/[gentoo]/xml/htdocs/doc/en/distcc.xml
Gentoo

Diff of /xml/htdocs/doc/en/distcc.xml

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

Revision 1.21 Revision 1.22
1<?xml version='1.0' encoding="UTF-8"?> 1<?xml version='1.0' encoding="UTF-8"?>
2<!DOCTYPE guide SYSTEM "/dtd/guide.dtd"> 2<!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/distcc.xml,v 1.21 2004/07/28 14:38:25 swift Exp $ --> 3<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/distcc.xml,v 1.22 2004/08/22 20:51:08 neysx Exp $ -->
4
5<!--
6 Synchronised with rev 1.15 on
7 http://www.thedoh.com/viewcvs/distcc/html/distcc.xml
8-->
9 4
10<guide link="/doc/en/distcc.xml"> 5<guide link="/doc/en/distcc.xml">
11 6
12<title>Gentoo Distcc Documentation</title> 7<title>Gentoo Distcc Documentation</title>
13 8
14<author title="Author"> 9<author title="Author">
15 <mail link="lisa@gentoo.org">Lisa Seelye</mail> 10 <mail link="lisa@gentoo.org">Lisa Seelye</mail>
16</author> 11</author>
17 12
18<author title="Editor"> 13<author title="Editor">
19 <mail link="vapier@gentoo.org">Mike Frysinger</mail> 14 <mail link="vapier@gentoo.org">Mike Frysinger</mail>
20</author> 15</author>
21 16
22<author title="Editor"> 17<author title="Editor">
23 <mail link="erwin@gentoo.org">Erwin</mail> 18 <mail link="erwin@gentoo.org">Erwin</mail>
31 <mail link="pylon@gentoo.org">Lars Weiler</mail> 26 <mail link="pylon@gentoo.org">Lars Weiler</mail>
32</author> 27</author>
33 28
34<author title="Reviewer"> 29<author title="Reviewer">
35 <mail link="blubber@gentoo.org">Tiemo Kieft</mail> 30 <mail link="blubber@gentoo.org">Tiemo Kieft</mail>
36</author> 31</author>
37 32
38<abstract> 33<abstract>
39This document serves as a HOWTO for using distcc with Gentoo. 34This document serves as a HOWTO for using distcc with Gentoo.
40</abstract> 35</abstract>
41 36
42<!-- The content of this document is licensed under the CC-BY-SA license --> 37<!-- The content of this document is licensed under the CC-BY-SA license -->
43<!-- See http://creativecommons.org/licenses/by-sa/1.0 --> 38<!-- See http://creativecommons.org/licenses/by-sa/1.0 -->
44<license/> 39<license/>
45 40
46<version>1.7.2</version> 41<version>1.8</version>
47<date>July 28, 2004</date> 42<date>August 22, 2004</date>
48 43
49<chapter> 44<chapter>
50<title>Introduction</title> 45<title>Introduction</title>
51<section> 46<section>
52<title>What is distcc?</title> 47<title>What is distcc?</title>
53<body> 48<body>
54 49
55<p> 50<p>
56Distcc is a program designed to distribute compiling tasks across a network to 51Distcc is a program designed to distribute compiling tasks across a network to
57participating hosts. It is comprised of a server, <c>distccd</c> and a client 52participating hosts. It is comprised of a server, <c>distccd</c> and a client
58program, <c>distcc</c>. Distcc can work transparently with <uri 53program, <c>distcc</c>. Distcc can work transparently with <uri
59link="http://ccache.samba.org">ccache</uri> and Portage with a little work. 54link="http://ccache.samba.org">ccache</uri> and Portage with a little work.
60</p> 55</p>
61 56
62</body> 57</body>
112</body> 107</body>
113</section> 108</section>
114<section> 109<section>
115<title>Setting up Portage to use Distcc</title> 110<title>Setting up Portage to use Distcc</title>
116<body> 111<body>
117 112
118<p> 113<p>
119Setting up distcc is easy to do with Portage. Execute the following steps on 114Setting up distcc is easy to do with Portage. Execute the following steps on
120each system that should participate in the distributed compiling: 115each system that should participate in the distributed compiling:
121</p> 116</p>
122 117
123<pre caption="Integrating Distcc and Portage"> 118<pre caption="Integrating Distcc and Portage">
124# <i>emerge distcc</i> 119# <i>emerge distcc</i>
125# <i>nano -w /etc/make.conf</i> 120# <i>nano -w /etc/make.conf</i>
126<comment>(Set N to a suitable number for your particular setup)</comment> 121<comment>(Set N to a suitable number for your particular setup)</comment>
127<comment>(A common heuristic strategy is twice the number of CPUs + 1)</comment> 122<comment>(A common heuristic strategy is twice the number of total CPUs + 1)</comment>
128MAKEOPTS="-jN" 123MAKEOPTS="-jN"
129<comment>(Add distcc to your FEATURES variable)</comment> 124<comment>(Add distcc to your FEATURES variable)</comment>
130FEATURES="distcc" 125FEATURES="distcc"
131</pre> 126</pre>
132 127
133<p> 128<p>
134Next you have to specify what hosts you want to use. Use the 129Next you have to specify what hosts you want to use. Use the
135<c>distcc-config</c> command to set the list of hosts. Here is an example of 130<c>distcc-config</c> command to set the list of hosts. Here is an example of
136some hosts that might be in your list: 131some hosts that might be in your list:
137</p> 132</p>
138 133
139<pre caption="Examples of host definitions"> 134<pre caption="Examples of host definitions">
140192.168.0.1 192.168.0.2 192.168.0.3 135192.168.0.1 192.168.0.2 192.168.0.3
141192.168.0.1/2 192.168.0.2 192.168.0.3/10 136192.168.0.1/2 192.168.0.2 192.168.0.3/10
142192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4 137192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4
190<body> 185<body>
191 186
192<p> 187<p>
193This is in some cases easier than the Portage setup. What you have to do is 188This is in some cases easier than the Portage setup. What you have to do is
194update your <c>PATH</c> variable to include <path>/usr/lib/distcc/bin</path> 189update your <c>PATH</c> variable to include <path>/usr/lib/distcc/bin</path>
195in front of the directory that contains <c>gcc</c> (<path>/usr/bin</path>). 190in front of the directory that contains <c>gcc</c> (<path>/usr/bin</path>).
196However, there is a caveat. If you use ccache you have to put distcc after 191However, there is a caveat. If you use ccache you have to put distcc after
197the ccache part: 192the ccache part:
198</p> 193</p>
199 194
200<pre caption="Setting your path"> 195<pre caption="Setting your path">
201# <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i> 196# <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
202</pre> 197</pre>
203 198
204<p> 199<p>
205Then, as you would normally type <c>make</c>, you would type <c>make -jN</c> 200Then, as you would normally type <c>make</c>, you would type <c>make -jN</c>
206(where N is an integer). The value of N depends on your network and the types 201(where N is an integer). The value of N depends on your network and the types
207of computers you are using to compile. For me, I have a dual processor P3 and 202of computers you are using to compile. For me, I have a dual processor P3 and
208a K6-2/400 that compiles everything for my desktop (a 1200MHz Athlon 203a K6-2/400 that compiles everything for my desktop (a 1200MHz Athlon
209Thunderbird) and I use -j5. Test your own settings to find the most 204Thunderbird) and I use -j5. Test your own settings to find the number that
210performant number. 205yields the best performance.
211</p> 206</p>
212 207
213</body> 208</body>
214</section> 209</section>
215</chapter> 210</chapter>
216 211
217<chapter> 212<chapter>
218<title>Cross-Compiling</title> 213<title>Cross-Compiling</title>
219<section> 214<section>
220<title>A Note on Cross-Compiling</title> 215<title>A Note on Cross-Compiling</title>
221<body> 216<body>
222 217
223<p> 218<p>
224Cross-compiling is using one architecture to build programs for another 219Cross-compiling is using one architecture to build programs for another
225architecture. This can be as simple as using an Athlon (i686) to build a 220architecture. This can be as simple as using an Athlon (i686) to build a
226program for a K6-2 (i586), or using a Sparc to build a program for a ppc. 221program for a K6-2 (i586), or using a Sparc to build a program for a ppc.
227</p> 222</p>
228 223
229</body> 224</body>
230</section> 225</section>
231<section> 226<section>
232<title>A Personal Note on Cross-compiling</title> 227<title>An Introduction to Cross-compiling</title>
233<body> 228<body>
234 229
235<p>
236I'd love to be able to help out on the Gentoo Cross-compiling effort, but I
237don't have any non-x86 machines. I can create things that work <e>in
238theory</e>, but I have to rely on other people to test what I write.
239That works to an extent, but it's tough. ;-)
240</p> 230<p>
241 231If you want to give Cross-compiling a try you can emerge the crossdev package.
242<p> 232I can't support it since I didn't write it, but that's a good way to start
243In the near future I hope to acquire a Sparc of some type to put Gentoo on so 233cross-compiling.
244I can play around in-house.
245</p> 234</p>
246 235
247</body> 236</body>
248</section> 237</section>
249</chapter> 238</chapter>
250 239
251<chapter id="bootstrapping"> 240<chapter id="bootstrapping">
252<title>Using Distcc to Bootstrap</title> 241<title>Using Distcc to Bootstrap</title>
253<section> 242<section>
254<title>Step 1: Configure Portage </title> 243<title>Step 1: Configure Portage </title>
255<body> 244<body>
256 245
257<p> 246<p>
258Boot your new box with a Gentoo Linux LiveCD and follow the <uri 247Boot your new box with a Gentoo Linux LiveCD and follow the <uri
259link="/doc/en/handbook/handbook-x86.xml?part=1">installation instructions</uri> 248link="/doc/en/handbook/handbook-x86.xml?part=1">installation instructions</uri>
260up until the bootstrapping part. Then configure Portage to use distcc: 249up until the bootstrapping part. Then configure Portage to use distcc:
261</p> 250</p>
262 251
263<pre caption="Preliminary Setup"> 252<pre caption="Preliminary Setup">
264# <i>nano -w /etc/make.conf</i> 253# <i>nano -w /etc/make.conf</i>
265<comment>(Add distcc to the FEATURES variable)</comment> 254<comment>(Add distcc to the FEATURES variable)</comment>
266FEATURES="distcc" 255FEATURES="distcc"
267<comment>(Set distcc's directory for temporary space)</comment>
268DISTCC_DIR="${PORTAGE_TMPDIR}/.distcc"
269<comment>(Modify MAKEOPTS to include -jN with N an appropriate integer)</comment> 256<comment>(Modify MAKEOPTS to include -jN with N an appropriate integer)</comment>
270MAKEOPTS="-jN" 257MAKEOPTS="-jN"
271</pre> 258</pre>
272 259
260<pre caption="Setting your path">
261# <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
262</pre>
263
273</body> 264</body>
274</section> 265</section>
275<section> 266<section>
276<title>Step 2: Getting Distcc</title> 267<title>Step 2: Getting Distcc</title>
277<body> 268<body>
278 269
270<p>
279<p>Before the installation of distcc, a user called distcc must be added to the 271Before the installation of distcc, a user called distcc must be added to the
280<path>/etc/passwd</path>:</p> 272<path>/etc/passwd</path>:
273</p>
281 274
282<pre caption="Create user distcc"> 275<pre caption="Create user distcc">
283# <i>echo "distcc:x:240:2:distccd:/dev/null:/bin/false" &gt;&gt; /etc/passwd</i> 276# <i>echo "distcc:x:240:2:distccd:/dev/null:/bin/false" &gt;&gt; /etc/passwd</i>
284</pre> 277</pre>
285 278
286<p> 279<p>
287Install distcc: 280Install distcc:
288</p> 281</p>
289 282
290<pre caption="Getting Distcc on the new box"> 283<pre caption="Getting Distcc on the new box">
291# <i>USE='-*' emerge --nodeps distcc</i> 284# <i>USE='-*' emerge --nodeps distcc</i>
292</pre> 285</pre>
293 286
294</body> 287</body>
295</section> 288</section>
296<section> 289<section>
297<title>Step 3: Setting Up Distcc</title> 290<title>Step 3: Setting Up Distcc</title>
298<body> 291<body>
299 292
300<p> 293<p>
301Run <c>distcc-config</c> to setup distcc: 294Run <c>distcc-config</c> to setup distcc:
302</p> 295</p>
303 296
304<pre caption="Final distcc setup"> 297<pre caption="Final distcc setup">
305<comment>(Substitute host1, host2, ... with the IP number(s) of the participating hosts)</comment> 298<comment>(Substitute host1, host2, ... with the IP number(s) of the participating hosts)</comment>
306# <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i> 299# <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i>
300<comment>An example: <i>/usr/bin/distcc-config --set-hosts "localhost 192.168.0.4 192.168.0.6"</i></comment>
307</pre> 301</pre>
308 302
309<p> 303<p>
310distcc is now set up to bootstrap! Continue the official installation 304distcc is now set up to bootstrap! Continue the official installation
311instructions and <e>don't forget</e> to re-emerge distcc after <c>emerge 305instructions and <e>don't forget</e> to re-emerge distcc after <c>emerge
312system</c>. This is to make sure that all of the dependencies you want are 306system</c>. This is to make sure that all of the dependencies you want are
313installed as well. 307installed as well.
314</p> 308</p>
315 309
316<note> 310<note>
317During bootstrap and <c>emerge system</c> Distcc may not appear to be used. 311During bootstrap and <c>emerge system</c> Distcc may not appear to be used.
318This is expected as some ebuilds do not work well with distcc, so they 312This is expected as some ebuilds do not work well with distcc, so they
319intentionally disable it. 313intentionally disable it.
320</note> 314</note>
321 315
388Distcc ships with two monitors. The text-based one is always built and is 382Distcc ships with two monitors. The text-based one is always built and is
389called <c>distccmon-text</c>. Running it for the first time can be a bit 383called <c>distccmon-text</c>. Running it for the first time can be a bit
390confusing, but it is really quite easy to use. If you run the program with no 384confusing, but it is really quite easy to use. If you run the program with no
391parameter it will run once. However, if you pass it a number it will update 385parameter it will run once. However, if you pass it a number it will update
392every N seconds, where N is the argument you passed. 386every N seconds, where N is the argument you passed.
393</p> 387</p>
394 388
395<p> 389<p>
396The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c> 390The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c>
397in your <c>USE</c> flags. This one is GTK+ based, runs in an X environment 391in your <c>USE</c> flags. This one is GTK+ based, runs in an X environment
398and it is quite lovely. 392and it is quite lovely.
399</p> 393</p>
400 394
401<pre caption="Starting the monitors"> 395<pre caption="Starting the monitors">
402# <i>distccmon-text N</i> 396# <i>distccmon-text N</i>
403<comment>(Or ...)</comment> 397<comment>(Or)</comment>
404# <i>distccmon-gnome</i> 398# <i>distccmon-gnome</i>
405</pre> 399<comment>To monitor Portage's distcc usage you can use:</comment>
406 400# <i>DISTCC_DIR=`portageq envvar DISTCC_DIR` distccmon-text N</i>
407</body> 401<comment>(Or)</comment>
408</section> 402# <i>DISTCC_DIR=`portageq envvar DISTCC_DIR` distccmon-gnome</i>
409</chapter>
410
411<chapter>
412<title>Future Plans for Distcc and Gentoo</title>
413<section>
414<title>distcc-subnetscan</title>
415<body>
416
417<p>
418<c>distcc-subnetscan</c> is a perl program in development that will scan a
419subnet for hosts that have a participating distcc daemon. Even this will be
420enhanced to test if the daemon on a remote host will conform to a specific
421<c>CHOST</c> setting to make cross-compiling easier.
422</p> 403</pre>
423 404
424<p> 405<impo>
425The perl script is being kept at <uri link="http://dev.gentoo.org/~lisa/proj/distcc/distcc-subnetscan/distcc-subnetscan.html">lisa's 406Notice the backticks above. <c>portageq</c> is a part of Portage that will
426dev page</uri> until a more formal home can be found. 407return a piece of information (in this case what DISTCC_DIR is set to). For
427</p> 408simplicity, most users can use <c>DISTCC_DIR=/var/tmp/portage/.distcc
409distccmon-text N</c>
410</impo>
428 411
429</body> 412</body>
430</section> 413</section>
431</chapter> 414</chapter>
432</guide> 415</guide>

Legend:
Removed from v.1.21  
changed lines
  Added in v.1.22

  ViewVC Help
Powered by ViewVC 1.1.20