/[gentoo]/xml/htdocs/doc/en/distcc.xml
Gentoo

Diff of /xml/htdocs/doc/en/distcc.xml

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

Revision 1.27 Revision 1.28
1<?xml version='1.0' encoding="UTF-8"?> 1<?xml version='1.0' encoding="UTF-8"?>
2<!DOCTYPE guide SYSTEM "/dtd/guide.dtd"> 2<!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/distcc.xml,v 1.27 2005/07/01 22:35:19 smithj Exp $ --> 3<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/distcc.xml,v 1.28 2005/07/01 22:42:42 smithj Exp $ -->
4 4
5<guide link="/doc/en/distcc.xml"> 5<guide link="/doc/en/distcc.xml">
6 6
7<title>Gentoo Distcc Documentation</title> 7<title>Gentoo Distcc Documentation</title>
8 8
9<author title="Author"> 9<author title="Author">
10 <mail link="lisa@gentoo.org">Lisa Seelye</mail> 10 <mail link="lisa@gentoo.org">Lisa Seelye</mail>
11</author> 11</author>
12 12
13<author title="Editor"> 13<author title="Editor">
14 <mail link="vapier@gentoo.org">Mike Frysinger</mail> 14 <mail link="vapier@gentoo.org">Mike Frysinger</mail>
15</author> 15</author>
16 16
17<author title="Editor"> 17<author title="Editor">
18 <mail link="erwin@gentoo.org">Erwin</mail> 18 <mail link="erwin@gentoo.org">Erwin</mail>
… …
26 <mail link="pylon@gentoo.org">Lars Weiler</mail> 26 <mail link="pylon@gentoo.org">Lars Weiler</mail>
27</author> 27</author>
28 28
29<author title="Reviewer"> 29<author title="Reviewer">
30 <mail link="blubber@gentoo.org">Tiemo Kieft</mail> 30 <mail link="blubber@gentoo.org">Tiemo Kieft</mail>
31</author> 31</author>
32 32
33<abstract> 33<abstract>
34This document serves as a HOWTO for using distcc with Gentoo. 34This document serves as a HOWTO for using distcc with Gentoo.
35</abstract> 35</abstract>
36 36
37<!-- The content of this document is licensed under the CC-BY-SA license --> 37<!-- The content of this document is licensed under the CC-BY-SA license -->
38<!-- See http://creativecommons.org/licenses/by-sa/1.0 --> 38<!-- See http://creativecommons.org/licenses/by-sa/1.0 -->
39<license/> 39<license/>
40 40
41<version>1.11</version> 41<version>1.13</version>
42<date>2005-03-25</date> 42<date>2005-07-01</date>
43 43
44<chapter> 44<chapter>
45<title>Introduction</title> 45<title>Introduction</title>
46<section> 46<section>
47<title>What is distcc?</title> 47<title>What is distcc?</title>
48<body> 48<body>
49 49
50<p> 50<p>
51Distcc is a program designed to distribute compiling tasks across a network to 51Distcc is a program designed to distribute compiling tasks across a network to
52participating hosts. It is comprised of a server, <c>distccd</c>, and a client 52participating hosts. It is comprised of a server, <c>distccd</c>, and a client
53program, <c>distcc</c>. Distcc can work transparently with <uri 53program, <c>distcc</c>. Distcc can work transparently with <uri
54link="http://ccache.samba.org">ccache</uri> and Portage with a little work. 54link="http://ccache.samba.org">ccache</uri>, Portage, and Automake with a
55little setup.
55</p> 56</p>
56 57
57</body> 58</body>
58</section> 59</section>
59<section> 60<section>
60<title>Using distcc to bootstrap</title> 61<title>Using distcc to bootstrap</title>
61<body> 62<body>
62 63
63<p> 64<p>
64If you are planning on using distcc to help you bootstrap a Gentoo installation, 65If you are planning on using distcc to help you bootstrap a Gentoo
65make sure you read the section <uri link="#bootstrapping">Using distcc to 66installation, make sure you read the section <uri link="#bootstrapping">Using
66Bootstrap</uri>, which is situated further down in this document. 67distcc to Bootstrap</uri>, which is situated further down in this document.
67</p> 68</p>
68 69
69</body> 70</body>
70</section> 71</section>
71</chapter> 72</chapter>
72 73
73<chapter> 74<chapter>
74<title>Setup</title> 75<title>Setup</title>
75<section> 76<section>
76<title>Dependencies</title> 77<title>Dependencies</title>
77<body> 78<body>
78 79
79<p> 80<p>
80In order to use Distcc, all of the computers on your network need to 81In order to use Distcc, all of the computers on your network need to have the
81have the same GCC versions. Mixing 3.3.x (where the x varies) is okay, but 82same GCC versions. For example, mixing 3.3.x (where the x varies) is okay, but
82mixing 3.3.x and 3.2.x <b>will</b> result in your programs having errors in them. 83mixing 3.3.x with 3.2.x <b>may</b> result in compilation errors or runtime
84errors.
83</p> 85</p>
84 86
85</body> 87</body>
86</section> 88</section>
87<section> 89<section>
88<title>Installing Distcc</title> 90<title>Installing Distcc</title>
89<body> 91<body>
90 92
91<p> 93<p>
92There are a couple of options you should be aware of before you start 94There are a couple of options you should be aware of before you start
93installing distcc. 95installing distcc.
94</p> 96</p>
95 97
96<p> 98<p>
97Distcc ships with a graphical monitor to monitor tasks that your computer is 99Distcc ships with a graphical monitor to monitor tasks that your computer is
… …
107</body> 109</body>
108</section> 110</section>
109<section> 111<section>
110<title>Setting up Portage to use Distcc</title> 112<title>Setting up Portage to use Distcc</title>
111<body> 113<body>
112 114
113<p> 115<p>
114Setting up Portage to use distcc is easy. Execute the following steps on 116Setting up Portage to use distcc is easy. Execute the following steps on
115each system that should participate in the distributed compiling: 117each system that should participate in the distributed compiling:
116</p> 118</p>
117 119
118<pre caption="Integrating Distcc and Portage"> 120<pre caption="Integrating Distcc and Portage">
119# <i>emerge distcc</i> 121# <i>emerge distcc</i>
120# <i>nano -w /etc/make.conf</i> 122# <i>nano -w /etc/make.conf</i>
121<comment>(Set N to a suitable number for your particular setup)</comment> 123<comment>(Set N to a suitable number for your particular setup)</comment>
122<comment>(A common heuristic strategy is twice the number of total CPUs + 1)</comment> 124<comment>(A common strategy is setting N as twice the number of total CPUs + 1
125available)</comment>
123MAKEOPTS="-jN" 126MAKEOPTS="-jN"
124<comment>(Add distcc to your FEATURES variable)</comment> 127<comment>(Add distcc to your FEATURES)</comment>
125FEATURES="distcc" 128FEATURES="distcc"
126</pre> 129</pre>
127 130
128</body> 131</body>
129</section> 132</section>
130<section> 133<section>
131<title>Specifying Participating Hosts</title> 134<title>Specifying Participating Hosts</title>
132<body> 135<body>
133 136
134<p> 137<p>
135Use the <c>distcc-config</c> command to set the list of hosts. Here is an example of 138Use the <c>distcc-config</c> command to set the list of hosts. Here is an
136some hosts that might be in your list: 139example of some hosts that might be in your list:
137</p> 140</p>
138 141
139<pre caption="Examples of host definitions"> 142<pre caption="Examples of host definitions">
140192.168.0.1 192.168.0.2 192.168.0.3 143192.168.0.1 192.168.0.2 192.168.0.3
141192.168.0.1/2 192.168.0.2 192.168.0.3/10 144192.168.0.1/2 192.168.0.2 192.168.0.3/10
142192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4 145192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4
143@192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3 146@192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3
144<comment>(There are also several other methods of setting up hosts. See the 147<comment>(There are also several other methods of setting up hosts. See the
145distcc manpage for more details.)</comment> 148distcc manpage for more details.)</comment>
149<comment>If you wish to compile on the local machine you should put 'localhost'
150in the hosts list. Conversely if you do not wish to use the local machine to
151compile (which is often the case) omit it from the hosts list. On a slow
152machine using localhost may actually slow things down. Make sure to test your
153settings for performance.</comment>
146</pre> 154</pre>
147 155
148<p> 156<p>
149It may all look complicated, but in most cases a variant of line 1 or 2 will 157It may all look complicated, but in most cases a variant of line 1 or 2 will
150work. 158work.
151</p> 159</p>
152 160
153<ul>
154 <li>Line 1 is just a space-delimited list of hosts</li>
155 <li>
156 Line 2 is a list of hosts that specifies the maximum amount of jobs (by
157 defining the <c>/N</c> setting) to send that host at any given time
158 </li>
159</ul>
160
161<p> 161<p>
162Since most people won't be using lines 3 or 4, I'll <uri 162Since most people won't be using lines 3 or 4, I'll <uri
163link="http://distcc.samba.org/man/distcc_1.html">point you to</uri> the distcc 163link="http://distcc.samba.org/man/distcc_1.html">refer to</uri> the distcc
164docs for more information. 164docs (man distcc) for more information.
165</p> 165</p>
166 166
167<p> 167<p>
168For instance, to set the first line in the previous example: 168For instance, to set the first line in the previous example:
169</p> 169</p>
170 170
171<pre caption="Sample command to set the hosts"> 171<pre caption="Sample command to set the hosts">
172# <i>/usr/bin/distcc-config --set-hosts "192.168.0.1 192.168.0.2 192.168.0.3"</i> 172# <i>/usr/bin/distcc-config --set-hosts "192.168.0.1 192.168.0.2 192.168.0.3"</i>
173</pre> 173</pre>
174 174
175<p> 175<p>
176Edit <path>/etc/conf.d/distccd</path> to your needs and be sure to set the 176Edit <path>/etc/conf.d/distccd</path> to your needs and be sure to set the
177<c>--allow</c> directive to allow only hosts you trust. For added security, you should also use the <c>--listen</c> 177<c>--allow</c> directive to allow only hosts you trust. For added security,
178directive to tell the distcc daemon what IP to listen on (for multi-homed 178you should also use the <c>--listen</c> directive to tell the distcc daemon
179systems). More information on distcc security can be found at <uri link="http://distcc.samba.org/security.html">Distcc Security 179what IP to listen on (for multi-homed systems). More information on distcc
180Design</uri>. 180security can be found at <uri
181link="http://distcc.samba.org/security.html">Distcc Security Design</uri>.
181</p> 182</p>
183
184<impo>
185It is important to use --allow and --listen. Please read the distccd manpage
186or the above seurity document for more information.
187</impo>
188
182 189
183<p> 190<p>
184Now start the distcc daemon on all the participating computers: 191Now start the distcc daemon on all the participating computers:
185</p> 192</p>
186 193
187<pre caption="Starting the distcc daemon"> 194<pre caption="Starting the distcc daemon">
188<comment>(Add distccd to the default runlevel)</comment> 195<comment>(Add distccd to the default runlevel)</comment>
189# <i>rc-update add distccd default</i> 196# <i>rc-update add distccd default</i>
190<comment>(Start the distcc daemon)</comment> 197<comment>(Start the distcc daemon)</comment>
191# <i>/etc/init.d/distccd start</i> 198# <i>/etc/init.d/distccd start</i>
192</pre> 199</pre>
193 200
194</body> 201</body>
195</section> 202</section>
196<section> 203<section>
197<title>Setting up Distcc to Work With Automake</title> 204<title>Setting up Distcc to Work With Automake</title>
198<body> 205<body>
199 206
200<p> 207<p>
201This is, in some cases, easier than the Portage setup. What you have to do is 208This is, in some cases, easier than the Portage setup. What you have to do is
202update your <c>PATH</c> variable to include <path>/usr/lib/distcc/bin</path> 209update your <c>PATH</c> variable to include <path>/usr/lib/distcc/bin</path>
203in front of the directory that contains <c>gcc</c> (<path>/usr/bin</path>). 210in front of the directory that contains <c>gcc</c> (<path>/usr/bin</path>).
204However, there is a caveat. If you use ccache you have to put distcc after 211However, there is a caveat. If you use ccache you have to put distcc after
205the ccache part: 212the ccache part:
206</p> 213</p>
207 214
208<pre caption="Setting your path"> 215<pre caption="Setting your path">
209# <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i> 216# <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
217<comment>You can put this in your .bashrc or equivelant file to have the PATH
218set every time you log in</comment>
210</pre> 219</pre>
211 220
212<p> 221<p>
213Then, as you would normally type <c>make</c>, you would type <c>make -jN</c> 222Then, as you would normally type <c>make</c>, you would type <c>make -jN</c>
214(where N is an integer). The value of N depends on your network and the types 223(where N is an integer). The value of N depends on your network and the types
215of computers you are using to compile. For me, I have a dual processor P3 and 224of computers you are using to compile. Test your own settings to find the
216a K6-2/400 that compiles everything for my desktop (a 1200MHz Athlon
217Thunderbird) and I use -j5. Test your own settings to find the number that
218yields the best performance. 225number that yields the best performance.
219</p> 226</p>
220 227
221</body> 228</body>
222</section> 229</section>
223</chapter> 230</chapter>
224 231
225<chapter> 232<chapter>
226<title>Cross-Compiling</title> 233<title>Cross-Compiling</title>
227<section> 234<section>
228<title>A Note on Cross-Compiling</title> 235<title>A Note on Cross-Compiling</title>
229<body> 236<body>
230 237
231<p> 238<p>
232Cross-compiling is using one architecture to build programs for another 239Cross-compiling is using one architecture to build programs for another
233architecture. This can be as simple as using an Athlon (i686) to build a 240architecture. This can be as simple as using an Athlon (i686) to build a
… …
252 259
253<chapter id="bootstrapping"> 260<chapter id="bootstrapping">
254<title>Using Distcc to Bootstrap</title> 261<title>Using Distcc to Bootstrap</title>
255<section> 262<section>
256<title>Step 1: Configure Portage</title> 263<title>Step 1: Configure Portage</title>
257<body> 264<body>
258 265
259<p> 266<p>
260Boot your new box with a Gentoo Linux LiveCD and follow the <uri 267Boot your new box with a Gentoo Linux LiveCD and follow the <uri
261link="/doc/en/handbook/handbook-x86.xml?part=1">installation instructions</uri> 268link="/doc/en/handbook/handbook-x86.xml?part=1">installation instructions</uri>
262up until the bootstrapping part. Then configure Portage to use distcc: 269up until the bootstrapping part. Then configure Portage to use distcc:
263</p> 270</p>
264 271
265<pre caption="Preliminary Setup"> 272<pre caption="Preliminary Setup">
266# <i>nano -w /etc/make.conf</i> 273# <i>nano -w /etc/make.conf</i>
267<comment>(Add distcc to the FEATURES variable)</comment> 274<comment>(Add distcc to the FEATURES</comment>
268FEATURES="distcc" 275FEATURES="distcc"
269<comment>(Modify MAKEOPTS to include -jN with N an appropriate integer)</comment> 276<comment>(Modify MAKEOPTS to include -jN, where N is twice the number of CPUs
277+1 available)</comment>
270MAKEOPTS="-jN" 278MAKEOPTS="-jN"
271</pre> 279</pre>
272 280
273<pre caption="Setting your path"> 281<pre caption="Setting your path">
274# <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i> 282# <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
275</pre> 283</pre>
276 284
277</body> 285</body>
278</section> 286</section>
279<section> 287<section>
280<title>Step 2: Getting Distcc</title> 288<title>Step 2: Getting Distcc</title>
281<body> 289<body>
282 290
283<p> 291<p>
284Before the installation of distcc, a user called distcc must be added to the 292Before the installation of distcc, a user called distcc must be added to the
285<path>/etc/passwd</path>: 293<path>/etc/passwd</path>:
286</p> 294</p>
287 295
288<pre caption="Create user distcc"> 296<pre caption="Create user distcc">
289# <i>echo "distcc:x:240:2:distccd:/dev/null:/bin/false" &gt;&gt; /etc/passwd</i> 297# <i>echo "distcc:x:240:2:distccd:/dev/null:/bin/false" &gt;&gt;/etc/passwd</i>
290</pre> 298</pre>
291 299
292<impo> 300<impo>
293It is important to note that adding users like this is very bad. We only do 301It is important to note that adding users like this is very bad. We only do
294it here because there is no <c>useradd</c> utility (which you normally use for 302it here because there is no <c>useradd</c> utility (which you normally use for
295adding users) yet at this stage of installation. 303adding users) yet at this stage of installation.
296</impo> 304</impo>
297 305
298<p> 306<p>
299Install distcc: 307Install distcc:
300</p> 308</p>
301 309
302<pre caption="Getting Distcc on the new box"> 310<pre caption="Getting Distcc on the new box">
303# <i>USE='-*' emerge --nodeps sys-devel/distcc</i> 311# <i>USE='-*' emerge --nodeps sys-devel/distcc</i>
304</pre> 312</pre>
305 313
306</body> 314</body>
307</section> 315</section>
308<section> 316<section>
309<title>Step 3: Setting Up Distcc</title> 317<title>Step 3: Setting Up Distcc</title>
310<body> 318<body>
311 319
312<p> 320<p>
313Run <c>distcc-config --install</c> to setup distcc: 321Run <c>distcc-config --install</c> to setup distcc:
314</p> 322</p>
315 323
316<pre caption="Final distcc setup"> 324<pre caption="Final distcc setup">
317<comment>(Substitute host1, host2, ... with the IP number(s) of the participating hosts)</comment> 325<comment>(Substitute host1, host2, ... with the IP number(s) of the
326participating hosts)</comment>
318# <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i> 327# <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i>
319<comment>An example: <i>/usr/bin/distcc-config --set-hosts "localhost 192.168.0.4 192.168.0.6"</i></comment> 328<comment>An example: <i>/usr/bin/distcc-config --set-hosts "localhost
329192.168.0.4 192.168.0.6"</i></comment>
320</pre> 330</pre>
321 331
322<p> 332<p>
323Distcc is now set up to bootstrap! Continue with the official installation 333Distcc is now set up to bootstrap! Continue with the official installation
324instructions and <e>do not forget</e> to re-emerge distcc after <c>emerge 334instructions and <e>do not forget</e> to re-emerge distcc after <c>emerge
325system</c>. This is to make sure that all of the dependencies you want are 335system</c>. This is to make sure that all of the dependencies you want are
326installed as well. 336installed as well.
327</p> 337</p>
328 338
329<note> 339<note>
330During bootstrap and <c>emerge system</c> distcc may not appear to be used. 340During bootstrap and <c>emerge system</c> distcc may not appear to be used.
331This is expected as some ebuilds do not work well with distcc, so they 341This is expected as some ebuilds do not work well with distcc, so they
332intentionally disable it. 342intentionally disable it.
333</note> 343</note>
334 344
… …
379</section> 389</section>
380<section> 390<section>
381<title>Mixed GCC Versions</title> 391<title>Mixed GCC Versions</title>
382<body> 392<body>
383 393
384<p> 394<p>
385If you have different GCC versions on your hosts, there will likely be very 395If you have different GCC versions on your hosts, there will likely be very
386weird problems. The solution is to make certain all hosts have the same GCC 396weird problems. The solution is to make certain all hosts have the same GCC
387version. 397version.
388</p> 398</p>
389 399
390<p> 400<p>
391Recent Portage updates have made Portage use <c>${CHOST}-gcc</c> instead of 401Recent Portage updates have made Portage use <c>${CHOST}-gcc</c> instead of
392<c>gcc</c>. This means that if you're mixing i686 machines with other types 402<c>gcc</c>. This means that if you're mixing i686 machines with other types
393(i386, i586) you will run into problems. A workaround for this may be to 403(i386, i586) you will run into problems. A workaround for this may be to
394<c>export CC='gcc' CXX='c++'</c> or to put it in <path>/etc/make.conf</path>, 404<c>export CC='gcc' CXX='c++'</c> or to put it in <path>/etc/make.conf</path>.
395but that needs testing. If it works for you please email me,
396<mail link="lisa@gentoo.org">Lisa Seelye</mail>.
397</p> 405</p>
406
407<impo>
408Doing this explicitly redefines some behaviour of Portage and may have some
409weird results in the future. Only do this if you're mixing CHOSTs.
410</impo>
411
398 412
399</body> 413</body>
400</section> 414</section>
401</chapter> 415</chapter>
402 416
403<chapter> 417<chapter>
404<title>Distcc Extras</title> 418<title>Distcc Extras</title>
405<section> 419<section>
406<title>Distcc Monitors</title> 420<title>Distcc Monitors</title>
407<body> 421<body>
408 422
409<p> 423<p>
410Distcc ships with two monitors. The text-based one is always built and is 424Distcc ships with two monitors. The text-based one is always built and is
411called <c>distccmon-text</c>. Running it for the first time can be a bit 425called <c>distccmon-text</c>. Running it for the first time can be a bit
412confusing, but it is really quite easy to use. If you run the program with no 426confusing, but it is really quite easy to use. If you run the program with no
413parameter it will run once. However, if you pass it a number it will update 427parameter it will run once. However, if you pass it a number it will update
414every N seconds, where N is the argument you passed. 428every N seconds, where N is the argument you passed.
415</p> 429</p>
416 430
417<p> 431<p>
418The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c> 432The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c>
419in your <c>USE</c> flags. This one is GTK+ based, runs in an X environment 433in your <c>USE</c> flags. This one is GTK+ based, runs in an X environment
420and it is quite lovely. 434and it is quite lovely. For Gentoo the GUI monitor has been called
435<c>distccmon-gui</c> for less confusion. Elsewhere it may be referred to as
436<c>distccmon-gnome</c>.
421</p> 437</p>
422 438
423<pre caption="Starting the monitors"> 439<pre caption="Starting the monitors">
424# <i>distccmon-text N</i> 440# <i>distccmon-text N</i>
425<comment>(Or)</comment> 441<comment>(Or)</comment>
426# <i>distccmon-gnome</i> 442# <i>distccmon-gui</i>
427<comment>To monitor Portage's distcc usage you can use:</comment> 443<comment>To monitor Portage's distcc usage you can use:</comment>
428# <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-text N</i> 444# <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-text N</i>
429<comment>(Or)</comment> 445<comment>(Or)</comment>
430# <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-gnome</i> 446# <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-gui</i>
431</pre> 447</pre>
432 448
433<impo> 449<impo>
434If your distcc directory is elsewhere, change the DISTCC_DIR variable 450If your distcc directory is elsewhere, change the DISTCC_DIR variable
435accordingly. 451accordingly.
436</impo> 452</impo>
437 453
438</body> 454</body>
439</section> 455</section>
440</chapter> 456</chapter>
441</guide> 457</guide>
Legend:
Removed from v.1.27  
changed lines
  Added in v.1.28
  ViewVC Help
Powered by ViewVC 1.1.13