/[gentoo]/xml/htdocs/doc/en/distcc.xml
Gentoo

Contents of /xml/htdocs/doc/en/distcc.xml

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.30 - (hide annotations) (download) (as text)
Sun Jul 3 17:33:01 2005 UTC (8 years, 9 months ago) by alin
Branch: MAIN
Changes since 1.29: +2 -2 lines
File MIME type: application/xml
Fixed minor typo. No content change

1 swift 1.1 <?xml version='1.0' encoding="UTF-8"?>
2 blubber 1.3 <!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3 alin 1.30 <!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/distcc.xml,v 1.29 2005/07/02 04:50:55 smithj Exp $ -->
4 blubber 1.2
5 swift 1.1 <guide link="/doc/en/distcc.xml">
6    
7     <title>Gentoo Distcc Documentation</title>
8    
9     <author title="Author">
10 swift 1.10 <mail link="lisa@gentoo.org">Lisa Seelye</mail>
11 erwin 1.4 </author>
12 swift 1.17
13 erwin 1.4 <author title="Editor">
14 swift 1.10 <mail link="vapier@gentoo.org">Mike Frysinger</mail>
15 erwin 1.4 </author>
16 swift 1.17
17 erwin 1.4 <author title="Editor">
18 swift 1.10 <mail link="erwin@gentoo.org">Erwin</mail>
19 blubber 1.2 </author>
20 swift 1.17
21     <author title="Editor">
22 swift 1.10 <mail link="swift@gentoo.org">Sven Vermeulen</mail>
23 swift 1.1 </author>
24 swift 1.17
25 pylon 1.20 <author title="Editor">
26     <mail link="pylon@gentoo.org">Lars Weiler</mail>
27     </author>
28    
29 swift 1.17 <author title="Reviewer">
30 swift 1.10 <mail link="blubber@gentoo.org">Tiemo Kieft</mail>
31 swift 1.1 </author>
32    
33 swift 1.15 <abstract>
34     This document serves as a HOWTO for using distcc with Gentoo.
35     </abstract>
36    
37 swift 1.17 <!-- The content of this document is licensed under the CC-BY-SA license -->
38     <!-- See http://creativecommons.org/licenses/by-sa/1.0 -->
39 swift 1.6 <license/>
40    
41 smithj 1.29 <version>1.12</version>
42 smithj 1.28 <date>2005-07-01</date>
43 swift 1.1
44     <chapter>
45 swift 1.10 <title>Introduction</title>
46     <section>
47     <title>What is distcc?</title>
48     <body>
49    
50     <p>
51     Distcc is a program designed to distribute compiling tasks across a network to
52 smithj 1.27 participating hosts. It is comprised of a server, <c>distccd</c>, and a client
53 swift 1.10 program, <c>distcc</c>. Distcc can work transparently with <uri
54 smithj 1.28 link="http://ccache.samba.org">ccache</uri>, Portage, and Automake with a
55     little setup.
56 swift 1.17 </p>
57    
58     </body>
59     </section>
60     <section>
61     <title>Using distcc to bootstrap</title>
62     <body>
63    
64     <p>
65 smithj 1.28 If you are planning on using distcc to help you bootstrap a Gentoo
66     installation, make sure you read the section <uri link="#bootstrapping">Using
67     distcc to Bootstrap</uri>, which is situated further down in this document.
68 swift 1.10 </p>
69    
70     </body>
71     </section>
72 swift 1.8 </chapter>
73    
74     <chapter>
75 swift 1.10 <title>Setup</title>
76     <section>
77     <title>Dependencies</title>
78     <body>
79    
80 swift 1.17 <p>
81 smithj 1.28 In order to use Distcc, all of the computers on your network need to have the
82     same GCC versions. For example, mixing 3.3.x (where the x varies) is okay, but
83     mixing 3.3.x with 3.2.x <b>may</b> result in compilation errors or runtime
84     errors.
85 swift 1.17 </p>
86 swift 1.10
87     </body>
88     </section>
89     <section>
90     <title>Installing Distcc</title>
91     <body>
92    
93     <p>
94 swift 1.25 There are a couple of options you should be aware of before you start
95 swift 1.17 installing distcc.
96 swift 1.10 </p>
97    
98     <p>
99     Distcc ships with a graphical monitor to monitor tasks that your computer is
100     sending away for compilation. If you use Gnome then put 'gnome' in your USE
101     flags. However, if you don't use Gnome and would still like to have the
102     monitor then you should put 'gtk' in your USE flags.
103     </p>
104    
105 swift 1.17 <pre caption="Installing distcc">
106     # <i>emerge distcc</i>
107     </pre>
108    
109 swift 1.10 </body>
110     </section>
111     <section>
112     <title>Setting up Portage to use Distcc</title>
113     <body>
114    
115     <p>
116 swift 1.25 Setting up Portage to use distcc is easy. Execute the following steps on
117 swift 1.17 each system that should participate in the distributed compiling:
118 swift 1.10 </p>
119    
120     <pre caption="Integrating Distcc and Portage">
121 swift 1.1 # <i>emerge distcc</i>
122     # <i>nano -w /etc/make.conf</i>
123 swift 1.17 <comment>(Set N to a suitable number for your particular setup)</comment>
124 smithj 1.28 <comment>(A common strategy is setting N as twice the number of total CPUs + 1
125     available)</comment>
126 swift 1.17 MAKEOPTS="-jN"
127 smithj 1.28 <comment>(Add distcc to your FEATURES)</comment>
128 swift 1.17 FEATURES="distcc"
129 swift 1.5 </pre>
130 swift 1.1
131 swift 1.25 </body>
132     </section>
133     <section>
134     <title>Specifying Participating Hosts</title>
135     <body>
136    
137 swift 1.10 <p>
138 smithj 1.28 Use the <c>distcc-config</c> command to set the list of hosts. Here is an
139     example of some hosts that might be in your list:
140 swift 1.10 </p>
141    
142     <pre caption="Examples of host definitions">
143 swift 1.8 192.168.0.1 192.168.0.2 192.168.0.3
144     192.168.0.1/2 192.168.0.2 192.168.0.3/10
145     192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4
146     @192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3
147 swift 1.17 <comment>(There are also several other methods of setting up hosts. See the
148     distcc manpage for more details.)</comment>
149 smithj 1.28 <comment>If you wish to compile on the local machine you should put 'localhost'
150     in the hosts list. Conversely if you do not wish to use the local machine to
151     compile (which is often the case) omit it from the hosts list. On a slow
152     machine using localhost may actually slow things down. Make sure to test your
153     settings for performance.</comment>
154 swift 1.8 </pre>
155 swift 1.10
156     <p>
157     It may all look complicated, but in most cases a variant of line 1 or 2 will
158 swift 1.17 work.
159 swift 1.10 </p>
160    
161     <p>
162 swift 1.17 Since most people won't be using lines 3 or 4, I'll <uri
163 smithj 1.28 link="http://distcc.samba.org/man/distcc_1.html">refer to</uri> the distcc
164     docs (man distcc) for more information.
165 swift 1.17 </p>
166    
167     <p>
168     For instance, to set the first line in the previous example:
169 swift 1.10 </p>
170    
171     <pre caption="Sample command to set the hosts">
172 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "192.168.0.1 192.168.0.2 192.168.0.3"</i>
173 swift 1.8 </pre>
174 swift 1.1
175 swift 1.10 <p>
176 swift 1.23 Edit <path>/etc/conf.d/distccd</path> to your needs and be sure to set the
177 smithj 1.28 <c>--allow</c> directive to allow only hosts you trust. For added security,
178     you should also use the <c>--listen</c> directive to tell the distcc daemon
179     what IP to listen on (for multi-homed systems). More information on distcc
180     security can be found at <uri
181     link="http://distcc.samba.org/security.html">Distcc Security Design</uri>.
182 swift 1.23 </p>
183    
184 smithj 1.28 <impo>
185     It is important to use --allow and --listen. Please read the distccd manpage
186 alin 1.30 or the above security document for more information.
187 smithj 1.28 </impo>
188    
189    
190 swift 1.23 <p>
191 swift 1.17 Now start the distcc daemon on all the participating computers:
192 swift 1.10 </p>
193    
194 swift 1.17 <pre caption="Starting the distcc daemon">
195     <comment>(Add distccd to the default runlevel)</comment>
196     # <i>rc-update add distccd default</i>
197     <comment>(Start the distcc daemon)</comment>
198     # <i>/etc/init.d/distccd start</i>
199 swift 1.8 </pre>
200 swift 1.10
201     </body>
202     </section>
203     <section>
204     <title>Setting up Distcc to Work With Automake</title>
205     <body>
206    
207     <p>
208 swift 1.25 This is, in some cases, easier than the Portage setup. What you have to do is
209 swift 1.17 update your <c>PATH</c> variable to include <path>/usr/lib/distcc/bin</path>
210     in front of the directory that contains <c>gcc</c> (<path>/usr/bin</path>).
211 swift 1.10 However, there is a caveat. If you use ccache you have to put distcc after
212 swift 1.17 the ccache part:
213 swift 1.10 </p>
214    
215     <pre caption="Setting your path">
216 swift 1.1 # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
217 smithj 1.28 <comment>You can put this in your .bashrc or equivelant file to have the PATH
218     set every time you log in</comment>
219 swift 1.8 </pre>
220 swift 1.10
221     <p>
222 neysx 1.22 Then, as you would normally type <c>make</c>, you would type <c>make -jN</c>
223     (where N is an integer). The value of N depends on your network and the types
224 smithj 1.28 of computers you are using to compile. Test your own settings to find the
225     number that yields the best performance.
226 swift 1.10 </p>
227    
228     </body>
229     </section>
230 blubber 1.2 </chapter>
231 swift 1.1
232 blubber 1.2 <chapter>
233 swift 1.17 <title>Cross-Compiling</title>
234     <section>
235     <title>A Note on Cross-Compiling</title>
236     <body>
237    
238     <p>
239     Cross-compiling is using one architecture to build programs for another
240     architecture. This can be as simple as using an Athlon (i686) to build a
241     program for a K6-2 (i586), or using a Sparc to build a program for a ppc.
242     </p>
243    
244     </body>
245     </section>
246 swift 1.10 <section>
247 swift 1.25 <title>An Introduction to Cross-Compiling</title>
248 swift 1.10 <body>
249    
250     <p>
251 swift 1.25 If you want to give cross-compiling a try you can to follow <uri
252 swift 1.24 link="http://dev.gentoo.org/~vapier/CROSS-COMPILE-HOWTO">The Cross Compile
253 swift 1.25 HOWTO</uri>; Crossdev is deprecated.
254 swift 1.17 </p>
255    
256     </body>
257     </section>
258     </chapter>
259    
260     <chapter id="bootstrapping">
261     <title>Using Distcc to Bootstrap</title>
262     <section>
263 swift 1.24 <title>Step 1: Configure Portage</title>
264 swift 1.17 <body>
265 swift 1.10
266     <p>
267 swift 1.17 Boot your new box with a Gentoo Linux LiveCD and follow the <uri
268     link="/doc/en/handbook/handbook-x86.xml?part=1">installation instructions</uri>
269     up until the bootstrapping part. Then configure Portage to use distcc:
270 swift 1.10 </p>
271    
272 swift 1.17 <pre caption="Preliminary Setup">
273     # <i>nano -w /etc/make.conf</i>
274 smithj 1.28 <comment>(Add distcc to the FEATURES</comment>
275 swift 1.17 FEATURES="distcc"
276 smithj 1.28 <comment>(Modify MAKEOPTS to include -jN, where N is twice the number of CPUs
277     +1 available)</comment>
278 swift 1.17 MAKEOPTS="-jN"
279 swift 1.8 </pre>
280 swift 1.10
281 neysx 1.22 <pre caption="Setting your path">
282     # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
283     </pre>
284    
285 swift 1.10 </body>
286     </section>
287     <section>
288     <title>Step 2: Getting Distcc</title>
289     <body>
290    
291 neysx 1.22 <p>
292     Before the installation of distcc, a user called distcc must be added to the
293     <path>/etc/passwd</path>:
294     </p>
295 pylon 1.20
296     <pre caption="Create user distcc">
297 smithj 1.28 # <i>echo "distcc:x:240:2:distccd:/dev/null:/bin/false" &gt;&gt;/etc/passwd</i>
298 pylon 1.20 </pre>
299    
300 swift 1.24 <impo>
301     It is important to note that adding users like this is very bad. We only do
302     it here because there is no <c>useradd</c> utility (which you normally use for
303     adding users) yet at this stage of installation.
304     </impo>
305    
306 swift 1.10 <p>
307 swift 1.17 Install distcc:
308 swift 1.10 </p>
309    
310     <pre caption="Getting Distcc on the new box">
311 swift 1.24 # <i>USE='-*' emerge --nodeps sys-devel/distcc</i>
312 swift 1.8 </pre>
313 swift 1.10
314     </body>
315     </section>
316     <section>
317 swift 1.17 <title>Step 3: Setting Up Distcc</title>
318 swift 1.10 <body>
319    
320     <p>
321 swift 1.24 Run <c>distcc-config --install</c> to setup distcc:
322 swift 1.10 </p>
323    
324     <pre caption="Final distcc setup">
325 smithj 1.28 <comment>(Substitute host1, host2, ... with the IP number(s) of the
326     participating hosts)</comment>
327 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i>
328 smithj 1.28 <comment>An example: <i>/usr/bin/distcc-config --set-hosts "localhost
329     192.168.0.4 192.168.0.6"</i></comment>
330 swift 1.17 </pre>
331 swift 1.10
332     <p>
333 swift 1.25 Distcc is now set up to bootstrap! Continue with the official installation
334 swift 1.24 instructions and <e>do not forget</e> to re-emerge distcc after <c>emerge
335 swift 1.17 system</c>. This is to make sure that all of the dependencies you want are
336     installed as well.
337 swift 1.10 </p>
338    
339 swift 1.12 <note>
340 swift 1.25 During bootstrap and <c>emerge system</c> distcc may not appear to be used.
341 swift 1.17 This is expected as some ebuilds do not work well with distcc, so they
342     intentionally disable it.
343 swift 1.12 </note>
344    
345 swift 1.10 </body>
346     </section>
347 blubber 1.2 </chapter>
348 swift 1.1
349 swift 1.5 <chapter>
350 swift 1.10 <title>Troubleshooting</title>
351     <section>
352     <title>Mozilla and Xfree</title>
353     <body>
354    
355     <p>
356 swift 1.17 As you emerge various packages, you'll notice that some of them aren't being
357     distributed (and aren't being built in parallel). This is because the
358     developers of the Mozilla and Xfree ebuilds intentionally disable parallel
359     building because it is known to cause problems.
360 swift 1.10 </p>
361    
362     <p>
363 swift 1.17 Sometimes distcc might cause a package to fail to compile. If this happens
364     for you, please <uri link="http://bugs.gentoo.org">report</uri> it to us.
365 swift 1.10 </p>
366    
367     </body>
368     </section>
369     <section>
370     <title>A Mixture of hardened-gcc and non-hardened-gcc Hosts Will Be Faulty</title>
371     <body>
372    
373     <p>
374     With such a long title any explanation here is almost irrelevent. However, if
375     you plan to use distcc across hosts that have the <uri
376 swift 1.17 link="/proj/en/hardened/etdyn-ssp.xml">PaX/hardened-gcc</uri> and some that do
377     not, you will run into problems.
378 swift 1.10 </p>
379    
380     <p>
381     The solution requires a little foresight on your part; you have to run
382     <c>hardened-gcc -R</c> on the host that has PaX/hardened-gcc, or you have to
383     enable PaX protections in your kernel and <c>emerge hardened-gcc</c>. Both of
384     which are a good thing to do since for the most part the protections offered
385     by both packages is a good thing and is transparent to the user.
386     </p>
387    
388     </body>
389     </section>
390     <section>
391     <title>Mixed GCC Versions</title>
392     <body>
393    
394     <p>
395 swift 1.17 If you have different GCC versions on your hosts, there will likely be very
396 swift 1.10 weird problems. The solution is to make certain all hosts have the same GCC
397     version.
398     </p>
399    
400 swift 1.24 <p>
401     Recent Portage updates have made Portage use <c>${CHOST}-gcc</c> instead of
402     <c>gcc</c>. This means that if you're mixing i686 machines with other types
403     (i386, i586) you will run into problems. A workaround for this may be to
404 smithj 1.28 <c>export CC='gcc' CXX='c++'</c> or to put it in <path>/etc/make.conf</path>.
405 swift 1.24 </p>
406    
407 smithj 1.28 <impo>
408     Doing this explicitly redefines some behaviour of Portage and may have some
409     weird results in the future. Only do this if you're mixing CHOSTs.
410     </impo>
411    
412    
413 swift 1.10 </body>
414     </section>
415 swift 1.5 </chapter>
416 swift 1.1
417     <chapter>
418 swift 1.10 <title>Distcc Extras</title>
419     <section>
420     <title>Distcc Monitors</title>
421     <body>
422    
423     <p>
424 swift 1.17 Distcc ships with two monitors. The text-based one is always built and is
425 swift 1.10 called <c>distccmon-text</c>. Running it for the first time can be a bit
426     confusing, but it is really quite easy to use. If you run the program with no
427     parameter it will run once. However, if you pass it a number it will update
428     every N seconds, where N is the argument you passed.
429     </p>
430    
431     <p>
432 smithj 1.28 The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c>
433     in your <c>USE</c> flags. This one is GTK+ based, runs in an X environment
434     and it is quite lovely. For Gentoo the GUI monitor has been called
435     <c>distccmon-gui</c> for less confusion. Elsewhere it may be referred to as
436     <c>distccmon-gnome</c>.
437 swift 1.10 </p>
438    
439     <pre caption="Starting the monitors">
440 swift 1.17 # <i>distccmon-text N</i>
441 neysx 1.22 <comment>(Or)</comment>
442 smithj 1.28 # <i>distccmon-gui</i>
443 neysx 1.22 <comment>To monitor Portage's distcc usage you can use:</comment>
444 swift 1.26 # <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-text N</i>
445 neysx 1.22 <comment>(Or)</comment>
446 smithj 1.28 # <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-gui</i>
447 swift 1.8 </pre>
448 swift 1.10
449 neysx 1.22 <impo>
450 swift 1.26 If your distcc directory is elsewhere, change the DISTCC_DIR variable
451     accordingly.
452 neysx 1.22 </impo>
453 swift 1.10
454     </body>
455     </section>
456 swift 1.1 </chapter>
457     </guide>

  ViewVC Help
Powered by ViewVC 1.1.20