/[gentoo]/xml/htdocs/doc/en/distcc.xml
Gentoo

Contents of /xml/htdocs/doc/en/distcc.xml

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.34 - (hide annotations) (download) (as text)
Tue Feb 21 16:50:58 2006 UTC (8 years, 10 months ago) by jkt
Branch: MAIN
Changes since 1.33: +5 -5 lines
File MIME type: application/xml
Fixed link and license stuff

1 swift 1.1 <?xml version='1.0' encoding="UTF-8"?>
2 blubber 1.3 <!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3 jkt 1.34 <!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/distcc.xml,v 1.33 2006/02/18 23:21:44 nightmorph Exp $ -->
4 blubber 1.2
5 swift 1.1 <guide link="/doc/en/distcc.xml">
6    
7     <title>Gentoo Distcc Documentation</title>
8    
9     <author title="Author">
10 swift 1.10 <mail link="lisa@gentoo.org">Lisa Seelye</mail>
11 erwin 1.4 </author>
12     <author title="Editor">
13 swift 1.10 <mail link="vapier@gentoo.org">Mike Frysinger</mail>
14 erwin 1.4 </author>
15     <author title="Editor">
16 swift 1.10 <mail link="erwin@gentoo.org">Erwin</mail>
17 blubber 1.2 </author>
18 swift 1.17 <author title="Editor">
19 swift 1.10 <mail link="swift@gentoo.org">Sven Vermeulen</mail>
20 swift 1.1 </author>
21 pylon 1.20 <author title="Editor">
22     <mail link="pylon@gentoo.org">Lars Weiler</mail>
23     </author>
24 swift 1.17 <author title="Reviewer">
25 swift 1.10 <mail link="blubber@gentoo.org">Tiemo Kieft</mail>
26 swift 1.1 </author>
27    
28 swift 1.15 <abstract>
29     This document serves as a HOWTO for using distcc with Gentoo.
30     </abstract>
31    
32 swift 1.17 <!-- The content of this document is licensed under the CC-BY-SA license -->
33 jkt 1.34 <!-- See http://creativecommons.org/licenses/by-sa/2.5 -->
34 swift 1.6 <license/>
35    
36 jkt 1.34 <version>1.15</version>
37 nightmorph 1.33 <date>2006-02-18</date>
38 swift 1.1
39     <chapter>
40 swift 1.10 <title>Introduction</title>
41     <section>
42     <title>What is distcc?</title>
43     <body>
44    
45     <p>
46     Distcc is a program designed to distribute compiling tasks across a network to
47 smithj 1.27 participating hosts. It is comprised of a server, <c>distccd</c>, and a client
48 swift 1.10 program, <c>distcc</c>. Distcc can work transparently with <uri
49 smithj 1.28 link="http://ccache.samba.org">ccache</uri>, Portage, and Automake with a
50     little setup.
51 swift 1.17 </p>
52    
53     </body>
54     </section>
55     <section>
56     <title>Using distcc to bootstrap</title>
57     <body>
58    
59     <p>
60 smithj 1.28 If you are planning on using distcc to help you bootstrap a Gentoo
61     installation, make sure you read the section <uri link="#bootstrapping">Using
62     distcc to Bootstrap</uri>, which is situated further down in this document.
63 swift 1.10 </p>
64    
65     </body>
66     </section>
67 swift 1.8 </chapter>
68    
69     <chapter>
70 swift 1.10 <title>Setup</title>
71     <section>
72     <title>Dependencies</title>
73     <body>
74    
75 swift 1.17 <p>
76 smithj 1.28 In order to use Distcc, all of the computers on your network need to have the
77     same GCC versions. For example, mixing 3.3.x (where the x varies) is okay, but
78     mixing 3.3.x with 3.2.x <b>may</b> result in compilation errors or runtime
79     errors.
80 swift 1.17 </p>
81 swift 1.10
82     </body>
83     </section>
84     <section>
85     <title>Installing Distcc</title>
86     <body>
87    
88     <p>
89 swift 1.25 There are a couple of options you should be aware of before you start
90 swift 1.17 installing distcc.
91 swift 1.10 </p>
92    
93     <p>
94     Distcc ships with a graphical monitor to monitor tasks that your computer is
95     sending away for compilation. If you use Gnome then put 'gnome' in your USE
96     flags. However, if you don't use Gnome and would still like to have the
97     monitor then you should put 'gtk' in your USE flags.
98     </p>
99    
100 swift 1.17 <pre caption="Installing distcc">
101     # <i>emerge distcc</i>
102     </pre>
103    
104 swift 1.10 </body>
105     </section>
106     <section>
107     <title>Setting up Portage to use Distcc</title>
108     <body>
109    
110     <p>
111 swift 1.25 Setting up Portage to use distcc is easy. Execute the following steps on
112 swift 1.17 each system that should participate in the distributed compiling:
113 swift 1.10 </p>
114    
115     <pre caption="Integrating Distcc and Portage">
116 swift 1.1 # <i>emerge distcc</i>
117     # <i>nano -w /etc/make.conf</i>
118 swift 1.17 <comment>(Set N to a suitable number for your particular setup)</comment>
119 smithj 1.28 <comment>(A common strategy is setting N as twice the number of total CPUs + 1
120     available)</comment>
121 swift 1.17 MAKEOPTS="-jN"
122 smithj 1.28 <comment>(Add distcc to your FEATURES)</comment>
123 swift 1.17 FEATURES="distcc"
124 swift 1.5 </pre>
125 swift 1.1
126 swift 1.25 </body>
127     </section>
128     <section>
129     <title>Specifying Participating Hosts</title>
130     <body>
131    
132 swift 1.10 <p>
133 smithj 1.28 Use the <c>distcc-config</c> command to set the list of hosts. Here is an
134     example of some hosts that might be in your list:
135 swift 1.10 </p>
136    
137     <pre caption="Examples of host definitions">
138 swift 1.8 192.168.0.1 192.168.0.2 192.168.0.3
139     192.168.0.1/2 192.168.0.2 192.168.0.3/10
140     192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4
141     @192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3
142 swift 1.17 <comment>(There are also several other methods of setting up hosts. See the
143     distcc manpage for more details.)</comment>
144 smithj 1.28 <comment>If you wish to compile on the local machine you should put 'localhost'
145     in the hosts list. Conversely if you do not wish to use the local machine to
146     compile (which is often the case) omit it from the hosts list. On a slow
147     machine using localhost may actually slow things down. Make sure to test your
148     settings for performance.</comment>
149 swift 1.8 </pre>
150 swift 1.10
151     <p>
152     It may all look complicated, but in most cases a variant of line 1 or 2 will
153 swift 1.17 work.
154 swift 1.10 </p>
155    
156     <p>
157 swift 1.17 Since most people won't be using lines 3 or 4, I'll <uri
158 smithj 1.28 link="http://distcc.samba.org/man/distcc_1.html">refer to</uri> the distcc
159     docs (man distcc) for more information.
160 swift 1.17 </p>
161    
162     <p>
163     For instance, to set the first line in the previous example:
164 swift 1.10 </p>
165    
166     <pre caption="Sample command to set the hosts">
167 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "192.168.0.1 192.168.0.2 192.168.0.3"</i>
168 swift 1.8 </pre>
169 swift 1.1
170 swift 1.10 <p>
171 swift 1.23 Edit <path>/etc/conf.d/distccd</path> to your needs and be sure to set the
172 smithj 1.28 <c>--allow</c> directive to allow only hosts you trust. For added security,
173     you should also use the <c>--listen</c> directive to tell the distcc daemon
174     what IP to listen on (for multi-homed systems). More information on distcc
175     security can be found at <uri
176     link="http://distcc.samba.org/security.html">Distcc Security Design</uri>.
177 swift 1.23 </p>
178    
179 smithj 1.28 <impo>
180     It is important to use --allow and --listen. Please read the distccd manpage
181 alin 1.30 or the above security document for more information.
182 smithj 1.28 </impo>
183    
184    
185 swift 1.23 <p>
186 swift 1.17 Now start the distcc daemon on all the participating computers:
187 swift 1.10 </p>
188    
189 swift 1.17 <pre caption="Starting the distcc daemon">
190     <comment>(Add distccd to the default runlevel)</comment>
191     # <i>rc-update add distccd default</i>
192     <comment>(Start the distcc daemon)</comment>
193     # <i>/etc/init.d/distccd start</i>
194 swift 1.8 </pre>
195 swift 1.10
196     </body>
197     </section>
198     <section>
199     <title>Setting up Distcc to Work With Automake</title>
200     <body>
201    
202     <p>
203 swift 1.25 This is, in some cases, easier than the Portage setup. What you have to do is
204 swift 1.17 update your <c>PATH</c> variable to include <path>/usr/lib/distcc/bin</path>
205     in front of the directory that contains <c>gcc</c> (<path>/usr/bin</path>).
206 swift 1.10 However, there is a caveat. If you use ccache you have to put distcc after
207 swift 1.17 the ccache part:
208 swift 1.10 </p>
209    
210     <pre caption="Setting your path">
211 swift 1.1 # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
212 smithj 1.28 <comment>You can put this in your .bashrc or equivelant file to have the PATH
213     set every time you log in</comment>
214 swift 1.8 </pre>
215 swift 1.10
216     <p>
217 neysx 1.22 Then, as you would normally type <c>make</c>, you would type <c>make -jN</c>
218     (where N is an integer). The value of N depends on your network and the types
219 smithj 1.28 of computers you are using to compile. Test your own settings to find the
220     number that yields the best performance.
221 swift 1.10 </p>
222    
223     </body>
224     </section>
225 blubber 1.2 </chapter>
226 swift 1.1
227 blubber 1.2 <chapter>
228 swift 1.17 <title>Cross-Compiling</title>
229     <section>
230 swift 1.10 <body>
231    
232     <p>
233 neysx 1.32 Cross-compiling is using one architecture to build programs for another
234     architecture. This can be as simple as using an Athlon (i686) to build a
235     program for a K6-2 (i586), or using a Sparc to build a program for a ppc. This
236     is documented in our <uri link="/doc/en/cross-compiling-distcc.xml">DistCC
237     Cross-compiling Guide</uri>.
238 swift 1.17 </p>
239    
240     </body>
241     </section>
242     </chapter>
243    
244     <chapter id="bootstrapping">
245     <title>Using Distcc to Bootstrap</title>
246     <section>
247 swift 1.24 <title>Step 1: Configure Portage</title>
248 swift 1.17 <body>
249 nightmorph 1.33 <!--
250     Note that this will need to become a link to the old 2005.1 networked handbooks
251     once the 2006.0 handbooks are released. Until then, the link can stay.
252     -->
253     <p>
254     Boot your new box with a Gentoo Linux LiveCD and follow the <uri
255     link="/doc/en/handbook/handbook-x86.xml?part=1">installation instructions</uri>
256     up until the bootstrapping part. (See the <uri
257 jkt 1.34 link="/doc/en/faq.xml#stage12">Gentoo FAQ</uri> for more information about
258     bootstrapping.) Then configure Portage to use distcc:
259 swift 1.10 </p>
260    
261 swift 1.17 <pre caption="Preliminary Setup">
262     # <i>nano -w /etc/make.conf</i>
263 smithj 1.28 <comment>(Add distcc to the FEATURES</comment>
264 swift 1.17 FEATURES="distcc"
265 smithj 1.28 <comment>(Modify MAKEOPTS to include -jN, where N is twice the number of CPUs
266     +1 available)</comment>
267 swift 1.17 MAKEOPTS="-jN"
268 swift 1.8 </pre>
269 swift 1.10
270 neysx 1.22 <pre caption="Setting your path">
271     # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
272     </pre>
273    
274 swift 1.10 </body>
275     </section>
276     <section>
277     <title>Step 2: Getting Distcc</title>
278     <body>
279    
280 neysx 1.22 <p>
281     Before the installation of distcc, a user called distcc must be added to the
282     <path>/etc/passwd</path>:
283     </p>
284 pylon 1.20
285     <pre caption="Create user distcc">
286 smithj 1.28 # <i>echo "distcc:x:240:2:distccd:/dev/null:/bin/false" &gt;&gt;/etc/passwd</i>
287 pylon 1.20 </pre>
288    
289 swift 1.24 <impo>
290     It is important to note that adding users like this is very bad. We only do
291     it here because there is no <c>useradd</c> utility (which you normally use for
292     adding users) yet at this stage of installation.
293     </impo>
294    
295 swift 1.10 <p>
296 swift 1.17 Install distcc:
297 swift 1.10 </p>
298    
299     <pre caption="Getting Distcc on the new box">
300 swift 1.24 # <i>USE='-*' emerge --nodeps sys-devel/distcc</i>
301 swift 1.8 </pre>
302 swift 1.10
303     </body>
304     </section>
305     <section>
306 swift 1.17 <title>Step 3: Setting Up Distcc</title>
307 swift 1.10 <body>
308    
309     <p>
310 swift 1.24 Run <c>distcc-config --install</c> to setup distcc:
311 swift 1.10 </p>
312    
313     <pre caption="Final distcc setup">
314 smithj 1.28 <comment>(Substitute host1, host2, ... with the IP number(s) of the
315     participating hosts)</comment>
316 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i>
317 smithj 1.28 <comment>An example: <i>/usr/bin/distcc-config --set-hosts "localhost
318     192.168.0.4 192.168.0.6"</i></comment>
319 neysx 1.31 </pre>
320 swift 1.10
321     <p>
322 swift 1.25 Distcc is now set up to bootstrap! Continue with the official installation
323 swift 1.24 instructions and <e>do not forget</e> to re-emerge distcc after <c>emerge
324 swift 1.17 system</c>. This is to make sure that all of the dependencies you want are
325     installed as well.
326 swift 1.10 </p>
327    
328 swift 1.12 <note>
329 swift 1.25 During bootstrap and <c>emerge system</c> distcc may not appear to be used.
330 swift 1.17 This is expected as some ebuilds do not work well with distcc, so they
331     intentionally disable it.
332 swift 1.12 </note>
333    
334 swift 1.10 </body>
335     </section>
336 blubber 1.2 </chapter>
337 swift 1.1
338 swift 1.5 <chapter>
339 swift 1.10 <title>Troubleshooting</title>
340     <section>
341 nightmorph 1.33 <title>Mozilla and Xorg</title>
342 swift 1.10 <body>
343    
344     <p>
345 nightmorph 1.33 As you emerge various packages, you'll notice that some of them aren't being
346     distributed (and aren't being built in parallel). This is because the
347     developers of the Mozilla and Xorg-X11 ebuilds intentionally disable parallel
348 swift 1.17 building because it is known to cause problems.
349 swift 1.10 </p>
350    
351     <p>
352 swift 1.17 Sometimes distcc might cause a package to fail to compile. If this happens
353     for you, please <uri link="http://bugs.gentoo.org">report</uri> it to us.
354 swift 1.10 </p>
355    
356     </body>
357     </section>
358     <section>
359     <title>Mixed GCC Versions</title>
360     <body>
361    
362     <p>
363 swift 1.17 If you have different GCC versions on your hosts, there will likely be very
364 swift 1.10 weird problems. The solution is to make certain all hosts have the same GCC
365     version.
366     </p>
367    
368 swift 1.24 <p>
369     Recent Portage updates have made Portage use <c>${CHOST}-gcc</c> instead of
370     <c>gcc</c>. This means that if you're mixing i686 machines with other types
371     (i386, i586) you will run into problems. A workaround for this may be to
372 smithj 1.28 <c>export CC='gcc' CXX='c++'</c> or to put it in <path>/etc/make.conf</path>.
373 swift 1.24 </p>
374    
375 smithj 1.28 <impo>
376     Doing this explicitly redefines some behaviour of Portage and may have some
377     weird results in the future. Only do this if you're mixing CHOSTs.
378     </impo>
379    
380    
381 swift 1.10 </body>
382     </section>
383 swift 1.5 </chapter>
384 swift 1.1
385     <chapter>
386 swift 1.10 <title>Distcc Extras</title>
387     <section>
388     <title>Distcc Monitors</title>
389     <body>
390    
391     <p>
392 swift 1.17 Distcc ships with two monitors. The text-based one is always built and is
393 swift 1.10 called <c>distccmon-text</c>. Running it for the first time can be a bit
394     confusing, but it is really quite easy to use. If you run the program with no
395     parameter it will run once. However, if you pass it a number it will update
396     every N seconds, where N is the argument you passed.
397     </p>
398    
399     <p>
400 smithj 1.28 The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c>
401     in your <c>USE</c> flags. This one is GTK+ based, runs in an X environment
402     and it is quite lovely. For Gentoo the GUI monitor has been called
403     <c>distccmon-gui</c> for less confusion. Elsewhere it may be referred to as
404     <c>distccmon-gnome</c>.
405 swift 1.10 </p>
406    
407     <pre caption="Starting the monitors">
408 swift 1.17 # <i>distccmon-text N</i>
409 neysx 1.22 <comment>(Or)</comment>
410 smithj 1.28 # <i>distccmon-gui</i>
411 neysx 1.22 <comment>To monitor Portage's distcc usage you can use:</comment>
412 swift 1.26 # <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-text N</i>
413 neysx 1.22 <comment>(Or)</comment>
414 smithj 1.28 # <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-gui</i>
415 swift 1.8 </pre>
416 swift 1.10
417 neysx 1.22 <impo>
418 swift 1.26 If your distcc directory is elsewhere, change the DISTCC_DIR variable
419     accordingly.
420 neysx 1.22 </impo>
421 swift 1.10
422     </body>
423     </section>
424 swift 1.1 </chapter>
425     </guide>

  ViewVC Help
Powered by ViewVC 1.1.20