/[gentoo]/xml/htdocs/doc/en/distcc.xml
Gentoo

Contents of /xml/htdocs/doc/en/distcc.xml

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.27 - (hide annotations) (download) (as text)
Fri Jul 1 22:35:19 2005 UTC (9 years, 3 months ago) by smithj
Branch: MAIN
Changes since 1.26: +3 -3 lines
File MIME type: application/xml
#42712: a few minor english fixes

1 swift 1.1 <?xml version='1.0' encoding="UTF-8"?>
2 blubber 1.3 <!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3 smithj 1.27 <!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/distcc.xml,v 1.26 2005/03/25 14:43:23 swift Exp $ -->
4 blubber 1.2
5 swift 1.1 <guide link="/doc/en/distcc.xml">
6    
7     <title>Gentoo Distcc Documentation</title>
8    
9     <author title="Author">
10 swift 1.10 <mail link="lisa@gentoo.org">Lisa Seelye</mail>
11 erwin 1.4 </author>
12 swift 1.17
13 erwin 1.4 <author title="Editor">
14 swift 1.10 <mail link="vapier@gentoo.org">Mike Frysinger</mail>
15 erwin 1.4 </author>
16 swift 1.17
17 erwin 1.4 <author title="Editor">
18 swift 1.10 <mail link="erwin@gentoo.org">Erwin</mail>
19 blubber 1.2 </author>
20 swift 1.17
21     <author title="Editor">
22 swift 1.10 <mail link="swift@gentoo.org">Sven Vermeulen</mail>
23 swift 1.1 </author>
24 swift 1.17
25 pylon 1.20 <author title="Editor">
26     <mail link="pylon@gentoo.org">Lars Weiler</mail>
27     </author>
28    
29 swift 1.17 <author title="Reviewer">
30 swift 1.10 <mail link="blubber@gentoo.org">Tiemo Kieft</mail>
31 swift 1.1 </author>
32    
33 swift 1.15 <abstract>
34     This document serves as a HOWTO for using distcc with Gentoo.
35     </abstract>
36    
37 swift 1.17 <!-- The content of this document is licensed under the CC-BY-SA license -->
38     <!-- See http://creativecommons.org/licenses/by-sa/1.0 -->
39 swift 1.6 <license/>
40    
41 swift 1.26 <version>1.11</version>
42     <date>2005-03-25</date>
43 swift 1.1
44     <chapter>
45 swift 1.10 <title>Introduction</title>
46     <section>
47     <title>What is distcc?</title>
48     <body>
49    
50     <p>
51     Distcc is a program designed to distribute compiling tasks across a network to
52 smithj 1.27 participating hosts. It is comprised of a server, <c>distccd</c>, and a client
53 swift 1.10 program, <c>distcc</c>. Distcc can work transparently with <uri
54 swift 1.17 link="http://ccache.samba.org">ccache</uri> and Portage with a little work.
55     </p>
56    
57     </body>
58     </section>
59     <section>
60     <title>Using distcc to bootstrap</title>
61     <body>
62    
63     <p>
64     If you are planning on using distcc to help you bootstrap a Gentoo installation,
65 smithj 1.27 make sure you read the section <uri link="#bootstrapping">Using distcc to
66 swift 1.25 Bootstrap</uri>, which is situated further down in this document.
67 swift 1.10 </p>
68    
69     </body>
70     </section>
71 swift 1.8 </chapter>
72    
73     <chapter>
74 swift 1.10 <title>Setup</title>
75     <section>
76     <title>Dependencies</title>
77     <body>
78    
79 swift 1.17 <p>
80 swift 1.25 In order to use Distcc, all of the computers on your network need to
81     have the same GCC versions. Mixing 3.3.x (where the x varies) is okay, but
82     mixing 3.3.x and 3.2.x <b>will</b> result in your programs having errors in them.
83 swift 1.17 </p>
84 swift 1.10
85     </body>
86     </section>
87     <section>
88     <title>Installing Distcc</title>
89     <body>
90    
91     <p>
92 swift 1.25 There are a couple of options you should be aware of before you start
93 swift 1.17 installing distcc.
94 swift 1.10 </p>
95    
96     <p>
97     Distcc ships with a graphical monitor to monitor tasks that your computer is
98     sending away for compilation. If you use Gnome then put 'gnome' in your USE
99     flags. However, if you don't use Gnome and would still like to have the
100     monitor then you should put 'gtk' in your USE flags.
101     </p>
102    
103 swift 1.17 <pre caption="Installing distcc">
104     # <i>emerge distcc</i>
105     </pre>
106    
107 swift 1.10 </body>
108     </section>
109     <section>
110     <title>Setting up Portage to use Distcc</title>
111     <body>
112    
113     <p>
114 swift 1.25 Setting up Portage to use distcc is easy. Execute the following steps on
115 swift 1.17 each system that should participate in the distributed compiling:
116 swift 1.10 </p>
117    
118     <pre caption="Integrating Distcc and Portage">
119 swift 1.1 # <i>emerge distcc</i>
120     # <i>nano -w /etc/make.conf</i>
121 swift 1.17 <comment>(Set N to a suitable number for your particular setup)</comment>
122 neysx 1.22 <comment>(A common heuristic strategy is twice the number of total CPUs + 1)</comment>
123 swift 1.17 MAKEOPTS="-jN"
124     <comment>(Add distcc to your FEATURES variable)</comment>
125     FEATURES="distcc"
126 swift 1.5 </pre>
127 swift 1.1
128 swift 1.25 </body>
129     </section>
130     <section>
131     <title>Specifying Participating Hosts</title>
132     <body>
133    
134 swift 1.10 <p>
135 swift 1.25 Use the <c>distcc-config</c> command to set the list of hosts. Here is an example of
136 swift 1.17 some hosts that might be in your list:
137 swift 1.10 </p>
138    
139     <pre caption="Examples of host definitions">
140 swift 1.8 192.168.0.1 192.168.0.2 192.168.0.3
141     192.168.0.1/2 192.168.0.2 192.168.0.3/10
142     192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4
143     @192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3
144 swift 1.17 <comment>(There are also several other methods of setting up hosts. See the
145     distcc manpage for more details.)</comment>
146 swift 1.8 </pre>
147 swift 1.10
148     <p>
149     It may all look complicated, but in most cases a variant of line 1 or 2 will
150 swift 1.17 work.
151 swift 1.10 </p>
152    
153 swift 1.17 <ul>
154     <li>Line 1 is just a space-delimited list of hosts</li>
155     <li>
156     Line 2 is a list of hosts that specifies the maximum amount of jobs (by
157     defining the <c>/N</c> setting) to send that host at any given time
158     </li>
159     </ul>
160    
161 swift 1.10 <p>
162 swift 1.17 Since most people won't be using lines 3 or 4, I'll <uri
163     link="http://distcc.samba.org/man/distcc_1.html">point you to</uri> the distcc
164     docs for more information.
165     </p>
166    
167     <p>
168     For instance, to set the first line in the previous example:
169 swift 1.10 </p>
170    
171     <pre caption="Sample command to set the hosts">
172 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "192.168.0.1 192.168.0.2 192.168.0.3"</i>
173 swift 1.8 </pre>
174 swift 1.1
175 swift 1.10 <p>
176 swift 1.23 Edit <path>/etc/conf.d/distccd</path> to your needs and be sure to set the
177 swift 1.25 <c>--allow</c> directive to allow only hosts you trust. For added security, you should also use the <c>--listen</c>
178     directive to tell the distcc daemon what IP to listen on (for multi-homed
179     systems). More information on distcc security can be found at <uri link="http://distcc.samba.org/security.html">Distcc Security
180     Design</uri>.
181 swift 1.23 </p>
182    
183     <p>
184 swift 1.17 Now start the distcc daemon on all the participating computers:
185 swift 1.10 </p>
186    
187 swift 1.17 <pre caption="Starting the distcc daemon">
188     <comment>(Add distccd to the default runlevel)</comment>
189     # <i>rc-update add distccd default</i>
190     <comment>(Start the distcc daemon)</comment>
191     # <i>/etc/init.d/distccd start</i>
192 swift 1.8 </pre>
193 swift 1.10
194     </body>
195     </section>
196     <section>
197     <title>Setting up Distcc to Work With Automake</title>
198     <body>
199    
200     <p>
201 swift 1.25 This is, in some cases, easier than the Portage setup. What you have to do is
202 swift 1.17 update your <c>PATH</c> variable to include <path>/usr/lib/distcc/bin</path>
203     in front of the directory that contains <c>gcc</c> (<path>/usr/bin</path>).
204 swift 1.10 However, there is a caveat. If you use ccache you have to put distcc after
205 swift 1.17 the ccache part:
206 swift 1.10 </p>
207    
208     <pre caption="Setting your path">
209 swift 1.1 # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
210 swift 1.8 </pre>
211 swift 1.10
212     <p>
213 neysx 1.22 Then, as you would normally type <c>make</c>, you would type <c>make -jN</c>
214     (where N is an integer). The value of N depends on your network and the types
215     of computers you are using to compile. For me, I have a dual processor P3 and
216     a K6-2/400 that compiles everything for my desktop (a 1200MHz Athlon
217     Thunderbird) and I use -j5. Test your own settings to find the number that
218     yields the best performance.
219 swift 1.10 </p>
220    
221     </body>
222     </section>
223 blubber 1.2 </chapter>
224 swift 1.1
225 blubber 1.2 <chapter>
226 swift 1.17 <title>Cross-Compiling</title>
227     <section>
228     <title>A Note on Cross-Compiling</title>
229     <body>
230    
231     <p>
232     Cross-compiling is using one architecture to build programs for another
233     architecture. This can be as simple as using an Athlon (i686) to build a
234     program for a K6-2 (i586), or using a Sparc to build a program for a ppc.
235     </p>
236    
237     </body>
238     </section>
239 swift 1.10 <section>
240 swift 1.25 <title>An Introduction to Cross-Compiling</title>
241 swift 1.10 <body>
242    
243     <p>
244 swift 1.25 If you want to give cross-compiling a try you can to follow <uri
245 swift 1.24 link="http://dev.gentoo.org/~vapier/CROSS-COMPILE-HOWTO">The Cross Compile
246 swift 1.25 HOWTO</uri>; Crossdev is deprecated.
247 swift 1.17 </p>
248    
249     </body>
250     </section>
251     </chapter>
252    
253     <chapter id="bootstrapping">
254     <title>Using Distcc to Bootstrap</title>
255     <section>
256 swift 1.24 <title>Step 1: Configure Portage</title>
257 swift 1.17 <body>
258 swift 1.10
259     <p>
260 swift 1.17 Boot your new box with a Gentoo Linux LiveCD and follow the <uri
261     link="/doc/en/handbook/handbook-x86.xml?part=1">installation instructions</uri>
262     up until the bootstrapping part. Then configure Portage to use distcc:
263 swift 1.10 </p>
264    
265 swift 1.17 <pre caption="Preliminary Setup">
266     # <i>nano -w /etc/make.conf</i>
267     <comment>(Add distcc to the FEATURES variable)</comment>
268     FEATURES="distcc"
269     <comment>(Modify MAKEOPTS to include -jN with N an appropriate integer)</comment>
270     MAKEOPTS="-jN"
271 swift 1.8 </pre>
272 swift 1.10
273 neysx 1.22 <pre caption="Setting your path">
274     # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
275     </pre>
276    
277 swift 1.10 </body>
278     </section>
279     <section>
280     <title>Step 2: Getting Distcc</title>
281     <body>
282    
283 neysx 1.22 <p>
284     Before the installation of distcc, a user called distcc must be added to the
285     <path>/etc/passwd</path>:
286     </p>
287 pylon 1.20
288     <pre caption="Create user distcc">
289     # <i>echo "distcc:x:240:2:distccd:/dev/null:/bin/false" &gt;&gt; /etc/passwd</i>
290     </pre>
291    
292 swift 1.24 <impo>
293     It is important to note that adding users like this is very bad. We only do
294     it here because there is no <c>useradd</c> utility (which you normally use for
295     adding users) yet at this stage of installation.
296     </impo>
297    
298 swift 1.10 <p>
299 swift 1.17 Install distcc:
300 swift 1.10 </p>
301    
302     <pre caption="Getting Distcc on the new box">
303 swift 1.24 # <i>USE='-*' emerge --nodeps sys-devel/distcc</i>
304 swift 1.8 </pre>
305 swift 1.10
306     </body>
307     </section>
308     <section>
309 swift 1.17 <title>Step 3: Setting Up Distcc</title>
310 swift 1.10 <body>
311    
312     <p>
313 swift 1.24 Run <c>distcc-config --install</c> to setup distcc:
314 swift 1.10 </p>
315    
316     <pre caption="Final distcc setup">
317 swift 1.17 <comment>(Substitute host1, host2, ... with the IP number(s) of the participating hosts)</comment>
318 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i>
319 neysx 1.22 <comment>An example: <i>/usr/bin/distcc-config --set-hosts "localhost 192.168.0.4 192.168.0.6"</i></comment>
320 swift 1.17 </pre>
321 swift 1.10
322     <p>
323 swift 1.25 Distcc is now set up to bootstrap! Continue with the official installation
324 swift 1.24 instructions and <e>do not forget</e> to re-emerge distcc after <c>emerge
325 swift 1.17 system</c>. This is to make sure that all of the dependencies you want are
326     installed as well.
327 swift 1.10 </p>
328    
329 swift 1.12 <note>
330 swift 1.25 During bootstrap and <c>emerge system</c> distcc may not appear to be used.
331 swift 1.17 This is expected as some ebuilds do not work well with distcc, so they
332     intentionally disable it.
333 swift 1.12 </note>
334    
335 swift 1.10 </body>
336     </section>
337 blubber 1.2 </chapter>
338 swift 1.1
339 swift 1.5 <chapter>
340 swift 1.10 <title>Troubleshooting</title>
341     <section>
342     <title>Mozilla and Xfree</title>
343     <body>
344    
345     <p>
346 swift 1.17 As you emerge various packages, you'll notice that some of them aren't being
347     distributed (and aren't being built in parallel). This is because the
348     developers of the Mozilla and Xfree ebuilds intentionally disable parallel
349     building because it is known to cause problems.
350 swift 1.10 </p>
351    
352     <p>
353 swift 1.17 Sometimes distcc might cause a package to fail to compile. If this happens
354     for you, please <uri link="http://bugs.gentoo.org">report</uri> it to us.
355 swift 1.10 </p>
356    
357     </body>
358     </section>
359     <section>
360     <title>A Mixture of hardened-gcc and non-hardened-gcc Hosts Will Be Faulty</title>
361     <body>
362    
363     <p>
364     With such a long title any explanation here is almost irrelevent. However, if
365     you plan to use distcc across hosts that have the <uri
366 swift 1.17 link="/proj/en/hardened/etdyn-ssp.xml">PaX/hardened-gcc</uri> and some that do
367     not, you will run into problems.
368 swift 1.10 </p>
369    
370     <p>
371     The solution requires a little foresight on your part; you have to run
372     <c>hardened-gcc -R</c> on the host that has PaX/hardened-gcc, or you have to
373     enable PaX protections in your kernel and <c>emerge hardened-gcc</c>. Both of
374     which are a good thing to do since for the most part the protections offered
375     by both packages is a good thing and is transparent to the user.
376     </p>
377    
378     </body>
379     </section>
380     <section>
381     <title>Mixed GCC Versions</title>
382     <body>
383    
384     <p>
385 swift 1.17 If you have different GCC versions on your hosts, there will likely be very
386 swift 1.10 weird problems. The solution is to make certain all hosts have the same GCC
387     version.
388     </p>
389    
390 swift 1.24 <p>
391     Recent Portage updates have made Portage use <c>${CHOST}-gcc</c> instead of
392     <c>gcc</c>. This means that if you're mixing i686 machines with other types
393     (i386, i586) you will run into problems. A workaround for this may be to
394     <c>export CC='gcc' CXX='c++'</c> or to put it in <path>/etc/make.conf</path>,
395     but that needs testing. If it works for you please email me,
396     <mail link="lisa@gentoo.org">Lisa Seelye</mail>.
397     </p>
398    
399 swift 1.10 </body>
400     </section>
401 swift 1.5 </chapter>
402 swift 1.1
403     <chapter>
404 swift 1.10 <title>Distcc Extras</title>
405     <section>
406     <title>Distcc Monitors</title>
407     <body>
408    
409     <p>
410 swift 1.17 Distcc ships with two monitors. The text-based one is always built and is
411 swift 1.10 called <c>distccmon-text</c>. Running it for the first time can be a bit
412     confusing, but it is really quite easy to use. If you run the program with no
413     parameter it will run once. However, if you pass it a number it will update
414     every N seconds, where N is the argument you passed.
415     </p>
416    
417     <p>
418     The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c>
419 swift 1.17 in your <c>USE</c> flags. This one is GTK+ based, runs in an X environment
420 swift 1.10 and it is quite lovely.
421     </p>
422    
423     <pre caption="Starting the monitors">
424 swift 1.17 # <i>distccmon-text N</i>
425 neysx 1.22 <comment>(Or)</comment>
426 swift 1.17 # <i>distccmon-gnome</i>
427 neysx 1.22 <comment>To monitor Portage's distcc usage you can use:</comment>
428 swift 1.26 # <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-text N</i>
429 neysx 1.22 <comment>(Or)</comment>
430 swift 1.26 # <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-gnome</i>
431 swift 1.8 </pre>
432 swift 1.10
433 neysx 1.22 <impo>
434 swift 1.26 If your distcc directory is elsewhere, change the DISTCC_DIR variable
435     accordingly.
436 neysx 1.22 </impo>
437 swift 1.10
438     </body>
439     </section>
440 swift 1.1 </chapter>
441     </guide>

  ViewVC Help
Powered by ViewVC 1.1.20