/[gentoo]/xml/htdocs/doc/en/distcc.xml
Gentoo

Contents of /xml/htdocs/doc/en/distcc.xml

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.20 - (hide annotations) (download) (as text)
Sun Jul 11 13:13:49 2004 UTC (9 years, 9 months ago) by pylon
Branch: MAIN
Changes since 1.19: +15 -6 lines
File MIME type: application/xml
The user distcc must be created before installing distcc for bootstrap usage.

1 swift 1.1 <?xml version='1.0' encoding="UTF-8"?>
2 blubber 1.3 <!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3 pylon 1.20 <!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/distcc.xml,v 1.19 2004/07/02 15:47:15 swift Exp $ -->
4 swift 1.17
5     <!--
6 swift 1.18 Synchronised with rev 1.15 on
7 swift 1.17 http://www.thedoh.com/viewcvs/distcc/html/distcc.xml
8     -->
9 blubber 1.2
10 swift 1.1 <guide link="/doc/en/distcc.xml">
11    
12     <title>Gentoo Distcc Documentation</title>
13    
14     <author title="Author">
15 swift 1.10 <mail link="lisa@gentoo.org">Lisa Seelye</mail>
16 erwin 1.4 </author>
17 swift 1.17
18 erwin 1.4 <author title="Editor">
19 swift 1.10 <mail link="vapier@gentoo.org">Mike Frysinger</mail>
20 erwin 1.4 </author>
21 swift 1.17
22 erwin 1.4 <author title="Editor">
23 swift 1.10 <mail link="erwin@gentoo.org">Erwin</mail>
24 blubber 1.2 </author>
25 swift 1.17
26     <author title="Editor">
27 swift 1.10 <mail link="swift@gentoo.org">Sven Vermeulen</mail>
28 swift 1.1 </author>
29 swift 1.17
30 pylon 1.20 <author title="Editor">
31     <mail link="pylon@gentoo.org">Lars Weiler</mail>
32     </author>
33    
34 swift 1.17 <author title="Reviewer">
35 swift 1.10 <mail link="blubber@gentoo.org">Tiemo Kieft</mail>
36 swift 1.1 </author>
37    
38 swift 1.15 <abstract>
39     This document serves as a HOWTO for using distcc with Gentoo.
40     </abstract>
41    
42 swift 1.17 <!-- The content of this document is licensed under the CC-BY-SA license -->
43     <!-- See http://creativecommons.org/licenses/by-sa/1.0 -->
44 swift 1.6 <license/>
45    
46 pylon 1.20 <version>1.7.1</version>
47     <date>July 11, 2004</date>
48 swift 1.1
49     <chapter>
50 swift 1.10 <title>Introduction</title>
51     <section>
52     <title>What is distcc?</title>
53     <body>
54    
55     <p>
56     Distcc is a program designed to distribute compiling tasks across a network to
57     participating hosts. It is comprised of a server, <c>distccd</c> and a client
58     program, <c>distcc</c>. Distcc can work transparently with <uri
59 swift 1.17 link="http://ccache.samba.org">ccache</uri> and Portage with a little work.
60     </p>
61    
62     </body>
63     </section>
64     <section>
65     <title>Using distcc to bootstrap</title>
66     <body>
67    
68     <p>
69     If you are planning on using distcc to help you bootstrap a Gentoo installation,
70     make sure you read the part on <uri link="#bootstrapping">Using distcc to
71     Bootstrap</uri> which is situated further down in this document.
72 swift 1.10 </p>
73    
74     </body>
75     </section>
76 swift 1.8 </chapter>
77    
78     <chapter>
79 swift 1.10 <title>Setup</title>
80     <section>
81     <title>Dependencies</title>
82     <body>
83    
84 swift 1.17 <p>
85     To start off with, to use Distcc all of the computers on your network need to
86     have the same GCC versions. Mixing 3.3.x (where the x varies) is okay, but
87     mixing 3.3.x and 3.2.x is not: your programs <b>will</b> have errors in it.
88     </p>
89 swift 1.10
90     </body>
91     </section>
92     <section>
93     <title>Installing Distcc</title>
94     <body>
95    
96     <p>
97 swift 1.17 There are a couple of options you should know about before you start
98     installing distcc.
99 swift 1.10 </p>
100    
101     <p>
102     Distcc ships with a graphical monitor to monitor tasks that your computer is
103     sending away for compilation. If you use Gnome then put 'gnome' in your USE
104     flags. However, if you don't use Gnome and would still like to have the
105     monitor then you should put 'gtk' in your USE flags.
106     </p>
107    
108 swift 1.17 <pre caption="Installing distcc">
109     # <i>emerge distcc</i>
110     </pre>
111    
112 swift 1.10 </body>
113     </section>
114     <section>
115     <title>Setting up Portage to use Distcc</title>
116     <body>
117    
118     <p>
119 swift 1.17 Setting up distcc is easy to do with Portage. Execute the following steps on
120     each system that should participate in the distributed compiling:
121 swift 1.10 </p>
122    
123     <pre caption="Integrating Distcc and Portage">
124 swift 1.1 # <i>emerge distcc</i>
125     # <i>nano -w /etc/make.conf</i>
126 swift 1.17 <comment>(Set N to a suitable number for your particular setup)</comment>
127     <comment>(A common heuristic strategy is twice the number of CPUs + 1)</comment>
128     MAKEOPTS="-jN"
129     <comment>(Add distcc to your FEATURES variable)</comment>
130     FEATURES="distcc"
131 swift 1.5 </pre>
132 swift 1.1
133 swift 1.10 <p>
134 swift 1.17 Next you have to specify what hosts you want to use. Use the
135     <c>distcc-config</c> command to set the list of hosts. Here is an example of
136     some hosts that might be in your list:
137 swift 1.10 </p>
138    
139     <pre caption="Examples of host definitions">
140 swift 1.8 192.168.0.1 192.168.0.2 192.168.0.3
141     192.168.0.1/2 192.168.0.2 192.168.0.3/10
142     192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4
143     @192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3
144 swift 1.17 <comment>(There are also several other methods of setting up hosts. See the
145     distcc manpage for more details.)</comment>
146 swift 1.8 </pre>
147 swift 1.10
148     <p>
149     It may all look complicated, but in most cases a variant of line 1 or 2 will
150 swift 1.17 work.
151 swift 1.10 </p>
152    
153 swift 1.17 <ul>
154     <li>Line 1 is just a space-delimited list of hosts</li>
155     <li>
156     Line 2 is a list of hosts that specifies the maximum amount of jobs (by
157     defining the <c>/N</c> setting) to send that host at any given time
158     </li>
159     </ul>
160    
161 swift 1.10 <p>
162 swift 1.17 Since most people won't be using lines 3 or 4, I'll <uri
163     link="http://distcc.samba.org/man/distcc_1.html">point you to</uri> the distcc
164     docs for more information.
165     </p>
166    
167     <p>
168     For instance, to set the first line in the previous example:
169 swift 1.10 </p>
170    
171     <pre caption="Sample command to set the hosts">
172 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "192.168.0.1 192.168.0.2 192.168.0.3"</i>
173 swift 1.8 </pre>
174 swift 1.1
175 swift 1.10 <p>
176 swift 1.17 Now start the distcc daemon on all the participating computers:
177 swift 1.10 </p>
178    
179 swift 1.17 <pre caption="Starting the distcc daemon">
180     <comment>(Add distccd to the default runlevel)</comment>
181     # <i>rc-update add distccd default</i>
182     <comment>(Start the distcc daemon)</comment>
183     # <i>/etc/init.d/distccd start</i>
184 swift 1.8 </pre>
185 swift 1.10
186     </body>
187     </section>
188     <section>
189     <title>Setting up Distcc to Work With Automake</title>
190     <body>
191    
192     <p>
193     This is in some cases easier than the Portage setup. What you have to do is
194 swift 1.17 update your <c>PATH</c> variable to include <path>/usr/lib/distcc/bin</path>
195     in front of the directory that contains <c>gcc</c> (<path>/usr/bin</path>).
196 swift 1.10 However, there is a caveat. If you use ccache you have to put distcc after
197 swift 1.17 the ccache part:
198 swift 1.10 </p>
199    
200     <pre caption="Setting your path">
201 swift 1.1 # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
202 swift 1.8 </pre>
203 swift 1.10
204     <p>
205     Then, as you would normally type <c>make</c>, you would type <c>make -jN</c>
206 swift 1.17 (where N is an integer). The value of N depends on your network and the types
207     of computers you are using to compile. For me, I have a dual processor P3 and
208     a K6-2/400 that compiles everything for my desktop (a 1200MHz Athlon
209     Thunderbird) and I use -j5. Test your own settings to find the most
210     performant number.
211 swift 1.10 </p>
212    
213     </body>
214     </section>
215 blubber 1.2 </chapter>
216 swift 1.1
217 blubber 1.2 <chapter>
218 swift 1.17 <title>Cross-Compiling</title>
219     <section>
220     <title>A Note on Cross-Compiling</title>
221     <body>
222    
223     <p>
224     Cross-compiling is using one architecture to build programs for another
225     architecture. This can be as simple as using an Athlon (i686) to build a
226     program for a K6-2 (i586), or using a Sparc to build a program for a ppc.
227     </p>
228    
229     </body>
230     </section>
231 swift 1.10 <section>
232 swift 1.17 <title>A Personal Note on Cross-compiling</title>
233 swift 1.10 <body>
234    
235     <p>
236 swift 1.17 I'd love to be able to help out on the Gentoo Cross-compiling effort, but I
237     don't have any non-x86 machines. I can create things that work <e>in
238     theory</e>, but I have to rely on other people to test what I write.
239     That works to an extent, but it's tough. ;-)
240 swift 1.10 </p>
241    
242 swift 1.17 <p>
243     In the near future I hope to acquire a Sparc of some type to put Gentoo on so
244     I can play around in-house.
245     </p>
246    
247     </body>
248     </section>
249     </chapter>
250    
251     <chapter id="bootstrapping">
252     <title>Using Distcc to Bootstrap</title>
253     <section>
254     <title>Step 1: Configure Portage </title>
255     <body>
256 swift 1.10
257     <p>
258 swift 1.17 Boot your new box with a Gentoo Linux LiveCD and follow the <uri
259     link="/doc/en/handbook/handbook-x86.xml?part=1">installation instructions</uri>
260     up until the bootstrapping part. Then configure Portage to use distcc:
261 swift 1.10 </p>
262    
263 swift 1.17 <pre caption="Preliminary Setup">
264     # <i>nano -w /etc/make.conf</i>
265     <comment>(Add distcc to the FEATURES variable)</comment>
266     FEATURES="distcc"
267     <comment>(Set distcc's directory for temporary space)</comment>
268     DISTCC_DIR="${PORTAGE_TMPDIR}/.distcc"
269     <comment>(Modify MAKEOPTS to include -jN with N an appropriate integer)</comment>
270     MAKEOPTS="-jN"
271 swift 1.8 </pre>
272 swift 1.10
273     </body>
274     </section>
275     <section>
276     <title>Step 2: Getting Distcc</title>
277     <body>
278    
279 pylon 1.20 <p>Before the installation of distcc, a user called distcc must be added to the
280     <path>/etc/passwd</path>:</p>
281    
282     <pre caption="Create user distcc">
283     # <i>echo "distcc:x:240:2:distccd:/dev/null:/bin/false" &gt;&gt; /etc/passwd</i>
284     </pre>
285    
286 swift 1.10 <p>
287 swift 1.17 Install distcc:
288 swift 1.10 </p>
289    
290     <pre caption="Getting Distcc on the new box">
291 swift 1.11 # <i>USE='-*' emerge --nodeps distcc</i>
292 swift 1.8 </pre>
293 swift 1.10
294     </body>
295     </section>
296     <section>
297 swift 1.17 <title>Step 3: Setting Up Distcc</title>
298 swift 1.10 <body>
299    
300     <p>
301 pylon 1.20 Run <c>distcc-config</c> to setup distcc:
302 swift 1.10 </p>
303    
304     <pre caption="Final distcc setup">
305 swift 1.17 <comment>(Substitute host1, host2, ... with the IP number(s) of the participating hosts)</comment>
306 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i>
307 swift 1.17 </pre>
308 swift 1.10
309     <p>
310 swift 1.17 distcc is now set up to bootstrap! Continue the official installation
311     instructions and <e>don't forget</e> to re-emerge distcc after <c>emerge
312     system</c>. This is to make sure that all of the dependencies you want are
313     installed as well.
314 swift 1.10 </p>
315    
316 swift 1.12 <note>
317 swift 1.17 During bootstrap and <c>emerge system</c> Distcc may not appear to be used.
318     This is expected as some ebuilds do not work well with distcc, so they
319     intentionally disable it.
320 swift 1.12 </note>
321    
322 swift 1.10 </body>
323     </section>
324 blubber 1.2 </chapter>
325 swift 1.1
326 swift 1.5 <chapter>
327 swift 1.10 <title>Troubleshooting</title>
328     <section>
329     <title>Mozilla and Xfree</title>
330     <body>
331    
332     <p>
333 swift 1.17 As you emerge various packages, you'll notice that some of them aren't being
334     distributed (and aren't being built in parallel). This is because the
335     developers of the Mozilla and Xfree ebuilds intentionally disable parallel
336     building because it is known to cause problems.
337 swift 1.10 </p>
338    
339     <p>
340 swift 1.17 Sometimes distcc might cause a package to fail to compile. If this happens
341     for you, please <uri link="http://bugs.gentoo.org">report</uri> it to us.
342 swift 1.10 </p>
343    
344     </body>
345     </section>
346     <section>
347     <title>A Mixture of hardened-gcc and non-hardened-gcc Hosts Will Be Faulty</title>
348     <body>
349    
350     <p>
351     With such a long title any explanation here is almost irrelevent. However, if
352     you plan to use distcc across hosts that have the <uri
353 swift 1.17 link="/proj/en/hardened/etdyn-ssp.xml">PaX/hardened-gcc</uri> and some that do
354     not, you will run into problems.
355 swift 1.10 </p>
356    
357     <p>
358     The solution requires a little foresight on your part; you have to run
359     <c>hardened-gcc -R</c> on the host that has PaX/hardened-gcc, or you have to
360     enable PaX protections in your kernel and <c>emerge hardened-gcc</c>. Both of
361     which are a good thing to do since for the most part the protections offered
362     by both packages is a good thing and is transparent to the user.
363     </p>
364    
365     </body>
366     </section>
367     <section>
368     <title>Mixed GCC Versions</title>
369     <body>
370    
371     <p>
372 swift 1.17 If you have different GCC versions on your hosts, there will likely be very
373 swift 1.10 weird problems. The solution is to make certain all hosts have the same GCC
374     version.
375     </p>
376    
377     </body>
378     </section>
379 swift 1.5 </chapter>
380 swift 1.1
381     <chapter>
382 swift 1.10 <title>Distcc Extras</title>
383     <section>
384     <title>Distcc Monitors</title>
385     <body>
386    
387     <p>
388 swift 1.17 Distcc ships with two monitors. The text-based one is always built and is
389 swift 1.10 called <c>distccmon-text</c>. Running it for the first time can be a bit
390     confusing, but it is really quite easy to use. If you run the program with no
391     parameter it will run once. However, if you pass it a number it will update
392     every N seconds, where N is the argument you passed.
393     </p>
394    
395     <p>
396     The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c>
397 swift 1.17 in your <c>USE</c> flags. This one is GTK+ based, runs in an X environment
398 swift 1.10 and it is quite lovely.
399     </p>
400    
401     <pre caption="Starting the monitors">
402 swift 1.17 # <i>distccmon-text N</i>
403     <comment>(Or ...)</comment>
404     # <i>distccmon-gnome</i>
405 swift 1.8 </pre>
406 swift 1.10
407     </body>
408     </section>
409 swift 1.1 </chapter>
410    
411     <chapter>
412 swift 1.10 <title>Future Plans for Distcc and Gentoo</title>
413     <section>
414     <title>distcc-subnetscan</title>
415     <body>
416    
417     <p>
418     <c>distcc-subnetscan</c> is a perl program in development that will scan a
419     subnet for hosts that have a participating distcc daemon. Even this will be
420     enhanced to test if the daemon on a remote host will conform to a specific
421     <c>CHOST</c> setting to make cross-compiling easier.
422     </p>
423    
424     <p>
425 swift 1.17 The perl script is being kept at <uri link="http://dev.gentoo.org/~lisa/distcc/distcc-subnetscan/distcc-subnetscan.pl">lisa's
426     dev page</uri> until a more formal home can be found.
427 swift 1.10 </p>
428    
429     </body>
430     </section>
431 swift 1.1 </chapter>
432     </guide>

  ViewVC Help
Powered by ViewVC 1.1.20