/[gentoo]/xml/htdocs/doc/en/distcc.xml
Gentoo

Contents of /xml/htdocs/doc/en/distcc.xml

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.10 - (hide annotations) (download) (as text)
Wed Nov 19 05:40:27 2003 UTC (11 years, 1 month ago) by swift
Branch: MAIN
Changes since 1.9: +347 -147 lines
File MIME type: application/xml
Coding style

1 swift 1.1 <?xml version='1.0' encoding="UTF-8"?>
2 blubber 1.3 <!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3 swift 1.10 <!-- $Header: /home/cvsroot/gentoo/xml/htdocs/doc/en/distcc.xml,v 1.9 2003/10/12 17:38:25 swift Exp $ -->
4 blubber 1.2
5 swift 1.1 <guide link="/doc/en/distcc.xml">
6    
7     <title>Gentoo Distcc Documentation</title>
8    
9     <author title="Author">
10 swift 1.10 <mail link="lisa@gentoo.org">Lisa Seelye</mail>
11 erwin 1.4 </author>
12     <author title="Editor">
13 swift 1.10 <mail link="vapier@gentoo.org">Mike Frysinger</mail>
14 erwin 1.4 </author>
15     <author title="Editor">
16 swift 1.10 <mail link="erwin@gentoo.org">Erwin</mail>
17 blubber 1.2 </author>
18     <author title="Reviewer">
19 swift 1.10 <mail link="swift@gentoo.org">Sven Vermeulen</mail>
20 swift 1.1 </author>
21     <author title="Reviewer">
22 swift 1.10 <mail link="blubber@gentoo.org">Tiemo Kieft</mail>
23 swift 1.1 </author>
24    
25 swift 1.6 <license/>
26    
27 swift 1.9 <version>1.2.4</version>
28     <date>October 12, 2003</date>
29 swift 1.1
30 swift 1.10 <abstract>
31     This document serves as a HOWTO for using distcc with Gentoo.
32     </abstract>
33 swift 1.1
34     <chapter>
35 swift 1.10 <title>Introduction</title>
36     <section>
37     <title>What is distcc?</title>
38     <body>
39    
40     <p>
41     Distcc is a program designed to distribute compiling tasks across a network to
42     participating hosts. It is comprised of a server, <c>distccd</c> and a client
43     program, <c>distcc</c>. Distcc can work transparently with <uri
44     link="http://ccache.samba.org">ccache</uri> and Portage with a little set up.
45     </p>
46    
47     </body>
48     </section>
49 swift 1.8 </chapter>
50    
51     <chapter>
52 swift 1.10 <title>Setup</title>
53     <section>
54     <title>Dependencies</title>
55     <body>
56    
57     <pre caption="Distcc dependencies (2.8 through 2.10)">
58 swift 1.1 &gt;=sys-apps/portage-2.0.46-r11
59 swift 1.8 <codenote>Distcc-2.11 and on require &gt;=sys-apps/portage-2.0.49-r6</codenote>
60 swift 1.1 &gt;=sys-devel/gcc-config-1.3.1
61 erwin 1.4 sys-apps/shadow
62 swift 1.8 <codenote>(As of version 2.8 and up until 2.11) And the following optional dependencies when you have <i>gtk</i> in your <i>USE</i> flags</codenote>
63 swift 1.1 &gt;=x11-libs/gtk+-2.2.1
64 swift 1.8 </pre>
65 swift 1.10
66     <pre caption="Distcc Dependencies (post 2.11.1)">
67 swift 1.8 &gt;=sys-apps/portage-2.0.49-r6
68     &gt;=sys-devel/gcc-config-1.3.1
69     sys-apps/shadow
70     <codenote>In this revision you may chose between a Gnome and GTK GUI monitor, they have the following added dependencies</codenote>
71     <codenote>For GTK:</codenote>
72     &gt;=x11-libs/gtk+-2.0.0
73     &gt;=gnome-base/libglade-2.0.0
74     x11-libs/pango
75     <codenote>For Gnome:</codenote>
76     &gt;=x11-libs/gtk+-2.0.0
77     &gt;=gnome-base/libglade-2.0.0
78     x11-libs/pango
79     &gt;=gnome-base/libgnomeui-2.0.0.0
80     &gt;=gnome-base/libgnome-2.0.0
81     </pre>
82 swift 1.10
83     </body>
84     </section>
85     <section>
86     <title>Installing Distcc</title>
87     <body>
88    
89     <p>
90     Installing Distcc is very easy. Simply set your USE flags and <c>emerge
91     distcc</c>. But, there's a couple of options you should know about.
92     </p>
93    
94     <p>
95     Distcc ships with a graphical monitor to monitor tasks that your computer is
96     sending away for compilation. If you use Gnome then put 'gnome' in your USE
97     flags. However, if you don't use Gnome and would still like to have the
98     monitor then you should put 'gtk' in your USE flags.
99     </p>
100    
101     </body>
102     </section>
103     <section>
104     <title>Setting up Portage to use Distcc</title>
105     <body>
106    
107     <p>
108     Setting up distcc is very easy to do with Portage. Follow these simple steps on
109     each computer you want to use distcc on:
110     </p>
111    
112     <pre caption="Integrating Distcc and Portage">
113 swift 1.1 # <i>emerge distcc</i>
114     # <i>nano -w /etc/make.conf</i>
115 swift 1.5 </pre>
116 swift 1.1
117 swift 1.10 <impo>
118     If you use distcc-2.11.1 or greater with &gt;=portage-2.0.49-r6, then all you
119     need to do as far as setup is to set your hosts (see next code block), modify
120     MAKEOPTS in <path>/etc/make.conf</path>, and add 'distcc' to FEATURES in
121     <path>/etc/make.conf</path>.
122     </impo>
123    
124     <note>
125     Edit your FEATURES flags to include &quot;distcc&quot;
126     </note>
127    
128     <note>
129     You should also uncomment the PORTAGE_TMPDIR line in
130     <path>/etc/make.conf</path>.
131     </note>
132    
133     <note>
134     Set <c>DISTCC_DIR=${PORTAGE_TMPDIR}/portage/.distcc</c> near the end of the file
135     (last line is okay).
136     </note>
137    
138     <p>
139     Next you have to specify what hosts you want to use. To do this you can use
140     the <c>distcc-config</c> command to set the list of hosts. Here is an example
141     of some hosts that might be in your list:
142     </p>
143    
144     <pre caption="Examples of host definitions">
145 swift 1.8 192.168.0.1 192.168.0.2 192.168.0.3
146     192.168.0.1/2 192.168.0.2 192.168.0.3/10
147     192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4
148     @192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3
149     </pre>
150 swift 1.10
151     <p>
152     It may all look complicated, but in most cases a variant of line 1 or 2 will
153     work. An explanation of each line is: Line 1 is just a space-delimited list
154     of hosts that will use default everything. Line 2 is a list of hosts that
155     specifies the maximum number of jobs (by use of the /N) to send that host at
156     any given time (specified with the <c>/n</c>). Since most people won't be
157     using lines 3 or 4, I'll <uri link="http://distcc.samba.org/man/distcc_1.html">
158     point you to</uri> the distcc docs for more information.
159     </p>
160    
161     <p>
162     A sample command to set the hosts (for line 1) is:
163     </p>
164    
165     <pre caption="Sample command to set the hosts">
166 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "192.168.0.1 192.168.0.2 192.168.0.3"</i>
167 swift 1.8 </pre>
168 swift 1.1
169 swift 1.10 <p>
170     The final step to integrating distcc into Portage is to re-open your
171     <path>/etc/make.conf</path> and edit <c>MAKEOPTS</c> to include <c>-jN</c>
172     (where N is an integer). Typically you will want to set this to the total
173     number of processors in your network plus one.
174     </p>
175    
176     <pre caption="Final steps in make.conf">
177 swift 1.1 # <i>nano -w /etc/make.conf</i>
178 swift 1.5 MAKEOPTS=-jN
179 swift 1.8 </pre>
180 swift 1.10
181     <impo>
182     Don't forget to start the distcc daemon! <c>/etc/init.d/distccd start</c>
183     </impo>
184    
185     </body>
186     </section>
187     <section>
188     <title>Setting up Distcc to Work With Automake</title>
189     <body>
190    
191     <p>
192     This is in some cases easier than the Portage setup. What you have to do is
193     update your <c>PATH</c> environmental variable to include
194     <path>/usr/lib/distcc/bin</path> before where gcc (<path>/usr/bin</path>).
195     However, there is a caveat. If you use ccache you have to put distcc after
196     the ccache part. Like so:
197     </p>
198    
199     <pre caption="Setting your path">
200 swift 1.1 # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
201 swift 1.8 </pre>
202 swift 1.10
203     <p>
204     Then, as you would normally type <c>make</c>, you would type <c>make -jN</c>
205     (where N is an integer). A safe number to use for N varies on your network
206     and the types of computers you are using to compile. For me, I have a dual
207     processor P3 and a K6-2/400 that compile everything for my desktop (a 1200MHz
208     Athlon Thunderbird) and I use -j5. Test your own settings to find the right
209     number.
210     </p>
211    
212     </body>
213     </section>
214 blubber 1.2 </chapter>
215 swift 1.1
216 blubber 1.2 <chapter>
217 swift 1.10 <title>Cross-Compiling</title>
218     <section>
219     <title>A Note on Cross-Compiling</title>
220     <body>
221    
222     <p>
223     Cross-compiling is using one architecture to build programs for another
224     architecture. This can be as simple as using an Athlon (i686) to build a
225     program for a K6-2 (i586), or using a Sparc to build a program for a ppc.
226     </p>
227    
228     </body>
229     </section>
230     <section>
231     <title>A Personal Note on Cross-compiling</title>
232     <body>
233    
234     <p>
235     I'd love to be able to help out on the Gentoo Cross-compiling effort, but I
236     don't have any non-x86 machines. I can create things that work <e>in
237     theory</e>, but I have to rely on other people to test what I write. That
238     works to an extent, but it's tough. ;-)
239     </p>
240    
241     <p>
242     In the near future I hope to acquire a Sparc of some type to put Gentoo on so
243     I can play around in-house.
244     </p>
245    
246     </body>
247     </section>
248 swift 1.1 </chapter>
249    
250     <chapter>
251 swift 1.10 <title>Using Distcc to Bootstrap</title>
252     <section>
253     <title>Step 1: Setup on New Box</title>
254     <body>
255    
256     <p>
257     Next, you have to boot your new box with a Gentoo Linux LiveCD and follow all
258     of the steps up until the bootstrapping. Then proceed with a little preliminary
259     setup on the new box.
260     </p>
261    
262     <pre caption="Preliminary Setup">
263 blubber 1.2 <codenote>Here we add distcc to FEATURES</codenote>
264 swift 1.8 # <i>nano -w /etc/make.conf</i>
265     <codenote>Add &quot;distcc&quot; to FEATURES</codenote>
266     <codenote>Modify MAKEOPTS in <path>/etc/make.conf to include -jN (as described above)</path></codenote>
267     </pre>
268 swift 1.10
269     <p>
270     Then add the distcc user to your <path>/etc/passwd</path>:
271     </p>
272    
273     <pre caption="Add the distcc user to /etc/passwd">
274 swift 1.5 # <i>echo "distcc:x:240:2:distccd:/dev/null:/bin/false" &gt;&gt;/etc/passwd</i>
275 erwin 1.4 <codenote>Do not forget the `<i>&gt;&gt;</i>'</codenote>
276 swift 1.8 </pre>
277 swift 1.10
278     </body>
279     </section>
280     <section>
281     <title>Step 2: Getting Distcc</title>
282     <body>
283    
284     <p>
285     Next you will want to issue the following command inside the chroot on the new
286     box.
287     </p>
288    
289     <pre caption="Getting Distcc on the new box">
290 swift 1.8 # <i>emerge --nodeps distcc</i>
291     </pre>
292 swift 1.10
293     </body>
294     </section>
295     <section>
296     <title>Step 3: Setting Up Distcc Itself</title>
297     <body>
298    
299     <p>
300     Now set up distcc itself...
301     </p>
302    
303     <pre caption="Final distcc setup">
304 erwin 1.4 # <i>/usr/bin/distcc-config --install</i>
305     # <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i>
306 swift 1.8 </pre>
307 swift 1.10
308     <p>
309     distcc should be set up now to bootstrap! Continue the official install guide,
310     and don't forget to re-emerge distcc after <c>emerge system</c>.
311     </p>
312    
313     </body>
314     </section>
315 blubber 1.2 </chapter>
316 swift 1.1
317 swift 1.5 <chapter>
318 swift 1.10 <title>Troubleshooting</title>
319     <section>
320     <title>Mozilla and Xfree</title>
321     <body>
322    
323     <p>
324     As you emerge various packages, you'll notice that they aren't being distributed
325     (and indeed aren't being built in parallel). This is because the developers of
326     Mozilla and Xfree .ebuilds disabled parallel building because it is known to
327     cause problems. This isn't necessarily a distcc problem.
328     </p>
329    
330     <p>
331     That isn't to say that sometimes distcc will cause a package to fail to compile.
332     </p>
333    
334     </body>
335     </section>
336     <section>
337     <title>A Mixture of hardened-gcc and non-hardened-gcc Hosts Will Be Faulty</title>
338     <body>
339    
340     <p>
341     With such a long title any explanation here is almost irrelevent. However, if
342     you plan to use distcc across hosts that have the <uri
343     link="http://www.gentoo.org/proj/en/hardened/etdyn-ssp.xml">PaX/hardened-gcc</uri>
344     and some that do not, you will run into problems.
345     </p>
346    
347     <p>
348     The solution requires a little foresight on your part; you have to run
349     <c>hardened-gcc -R</c> on the host that has PaX/hardened-gcc, or you have to
350     enable PaX protections in your kernel and <c>emerge hardened-gcc</c>. Both of
351     which are a good thing to do since for the most part the protections offered
352     by both packages is a good thing and is transparent to the user.
353     </p>
354    
355     </body>
356     </section>
357     <section>
358     <title>Mixed GCC Versions</title>
359     <body>
360    
361     <p>
362     If you have differing versions of GCC on your hosts, there will likely be very
363     weird problems. The solution is to make certain all hosts have the same GCC
364     version.
365     </p>
366    
367     </body>
368     </section>
369 swift 1.5 </chapter>
370 swift 1.1
371     <chapter>
372 swift 1.10 <title>Distcc Extras</title>
373     <section>
374     <title>Distcc Monitors</title>
375     <body>
376    
377     <p>
378     Distcc ships with two monitors. The text-based one is always built, it is
379     called <c>distccmon-text</c>. Running it for the first time can be a bit
380     confusing, but it is really quite easy to use. If you run the program with no
381     parameter it will run once. However, if you pass it a number it will update
382     every N seconds, where N is the argument you passed.
383     </p>
384    
385     <p>
386     The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c>
387     in your <c>USE</c> flags. This one is GTK+ based and runs in an X environment,
388     and it is quite lovely.
389     </p>
390    
391     <p>
392     There is a caveat to using these programs. If you want to monitor any emerge,
393     you have to start the monitor like so:
394     </p>
395    
396     <pre caption="Starting the monitors">
397 swift 1.8 # <i>DISTCC_DIR=/path/to/distccdir distccmon-text N</i>
398 swift 1.1 <codenote>Or you can start the graphical monitor...</codenote>
399 swift 1.8 # <i>DISTCC_DIR=/path/to/distccdir distccmon-gnome</i>
400     </pre>
401 swift 1.10
402     <impo>
403     If you are using Portage and used the above example, this will be
404     <c>DISTCC_DIR=/var/tmp/portage/.distcc</c>.
405     </impo>
406    
407     <note>
408     If you aren't using Portage you do not need to specify DISTCC_DIR (i.e., just
409     type distccmon-text N).
410     </note>
411    
412     </body>
413     </section>
414 swift 1.1 </chapter>
415    
416     <chapter>
417 swift 1.10 <title>Future Plans for Distcc and Gentoo</title>
418     <section>
419     <title>distcc-subnetscan</title>
420     <body>
421    
422     <p>
423     <c>distcc-subnetscan</c> is a perl program in development that will scan a
424     subnet for hosts that have a participating distcc daemon. Even this will be
425     enhanced to test if the daemon on a remote host will conform to a specific
426     <c>CHOST</c> setting to make cross-compiling easier.
427     </p>
428    
429     <p>
430     The perl script is being kept at <uri link="http://dev.gentoo.org/~lisa/distcc/distcc-subnetscan/distcc-subnetscan.pl">
431     right here</uri> until a more formal home can be found.
432     </p>
433    
434     </body>
435     </section>
436     <section>
437     <title>distcc-config</title>
438     <body>
439    
440     <p>
441     <c>distcc-config</c>, the userland configuration tool for distcc is out of date.
442     Soon it will be rewritten to be brought up the current version of distcc.
443     </p>
444    
445     </body>
446     </section>
447 swift 1.1 </chapter>
448     </guide>

  ViewVC Help
Powered by ViewVC 1.1.20