/[gentoo]/xml/htdocs/doc/en/distcc.xml
Gentoo

Contents of /xml/htdocs/doc/en/distcc.xml

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.19 - (hide annotations) (download) (as text)
Fri Jul 2 15:47:15 2004 UTC (10 years, 5 months ago) by swift
Branch: MAIN
Changes since 1.18: +3 -5 lines
File MIME type: application/xml
#53015 - Remove DISTCC_DIR entry from make.conf

1 swift 1.1 <?xml version='1.0' encoding="UTF-8"?>
2 blubber 1.3 <!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3 swift 1.19 <!-- $Header: /home/cvsroot/gentoo/xml/htdocs/doc/en/distcc.xml,v 1.18 2004/05/12 14:44:19 swift Exp $ -->
4 swift 1.17
5     <!--
6 swift 1.18 Synchronised with rev 1.15 on
7 swift 1.17 http://www.thedoh.com/viewcvs/distcc/html/distcc.xml
8     -->
9 blubber 1.2
10 swift 1.1 <guide link="/doc/en/distcc.xml">
11    
12     <title>Gentoo Distcc Documentation</title>
13    
14     <author title="Author">
15 swift 1.10 <mail link="lisa@gentoo.org">Lisa Seelye</mail>
16 erwin 1.4 </author>
17 swift 1.17
18 erwin 1.4 <author title="Editor">
19 swift 1.10 <mail link="vapier@gentoo.org">Mike Frysinger</mail>
20 erwin 1.4 </author>
21 swift 1.17
22 erwin 1.4 <author title="Editor">
23 swift 1.10 <mail link="erwin@gentoo.org">Erwin</mail>
24 blubber 1.2 </author>
25 swift 1.17
26     <author title="Editor">
27 swift 1.10 <mail link="swift@gentoo.org">Sven Vermeulen</mail>
28 swift 1.1 </author>
29 swift 1.17
30     <author title="Reviewer">
31 swift 1.10 <mail link="blubber@gentoo.org">Tiemo Kieft</mail>
32 swift 1.1 </author>
33    
34 swift 1.15 <abstract>
35     This document serves as a HOWTO for using distcc with Gentoo.
36     </abstract>
37    
38 swift 1.17 <!-- The content of this document is licensed under the CC-BY-SA license -->
39     <!-- See http://creativecommons.org/licenses/by-sa/1.0 -->
40 swift 1.6 <license/>
41    
42 swift 1.19 <version>1.7</version>
43     <date>July 02, 2004</date>
44 swift 1.1
45     <chapter>
46 swift 1.10 <title>Introduction</title>
47     <section>
48     <title>What is distcc?</title>
49     <body>
50    
51     <p>
52     Distcc is a program designed to distribute compiling tasks across a network to
53     participating hosts. It is comprised of a server, <c>distccd</c> and a client
54     program, <c>distcc</c>. Distcc can work transparently with <uri
55 swift 1.17 link="http://ccache.samba.org">ccache</uri> and Portage with a little work.
56     </p>
57    
58     </body>
59     </section>
60     <section>
61     <title>Using distcc to bootstrap</title>
62     <body>
63    
64     <p>
65     If you are planning on using distcc to help you bootstrap a Gentoo installation,
66     make sure you read the part on <uri link="#bootstrapping">Using distcc to
67     Bootstrap</uri> which is situated further down in this document.
68 swift 1.10 </p>
69    
70     </body>
71     </section>
72 swift 1.8 </chapter>
73    
74     <chapter>
75 swift 1.10 <title>Setup</title>
76     <section>
77     <title>Dependencies</title>
78     <body>
79    
80 swift 1.17 <p>
81     To start off with, to use Distcc all of the computers on your network need to
82     have the same GCC versions. Mixing 3.3.x (where the x varies) is okay, but
83     mixing 3.3.x and 3.2.x is not: your programs <b>will</b> have errors in it.
84     </p>
85 swift 1.10
86     </body>
87     </section>
88     <section>
89     <title>Installing Distcc</title>
90     <body>
91    
92     <p>
93 swift 1.17 There are a couple of options you should know about before you start
94     installing distcc.
95 swift 1.10 </p>
96    
97     <p>
98     Distcc ships with a graphical monitor to monitor tasks that your computer is
99     sending away for compilation. If you use Gnome then put 'gnome' in your USE
100     flags. However, if you don't use Gnome and would still like to have the
101     monitor then you should put 'gtk' in your USE flags.
102     </p>
103    
104 swift 1.17 <pre caption="Installing distcc">
105     # <i>emerge distcc</i>
106     </pre>
107    
108 swift 1.10 </body>
109     </section>
110     <section>
111     <title>Setting up Portage to use Distcc</title>
112     <body>
113    
114     <p>
115 swift 1.17 Setting up distcc is easy to do with Portage. Execute the following steps on
116     each system that should participate in the distributed compiling:
117 swift 1.10 </p>
118    
119     <pre caption="Integrating Distcc and Portage">
120 swift 1.1 # <i>emerge distcc</i>
121     # <i>nano -w /etc/make.conf</i>
122 swift 1.17 <comment>(Set N to a suitable number for your particular setup)</comment>
123     <comment>(A common heuristic strategy is twice the number of CPUs + 1)</comment>
124     MAKEOPTS="-jN"
125     <comment>(Add distcc to your FEATURES variable)</comment>
126     FEATURES="distcc"
127 swift 1.5 </pre>
128 swift 1.1
129 swift 1.10 <p>
130 swift 1.17 Next you have to specify what hosts you want to use. Use the
131     <c>distcc-config</c> command to set the list of hosts. Here is an example of
132     some hosts that might be in your list:
133 swift 1.10 </p>
134    
135     <pre caption="Examples of host definitions">
136 swift 1.8 192.168.0.1 192.168.0.2 192.168.0.3
137     192.168.0.1/2 192.168.0.2 192.168.0.3/10
138     192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4
139     @192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3
140 swift 1.17 <comment>(There are also several other methods of setting up hosts. See the
141     distcc manpage for more details.)</comment>
142 swift 1.8 </pre>
143 swift 1.10
144     <p>
145     It may all look complicated, but in most cases a variant of line 1 or 2 will
146 swift 1.17 work.
147 swift 1.10 </p>
148    
149 swift 1.17 <ul>
150     <li>Line 1 is just a space-delimited list of hosts</li>
151     <li>
152     Line 2 is a list of hosts that specifies the maximum amount of jobs (by
153     defining the <c>/N</c> setting) to send that host at any given time
154     </li>
155     </ul>
156    
157 swift 1.10 <p>
158 swift 1.17 Since most people won't be using lines 3 or 4, I'll <uri
159     link="http://distcc.samba.org/man/distcc_1.html">point you to</uri> the distcc
160     docs for more information.
161     </p>
162    
163     <p>
164     For instance, to set the first line in the previous example:
165 swift 1.10 </p>
166    
167     <pre caption="Sample command to set the hosts">
168 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "192.168.0.1 192.168.0.2 192.168.0.3"</i>
169 swift 1.8 </pre>
170 swift 1.1
171 swift 1.10 <p>
172 swift 1.17 Now start the distcc daemon on all the participating computers:
173 swift 1.10 </p>
174    
175 swift 1.17 <pre caption="Starting the distcc daemon">
176     <comment>(Add distccd to the default runlevel)</comment>
177     # <i>rc-update add distccd default</i>
178     <comment>(Start the distcc daemon)</comment>
179     # <i>/etc/init.d/distccd start</i>
180 swift 1.8 </pre>
181 swift 1.10
182     </body>
183     </section>
184     <section>
185     <title>Setting up Distcc to Work With Automake</title>
186     <body>
187    
188     <p>
189     This is in some cases easier than the Portage setup. What you have to do is
190 swift 1.17 update your <c>PATH</c> variable to include <path>/usr/lib/distcc/bin</path>
191     in front of the directory that contains <c>gcc</c> (<path>/usr/bin</path>).
192 swift 1.10 However, there is a caveat. If you use ccache you have to put distcc after
193 swift 1.17 the ccache part:
194 swift 1.10 </p>
195    
196     <pre caption="Setting your path">
197 swift 1.1 # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
198 swift 1.8 </pre>
199 swift 1.10
200     <p>
201     Then, as you would normally type <c>make</c>, you would type <c>make -jN</c>
202 swift 1.17 (where N is an integer). The value of N depends on your network and the types
203     of computers you are using to compile. For me, I have a dual processor P3 and
204     a K6-2/400 that compiles everything for my desktop (a 1200MHz Athlon
205     Thunderbird) and I use -j5. Test your own settings to find the most
206     performant number.
207 swift 1.10 </p>
208    
209     </body>
210     </section>
211 blubber 1.2 </chapter>
212 swift 1.1
213 blubber 1.2 <chapter>
214 swift 1.17 <title>Cross-Compiling</title>
215     <section>
216     <title>A Note on Cross-Compiling</title>
217     <body>
218    
219     <p>
220     Cross-compiling is using one architecture to build programs for another
221     architecture. This can be as simple as using an Athlon (i686) to build a
222     program for a K6-2 (i586), or using a Sparc to build a program for a ppc.
223     </p>
224    
225     </body>
226     </section>
227 swift 1.10 <section>
228 swift 1.17 <title>A Personal Note on Cross-compiling</title>
229 swift 1.10 <body>
230    
231     <p>
232 swift 1.17 I'd love to be able to help out on the Gentoo Cross-compiling effort, but I
233     don't have any non-x86 machines. I can create things that work <e>in
234     theory</e>, but I have to rely on other people to test what I write.
235     That works to an extent, but it's tough. ;-)
236 swift 1.10 </p>
237    
238 swift 1.17 <p>
239     In the near future I hope to acquire a Sparc of some type to put Gentoo on so
240     I can play around in-house.
241     </p>
242    
243     </body>
244     </section>
245     </chapter>
246    
247     <chapter id="bootstrapping">
248     <title>Using Distcc to Bootstrap</title>
249     <section>
250     <title>Step 1: Configure Portage </title>
251     <body>
252 swift 1.10
253     <p>
254 swift 1.17 Boot your new box with a Gentoo Linux LiveCD and follow the <uri
255     link="/doc/en/handbook/handbook-x86.xml?part=1">installation instructions</uri>
256     up until the bootstrapping part. Then configure Portage to use distcc:
257 swift 1.10 </p>
258    
259 swift 1.17 <pre caption="Preliminary Setup">
260     # <i>nano -w /etc/make.conf</i>
261     <comment>(Add distcc to the FEATURES variable)</comment>
262     FEATURES="distcc"
263     <comment>(Set distcc's directory for temporary space)</comment>
264     DISTCC_DIR="${PORTAGE_TMPDIR}/.distcc"
265     <comment>(Modify MAKEOPTS to include -jN with N an appropriate integer)</comment>
266     MAKEOPTS="-jN"
267 swift 1.8 </pre>
268 swift 1.10
269     </body>
270     </section>
271     <section>
272     <title>Step 2: Getting Distcc</title>
273     <body>
274    
275     <p>
276 swift 1.17 Install distcc:
277 swift 1.10 </p>
278    
279     <pre caption="Getting Distcc on the new box">
280 swift 1.11 # <i>USE='-*' emerge --nodeps distcc</i>
281 swift 1.8 </pre>
282 swift 1.10
283     </body>
284     </section>
285     <section>
286 swift 1.17 <title>Step 3: Setting Up Distcc</title>
287 swift 1.10 <body>
288    
289     <p>
290 swift 1.18 Add a "distcc" user to the <path>/etc/passwd</path> file and run
291     <c>distcc-config</c> to setup distcc:
292 swift 1.10 </p>
293    
294     <pre caption="Final distcc setup">
295 swift 1.18 # <i>echo "distcc:x:240:2:distccd:/dev/null:/bin/false" &gt;&gt; /etc/passwd</i>
296 swift 1.17 <comment>(Substitute host1, host2, ... with the IP number(s) of the participating hosts)</comment>
297 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i>
298 swift 1.17 </pre>
299 swift 1.10
300     <p>
301 swift 1.17 distcc is now set up to bootstrap! Continue the official installation
302     instructions and <e>don't forget</e> to re-emerge distcc after <c>emerge
303     system</c>. This is to make sure that all of the dependencies you want are
304     installed as well.
305 swift 1.10 </p>
306    
307 swift 1.12 <note>
308 swift 1.17 During bootstrap and <c>emerge system</c> Distcc may not appear to be used.
309     This is expected as some ebuilds do not work well with distcc, so they
310     intentionally disable it.
311 swift 1.12 </note>
312    
313 swift 1.10 </body>
314     </section>
315 blubber 1.2 </chapter>
316 swift 1.1
317 swift 1.5 <chapter>
318 swift 1.10 <title>Troubleshooting</title>
319     <section>
320     <title>Mozilla and Xfree</title>
321     <body>
322    
323     <p>
324 swift 1.17 As you emerge various packages, you'll notice that some of them aren't being
325     distributed (and aren't being built in parallel). This is because the
326     developers of the Mozilla and Xfree ebuilds intentionally disable parallel
327     building because it is known to cause problems.
328 swift 1.10 </p>
329    
330     <p>
331 swift 1.17 Sometimes distcc might cause a package to fail to compile. If this happens
332     for you, please <uri link="http://bugs.gentoo.org">report</uri> it to us.
333 swift 1.10 </p>
334    
335     </body>
336     </section>
337     <section>
338     <title>A Mixture of hardened-gcc and non-hardened-gcc Hosts Will Be Faulty</title>
339     <body>
340    
341     <p>
342     With such a long title any explanation here is almost irrelevent. However, if
343     you plan to use distcc across hosts that have the <uri
344 swift 1.17 link="/proj/en/hardened/etdyn-ssp.xml">PaX/hardened-gcc</uri> and some that do
345     not, you will run into problems.
346 swift 1.10 </p>
347    
348     <p>
349     The solution requires a little foresight on your part; you have to run
350     <c>hardened-gcc -R</c> on the host that has PaX/hardened-gcc, or you have to
351     enable PaX protections in your kernel and <c>emerge hardened-gcc</c>. Both of
352     which are a good thing to do since for the most part the protections offered
353     by both packages is a good thing and is transparent to the user.
354     </p>
355    
356     </body>
357     </section>
358     <section>
359     <title>Mixed GCC Versions</title>
360     <body>
361    
362     <p>
363 swift 1.17 If you have different GCC versions on your hosts, there will likely be very
364 swift 1.10 weird problems. The solution is to make certain all hosts have the same GCC
365     version.
366     </p>
367    
368     </body>
369     </section>
370 swift 1.5 </chapter>
371 swift 1.1
372     <chapter>
373 swift 1.10 <title>Distcc Extras</title>
374     <section>
375     <title>Distcc Monitors</title>
376     <body>
377    
378     <p>
379 swift 1.17 Distcc ships with two monitors. The text-based one is always built and is
380 swift 1.10 called <c>distccmon-text</c>. Running it for the first time can be a bit
381     confusing, but it is really quite easy to use. If you run the program with no
382     parameter it will run once. However, if you pass it a number it will update
383     every N seconds, where N is the argument you passed.
384     </p>
385    
386     <p>
387     The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c>
388 swift 1.17 in your <c>USE</c> flags. This one is GTK+ based, runs in an X environment
389 swift 1.10 and it is quite lovely.
390     </p>
391    
392     <pre caption="Starting the monitors">
393 swift 1.17 # <i>distccmon-text N</i>
394     <comment>(Or ...)</comment>
395     # <i>distccmon-gnome</i>
396 swift 1.8 </pre>
397 swift 1.10
398     </body>
399     </section>
400 swift 1.1 </chapter>
401    
402     <chapter>
403 swift 1.10 <title>Future Plans for Distcc and Gentoo</title>
404     <section>
405     <title>distcc-subnetscan</title>
406     <body>
407    
408     <p>
409     <c>distcc-subnetscan</c> is a perl program in development that will scan a
410     subnet for hosts that have a participating distcc daemon. Even this will be
411     enhanced to test if the daemon on a remote host will conform to a specific
412     <c>CHOST</c> setting to make cross-compiling easier.
413     </p>
414    
415     <p>
416 swift 1.17 The perl script is being kept at <uri link="http://dev.gentoo.org/~lisa/distcc/distcc-subnetscan/distcc-subnetscan.pl">lisa's
417     dev page</uri> until a more formal home can be found.
418 swift 1.10 </p>
419    
420     </body>
421     </section>
422 swift 1.1 </chapter>
423     </guide>

  ViewVC Help
Powered by ViewVC 1.1.20