/[gentoo]/xml/htdocs/doc/en/distcc.xml
Gentoo

Contents of /xml/htdocs/doc/en/distcc.xml

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.17 - (hide annotations) (download) (as text)
Sun May 9 10:16:34 2004 UTC (10 years, 7 months ago) by swift
Branch: MAIN
Changes since 1.16: +155 -178 lines
File MIME type: application/xml
Various updates, including removal of obsoleted paragraphs, removal of duplicate
information, including reference to bootstrapping section, removing the many
colorful <impo>, <warn> and <note>'s and fixing language. Also added the
<license/> tag:
11:57 <@SwifT> lisa: ping (final one :)
11:57 <@lisa> SwifT: hi
11:58 <@SwifT> lisa: may I put the document available under the cc-by-sa
               license?
11:58 <@lisa> SwifT: cc-by-sa ?
11:58 <@SwifT> lisa: creative commons attribution - share alike
11:58 <@SwifT> lisa: it's a free documentation license
11:58 <@lisa> sure, go ahead... dont forget to send me the diff
              ok ?
11:58 <@SwifT> sure
11:59 <@SwifT> tx

As a sidenote I also added a comment referring to lisa's viewcvs where she keeps
the master document.

1 swift 1.1 <?xml version='1.0' encoding="UTF-8"?>
2 blubber 1.3 <!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3 swift 1.17 <!-- $Header: /home/cvsroot/distcc/html/distcc.xml,v 1.13 2004/01/09 19:56:37 lisa Exp $ -->
4    
5     <!--
6     Synchronised with rev 1.14 on
7     http://www.thedoh.com/viewcvs/distcc/html/distcc.xml
8     -->
9 blubber 1.2
10 swift 1.1 <guide link="/doc/en/distcc.xml">
11    
12     <title>Gentoo Distcc Documentation</title>
13    
14     <author title="Author">
15 swift 1.10 <mail link="lisa@gentoo.org">Lisa Seelye</mail>
16 erwin 1.4 </author>
17 swift 1.17
18 erwin 1.4 <author title="Editor">
19 swift 1.10 <mail link="vapier@gentoo.org">Mike Frysinger</mail>
20 erwin 1.4 </author>
21 swift 1.17
22 erwin 1.4 <author title="Editor">
23 swift 1.10 <mail link="erwin@gentoo.org">Erwin</mail>
24 blubber 1.2 </author>
25 swift 1.17
26     <author title="Editor">
27 swift 1.10 <mail link="swift@gentoo.org">Sven Vermeulen</mail>
28 swift 1.1 </author>
29 swift 1.17
30     <author title="Reviewer">
31 swift 1.10 <mail link="blubber@gentoo.org">Tiemo Kieft</mail>
32 swift 1.1 </author>
33    
34 swift 1.15 <abstract>
35     This document serves as a HOWTO for using distcc with Gentoo.
36     </abstract>
37    
38 swift 1.17 <!-- The content of this document is licensed under the CC-BY-SA license -->
39     <!-- See http://creativecommons.org/licenses/by-sa/1.0 -->
40 swift 1.6 <license/>
41    
42 swift 1.17 <version>1.5</version>
43     <date>May 09, 2004</date>
44 swift 1.1
45     <chapter>
46 swift 1.10 <title>Introduction</title>
47     <section>
48     <title>What is distcc?</title>
49     <body>
50    
51     <p>
52     Distcc is a program designed to distribute compiling tasks across a network to
53     participating hosts. It is comprised of a server, <c>distccd</c> and a client
54     program, <c>distcc</c>. Distcc can work transparently with <uri
55 swift 1.17 link="http://ccache.samba.org">ccache</uri> and Portage with a little work.
56     </p>
57    
58     </body>
59     </section>
60     <section>
61     <title>Using distcc to bootstrap</title>
62     <body>
63    
64     <p>
65     If you are planning on using distcc to help you bootstrap a Gentoo installation,
66     make sure you read the part on <uri link="#bootstrapping">Using distcc to
67     Bootstrap</uri> which is situated further down in this document.
68 swift 1.10 </p>
69    
70     </body>
71     </section>
72 swift 1.8 </chapter>
73    
74     <chapter>
75 swift 1.10 <title>Setup</title>
76     <section>
77     <title>Dependencies</title>
78     <body>
79    
80 swift 1.17 <p>
81     To start off with, to use Distcc all of the computers on your network need to
82     have the same GCC versions. Mixing 3.3.x (where the x varies) is okay, but
83     mixing 3.3.x and 3.2.x is not: your programs <b>will</b> have errors in it.
84     </p>
85 swift 1.10
86     </body>
87     </section>
88     <section>
89     <title>Installing Distcc</title>
90     <body>
91    
92     <p>
93 swift 1.17 There are a couple of options you should know about before you start
94     installing distcc.
95 swift 1.10 </p>
96    
97     <p>
98     Distcc ships with a graphical monitor to monitor tasks that your computer is
99     sending away for compilation. If you use Gnome then put 'gnome' in your USE
100     flags. However, if you don't use Gnome and would still like to have the
101     monitor then you should put 'gtk' in your USE flags.
102     </p>
103    
104 swift 1.17 <pre caption="Installing distcc">
105     # <i>emerge distcc</i>
106     </pre>
107    
108 swift 1.10 </body>
109     </section>
110     <section>
111     <title>Setting up Portage to use Distcc</title>
112     <body>
113    
114     <p>
115 swift 1.17 Setting up distcc is easy to do with Portage. Execute the following steps on
116     each system that should participate in the distributed compiling:
117 swift 1.10 </p>
118    
119     <pre caption="Integrating Distcc and Portage">
120 swift 1.1 # <i>emerge distcc</i>
121     # <i>nano -w /etc/make.conf</i>
122 swift 1.17 <comment>(Set N to a suitable number for your particular setup)</comment>
123     <comment>(A common heuristic strategy is twice the number of CPUs + 1)</comment>
124     MAKEOPTS="-jN"
125     <comment>(Add distcc to your FEATURES variable)</comment>
126     FEATURES="distcc"
127     <comment>(Set distcc's directory for temporary space)</comment>
128     DISTCC_DIR="${PORTAGE_TMPDIR}/.distcc"
129 swift 1.5 </pre>
130 swift 1.1
131 swift 1.10 <p>
132 swift 1.17 Next you have to specify what hosts you want to use. Use the
133     <c>distcc-config</c> command to set the list of hosts. Here is an example of
134     some hosts that might be in your list:
135 swift 1.10 </p>
136    
137     <pre caption="Examples of host definitions">
138 swift 1.8 192.168.0.1 192.168.0.2 192.168.0.3
139     192.168.0.1/2 192.168.0.2 192.168.0.3/10
140     192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4
141     @192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3
142 swift 1.17 <comment>(There are also several other methods of setting up hosts. See the
143     distcc manpage for more details.)</comment>
144 swift 1.8 </pre>
145 swift 1.10
146     <p>
147     It may all look complicated, but in most cases a variant of line 1 or 2 will
148 swift 1.17 work.
149 swift 1.10 </p>
150    
151 swift 1.17 <ul>
152     <li>Line 1 is just a space-delimited list of hosts</li>
153     <li>
154     Line 2 is a list of hosts that specifies the maximum amount of jobs (by
155     defining the <c>/N</c> setting) to send that host at any given time
156     </li>
157     </ul>
158    
159 swift 1.10 <p>
160 swift 1.17 Since most people won't be using lines 3 or 4, I'll <uri
161     link="http://distcc.samba.org/man/distcc_1.html">point you to</uri> the distcc
162     docs for more information.
163     </p>
164    
165     <p>
166     For instance, to set the first line in the previous example:
167 swift 1.10 </p>
168    
169     <pre caption="Sample command to set the hosts">
170 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "192.168.0.1 192.168.0.2 192.168.0.3"</i>
171 swift 1.8 </pre>
172 swift 1.1
173 swift 1.10 <p>
174 swift 1.17 Now start the distcc daemon on all the participating computers:
175 swift 1.10 </p>
176    
177 swift 1.17 <pre caption="Starting the distcc daemon">
178     <comment>(Add distccd to the default runlevel)</comment>
179     # <i>rc-update add distccd default</i>
180     <comment>(Start the distcc daemon)</comment>
181     # <i>/etc/init.d/distccd start</i>
182 swift 1.8 </pre>
183 swift 1.10
184     </body>
185     </section>
186     <section>
187     <title>Setting up Distcc to Work With Automake</title>
188     <body>
189    
190     <p>
191     This is in some cases easier than the Portage setup. What you have to do is
192 swift 1.17 update your <c>PATH</c> variable to include <path>/usr/lib/distcc/bin</path>
193     in front of the directory that contains <c>gcc</c> (<path>/usr/bin</path>).
194 swift 1.10 However, there is a caveat. If you use ccache you have to put distcc after
195 swift 1.17 the ccache part:
196 swift 1.10 </p>
197    
198     <pre caption="Setting your path">
199 swift 1.1 # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
200 swift 1.8 </pre>
201 swift 1.10
202     <p>
203     Then, as you would normally type <c>make</c>, you would type <c>make -jN</c>
204 swift 1.17 (where N is an integer). The value of N depends on your network and the types
205     of computers you are using to compile. For me, I have a dual processor P3 and
206     a K6-2/400 that compiles everything for my desktop (a 1200MHz Athlon
207     Thunderbird) and I use -j5. Test your own settings to find the most
208     performant number.
209 swift 1.10 </p>
210    
211     </body>
212     </section>
213 blubber 1.2 </chapter>
214 swift 1.1
215 blubber 1.2 <chapter>
216 swift 1.17 <title>Cross-Compiling</title>
217     <section>
218     <title>A Note on Cross-Compiling</title>
219     <body>
220    
221     <p>
222     Cross-compiling is using one architecture to build programs for another
223     architecture. This can be as simple as using an Athlon (i686) to build a
224     program for a K6-2 (i586), or using a Sparc to build a program for a ppc.
225     </p>
226    
227     </body>
228     </section>
229 swift 1.10 <section>
230 swift 1.17 <title>A Personal Note on Cross-compiling</title>
231 swift 1.10 <body>
232    
233     <p>
234 swift 1.17 I'd love to be able to help out on the Gentoo Cross-compiling effort, but I
235     don't have any non-x86 machines. I can create things that work <e>in
236     theory</e>, but I have to rely on other people to test what I write.
237     That works to an extent, but it's tough. ;-)
238 swift 1.10 </p>
239    
240 swift 1.17 <p>
241     In the near future I hope to acquire a Sparc of some type to put Gentoo on so
242     I can play around in-house.
243     </p>
244    
245     </body>
246     </section>
247     </chapter>
248    
249     <chapter id="bootstrapping">
250     <title>Using Distcc to Bootstrap</title>
251     <section>
252     <title>Step 1: Configure Portage </title>
253     <body>
254 swift 1.10
255     <p>
256 swift 1.17 Boot your new box with a Gentoo Linux LiveCD and follow the <uri
257     link="/doc/en/handbook/handbook-x86.xml?part=1">installation instructions</uri>
258     up until the bootstrapping part. Then configure Portage to use distcc:
259 swift 1.10 </p>
260    
261 swift 1.17 <pre caption="Preliminary Setup">
262     # <i>nano -w /etc/make.conf</i>
263     <comment>(Add distcc to the FEATURES variable)</comment>
264     FEATURES="distcc"
265     <comment>(Set distcc's directory for temporary space)</comment>
266     DISTCC_DIR="${PORTAGE_TMPDIR}/.distcc"
267     <comment>(Modify MAKEOPTS to include -jN with N an appropriate integer)</comment>
268     MAKEOPTS="-jN"
269 swift 1.8 </pre>
270 swift 1.10
271     </body>
272     </section>
273     <section>
274     <title>Step 2: Getting Distcc</title>
275     <body>
276    
277     <p>
278 swift 1.17 Install distcc:
279 swift 1.10 </p>
280    
281     <pre caption="Getting Distcc on the new box">
282 swift 1.11 # <i>USE='-*' emerge --nodeps distcc</i>
283 swift 1.8 </pre>
284 swift 1.10
285     </body>
286     </section>
287     <section>
288 swift 1.17 <title>Step 3: Setting Up Distcc</title>
289 swift 1.10 <body>
290    
291     <p>
292 swift 1.17 Now run <c>distcc-config</c> to setup distcc:
293 swift 1.10 </p>
294    
295     <pre caption="Final distcc setup">
296 erwin 1.4 # <i>/usr/bin/distcc-config --install</i>
297 swift 1.17 <comment>(Substitute host1, host2, ... with the IP number(s) of the participating hosts)</comment>
298 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i>
299 swift 1.17 </pre>
300 swift 1.10
301     <p>
302 swift 1.17 distcc is now set up to bootstrap! Continue the official installation
303     instructions and <e>don't forget</e> to re-emerge distcc after <c>emerge
304     system</c>. This is to make sure that all of the dependencies you want are
305     installed as well.
306 swift 1.10 </p>
307    
308 swift 1.12 <note>
309 swift 1.17 During bootstrap and <c>emerge system</c> Distcc may not appear to be used.
310     This is expected as some ebuilds do not work well with distcc, so they
311     intentionally disable it.
312 swift 1.12 </note>
313    
314 swift 1.10 </body>
315     </section>
316 blubber 1.2 </chapter>
317 swift 1.1
318 swift 1.5 <chapter>
319 swift 1.10 <title>Troubleshooting</title>
320     <section>
321     <title>Mozilla and Xfree</title>
322     <body>
323    
324     <p>
325 swift 1.17 As you emerge various packages, you'll notice that some of them aren't being
326     distributed (and aren't being built in parallel). This is because the
327     developers of the Mozilla and Xfree ebuilds intentionally disable parallel
328     building because it is known to cause problems.
329 swift 1.10 </p>
330    
331     <p>
332 swift 1.17 Sometimes distcc might cause a package to fail to compile. If this happens
333     for you, please <uri link="http://bugs.gentoo.org">report</uri> it to us.
334 swift 1.10 </p>
335    
336     </body>
337     </section>
338     <section>
339     <title>A Mixture of hardened-gcc and non-hardened-gcc Hosts Will Be Faulty</title>
340     <body>
341    
342     <p>
343     With such a long title any explanation here is almost irrelevent. However, if
344     you plan to use distcc across hosts that have the <uri
345 swift 1.17 link="/proj/en/hardened/etdyn-ssp.xml">PaX/hardened-gcc</uri> and some that do
346     not, you will run into problems.
347 swift 1.10 </p>
348    
349     <p>
350     The solution requires a little foresight on your part; you have to run
351     <c>hardened-gcc -R</c> on the host that has PaX/hardened-gcc, or you have to
352     enable PaX protections in your kernel and <c>emerge hardened-gcc</c>. Both of
353     which are a good thing to do since for the most part the protections offered
354     by both packages is a good thing and is transparent to the user.
355     </p>
356    
357     </body>
358     </section>
359     <section>
360     <title>Mixed GCC Versions</title>
361     <body>
362    
363     <p>
364 swift 1.17 If you have different GCC versions on your hosts, there will likely be very
365 swift 1.10 weird problems. The solution is to make certain all hosts have the same GCC
366     version.
367     </p>
368    
369     </body>
370     </section>
371 swift 1.5 </chapter>
372 swift 1.1
373     <chapter>
374 swift 1.10 <title>Distcc Extras</title>
375     <section>
376     <title>Distcc Monitors</title>
377     <body>
378    
379     <p>
380 swift 1.17 Distcc ships with two monitors. The text-based one is always built and is
381 swift 1.10 called <c>distccmon-text</c>. Running it for the first time can be a bit
382     confusing, but it is really quite easy to use. If you run the program with no
383     parameter it will run once. However, if you pass it a number it will update
384     every N seconds, where N is the argument you passed.
385     </p>
386    
387     <p>
388     The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c>
389 swift 1.17 in your <c>USE</c> flags. This one is GTK+ based, runs in an X environment
390 swift 1.10 and it is quite lovely.
391     </p>
392    
393     <pre caption="Starting the monitors">
394 swift 1.17 # <i>distccmon-text N</i>
395     <comment>(Or ...)</comment>
396     # <i>distccmon-gnome</i>
397 swift 1.8 </pre>
398 swift 1.10
399     </body>
400     </section>
401 swift 1.1 </chapter>
402    
403     <chapter>
404 swift 1.10 <title>Future Plans for Distcc and Gentoo</title>
405     <section>
406     <title>distcc-subnetscan</title>
407     <body>
408    
409     <p>
410     <c>distcc-subnetscan</c> is a perl program in development that will scan a
411     subnet for hosts that have a participating distcc daemon. Even this will be
412     enhanced to test if the daemon on a remote host will conform to a specific
413     <c>CHOST</c> setting to make cross-compiling easier.
414     </p>
415    
416     <p>
417 swift 1.17 The perl script is being kept at <uri link="http://dev.gentoo.org/~lisa/distcc/distcc-subnetscan/distcc-subnetscan.pl">lisa's
418     dev page</uri> until a more formal home can be found.
419 swift 1.10 </p>
420    
421     </body>
422     </section>
423 swift 1.1 </chapter>
424     </guide>

  ViewVC Help
Powered by ViewVC 1.1.20