/[gentoo]/xml/htdocs/doc/en/distcc.xml
Gentoo

Contents of /xml/htdocs/doc/en/distcc.xml

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.40 - (hide annotations) (download) (as text)
Thu Jun 21 23:46:16 2007 UTC (7 years, 2 months ago) by rane
Branch: MAIN
Changes since 1.39: +6 -22 lines
File MIME type: application/xml
#182759, added emerge distcc-config, removed adding distcc user as ebuild handles that

1 swift 1.1 <?xml version='1.0' encoding="UTF-8"?>
2 blubber 1.3 <!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3 rane 1.40 <!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/distcc.xml,v 1.39 2006/12/24 17:54:01 nightmorph Exp $ -->
4 blubber 1.2
5 swift 1.1 <guide link="/doc/en/distcc.xml">
6    
7     <title>Gentoo Distcc Documentation</title>
8    
9     <author title="Author">
10 swift 1.10 <mail link="lisa@gentoo.org">Lisa Seelye</mail>
11 erwin 1.4 </author>
12     <author title="Editor">
13 swift 1.10 <mail link="vapier@gentoo.org">Mike Frysinger</mail>
14 erwin 1.4 </author>
15     <author title="Editor">
16 swift 1.10 <mail link="erwin@gentoo.org">Erwin</mail>
17 blubber 1.2 </author>
18 swift 1.17 <author title="Editor">
19 nightmorph 1.38 <mail link="swift@gentoo.org">Sven Vermeulen</mail>
20 swift 1.1 </author>
21 pylon 1.20 <author title="Editor">
22     <mail link="pylon@gentoo.org">Lars Weiler</mail>
23     </author>
24 swift 1.17 <author title="Reviewer">
25 swift 1.10 <mail link="blubber@gentoo.org">Tiemo Kieft</mail>
26 swift 1.1 </author>
27    
28 swift 1.15 <abstract>
29     This document serves as a HOWTO for using distcc with Gentoo.
30     </abstract>
31    
32 swift 1.17 <!-- The content of this document is licensed under the CC-BY-SA license -->
33 jkt 1.34 <!-- See http://creativecommons.org/licenses/by-sa/2.5 -->
34 swift 1.6 <license/>
35    
36 rane 1.40 <version>1.19</version>
37     <date>2007-06-22</date>
38 swift 1.1
39     <chapter>
40 swift 1.10 <title>Introduction</title>
41     <section>
42     <title>What is distcc?</title>
43     <body>
44    
45     <p>
46     Distcc is a program designed to distribute compiling tasks across a network to
47 smithj 1.27 participating hosts. It is comprised of a server, <c>distccd</c>, and a client
48 swift 1.10 program, <c>distcc</c>. Distcc can work transparently with <uri
49 smithj 1.28 link="http://ccache.samba.org">ccache</uri>, Portage, and Automake with a
50     little setup.
51 swift 1.17 </p>
52    
53     </body>
54     </section>
55     <section>
56     <title>Using distcc to bootstrap</title>
57     <body>
58    
59     <p>
60 smithj 1.28 If you are planning on using distcc to help you bootstrap a Gentoo
61     installation, make sure you read the section <uri link="#bootstrapping">Using
62     distcc to Bootstrap</uri>, which is situated further down in this document.
63 swift 1.10 </p>
64    
65     </body>
66     </section>
67 swift 1.8 </chapter>
68    
69     <chapter>
70 swift 1.10 <title>Setup</title>
71     <section>
72     <title>Dependencies</title>
73     <body>
74    
75 swift 1.17 <p>
76 smithj 1.28 In order to use Distcc, all of the computers on your network need to have the
77     same GCC versions. For example, mixing 3.3.x (where the x varies) is okay, but
78     mixing 3.3.x with 3.2.x <b>may</b> result in compilation errors or runtime
79     errors.
80 swift 1.17 </p>
81 swift 1.10
82     </body>
83     </section>
84     <section>
85     <title>Installing Distcc</title>
86     <body>
87    
88     <p>
89 swift 1.25 There are a couple of options you should be aware of before you start
90 swift 1.17 installing distcc.
91 swift 1.10 </p>
92    
93     <p>
94     Distcc ships with a graphical monitor to monitor tasks that your computer is
95     sending away for compilation. If you use Gnome then put 'gnome' in your USE
96     flags. However, if you don't use Gnome and would still like to have the
97     monitor then you should put 'gtk' in your USE flags.
98     </p>
99    
100 swift 1.17 <pre caption="Installing distcc">
101     # <i>emerge distcc</i>
102     </pre>
103    
104 nightmorph 1.36 <impo>
105     Remember, you must be sure to install distcc on all of your participating
106     machines.
107     </impo>
108    
109 swift 1.10 </body>
110     </section>
111     <section>
112     <title>Setting up Portage to use Distcc</title>
113     <body>
114    
115     <p>
116 swift 1.25 Setting up Portage to use distcc is easy. Execute the following steps on
117 swift 1.17 each system that should participate in the distributed compiling:
118 swift 1.10 </p>
119    
120     <pre caption="Integrating Distcc and Portage">
121 rane 1.40 # <i>emerge distcc distcc-config</i>
122 swift 1.1 # <i>nano -w /etc/make.conf</i>
123 swift 1.17 <comment>(Set N to a suitable number for your particular setup)</comment>
124 rane 1.40 <comment>(A common strategy is setting N as twice the number of total CPUs + 1 available)</comment>
125 swift 1.17 MAKEOPTS="-jN"
126 smithj 1.28 <comment>(Add distcc to your FEATURES)</comment>
127 swift 1.17 FEATURES="distcc"
128 swift 1.5 </pre>
129 swift 1.1
130 swift 1.25 </body>
131     </section>
132     <section>
133     <title>Specifying Participating Hosts</title>
134     <body>
135    
136 swift 1.10 <p>
137 smithj 1.28 Use the <c>distcc-config</c> command to set the list of hosts. Here is an
138     example of some hosts that might be in your list:
139 swift 1.10 </p>
140    
141     <pre caption="Examples of host definitions">
142 swift 1.8 192.168.0.1 192.168.0.2 192.168.0.3
143     192.168.0.1/2 192.168.0.2 192.168.0.3/10
144     192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4
145     @192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3
146 swift 1.17 <comment>(There are also several other methods of setting up hosts. See the
147     distcc manpage for more details.)</comment>
148 smithj 1.28 <comment>If you wish to compile on the local machine you should put 'localhost'
149     in the hosts list. Conversely if you do not wish to use the local machine to
150     compile (which is often the case) omit it from the hosts list. On a slow
151     machine using localhost may actually slow things down. Make sure to test your
152     settings for performance.</comment>
153 swift 1.8 </pre>
154 swift 1.10
155     <p>
156     It may all look complicated, but in most cases a variant of line 1 or 2 will
157 swift 1.17 work.
158 swift 1.10 </p>
159    
160     <p>
161 swift 1.17 Since most people won't be using lines 3 or 4, I'll <uri
162 smithj 1.28 link="http://distcc.samba.org/man/distcc_1.html">refer to</uri> the distcc
163     docs (man distcc) for more information.
164 swift 1.17 </p>
165    
166     <p>
167     For instance, to set the first line in the previous example:
168 swift 1.10 </p>
169    
170     <pre caption="Sample command to set the hosts">
171 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "192.168.0.1 192.168.0.2 192.168.0.3"</i>
172 swift 1.8 </pre>
173 swift 1.1
174 swift 1.10 <p>
175 swift 1.23 Edit <path>/etc/conf.d/distccd</path> to your needs and be sure to set the
176 smithj 1.28 <c>--allow</c> directive to allow only hosts you trust. For added security,
177     you should also use the <c>--listen</c> directive to tell the distcc daemon
178     what IP to listen on (for multi-homed systems). More information on distcc
179     security can be found at <uri
180     link="http://distcc.samba.org/security.html">Distcc Security Design</uri>.
181 swift 1.23 </p>
182    
183 smithj 1.28 <impo>
184     It is important to use --allow and --listen. Please read the distccd manpage
185 alin 1.30 or the above security document for more information.
186 smithj 1.28 </impo>
187    
188    
189 swift 1.23 <p>
190 swift 1.17 Now start the distcc daemon on all the participating computers:
191 swift 1.10 </p>
192    
193 swift 1.17 <pre caption="Starting the distcc daemon">
194     <comment>(Add distccd to the default runlevel)</comment>
195     # <i>rc-update add distccd default</i>
196     <comment>(Start the distcc daemon)</comment>
197     # <i>/etc/init.d/distccd start</i>
198 swift 1.8 </pre>
199 swift 1.10
200     </body>
201     </section>
202     <section>
203     <title>Setting up Distcc to Work With Automake</title>
204     <body>
205    
206     <p>
207 swift 1.25 This is, in some cases, easier than the Portage setup. What you have to do is
208 swift 1.17 update your <c>PATH</c> variable to include <path>/usr/lib/distcc/bin</path>
209     in front of the directory that contains <c>gcc</c> (<path>/usr/bin</path>).
210 swift 1.10 However, there is a caveat. If you use ccache you have to put distcc after
211 swift 1.17 the ccache part:
212 swift 1.10 </p>
213    
214     <pre caption="Setting your path">
215 swift 1.1 # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
216 nightmorph 1.39 <comment>(You can put this in your ~/.bashrc or equivalent file to have the PATH
217     set every time you log in)</comment>
218 swift 1.8 </pre>
219 swift 1.10
220     <p>
221 neysx 1.22 Then, as you would normally type <c>make</c>, you would type <c>make -jN</c>
222     (where N is an integer). The value of N depends on your network and the types
223 smithj 1.28 of computers you are using to compile. Test your own settings to find the
224     number that yields the best performance.
225 swift 1.10 </p>
226    
227     </body>
228     </section>
229 blubber 1.2 </chapter>
230 swift 1.1
231 blubber 1.2 <chapter>
232 swift 1.17 <title>Cross-Compiling</title>
233     <section>
234 swift 1.10 <body>
235    
236     <p>
237 neysx 1.32 Cross-compiling is using one architecture to build programs for another
238     architecture. This can be as simple as using an Athlon (i686) to build a
239     program for a K6-2 (i586), or using a Sparc to build a program for a ppc. This
240     is documented in our <uri link="/doc/en/cross-compiling-distcc.xml">DistCC
241     Cross-compiling Guide</uri>.
242 swift 1.17 </p>
243    
244     </body>
245     </section>
246     </chapter>
247    
248     <chapter id="bootstrapping">
249     <title>Using Distcc to Bootstrap</title>
250     <section>
251 swift 1.24 <title>Step 1: Configure Portage</title>
252 swift 1.17 <body>
253 nightmorph 1.33 <!--
254     Note that this will need to become a link to the old 2005.1 networked handbooks
255     once the 2006.0 handbooks are released. Until then, the link can stay.
256     -->
257     <p>
258     Boot your new box with a Gentoo Linux LiveCD and follow the <uri
259     link="/doc/en/handbook/handbook-x86.xml?part=1">installation instructions</uri>
260     up until the bootstrapping part. (See the <uri
261 jkt 1.34 link="/doc/en/faq.xml#stage12">Gentoo FAQ</uri> for more information about
262     bootstrapping.) Then configure Portage to use distcc:
263 swift 1.10 </p>
264    
265 swift 1.17 <pre caption="Preliminary Setup">
266     # <i>nano -w /etc/make.conf</i>
267 smithj 1.28 <comment>(Add distcc to the FEATURES</comment>
268 swift 1.17 FEATURES="distcc"
269 smithj 1.28 <comment>(Modify MAKEOPTS to include -jN, where N is twice the number of CPUs
270     +1 available)</comment>
271 swift 1.17 MAKEOPTS="-jN"
272 swift 1.8 </pre>
273 swift 1.10
274 neysx 1.22 <pre caption="Setting your path">
275     # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
276     </pre>
277    
278 swift 1.10 </body>
279     </section>
280     <section>
281     <title>Step 2: Getting Distcc</title>
282     <body>
283    
284 neysx 1.22 <p>
285 swift 1.17 Install distcc:
286 swift 1.10 </p>
287    
288     <pre caption="Getting Distcc on the new box">
289 rane 1.40 # <i>USE='-*' emerge --nodeps sys-devel/distcc sys-devel/distcc-config</i>
290 swift 1.8 </pre>
291 swift 1.10
292     </body>
293     </section>
294     <section>
295 swift 1.17 <title>Step 3: Setting Up Distcc</title>
296 swift 1.10 <body>
297    
298     <p>
299 swift 1.24 Run <c>distcc-config --install</c> to setup distcc:
300 swift 1.10 </p>
301    
302     <pre caption="Final distcc setup">
303 smithj 1.28 <comment>(Substitute host1, host2, ... with the IP number(s) of the
304     participating hosts)</comment>
305 erwin 1.4 # <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i>
306 smithj 1.28 <comment>An example: <i>/usr/bin/distcc-config --set-hosts "localhost
307     192.168.0.4 192.168.0.6"</i></comment>
308 neysx 1.31 </pre>
309 swift 1.10
310     <p>
311 swift 1.25 Distcc is now set up to bootstrap! Continue with the official installation
312 swift 1.24 instructions and <e>do not forget</e> to re-emerge distcc after <c>emerge
313 swift 1.17 system</c>. This is to make sure that all of the dependencies you want are
314     installed as well.
315 swift 1.10 </p>
316    
317 swift 1.12 <note>
318 swift 1.25 During bootstrap and <c>emerge system</c> distcc may not appear to be used.
319 swift 1.17 This is expected as some ebuilds do not work well with distcc, so they
320     intentionally disable it.
321 swift 1.12 </note>
322    
323 swift 1.10 </body>
324     </section>
325 blubber 1.2 </chapter>
326 swift 1.1
327 swift 1.5 <chapter>
328 swift 1.10 <title>Troubleshooting</title>
329     <section>
330 fox2mike 1.35 <title>Some Packages Don't Use Distcc</title>
331 swift 1.10 <body>
332    
333     <p>
334 nightmorph 1.33 As you emerge various packages, you'll notice that some of them aren't being
335 fox2mike 1.35 distributed (and aren't being built in parallel). This may happen because the
336     package's Makefile doesn't support parallel operations or the maintainer of the
337     ebuild has explicitly disabled parallel operations due to a known problem.
338 swift 1.10 </p>
339    
340     <p>
341 swift 1.17 Sometimes distcc might cause a package to fail to compile. If this happens
342     for you, please <uri link="http://bugs.gentoo.org">report</uri> it to us.
343 swift 1.10 </p>
344    
345     </body>
346     </section>
347     <section>
348     <title>Mixed GCC Versions</title>
349     <body>
350    
351     <p>
352 swift 1.17 If you have different GCC versions on your hosts, there will likely be very
353 swift 1.10 weird problems. The solution is to make certain all hosts have the same GCC
354     version.
355     </p>
356    
357 swift 1.24 <p>
358     Recent Portage updates have made Portage use <c>${CHOST}-gcc</c> instead of
359     <c>gcc</c>. This means that if you're mixing i686 machines with other types
360     (i386, i586) you will run into problems. A workaround for this may be to
361 smithj 1.28 <c>export CC='gcc' CXX='c++'</c> or to put it in <path>/etc/make.conf</path>.
362 swift 1.24 </p>
363    
364 smithj 1.28 <impo>
365     Doing this explicitly redefines some behaviour of Portage and may have some
366     weird results in the future. Only do this if you're mixing CHOSTs.
367     </impo>
368    
369    
370 swift 1.10 </body>
371     </section>
372 swift 1.5 </chapter>
373 swift 1.1
374     <chapter>
375 swift 1.10 <title>Distcc Extras</title>
376     <section>
377     <title>Distcc Monitors</title>
378     <body>
379    
380     <p>
381 swift 1.17 Distcc ships with two monitors. The text-based one is always built and is
382 swift 1.10 called <c>distccmon-text</c>. Running it for the first time can be a bit
383     confusing, but it is really quite easy to use. If you run the program with no
384     parameter it will run once. However, if you pass it a number it will update
385     every N seconds, where N is the argument you passed.
386     </p>
387    
388     <p>
389 smithj 1.28 The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c>
390     in your <c>USE</c> flags. This one is GTK+ based, runs in an X environment
391     and it is quite lovely. For Gentoo the GUI monitor has been called
392     <c>distccmon-gui</c> for less confusion. Elsewhere it may be referred to as
393     <c>distccmon-gnome</c>.
394 swift 1.10 </p>
395    
396     <pre caption="Starting the monitors">
397 swift 1.17 # <i>distccmon-text N</i>
398 neysx 1.22 <comment>(Or)</comment>
399 smithj 1.28 # <i>distccmon-gui</i>
400 neysx 1.22 <comment>To monitor Portage's distcc usage you can use:</comment>
401 swift 1.26 # <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-text N</i>
402 neysx 1.22 <comment>(Or)</comment>
403 smithj 1.28 # <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-gui</i>
404 swift 1.8 </pre>
405 swift 1.10
406 neysx 1.22 <impo>
407 swift 1.26 If your distcc directory is elsewhere, change the DISTCC_DIR variable
408     accordingly.
409 neysx 1.22 </impo>
410 swift 1.10
411     </body>
412     </section>
413 swift 1.1 </chapter>
414     </guide>

  ViewVC Help
Powered by ViewVC 1.1.20