/[gentoo]/xml/htdocs/doc/en/distcc.xml
Gentoo

Contents of /xml/htdocs/doc/en/distcc.xml

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.33 - (show annotations) (download) (as text)
Sat Feb 18 23:21:44 2006 UTC (8 years, 10 months ago) by nightmorph
Branch: MAIN
Changes since 1.32: +17 -33 lines
File MIME type: application/xml
updated distcc guide for bug 123231

1 <?xml version='1.0' encoding="UTF-8"?>
2 <!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3 <!-- $Header: $ -->
4
5 <guide link="/doc/en/distcc.xml">
6
7 <title>Gentoo Distcc Documentation</title>
8
9 <author title="Author">
10 <mail link="lisa@gentoo.org">Lisa Seelye</mail>
11 </author>
12 <author title="Editor">
13 <mail link="vapier@gentoo.org">Mike Frysinger</mail>
14 </author>
15 <author title="Editor">
16 <mail link="erwin@gentoo.org">Erwin</mail>
17 </author>
18 <author title="Editor">
19 <mail link="swift@gentoo.org">Sven Vermeulen</mail>
20 </author>
21 <author title="Editor">
22 <mail link="pylon@gentoo.org">Lars Weiler</mail>
23 </author>
24 <author title="Reviewer">
25 <mail link="blubber@gentoo.org">Tiemo Kieft</mail>
26 </author>
27
28 <abstract>
29 This document serves as a HOWTO for using distcc with Gentoo.
30 </abstract>
31
32 <!-- The content of this document is licensed under the CC-BY-SA license -->
33 <!-- See http://creativecommons.org/licenses/by-sa/1.0 -->
34 <license/>
35
36 <version>1.14</version>
37 <date>2006-02-18</date>
38
39 <chapter>
40 <title>Introduction</title>
41 <section>
42 <title>What is distcc?</title>
43 <body>
44
45 <p>
46 Distcc is a program designed to distribute compiling tasks across a network to
47 participating hosts. It is comprised of a server, <c>distccd</c>, and a client
48 program, <c>distcc</c>. Distcc can work transparently with <uri
49 link="http://ccache.samba.org">ccache</uri>, Portage, and Automake with a
50 little setup.
51 </p>
52
53 </body>
54 </section>
55 <section>
56 <title>Using distcc to bootstrap</title>
57 <body>
58
59 <p>
60 If you are planning on using distcc to help you bootstrap a Gentoo
61 installation, make sure you read the section <uri link="#bootstrapping">Using
62 distcc to Bootstrap</uri>, which is situated further down in this document.
63 </p>
64
65 </body>
66 </section>
67 </chapter>
68
69 <chapter>
70 <title>Setup</title>
71 <section>
72 <title>Dependencies</title>
73 <body>
74
75 <p>
76 In order to use Distcc, all of the computers on your network need to have the
77 same GCC versions. For example, mixing 3.3.x (where the x varies) is okay, but
78 mixing 3.3.x with 3.2.x <b>may</b> result in compilation errors or runtime
79 errors.
80 </p>
81
82 </body>
83 </section>
84 <section>
85 <title>Installing Distcc</title>
86 <body>
87
88 <p>
89 There are a couple of options you should be aware of before you start
90 installing distcc.
91 </p>
92
93 <p>
94 Distcc ships with a graphical monitor to monitor tasks that your computer is
95 sending away for compilation. If you use Gnome then put 'gnome' in your USE
96 flags. However, if you don't use Gnome and would still like to have the
97 monitor then you should put 'gtk' in your USE flags.
98 </p>
99
100 <pre caption="Installing distcc">
101 # <i>emerge distcc</i>
102 </pre>
103
104 </body>
105 </section>
106 <section>
107 <title>Setting up Portage to use Distcc</title>
108 <body>
109
110 <p>
111 Setting up Portage to use distcc is easy. Execute the following steps on
112 each system that should participate in the distributed compiling:
113 </p>
114
115 <pre caption="Integrating Distcc and Portage">
116 # <i>emerge distcc</i>
117 # <i>nano -w /etc/make.conf</i>
118 <comment>(Set N to a suitable number for your particular setup)</comment>
119 <comment>(A common strategy is setting N as twice the number of total CPUs + 1
120 available)</comment>
121 MAKEOPTS="-jN"
122 <comment>(Add distcc to your FEATURES)</comment>
123 FEATURES="distcc"
124 </pre>
125
126 </body>
127 </section>
128 <section>
129 <title>Specifying Participating Hosts</title>
130 <body>
131
132 <p>
133 Use the <c>distcc-config</c> command to set the list of hosts. Here is an
134 example of some hosts that might be in your list:
135 </p>
136
137 <pre caption="Examples of host definitions">
138 192.168.0.1 192.168.0.2 192.168.0.3
139 192.168.0.1/2 192.168.0.2 192.168.0.3/10
140 192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4
141 @192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3
142 <comment>(There are also several other methods of setting up hosts. See the
143 distcc manpage for more details.)</comment>
144 <comment>If you wish to compile on the local machine you should put 'localhost'
145 in the hosts list. Conversely if you do not wish to use the local machine to
146 compile (which is often the case) omit it from the hosts list. On a slow
147 machine using localhost may actually slow things down. Make sure to test your
148 settings for performance.</comment>
149 </pre>
150
151 <p>
152 It may all look complicated, but in most cases a variant of line 1 or 2 will
153 work.
154 </p>
155
156 <p>
157 Since most people won't be using lines 3 or 4, I'll <uri
158 link="http://distcc.samba.org/man/distcc_1.html">refer to</uri> the distcc
159 docs (man distcc) for more information.
160 </p>
161
162 <p>
163 For instance, to set the first line in the previous example:
164 </p>
165
166 <pre caption="Sample command to set the hosts">
167 # <i>/usr/bin/distcc-config --set-hosts "192.168.0.1 192.168.0.2 192.168.0.3"</i>
168 </pre>
169
170 <p>
171 Edit <path>/etc/conf.d/distccd</path> to your needs and be sure to set the
172 <c>--allow</c> directive to allow only hosts you trust. For added security,
173 you should also use the <c>--listen</c> directive to tell the distcc daemon
174 what IP to listen on (for multi-homed systems). More information on distcc
175 security can be found at <uri
176 link="http://distcc.samba.org/security.html">Distcc Security Design</uri>.
177 </p>
178
179 <impo>
180 It is important to use --allow and --listen. Please read the distccd manpage
181 or the above security document for more information.
182 </impo>
183
184
185 <p>
186 Now start the distcc daemon on all the participating computers:
187 </p>
188
189 <pre caption="Starting the distcc daemon">
190 <comment>(Add distccd to the default runlevel)</comment>
191 # <i>rc-update add distccd default</i>
192 <comment>(Start the distcc daemon)</comment>
193 # <i>/etc/init.d/distccd start</i>
194 </pre>
195
196 </body>
197 </section>
198 <section>
199 <title>Setting up Distcc to Work With Automake</title>
200 <body>
201
202 <p>
203 This is, in some cases, easier than the Portage setup. What you have to do is
204 update your <c>PATH</c> variable to include <path>/usr/lib/distcc/bin</path>
205 in front of the directory that contains <c>gcc</c> (<path>/usr/bin</path>).
206 However, there is a caveat. If you use ccache you have to put distcc after
207 the ccache part:
208 </p>
209
210 <pre caption="Setting your path">
211 # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
212 <comment>You can put this in your .bashrc or equivelant file to have the PATH
213 set every time you log in</comment>
214 </pre>
215
216 <p>
217 Then, as you would normally type <c>make</c>, you would type <c>make -jN</c>
218 (where N is an integer). The value of N depends on your network and the types
219 of computers you are using to compile. Test your own settings to find the
220 number that yields the best performance.
221 </p>
222
223 </body>
224 </section>
225 </chapter>
226
227 <chapter>
228 <title>Cross-Compiling</title>
229 <section>
230 <body>
231
232 <p>
233 Cross-compiling is using one architecture to build programs for another
234 architecture. This can be as simple as using an Athlon (i686) to build a
235 program for a K6-2 (i586), or using a Sparc to build a program for a ppc. This
236 is documented in our <uri link="/doc/en/cross-compiling-distcc.xml">DistCC
237 Cross-compiling Guide</uri>.
238 </p>
239
240 </body>
241 </section>
242 </chapter>
243
244 <chapter id="bootstrapping">
245 <title>Using Distcc to Bootstrap</title>
246 <section>
247 <title>Step 1: Configure Portage</title>
248 <body>
249 <!--
250 Note that this will need to become a link to the old 2005.1 networked handbooks
251 once the 2006.0 handbooks are released. Until then, the link can stay.
252 -->
253 <p>
254 Boot your new box with a Gentoo Linux LiveCD and follow the <uri
255 link="/doc/en/handbook/handbook-x86.xml?part=1">installation instructions</uri>
256 up until the bootstrapping part. (See the <uri
257 link="http://www.gentoo.org/doc/en/faq.xml#stage12">Gentoo FAQ</uri> for more
258 information about bootstrapping.) Then configure Portage to use distcc:
259 </p>
260
261 <pre caption="Preliminary Setup">
262 # <i>nano -w /etc/make.conf</i>
263 <comment>(Add distcc to the FEATURES</comment>
264 FEATURES="distcc"
265 <comment>(Modify MAKEOPTS to include -jN, where N is twice the number of CPUs
266 +1 available)</comment>
267 MAKEOPTS="-jN"
268 </pre>
269
270 <pre caption="Setting your path">
271 # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
272 </pre>
273
274 </body>
275 </section>
276 <section>
277 <title>Step 2: Getting Distcc</title>
278 <body>
279
280 <p>
281 Before the installation of distcc, a user called distcc must be added to the
282 <path>/etc/passwd</path>:
283 </p>
284
285 <pre caption="Create user distcc">
286 # <i>echo "distcc:x:240:2:distccd:/dev/null:/bin/false" &gt;&gt;/etc/passwd</i>
287 </pre>
288
289 <impo>
290 It is important to note that adding users like this is very bad. We only do
291 it here because there is no <c>useradd</c> utility (which you normally use for
292 adding users) yet at this stage of installation.
293 </impo>
294
295 <p>
296 Install distcc:
297 </p>
298
299 <pre caption="Getting Distcc on the new box">
300 # <i>USE='-*' emerge --nodeps sys-devel/distcc</i>
301 </pre>
302
303 </body>
304 </section>
305 <section>
306 <title>Step 3: Setting Up Distcc</title>
307 <body>
308
309 <p>
310 Run <c>distcc-config --install</c> to setup distcc:
311 </p>
312
313 <pre caption="Final distcc setup">
314 <comment>(Substitute host1, host2, ... with the IP number(s) of the
315 participating hosts)</comment>
316 # <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i>
317 <comment>An example: <i>/usr/bin/distcc-config --set-hosts "localhost
318 192.168.0.4 192.168.0.6"</i></comment>
319 </pre>
320
321 <p>
322 Distcc is now set up to bootstrap! Continue with the official installation
323 instructions and <e>do not forget</e> to re-emerge distcc after <c>emerge
324 system</c>. This is to make sure that all of the dependencies you want are
325 installed as well.
326 </p>
327
328 <note>
329 During bootstrap and <c>emerge system</c> distcc may not appear to be used.
330 This is expected as some ebuilds do not work well with distcc, so they
331 intentionally disable it.
332 </note>
333
334 </body>
335 </section>
336 </chapter>
337
338 <chapter>
339 <title>Troubleshooting</title>
340 <section>
341 <title>Mozilla and Xorg</title>
342 <body>
343
344 <p>
345 As you emerge various packages, you'll notice that some of them aren't being
346 distributed (and aren't being built in parallel). This is because the
347 developers of the Mozilla and Xorg-X11 ebuilds intentionally disable parallel
348 building because it is known to cause problems.
349 </p>
350
351 <p>
352 Sometimes distcc might cause a package to fail to compile. If this happens
353 for you, please <uri link="http://bugs.gentoo.org">report</uri> it to us.
354 </p>
355
356 </body>
357 </section>
358 <section>
359 <title>Mixed GCC Versions</title>
360 <body>
361
362 <p>
363 If you have different GCC versions on your hosts, there will likely be very
364 weird problems. The solution is to make certain all hosts have the same GCC
365 version.
366 </p>
367
368 <p>
369 Recent Portage updates have made Portage use <c>${CHOST}-gcc</c> instead of
370 <c>gcc</c>. This means that if you're mixing i686 machines with other types
371 (i386, i586) you will run into problems. A workaround for this may be to
372 <c>export CC='gcc' CXX='c++'</c> or to put it in <path>/etc/make.conf</path>.
373 </p>
374
375 <impo>
376 Doing this explicitly redefines some behaviour of Portage and may have some
377 weird results in the future. Only do this if you're mixing CHOSTs.
378 </impo>
379
380
381 </body>
382 </section>
383 </chapter>
384
385 <chapter>
386 <title>Distcc Extras</title>
387 <section>
388 <title>Distcc Monitors</title>
389 <body>
390
391 <p>
392 Distcc ships with two monitors. The text-based one is always built and is
393 called <c>distccmon-text</c>. Running it for the first time can be a bit
394 confusing, but it is really quite easy to use. If you run the program with no
395 parameter it will run once. However, if you pass it a number it will update
396 every N seconds, where N is the argument you passed.
397 </p>
398
399 <p>
400 The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c>
401 in your <c>USE</c> flags. This one is GTK+ based, runs in an X environment
402 and it is quite lovely. For Gentoo the GUI monitor has been called
403 <c>distccmon-gui</c> for less confusion. Elsewhere it may be referred to as
404 <c>distccmon-gnome</c>.
405 </p>
406
407 <pre caption="Starting the monitors">
408 # <i>distccmon-text N</i>
409 <comment>(Or)</comment>
410 # <i>distccmon-gui</i>
411 <comment>To monitor Portage's distcc usage you can use:</comment>
412 # <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-text N</i>
413 <comment>(Or)</comment>
414 # <i>DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-gui</i>
415 </pre>
416
417 <impo>
418 If your distcc directory is elsewhere, change the DISTCC_DIR variable
419 accordingly.
420 </impo>
421
422 </body>
423 </section>
424 </chapter>
425 </guide>

  ViewVC Help
Powered by ViewVC 1.1.20