/[gentoo]/xml/htdocs/doc/en/distcc.xml
Gentoo

Contents of /xml/htdocs/doc/en/distcc.xml

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.17 - (show annotations) (download) (as text)
Sun May 9 10:16:34 2004 UTC (10 years, 3 months ago) by swift
Branch: MAIN
Changes since 1.16: +155 -178 lines
File MIME type: application/xml
Various updates, including removal of obsoleted paragraphs, removal of duplicate
information, including reference to bootstrapping section, removing the many
colorful <impo>, <warn> and <note>'s and fixing language. Also added the
<license/> tag:
11:57 <@SwifT> lisa: ping (final one :)
11:57 <@lisa> SwifT: hi
11:58 <@SwifT> lisa: may I put the document available under the cc-by-sa
               license?
11:58 <@lisa> SwifT: cc-by-sa ?
11:58 <@SwifT> lisa: creative commons attribution - share alike
11:58 <@SwifT> lisa: it's a free documentation license
11:58 <@lisa> sure, go ahead... dont forget to send me the diff
              ok ?
11:58 <@SwifT> sure
11:59 <@SwifT> tx

As a sidenote I also added a comment referring to lisa's viewcvs where she keeps
the master document.

1 <?xml version='1.0' encoding="UTF-8"?>
2 <!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3 <!-- $Header: /home/cvsroot/distcc/html/distcc.xml,v 1.13 2004/01/09 19:56:37 lisa Exp $ -->
4
5 <!--
6 Synchronised with rev 1.14 on
7 http://www.thedoh.com/viewcvs/distcc/html/distcc.xml
8 -->
9
10 <guide link="/doc/en/distcc.xml">
11
12 <title>Gentoo Distcc Documentation</title>
13
14 <author title="Author">
15 <mail link="lisa@gentoo.org">Lisa Seelye</mail>
16 </author>
17
18 <author title="Editor">
19 <mail link="vapier@gentoo.org">Mike Frysinger</mail>
20 </author>
21
22 <author title="Editor">
23 <mail link="erwin@gentoo.org">Erwin</mail>
24 </author>
25
26 <author title="Editor">
27 <mail link="swift@gentoo.org">Sven Vermeulen</mail>
28 </author>
29
30 <author title="Reviewer">
31 <mail link="blubber@gentoo.org">Tiemo Kieft</mail>
32 </author>
33
34 <abstract>
35 This document serves as a HOWTO for using distcc with Gentoo.
36 </abstract>
37
38 <!-- The content of this document is licensed under the CC-BY-SA license -->
39 <!-- See http://creativecommons.org/licenses/by-sa/1.0 -->
40 <license/>
41
42 <version>1.5</version>
43 <date>May 09, 2004</date>
44
45 <chapter>
46 <title>Introduction</title>
47 <section>
48 <title>What is distcc?</title>
49 <body>
50
51 <p>
52 Distcc is a program designed to distribute compiling tasks across a network to
53 participating hosts. It is comprised of a server, <c>distccd</c> and a client
54 program, <c>distcc</c>. Distcc can work transparently with <uri
55 link="http://ccache.samba.org">ccache</uri> and Portage with a little work.
56 </p>
57
58 </body>
59 </section>
60 <section>
61 <title>Using distcc to bootstrap</title>
62 <body>
63
64 <p>
65 If you are planning on using distcc to help you bootstrap a Gentoo installation,
66 make sure you read the part on <uri link="#bootstrapping">Using distcc to
67 Bootstrap</uri> which is situated further down in this document.
68 </p>
69
70 </body>
71 </section>
72 </chapter>
73
74 <chapter>
75 <title>Setup</title>
76 <section>
77 <title>Dependencies</title>
78 <body>
79
80 <p>
81 To start off with, to use Distcc all of the computers on your network need to
82 have the same GCC versions. Mixing 3.3.x (where the x varies) is okay, but
83 mixing 3.3.x and 3.2.x is not: your programs <b>will</b> have errors in it.
84 </p>
85
86 </body>
87 </section>
88 <section>
89 <title>Installing Distcc</title>
90 <body>
91
92 <p>
93 There are a couple of options you should know about before you start
94 installing distcc.
95 </p>
96
97 <p>
98 Distcc ships with a graphical monitor to monitor tasks that your computer is
99 sending away for compilation. If you use Gnome then put 'gnome' in your USE
100 flags. However, if you don't use Gnome and would still like to have the
101 monitor then you should put 'gtk' in your USE flags.
102 </p>
103
104 <pre caption="Installing distcc">
105 # <i>emerge distcc</i>
106 </pre>
107
108 </body>
109 </section>
110 <section>
111 <title>Setting up Portage to use Distcc</title>
112 <body>
113
114 <p>
115 Setting up distcc is easy to do with Portage. Execute the following steps on
116 each system that should participate in the distributed compiling:
117 </p>
118
119 <pre caption="Integrating Distcc and Portage">
120 # <i>emerge distcc</i>
121 # <i>nano -w /etc/make.conf</i>
122 <comment>(Set N to a suitable number for your particular setup)</comment>
123 <comment>(A common heuristic strategy is twice the number of CPUs + 1)</comment>
124 MAKEOPTS="-jN"
125 <comment>(Add distcc to your FEATURES variable)</comment>
126 FEATURES="distcc"
127 <comment>(Set distcc's directory for temporary space)</comment>
128 DISTCC_DIR="${PORTAGE_TMPDIR}/.distcc"
129 </pre>
130
131 <p>
132 Next you have to specify what hosts you want to use. Use the
133 <c>distcc-config</c> command to set the list of hosts. Here is an example of
134 some hosts that might be in your list:
135 </p>
136
137 <pre caption="Examples of host definitions">
138 192.168.0.1 192.168.0.2 192.168.0.3
139 192.168.0.1/2 192.168.0.2 192.168.0.3/10
140 192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4
141 @192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3
142 <comment>(There are also several other methods of setting up hosts. See the
143 distcc manpage for more details.)</comment>
144 </pre>
145
146 <p>
147 It may all look complicated, but in most cases a variant of line 1 or 2 will
148 work.
149 </p>
150
151 <ul>
152 <li>Line 1 is just a space-delimited list of hosts</li>
153 <li>
154 Line 2 is a list of hosts that specifies the maximum amount of jobs (by
155 defining the <c>/N</c> setting) to send that host at any given time
156 </li>
157 </ul>
158
159 <p>
160 Since most people won't be using lines 3 or 4, I'll <uri
161 link="http://distcc.samba.org/man/distcc_1.html">point you to</uri> the distcc
162 docs for more information.
163 </p>
164
165 <p>
166 For instance, to set the first line in the previous example:
167 </p>
168
169 <pre caption="Sample command to set the hosts">
170 # <i>/usr/bin/distcc-config --set-hosts "192.168.0.1 192.168.0.2 192.168.0.3"</i>
171 </pre>
172
173 <p>
174 Now start the distcc daemon on all the participating computers:
175 </p>
176
177 <pre caption="Starting the distcc daemon">
178 <comment>(Add distccd to the default runlevel)</comment>
179 # <i>rc-update add distccd default</i>
180 <comment>(Start the distcc daemon)</comment>
181 # <i>/etc/init.d/distccd start</i>
182 </pre>
183
184 </body>
185 </section>
186 <section>
187 <title>Setting up Distcc to Work With Automake</title>
188 <body>
189
190 <p>
191 This is in some cases easier than the Portage setup. What you have to do is
192 update your <c>PATH</c> variable to include <path>/usr/lib/distcc/bin</path>
193 in front of the directory that contains <c>gcc</c> (<path>/usr/bin</path>).
194 However, there is a caveat. If you use ccache you have to put distcc after
195 the ccache part:
196 </p>
197
198 <pre caption="Setting your path">
199 # <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
200 </pre>
201
202 <p>
203 Then, as you would normally type <c>make</c>, you would type <c>make -jN</c>
204 (where N is an integer). The value of N depends on your network and the types
205 of computers you are using to compile. For me, I have a dual processor P3 and
206 a K6-2/400 that compiles everything for my desktop (a 1200MHz Athlon
207 Thunderbird) and I use -j5. Test your own settings to find the most
208 performant number.
209 </p>
210
211 </body>
212 </section>
213 </chapter>
214
215 <chapter>
216 <title>Cross-Compiling</title>
217 <section>
218 <title>A Note on Cross-Compiling</title>
219 <body>
220
221 <p>
222 Cross-compiling is using one architecture to build programs for another
223 architecture. This can be as simple as using an Athlon (i686) to build a
224 program for a K6-2 (i586), or using a Sparc to build a program for a ppc.
225 </p>
226
227 </body>
228 </section>
229 <section>
230 <title>A Personal Note on Cross-compiling</title>
231 <body>
232
233 <p>
234 I'd love to be able to help out on the Gentoo Cross-compiling effort, but I
235 don't have any non-x86 machines. I can create things that work <e>in
236 theory</e>, but I have to rely on other people to test what I write.
237 That works to an extent, but it's tough. ;-)
238 </p>
239
240 <p>
241 In the near future I hope to acquire a Sparc of some type to put Gentoo on so
242 I can play around in-house.
243 </p>
244
245 </body>
246 </section>
247 </chapter>
248
249 <chapter id="bootstrapping">
250 <title>Using Distcc to Bootstrap</title>
251 <section>
252 <title>Step 1: Configure Portage </title>
253 <body>
254
255 <p>
256 Boot your new box with a Gentoo Linux LiveCD and follow the <uri
257 link="/doc/en/handbook/handbook-x86.xml?part=1">installation instructions</uri>
258 up until the bootstrapping part. Then configure Portage to use distcc:
259 </p>
260
261 <pre caption="Preliminary Setup">
262 # <i>nano -w /etc/make.conf</i>
263 <comment>(Add distcc to the FEATURES variable)</comment>
264 FEATURES="distcc"
265 <comment>(Set distcc's directory for temporary space)</comment>
266 DISTCC_DIR="${PORTAGE_TMPDIR}/.distcc"
267 <comment>(Modify MAKEOPTS to include -jN with N an appropriate integer)</comment>
268 MAKEOPTS="-jN"
269 </pre>
270
271 </body>
272 </section>
273 <section>
274 <title>Step 2: Getting Distcc</title>
275 <body>
276
277 <p>
278 Install distcc:
279 </p>
280
281 <pre caption="Getting Distcc on the new box">
282 # <i>USE='-*' emerge --nodeps distcc</i>
283 </pre>
284
285 </body>
286 </section>
287 <section>
288 <title>Step 3: Setting Up Distcc</title>
289 <body>
290
291 <p>
292 Now run <c>distcc-config</c> to setup distcc:
293 </p>
294
295 <pre caption="Final distcc setup">
296 # <i>/usr/bin/distcc-config --install</i>
297 <comment>(Substitute host1, host2, ... with the IP number(s) of the participating hosts)</comment>
298 # <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i>
299 </pre>
300
301 <p>
302 distcc is now set up to bootstrap! Continue the official installation
303 instructions and <e>don't forget</e> to re-emerge distcc after <c>emerge
304 system</c>. This is to make sure that all of the dependencies you want are
305 installed as well.
306 </p>
307
308 <note>
309 During bootstrap and <c>emerge system</c> Distcc may not appear to be used.
310 This is expected as some ebuilds do not work well with distcc, so they
311 intentionally disable it.
312 </note>
313
314 </body>
315 </section>
316 </chapter>
317
318 <chapter>
319 <title>Troubleshooting</title>
320 <section>
321 <title>Mozilla and Xfree</title>
322 <body>
323
324 <p>
325 As you emerge various packages, you'll notice that some of them aren't being
326 distributed (and aren't being built in parallel). This is because the
327 developers of the Mozilla and Xfree ebuilds intentionally disable parallel
328 building because it is known to cause problems.
329 </p>
330
331 <p>
332 Sometimes distcc might cause a package to fail to compile. If this happens
333 for you, please <uri link="http://bugs.gentoo.org">report</uri> it to us.
334 </p>
335
336 </body>
337 </section>
338 <section>
339 <title>A Mixture of hardened-gcc and non-hardened-gcc Hosts Will Be Faulty</title>
340 <body>
341
342 <p>
343 With such a long title any explanation here is almost irrelevent. However, if
344 you plan to use distcc across hosts that have the <uri
345 link="/proj/en/hardened/etdyn-ssp.xml">PaX/hardened-gcc</uri> and some that do
346 not, you will run into problems.
347 </p>
348
349 <p>
350 The solution requires a little foresight on your part; you have to run
351 <c>hardened-gcc -R</c> on the host that has PaX/hardened-gcc, or you have to
352 enable PaX protections in your kernel and <c>emerge hardened-gcc</c>. Both of
353 which are a good thing to do since for the most part the protections offered
354 by both packages is a good thing and is transparent to the user.
355 </p>
356
357 </body>
358 </section>
359 <section>
360 <title>Mixed GCC Versions</title>
361 <body>
362
363 <p>
364 If you have different GCC versions on your hosts, there will likely be very
365 weird problems. The solution is to make certain all hosts have the same GCC
366 version.
367 </p>
368
369 </body>
370 </section>
371 </chapter>
372
373 <chapter>
374 <title>Distcc Extras</title>
375 <section>
376 <title>Distcc Monitors</title>
377 <body>
378
379 <p>
380 Distcc ships with two monitors. The text-based one is always built and is
381 called <c>distccmon-text</c>. Running it for the first time can be a bit
382 confusing, but it is really quite easy to use. If you run the program with no
383 parameter it will run once. However, if you pass it a number it will update
384 every N seconds, where N is the argument you passed.
385 </p>
386
387 <p>
388 The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c>
389 in your <c>USE</c> flags. This one is GTK+ based, runs in an X environment
390 and it is quite lovely.
391 </p>
392
393 <pre caption="Starting the monitors">
394 # <i>distccmon-text N</i>
395 <comment>(Or ...)</comment>
396 # <i>distccmon-gnome</i>
397 </pre>
398
399 </body>
400 </section>
401 </chapter>
402
403 <chapter>
404 <title>Future Plans for Distcc and Gentoo</title>
405 <section>
406 <title>distcc-subnetscan</title>
407 <body>
408
409 <p>
410 <c>distcc-subnetscan</c> is a perl program in development that will scan a
411 subnet for hosts that have a participating distcc daemon. Even this will be
412 enhanced to test if the daemon on a remote host will conform to a specific
413 <c>CHOST</c> setting to make cross-compiling easier.
414 </p>
415
416 <p>
417 The perl script is being kept at <uri link="http://dev.gentoo.org/~lisa/distcc/distcc-subnetscan/distcc-subnetscan.pl">lisa's
418 dev page</uri> until a more formal home can be found.
419 </p>
420
421 </body>
422 </section>
423 </chapter>
424 </guide>

  ViewVC Help
Powered by ViewVC 1.1.20