/[gentoo]/xml/htdocs/doc/en/distcc.xml
Gentoo

Diff of /xml/htdocs/doc/en/distcc.xml

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

Revision 1.7 Revision 1.8
1<?xml version='1.0' encoding="UTF-8"?> 1<?xml version='1.0' encoding="UTF-8"?>
2<!DOCTYPE guide SYSTEM "/dtd/guide.dtd"> 2<!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/distcc.xml,v 1.7 2003/10/09 13:36:14 swift Exp $ --> 3<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/distcc.xml,v 1.8 2003/10/11 17:21:58 swift Exp $ -->
4 4
5<guide link="/doc/en/distcc.xml"> 5<guide link="/doc/en/distcc.xml">
6 6
7<title>Gentoo Distcc Documentation</title> 7<title>Gentoo Distcc Documentation</title>
8 8
26 <mail link="blubber@gentoo.org">Tiemo Kieft</mail> 26 <mail link="blubber@gentoo.org">Tiemo Kieft</mail>
27</author> 27</author>
28 28
29<license/> 29<license/>
30 30
31<version>1.2.2</version> 31<version>1.2.3</version>
32<date>October 9, 2003</date> 32<date>October 11, 2003</date>
33 33
34<abstract>This document serves as a HOWTO for using distcc with Gentoo.</abstract> 34<abstract>This document serves as a HOWTO for using distcc with Gentoo.</abstract>
35 35
36<chapter> 36<chapter>
37 <title>Introduction</title> 37 <title>Introduction</title>
39 <title>What is distcc?</title> 39 <title>What is distcc?</title>
40 <body> 40 <body>
41 <p>Distcc is a program designed to distribute compiling tasks across a network to participating hosts. It is comprised of a server, <c>distccd</c> and a client program, <c>distcc</c>. Distcc can work transparently with <uri link="http://ccache.samba.org">ccache</uri> and Portage with a little set up.</p> 41 <p>Distcc is a program designed to distribute compiling tasks across a network to participating hosts. It is comprised of a server, <c>distccd</c> and a client program, <c>distcc</c>. Distcc can work transparently with <uri link="http://ccache.samba.org">ccache</uri> and Portage with a little set up.</p>
42 </body> 42 </body>
43 </section> 43 </section>
44 44</chapter>
45
46<chapter>
47 <title>Setup</title>
45 <section> 48 <section>
46 <title>Dependencies</title> 49 <title>Dependencies</title>
47 <body> 50 <body>
48 <pre caption="Distcc dependencies"> 51 <pre caption="Distcc dependencies (2.8 through 2.10)">
49&gt;=sys-apps/portage-2.0.46-r11 52&gt;=sys-apps/portage-2.0.46-r11
53<codenote>Distcc-2.11 and on require &gt;=sys-apps/portage-2.0.49-r6</codenote>
50&gt;=sys-devel/gcc-config-1.3.1 54&gt;=sys-devel/gcc-config-1.3.1
51sys-apps/shadow 55sys-apps/shadow
52<codenote>Distcc has the following optional dependencies when you have gtk in your USE flags (As of version 2.8)</codenote> 56<codenote>(As of version 2.8 and up until 2.11) And the following optional dependencies when you have <i>gtk</i> in your <i>USE</i> flags</codenote>
53&gt;=x11-libs/gtk+-2.2.1 57&gt;=x11-libs/gtk+-2.2.1
54 </pre> 58</pre>
59 <pre caption="Distcc Dependencies (post 2.11.1)">
60&gt;=sys-apps/portage-2.0.49-r6
61&gt;=sys-devel/gcc-config-1.3.1
62sys-apps/shadow
63<codenote>In this revision you may chose between a Gnome and GTK GUI monitor, they have the following added dependencies</codenote>
64<codenote>For GTK:</codenote>
65&gt;=x11-libs/gtk+-2.0.0
66&gt;=gnome-base/libglade-2.0.0
67x11-libs/pango
68<codenote>For Gnome:</codenote>
69&gt;=x11-libs/gtk+-2.0.0
70&gt;=gnome-base/libglade-2.0.0
71x11-libs/pango
72&gt;=gnome-base/libgnomeui-2.0.0.0
73&gt;=gnome-base/libgnome-2.0.0
74</pre>
75 </body>
76 </section>
77 <section>
78 <title>Installing Distcc</title>
79 <body>
80 <p>Installing Distcc is very easy. Simply set your USE flags and <i>emerge distcc</i>. But, there's a couple of options you should know about.</p>
81 <p>Distcc ships with a graphical monitor to monitor tasks that your computer is sending away for compilation. If you use Gnome then put 'gnome' in your USE flags. However, if you don't use Gnome and would still like to have the monitor then you should put 'gtk' in your USE flags.</p>
55 </body> 82 </body>
56 </section> 83 </section>
57</chapter>
58
59<chapter>
60 <title>Setup</title>
61 <section> 84 <section>
62 <title>Setting up Portage to use Distcc</title> 85 <title>Setting up Portage to use Distcc</title>
63 <body> 86 <body>
64 <p>Setting up distcc is very easy to do with Portage. Follow these simple steps on each computer you want to use distcc on:</p> 87 <p>Setting up distcc is very easy to do with Portage. Follow these simple steps on each computer you want to use distcc on:</p>
65 <pre caption="Integrating Distcc and Portage"> 88 <pre caption="Integrating Distcc and Portage">
66# <i>emerge distcc</i> 89# <i>emerge distcc</i>
67# <i>nano -w /etc/make.conf</i> 90# <i>nano -w /etc/make.conf</i>
68</pre> 91</pre>
92 <impo>If you use distcc-2.11.1 or greater with &gt;=portage-2.0.49-r6, then all you need to do as far as setup is to set your hosts (see next code block), modify MAKEOPTS in <path>/etc/make.conf</path>, and add 'distcc' to FEATURES in <path>/etc/make.conf</path>.</impo>
69 <note>Edit your FEATURES flags to include &quot;distcc&quot;</note> 93 <note>Edit your FEATURES flags to include &quot;distcc&quot;</note>
70 <note>Set <c>DISTCC_DIR=${PORTAGE_TMPDIR}/portage/.distcc</c></note>
71 <note>You should also uncomment the DISTCC_TMPDIR line in <path>/etc/conf.d/distccd</path>.</note> 94 <note>You should also uncomment the DISTCC_TMPDIR line in <path>/etc/make.conf</path>.</note>
72 95 <note>Set <i>DISTCC_DIR=${PORTAGE_TMPDIR}/portage/.distcc</i> near the end of the file (last line is okay)</note>
73 96
74 <p>Next you have to specify what hosts you want to use. To do this you can use the <c>distcc-config</c> command to set the list of hosts. Here is an example of some hosts that might be in your list:</p> 97 <p>Next you have to specify what hosts you want to use. To do this you can use the <c>distcc-config</c> command to set the list of hosts. Here is an example of some hosts that might be in your list:</p>
75 <pre caption="Examples of host definitions"> 98 <pre caption="Examples of host definitions">
76192.168.0.1 192.168.0.2 192.168.0.3 99192.168.0.1 192.168.0.2 192.168.0.3
77192.168.0.1/2 192.168.0.2 192.168.0.3/10 100192.168.0.1/2 192.168.0.2 192.168.0.3/10
78192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4 101192.168.0.1:4000/2 192.168.0.2/1 192.168.0.3:3632/4
79@192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3 102@192.168.0.1 @192.168.0.2:/usr/bin/distccd 192.168.0.3
80 </pre> 103</pre>
81 <p>It may all look complicated, but in most cases a variant of line 1 or 2 will work. An explanation of each line is: Line 1 is just a space-delimited list of hosts that will use default everything. Line 2 is a list of hosts that specifies the maximum number of jobs (by use of the /N) to send that host at any given time (specified with the <c>/n</c>). Since most people won't be using lines 3 or 4, I'll <uri link="http://distcc.samba.org/man/distcc_1.html">point you to</uri> the distcc docs for more information.</p> 104 <p>It may all look complicated, but in most cases a variant of line 1 or 2 will work. An explanation of each line is: Line 1 is just a space-delimited list of hosts that will use default everything. Line 2 is a list of hosts that specifies the maximum number of jobs (by use of the /N) to send that host at any given time (specified with the <c>/n</c>). Since most people won't be using lines 3 or 4, I'll <uri link="http://distcc.samba.org/man/distcc_1.html">point you to</uri> the distcc docs for more information.</p>
82 <p>A sample command to set the hosts (for line 1) is:</p> 105 <p>A sample command to set the hosts (for line 1) is:</p>
83 <pre caption="Sample command to set the hosts"> 106 <pre caption="Sample command to set the hosts">
84# <i>/usr/bin/distcc-config --set-hosts "192.168.0.1 192.168.0.2 192.168.0.3"</i> 107# <i>/usr/bin/distcc-config --set-hosts "192.168.0.1 192.168.0.2 192.168.0.3"</i>
85 </pre> 108</pre>
86 109
87 <p>The final step to integrating distcc into Portage is to re-open your <path>/etc/make.conf</path> and edit <c>MAKEOPTS</c> to include <c>-jN</c> (where N is an integer). Typically you will want to set this to the total number of processors in your network plus one.</p> 110 <p>The final step to integrating distcc into Portage is to re-open your <path>/etc/make.conf</path> and edit <c>MAKEOPTS</c> to include <c>-jN</c> (where N is an integer). Typically you will want to set this to the total number of processors in your network plus one.</p>
88 <pre caption="Final steps in make.conf"> 111 <pre caption="Final steps in make.conf">
89# <i>nano -w /etc/make.conf</i> 112# <i>nano -w /etc/make.conf</i>
90MAKEOPTS=-jN 113MAKEOPTS=-jN
91 </pre> 114</pre>
92 <impo>Don't forget to start the distcc daemon! <c>/etc/init.d/distccd start</c></impo> 115 <impo>Don't forget to start the distcc daemon! <c>/etc/init.d/distccd start</c></impo>
93 </body> 116 </body>
94 </section> 117 </section>
95 <section> 118 <section>
96 <title>Setting up Distcc to Work With Automake</title> 119 <title>Setting up Distcc to Work With Automake</title>
97 <body> 120 <body>
98 <p>This is in some cases easier than the Portage setup. What you have to do is update your <c>PATH</c> environment variable to include <path>/usr/lib/distcc/bin</path> before where gcc (<path>/usr/bin</path>). However, there is a caveat. If you use ccache you have to put distcc after the ccache part. Like so:</p> 121 <p>This is in some cases easier than the Portage setup. What you have to do is update your <c>PATH</c> environmental variable to include <path>/usr/lib/distcc/bin</path> before where gcc (<path>/usr/bin</path>). However, there is a caveat. If you use ccache you have to put distcc after the ccache part. Like so:</p>
99 <pre caption="Setting your path"> 122 <pre caption="Setting your path">
100# <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i> 123# <i>export PATH="/usr/lib/ccache/bin:/usr/lib/distcc/bin:${PATH}"</i>
101 </pre> 124</pre>
102 <p>Then, as you would normally type <c>make</c>, you would type <c>make -jN</c> (where N is an integer). A safe number to use for N varries on your network and the types of computers you are using to compile. For me, I have a dual processor P3 and a K6-2/400 that compile everything for my desktop (a 1200MHz Athlon Thunderbird) and I use -j5. Test your own settings to find the right number.</p> 125 <p>Then, as you would normally type <c>make</c>, you would type <c>make -jN</c> (where N is an integer). A safe number to use for N varies on your network and the types of computers you are using to compile. For me, I have a dual processor P3 and a K6-2/400 that compile everything for my desktop (a 1200MHz Athlon Thunderbird) and I use -j5. Test your own settings to find the right number.</p>
103 </body> 126 </body>
104 </section> 127 </section>
105</chapter> 128</chapter>
106 129
107<chapter> 130<chapter>
122</chapter> 145</chapter>
123 146
124<chapter> 147<chapter>
125 <title>Using Distcc to Bootstrap</title> 148 <title>Using Distcc to Bootstrap</title>
126 <section> 149 <section>
127 <title>Step 1: Build the Tarball</title> 150 <title>Step 1: Setup on New Box</title>
128 <body>
129 <p>This section is helpful if you want to use Distcc during the Gentoo installation. Surprisingly it isn't that hard. What you'll need is another computer running Gentoo and a way to transfer a tarball to your new system (I use scp).</p>
130 <p>First you have to build a tarball of Distcc and include a library (<path>/usr/lib/libpopt.so.*</path>).</p>
131 <pre caption="Building the tarball">
132# <i>USE='-gtk -selinux' emerge --buildpkg distcc</i>
133# <i>mkdir -p /tmp/distcc/usr/lib</i>
134# <i>cp /usr/portage/packages/sys-devel/distcc-DISTCC_VERSION.tbz2 /tmp/distcc/</i>
135# <i>cp /usr/lib/libpopt.so.* /tmp/distcc/usr/lib</i>
136# <i>cd /tmp/distcc</i>
137# <i>tar cfzv distcc.tar.gz *</i>
138<codenote>Be sure you replace DISTCC_VERSION with the version (and Gentoo revision) of Distcc, <i>like 2.10-r1</i></codenote>
139 </pre>
140 </body> 151 <body>
141 </section>
142 <section>
143 <title>Step 2: Setup on New Box</title>
144 <body>
145 <impo>Do not forget to start sshd and reset the root password for the LiveCD!</impo>
146 <p>Next, you have to boot your new box with a Gentoo Linux LiveCD and follow all of the steps up until the bootstrapping. Then proceed with a little preliminary setup on the new box.</p> 152 <p>Next, you have to boot your new box with a Gentoo Linux LiveCD and follow all of the steps up until the bootstrapping. Then proceed with a little preliminary setup on the new box.</p>
147 <pre caption="Preliminary Setup"> 153 <pre caption="Preliminary Setup">
148<codenote>Here we add distcc to FEATURES</codenote> 154<codenote>Here we add distcc to FEATURES</codenote>
149# <i>echo "FEATURES=\"\${FEATURES} distcc\"" &gt;&gt; /etc/make.conf</i> 155# <i>nano -w /etc/make.conf</i>
150<codenote>You can also use <i>nano</i> to edit <path>/etc/make.conf</path> and manually add distcc to FEATURES.</codenote> 156<codenote>Add &quot;distcc&quot; to FEATURES</codenote>
151# <i>echo "MAKEOPTS=\"\${MAKEOPTS} -jN\"" &gt;&gt; /etc/make.conf</i> 157<codenote>Modify MAKEOPTS in <path>/etc/make.conf to include -jN (as described above)</path></codenote>
152<codenote>Just like above, you can use <i>nano</i> to edit <path>/etc/make.conf</path> and manually change MAKEOPTS.</codenote>
153 </pre> 158</pre>
154 <p>Then add the distcc user to your <path>/etc/passwd</path>:</p> 159 <p>Then add the distcc user to your <path>/etc/passwd</path>:</p>
155 <pre caption="Add the distcc user to /etc/passwd"> 160 <pre caption="Add the distcc user to /etc/passwd">
156# <i>echo "distcc:x:240:2:distccd:/dev/null:/bin/false" &gt;&gt;/etc/passwd</i> 161# <i>echo "distcc:x:240:2:distccd:/dev/null:/bin/false" &gt;&gt;/etc/passwd</i>
157<codenote>Do not forget the `<i>&gt;&gt;</i>'</codenote> 162<codenote>Do not forget the `<i>&gt;&gt;</i>'</codenote>
158 </pre> 163</pre>
159 </body> 164 </body>
160 </section>
161 <section> 165 </section>
162 <title>Step 3: Copy the Tarball</title>
163 <body>
164 <p>Next you will want to copy the distcc tarball that you made.</p>
165 <pre caption="Copy the tarball to the new box">
166<codenote>Execute this on the "old" box</codenote>
167# <i>scp /tmp/distcc/distcc.tar.gz root@ip.of.new.box:/mnt/gentoo/</i>
168<codenote>Substitute <i>ip.of.new.box</i> for your new box's IP</codenote>
169 </pre>
170 </body>
171 </section> 166 <section>
172 167 <title>Step 2: Getting Distcc</title>
168 <body>
169 <p>Next you will want to issue the following command inside the chroot on the new box.</p>
170 <pre caption="Getting Distcc on the new box">
171# <i>emerge --nodeps distcc</i>
172</pre>
173 </body>
173 <section> 174 </section>
174 <title>Step 4: Unpacking the Tarball</title>
175 <body>
176 <pre caption="Unpack the tarball">
177# <i>tar xvfz distcc.tar.gz</i>
178# <i>tar xvfjp distcc-DISTCC_VERSION.tbz2</i>
179<codenote>Be sure you replace DISTCC_VERSION with the version (and Gentoo revision) of Distcc, <i>like 2.10-r1</i></codenote>
180 </pre>
181 </body>
182 </section> 175 <section>
183
184 <section>
185 <title>Step 5: Setting Up Distcc Itself</title> 176 <title>Step 3: Setting Up Distcc Itself</title>
186 <body> 177 <body>
187 <p>Now set up distcc itself...</p> 178 <p>Now set up distcc itself...</p>
188 <pre caption="Final distcc setup"> 179 <pre caption="Final distcc setup">
189# <i>/usr/bin/distcc-config --install</i> 180# <i>/usr/bin/distcc-config --install</i>
190# <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i> 181# <i>/usr/bin/distcc-config --set-hosts "localhost host1 host2 host3 ..."</i>
191 </pre> 182</pre>
192 <p>distcc should be set up now to bootstrap! Continue the official install guide, and don't forget to re-emerge distcc after <c>emerge system</c>.</p> 183 <p>distcc should be set up now to bootstrap! Continue the official install guide, and don't forget to re-emerge distcc after <c>emerge system</c>.</p>
193 </body> 184 </body>
194 </section> 185 </section>
195</chapter> 186</chapter>
196 187
197<chapter> 188<chapter>
198 <title>Known Problems With Distcc</title> 189 <title>Troubleshooting</title>
199 <section> 190 <section>
200 <title>Mozilla and Xfree</title> 191 <title>Mozilla and Xfree</title>
201 <body> 192 <body>
202 <p>As you emerge various packages, you'll notice that they aren't being distributed (and indeed aren't being built in parallel). This is because the developers of Mozilla and Xfree .ebuilds disabled parallel building because it is known to cause problems. This isn't necessarily a distcc problem.</p> 193 <p>As you emerge various packages, you'll notice that they aren't being distributed (and indeed aren't being built in parallel). This is because the developers of Mozilla and Xfree .ebuilds disabled parallel building because it is known to cause problems. This isn't necessarily a distcc problem.</p>
203 <p>That isn't to say that sometimes distcc will cause a package to fail to compile.</p> 194 <p>That isn't to say that sometimes distcc will cause a package to fail to compile.</p>
222 <title>Distcc Extras</title> 213 <title>Distcc Extras</title>
223 <section> 214 <section>
224 <title>Distcc Monitors</title> 215 <title>Distcc Monitors</title>
225 <body> 216 <body>
226 <p>Distcc ships with two monitors. The text-based one is always built, it is called <c>distccmon-text</c>. Running it for the first time can be a bit confusing, but it is really quite easy to use. If you run the program with no parameter it will run once. However, if you pass it a number it will update every N seconds, where N is the argument you passed.</p> 217 <p>Distcc ships with two monitors. The text-based one is always built, it is called <c>distccmon-text</c>. Running it for the first time can be a bit confusing, but it is really quite easy to use. If you run the program with no parameter it will run once. However, if you pass it a number it will update every N seconds, where N is the argument you passed.</p>
227 <p>The other monitor is only turned on if you enabled <c>gtk</c> in your <c>USE</c> flags. This one is GTK+ based and runs in an X environment, and it is quite lovely.</p> 218 <p>The other monitor is only turned on if you enabled <c>gtk</c> or <c>gnome</c> in your <c>USE</c> flags. This one is GTK+ based and runs in an X environment, and it is quite lovely.</p>
228 <p>There is a caveat to using these programs. If you want to monitor any emerge, you have to start the monitor like so:</p> 219 <p>There is a caveat to using these programs. If you want to monitor any emerge, you have to start the monitor like so:</p>
229 <pre caption="Starting the monitors"> 220 <pre caption="Starting the monitors">
230# <i>DISTCC_DIR=/path/to/distccdir portage distccmon-text N</i> 221# <i>DISTCC_DIR=/path/to/distccdir distccmon-text N</i>
231<codenote>Or you can start the graphical monitor...</codenote> 222<codenote>Or you can start the graphical monitor...</codenote>
232# <i>DISTCC_DIR=/path/to/distccdir portage distccmon-gnome</i> 223# <i>DISTCC_DIR=/path/to/distccdir distccmon-gnome</i>
233 </pre> 224</pre>
225 <impo>If you are using Portage and used the above example, this will be <c>DISTCC_DIR=/var/tmp/portage/.distcc</c>.</impo>
234 <note>If you aren't using Portage you do not need to specify DISTCC_DIR (i.e., just type distccmon-text N).</note> 226 <note>If you aren't using Portage you do not need to specify DISTCC_DIR (i.e., just type distccmon-text N).</note>
235 <impo>If you are using Portage and used the above example, this will be <c>DISTCC_DIR=/var/tmp/portage/.distcc</c>.</impo>
236 </body> 227 </body>
237 </section> 228 </section>
238</chapter> 229</chapter>
239 230
240<chapter> 231<chapter>

Legend:
Removed from v.1.7  
changed lines
  Added in v.1.8

  ViewVC Help
Powered by ViewVC 1.1.20