en/handbook/hb-working-features.xml

<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE sections SYSTEM "/dtd/book.dtd">

<!-- The content of this document is licensed under the CC-BY-SA license -->
<!-- See http://creativecommons.org/licenses/by-sa/1.0 -->

<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/handbook/hb-working-features.xml,v 1.11 2004/03/01 05:43:41 swift Exp $ -->

<sections>
<section>
<title>DistCC</title>
<subsection>
<title>What is DistCC?</title>
<body>

<p>
<c>distcc</c> is a program to distribute compilations across several, not 
necessarily identical, machines on a network. The <c>distcc</c> client sends all
necessary information to the available DistCC servers (running <c>distccd</c>)
so they can compile pieces of source code for the client. The net result is a
faster compilation time.
</p>

<p>
You can find more elaborate information about <c>distcc</c> (and information on
how to have it work with Gentoo) in our <uri link="/doc/en/distcc.xml">Gentoo
Distcc Documentation</uri>.
</p>

</body>
</subsection>
<subsection>
<title>Installing DistCC</title>
<body>

<p>
Distcc ships with a graphical monitor to monitor tasks that your computer is
sending away for compilation. If you use Gnome then put 'gnome' in your
<c>USE</c> setting. However, if you don't use Gnome and would still like to have
the monitor then you should put 'gtk' in your <c>USE</c> setting.
</p>

<p>
Installing distcc is, as is with all software available through Gentoo's
Portage, extremely easy:
</p>

<note>
From now on, as you now know how to install binary packages if you want, we will
omit the <c>--usepkg</c> option throughout the rest of the Gentoo Handbook.
</note>

<pre caption="Installing Distcc">
# <i>emerge distcc</i>
</pre>

<p>
Now setup distcc to use 2G of caching space:
</p>

<pre caption="Setting up distcc">
# <i>ccache -M 2G</i>
</pre>

</body>
</subsection>
<subsection>
<title>Activating Portage Support</title>
<body>

<p>
Well, if installation is easy, the rest should be easy too :) So let us quickly
activate the Portage support for <c>distcc</c>.
</p>

<p>
First, open <path>/etc/make.conf</path> and edit the <c>FEATURES</c> variable so
it contains the <c>distcc</c> keyword. Next, edit the <c>MAKEOPTS</c> variable
so it reads <c>-jX</c> with <c>X</c> the number of CPUs that run <c>distccd</c>
(including the current host) plus one:
</p>

<pre caption="Possible MAKEOPTS setting in /etc/make.conf">
<comment># Suppose you have 2 single-CPU distccd hosts excluding this host:</comment>
MAKEOPTS="-j4"
</pre>

<p>
Now, still inside <path>/etc/make.conf</path>, uncomment the
<c>PORTAGE_TMPDIR</c> line and add the following line at the end
of the file:
</p>

<pre caption="Add an extra, distcc-specific variable to /etc/make.conf">
<comment># Don't forget to uncomment the PORTAGE_TMPDIR variable</comment>
DISTCC_DIR=${PORTAGE_TMPDIR}/portage/.distcc
</pre>

<p>
Now run <c>distcc-config</c> and enter the list of available DistCC servers. For
a simple example we assume that the available DistCC servers are
<c>192.168.1.102</c> (the current host), <c>192.168.1.103</c> and 
<c>192.168.1.104</c> (two "remote" hosts):
</p>

<pre caption="Configuring distcc to use three available DistCC servers">
# <i>distcc-config --set-hosts "192.168.1.102 192.168.1.103 192.168.1.104"</i>
</pre>

<p>
Of course, don't forget to run the <c>distccd</c> daemon too:
</p>

<pre caption="Starting the distcc daemon">
# <i>/etc/init.d/distccd start</i>
</pre>

<p>
Congratulations, your system will now use distributed compiling! For more
in-depth information about DistCC and Gentoo, please read our <uri
link="/doc/en/distcc.xml">Gentoo DistCC Documentation</uri>.
</p>

</body>
</subsection>
</section>
<section>
<title>ccache</title>
<subsection>
<title>What is ccache?</title>
<body>

<p>
<c>ccache</c> is a fast compiler cache. When you compile a program, it will
cache intermediate results so that, when you ever recompile the same program,
the compilation time is greatly reduced. In common compilations this can result
in 5 to 10 times faster compilation times.
</p>

<p>
If you are interested in the ins and outs of <c>ccache</c>, please visit the
<uri link="http://ccache.samba.org">ccache homepage</uri>.
</p>

</body>
</subsection>
<subsection>
<title>Installing ccache</title>
<body>

<p>
Installing <c>ccache</c> with Gentoo is a breeze. Just emerge it and you're done
:)
</p>

<pre caption="Installing ccache">
# <i>emerge ccache</i>
</pre>

</body>
</subsection>
<subsection>
<title>Tweaking ccache</title>
<body>

<p>
Portage will automatically use ccache if it's installed. However, you can alter
the default cache size used by ccache by providing a <c>CCACHE_SIZE</c> variable
in <path>/etc/make.conf</path>:
</p>

<pre caption="Editing CCACHE_SIZE in /etc/make.conf">
CCACHE_SIZE="2G"
</pre>

<p>
As of now, Portage will use <c>ccache</c> to speed up compilations where
possible. If you are uncertain that <c>ccache</c> works, you can run <c>ccache
-s</c> to view the <c>ccache</c> statistics:
</p>

<pre caption="Viewing ccache statistics">
# <i>ccache -s</i>
</pre>

</body>
</subsection>
<subsection>
<title>Using ccache for non-Portage C compiling</title>
<body>

<p>
If you would like to use ccache for non-Portage compilations, add
<path>/usr/lib/ccache/bin</path> to the beginning of your <c>PATH</c> variable
(before <path>/usr/bin</path>). This can be accomplished by editing
<path>/etc/env.d/00basic</path>:
</p>

<pre caption="Editing 00basic">
# <i>nano -w /etc/env.d/00basic</i>
# <i>env-update &amp;&amp; source /etc/profile</i>
</pre>

</body>
</subsection>
</section>
<section>
<title>Binary Packages</title>
<subsection>
<title>Creating binary packages</title>
<body>

<p>
We have already discussed how to work with prebuilt packages, but how do you
create your own prebuilt packages?
</p>

<p>
If the package is already installed, you can use the <c>quickpkg</c> command
which will make a tarball of the installed files. This is very interesting for
backup purposes!
</p>

<pre caption="Using quickpkg">
# <i>quickpkg gnumeric</i>
</pre>

<p>
If the package isn't installed yet, you can install it using <c>emerge</c> and
ask to build a binary package too. <c>emerge</c> uses the <c>--buildpkg</c>
option (<c>-b</c> in short) for this:
</p>

<pre caption="Installing gnumeric and building binary packages too">
# <i>emerge --buildpkg gnumeric</i>
</pre>

<p>
If you want Portage to do this by default, you should set the <c>buildpkg</c>
keyword in the <c>FEATURES</c> variable declared in <path>/etc/make.conf</path>.
</p>

<pre caption="Automatically creating binary packages">
FEATURES="buildpkg"
</pre>

<p>
If you don't want to install the software, but only build the package, you can
use the <c>--buildpkgonly</c> option (<c>-B</c> in short):
</p>

<pre caption="Building a binary package for gnumeric">
# <i>emerge --buildpkgonly gnumeric</i>
</pre>

</body>
</subsection>
</section>
<section>
<title>Security Related Features</title>
<subsection>
<title>Sandbox</title>
<body>

<p>
While building and installing packages, Portage uses a <e>sandbox</e> to protect
your live system. This means that, as long as the package isn't installed on
your system, the package cannot touch any file outside the sandbox. This 
ensures that Portage knows what files are created and modified by a package.
</p>

<p>
When the package compilation is finished, Portage will "preinstall" the package
in the sandbox, registering what files are placed and where. It will then move
those files from the sandbox on your live system.
</p>

</body>
</subsection>
<subsection>
<title>User Privileges</title>
<body>

<p>
Portage also supports building packages as non-root user (more precisely, as
user "portage", group "portage"). This improves the security during the
build process. You can opt to use user privileges with or without sandboxing. Of
course, it goes without saying that user privileges and sandboxing is the most
preferred method :)
</p>

</body>
</subsection>
<subsection>
<title>Activating sandbox and/or userpriv</title>
<body>

<p>
Portage will use <c>sandbox</c> per default. If you want <c>userpriv</c>, you
should add it to the <c>FEATURES</c> variable. Note that activating
<c>userpriv</c> will drop <c>sandbox</c> support, unless you also activate
<c>usersandbox</c>:
</p>

<pre caption="Activating userpriv and usersandbox">
FEATURES="userpriv usersandbox"
</pre>

<warn>
Do <e>not</e> remove <c>sandbox</c> from the <c>FEATURES</c> variable!
</warn>

</body>
</subsection>
<subsection>
<title>Strict Checking</title>
<body>

<p>
Portage can be asked to react strongly to possibly dangerous conditions (such as
missing or incorrect Manifest files). To activate this strict checking, add the
<c>strict</c> keyword to the <c>FEATURES</c> variable:
</p>

<pre caption="Activating strict checking">
FEATURES="strict"
</pre>

</body>
</subsection>
<subsection>
<title>Smart File System Permissions</title>
<body>

<p>
Portage can be told to automatically deal with potentially dangerous file
permissions that could pose a security risk. It does this by removing the
"group" and "other" readable bits on setuid files and removing the "other"
readable bit on setgid files in the pre install phase.  To activate the
smart file permissions, add the <c>sfperms</c> keyword to the
<c>FEATURES</c> variable:
</p>

<pre caption="Activating smart file system permissions">
FEATURES="sfperms"
</pre>

</body>
</subsection>
</section>
<section>
<title>Other Features</title>
<subsection>
<title>Portage Help</title>
<body>

<p>
There are several other keywords you can place in the <c>FEATURES</c> variable.
Most of them are targeted towards developers and less interesting for the casual
user. If you are interested in learning more about these features (or Portage
generally), don't forget to read the <path>make.conf</path> manpage we provide.
</p>

<pre caption="More Portage-related information">
# <i>man make.conf</i>
</pre>

</body>
</subsection>
</section>
</sections>
1	swift	1.10	<?xml version='1.0' encoding='UTF-8'?>
2			<!DOCTYPE sections SYSTEM "/dtd/book.dtd">
3
4	swift	1.1	<!-- The content of this document is licensed under the CC-BY-SA license -->
5			<!-- See http://creativecommons.org/licenses/by-sa/1.0 -->
6
7	swift	1.12	<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/handbook/hb-working-features.xml,v 1.11 2004/03/01 05:43:41 swift Exp $ -->
8	swift	1.1
9			<sections>
10			<section>
11			<title>DistCC</title>
12			<subsection>
13			<title>What is DistCC?</title>
14			<body>
15
16	swift	1.2	<p>
17			<c>distcc</c> is a program to distribute compilations across several, not
18			necessarily identical, machines on a network. The <c>distcc</c> client sends all
19			necessary information to the available DistCC servers (running <c>distccd</c>)
20	swift	1.9	so they can compile pieces of source code for the client. The net result is a
21	swift	1.2	faster compilation time.
22			</p>
23
24			<p>
25			You can find more elaborate information about <c>distcc</c> (and information on
26			how to have it work with Gentoo) in our <uri link="/doc/en/distcc.xml">Gentoo
27			Distcc Documentation</uri>.
28			</p>
29
30	swift	1.1	</body>
31			</subsection>
32			<subsection>
33			<title>Installing DistCC</title>
34			<body>
35
36	swift	1.2	<p>
37			Distcc ships with a graphical monitor to monitor tasks that your computer is
38			sending away for compilation. If you use Gnome then put 'gnome' in your
39			<c>USE</c> setting. However, if you don't use Gnome and would still like to have
40			the monitor then you should put 'gtk' in your <c>USE</c> setting.
41			</p>
42
43			<p>
44			Installing distcc is, as is with all software available through Gentoo's
45			Portage, extremely easy:
46			</p>
47
48			<note>
49			From now on, as you now know how to install binary packages if you want, we will
50			omit the <c>--usepkg</c> option throughout the rest of the Gentoo Handbook.
51			</note>
52
53			<pre caption="Installing Distcc">
54			# <i>emerge distcc</i>
55			</pre>
56
57	swift	1.12	<p>
58			Now setup distcc to use 2G of caching space:
59			</p>
60
61			<pre caption="Setting up distcc">
62			# <i>ccache -M 2G</i>
63			</pre>
64
65	swift	1.1	</body>
66			</subsection>
67			<subsection>
68			<title>Activating Portage Support</title>
69			<body>
70
71	swift	1.2	<p>
72			Well, if installation is easy, the rest should be easy too :) So let us quickly
73			activate the Portage support for <c>distcc</c>.
74			</p>
75
76			<p>
77			First, open <path>/etc/make.conf</path> and edit the <c>FEATURES</c> variable so
78			it contains the <c>distcc</c> keyword. Next, edit the <c>MAKEOPTS</c> variable
79	swift	1.3	so it reads <c>-jX</c> with <c>X</c> the number of CPUs that run <c>distccd</c>
80			(including the current host) plus one:
81	swift	1.2	</p>
82
83			<pre caption="Possible MAKEOPTS setting in /etc/make.conf">
84	swift	1.3	<comment># Suppose you have 2 single-CPU distccd hosts excluding this host:</comment>
85			MAKEOPTS="-j4"
86	swift	1.2	</pre>
87
88			<p>
89			Now, still inside <path>/etc/make.conf</path>, uncomment the
90			<c>PORTAGE_TMPDIR</c> line and add the following line at the end
91			of the file:
92			</p>
93
94			<pre caption="Add an extra, distcc-specific variable to /etc/make.conf">
95			<comment># Don't forget to uncomment the PORTAGE_TMPDIR variable</comment>
96			DISTCC_DIR=${PORTAGE_TMPDIR}/portage/.distcc
97			</pre>
98	swift	1.3
99			<p>
100			Now run <c>distcc-config</c> and enter the list of available DistCC servers. For
101			a simple example we assume that the available DistCC servers are
102			<c>192.168.1.102</c> (the current host), <c>192.168.1.103</c> and
103			<c>192.168.1.104</c> (two "remote" hosts):
104			</p>
105
106			<pre caption="Configuring distcc to use three available DistCC servers">
107			# <i>distcc-config --set-hosts "192.168.1.102 192.168.1.103 192.168.1.104"</i>
108			</pre>
109
110			<p>
111			Of course, don't forget to run the <c>distccd</c> daemon too:
112			</p>
113
114			<pre caption="Starting the distcc daemon">
115			# <i>/etc/init.d/distccd start</i>
116			</pre>
117
118			<p>
119			Congratulations, your system will now use distributed compiling! For more
120			in-depth information about DistCC and Gentoo, please read our <uri
121			link="/doc/en/distcc.xml">Gentoo DistCC Documentation</uri>.
122			</p>
123	swift	1.2
124	swift	1.1	</body>
125			</subsection>
126			</section>
127			<section>
128	swift	1.4	<title>ccache</title>
129	swift	1.1	<subsection>
130	swift	1.4	<title>What is ccache?</title>
131	swift	1.1	<body>
132
133	swift	1.4	<p>
134			<c>ccache</c> is a fast compiler cache. When you compile a program, it will
135			cache intermediate results so that, when you ever recompile the same program,
136			the compilation time is greatly reduced. In common compilations this can result
137			in 5 to 10 times faster compilation times.
138			</p>
139
140			<p>
141			If you are interested in the ins and outs of <c>ccache</c>, please visit the
142			<uri link="http://ccache.samba.org">ccache homepage</uri>.
143			</p>
144
145	swift	1.1	</body>
146			</subsection>
147			<subsection>
148	swift	1.4	<title>Installing ccache</title>
149	swift	1.1	<body>
150
151	swift	1.4	<p>
152			Installing <c>ccache</c> with Gentoo is a breeze. Just emerge it and you're done
153			:)
154			</p>
155
156			<pre caption="Installing ccache">
157			# <i>emerge ccache</i>
158			</pre>
159
160	swift	1.1	</body>
161			</subsection>
162			<subsection>
163	swift	1.12	<title>Tweaking ccache</title>
164	swift	1.1	<body>
165	swift	1.4
166			<p>
167	swift	1.12	Portage will automatically use ccache if it's installed. However, you can alter
168			the default cache size used by ccache by providing a <c>CCACHE_SIZE</c> variable
169			in <path>/etc/make.conf</path>:
170	swift	1.4	</p>
171
172			<pre caption="Editing CCACHE_SIZE in /etc/make.conf">
173			CCACHE_SIZE="2G"
174			</pre>
175
176			<p>
177			As of now, Portage will use <c>ccache</c> to speed up compilations where
178			possible. If you are uncertain that <c>ccache</c> works, you can run <c>ccache
179			-s</c> to view the <c>ccache</c> statistics:
180			</p>
181
182			<pre caption="Viewing ccache statistics">
183			# <i>ccache -s</i>
184			</pre>
185	swift	1.1
186			</body>
187			</subsection>
188	swift	1.12	<subsection>
189			<title>Using ccache for non-Portage C compiling</title>
190			<body>
191
192			<p>
193			If you would like to use ccache for non-Portage compilations, add
194			<path>/usr/lib/ccache/bin</path> to the beginning of your <c>PATH</c> variable
195			(before <path>/usr/bin</path>). This can be accomplished by editing
196			<path>/etc/env.d/00basic</path>:
197			</p>
198
199			<pre caption="Editing 00basic">
200			# <i>nano -w /etc/env.d/00basic</i>
201			# <i>env-update && source /etc/profile</i>
202			</pre>
203
204			</body>
205			</subsection>
206	swift	1.1	</section>
207			<section>
208			<title>Binary Packages</title>
209			<subsection>
210			<title>Creating binary packages</title>
211			<body>
212
213	swift	1.2	<p>
214	swift	1.8	We have already discussed how to work with prebuilt packages, but how do you
215			create your own prebuilt packages?
216	swift	1.2	</p>
217
218			<p>
219			If the package is already installed, you can use the <c>quickpkg</c> command
220			which will make a tarball of the installed files. This is very interesting for
221			backup purposes!
222			</p>
223
224			<pre caption="Using quickpkg">
225			# <i>quickpkg gnumeric</i>
226			</pre>
227
228			<p>
229			If the package isn't installed yet, you can install it using <c>emerge</c> and
230			ask to build a binary package too. <c>emerge</c> uses the <c>--buildpkg</c>
231			option (<c>-b</c> in short) for this:
232			</p>
233
234			<pre caption="Installing gnumeric and building binary packages too">
235			# <i>emerge --buildpkg gnumeric</i>
236			</pre>
237
238			<p>
239			If you want Portage to do this by default, you should set the <c>buildpkg</c>
240			keyword in the <c>FEATURES</c> variable declared in <path>/etc/make.conf</path>.
241			</p>
242	swift	1.6
243			<pre caption="Automatically creating binary packages">
244			FEATURES="buildpkg"
245			</pre>
246	swift	1.2
247			<p>
248			If you don't want to install the software, but only build the package, you can
249			use the <c>--buildpkgonly</c> option (<c>-B</c> in short):
250			</p>
251
252			<pre caption="Building a binary package for gnumeric">
253			# <i>emerge --buildpkgonly gnumeric</i>
254	swift	1.5	</pre>
255
256			</body>
257			</subsection>
258			</section>
259			<section>
260			<title>Security Related Features</title>
261			<subsection>
262			<title>Sandbox</title>
263			<body>
264
265			<p>
266			While building and installing packages, Portage uses a <e>sandbox</e> to protect
267			your live system. This means that, as long as the package isn't installed on
268			your system, the package cannot touch any file outside the sandbox. This
269			ensures that Portage knows what files are created and modified by a package.
270			</p>
271
272			<p>
273			When the package compilation is finished, Portage will "preinstall" the package
274			in the sandbox, registering what files are placed and where. It will then move
275			those files from the sandbox on your live system.
276			</p>
277
278			</body>
279			</subsection>
280			<subsection>
281			<title>User Privileges</title>
282			<body>
283
284			<p>
285			Portage also supports building packages as non-root user (more precisely, as
286			user "portage", group "portage"). This improves the security during the
287			build process. You can opt to use user privileges with or without sandboxing. Of
288			course, it goes without saying that user privileges and sandboxing is the most
289			preferred method :)
290			</p>
291
292			</body>
293			</subsection>
294			<subsection>
295			<title>Activating sandbox and/or userpriv</title>
296			<body>
297
298			<p>
299			Portage will use <c>sandbox</c> per default. If you want <c>userpriv</c>, you
300			should add it to the <c>FEATURES</c> variable. Note that activating
301			<c>userpriv</c> will drop <c>sandbox</c> support, unless you also activate
302			<c>usersandbox</c>:
303			</p>
304
305			<pre caption="Activating userpriv and usersandbox">
306			FEATURES="userpriv usersandbox"
307			</pre>
308
309			<warn>
310			Do <e>not</e> remove <c>sandbox</c> from the <c>FEATURES</c> variable!
311			</warn>
312
313			</body>
314			</subsection>
315			<subsection>
316			<title>Strict Checking</title>
317			<body>
318
319			<p>
320			Portage can be asked to react strongly to possibly dangerous conditions (such as
321			missing or incorrect Manifest files). To activate this strict checking, add the
322			<c>strict</c> keyword to the <c>FEATURES</c> variable:
323			</p>
324
325			<pre caption="Activating strict checking">
326			FEATURES="strict"
327			</pre>
328
329			</body>
330			</subsection>
331	swift	1.7	<subsection>
332			<title>Smart File System Permissions</title>
333			<body>
334
335			<p>
336			Portage can be told to automatically deal with potentially dangerous file
337			permissions that could pose a security risk. It does this by removing the
338			"group" and "other" readable bits on setuid files and removing the "other"
339			readable bit on setgid files in the pre install phase. To activate the
340			smart file permissions, add the <c>sfperms</c> keyword to the
341			<c>FEATURES</c> variable:
342			</p>
343
344			<pre caption="Activating smart file system permissions">
345			FEATURES="sfperms"
346			</pre>
347
348			</body>
349			</subsection>
350	swift	1.5	</section>
351			<section>
352			<title>Other Features</title>
353			<subsection>
354			<title>Portage Help</title>
355			<body>
356
357			<p>
358			There are several other keywords you can place in the <c>FEATURES</c> variable.
359			Most of them are targeted towards developers and less interesting for the casual
360			user. If you are interested in learning more about these features (or Portage
361			generally), don't forget to read the <path>make.conf</path> manpage we provide.
362			</p>
363
364			<pre caption="More Portage-related information">
365			# <i>man make.conf</i>
366	swift	1.2	</pre>
367	swift	1.1
368			</body>
369			</subsection>
370			</section>
371			</sections>