/[gentoo]/xml/htdocs/doc/en/handbook/hb-working-features.xml
Gentoo

Contents of /xml/htdocs/doc/en/handbook/hb-working-features.xml

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.40 - (show annotations) (download) (as text)
Sun Nov 27 18:25:28 2011 UTC (2 years, 9 months ago) by swift
Branch: MAIN
Changes since 1.39: +10 -5 lines
File MIME type: application/xml
Portage tree is signed with new key (cfr mail on gentoo-dev-announce)

1 <?xml version='1.0' encoding='UTF-8'?>
2 <!DOCTYPE sections SYSTEM "/dtd/book.dtd">
3
4 <!-- The content of this document is licensed under the CC-BY-SA license -->
5 <!-- See http://creativecommons.org/licenses/by-sa/2.5 -->
6
7 <!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/handbook/hb-working-features.xml,v 1.39 2011/10/27 18:33:42 swift Exp $ -->
8
9 <sections>
10
11 <abstract>
12 Discover the features Portage has, such as support for distributed compiling,
13 ccache and more.
14 </abstract>
15
16 <version>4</version>
17 <date>2011-11-27</date>
18
19 <section>
20 <title>Portage Features</title>
21 <body>
22
23 <p>
24 Portage has several additional features that makes your Gentoo experience even
25 better. Many of these features rely on certain software tools that improve
26 performance, reliability, security, ...
27 </p>
28
29 <p>
30 To enable or disable certain Portage features you need to edit
31 <path>/etc/make.conf</path>'s <c>FEATURES</c> variable which contains the
32 various feature keywords, separated by white space. In several cases you
33 will also need to install the additional tool on which the feature relies.
34 </p>
35
36 <p>
37 Not all features that Portage supports are listed here. For a full overview,
38 please consult the <path>make.conf</path> man page:
39 </p>
40
41 <pre caption="Consulting the make.conf man page">
42 $ <i>man make.conf</i>
43 </pre>
44
45 <p>
46 To find out what FEATURES are default set, run <c>emerge --info</c> and search
47 for the FEATURES variable or grep it out:
48 </p>
49
50 <pre caption="Finding out the FEATURES that are already set">
51 $ <i>emerge --info | grep FEATURES</i>
52 </pre>
53
54 </body>
55 </section>
56 <section>
57 <title>Distributed Compiling</title>
58 <subsection>
59 <title>Using distcc</title>
60 <body>
61
62 <p>
63 <c>distcc</c> is a program to distribute compilations across several, not
64 necessarily identical, machines on a network. The <c>distcc</c> client sends all
65 necessary information to the available distcc servers (running <c>distccd</c>)
66 so they can compile pieces of source code for the client. The net result is a
67 faster compilation time.
68 </p>
69
70 <p>
71 You can find more information about <c>distcc</c> (and how to have it work
72 with Gentoo) in our <uri link="/doc/en/distcc.xml">Gentoo Distcc
73 Documentation</uri>.
74 </p>
75
76 </body>
77 </subsection>
78 <subsection>
79 <title>Installing distcc</title>
80 <body>
81
82 <p>
83 Distcc ships with a graphical monitor to monitor tasks that your computer is
84 sending away for compilation. If you use Gnome then put 'gnome' in your USE
85 variable. However, if you don't use Gnome and would still like to have the
86 monitor then you should put 'gtk' in your USE variable.
87 </p>
88
89 <pre caption="Installing distcc">
90 # <i>emerge distcc</i>
91 </pre>
92
93 </body>
94 </subsection>
95 <subsection>
96 <title>Activating Portage Support</title>
97 <body>
98
99 <p>
100 Add <c>distcc</c> to the FEATURES variable inside <path>/etc/make.conf</path>.
101 Next, edit the MAKEOPTS variable to your liking. A known guideline is to fill in
102 "-jX" with X the number of CPUs that run <c>distccd</c> (including the current
103 host) plus one, but you might have better results with other numbers.
104 </p>
105
106 <p>
107 Now run <c>distcc-config</c> and enter the list of available distcc servers. For
108 a simple example we assume that the available DistCC servers are 192.168.1.102
109 (the current host), 192.168.1.103 and 192.168.1.104 (two "remote" hosts):
110 </p>
111
112 <pre caption="Configuring distcc to use three available distcc servers">
113 # <i>distcc-config --set-hosts "192.168.1.102 192.168.1.103 192.168.1.104"</i>
114 </pre>
115
116 <p>
117 Don't forget to run the <c>distccd</c> daemon as well:
118 </p>
119
120 <pre caption="Starting the distccd daemons">
121 # <i>rc-update add distccd default</i>
122 # <i>/etc/init.d/distccd start</i>
123 </pre>
124
125 </body>
126 </subsection>
127 </section>
128 <section>
129 <title>Caching Compilation</title>
130 <subsection>
131 <title>About ccache</title>
132 <body>
133
134 <p>
135 <c>ccache</c> is a fast compiler cache. When you compile a program, it will
136 cache intermediate results so that, whenever you recompile the same program, the
137 compilation time is greatly reduced. The first time you run ccache, it will be
138 much slower than a normal compilation. Subsequent recompiles should be faster.
139 ccache is only helpful if you will be recompiling the same application many
140 times; thus it's mostly only useful for software developers.
141 </p>
142
143 <p>
144 If you are interested in the ins and outs of ccache, please visit the
145 <uri link="http://ccache.samba.org">ccache homepage</uri>.
146 </p>
147
148 <warn>
149 <c>ccache</c> is known to cause numerous compilation failures. Sometimes ccache
150 will retain stale code objects or corrupted files, which can lead to packages
151 that cannot be emerged. If this happens (if you receive errors like "File not
152 recognized: File truncated"), try recompiling the application with ccache
153 disabled (<c>FEATURES="-ccache"</c> in <path>/etc/make.conf</path>)
154 <e>before</e> reporting a bug. Unless you are doing development work, <e>do not
155 enable ccache</e>.
156 </warn>
157
158 </body>
159 </subsection>
160 <subsection>
161 <title>Installing ccache</title>
162 <body>
163
164 <p>
165 To install <c>ccache</c>, run <c>emerge ccache</c>:
166 </p>
167
168 <pre caption="Installing ccache">
169 # <i>emerge ccache</i>
170 </pre>
171
172 </body>
173 </subsection>
174 <subsection>
175 <title>Activating Portage Support</title>
176 <body>
177
178 <p>
179 Open <path>/etc/make.conf</path> and add <c>ccache</c> to the FEATURES variable.
180 Next, add a new variable called CCACHE_SIZE and set it to "2G":
181 </p>
182
183 <pre caption="Editing CCACHE_SIZE in /etc/make.conf">
184 CCACHE_SIZE="2G"
185 </pre>
186
187 <p>
188 To check if ccache functions, ask ccache to provide you with its statistics.
189 Because Portage uses a different ccache home directory, you need to set the
190 <c>CCACHE_DIR</c> variable as well:
191 </p>
192
193 <pre caption="Viewing ccache statistics">
194 # <i>CCACHE_DIR="/var/tmp/ccache" ccache -s</i>
195 </pre>
196
197 <p>
198 The <path>/var/tmp/ccache</path> location is Portage' default ccache home
199 directory; if you want to alter this setting you can set the <c>CCACHE_DIR</c>
200 variable in <path>/etc/make.conf</path>.
201 </p>
202
203 <p>
204 However, if you would run <c>ccache</c>, it would use the default location of
205 <path>${HOME}/.ccache</path>, which is why you needed to set the
206 <c>CCACHE_DIR</c> variable when asking for the (Portage) ccache statistics.
207 </p>
208
209 </body>
210 </subsection>
211 <subsection>
212 <title>Using ccache for non-Portage C Compiling</title>
213 <body>
214
215 <p>
216 If you would like to use ccache for non-Portage compilations, add
217 <path>/usr/lib/ccache/bin</path> to the beginning of your PATH variable (before
218 <path>/usr/bin</path>). This can be accomplished by editing
219 <path>.bash_profile</path> in your user's home directory. Using
220 <path>.bash_profile</path> is one way to define PATH variables.
221 </p>
222
223 <pre caption="Editing .bash_profile">
224 PATH="<i>/usr/lib/ccache/bin</i>:/opt/bin:${PATH}"
225 </pre>
226
227 </body>
228 </subsection>
229 </section>
230 <section>
231 <title>Binary Package Support</title>
232 <subsection>
233 <title>Creating Prebuilt Packages</title>
234 <body>
235
236 <p>
237 Portage supports the installation of prebuilt packages. Even though Gentoo does
238 not provide prebuilt packages by itself (except for the GRP snapshots) Portage
239 can be made fully aware of prebuilt packages.
240 </p>
241
242 <p>
243 To create a prebuilt package you can use <c>quickpkg</c> if the package is
244 already installed on your system, or <c>emerge</c> with the <c>--buildpkg</c> or
245 <c>--buildpkgonly</c> options.
246 </p>
247
248 <p>
249 If you want Portage to create prebuilt packages of every single package you
250 install, add <c>buildpkg</c> to the FEATURES variable.
251 </p>
252
253 <p>
254 More extended support for creating prebuilt package sets can be obtained with
255 <c>catalyst</c>. For more information on catalyst please read the <uri
256 link="/proj/en/releng/catalyst/faq.xml">Catalyst Frequently Asked
257 Questions</uri>.
258 </p>
259
260 </body>
261 </subsection>
262 <subsection>
263 <title>Installing Prebuilt Packages</title>
264 <body>
265
266 <p>
267 Although Gentoo doesn't provide one, you can create a central repository where
268 you store prebuilt packages. If you want to use this repository, you need to
269 make Portage aware of it by having the PORTAGE_BINHOST variable point to
270 it. For instance, if the prebuilt packages are on ftp://buildhost/gentoo:
271 </p>
272
273 <pre caption="Setting PORTAGE_BINHOST in /etc/make.conf">
274 PORTAGE_BINHOST="ftp://buildhost/gentoo"
275 </pre>
276
277 <p>
278 When you want to install a prebuilt package, add the <c>--getbinpkg</c> option
279 to the emerge command alongside of the <c>--usepkg</c> option. The former tells
280 emerge to download the prebuilt package from the previously defined server
281 while the latter asks emerge to try to install the prebuilt package first before
282 fetching the sources and compiling it.
283 </p>
284
285 <p>
286 For instance, to install <c>gnumeric</c> with prebuilt packages:
287 </p>
288
289 <pre caption="Installing the gnumeric prebuilt package">
290 # <i>emerge --usepkg --getbinpkg gnumeric</i>
291 </pre>
292
293 <p>
294 More information about emerge's prebuilt package options can be found in the
295 emerge man page:
296 </p>
297
298 <pre caption="Reading the emerge man page">
299 $ <i>man emerge</i>
300 </pre>
301
302 </body>
303 </subsection>
304 </section>
305 <section>
306 <title>Fetching Files</title>
307 <subsection>
308 <title>Parallel fetch</title>
309 <body>
310
311 <p>
312 When you are emerging a series of packages, Portage can fetch the source files
313 for the next package in the list even while it is compiling another package,
314 thus shortening compile times. To make use of this capability, add
315 "parallel-fetch" to your FEATURES.
316 </p>
317
318 </body>
319 </subsection>
320 <subsection>
321 <title>Userfetch</title>
322 <body>
323
324 <p>
325 When Portage is run as root, FEATURES="userfetch" will allow Portage to drop
326 root privileges while fetching package sources. This is a small security
327 improvement.
328 </p>
329
330 </body>
331 </subsection>
332 </section>
333 <section id="webrsync-gpg">
334 <title>Pulling Validated Portage Tree Snapshots</title>
335 <body>
336
337 <p>
338 As an administrator, you can opt to only update your local Portage tree with a
339 cryptographically validated Portage tree snapshot as released by the Gentoo
340 infrastructure. This ensures that no rogue rsync mirror is adding unwanted code
341 or packages in the tree you are downloading.
342 </p>
343
344 <p>
345 To configure Portage, first create a truststore in which you download and accept
346 the keys of the Gentoo Infrastructure responsible for signing the Portage tree
347 snapshots. Of course, if you want to, you can validate this GPG key as per the
348 <uri link="/doc/en/gnupg-user.xml#doc_chap2_sect4">proper guidelines</uri>
349 (like checking the key fingerprint <c>AE54 54F9 67B5 6AB0 9AE1 6064 0838
350 C26E 239C 75C4</c> for key <c>0x239C75C4</c> or <c>DCD0 5B71 EAB9 4199 527F 44AC
351 DB6B 8C1F 96D8 BF6D</c> for key <c>0x96D8BF6D</c>).
352 </p>
353
354 <!-- Key 0x239C75C4 is superceded by 0x96D8BF6D, but keep it in the documents
355 for a while to ensure continuity -->
356
357 <pre caption="Creating a truststore for Portage">
358 # <i>mkdir -p /etc/portage/gpg</i>
359 # <i>chmod 0700 /etc/portage/gpg</i>
360 # <i>gpg --homedir /etc/portage/gpg --keyserver subkeys.pgp.net --recv-keys 0x239C75C4 0x96D8BF6D</i>
361 # <i>gpg --homedir /etc/portage/gpg --edit-key 0x239C75C4 trust</i>
362 # <i>gpg --homedir /etc/portage/gpg --edit-key 0x96D8BF6D trust</i>
363 </pre>
364
365 <p>
366 Next, edit <path>/etc/make.conf</path> and enable support for validating the
367 signed Portage tree snapshots (using <c>FEATURES="webrsync-gpg"</c>) and
368 disabling updating the Portage tree using the regular <c>emerge --sync</c>
369 method.
370 </p>
371
372 <pre caption="Updating Portage for signed tree validation">
373 FEATURES="webrsync-gpg"
374 PORTAGE_GPG_DIR="/etc/portage/gpg"
375 SYNC=""
376 </pre>
377
378 <p>
379 That's it. Next time you run <c>emerge-webrsync</c>, only the snapshots with
380 a valid signature will be expanded on your file system.
381 </p>
382
383
384 </body>
385 </section>
386
387 </sections>

  ViewVC Help
Powered by ViewVC 1.1.20