en/handbook/hb-portage-advanced.xml

<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE sections SYSTEM "/dtd/book.dtd">

<!-- The content of this document is licensed under the CC-BY-SA license -->
<!-- See http://creativecommons.org/licenses/by-sa/2.5 -->

<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/handbook/hb-portage-advanced.xml,v 1.1 2012/01/15 09:55:35 swift Exp $ -->

<sections>

<abstract>
As times goes by, Portage evolves and matures further and further. Additional
features are continuously being put in - many of these are only of use by more
advanced users. This chapter will go into more detail of these specific
features.
</abstract>

<version>2</version>
<date>2012-02-07</date>

<section>
<title>Introduction</title>
<body>

<p>
For most users, the information received thus far is sufficient for all their
Linux operations. But Portage is capable of much more; many of its features are
for advanced users or only applicable in specific corner cases. Still, that
would not be an excuse not to document them.
</p>

<p>
Of course, with lots of flexibility comes a huge list of potential cases. It
will not be possible to document them all here. Instead, we hope to focus on
some generic issues which you can then bend to fit your own needs. If you have
need for more specific tweaks and tips, you might find them on the <uri
link="https://wiki.gentoo.org">Gentoo WiKi</uri> instead.
</p>

<p>
Most, if not all of these additional features can be easily found by digging
through the manual pages that portage provides:
</p>

<pre caption="Reading up on portage man pages">
$ <i>man portage</i>
$ <i>man make.conf</i>
</pre>

<p>
Finally, know that these are advanced features which, if not worked with
correctly, can make debugging and troubleshooting very difficult. Make sure you
mention these if you think you hit a bug and want to open a bugreport.
</p>

</body>
</section>

<section>
<title>Per-Package Environment Variables</title>
<subsection>
<title>Using /etc/portage/env</title>
<body>

<p>
By default, package builds will use the environment variables defined in
<path>/etc/make.conf</path>, such as <c>CFLAGS</c>, <c>MAKEOPTS</c> and more. In
some cases though, you might want to provide different variables for specific
packages. To do so, Portage supports the use of <path>/etc/portage/env</path>
and <path>/etc/portage/package.env</path>.
</p>

<p>
The <path>/etc/portage/package.env</path> file contains the list of packages for
which you want deviating variables as well as a specific identifier that tells
Portage which changes you want. The identifier name you pick yourself, Portage
will look for the variables in the <path>/etc/portage/env/&lt;identifier&gt;</path>
file.
</p>

</body>
</subsection>
<subsection>
<title>Example: Using debugging for specific packages</title>
<body>

<p>
As an example, we enable debugging for the <path>media-video/mplayer</path>
package.
</p>

<p>
First of all, we set the debugging variables in a file called
<path>/etc/portage/env/debug-cflags</path>. The name is arbitrarily chosen, but
of course reflects the reason of the deviation to make it more obvious later why
a deviation was put in.
</p>

<pre caption="/etc/portage/env/debug-cflags content">
CFLAGS="-O2 -ggdb -pipe"
FEATURES="${FEATURES} nostrip"
</pre>

<p>
Next, we tag the <path>media-video/mplayer</path> package to use this content:
</p>

<pre caption="/etc/portage/package.env content">
media-video/mplayer debug-cflags
</pre>

</body>
</subsection>
</section>

<section>
<title>Hooking In the Emerge Process</title>
<subsection>
<title>Using /etc/portage/bashrc and affiliated files</title>
<body>

<p>
When Portage works with ebuilds, it uses a bash environment in which it calls
the various build functions (like src_prepare, src_configure, pkg_postinst,
etc.). But Portage also allows you to set up a bash environment yourself.
</p>

<p>
The advantage of using your own bash environment is that you can hook in the
emerge process during each step it performs. This can be done for every emerge
(through <path>/etc/portage/bashrc</path>) or by using per-package environments
(through <path>/etc/portage/env</path> as discussed earlier).
</p>

<p>
To hook in the process, the bash environment can listen to the variables
<c>EBUILD_PHASE</c>, <c>CATEGORY</c> as well as the variables that are always
available during ebuild development (such as <c>P</c>, <c>PF</c>, ...). Based on
the values of these variables, you can then execute additional steps.
</p>

</body>
</subsection>
<subsection>
<title>Example: Updating File Databases</title>
<body>

<p>
In this example, we'll use <path>/etc/portage/bashrc</path> to call some file
database applications to ensure their databases are up to date with the system.
The applications used in the example are <c>aide</c> (an intrusion detection
tool) and <c>updatedb</c> (to use with <c>locate</c>), but these are meant as
examples. Do not consider this as a HOWTO for AIDE ;-)
</p>

<p>
To use <path>/etc/portage/bashrc</path> for this case, we need to "hook" in the
<c>postrm</c> (after removal of files) and <c>postinst</c> (after installation
of files) functions, because that is when the files on the file system have been
changed.
</p>

<pre caption="Example /etc/portage/bashrc">
if [ "${EBUILD_PHASE}" == "postinst"] || [ "${EBUILD_PHASE}" == "postrm" ];
then
  echo ":: Calling aide --update to update its database";
  aide --update;
  echo ":: Calling updatedb to update its database";
  updatedb;
fi
</pre>

</body>
</subsection>
</section>

<section>
<title>Executing Tasks After --sync</title>
<subsection>
<title>The /etc/portage/postsync.d location</title>
<body>

<p>
Until now we've talked about hooking into the ebuild processes. However, Portage
also has another important function: updating the Portage tree. In order to run
tasks after updating the Portage tree, put a script inside
<path>/etc/portage/postsync.d</path> and make sure it is marked as executable.
</p>

</body>
</subsection>
<subsection>
<title>Example: Running eix-update</title>
<body>

<p>
If you didn't use <c>eix-sync</c> to update the tree, you can still have its
database updated after running <c>emerge --sync</c> (or <c>emerge-webrsync</c>)
by putting a symlink to <path>/usr/bin/eix</path> called <path>eix-update</path>
in <path>/etc/portage/postsync.d</path>.
</p>

<pre caption="Running eix-update after a sync operation">
# <i>ln -s /usr/bin/eix /etc/portage/postsync.d/eix-update</i>
</pre>

<note>
If you rather use a different name, you will need to make a script that calls
<c>/usr/bin/eix-update</c> instead. The <c>eix</c> binary looks at how it has
been called to find out which function it has to execute. If you put in a
symlink to <c>eix</c> that isn't called <c>eix-update</c>, it will not run
correctly.
</note>

</body>
</subsection>
</section>

<section>
<title>Overriding Profile Settings</title>
<subsection>
<title>The /etc/portage/profile location</title>
<body>

<p>
By default, Gentoo uses the settings contained in the profile pointed to by
<path>/etc/make.profile</path> (a symbolic link to the right profile directory).
These profiles define both specific settings as well as inherit settings from
other profiles (through their <path>parent</path> file).
</p>

<p>
By using <path>/etc/portage/profile</path>, you can override profile settings
such as <path>packages</path> (what packages are considered to be part of the
system set), <path>virtuals</path> (default packages when pulling in a virtual)
and more.
</p>

</body>
</subsection>
<subsection>
<title>Example: Adding nfs-utils to the System Set</title>
<body>

<p>
If you use NFS-based file systems for rather critical file systems, you might
want to have <path>net-fs/nfs-utils</path> "protected" as a system package,
causing Portage to heavily warn you if it would be deleted.
</p>

<p>
To accomplish that, we add the package to
<path>/etc/portage/profile/packages</path>, prepended with a <c>*</c>:
</p>

<pre caption="/etc/portage/profile/packages content">
*net-fs/nfs-utils
</pre>

</body>
</subsection>
</section>

<section>
<title>Applying Non-Standard Patches</title>
<subsection>
<title>Using epatch_user</title>
<body>

<p>
To manage several ebuilds in a similar manner, ebuild developers use
<e>eclasses</e> (sort-of shell libraries) that define commonly used functions.
One of these eclasses is <path>eutils.eclass</path> which offers an interesting
function called <c>epatch_user</c>.
</p>

<p>
The <c>epatch_user</c> function applies source code patches that are found in
<path>/etc/portage/patches/&lt;category&gt;/&lt;package&gt;[-&lt;version&gt;[-&lt;revision&gt;]]</path>,
whatever directory is found first. Sadly, not all ebuilds automatically call
this function so just putting your patch in this location might not always work.
</p>

<p>
Luckily, with the information provided above, you can call this function by
hooking into, for instance, the <c>prepare</c> phase. The function can be called
as many times as you like - it will only apply the patches once.
</p>

</body>
</subsection>
<subsection>
<title>Example: Applying Patches to Firefox</title>
<body>

<p>
The <path>www-client/firefox</path> package is one of the few that already call
<c>epatch_user</c> from within the ebuild, so you do not need to override
anything specific. 
</p>

<p>
If you need to patch firefox (for instance because a developer asked you to to
support you in a bug you reported), put the patch in
<path>/etc/portage/patches/www-client/firefox</path> (probably best to use the
full name, including version so that the patch does not interfere with later
versions) and rebuild firefox.
</p>

</body>
</subsection>
</section>

</sections>
1	swift	1.1	<?xml version='1.0' encoding='UTF-8'?>
2			<!DOCTYPE sections SYSTEM "/dtd/book.dtd">
3
4			<!-- The content of this document is licensed under the CC-BY-SA license -->
5			<!-- See http://creativecommons.org/licenses/by-sa/2.5 -->
6
7	swift	1.2	<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/handbook/hb-portage-advanced.xml,v 1.1 2012/01/15 09:55:35 swift Exp $ -->
8	swift	1.1
9			<sections>
10
11			<abstract>
12			As times goes by, Portage evolves and matures further and further. Additional
13			features are continuously being put in - many of these are only of use by more
14			advanced users. This chapter will go into more detail of these specific
15			features.
16			</abstract>
17
18	swift	1.2	<version>2</version>
19			<date>2012-02-07</date>
20	swift	1.1
21			<section>
22			<title>Introduction</title>
23			<body>
24
25			<p>
26			For most users, the information received thus far is sufficient for all their
27			Linux operations. But Portage is capable of much more; many of its features are
28			for advanced users or only applicable in specific corner cases. Still, that
29			would not be an excuse not to document them.
30			</p>
31
32			<p>
33			Of course, with lots of flexibility comes a huge list of potential cases. It
34			will not be possible to document them all here. Instead, we hope to focus on
35			some generic issues which you can then bend to fit your own needs. If you have
36			need for more specific tweaks and tips, you might find them on the <uri
37			link="https://wiki.gentoo.org">Gentoo WiKi</uri> instead.
38			</p>
39
40			<p>
41			Most, if not all of these additional features can be easily found by digging
42			through the manual pages that portage provides:
43			</p>
44
45			<pre caption="Reading up on portage man pages">
46			$ <i>man portage</i>
47			$ <i>man make.conf</i>
48			</pre>
49
50			<p>
51			Finally, know that these are advanced features which, if not worked with
52			correctly, can make debugging and troubleshooting very difficult. Make sure you
53			mention these if you think you hit a bug and want to open a bugreport.
54			</p>
55
56			</body>
57			</section>
58
59			<section>
60			<title>Per-Package Environment Variables</title>
61			<subsection>
62			<title>Using /etc/portage/env</title>
63			<body>
64
65			<p>
66			By default, package builds will use the environment variables defined in
67			<path>/etc/make.conf</path>, such as <c>CFLAGS</c>, <c>MAKEOPTS</c> and more. In
68			some cases though, you might want to provide different variables for specific
69			packages. To do so, Portage supports the use of <path>/etc/portage/env</path>
70			and <path>/etc/portage/package.env</path>.
71			</p>
72
73			<p>
74			The <path>/etc/portage/package.env</path> file contains the list of packages for
75			which you want deviating variables as well as a specific identifier that tells
76			Portage which changes you want. The identifier name you pick yourself, Portage
77			will look for the variables in the <path>/etc/portage/env/<identifier></path>
78			file.
79			</p>
80
81			</body>
82			</subsection>
83			<subsection>
84			<title>Example: Using debugging for specific packages</title>
85			<body>
86
87			<p>
88			As an example, we enable debugging for the <path>media-video/mplayer</path>
89			package.
90			</p>
91
92			<p>
93			First of all, we set the debugging variables in a file called
94			<path>/etc/portage/env/debug-cflags</path>. The name is arbitrarily chosen, but
95			of course reflects the reason of the deviation to make it more obvious later why
96			a deviation was put in.
97			</p>
98
99			<pre caption="/etc/portage/env/debug-cflags content">
100			CFLAGS="-O2 -ggdb -pipe"
101			FEATURES="${FEATURES} nostrip"
102			</pre>
103
104			<p>
105			Next, we tag the <path>media-video/mplayer</path> package to use this content:
106			</p>
107
108			<pre caption="/etc/portage/package.env content">
109			media-video/mplayer debug-cflags
110			</pre>
111
112			</body>
113			</subsection>
114			</section>
115
116			<section>
117			<title>Hooking In the Emerge Process</title>
118			<subsection>
119			<title>Using /etc/portage/bashrc and affiliated files</title>
120			<body>
121
122			<p>
123			When Portage works with ebuilds, it uses a bash environment in which it calls
124	swift	1.2	the various build functions (like src_prepare, src_configure, pkg_postinst,
125	swift	1.1	etc.). But Portage also allows you to set up a bash environment yourself.
126			</p>
127
128			<p>
129			The advantage of using your own bash environment is that you can hook in the
130			emerge process during each step it performs. This can be done for every emerge
131			(through <path>/etc/portage/bashrc</path>) or by using per-package environments
132			(through <path>/etc/portage/env</path> as discussed earlier).
133			</p>
134
135			<p>
136			To hook in the process, the bash environment can listen to the variables
137			<c>EBUILD_PHASE</c>, <c>CATEGORY</c> as well as the variables that are always
138			available during ebuild development (such as <c>P</c>, <c>PF</c>, ...). Based on
139			the values of these variables, you can then execute additional steps.
140			</p>
141
142			</body>
143			</subsection>
144			<subsection>
145			<title>Example: Updating File Databases</title>
146			<body>
147
148			<p>
149			In this example, we'll use <path>/etc/portage/bashrc</path> to call some file
150			database applications to ensure their databases are up to date with the system.
151			The applications used in the example are <c>aide</c> (an intrusion detection
152			tool) and <c>updatedb</c> (to use with <c>locate</c>), but these are meant as
153			examples. Do not consider this as a HOWTO for AIDE ;-)
154			</p>
155
156			<p>
157			To use <path>/etc/portage/bashrc</path> for this case, we need to "hook" in the
158			<c>postrm</c> (after removal of files) and <c>postinst</c> (after installation
159			of files) functions, because that is when the files on the file system have been
160			changed.
161			</p>
162
163			<pre caption="Example /etc/portage/bashrc">
164			if [ "${EBUILD_PHASE}" == "postinst"] \|\| [ "${EBUILD_PHASE}" == "postrm" ];
165			then
166			echo ":: Calling aide --update to update its database";
167			aide --update;
168			echo ":: Calling updatedb to update its database";
169			updatedb;
170			fi
171			</pre>
172
173			</body>
174			</subsection>
175			</section>
176
177			<section>
178			<title>Executing Tasks After --sync</title>
179			<subsection>
180			<title>The /etc/portage/postsync.d location</title>
181			<body>
182
183			<p>
184			Until now we've talked about hooking into the ebuild processes. However, Portage
185			also has another important function: updating the Portage tree. In order to run
186			tasks after updating the Portage tree, put a script inside
187			<path>/etc/portage/postsync.d</path> and make sure it is marked as executable.
188			</p>
189
190			</body>
191			</subsection>
192			<subsection>
193			<title>Example: Running eix-update</title>
194			<body>
195
196			<p>
197			If you didn't use <c>eix-sync</c> to update the tree, you can still have its
198			database updated after running <c>emerge --sync</c> (or <c>emerge-webrsync</c>)
199			by putting a symlink to <path>/usr/bin/eix</path> called <path>eix-update</path>
200			in <path>/etc/portage/postsync.d</path>.
201			</p>
202
203			<pre caption="Running eix-update after a sync operation">
204			# <i>ln -s /usr/bin/eix /etc/portage/postsync.d/eix-update</i>
205			</pre>
206
207			<note>
208			If you rather use a different name, you will need to make a script that calls
209			<c>/usr/bin/eix-update</c> instead. The <c>eix</c> binary looks at how it has
210			been called to find out which function it has to execute. If you put in a
211			symlink to <c>eix</c> that isn't called <c>eix-update</c>, it will not run
212			correctly.
213			</note>
214
215			</body>
216			</subsection>
217			</section>
218
219			<section>
220			<title>Overriding Profile Settings</title>
221			<subsection>
222			<title>The /etc/portage/profile location</title>
223			<body>
224
225			<p>
226			By default, Gentoo uses the settings contained in the profile pointed to by
227			<path>/etc/make.profile</path> (a symbolic link to the right profile directory).
228			These profiles define both specific settings as well as inherit settings from
229			other profiles (through their <path>parent</path> file).
230			</p>
231
232			<p>
233			By using <path>/etc/portage/profile</path>, you can override profile settings
234			such as <path>packages</path> (what packages are considered to be part of the
235			system set), <path>virtuals</path> (default packages when pulling in a virtual)
236			and more.
237			</p>
238
239			</body>
240			</subsection>
241			<subsection>
242			<title>Example: Adding nfs-utils to the System Set</title>
243			<body>
244
245			<p>
246			If you use NFS-based file systems for rather critical file systems, you might
247			want to have <path>net-fs/nfs-utils</path> "protected" as a system package,
248			causing Portage to heavily warn you if it would be deleted.
249			</p>
250
251			<p>
252			To accomplish that, we add the package to
253			<path>/etc/portage/profile/packages</path>, prepended with a <c>*</c>:
254			</p>
255
256			<pre caption="/etc/portage/profile/packages content">
257			*net-fs/nfs-utils
258			</pre>
259
260			</body>
261			</subsection>
262			</section>
263
264			<section>
265			<title>Applying Non-Standard Patches</title>
266			<subsection>
267			<title>Using epatch_user</title>
268			<body>
269
270			<p>
271			To manage several ebuilds in a similar manner, ebuild developers use
272			<e>eclasses</e> (sort-of shell libraries) that define commonly used functions.
273			One of these eclasses is <path>eutils.eclass</path> which offers an interesting
274			function called <c>epatch_user</c>.
275			</p>
276
277			<p>
278			The <c>epatch_user</c> function applies source code patches that are found in
279			<path>/etc/portage/patches/<category>/<package>[-<version>[-<revision>]]</path>,
280			whatever directory is found first. Sadly, not all ebuilds automatically call
281			this function so just putting your patch in this location might not always work.
282			</p>
283
284			<p>
285			Luckily, with the information provided above, you can call this function by
286			hooking into, for instance, the <c>prepare</c> phase. The function can be called
287			as many times as you like - it will only apply the patches once.
288			</p>
289
290			</body>
291			</subsection>
292			<subsection>
293			<title>Example: Applying Patches to Firefox</title>
294			<body>
295
296			<p>
297			The <path>www-client/firefox</path> package is one of the few that already call
298			<c>epatch_user</c> from within the ebuild, so you do not need to override
299			anything specific.
300			</p>
301
302			<p>
303			If you need to patch firefox (for instance because a developer asked you to to
304			support you in a bug you reported), put the patch in
305			<path>/etc/portage/patches/www-client/firefox</path> (probably best to use the
306			full name, including version so that the patch does not interfere with later
307			versions) and rebuild firefox.
308			</p>
309
310			</body>
311			</subsection>
312			</section>
313
314			</sections>