Contents of /xml/htdocs/proj/en/glep/glep-0047.html

Parent Directory Parent Directory | Revision Log Revision Log

Revision 1.9 - (show annotations) (download) (as text)
Sun Oct 14 17:00:15 2007 UTC (11 years, 5 months ago) by antarus
Branch: MAIN
Changes since 1.8: +4 -251 lines
File MIME type: text/html
the canary on 53 went well, changing the rest

1 <?xml version="1.0" encoding="utf-8" ?>
2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
3 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
5 <head>
6 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
7 <meta name="generator" content="Docutils 0.4: http://docutils.sourceforge.net/" />
8 <title>GLEP 47 -- Creating 'safe' environment variables</title>
9 <link rel="stylesheet" href="tools/glep.css" type="text/css" />
10 </head>
11 <body bgcolor="white">
12 <table class="navigation" cellpadding="0" cellspacing="0"
13 width="100%" border="0">
14 <tr><td class="navicon" width="150" height="35">
15 <a href="http://www.gentoo.org/" title="Gentoo Linux Home Page">
16 <img src="http://www.gentoo.org/images/gentoo-new.gif" alt="[Gentoo]"
17 border="0" width="150" height="35" /></a></td>
18 <td class="textlinks" align="left">
19 [<b><a href="http://www.gentoo.org/">Gentoo Linux Home</a></b>]
20 [<b><a href="http://www.gentoo.org/proj/en/glep">GLEP Index</a></b>]
21 [<b><a href="http://www.gentoo.org/proj/en/glep/glep-0047.txt">GLEP Source</a></b>]
22 </td></tr></table>
23 <table class="rfc2822 docutils field-list" frame="void" rules="none">
24 <col class="field-name" />
25 <col class="field-body" />
26 <tbody valign="top">
27 <tr class="field"><th class="field-name">GLEP:</th><td class="field-body">47</td>
28 </tr>
29 <tr class="field"><th class="field-name">Title:</th><td class="field-body">Creating 'safe' environment variables</td>
30 </tr>
31 <tr class="field"><th class="field-name">Version:</th><td class="field-body">1.6</td>
32 </tr>
33 <tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference" href="http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/proj/en/glep/glep-0047.txt?cvsroot=gentoo">2007/07/22 10:03:18</a></td>
34 </tr>
35 <tr class="field"><th class="field-name">Author:</th><td class="field-body">Diego Pettenò, Fabian Groffen</td>
36 </tr>
37 <tr class="field"><th class="field-name">Status:</th><td class="field-body">Rejected</td>
38 </tr>
39 <tr class="field"><th class="field-name">Type:</th><td class="field-body">Standards Track</td>
40 </tr>
41 <tr class="field"><th class="field-name">Content-Type:</th><td class="field-body"><a class="reference" href="glep-0002.html">text/x-rst</a></td>
42 </tr>
43 <tr class="field"><th class="field-name">Created:</th><td class="field-body">14-Oct-2005</td>
44 </tr>
45 <tr class="field"><th class="field-name">Post-History:</th><td class="field-body">09-Feb-2006</td>
46 </tr>
47 </tbody>
48 </table>
49 <hr />
50 <div class="contents topic">
51 <p class="topic-title first"><a id="contents" name="contents">Contents</a></p>
52 <ul class="simple">
53 <li><a class="reference" href="#credits" id="id5" name="id5">Credits</a></li>
54 <li><a class="reference" href="#abstract" id="id6" name="id6">Abstract</a></li>
55 <li><a class="reference" href="#motivation" id="id7" name="id7">Motivation</a></li>
56 <li><a class="reference" href="#rationale" id="id8" name="id8">Rationale</a></li>
57 <li><a class="reference" href="#backwards-compatibility" id="id9" name="id9">Backwards Compatibility</a></li>
58 <li><a class="reference" href="#specification" id="id10" name="id10">Specification</a><ul>
59 <li><a class="reference" href="#variable-assignment" id="id11" name="id11">Variable Assignment</a></li>
60 </ul>
61 </li>
62 <li><a class="reference" href="#references" id="id12" name="id12">References</a></li>
63 <li><a class="reference" href="#copyright" id="id13" name="id13">Copyright</a></li>
64 </ul>
65 </div>
66 <div class="section">
67 <h1><a class="toc-backref" href="#id5" id="credits" name="credits">Credits</a></h1>
68 <p>The text of this GLEP is a result of a discussion and input of the
69 following persons, in no particular order: Mike Frysinger, Diego
70 Pettenò, Fabian Groffen and Finn Thain.</p>
71 </div>
72 <div class="section">
73 <h1><a class="toc-backref" href="#id6" id="abstract" name="abstract">Abstract</a></h1>
74 <p>In order for ebuilds and eclasses to be able to make host specific
75 decisions, it is necessary to have a number of environmental variables
76 which allow for such decisions. This GLEP introduces some measures that
77 need to be made to make these decisions 'safe', by making sure the
78 variables the decisions are based on are 'safe'. A small overlap with
79 GLEP 22 <a class="footnote-reference" href="#id3" id="id1" name="id1">[1]</a> is being handled in this GLEP where the use of 2-tuple
80 keywords are being kept instead of 4-tuple keywords. Additionally, the
81 <tt class="docutils literal"><span class="pre">ELIBC</span></tt>, <tt class="docutils literal"><span class="pre">KERNEL</span></tt> and <tt class="docutils literal"><span class="pre">ARCH</span></tt> get auto filled starting from
82 <tt class="docutils literal"><span class="pre">CHOST</span></tt> and the 2-tuple keyword, instead of solely from they 4-tuple
83 keyword as proposed in GLEP 22.</p>
84 <p>The destiny of the <tt class="docutils literal"><span class="pre">USERLAND</span></tt> variable is out of the scope of this
85 GLEP. Depending on its presence in the tree, it may be decided to set
86 this variable the same way we propose to set <tt class="docutils literal"><span class="pre">ELIBC</span></tt>, <tt class="docutils literal"><span class="pre">KERNEL</span></tt> and
87 <tt class="docutils literal"><span class="pre">ARCH</span></tt>, or alternatively, e.g. via the profiles.</p>
88 </div>
89 <div class="section">
90 <h1><a class="toc-backref" href="#id7" id="motivation" name="motivation">Motivation</a></h1>
91 <p>The Gentoo/Alt project is in an emerging state to get ready to serve a
92 plethora of 'alternative' configurations such as FreeBSD, NetBSD,
93 DragonflyBSD, GNU/kFreeBSD, Mac OS X, (Open)Darwin, (Open)Solaris and so
94 on. As such, the project is in need for a better grip on the actual
95 host being built on. This information on the host environment is
96 necessary to make proper (automated) decisions on settings that are
97 highly dependant on the build environment, such as platform or C-library
98 implementation.</p>
99 </div>
100 <div class="section">
101 <h1><a class="toc-backref" href="#id8" id="rationale" name="rationale">Rationale</a></h1>
102 <p>Gentoo's unique Portage system allows easy installation of applications
103 from source packages. Compiling sources is prone to many environmental
104 settings and availability of certain tools. Only recently the Gentoo
105 for FreeBSD project has started, as second Gentoo project that operates
106 on a foreign host operating system using foreign (non-GNU) C-libraries
107 and userland utilities. Such projects suffer from the current implicit
108 assumption made within Gentoo Portage's ebuilds that there is a single
109 type of operating system, C-libraries and system utilities. In order to
110 enable ebuilds -- and also eclasses -- to be aware of these
111 environmental differences, information regarding it should be supplied.
112 Since decisions based on this information can be vital, it is of high
113 importance that this information can be trusted and the values can be
114 considered 'safe' and correct.</p>
115 </div>
116 <div class="section">
117 <h1><a class="toc-backref" href="#id9" id="backwards-compatibility" name="backwards-compatibility">Backwards Compatibility</a></h1>
118 <p>The proposed keywording scheme in this GLEP is fully compatible with the
119 current situation of the portage tree, this in contrast to GLEP 22. The
120 variables provided by GLEP 22 can't be extracted from the new keyword,
121 but since GLEP 22-style keywords aren't in the tree at the moment, that
122 is not a problem. The same information can be extracted from the CHOST
123 variable, if necessary. No modifications to ebuilds will have to be
124 made.</p>
125 </div>
126 <div class="section">
127 <h1><a class="toc-backref" href="#id10" id="specification" name="specification">Specification</a></h1>
128 <p>Unlike GLEP 22 the currently used keyword scheme is not changed.
129 Instead of proposing a 4-tuple <a class="footnote-reference" href="#id4" id="id2" name="id2">[2]</a> keyword, a 2-tuple keyword is chosen
130 for archs that require them. Archs for which a 1-tuple keyword
131 suffices, can keep that keyword. Since this doesn't change anything to
132 the current situation in the tree, it is considered to be a big
133 advantage over the 4-tuple keyword from GLEP 22. This GLEP is an
134 official specification of the syntax of the keyword.</p>
135 <p>Keywords will consist out of two parts separated by a hyphen ('-'). The
136 part up to the first hyphen from the left of the keyword 2-tuple is the
137 architecture, such as ppc64, sparc and x86. Allowed characters for the
138 architecture name are in <tt class="docutils literal"><span class="pre">a-z0-9</span></tt>. The remaining part on the right of
139 the first hyphen from the left indicates the operating system or
140 distribution, such as linux, macos, darwin, obsd, et-cetera. If the
141 right hand part is omitted, it implies the operating system/distribution
142 type is Gentoo GNU/Linux. In such case the hyphen is also omitted, and
143 the keyword consists of solely the architecture. The operating system
144 or distribution name can consist out of characters in <tt class="docutils literal"><span class="pre">a-zA-Z0-9_+:-</span></tt>.
145 Please note that the hyphen is an allowed character, and therefore the
146 separation of the two fields in the keyword is only determinable by
147 scanning for the first hyphen character from the start of the keyword
148 string. Examples of keywords following this specification are
149 ppc-darwin and x86. This is fully compatible with the current use of
150 keywords in the tree.</p>
151 <p>The variables <tt class="docutils literal"><span class="pre">ELIBC</span></tt>, <tt class="docutils literal"><span class="pre">KERNEL</span></tt> and <tt class="docutils literal"><span class="pre">ARCH</span></tt> are currently set in
152 the profiles when other than their defaults for a GNU/Linux system.
153 They can as such easily be overridden and defined by the user. To
154 prevent this from happening, the variables should be auto filled by
155 Portage itself, based on the <tt class="docutils literal"><span class="pre">CHOST</span></tt> variable. While the <tt class="docutils literal"><span class="pre">CHOST</span></tt>
156 variable can be as easy as the others set by the user, it still is
157 assumed to be 'safe'. This assumption is grounded in the fact that the
158 variable itself is being used in various other places with the same
159 intention, and that an invalid <tt class="docutils literal"><span class="pre">CHOST</span></tt> will cause major malfunctioning
160 of the system. A user that changes the <tt class="docutils literal"><span class="pre">CHOST</span></tt> into something that is
161 not valid for the system, is already warned that this might render the
162 system unusable. Concluding, the 'safeness' of the <tt class="docutils literal"><span class="pre">CHOST</span></tt> variable
163 is based on externally assumed 'safeness', which's discussion falls
164 outside this GLEP.</p>
165 <p>Current USE-expansion of the variables is being maintained, as this
166 results in full backward compatibility. Since the variables themself
167 don't change in what they represent, but only how they are being
168 assigned, there should be no problem in maintaining them. Using
169 USE-expansion, conditional code can be written down in ebuilds, which is
170 not different from any existing methods at all:</p>
171 <pre class="literal-block">
172 ...
173 RDEPEND=&quot;elibc_FreeBSD? ( sys-libs/com_err )&quot;
174 ...
175 src_compile() {
176 ...
177 use elibc_FreeBSD &amp;&amp; myconf=&quot;${myconf} -Dlibc=/usr/lib/libc.a&quot;
178 ...
179 }
180 </pre>
181 <p>Alternatively, the variables <tt class="docutils literal"><span class="pre">ELIBC</span></tt>, <tt class="docutils literal"><span class="pre">KERNEL</span></tt> and <tt class="docutils literal"><span class="pre">ARCH</span></tt>
182 are available in the ebuild evironment and they can be used instead of
183 invoking <tt class="docutils literal"><span class="pre">xxx_Xxxx</span></tt> or in switch statements where they are actually
184 necessary.</p>
185 <p>A map file can be used to have the various <tt class="docutils literal"><span class="pre">CHOST</span></tt> values being
186 translated to the correct values for the four variables. This change is
187 invisible for ebuilds and eclasses, but allows to rely on these
188 variables as they are based on a 'safe' value -- the <tt class="docutils literal"><span class="pre">CHOST</span></tt> variable.
189 Ebuilds should not be sensitive to the keyword value, but use the
190 aforementioned four variables instead. They allow specific tests for
191 properties. If this is undesirable, the full <tt class="docutils literal"><span class="pre">CHOST</span></tt> variable can be
192 used to match a complete operating system.</p>
193 <div class="section">
194 <h2><a class="toc-backref" href="#id11" id="variable-assignment" name="variable-assignment">Variable Assignment</a></h2>
195 <p>The <tt class="docutils literal"><span class="pre">ELIBC</span></tt>, <tt class="docutils literal"><span class="pre">KERNEL</span></tt>, <tt class="docutils literal"><span class="pre">ARCH</span></tt> variables are filled from a profile
196 file. The file can be overlaid, such that the following entries in the
197 map file (on the left of the arrow) will result in the assigned
198 variables on the right hand side of the arrow:</p>
199 <pre class="literal-block">
200 *-*-linux-* -&gt; KERNEL=&quot;linux&quot;
201 *-*-*-gnu -&gt; ELIBC=&quot;glibc&quot;
202 *-*-kfreebsd-gnu -&gt; KERNEL=&quot;FreeBSD&quot; ELIBC=&quot;glibc&quot;
203 *-*-freebsd* -&gt; KERNEL=&quot;FreeBSD&quot; ELIBC=&quot;FreeBSD&quot;
204 *-*-darwin* -&gt; KERNEL=&quot;Darwin&quot; ELIBC=&quot;Darwin&quot;
205 *-*-netbsd* -&gt; KERNEL=&quot;NetBSD&quot; ELIBC=&quot;NetBSD&quot;
206 *-*-solaris* -&gt; KERNEL=&quot;Solaris&quot; ELIBC=&quot;Solaris&quot;
207 </pre>
208 <p>A way to achieve this is proposed by Mike Frysinger, which
209 suggests to have an env-map file, for instance filled with:</p>
210 <pre class="literal-block">
211 % cat env-map
212 *-linux-* KERNEL=linux
213 *-gnu ELIBC=glibc
214 x86_64-* ARCH=amd64
215 </pre>
216 <p>then the following bash script can be used to set the four variables to
217 their correct values:</p>
218 <pre class="literal-block">
219 % cat readmap
220 #!/bin/bash
223 [[ -z ${CHOST} ]] &amp;&amp; echo need chost
227 while read LINE ; do
228 set -- ${LINE}
229 targ=$1
230 shift
231 [[ ${CBUILD} == ${targ} ]] &amp;&amp; eval $&#64;
232 done &lt; env-map
235 </pre>
236 <p>Given the example env-map file, this script would result in:</p>
237 <pre class="literal-block">
238 % ./readmap x86_64-pc-linux-gnu
239 ARCH=amd64 KERNEL=linux ELIBC=glibc
240 </pre>
241 <p>The entries in the <tt class="docutils literal"><span class="pre">env-map</span></tt> file will be evaluated in a forward
242 linear full scan. A side-effect of this exhaustive search is that the
243 variables can be re-assigned if multiple entries match the given
244 <tt class="docutils literal"><span class="pre">CHOST</span></tt>. Because of this, the order of the entries does matter.
245 Because the <tt class="docutils literal"><span class="pre">env-map</span></tt> file size is assumed not to exceed the block
246 size of the file system, the performance penalty of a full scan versus
247 'first-hit-stop technique' is assumed to be minimal.</p>
248 <p>It should be noted, however, that the above bash script is a proof of
249 concept implementation. Since Portage is largerly written in Python, it
250 will be more efficient to write an equivalent of this code in Python
251 also. Coding wise, this is considered to be a non-issue, but the format
252 of the <tt class="docutils literal"><span class="pre">env-map</span></tt> file, and especially its wildcard characters, might
253 not be the best match with Python. For this purpose, the format
254 specification of the <tt class="docutils literal"><span class="pre">env-map</span></tt> file is deferred to the Python
255 implementation, and only the requirements are given here.</p>
256 <p>The <tt class="docutils literal"><span class="pre">env-map</span></tt> file should be capable of encoding a <tt class="docutils literal"><span class="pre">key</span></tt>, <tt class="docutils literal"><span class="pre">value</span></tt>
257 pair, where <tt class="docutils literal"><span class="pre">key</span></tt> is a (regular) expression that matches a
258 chost-string, and <tt class="docutils literal"><span class="pre">value</span></tt> contains at least one, distinct variable
259 assignment for the variables <tt class="docutils literal"><span class="pre">ARCH</span></tt>, <tt class="docutils literal"><span class="pre">KERNEL</span></tt> and <tt class="docutils literal"><span class="pre">ELIBC</span></tt>. The
260 interpreter of the <tt class="docutils literal"><span class="pre">env-map</span></tt> file must scan the file linearly and
261 continue trying to match the <tt class="docutils literal"><span class="pre">key</span></tt>s and assign variables if
262 appropriate until the end of file.</p>
263 <p>Since Portage will use the <tt class="docutils literal"><span class="pre">env-map</span></tt> file, the location of the file is
264 beyond the scope of this GLEP and up to the Portage implementors.</p>
265 </div>
266 </div>
267 <div class="section">
268 <h1><a class="toc-backref" href="#id12" id="references" name="references">References</a></h1>
269 <table class="docutils footnote" frame="void" id="id3" rules="none">
270 <colgroup><col class="label" /><col /></colgroup>
271 <tbody valign="top">
272 <tr><td class="label"><a class="fn-backref" href="#id1" name="id3">[1]</a></td><td>GLEP 22, New &quot;keyword&quot; system to incorporate various
273 userlands/kernels/archs, Goodyear,
274 (<a class="reference" href="http://glep.gentoo.org/glep-0022.html">http://glep.gentoo.org/glep-0022.html</a>)</td></tr>
275 </tbody>
276 </table>
277 <table class="docutils footnote" frame="void" id="id4" rules="none">
278 <colgroup><col class="label" /><col /></colgroup>
279 <tbody valign="top">
280 <tr><td class="label"><a class="fn-backref" href="#id2" name="id4">[2]</a></td><td>For the purpose of readability, we will refer to 1, 2 and
281 4-tuples, even though tuple in itself suggest a field consisting of
282 two values. For clarity: a 1-tuple describes a single value field,
283 while a 4-tuple describes a field consisting out of four values.</td></tr>
284 </tbody>
285 </table>
286 </div>
287 <div class="section">
288 <h1><a class="toc-backref" href="#id13" id="copyright" name="copyright">Copyright</a></h1>
289 <p>This document has been placed in the public domain.</p>
290 </div>
292 </div>
293 <div class="footer">
294 <hr class="footer" />
295 <a class="reference" href="glep-0047.txt">View document source</a>.
296 Generated on: 2007-10-13 13:39 UTC.
297 Generated by <a class="reference" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.
299 </div>
300 </body>
301 </html>

  ViewVC Help
Powered by ViewVC 1.1.20