en/glep/glep-0047.html

<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<!--
This HTML is auto-generated.  DO NOT EDIT THIS FILE!  If you are writing a new
PEP, see http://www.python.org/peps/pep-0001.html for instructions and links
to templates.  DO NOT USE THIS HTML FILE AS YOUR TEMPLATE!
-->
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  <meta name="generator" content="Docutils 0.4: http://docutils.sourceforge.net/" />
  <title>GLEP 47 -- Creating 'safe' environment variables</title>
  <style type="text/css">

/*
:Author: David Goodger
:Contact: goodger@users.sourceforge.net
:date: $Date: 2006/10/10 20:25:14 $
:version: $Revision: 1.5 $
:copyright: This stylesheet has been placed in the public domain.

Default cascading style sheet for the PEP HTML output of Docutils.
*/

.first {
  margin-top: 0 }

.last {
  margin-bottom: 0 }

.navigation {
  width: 100% ;
  background: #cc99ff ;
  margin-top: 0px ;
  margin-bottom: 0px }

.navigation .navicon {
  width: 150px ;
  height: 35px }

.navigation .textlinks {
  padding-left: 1em ;
  text-align: left }

.navigation td, .navigation th {
  padding-left: 0em ;
  padding-right: 0em ;
  vertical-align: middle }

.rfc2822 {
  margin-top: 0.5em ;
  margin-left: 0.5em ;
  margin-right: 0.5em ;
  margin-bottom: 0em }

.rfc2822 td {
  text-align: left }

.rfc2822 th.field-name {
  text-align: right ;
  font-family: sans-serif ;
  padding-right: 0.5em ;
  font-weight: bold ;
  margin-bottom: 0em }

a.toc-backref {
  text-decoration: none ;
  color: black }

body {
  margin: 0px ;
  margin-bottom: 1em ;
  padding: 0px }

dd {
  margin-bottom: 0.5em }

div.section {
  margin-left: 1em ;
  margin-right: 1em ;
  margin-bottom: 1.5em }

div.section div.section {
  margin-left: 0em ;
  margin-right: 0em ;
  margin-top: 1.5em }

div.abstract {
  margin: 2em 5em }

div.abstract p.topic-title {
  font-weight: bold ;
  text-align: center }

div.attention, div.caution, div.danger, div.error, div.hint,
div.important, div.note, div.tip, div.warning {
  margin: 2em ;
  border: medium outset ;
  padding: 1em }

div.attention p.admonition-title, div.caution p.admonition-title,
div.danger p.admonition-title, div.error p.admonition-title,
div.warning p.admonition-title {
  color: red ;
  font-weight: bold ;
  font-family: sans-serif }

div.hint p.admonition-title, div.important p.admonition-title,
div.note p.admonition-title, div.tip p.admonition-title {
  font-weight: bold ;
  font-family: sans-serif }

div.figure {
  margin-left: 2em }

div.footer, div.header {
  font-size: smaller }

div.footer {
  margin-left: 1em ;
  margin-right: 1em }

div.system-messages {
  margin: 5em }

div.system-messages h1 {
  color: red }

div.system-message {
  border: medium outset ;
  padding: 1em }

div.system-message p.system-message-title {
  color: red ;
  font-weight: bold }

div.topic {
  margin: 2em }

h1 {
  font-family: sans-serif ;
  font-size: large }

h2 {
  font-family: sans-serif ;
  font-size: medium }

h3 {
  font-family: sans-serif ;
  font-size: small }

h4 {
  font-family: sans-serif ;
  font-style: italic ;
  font-size: small }

h5 {
  font-family: sans-serif;
  font-size: x-small }

h6 {
  font-family: sans-serif;
  font-style: italic ;
  font-size: x-small }

.section hr {
  width: 75% }

ol.simple, ul.simple {
  margin-bottom: 1em }

ol.arabic {
  list-style: decimal }

ol.loweralpha {
  list-style: lower-alpha }

ol.upperalpha {
  list-style: upper-alpha }

ol.lowerroman {
  list-style: lower-roman }

ol.upperroman {
  list-style: upper-roman }

p.caption {
  font-style: italic }

p.credits {
  font-style: italic ;
  font-size: smaller }

p.label {
  white-space: nowrap }

p.topic-title {
  font-family: sans-serif ;
  font-weight: bold }

pre.line-block {
  font-family: serif ;
  font-size: 100% }

pre.literal-block, pre.doctest-block {
  margin-left: 2em ;
  margin-right: 2em ;
  background-color: #eeeeee }

span.classifier {
  font-family: sans-serif ;
  font-style: oblique }

span.classifier-delimiter {
  font-family: sans-serif ;
  font-weight: bold }

span.interpreted {
  font-family: sans-serif }

span.option-argument {
  font-style: italic }

span.pre {
  white-space: pre }

span.problematic {
  color: red }

table {
  margin-top: 0.5em ;
  margin-bottom: 0.5em }

td, th {
  padding-left: 0.5em ;
  padding-right: 0.5em ;
  vertical-align: top }

td.num {
  text-align: right }

th.field-name {
  font-weight: bold ;
  text-align: left ;
  white-space: nowrap }

h1 tt, h2 tt, h3 tt, h4 tt, h5 tt, h6 tt {
  font-size: 100% }

tt {
  background-color: #eeeeee }

ul.auto-toc {
  list-style-type: none }

</style>
</head>
<body bgcolor="white">
<table class="navigation" cellpadding="0" cellspacing="0"
       width="100%" border="0">
<tr><td class="navicon" width="150" height="35">
<a href="http://www.gentoo.org/" title="Gentoo Linux Home Page">
<img src="http://www.gentoo.org/images/gentoo-new.gif" alt="[Gentoo]"
 border="0" width="150" height="35" /></a></td>
<td class="textlinks" align="left">
[<b><a href="http://www.gentoo.org/">Gentoo Linux Home</a></b>]
[<b><a href="http://www.gentoo.org/proj/en/glep/">GLEP Index</a></b>]
[<b><a href="http://www.gentoo.org/proj/en/glep/glep-0047.txt">GLEP Source</a></b>]
</td></tr></table>
<table class="rfc2822 docutils field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field"><th class="field-name">GLEP:</th><td class="field-body">47</td>
</tr>
<tr class="field"><th class="field-name">Title:</th><td class="field-body">Creating 'safe' environment variables</td>
</tr>
<tr class="field"><th class="field-name">Version:</th><td class="field-body">1.4</td>
</tr>
<tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference" href="http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/proj/en/glep/glep-0047.txt?cvsroot=gentoo">2006/02/13 21:00:50</a></td>
</tr>
<tr class="field"><th class="field-name">Author:</th><td class="field-body">Diego Pettenò, Fabian Groffen</td>
</tr>
<tr class="field"><th class="field-name">Status:</th><td class="field-body">Draft</td>
</tr>
<tr class="field"><th class="field-name">Type:</th><td class="field-body">Standards Track</td>
</tr>
<tr class="field"><th class="field-name">Content-Type:</th><td class="field-body"><a class="reference" href="glep-0002.html">text/x-rst</a></td>
</tr>
<tr class="field"><th class="field-name">Created:</th><td class="field-body">14-Oct-2005</td>
</tr>
<tr class="field"><th class="field-name">Post-History:</th><td class="field-body">09-Feb-2006</td>
</tr>
</tbody>
</table>
<hr />
<div class="contents topic">
<p class="topic-title first"><a id="contents" name="contents">Contents</a></p>
<ul class="simple">
<li><a class="reference" href="#credits" id="id5" name="id5">Credits</a></li>
<li><a class="reference" href="#abstract" id="id6" name="id6">Abstract</a></li>
<li><a class="reference" href="#motivation" id="id7" name="id7">Motivation</a></li>
<li><a class="reference" href="#rationale" id="id8" name="id8">Rationale</a></li>
<li><a class="reference" href="#backwards-compatibility" id="id9" name="id9">Backwards Compatibility</a></li>
<li><a class="reference" href="#specification" id="id10" name="id10">Specification</a><ul>
<li><a class="reference" href="#variable-assignment" id="id11" name="id11">Variable Assignment</a></li>
</ul>
</li>
<li><a class="reference" href="#references" id="id12" name="id12">References</a></li>
<li><a class="reference" href="#copyright" id="id13" name="id13">Copyright</a></li>
</ul>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id5" id="credits" name="credits">Credits</a></h1>
<p>The text of this GLEP is a result of a discussion and input of the
following persons, in no particular order: Mike Frysinger, Diego
Pettenò, Fabian Groffen and Finn Thain.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id6" id="abstract" name="abstract">Abstract</a></h1>
<p>In order for ebuilds and eclasses to be able to make host specific
decisions, it is necessary to have a number of environmental variables
which allow for such decisions.  This GLEP introduces some measures that
need to be made to make these decisions 'safe', by making sure the
variables the decisions are based on are 'safe'.  A small overlap with
GLEP 22 <a class="footnote-reference" href="#id3" id="id1" name="id1">[1]</a> is being handled in this GLEP where the use of 2-tuple
keywords are being kept instead of 4-tuple keywords.  Additionally, the
<tt class="docutils literal"><span class="pre">ELIBC</span></tt>, <tt class="docutils literal"><span class="pre">KERNEL</span></tt> and <tt class="docutils literal"><span class="pre">ARCH</span></tt> get auto filled starting from
<tt class="docutils literal"><span class="pre">CHOST</span></tt> and the 2-tuple keyword, instead of solely from they 4-tuple
keyword as proposed in GLEP 22.</p>
<p>The destiny of the <tt class="docutils literal"><span class="pre">USERLAND</span></tt> variable is out of the scope of this
GLEP.  Depending on its presence in the tree, it may be decided to set
this variable the same way we propose to set <tt class="docutils literal"><span class="pre">ELIBC</span></tt>, <tt class="docutils literal"><span class="pre">KERNEL</span></tt> and
<tt class="docutils literal"><span class="pre">ARCH</span></tt>, or alternatively, e.g. via the profiles.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id7" id="motivation" name="motivation">Motivation</a></h1>
<p>The Gentoo/Alt project is in an emerging state to get ready to serve a
plethora of 'alternative' configurations such as FreeBSD, NetBSD,
DragonflyBSD, GNU/kFreeBSD, Mac OS X, (Open)Darwin, (Open)Solaris and so
on.  As such, the project is in need for a better grip on the actual
host being built on.  This information on the host environment is
necessary to make proper (automated) decisions on settings that are
highly dependant on the build environment, such as platform or C-library
implementation.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id8" id="rationale" name="rationale">Rationale</a></h1>
<p>Gentoo's unique Portage system allows easy installation of applications
from source packages.  Compiling sources is prone to many environmental
settings and availability of certain tools.  Only recently the Gentoo
for FreeBSD project has started, as second Gentoo project that operates
on a foreign host operating system using foreign (non-GNU) C-libraries
and userland utilities.  Such projects suffer from the current implicit
assumption made within Gentoo Portage's ebuilds that there is a single
type of operating system, C-libraries and system utilities.  In order to
enable ebuilds -- and also eclasses -- to be aware of these
environmental differences, information regarding it should be supplied.
Since decisions based on this information can be vital, it is of high
importance that this information can be trusted and the values can be
considered 'safe' and correct.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id9" id="backwards-compatibility" name="backwards-compatibility">Backwards Compatibility</a></h1>
<p>The proposed keywording scheme in this GLEP is fully compatible with the
current situation of the portage tree, this in contrast to GLEP 22.  The
variables provided by GLEP 22 can't be extracted from the new keyword,
but since GLEP 22-style keywords aren't in the tree at the moment, that
is not a problem.  The same information can be extracted from the CHOST
variable, if necessary.  No modifications to ebuilds will have to be
made.</p>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id10" id="specification" name="specification">Specification</a></h1>
<p>Unlike GLEP 22 the currently used keyword scheme is not changed.
Instead of proposing a 4-tuple <a class="footnote-reference" href="#id4" id="id2" name="id2">[2]</a> keyword, a 2-tuple keyword is chosen
for archs that require them.  Archs for which a 1-tuple keyword
suffices, can keep that keyword.  Since this doesn't change anything to
the current situation in the tree, it is considered to be a big
advantage over the 4-tuple keyword from GLEP 22.  This GLEP is an
official specification of the syntax of the keyword.</p>
<p>Keywords will consist out of two parts separated by a hyphen ('-').  The
part up to the first hyphen from the left of the keyword 2-tuple is the
architecture, such as ppc64, sparc and x86.  Allowed characters for the
architecture name are in <tt class="docutils literal"><span class="pre">a-z0-9</span></tt>.  The remaining part on the right of
the first hyphen from the left indicates the operating system or
distribution, such as linux, macos, darwin, obsd, et-cetera.  If the
right hand part is omitted, it implies the operating system/distribution
type is Gentoo GNU/Linux.  In such case the hyphen is also omitted, and
the keyword consists of solely the architecture.  The operating system
or distribution name can consist out of characters in <tt class="docutils literal"><span class="pre">a-zA-Z0-9_+:-</span></tt>.
Please note that the hyphen is an allowed character, and therefore the
separation of the two fields in the keyword is only determinable by
scanning for the first hyphen character from the start of the keyword
string.  Examples of keywords following this specification are
ppc-darwin and x86.  This is fully compatible with the current use of
keywords in the tree.</p>
<p>The variables <tt class="docutils literal"><span class="pre">ELIBC</span></tt>, <tt class="docutils literal"><span class="pre">KERNEL</span></tt> and <tt class="docutils literal"><span class="pre">ARCH</span></tt> are currently set in
the profiles when other than their defaults for a GNU/Linux system.
They can as such easily be overridden and defined by the user.  To
prevent this from happening, the variables should be auto filled by
Portage itself, based on the <tt class="docutils literal"><span class="pre">CHOST</span></tt> variable.  While the <tt class="docutils literal"><span class="pre">CHOST</span></tt>
variable can be as easy as the others set by the user, it still is
assumed to be 'safe'.  This assumption is grounded in the fact that the
variable itself is being used in various other places with the same
intention, and that an invalid <tt class="docutils literal"><span class="pre">CHOST</span></tt> will cause major malfunctioning
of the system.  A user that changes the <tt class="docutils literal"><span class="pre">CHOST</span></tt> into something that is
not valid for the system, is already warned that this might render the
system unusable.  Concluding, the 'safeness' of the <tt class="docutils literal"><span class="pre">CHOST</span></tt> variable
is based on externally assumed 'safeness', which's discussion falls
outside this GLEP.</p>
<p>Current USE-expansion of the variables is being maintained, as this
results in full backward compatibility.  Since the variables themself
don't change in what they represent, but only how they are being
assigned, there should be no problem in maintaining them.  Using
USE-expansion, conditional code can be written down in ebuilds, which is
not different from any existing methods at all:</p>
<pre class="literal-block">
...
RDEPEND=&quot;elibc_FreeBSD? ( sys-libs/com_err )&quot;
...
src_compile() {
    ...
    use elibc_FreeBSD &amp;&amp; myconf=&quot;${myconf} -Dlibc=/usr/lib/libc.a&quot;
    ...
}
</pre>
<p>Alternatively, the variables <tt class="docutils literal"><span class="pre">ELIBC</span></tt>, <tt class="docutils literal"><span class="pre">KERNEL</span></tt> and <tt class="docutils literal"><span class="pre">ARCH</span></tt>
are available in the ebuild evironment and they can be used instead of
invoking <tt class="docutils literal"><span class="pre">xxx_Xxxx</span></tt> or in switch statements where they are actually
necessary.</p>
<p>A map file can be used to have the various <tt class="docutils literal"><span class="pre">CHOST</span></tt> values being
translated to the correct values for the four variables.  This change is
invisible for ebuilds and eclasses, but allows to rely on these
variables as they are based on a 'safe' value -- the <tt class="docutils literal"><span class="pre">CHOST</span></tt> variable.
Ebuilds should not be sensitive to the keyword value, but use the
aforementioned four variables instead.  They allow specific tests for
properties.  If this is undesirable, the full <tt class="docutils literal"><span class="pre">CHOST</span></tt> variable can be
used to match a complete operating system.</p>
<div class="section">
<h2><a class="toc-backref" href="#id11" id="variable-assignment" name="variable-assignment">Variable Assignment</a></h2>
<p>The <tt class="docutils literal"><span class="pre">ELIBC</span></tt>, <tt class="docutils literal"><span class="pre">KERNEL</span></tt>, <tt class="docutils literal"><span class="pre">ARCH</span></tt> variables are filled from a profile
file.  The file can be overlaid, such that the following entries in the
map file (on the left of the arrow) will result in the assigned
variables on the right hand side of the arrow:</p>
<pre class="literal-block">
*-*-linux-*      -&gt; KERNEL=&quot;linux&quot;
*-*-*-gnu        -&gt; ELIBC=&quot;glibc&quot;
*-*-kfreebsd-gnu -&gt; KERNEL=&quot;FreeBSD&quot; ELIBC=&quot;glibc&quot;
*-*-freebsd*     -&gt; KERNEL=&quot;FreeBSD&quot; ELIBC=&quot;FreeBSD&quot;
*-*-darwin*      -&gt; KERNEL=&quot;Darwin&quot; ELIBC=&quot;Darwin&quot;
*-*-netbsd*      -&gt; KERNEL=&quot;NetBSD&quot; ELIBC=&quot;NetBSD&quot;
*-*-solaris*     -&gt; KERNEL=&quot;Solaris&quot; ELIBC=&quot;Solaris&quot;
</pre>
<p>A way to achieve this is proposed by Mike Frysinger, which
suggests to have an env-map file, for instance filled with:</p>
<pre class="literal-block">
% cat env-map
*-linux-* KERNEL=linux
*-gnu ELIBC=glibc
x86_64-* ARCH=amd64
</pre>
<p>then the following bash script can be used to set the four variables to
their correct values:</p>
<pre class="literal-block">
% cat readmap
#!/bin/bash

CBUILD=${CBUILD:-${CHOST=${CHOST:-$1}}}
[[ -z ${CHOST} ]] &amp;&amp; echo need chost

unset KERNEL ELIBC ARCH

while read LINE ; do
    set -- ${LINE}
    targ=$1
    shift
    [[ ${CBUILD} == ${targ} ]] &amp;&amp; eval $&#64;
done &lt; env-map

echo ARCH=${ARCH} KERNEL=${KERNEL} ELIBC=${ELIBC}
</pre>
<p>Given the example env-map file, this script would result in:</p>
<pre class="literal-block">
% ./readmap x86_64-pc-linux-gnu
ARCH=amd64 KERNEL=linux ELIBC=glibc
</pre>
<p>The entries in the <tt class="docutils literal"><span class="pre">env-map</span></tt> file will be evaluated in a forward
linear full scan.  A side-effect of this exhaustive search is that the
variables can be re-assigned if multiple entries match the given
<tt class="docutils literal"><span class="pre">CHOST</span></tt>.  Because of this, the order of the entries does matter.
Because the <tt class="docutils literal"><span class="pre">env-map</span></tt> file size is assumed not to exceed the block
size of the file system, the performance penalty of a full scan versus
'first-hit-stop technique' is assumed to be minimal.</p>
<p>It should be noted, however, that the above bash script is a proof of
concept implementation.  Since Portage is largerly written in Python, it
will be more efficient to write an equivalent of this code in Python
also.  Coding wise, this is considered to be a non-issue, but the format
of the <tt class="docutils literal"><span class="pre">env-map</span></tt> file, and especially its wildcard characters, might
not be the best match with Python.  For this purpose, the format
specification of the <tt class="docutils literal"><span class="pre">env-map</span></tt> file is deferred to the Python
implementation, and only the requirements are given here.</p>
<p>The <tt class="docutils literal"><span class="pre">env-map</span></tt> file should be capable of encoding a <tt class="docutils literal"><span class="pre">key</span></tt>, <tt class="docutils literal"><span class="pre">value</span></tt>
pair, where <tt class="docutils literal"><span class="pre">key</span></tt> is a (regular) expression that matches a
chost-string, and <tt class="docutils literal"><span class="pre">value</span></tt> contains at least one, distinct variable
assignment for the variables <tt class="docutils literal"><span class="pre">ARCH</span></tt>, <tt class="docutils literal"><span class="pre">KERNEL</span></tt> and <tt class="docutils literal"><span class="pre">ELIBC</span></tt>.  The
interpreter of the <tt class="docutils literal"><span class="pre">env-map</span></tt> file must scan the file linearly and
continue trying to match the <tt class="docutils literal"><span class="pre">key</span></tt>s and assign variables if
appropriate until the end of file.</p>
<p>Since Portage will use the <tt class="docutils literal"><span class="pre">env-map</span></tt> file, the location of the file is
beyond the scope of this GLEP and up to the Portage implementors.</p>
</div>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id12" id="references" name="references">References</a></h1>
<table class="docutils footnote" frame="void" id="id3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id1" name="id3">[1]</a></td><td>GLEP 22, New &quot;keyword&quot; system to incorporate various
userlands/kernels/archs, Goodyear,
(<a class="reference" href="http://glep.gentoo.org/glep-0022.html">http://glep.gentoo.org/glep-0022.html</a>)</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="id4" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#id2" name="id4">[2]</a></td><td>For the purpose of readability, we will refer to 1, 2 and
4-tuples, even though tuple in itself suggest a field consisting of
two values.  For clarity: a 1-tuple describes a single value field,
while a 4-tuple describes a field consisting out of four values.</td></tr>
</tbody>
</table>
</div>
<div class="section">
<h1><a class="toc-backref" href="#id13" id="copyright" name="copyright">Copyright</a></h1>
<p>This document has been placed in the public domain.</p>
</div>

</div>
<div class="footer">
<hr class="footer" />
<a class="reference" href="glep-0047.txt">View document source</a>.
Generated on: 2006-10-10 20:23 UTC.
Generated by <a class="reference" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.

</div>
</body>
</html>
