en/glep/glep-0047.txt

GLEP: 47
Title: Creating 'safe' environment variables
Version: $Revision: 1.8 $
Last-Modified: $Date: 2006/01/26 20:56:55 $
Author: Diego Pettenò, Fabian Groffen <{flameeyes,grobian}@gentoo.org>
Status: Active
Type: Standards Track
Content-Type: text/x-rst
Created: 14-Oct-2005
Post-History:


Credits
=======

The text of this GLEP is a result of a discussion and input of the
following persons, in no particular order: Mike Frysinger, Diego
Pettenò, Fabian Groffen and Finn Thain.


Abstract
========

In order for ebuilds and eclasses to be able to make host specific
decisions, it is necessary to have a number of environmental variables
which allow for such decisions.  This GLEP introduces some measures that
need to be made to make these decisions 'safe', by making sure the
variables the decisions are based on are 'safe'.  A small overlap with
GLEP 22 [1]_ is being handled in this GLEP where the use of 2-tuple
keywords are being kept instead of 4-tuple keywords.  Additionally, the
``ELIBC``, ``KERNEL`` and ``ARCH`` get auto filled starting from
``CHOST`` and the 2-tuple keyword, instead of solely from they 4-tuple
keyword as proposed in GLEP 22.
   
The destiny of the ``USERLAND`` variable is out of the scope of this
GLEP.  Depending on its presence in the tree, it may be decided to set
this variable the same way we propose to set ``ELIBC``, ``KERNEL`` and
``ARCH``, or alternatively, e.g. via the profiles.


Motivation
==========

The Gentoo/Alt project is in an emerging state to get ready to serve a
plethora of 'alternative' configurations such as FreeBSD, NetBSD,
DragonflyBSD, GNU/kFreeBSD, Mac OS X, (Open)Darwin, (Open)Solaris and so
on.  As such, the project is in need for a better grip on the actual
host being built on.  This information on the host environment is
necessary to make proper (automated) decisions on settings that are
highly dependant on the build environment, such as platform or as in
[2]_.


Rationale
=========

Gentoo's unique Portage system allows easy installation of applications
from source packages.  Compiling sources is prone to many environmental
settings and availability of certain tools.  Only recently the Gentoo
for FreeBSD project has started, as second Gentoo project that operates
on a foreign host operating system using foreign (non-GNU) C-libraries
and userland utilities.  Such projects suffer from the current implicit
assumption made within Gentoo Portage's ebuilds that there is a single
type of operating system, C-libraries and system utilities.  In order to
enable ebuilds -- and also eclasses -- to be aware of these
environmental differences, information regarding it should be supplied.
Since decisions based on this information can be vital, it is of high
importance that this information can be trusted and the values can be
considered 'safe' and correct.


Backwards Compatibility
=======================

The proposed keywording scheme in this GLEP is fully compatible with the
current situation of the portage tree, this in contrast to GLEP 22.  The
variables provided by GLEP 22 can't be extracted from the new keyword,
but since GLEP 22-style keywords aren't in the tree at the moment, that
is not a problem.  The same information can be extracted from the CHOST
variable, if necessary.  No modifications to ebuilds will have to be
made.


Specification
=============

Unlike GLEP 22 the current keyword scheme as used in practice is not
changed.  Instead of proposing a 4-tuple [3]_ keyword, a 2-tuple
keyword is chosen for archs that require them.  Archs for which a
1-tuple keyword suffices, keep that keyword.  Since this doesn't change
anything to the current situation in the tree, it is considered to be a
big advantage over the 4-tuple keyword from GLEP 22.  This GLEP is an
official specification of the syntax of the keyword.

Keywords will consist out of two parts separated by a hyphen ('-').  The
left hand part of the keyword 2-tuple is the architecture, such as
ppc64, sparc and x86.  The right hand part indicates the operating
system or distribution, such as linux, macos, darwin, obsd, etc.  If the
right hand part is omitted, it implies the operating system/distribution
type is GNU/Linux.  In such case the hyphen is also omitted.  Examples
of such keywords are ppc-darwin and x86.  This is fully compatible with
the current keywords used in the tree.

The variables ``ELIBC``, ``KERNEL`` and ``ARCH`` are currently set in
the profiles when other than their defaults for a GNU/Linux system.
They can as such easily be overridden and defined by the user.  To
prevent this from happening, the variables should be auto filled by
Portage itself, based on the ``CHOST`` variable.

A map file can be used to have the various ``CHOST`` values being
translated to the correct values for the four variables.  This change is
invisible for ebuilds and eclasses, but allows to rely on these
variables as they are based on a 'safe' value -- the ``CHOST`` variable.
Ebuilds should not be sensitive to the keyword value, but use the
aforementioned four variables instead.  They allow specific tests for
properties.  If this is undesirable, the full ``CHOST`` variable can be
used to match a complete operating system.

Current USE-expansion is being maintained, for backwards compatibility.
Since the expansion is based on the variables mentioned above which do
not change, but only in the way they are generated, there should be no
problem in maintaining them.


Variables
---------

The ``ELIBC``, ``KERNEL``, ``ARCH`` variables are filled from a profile
file.  The file can be overlaid, such that the following entries in the
map file (on the left of the arrow) will result in the assigned
variables on the right hand side of the arrow:

::

    *-*-linux-*      -> KERNEL="linux"  
    *-*-*-gnu        -> ELIBC="glibc"
    *-*-kfreebsd-gnu -> KERNEL="FreeBSD" ELIBC="glibc"
    *-*-freebsd*     -> KERNEL="FreeBSD" ELIBC="FreeBSD"
    *-*-darwin*      -> KERNEL="Darwin" ELIBC="Darwin"
    *-*-netbsd*      -> KERNEL="NetBSD" ELIBC="NetBSD"
    *-*-solaris*     -> KERNEL="Solaris" ELIBC="Solaris"

A way to achieve this is proposed by Mike Frysinger [4]_, which
suggests to have a env-map file, for instance filled with:

::

    % cat env-map
    *-linux-* KERNEL=linux
    *-gnu ELIBC=glibc
    x86_64-* ARCH=amd64

then the following bash script can be used to set the four variables to
their correct values:

::

    % cat readmap 
    #!/bin/bash 
    
    CBUILD=${CBUILD:-${CHOST=${CHOST:-$1}}} 
    [[ -z ${CHOST} ]] && echo need chost 
    
    unset KERNEL ELIBC ARCH 
    
    while read LINE ; do 
        set -- ${LINE} 
        targ=$1 
        shift 
        [[ ${CBUILD} == ${targ} ]] && eval $@ 
    done < env-map 
    
    echo ARCH=${ARCH} KERNEL=${KERNEL} ELIBC=${ELIBC}

Given the example env-map file, this script would result in:

::

    % ./readmap x86_64-pc-linux-gnu 
    ARCH=amd64 KERNEL=linux ELIBC=glibc

It should be noted, however, that the bash script is a proof of concept
implementation.  It cannot be used as Portage will need this
information, which is written in Python.  Hence, an equivalent of this
bash script should be written in Python to be able to use it within
Portage.  This is considered to be a non-issue coding wise.


References
==========

.. [1] GLEP 22, New "keyword" system to incorporate various
   userlands/kernels/archs, Goodyear,
   (http://glep.gentoo.org/glep-0022.html)

.. [2] For example in the perl ebuild, it is necessary to
   fill in the C-library part, which on a FreeBSD system is other than
   on a Linux system and currently is handled as follows:
   ::
   
   [[ ${ELIBC} == "FreeBSD" ]] && myconf="${myconf} -Dlibc=/usr/lib/libc.a"

.. [3] For the purpose of readability, we will refer to 1, 2 and
   4-tuples, even though tuple in itself suggest a field consisting of
   two values.  For clarity: a 1-tuple describes a single value field,
   while a 4-tuple decribes a field consisting out of four values.

.. [4] mailto:vapier [at) gentoo .dot org


Copyright
=========

This document has been placed in the public domain.
1	GLEP: 47
2	Title: Creating 'safe' environment variables
3	Version: $Revision: 1.8 $
4	Last-Modified: $Date: 2006/01/26 20:56:55 $
5	Author: Diego Pettenò, Fabian Groffen <{flameeyes,grobian}@gentoo.org>
6	Status: Active
7	Type: Standards Track
8	Content-Type: text/x-rst
9	Created: 14-Oct-2005
10	Post-History:
11
12
13	Credits
14	=======
15
16	The text of this GLEP is a result of a discussion and input of the
17	following persons, in no particular order: Mike Frysinger, Diego
18	Pettenò, Fabian Groffen and Finn Thain.
19
20
21	Abstract
22	========
23
24	In order for ebuilds and eclasses to be able to make host specific
25	decisions, it is necessary to have a number of environmental variables
26	which allow for such decisions. This GLEP introduces some measures that
27	need to be made to make these decisions 'safe', by making sure the
28	variables the decisions are based on are 'safe'. A small overlap with
29	GLEP 22 [1]_ is being handled in this GLEP where the use of 2-tuple
30	keywords are being kept instead of 4-tuple keywords. Additionally, the
31	``ELIBC``, ``KERNEL`` and ``ARCH`` get auto filled starting from
32	``CHOST`` and the 2-tuple keyword, instead of solely from they 4-tuple
33	keyword as proposed in GLEP 22.
34
35	The destiny of the ``USERLAND`` variable is out of the scope of this
36	GLEP. Depending on its presence in the tree, it may be decided to set
37	this variable the same way we propose to set ``ELIBC``, ``KERNEL`` and
38	``ARCH``, or alternatively, e.g. via the profiles.
39
40
41	Motivation
42	==========
43
44	The Gentoo/Alt project is in an emerging state to get ready to serve a
45	plethora of 'alternative' configurations such as FreeBSD, NetBSD,
46	DragonflyBSD, GNU/kFreeBSD, Mac OS X, (Open)Darwin, (Open)Solaris and so
47	on. As such, the project is in need for a better grip on the actual
48	host being built on. This information on the host environment is
49	necessary to make proper (automated) decisions on settings that are
50	highly dependant on the build environment, such as platform or as in
51	[2]_.
52
53
54	Rationale
55	=========
56
57	Gentoo's unique Portage system allows easy installation of applications
58	from source packages. Compiling sources is prone to many environmental
59	settings and availability of certain tools. Only recently the Gentoo
60	for FreeBSD project has started, as second Gentoo project that operates
61	on a foreign host operating system using foreign (non-GNU) C-libraries
62	and userland utilities. Such projects suffer from the current implicit
63	assumption made within Gentoo Portage's ebuilds that there is a single
64	type of operating system, C-libraries and system utilities. In order to
65	enable ebuilds -- and also eclasses -- to be aware of these
66	environmental differences, information regarding it should be supplied.
67	Since decisions based on this information can be vital, it is of high
68	importance that this information can be trusted and the values can be
69	considered 'safe' and correct.
70
71
72	Backwards Compatibility
73	=======================
74
75	The proposed keywording scheme in this GLEP is fully compatible with the
76	current situation of the portage tree, this in contrast to GLEP 22. The
77	variables provided by GLEP 22 can't be extracted from the new keyword,
78	but since GLEP 22-style keywords aren't in the tree at the moment, that
79	is not a problem. The same information can be extracted from the CHOST
80	variable, if necessary. No modifications to ebuilds will have to be
81	made.
82
83
84	Specification
85	=============
86
87	Unlike GLEP 22 the current keyword scheme as used in practice is not
88	changed. Instead of proposing a 4-tuple [3]_ keyword, a 2-tuple
89	keyword is chosen for archs that require them. Archs for which a
90	1-tuple keyword suffices, keep that keyword. Since this doesn't change
91	anything to the current situation in the tree, it is considered to be a
92	big advantage over the 4-tuple keyword from GLEP 22. This GLEP is an
93	official specification of the syntax of the keyword.
94
95	Keywords will consist out of two parts separated by a hyphen ('-'). The
96	left hand part of the keyword 2-tuple is the architecture, such as
97	ppc64, sparc and x86. The right hand part indicates the operating
98	system or distribution, such as linux, macos, darwin, obsd, etc. If the
99	right hand part is omitted, it implies the operating system/distribution
100	type is GNU/Linux. In such case the hyphen is also omitted. Examples
101	of such keywords are ppc-darwin and x86. This is fully compatible with
102	the current keywords used in the tree.
103
104	The variables ``ELIBC``, ``KERNEL`` and ``ARCH`` are currently set in
105	the profiles when other than their defaults for a GNU/Linux system.
106	They can as such easily be overridden and defined by the user. To
107	prevent this from happening, the variables should be auto filled by
108	Portage itself, based on the ``CHOST`` variable.
109
110	A map file can be used to have the various ``CHOST`` values being
111	translated to the correct values for the four variables. This change is
112	invisible for ebuilds and eclasses, but allows to rely on these
113	variables as they are based on a 'safe' value -- the ``CHOST`` variable.
114	Ebuilds should not be sensitive to the keyword value, but use the
115	aforementioned four variables instead. They allow specific tests for
116	properties. If this is undesirable, the full ``CHOST`` variable can be
117	used to match a complete operating system.
118
119	Current USE-expansion is being maintained, for backwards compatibility.
120	Since the expansion is based on the variables mentioned above which do
121	not change, but only in the way they are generated, there should be no
122	problem in maintaining them.
123
124
125	Variables
126	---------
127
128	The ``ELIBC``, ``KERNEL``, ``ARCH`` variables are filled from a profile
129	file. The file can be overlaid, such that the following entries in the
130	map file (on the left of the arrow) will result in the assigned
131	variables on the right hand side of the arrow:
132
133	::
134
135	--linux-* -> KERNEL="linux"
136	--*-gnu -> ELIBC="glibc"
137	--kfreebsd-gnu -> KERNEL="FreeBSD" ELIBC="glibc"
138	--freebsd* -> KERNEL="FreeBSD" ELIBC="FreeBSD"
139	--darwin* -> KERNEL="Darwin" ELIBC="Darwin"
140	--netbsd* -> KERNEL="NetBSD" ELIBC="NetBSD"
141	--solaris* -> KERNEL="Solaris" ELIBC="Solaris"
142
143	A way to achieve this is proposed by Mike Frysinger [4]_, which
144	suggests to have a env-map file, for instance filled with:
145
146	::
147
148	% cat env-map
149	-linux- KERNEL=linux
150	*-gnu ELIBC=glibc
151	x86_64-* ARCH=amd64
152
153	then the following bash script can be used to set the four variables to
154	their correct values:
155
156	::
157
158	% cat readmap
159	#!/bin/bash
160
161	CBUILD=${CBUILD:-${CHOST=${CHOST:-$1}}}
162	[[ -z ${CHOST} ]] && echo need chost
163
164	unset KERNEL ELIBC ARCH
165
166	while read LINE ; do
167	set -- ${LINE}
168	targ=$1
169	shift
170	[[ ${CBUILD} == ${targ} ]] && eval $@
171	done < env-map
172
173	echo ARCH=${ARCH} KERNEL=${KERNEL} ELIBC=${ELIBC}
174
175	Given the example env-map file, this script would result in:
176
177	::
178
179	% ./readmap x86_64-pc-linux-gnu
180	ARCH=amd64 KERNEL=linux ELIBC=glibc
181
182	It should be noted, however, that the bash script is a proof of concept
183	implementation. It cannot be used as Portage will need this
184	information, which is written in Python. Hence, an equivalent of this
185	bash script should be written in Python to be able to use it within
186	Portage. This is considered to be a non-issue coding wise.
187
188
189	References
190	==========
191
192	.. [1] GLEP 22, New "keyword" system to incorporate various
193	userlands/kernels/archs, Goodyear,
194	(http://glep.gentoo.org/glep-0022.html)
195
196	.. [2] For example in the perl ebuild, it is necessary to
197	fill in the C-library part, which on a FreeBSD system is other than
198	on a Linux system and currently is handled as follows:
199	::
200
201	[[ ${ELIBC} == "FreeBSD" ]] && myconf="${myconf} -Dlibc=/usr/lib/libc.a"
202
203	.. [3] For the purpose of readability, we will refer to 1, 2 and
204	4-tuples, even though tuple in itself suggest a field consisting of
205	two values. For clarity: a 1-tuple describes a single value field,
206	while a 4-tuple decribes a field consisting out of four values.
207
208	.. [4] mailto:vapier [at) gentoo .dot org
209
210
211
212	Copyright
213	=========
214
215	This document has been placed in the public domain.