en/glep/glep-0055.txt

GLEP: 55
Title: Use EAPI-suffixed ebuilds (.ebuild-EAPI)
Version: $Revision: 1.5 $
Last-Modified: $Date: 2009/05/17 20:56:53 $
Author: Piotr Jaroszyński <peper@gentoo.org>
Status: Rejected
Type: Standards Track
Content-Type: text/x-rst
Created: 17-Dec-2007
Post-History: 17-Dec-2007, 22-Dec-2007, 17-May-2009

  "A little learning is a dangerous thing; drink deep, or taste not the Pierian
  spring: there shallow draughts intoxicate the brain, and drinking largely
  sobers us again."

  -- Alexander Pope, An Essay on Criticism

Status
======

This GLEP was voted down by the Council in its meeting on 2010-08-23.
The Council rejected it again in its meeting on 2012-05-08, in favour
of parsing the EAPI from the bash assignment statement in ebuilds.

Abstract
========

This GLEP proposes usage of EAPI-suffixed file extensions for ebuilds (for
example, foo-1.2.3.ebuild-1).

Problem
=======

The current way of specifying the EAPI in ebuilds is flawed. In order to get the
EAPI the package manager needs to source the ebuild, which itself needs the EAPI
in the first place. Otherwise it imposes a serious limitation, namely every ebuild,
using any of the future EAPIs, will have to be source'able by old package
managers and hence there is no way to do any of the following:

  *  Change the behaviour of inherit in any way (for example, to extend or change
     eclass functionality).

  *  Add new global scope functions in any sane way.

  *  Extend versioning rules in an EAPI - for example, addition of the scm
     suffix - GLEP54 [#GLEP54]_ or allowing more sensible version formats like
     ``1-rc1``, ``1-alpha`` etc. to match upstream more closely.

  *  Use newer bash features.


Current behaviour
=================

Following subsections show what happens if you introduce any of the mentioned
changes in an ebuild and try to install it with portage 2.1.6.13.

Incompatible change of inherit (e.g. make it look in the package dir too)
-------------------------------------------------------------------------

``sys-apps/foo-1.ebuild``::

 EAPI="5"
 inherit "foo"

 DESCRIPTION=""
 HOMEPAGE=""
 SRC_URI=""
 ...

Result::

 *
 * ERROR: sys-apps/foo-1 failed.
 * Call stack:
 *               ebuild.sh, line 1879:  Called _source_ebuild
 *               ebuild.sh, line 1818:  Called source '/var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild'
 *            foo-1.ebuild, line    6:  Called inherit 'foo'
 *               ebuild.sh, line 1218:  Called die
 * The specific snippet of code:
 *              [ ! -e "$location" ] && die "${1}.eclass could not be found by inherit()"
 *  The die message:
 *   foo.eclass could not be found by inherit()
 *
 * If you need support, post the topmost build error, and the call stack if relevant.
 * This ebuild is from an overlay: '/var/lib/gentoo/repositories/peper/'
 *

 !!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
 !!! One of the following masked packages is required to complete your request:
 - sys-apps/foo-1 (masked by: corruption)

Current portage looks for eclasses only in the ``eclass`` directory of a
repository. This results in a fatal error and ebuild being masked by corruption
- might be pretty confusing to users.

New global scope function
-------------------------

``sys-apps/foo-1.ebuild``::

 EAPI="5"
 new_global_scope_function "foo"

 DESCRIPTION=""
 HOMEPAGE=""
 SRC_URI=""
 ...

Result::

 /var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 7: new_global_scope_function: command not found

 !!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
 !!! One of the following masked packages is required to complete your request:
 - sys-apps/foo-1 (masked by: EAPI 5)

 The current version of portage supports EAPI '2'. You must upgrade to a
 newer version of portage before EAPI masked packages can be installed.

Not that bad as user is advised to upgrade portage.

New version format
------------------

``sys-apps/foo-2-rc1.ebuild``::

 Invalid ebuild name: /var/lib/gentoo/repositories/peper/sys-apps/foo/foo-2-rc1.ebuild

 emerge: there are no ebuilds to satisfy "sys-apps/foo"

Not the best error message, especially if there are lots of them.

Use newer bash features
-----------------------

``|&`` is a new type of redirection added in bash-4. It cannot be used even in
local scope as bash still parses the whole ebuild.

``sys-apps/foo-1.ebuild``::

 EAPI="5"

 foo() {
    echo "foo" |& cat
 }

Result::

 /var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 8: syntax error near unexpected token `&'
 /var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 8: ` echo "foo" |& cat'
 *
 * ERROR: sys-apps/foo-1 failed.
 * Call stack:
 *               ebuild.sh, line 1879:  Called _source_ebuild
 *               ebuild.sh, line 1818:  Called die
 * The specific snippet of code:
 *      source "${EBUILD}" || die "error sourcing ebuild"
 *  The die message:
 *   error sourcing ebuild
 *
 * If you need support, post the topmost build error, and the call stack if relevant.
 * This ebuild is from an overlay: '/var/lib/gentoo/repositories/peper/'
 *                                                                                                                                                                                                       ... done!

 !!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
 !!! One of the following masked packages is required to complete your request:
 - sys-apps/foo-1 (masked by: corruption)

Again, not the best error.

Abstract solution
=================

A solution to this problem has to lift those limitations and the only way to do
it is to make the EAPI of an ebuild available to the package managers in a way
that doesn't require them to source the ebuild. Another important requirement is
for the solution to be backward compatible, which has the pleasant side-effect
of making the solution applicable in the Gentoo tree right away. Opposed to
waiting an arbitrary amount of time, which is never long enough anyway, as the
issues listed on the common portage problems page - [#PortageProblems]_ - show.

Proposed solution
=================

The proposed solution is to use EAPI-suffixed file extensions for ebuilds. This
allows package managers to trivially read the EAPI from the ebuild filename. It
is also backwards compatible, because currently ebuilds are recognised by the
``.ebuild`` file extension and hence EAPI-suffixed ebuilds are simply ignored by
the package managers.


Specification
=============

Ebuild filename extension syntax: ``ebuild[-<EAPI>]``, where ``[]`` denotes an
optional part, and ``<EAPI>`` is the EAPI of the ebuild.

The EAPI used by the ebuild is the EAPI included in the filename if it is set.
Otherwise the EAPI set inside the ebuild is used, which defaults to 0 (this is
the current behaviour).

Ebuilds with unsupported EAPIs are masked.

It should be considered an error to set the EAPI both in the filename and in the
ebuild.

Examples:

  *  ``pkg-1.ebuild``, no EAPI set inside the ebuild
       EAPI defaults to 0.

  *  ``pkg-2.ebuild-1``, no EAPI set inside the ebuild
       EAPI 1 is used.

  *  ``pkg-3.ebuild-1``, ``EAPI="1"``
       EAPI set in both places - error.

Note that it is still not permitted to have more than one ebuild with equal
category, package name, and version. Although it would have the advantage of
allowing authors to provide backwards compatible ebuilds, it would introduce
problems too. The first is the requirement to have strict EAPI ordering, the
second is ensuring that all the ebuilds for a single category/package-version
are equivalent, i.e. installing any of them has exactly the same effect on a
given system.

Also note that it is not a new restriction. It is already possible to illegally
have multiple versions with different EAPIs as e.g. ``1.0 == 1.00 == 1.00-r0``
and hence you could have ``foo-1.0.ebuild`` with EAPI X and ``foo-1.00.ebuild``
with EAPI Y.

Summary of ideas
================

EAPI-suffixed ebuilds (proposed solution)
-----------------------------------------

Properties:
 * Can be used right away: yes
 * Hurts performance: no

Some say it is clear and simple, others that it is ugly and unintuitive.

EAPI in the filename with one-time extension change
---------------------------------------------------

One of the proposed filename formats:
``<PKG>-<VER>.eapi-<EAPI>.eb``

Properties:
 * Can be used right away: yes
 * Hurts performance: no

This is equivalent to the proposed solution.

Some say it is better because the extension is static.

Easily fetchable EAPI inside the ebuild
---------------------------------------

Properties:
 * Can be used right away: no
 * Hurts performance: yes

Cannot be used right away as it would trigger the errors shown in Current
behaviour section for old package managers.

Performance decrease comes from the fact that with version format changes in the
picture package managers need EAPI to parse the ebuild's version. That means that merely
picking the best version of a package requires loading EAPI (from cache or the
ebuild) for each available ebuild.

Here is more or less how the package manager figures out the best available
version for a package with N versions available.

 * EAPI in the filename

   * Read the directory containing the package - readdir()
   * For each ebuild, read its EAPI and using that parse its version - no I/O
   * Sort the versions - no I/O
   * Going down from the highest to the lowest version

     * Get the metadata from cache - 2 x stat() + read()
     * break if the version is visible

 * EAPI in the ebuild

   * Read the directory containing the package - readdir()
   * For each ebuild load its metadata from cache to get its EAPI - N x (2 x stat() + read())
   * Sort the versions - no I/O
   * Going down from the highest to the lowest version

     * (metadata is already loaded) - no I/O
     * break if the version is visible - no I/O

The difference is in for how many versions the package manager needs to hit
cache. With EAPI in the ebuild it needs to do that for all versions, with EAPI
in the filename it depends on versions visibility.
For example, package foo has versions 1, 2, 3, 4, 5 and 6. 6 is masked, 5 is
~arch and 1,2,3 and 4 are arch. Say, the user accepts only arch for this
package. With EAPI in the filename it will read metadata only for versions
6, 5 and 4. With EAPI in the ebuild it needs to load metadata for all versions.

It's hard to say what's the avarage case, but surely the worst case scenario
(when only the lowest version is visible) is uncommon.

Easily fetchable EAPI inside the ebuild and one-time extension change
---------------------------------------------------------------------

Properties:
 * Can be used right away: yes
 * Hurts performance: yes

Performance decrease as described in the previous section.

Some say it is clear and simple, others that it is confusing and unintuitive,
because of the arbitrary format restrictions in what is a bash script otherwise.

Use different subdirectories for different EAPIs, i.e. cat/pkg/eapiX/
---------------------------------------------------------------------

Properties:
 * Can be used right away: yes
 * Hurts performance: yes

Performance decrease comes from the fact that it adds several more directory
reads.

Some say that it makes it much harder for maintainers to see what they have.

References
==========

.. [#GLEP54] GLEP 54, scm package version suffix
    (http://glep.gentoo.org/glep-0054.html)

.. [#PortageProblems] Common portage problems
    (http://www.gentoo.org/proj/en/portage/doc/common-problems.xml)

Copyright
=========

This document has been placed in the public domain.

.. vim: set tw=80 fileencoding=utf-8 spell spelllang=en et :
1	GLEP: 55
2	Title: Use EAPI-suffixed ebuilds (.ebuild-EAPI)
3	Version: $Revision: 1.5 $
4	Last-Modified: $Date: 2009/05/17 20:56:53 $
5	Author: Piotr Jaroszyński <peper@gentoo.org>
6	Status: Rejected
7	Type: Standards Track
8	Content-Type: text/x-rst
9	Created: 17-Dec-2007
10	Post-History: 17-Dec-2007, 22-Dec-2007, 17-May-2009
11
12	"A little learning is a dangerous thing; drink deep, or taste not the Pierian
13	spring: there shallow draughts intoxicate the brain, and drinking largely
14	sobers us again."
15
16	-- Alexander Pope, An Essay on Criticism
17
18	Status
19	======
20
21	This GLEP was voted down by the Council in its meeting on 2010-08-23.
22	The Council rejected it again in its meeting on 2012-05-08, in favour
23	of parsing the EAPI from the bash assignment statement in ebuilds.
24
25	Abstract
26	========
27
28	This GLEP proposes usage of EAPI-suffixed file extensions for ebuilds (for
29	example, foo-1.2.3.ebuild-1).
30
31	Problem
32	=======
33
34	The current way of specifying the EAPI in ebuilds is flawed. In order to get the
35	EAPI the package manager needs to source the ebuild, which itself needs the EAPI
36	in the first place. Otherwise it imposes a serious limitation, namely every ebuild,
37	using any of the future EAPIs, will have to be source'able by old package
38	managers and hence there is no way to do any of the following:
39
40	* Change the behaviour of inherit in any way (for example, to extend or change
41	eclass functionality).
42
43	* Add new global scope functions in any sane way.
44
45	* Extend versioning rules in an EAPI - for example, addition of the scm
46	suffix - GLEP54 [#GLEP54]_ or allowing more sensible version formats like
47	``1-rc1``, ``1-alpha`` etc. to match upstream more closely.
48
49	* Use newer bash features.
50
51
52	Current behaviour
53	=================
54
55	Following subsections show what happens if you introduce any of the mentioned
56	changes in an ebuild and try to install it with portage 2.1.6.13.
57
58	Incompatible change of inherit (e.g. make it look in the package dir too)
59	-------------------------------------------------------------------------
60
61	``sys-apps/foo-1.ebuild``::
62
63	EAPI="5"
64	inherit "foo"
65
66	DESCRIPTION=""
67	HOMEPAGE=""
68	SRC_URI=""
69	...
70
71	Result::
72
73	*
74	* ERROR: sys-apps/foo-1 failed.
75	* Call stack:
76	* ebuild.sh, line 1879: Called _source_ebuild
77	* ebuild.sh, line 1818: Called source '/var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild'
78	* foo-1.ebuild, line 6: Called inherit 'foo'
79	* ebuild.sh, line 1218: Called die
80	* The specific snippet of code:
81	* [ ! -e "$location" ] && die "${1}.eclass could not be found by inherit()"
82	* The die message:
83	* foo.eclass could not be found by inherit()
84	*
85	* If you need support, post the topmost build error, and the call stack if relevant.
86	* This ebuild is from an overlay: '/var/lib/gentoo/repositories/peper/'
87	*
88
89	!!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
90	!!! One of the following masked packages is required to complete your request:
91	- sys-apps/foo-1 (masked by: corruption)
92
93	Current portage looks for eclasses only in the ``eclass`` directory of a
94	repository. This results in a fatal error and ebuild being masked by corruption
95	- might be pretty confusing to users.
96
97	New global scope function
98	-------------------------
99
100	``sys-apps/foo-1.ebuild``::
101
102	EAPI="5"
103	new_global_scope_function "foo"
104
105	DESCRIPTION=""
106	HOMEPAGE=""
107	SRC_URI=""
108	...
109
110	Result::
111
112	/var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 7: new_global_scope_function: command not found
113
114	!!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
115	!!! One of the following masked packages is required to complete your request:
116	- sys-apps/foo-1 (masked by: EAPI 5)
117
118	The current version of portage supports EAPI '2'. You must upgrade to a
119	newer version of portage before EAPI masked packages can be installed.
120
121	Not that bad as user is advised to upgrade portage.
122
123	New version format
124	------------------
125
126	``sys-apps/foo-2-rc1.ebuild``::
127
128	Invalid ebuild name: /var/lib/gentoo/repositories/peper/sys-apps/foo/foo-2-rc1.ebuild
129
130	emerge: there are no ebuilds to satisfy "sys-apps/foo"
131
132	Not the best error message, especially if there are lots of them.
133
134	Use newer bash features
135	-----------------------
136
137	``\|&`` is a new type of redirection added in bash-4. It cannot be used even in
138	local scope as bash still parses the whole ebuild.
139
140	``sys-apps/foo-1.ebuild``::
141
142	EAPI="5"
143
144	foo() {
145	echo "foo" \|& cat
146	}
147
148	Result::
149
150	/var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 8: syntax error near unexpected token `&'
151	/var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 8: ` echo "foo" \|& cat'
152	*
153	* ERROR: sys-apps/foo-1 failed.
154	* Call stack:
155	* ebuild.sh, line 1879: Called _source_ebuild
156	* ebuild.sh, line 1818: Called die
157	* The specific snippet of code:
158	* source "${EBUILD}" \|\| die "error sourcing ebuild"
159	* The die message:
160	* error sourcing ebuild
161	*
162	* If you need support, post the topmost build error, and the call stack if relevant.
163	* This ebuild is from an overlay: '/var/lib/gentoo/repositories/peper/'
164	* ... done!
165
166	!!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
167	!!! One of the following masked packages is required to complete your request:
168	- sys-apps/foo-1 (masked by: corruption)
169
170	Again, not the best error.
171
172	Abstract solution
173	=================
174
175	A solution to this problem has to lift those limitations and the only way to do
176	it is to make the EAPI of an ebuild available to the package managers in a way
177	that doesn't require them to source the ebuild. Another important requirement is
178	for the solution to be backward compatible, which has the pleasant side-effect
179	of making the solution applicable in the Gentoo tree right away. Opposed to
180	waiting an arbitrary amount of time, which is never long enough anyway, as the
181	issues listed on the common portage problems page - [#PortageProblems]_ - show.
182
183	Proposed solution
184	=================
185
186	The proposed solution is to use EAPI-suffixed file extensions for ebuilds. This
187	allows package managers to trivially read the EAPI from the ebuild filename. It
188	is also backwards compatible, because currently ebuilds are recognised by the
189	``.ebuild`` file extension and hence EAPI-suffixed ebuilds are simply ignored by
190	the package managers.
191
192
193	Specification
194	=============
195
196	Ebuild filename extension syntax: ``ebuild[-<EAPI>]``, where ``[]`` denotes an
197	optional part, and ``<EAPI>`` is the EAPI of the ebuild.
198
199	The EAPI used by the ebuild is the EAPI included in the filename if it is set.
200	Otherwise the EAPI set inside the ebuild is used, which defaults to 0 (this is
201	the current behaviour).
202
203	Ebuilds with unsupported EAPIs are masked.
204
205	It should be considered an error to set the EAPI both in the filename and in the
206	ebuild.
207
208	Examples:
209
210	* ``pkg-1.ebuild``, no EAPI set inside the ebuild
211	EAPI defaults to 0.
212
213	* ``pkg-2.ebuild-1``, no EAPI set inside the ebuild
214	EAPI 1 is used.
215
216	* ``pkg-3.ebuild-1``, ``EAPI="1"``
217	EAPI set in both places - error.
218
219	Note that it is still not permitted to have more than one ebuild with equal
220	category, package name, and version. Although it would have the advantage of
221	allowing authors to provide backwards compatible ebuilds, it would introduce
222	problems too. The first is the requirement to have strict EAPI ordering, the
223	second is ensuring that all the ebuilds for a single category/package-version
224	are equivalent, i.e. installing any of them has exactly the same effect on a
225	given system.
226
227	Also note that it is not a new restriction. It is already possible to illegally
228	have multiple versions with different EAPIs as e.g. ``1.0 == 1.00 == 1.00-r0``
229	and hence you could have ``foo-1.0.ebuild`` with EAPI X and ``foo-1.00.ebuild``
230	with EAPI Y.
231
232	Summary of ideas
233	================
234
235	EAPI-suffixed ebuilds (proposed solution)
236	-----------------------------------------
237
238	Properties:
239	* Can be used right away: yes
240	* Hurts performance: no
241
242	Some say it is clear and simple, others that it is ugly and unintuitive.
243
244	EAPI in the filename with one-time extension change
245	---------------------------------------------------
246
247	One of the proposed filename formats:
248	``<PKG>-<VER>.eapi-<EAPI>.eb``
249
250	Properties:
251	* Can be used right away: yes
252	* Hurts performance: no
253
254	This is equivalent to the proposed solution.
255
256	Some say it is better because the extension is static.
257
258	Easily fetchable EAPI inside the ebuild
259	---------------------------------------
260
261	Properties:
262	* Can be used right away: no
263	* Hurts performance: yes
264
265	Cannot be used right away as it would trigger the errors shown in Current
266	behaviour section for old package managers.
267
268	Performance decrease comes from the fact that with version format changes in the
269	picture package managers need EAPI to parse the ebuild's version. That means that merely
270	picking the best version of a package requires loading EAPI (from cache or the
271	ebuild) for each available ebuild.
272
273	Here is more or less how the package manager figures out the best available
274	version for a package with N versions available.
275
276	* EAPI in the filename
277
278	* Read the directory containing the package - readdir()
279	* For each ebuild, read its EAPI and using that parse its version - no I/O
280	* Sort the versions - no I/O
281	* Going down from the highest to the lowest version
282
283	* Get the metadata from cache - 2 x stat() + read()
284	* break if the version is visible
285
286	* EAPI in the ebuild
287
288	* Read the directory containing the package - readdir()
289	* For each ebuild load its metadata from cache to get its EAPI - N x (2 x stat() + read())
290	* Sort the versions - no I/O
291	* Going down from the highest to the lowest version
292
293	* (metadata is already loaded) - no I/O
294	* break if the version is visible - no I/O
295
296	The difference is in for how many versions the package manager needs to hit
297	cache. With EAPI in the ebuild it needs to do that for all versions, with EAPI
298	in the filename it depends on versions visibility.
299	For example, package foo has versions 1, 2, 3, 4, 5 and 6. 6 is masked, 5 is
300	~arch and 1,2,3 and 4 are arch. Say, the user accepts only arch for this
301	package. With EAPI in the filename it will read metadata only for versions
302	6, 5 and 4. With EAPI in the ebuild it needs to load metadata for all versions.
303
304	It's hard to say what's the avarage case, but surely the worst case scenario
305	(when only the lowest version is visible) is uncommon.
306
307	Easily fetchable EAPI inside the ebuild and one-time extension change
308	---------------------------------------------------------------------
309
310	Properties:
311	* Can be used right away: yes
312	* Hurts performance: yes
313
314	Performance decrease as described in the previous section.
315
316	Some say it is clear and simple, others that it is confusing and unintuitive,
317	because of the arbitrary format restrictions in what is a bash script otherwise.
318
319	Use different subdirectories for different EAPIs, i.e. cat/pkg/eapiX/
320	---------------------------------------------------------------------
321
322	Properties:
323	* Can be used right away: yes
324	* Hurts performance: yes
325
326	Performance decrease comes from the fact that it adds several more directory
327	reads.
328
329	Some say that it makes it much harder for maintainers to see what they have.
330
331	References
332	==========
333
334	.. [#GLEP54] GLEP 54, scm package version suffix
335	(http://glep.gentoo.org/glep-0054.html)
336
337	.. [#PortageProblems] Common portage problems
338	(http://www.gentoo.org/proj/en/portage/doc/common-problems.xml)
339
340	Copyright
341	=========
342
343	This document has been placed in the public domain.
344
345	.. vim: set tw=80 fileencoding=utf-8 spell spelllang=en et :