en/glep/glep-0055.txt

GLEP: 55
Title: Use EAPI-suffixed ebuilds (.ebuild-EAPI)
Version: $Revision: 1.3 $
Last-Modified: $Date: 2009/05/17 15:50:27 $
Author: Piotr Jaroszyński <peper@gentoo.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 17-Dec-2007
Post-History: 17-Dec-2007, 22-Dec-2007, 17-May-2009

  "A little learning is a dangerous thing; drink deep, or taste not the Pierian
  spring: there shallow draughts intoxicate the brain, and drinking largely
  sobers us again."

  -- Alexander Pope, An Essay on Criticism

Abstract
========

This GLEP proposes usage of EAPI-suffixed file extensions for ebuilds (for
example, foo-1.2.3.ebuild-1).

Problem
=======

The current way of specifying the EAPI in ebuilds is flawed. In order to get the
EAPI the package manager needs to source the ebuild, which itself needs the EAPI
in the first place. Otherwise it imposes a serious limitation, namely every ebuild,
using any of the future EAPIs, will have to be source'able by old package
managers and hence there is no way to do any of the following:

  *  Change the behaviour of inherit in any way (for example, to extend or change
     eclass functionality).

  *  Add new global scope functions in any sane way.

  *  Extend versioning rules in an EAPI - for example, addition of the scm
     suffix - GLEP54 [#GLEP54]_ or allowing more sensible version formats like
     ``1-rc1``, ``1-alpha`` etc. to match upstream more closely.

  *  Use newer bash features.


Current behaviour
=================

Following subsections show what happens if you introduce any of the mentioned
changes in an ebuild and try to install it with portage 2.1.6.13.

Incompatible change of inherit (e.g. make it look in the package dir too)
-------------------------------------------------------------------------

``sys-apps/foo-1.ebuild``::

 EAPI="5"
 inherit "foo"

 DESCRIPTION=""
 HOMEPAGE=""
 SRC_URI=""
 ...

Result::

 *
 * ERROR: sys-apps/foo-1 failed.
 * Call stack:
 *               ebuild.sh, line 1879:  Called _source_ebuild
 *               ebuild.sh, line 1818:  Called source '/var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild'
 *            foo-1.ebuild, line    6:  Called inherit 'foo'
 *               ebuild.sh, line 1218:  Called die
 * The specific snippet of code:
 *              [ ! -e "$location" ] && die "${1}.eclass could not be found by inherit()"
 *  The die message:
 *   foo.eclass could not be found by inherit()
 *
 * If you need support, post the topmost build error, and the call stack if relevant.
 * This ebuild is from an overlay: '/var/lib/gentoo/repositories/peper/'
 *

 !!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
 !!! One of the following masked packages is required to complete your request:
 - sys-apps/foo-1 (masked by: corruption)

Current portage looks for eclasses only in the ``eclass`` directory of a
repository. This results in a fatal error and ebuild being masked by corruption
- might be pretty confusing to users.

New global scope function
-------------------------

``sys-apps/foo-1.ebuild``::

 EAPI="5"
 new_global_scope_function "foo"

 DESCRIPTION=""
 HOMEPAGE=""
 SRC_URI=""
 ...

Result::

 /var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 7: new_global_scope_function: command not found

 !!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
 !!! One of the following masked packages is required to complete your request:
 - sys-apps/foo-1 (masked by: EAPI 5)

 The current version of portage supports EAPI '2'. You must upgrade to a
 newer version of portage before EAPI masked packages can be installed.

Not that bad as user is advised to upgrade portage.

New version format
------------------

``sys-apps/foo-2-rc1.ebuild``::

 Invalid ebuild name: /var/lib/gentoo/repositories/peper/sys-apps/foo/foo-2-rc1.ebuild

 emerge: there are no ebuilds to satisfy "sys-apps/foo"

Not the best error message, especially if there are lots of them.

Use newer bash features
-----------------------

``|&`` is a new type of redirection added in bash-4. It cannot be used even in
local scope as bash still parses the whole ebuild.

``sys-apps/foo-1.ebuild``::

 EAPI="5"

 foo() {
    echo "foo" |& cat
 }

Result::

 /var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 8: syntax error near unexpected token `&'
 /var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 8: ` echo "foo" |& cat'
 *
 * ERROR: sys-apps/foo-1 failed.
 * Call stack:
 *               ebuild.sh, line 1879:  Called _source_ebuild
 *               ebuild.sh, line 1818:  Called die
 * The specific snippet of code:
 *      source "${EBUILD}" || die "error sourcing ebuild"
 *  The die message:
 *   error sourcing ebuild
 *
 * If you need support, post the topmost build error, and the call stack if relevant.
 * This ebuild is from an overlay: '/var/lib/gentoo/repositories/peper/'
 *                                                                                                                                                                                                       ... done!

 !!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
 !!! One of the following masked packages is required to complete your request:
 - sys-apps/foo-1 (masked by: corruption)

Again, not the best error.

Abstract solution
=================

A solution to this problem has to lift those limitations and the only way to do
it is to make the EAPI of an ebuild available to the package managers in a way
that doesn't require them to source the ebuild. Another important requirement is
for the solution to be backward compatible, which has the pleasant side-effect
of making the solution applicable in the Gentoo tree right away. Opposed to
waiting an arbitrary amount of time, which is never long enough anyway, as the
issues listed on the common portage problems page - [#PortageProblems]_ - show.

Proposed solution
=================

The proposed solution is to use EAPI-suffixed file extensions for ebuilds. This
allows package managers to trivially read the EAPI from the ebuild filename. It
is also backwards compatible, because currently ebuilds are recognised by the
``.ebuild`` file extension and hence EAPI-suffixed ebuilds are simply ignored by
the package managers.


Specification
=============

Ebuild filename extension syntax: ``ebuild[-<EAPI>]``, where ``[]`` denotes an
optional part, and ``<EAPI>`` is the EAPI of the ebuild.

The EAPI used by the ebuild is the EAPI included in the filename if it is set.
Otherwise the EAPI set inside the ebuild is used, which defaults to 0 (this is
the current behaviour).

Ebuilds with unsupported EAPIs are masked.

It should be considered an error to set the EAPI both in the filename and in the
ebuild.

Examples:

  *  ``pkg-1.ebuild``, no EAPI set inside the ebuild
       EAPI defaults to 0.

  *  ``pkg-2.ebuild-1``, no EAPI set inside the ebuild
       EAPI 1 is used.

  *  ``pkg-3.ebuild-1``, ``EAPI="1"``
       EAPI set in both places - error.

Note that it is still not permitted to have more than one ebuild with equal
category, package name, and version. Although it would have the advantage of
allowing authors to provide backwards compatible ebuilds, it would introduce
problems too. The first is the requirement to have strict EAPI ordering, the
second is ensuring that all the ebuilds for a single category/package-version
are equivalent, i.e. installing any of them has exactly the same effect on a
given system.

Summary of ideas
================

EAPI-suffixed ebuilds (proposed solution)
-----------------------------------------

Properties:
 * Can be used right away: yes
 * Hurts performance: no

Some say it is clear and simple, others that it is ugly and unintuitive.

EAPI in the filename with one-time extension change
---------------------------------------------------

One of the proposed filename formats:
``<PKG>-<VER>.eapi-<EAPI>.eb``

Properties:
 * Can be used right away: yes
 * Hurts performance: no

This is equivalent to the proposed solution.

Some say it is better because the extension is static.

Easily fetchable EAPI inside the ebuild
---------------------------------------

Properties:
 * Can be used right away: no
 * Hurts performance: yes

Cannot be used right away as it would trigger the errors shown in Current
behaviour section for old package managers.

Performance decrease comes from the fact that with version format changes in the
picture package managers need EAPI to parse the ebuild's version. That means that merely
picking the best version of a package requires loading EAPI (from cache or the
ebuild) for each available ebuild.


Easily fetchable EAPI inside the ebuild and one-time extension change
---------------------------------------------------------------------

Properties:
 * Can be used right away: yes
 * Hurts performance: yes

Performance decrease as described in the previous section.

Some say it is clear and simple, others that it is confusing and unintuitive,
because of the arbitrary format restrictions in what is a bash script otherwise.

Use different subdirectories for different EAPIs, i.e. cat/pkg/eapiX/
---------------------------------------------------------------------

Properties:
 * Can be used right away: yes
 * Hurts performance: yes

Performance decrease comes from the fact that it adds several more directory
reads.

Some say that it makes it much harder for maintainers to see what they have.

References
==========

.. [#GLEP54] GLEP 54, scm package version suffix
    (http://glep.gentoo.org/glep-0054.html)

.. [#PortageProblems] Common portage problems
    (http://www.gentoo.org/proj/en/portage/doc/common-problems.xml)

Copyright
=========

This document has been placed in the public domain.

.. vim: set tw=80 fileencoding=utf-8 spell spelllang=en et :
1	peper	1.1	GLEP: 55
2			Title: Use EAPI-suffixed ebuilds (.ebuild-EAPI)
3	peper	1.4	Version: $Revision: 1.3 $
4			Last-Modified: $Date: 2009/05/17 15:50:27 $
5	peper	1.1	Author: Piotr Jaroszyński <peper@gentoo.org>
6			Status: Draft
7			Type: Standards Track
8			Content-Type: text/x-rst
9			Created: 17-Dec-2007
10	peper	1.3	Post-History: 17-Dec-2007, 22-Dec-2007, 17-May-2009
11	peper	1.1
12			"A little learning is a dangerous thing; drink deep, or taste not the Pierian
13			spring: there shallow draughts intoxicate the brain, and drinking largely
14			sobers us again."
15
16			-- Alexander Pope, An Essay on Criticism
17
18			Abstract
19			========
20
21			This GLEP proposes usage of EAPI-suffixed file extensions for ebuilds (for
22			example, foo-1.2.3.ebuild-1).
23
24			Problem
25			=======
26
27			The current way of specifying the EAPI in ebuilds is flawed. In order to get the
28			EAPI the package manager needs to source the ebuild, which itself needs the EAPI
29			in the first place. Otherwise it imposes a serious limitation, namely every ebuild,
30			using any of the future EAPIs, will have to be source'able by old package
31	antarus	1.2	managers and hence there is no way to do any of the following:
32	peper	1.1
33	antarus	1.2	* Change the behaviour of inherit in any way (for example, to extend or change
34	peper	1.1	eclass functionality).
35
36			* Add new global scope functions in any sane way.
37
38			* Extend versioning rules in an EAPI - for example, addition of the scm
39	peper	1.3	suffix - GLEP54 [#GLEP54]_ or allowing more sensible version formats like
40			``1-rc1``, ``1-alpha`` etc. to match upstream more closely.
41	peper	1.1
42	peper	1.4	* Use newer bash features.
43
44	peper	1.1
45	peper	1.3	Current behaviour
46			=================
47
48			Following subsections show what happens if you introduce any of the mentioned
49			changes in an ebuild and try to install it with portage 2.1.6.13.
50
51			Incompatible change of inherit (e.g. make it look in the package dir too)
52			-------------------------------------------------------------------------
53
54			``sys-apps/foo-1.ebuild``::
55
56			EAPI="5"
57			inherit "foo"
58
59			DESCRIPTION=""
60			HOMEPAGE=""
61			SRC_URI=""
62			...
63
64			Result::
65
66			*
67			* ERROR: sys-apps/foo-1 failed.
68			* Call stack:
69			* ebuild.sh, line 1879: Called _source_ebuild
70			* ebuild.sh, line 1818: Called source '/var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild'
71			* foo-1.ebuild, line 6: Called inherit 'foo'
72			* ebuild.sh, line 1218: Called die
73			* The specific snippet of code:
74			* [ ! -e "$location" ] && die "${1}.eclass could not be found by inherit()"
75			* The die message:
76			* foo.eclass could not be found by inherit()
77			*
78			* If you need support, post the topmost build error, and the call stack if relevant.
79			* This ebuild is from an overlay: '/var/lib/gentoo/repositories/peper/'
80			*
81
82			!!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
83			!!! One of the following masked packages is required to complete your request:
84			- sys-apps/foo-1 (masked by: corruption)
85
86			Current portage looks for eclasses only in the ``eclass`` directory of a
87			repository. This results in a fatal error and ebuild being masked by corruption
88			- might be pretty confusing to users.
89
90			New global scope function
91			-------------------------
92
93			``sys-apps/foo-1.ebuild``::
94
95			EAPI="5"
96			new_global_scope_function "foo"
97
98			DESCRIPTION=""
99			HOMEPAGE=""
100			SRC_URI=""
101			...
102
103			Result::
104
105			/var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 7: new_global_scope_function: command not found
106
107			!!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
108			!!! One of the following masked packages is required to complete your request:
109			- sys-apps/foo-1 (masked by: EAPI 5)
110
111			The current version of portage supports EAPI '2'. You must upgrade to a
112			newer version of portage before EAPI masked packages can be installed.
113
114			Not that bad as user is advised to upgrade portage.
115
116			New version format
117			------------------
118
119			``sys-apps/foo-2-rc1.ebuild``::
120
121			Invalid ebuild name: /var/lib/gentoo/repositories/peper/sys-apps/foo/foo-2-rc1.ebuild
122
123			emerge: there are no ebuilds to satisfy "sys-apps/foo"
124
125			Not the best error message, especially if there are lots of them.
126
127	peper	1.4	Use newer bash features
128			-----------------------
129
130			``\|&`` is a new type of redirection added in bash-4. It cannot be used even in
131			local scope as bash still parses the whole ebuild.
132
133			``sys-apps/foo-1.ebuild``::
134
135			EAPI="5"
136
137			foo() {
138			echo "foo" \|& cat
139			}
140
141			Result::
142
143			/var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 8: syntax error near unexpected token `&'
144			/var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 8: ` echo "foo" \|& cat'
145			*
146			* ERROR: sys-apps/foo-1 failed.
147			* Call stack:
148			* ebuild.sh, line 1879: Called _source_ebuild
149			* ebuild.sh, line 1818: Called die
150			* The specific snippet of code:
151			* source "${EBUILD}" \|\| die "error sourcing ebuild"
152			* The die message:
153			* error sourcing ebuild
154			*
155			* If you need support, post the topmost build error, and the call stack if relevant.
156			* This ebuild is from an overlay: '/var/lib/gentoo/repositories/peper/'
157			* ... done!
158
159			!!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
160			!!! One of the following masked packages is required to complete your request:
161			- sys-apps/foo-1 (masked by: corruption)
162
163			Again, not the best error.
164
165	peper	1.1	Abstract solution
166			=================
167
168			A solution to this problem has to lift those limitations and the only way to do
169			it is to make the EAPI of an ebuild available to the package managers in a way
170			that doesn't require them to source the ebuild. Another important requirement is
171			for the solution to be backward compatible, which has the pleasant side-effect
172			of making the solution applicable in the Gentoo tree right away. Opposed to
173			waiting an arbitrary amount of time, which is never long enough anyway, as the
174			issues listed on the common portage problems page - [#PortageProblems]_ - show.
175
176			Proposed solution
177			=================
178
179			The proposed solution is to use EAPI-suffixed file extensions for ebuilds. This
180			allows package managers to trivially read the EAPI from the ebuild filename. It
181	antarus	1.2	is also backwards compatible, because currently ebuilds are recognised by the
182	peper	1.1	``.ebuild`` file extension and hence EAPI-suffixed ebuilds are simply ignored by
183			the package managers.
184
185
186			Specification
187			=============
188
189			Ebuild filename extension syntax: ``ebuild[-<EAPI>]``, where ``[]`` denotes an
190			optional part, and ``<EAPI>`` is the EAPI of the ebuild.
191
192	peper	1.3	The EAPI used by the ebuild is the EAPI included in the filename if it is set.
193			Otherwise the EAPI set inside the ebuild is used, which defaults to 0 (this is
194			the current behaviour).
195	peper	1.1
196			Ebuilds with unsupported EAPIs are masked.
197
198	peper	1.3	It should be considered an error to set the EAPI both in the filename and in the
199			ebuild.
200	peper	1.1
201			Examples:
202
203			* ``pkg-1.ebuild``, no EAPI set inside the ebuild
204	peper	1.3	EAPI defaults to 0.
205	peper	1.1
206			* ``pkg-2.ebuild-1``, no EAPI set inside the ebuild
207			EAPI 1 is used.
208
209	peper	1.3	* ``pkg-3.ebuild-1``, ``EAPI="1"``
210			EAPI set in both places - error.
211	peper	1.1
212			Note that it is still not permitted to have more than one ebuild with equal
213			category, package name, and version. Although it would have the advantage of
214	antarus	1.2	allowing authors to provide backwards compatible ebuilds, it would introduce
215			problems too. The first is the requirement to have strict EAPI ordering, the
216			second is ensuring that all the ebuilds for a single category/package-version
217			are equivalent, i.e. installing any of them has exactly the same effect on a
218			given system.
219	peper	1.1
220	peper	1.3	Summary of ideas
221			================
222
223			EAPI-suffixed ebuilds (proposed solution)
224			-----------------------------------------
225
226			Properties:
227			* Can be used right away: yes
228			* Hurts performance: no
229
230			Some say it is clear and simple, others that it is ugly and unintuitive.
231
232			EAPI in the filename with one-time extension change
233			---------------------------------------------------
234
235			One of the proposed filename formats:
236			``<PKG>-<VER>.eapi-<EAPI>.eb``
237
238			Properties:
239			* Can be used right away: yes
240			* Hurts performance: no
241
242			This is equivalent to the proposed solution.
243
244			Some say it is better because the extension is static.
245
246			Easily fetchable EAPI inside the ebuild
247			---------------------------------------
248
249			Properties:
250			* Can be used right away: no
251			* Hurts performance: yes
252
253			Cannot be used right away as it would trigger the errors shown in Current
254			behaviour section for old package managers.
255
256			Performance decrease comes from the fact that with version format changes in the
257			picture package managers need EAPI to parse the ebuild's version. That means that merely
258			picking the best version of a package requires loading EAPI (from cache or the
259			ebuild) for each available ebuild.
260
261
262			Easily fetchable EAPI inside the ebuild and one-time extension change
263			---------------------------------------------------------------------
264
265			Properties:
266			* Can be used right away: yes
267			* Hurts performance: yes
268
269			Performance decrease as described in the previous section.
270
271			Some say it is clear and simple, others that it is confusing and unintuitive,
272			because of the arbitrary format restrictions in what is a bash script otherwise.
273
274			Use different subdirectories for different EAPIs, i.e. cat/pkg/eapiX/
275			---------------------------------------------------------------------
276
277			Properties:
278			* Can be used right away: yes
279			* Hurts performance: yes
280	peper	1.1
281	peper	1.3	Performance decrease comes from the fact that it adds several more directory
282			reads.
283	peper	1.1
284	peper	1.3	Some say that it makes it much harder for maintainers to see what they have.
285	peper	1.1
286			References
287			==========
288
289			.. [#GLEP54] GLEP 54, scm package version suffix
290			(http://glep.gentoo.org/glep-0054.html)
291
292			.. [#PortageProblems] Common portage problems
293			(http://www.gentoo.org/proj/en/portage/doc/common-problems.xml)
294
295			Copyright
296			=========
297
298			This document has been placed in the public domain.
299
300			.. vim: set tw=80 fileencoding=utf-8 spell spelllang=en et :