/[gentoo]/xml/htdocs/proj/en/glep/glep-0055.txt
Gentoo

Contents of /xml/htdocs/proj/en/glep/glep-0055.txt

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.5 - (hide annotations) (download)
Sun May 17 20:56:53 2009 UTC (5 years, 4 months ago) by peper
Branch: MAIN
Changes since 1.4: +40 -2 lines
File MIME type: text/plain
Add a more detailed description of the performance hit.

1 peper 1.1 GLEP: 55
2     Title: Use EAPI-suffixed ebuilds (.ebuild-EAPI)
3 peper 1.5 Version: $Revision: 1.4 $
4     Last-Modified: $Date: 2009/05/17 17:00:30 $
5 peper 1.1 Author: Piotr JaroszyƄski <peper@gentoo.org>
6     Status: Draft
7     Type: Standards Track
8     Content-Type: text/x-rst
9     Created: 17-Dec-2007
10 peper 1.3 Post-History: 17-Dec-2007, 22-Dec-2007, 17-May-2009
11 peper 1.1
12     "A little learning is a dangerous thing; drink deep, or taste not the Pierian
13     spring: there shallow draughts intoxicate the brain, and drinking largely
14     sobers us again."
15    
16     -- Alexander Pope, An Essay on Criticism
17    
18     Abstract
19     ========
20    
21     This GLEP proposes usage of EAPI-suffixed file extensions for ebuilds (for
22     example, foo-1.2.3.ebuild-1).
23    
24     Problem
25     =======
26    
27     The current way of specifying the EAPI in ebuilds is flawed. In order to get the
28     EAPI the package manager needs to source the ebuild, which itself needs the EAPI
29     in the first place. Otherwise it imposes a serious limitation, namely every ebuild,
30     using any of the future EAPIs, will have to be source'able by old package
31 antarus 1.2 managers and hence there is no way to do any of the following:
32 peper 1.1
33 antarus 1.2 * Change the behaviour of inherit in any way (for example, to extend or change
34 peper 1.1 eclass functionality).
35    
36     * Add new global scope functions in any sane way.
37    
38     * Extend versioning rules in an EAPI - for example, addition of the scm
39 peper 1.3 suffix - GLEP54 [#GLEP54]_ or allowing more sensible version formats like
40     ``1-rc1``, ``1-alpha`` etc. to match upstream more closely.
41 peper 1.1
42 peper 1.4 * Use newer bash features.
43    
44 peper 1.1
45 peper 1.3 Current behaviour
46     =================
47    
48     Following subsections show what happens if you introduce any of the mentioned
49     changes in an ebuild and try to install it with portage 2.1.6.13.
50    
51     Incompatible change of inherit (e.g. make it look in the package dir too)
52     -------------------------------------------------------------------------
53    
54     ``sys-apps/foo-1.ebuild``::
55    
56     EAPI="5"
57     inherit "foo"
58    
59     DESCRIPTION=""
60     HOMEPAGE=""
61     SRC_URI=""
62     ...
63    
64     Result::
65    
66     *
67     * ERROR: sys-apps/foo-1 failed.
68     * Call stack:
69     * ebuild.sh, line 1879: Called _source_ebuild
70     * ebuild.sh, line 1818: Called source '/var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild'
71     * foo-1.ebuild, line 6: Called inherit 'foo'
72     * ebuild.sh, line 1218: Called die
73     * The specific snippet of code:
74     * [ ! -e "$location" ] && die "${1}.eclass could not be found by inherit()"
75     * The die message:
76     * foo.eclass could not be found by inherit()
77     *
78     * If you need support, post the topmost build error, and the call stack if relevant.
79     * This ebuild is from an overlay: '/var/lib/gentoo/repositories/peper/'
80     *
81    
82     !!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
83     !!! One of the following masked packages is required to complete your request:
84     - sys-apps/foo-1 (masked by: corruption)
85    
86     Current portage looks for eclasses only in the ``eclass`` directory of a
87     repository. This results in a fatal error and ebuild being masked by corruption
88     - might be pretty confusing to users.
89    
90     New global scope function
91     -------------------------
92    
93     ``sys-apps/foo-1.ebuild``::
94    
95     EAPI="5"
96     new_global_scope_function "foo"
97    
98     DESCRIPTION=""
99     HOMEPAGE=""
100     SRC_URI=""
101     ...
102    
103     Result::
104    
105     /var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 7: new_global_scope_function: command not found
106    
107     !!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
108     !!! One of the following masked packages is required to complete your request:
109     - sys-apps/foo-1 (masked by: EAPI 5)
110    
111     The current version of portage supports EAPI '2'. You must upgrade to a
112     newer version of portage before EAPI masked packages can be installed.
113    
114     Not that bad as user is advised to upgrade portage.
115    
116     New version format
117     ------------------
118    
119     ``sys-apps/foo-2-rc1.ebuild``::
120    
121     Invalid ebuild name: /var/lib/gentoo/repositories/peper/sys-apps/foo/foo-2-rc1.ebuild
122    
123     emerge: there are no ebuilds to satisfy "sys-apps/foo"
124    
125     Not the best error message, especially if there are lots of them.
126    
127 peper 1.4 Use newer bash features
128     -----------------------
129    
130     ``|&`` is a new type of redirection added in bash-4. It cannot be used even in
131     local scope as bash still parses the whole ebuild.
132    
133     ``sys-apps/foo-1.ebuild``::
134    
135     EAPI="5"
136    
137     foo() {
138     echo "foo" |& cat
139     }
140    
141     Result::
142    
143     /var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 8: syntax error near unexpected token `&'
144     /var/lib/gentoo/repositories/peper/sys-apps/foo/foo-1.ebuild: line 8: ` echo "foo" |& cat'
145     *
146     * ERROR: sys-apps/foo-1 failed.
147     * Call stack:
148     * ebuild.sh, line 1879: Called _source_ebuild
149     * ebuild.sh, line 1818: Called die
150     * The specific snippet of code:
151     * source "${EBUILD}" || die "error sourcing ebuild"
152     * The die message:
153     * error sourcing ebuild
154     *
155     * If you need support, post the topmost build error, and the call stack if relevant.
156     * This ebuild is from an overlay: '/var/lib/gentoo/repositories/peper/'
157     * ... done!
158    
159     !!! All ebuilds that could satisfy "sys-apps/foo" have been masked.
160     !!! One of the following masked packages is required to complete your request:
161     - sys-apps/foo-1 (masked by: corruption)
162    
163     Again, not the best error.
164    
165 peper 1.1 Abstract solution
166     =================
167    
168     A solution to this problem has to lift those limitations and the only way to do
169     it is to make the EAPI of an ebuild available to the package managers in a way
170     that doesn't require them to source the ebuild. Another important requirement is
171     for the solution to be backward compatible, which has the pleasant side-effect
172     of making the solution applicable in the Gentoo tree right away. Opposed to
173     waiting an arbitrary amount of time, which is never long enough anyway, as the
174     issues listed on the common portage problems page - [#PortageProblems]_ - show.
175    
176     Proposed solution
177     =================
178    
179     The proposed solution is to use EAPI-suffixed file extensions for ebuilds. This
180     allows package managers to trivially read the EAPI from the ebuild filename. It
181 antarus 1.2 is also backwards compatible, because currently ebuilds are recognised by the
182 peper 1.1 ``.ebuild`` file extension and hence EAPI-suffixed ebuilds are simply ignored by
183     the package managers.
184    
185    
186     Specification
187     =============
188    
189     Ebuild filename extension syntax: ``ebuild[-<EAPI>]``, where ``[]`` denotes an
190     optional part, and ``<EAPI>`` is the EAPI of the ebuild.
191    
192 peper 1.3 The EAPI used by the ebuild is the EAPI included in the filename if it is set.
193     Otherwise the EAPI set inside the ebuild is used, which defaults to 0 (this is
194     the current behaviour).
195 peper 1.1
196     Ebuilds with unsupported EAPIs are masked.
197    
198 peper 1.3 It should be considered an error to set the EAPI both in the filename and in the
199     ebuild.
200 peper 1.1
201     Examples:
202    
203     * ``pkg-1.ebuild``, no EAPI set inside the ebuild
204 peper 1.3 EAPI defaults to 0.
205 peper 1.1
206     * ``pkg-2.ebuild-1``, no EAPI set inside the ebuild
207     EAPI 1 is used.
208    
209 peper 1.3 * ``pkg-3.ebuild-1``, ``EAPI="1"``
210     EAPI set in both places - error.
211 peper 1.1
212     Note that it is still not permitted to have more than one ebuild with equal
213     category, package name, and version. Although it would have the advantage of
214 antarus 1.2 allowing authors to provide backwards compatible ebuilds, it would introduce
215     problems too. The first is the requirement to have strict EAPI ordering, the
216     second is ensuring that all the ebuilds for a single category/package-version
217     are equivalent, i.e. installing any of them has exactly the same effect on a
218     given system.
219 peper 1.1
220 peper 1.5 Also note that it is not a new restriction. It is already possible to illegally
221     have multiple versions with different EAPIs as e.g. ``1.0 == 1.00 == 1.00-r0``
222     and hence you could have ``foo-1.0.ebuild`` with EAPI X and ``foo-1.00.ebuild``
223     with EAPI Y.
224    
225 peper 1.3 Summary of ideas
226     ================
227    
228     EAPI-suffixed ebuilds (proposed solution)
229     -----------------------------------------
230    
231     Properties:
232     * Can be used right away: yes
233     * Hurts performance: no
234    
235     Some say it is clear and simple, others that it is ugly and unintuitive.
236    
237     EAPI in the filename with one-time extension change
238     ---------------------------------------------------
239    
240     One of the proposed filename formats:
241     ``<PKG>-<VER>.eapi-<EAPI>.eb``
242    
243     Properties:
244     * Can be used right away: yes
245     * Hurts performance: no
246    
247     This is equivalent to the proposed solution.
248    
249     Some say it is better because the extension is static.
250    
251     Easily fetchable EAPI inside the ebuild
252     ---------------------------------------
253    
254     Properties:
255     * Can be used right away: no
256     * Hurts performance: yes
257    
258     Cannot be used right away as it would trigger the errors shown in Current
259     behaviour section for old package managers.
260    
261     Performance decrease comes from the fact that with version format changes in the
262     picture package managers need EAPI to parse the ebuild's version. That means that merely
263     picking the best version of a package requires loading EAPI (from cache or the
264     ebuild) for each available ebuild.
265    
266 peper 1.5 Here is more or less how the package manager figures out the best available
267     version for a package with N versions available.
268    
269     * EAPI in the filename
270    
271     * Read the directory containing the package - readdir()
272     * For each ebuild, read its EAPI and using that parse its version - no I/O
273     * Sort the versions - no I/O
274     * Going down from the highest to the lowest version
275    
276     * Get the metadata from cache - 2 x stat() + read()
277     * break if the version is visible
278    
279     * EAPI in the ebuild
280    
281     * Read the directory containing the package - readdir()
282     * For each ebuild load its metadata from cache to get its EAPI - N x (2 x stat() + read())
283     * Sort the versions - no I/O
284     * Going down from the highest to the lowest version
285    
286     * (metadata is already loaded) - no I/O
287     * break if the version is visible - no I/O
288    
289     The difference is in for how many versions the package manager needs to hit
290     cache. With EAPI in the ebuild it needs to do that for all versions, with EAPI
291     in the filename it depends on versions visibility.
292     For example, package foo has versions 1, 2, 3, 4, 5 and 6. 6 is masked, 5 is
293     ~arch and 1,2,3 and 4 are arch. Say, the user accepts only arch for this
294     package. With EAPI in the filename it will read metadata only for versions
295     6, 5 and 4. With EAPI in the ebuild it needs to load metadata for all versions.
296    
297     It's hard to say what's the avarage case, but surely the worst case scenario
298     (when only the lowest version is visible) is uncommon.
299 peper 1.3
300     Easily fetchable EAPI inside the ebuild and one-time extension change
301     ---------------------------------------------------------------------
302    
303     Properties:
304     * Can be used right away: yes
305     * Hurts performance: yes
306    
307     Performance decrease as described in the previous section.
308    
309     Some say it is clear and simple, others that it is confusing and unintuitive,
310     because of the arbitrary format restrictions in what is a bash script otherwise.
311    
312     Use different subdirectories for different EAPIs, i.e. cat/pkg/eapiX/
313     ---------------------------------------------------------------------
314    
315     Properties:
316     * Can be used right away: yes
317     * Hurts performance: yes
318 peper 1.1
319 peper 1.3 Performance decrease comes from the fact that it adds several more directory
320     reads.
321 peper 1.1
322 peper 1.3 Some say that it makes it much harder for maintainers to see what they have.
323 peper 1.1
324     References
325     ==========
326    
327     .. [#GLEP54] GLEP 54, scm package version suffix
328     (http://glep.gentoo.org/glep-0054.html)
329    
330     .. [#PortageProblems] Common portage problems
331     (http://www.gentoo.org/proj/en/portage/doc/common-problems.xml)
332    
333     Copyright
334     =========
335    
336     This document has been placed in the public domain.
337    
338     .. vim: set tw=80 fileencoding=utf-8 spell spelllang=en et :

  ViewVC Help
Powered by ViewVC 1.1.20