/[gentoo]/xml/htdocs/proj/en/glep/glep-0033.txt
Gentoo

Diff of /xml/htdocs/proj/en/glep/glep-0033.txt

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

Revision 1.2 Revision 1.6
1GLEP: 33 1GLEP: 33
2Title: Eclass Restructure/Redesign 2Title: Eclass Restructure/Redesign
3Version: $Revision: 1.2 $ 3Version: $Revision: 1.6 $
4Last-Modified: $Date: 2005/03/06 20:33:20 $ 4Last-Modified: $Date: 2006/09/05 20:54:30 $
5Author: Brian Harring <ferringb@gentoo.org>, John Mylchreest <johnm@gentoo.org> 5Author: Brian Harring <ferringb@gentoo.org>, John Mylchreest <johnm@gentoo.org>
6Status: Draft 6Status: Moribund
7Type: Standards Track 7Type: Standards Track
8Content-Type: text/x-rst 8Content-Type: text/x-rst
9Created: 29-Jan-2005 9Created: 29-Jan-2005
10Post-History: 29-Jan-2005 10Post-History: 29-Jan-2005 6-Mar-2005 15-Sep-2005 5-Sep-2006
11 11
12Status
13======
14
15Approved by the Gentoo Council on 15 September 2005. As of Sept. 2006
16this GLEP is on hold, pending future revisions.
12 17
13Abstract 18Abstract
14======== 19========
15 20
16For any design, the transition from theoretical to applied exposes inadequacies 21For any design, the transition from theoretical to applied exposes inadequacies
36======================== 41========================
37 42
38Eclasses within the tree currently are a bit of a mess- they're forced to 43Eclasses within the tree currently are a bit of a mess- they're forced to
39maintain backwards compatibility w/ all previous functionality. In effect, 44maintain backwards compatibility w/ all previous functionality. In effect,
40their api is constant, and can only be added to- never changing the existing 45their api is constant, and can only be added to- never changing the existing
41functionality. This obviously is quite limiting, and leads to cruft accruing in 46functionality. This obviously is quite limiting, and leads to cruft accruing in
42eclasses as a eclasses design is refined. This needs to be dealt with prior to 47eclasses as a eclasses design is refined. This needs to be dealt with prior to
43eclass code reaching a critical mass where they become unmanageable/fragile 48eclass code reaching a critical mass where they become unmanageable/fragile
44(recent pushes for eclass versioning could be interpreted as proof of this). 49(recent pushes for eclass versioning could be interpreted as proof of this).
45 50
46Beyond that, eclasses were originally intended as a method to allow for ebuilds 51Beyond that, eclasses were originally intended as a method to allow for ebuilds
47to use a pre-existing block of code, rather then having to duplicate the code in 52to use a pre-existing block of code, rather then having to duplicate the code in
48each ebuild. This is a good thing, but there are ill effects that result from 53each ebuild. This is a good thing, but there are ill effects that result from
49the current design. Eclasses inherit other eclasses to get a single function- in 54the current design. Eclasses inherit other eclasses to get a single function- in
50doing so, modifying the the exported 'template' (default src_compile, default 55doing so, modifying the the exported 'template' (default src_compile, default
51src_unpack, various vars, etc). All the eclass designer was after was reusing a 56src_unpack, various vars, etc). All the eclass designer was after was reusing a
52function, not making their eclass sensitive to changes in the template of the 57function, not making their eclass sensitive to changes in the template of the
53eclass it's inheriting. The eclass designer -should- be aware of changes in the 58eclass it's inheriting. The eclass designer -should- be aware of changes in the
54function they're using, but shouldn't have to worry about their default src_* 59function they're using, but shouldn't have to worry about their default src_*
55and pkg_* functions being overwritten, let alone the env changes. 60and pkg_* functions being overwritten, let alone the env changes.
56 61
61rather then requiring them to be aware of what phase of eclass changes is in 66rather then requiring them to be aware of what phase of eclass changes is in
62progress. 67progress.
63 68
64By rolling all changes into one large change, a line is intentionally drawn in 69By rolling all changes into one large change, a line is intentionally drawn in
65the sand. Old eclasses allowed for this, behaved this way. New eclasses allow 70the sand. Old eclasses allowed for this, behaved this way. New eclasses allow
66for that, and behave this way. This should reduce misconceptions about what is 71for that, and behave this way. This should reduce misconceptions about what is
67allowed/possible with eclasses, thus reducing bugs that result from said 72allowed/possible with eclasses, thus reducing bugs that result from said
68misconceptions. 73misconceptions.
69 74
70A few words on elibs- think of them as a clear definition between behavioral 75A few words on elibs- think of them as a clear definition between behavioral
71functionality of an eclass, and the library functionality. Eclass's modify 76functionality of an eclass, and the library functionality. Eclass's modify
74 79
75Consider the majority of the portage bin/* scripts- these all are candidates for 80Consider the majority of the portage bin/* scripts- these all are candidates for
76being added to the tree as elibs, as is the bulk of eutils. 81being added to the tree as elibs, as is the bulk of eutils.
77 82
78 83
79Specification. 84Specification
80============== 85=============
81 86
82The various parts of this proposal are broken down into a set of changes and 87The various parts of this proposal are broken down into a set of changes and
83elaborations on why a proposed change is preferable. It's advisable to the 88elaborations on why a proposed change is preferable. It's advisable to the
84reader that this be read serially, rather then jumping around. 89reader that this be read serially, rather then jumping around.
85 90
108initialization of the library that is absolutely needed. Additionally, they 113initialization of the library that is absolutely needed. Additionally, they
109cannot modify any ebuild template functions- src_compile, src_unpack. Since they are 114cannot modify any ebuild template functions- src_compile, src_unpack. Since they are
110required to not modify the metadata keys, nor in any way affect the ebuild aside 115required to not modify the metadata keys, nor in any way affect the ebuild aside
111from providing functionality, they can be conditionally pulled in. They also 116from providing functionality, they can be conditionally pulled in. They also
112are allowed to pull in other elibs, but strictly just elibs- no eclasses, just 117are allowed to pull in other elibs, but strictly just elibs- no eclasses, just
113other elibs. A real world example would be the eutils eclass. 118other elibs. A real world example would be the eutils eclass.
114 119
115Portage, since the elib's don't modify metadata, isn't required to track elibs 120Portage, since the elib's don't modify metadata, isn't required to track elibs
116as it tracks eclasses. Thus a change in an elib doesn't result in half the tree 121as it tracks eclasses. Thus a change in an elib doesn't result in half the tree
117forced to be regenerated/marked stale when changed (this is more of an infra 122forced to be regenerated/marked stale when changed (this is more of an infra
118benefit, although regen's that take too long due to eclass changes have been 123benefit, although regen's that take too long due to eclass changes have been
119known to cause rsync issues due to missing timestamps). 124known to cause rsync issues due to missing timestamps).
120 125
121Elibs will not be available in the global scope of an eclass, or ebuild- nor during the 126Elibs will not be available in the global scope of an eclass, or ebuild- nor during the
122depends phase (basically a phase that sources the ebuild, to get it's metadata). Elib 127depends phase (basically a phase that sources the ebuild, to get its metadata). Elib
123calls in the global scope will be tracked, but the elib will not be loaded till just before 128calls in the global scope will be tracked, but the elib will not be loaded till just before
124the setup phase (pkg_setup). There are two reasons for this- first, it ensures elibs are 129the setup phase (pkg_setup). There are two reasons for this- first, it ensures elibs are
125completely incapable of modifying metadata. There is no room for confusion, late loading 130completely incapable of modifying metadata. There is no room for confusion, late loading
126of elibs gives you the functionality for all phases, except for depends- depends being the 131of elibs gives you the functionality for all phases, except for depends- depends being the
127only phase that is capable of specifying metadata. Second, as an added bonus, late 132only phase that is capable of specifying metadata. Second, as an added bonus, late
128loading reduces the amount of bash sourced for a regen- faster regens. This however is minor, 133loading reduces the amount of bash sourced for a regen- faster regens. This however is minor,
129and is an ancillary benefit of the first reason. 134and is an ancillary benefit of the first reason.
130 135
131There are a few further restrictions with elibs- mainly, elibs to load can only be specified 136There are a few further restrictions with elibs--mainly, elibs to load can only be specified
132in either global scope, or in the setup, unpack, compile, test, and install phases. You can 137in either global scope, or in the setup, unpack, compile, test, and install phases. You can
133not load elibs in prerm, postrm, preinst, and postinst. The reason being, for *rm phases, 138not load elibs in prerm, postrm, preinst, and postinst. The reason being, for \*rm phases,
134installed pkgs will have to look to the tree for the elib, which allows for api drift to cause 139installed pkgs will have to look to the tree for the elib, which allows for api drift to cause
135breakage. For *inst phases, same thing, except the culprit is binpkgs. 140breakage. For \*inst phases, same thing, except the culprit is binpkgs.
136 141
137There is a final restriction- elibs cannot change their exported api dependent on the api 142There is a final restriction--elibs cannot change their exported api dependent on the api
138(as some eclass do for example). The reason mainly being that elibs are loaded once- not 143(as some eclass do for example). The reason mainly being that elibs are loaded once--not
139multiple times, as eclasses are. 144multiple times, as eclasses are.
140 145
141To clarify, for example this is invalid. 146To clarify, for example this is invalid.
142:: 147::
148
143 if [[ -n ${SOME_VAR} ]]; then 149 if [[ -n ${SOME_VAR} ]]; then
144 func x() { echo "I'm accessible only via tweaking some var";} 150 func x() { echo "I'm accessible only via tweaking some var";}
145 else 151 else
146 func x() { echo "this is invalid, do not do it."; } 152 func x() { echo "this is invalid, do not do it."; }
147 fi 153 fi
148 154
149 155
150Regarding maintainability of elibs, it should be a less of a load then old 156Regarding maintainability of elibs, it should be a less of a load then old
151eclasses. One of the major issues with old eclasses is that their functions are 157eclasses. One of the major issues with old eclasses is that their functions are
152quite incestuous- they're bound tightly to the env they're defined in. This 158quite incestuous- they're bound tightly to the env they're defined in. This
153makes eclass functions a bit fragile- the restrictions on what can, and cannot 159makes eclass functions a bit fragile- the restrictions on what can, and cannot
154be done in elibs will address this, making functionality less fragile (thus a 160be done in elibs will address this, making functionality less fragile (thus a
155bit more maintainable). 161bit more maintainable).
156 162
157There is no need for backwards compatibility with elibs- they just must work 163There is no need for backwards compatibility with elibs- they just must work
168 174
169The reduced role of Eclasses, and a clarification of existing Eclass requirements 175The reduced role of Eclasses, and a clarification of existing Eclass requirements
170--------------------------------------------------------------------------------- 176---------------------------------------------------------------------------------
171 177
172Since elibs are now intended on holding common bash functionality, the focus of 178Since elibs are now intended on holding common bash functionality, the focus of
173eclasses should be in defining an appropriate template for ebuilds. For example, 179eclasses should be in defining an appropriate template for ebuilds. For example,
174defining common DEPENDS, RDEPENDS, src_compile functions, src_unpack, etc. 180defining common DEPENDS, RDEPENDS, src_compile functions, src_unpack, etc.
175Additionally, eclasses should pull in any elibs they need for functionality. 181Additionally, eclasses should pull in any elibs they need for functionality.
176 182
177Eclass functionality that isn't directly related to the metadata, or src_* and 183Eclass functionality that isn't directly related to the metadata, or src_* and
178pkg_* funcs should be shifted into elibs to allow for maximal code reuse. This 184pkg_* funcs should be shifted into elibs to allow for maximal code reuse. This
215 221
216 222
217The end of backwards compatibility... 223The end of backwards compatibility...
218------------------------------------- 224-------------------------------------
219 225
220With current eclasses, once the eclass is in use, it's api can no longer be 226With current eclasses, once the eclass is in use, its api can no longer be
221changed, nor can the eclass ever be removed from the tree. This is why we still 227changed, nor can the eclass ever be removed from the tree. This is why we still
222have *ancient* eclasses that are completely unused sitting in the tree, for 228have *ancient* eclasses that are completely unused sitting in the tree, for
223example inherit.eclass . The reason for this, not surprisingly is a portage 229example inherit.eclass. The reason for this, not surprisingly, is a portage
224deficiency- on unmerging an installed ebuild, portage used the eclass from the 230deficiency: on unmerging an installed ebuild, portage used the eclass from the
225current tree. 231current tree.
226 232
227For a real world example of this, if you merged a glibc 2 years back, whatever 233For a real world example of this, if you merged a glibc 2 years back, whatever
228eclasses it used must still be compatible, or you may not be able to unmerge the 234eclasses it used must still be compatible, or you may not be able to unmerge the
229older glibc version during an upgrade to a newer version. So either the glibc 235older glibc version during an upgrade to a newer version. So either the glibc
265 271
266In other words, these new eclasses would in effect, be old eclasses since older 272In other words, these new eclasses would in effect, be old eclasses since older
267portage versions could still access them. 273portage versions could still access them.
268 274
269 275
270Tree restructuring. 276Tree restructuring
271------------------- 277------------------
272 278
273There are only two way to block the existing (as of this writing) inherit 279There are only two way to block the existing (as of this writing) inherit
274functionality from accessing the new eclasses- either change the extension of 280functionality from accessing the new eclasses- either change the extension of
275eclasses to something other then 'eclass', or to have them stored in a separate 281eclasses to something other then 'eclass', or to have them stored in a separate
276subdirectory of the tree then eclass. 282subdirectory of the tree then eclass.
286eclasses, please reread the previous section. It's unfortunately a requirement 292eclasses, please reread the previous section. It's unfortunately a requirement
287to take advantage of all that the next major portage release will allow. 293to take advantage of all that the next major portage release will allow.
288 294
289The proposed directory structure is ${PORTDIR}/include/{eclass,elib}. 295The proposed directory structure is ${PORTDIR}/include/{eclass,elib}.
290Something like ${PORTDIR}/new-eclass, or ${PORTDIR}/eclass-ng could be used 296Something like ${PORTDIR}/new-eclass, or ${PORTDIR}/eclass-ng could be used
291(although many would cringe at the -ng), but such a name is unwise. Consider the 297(although many would cringe at the -ng), but such a name is unwise. Consider the
292possibility (likely a fact) that new eclasses someday may be found lacking, and 298possibility (likely a fact) that new eclasses someday may be found lacking, and
293refined further (version three as it were). Or perhaps we want to add yet more 299refined further (version three as it were). Or perhaps we want to add yet more
294functionality with direct relation to sourcing new files, and we would then need 300functionality with direct relation to sourcing new files, and we would then need
295to further populate ${PORTDIR}. 301to further populate ${PORTDIR}.
296 302
297The new-eclass directory will be (at least) 2 levels deep- for example: 303The new-eclass directory will be (at least) 2 levels deep- for example:
298 304
314to ensure no files are missing, and that nothing has been tainted. 320to ensure no files are missing, and that nothing has been tainted.
315 321
316The elib directory will be structured in the same way, for the same reasons. 322The elib directory will be structured in the same way, for the same reasons.
317 323
318Repoman will have to be extended to work within new eclass and elib groups, and 324Repoman will have to be extended to work within new eclass and elib groups, and
319to handle signing and committing. This is intentional, and a good thing. This 325to handle signing and committing. This is intentional, and a good thing. This
320gives repoman the possibility of doing sanity checks on elibs/new eclasses. 326gives repoman the possibility of doing sanity checks on elibs/new eclasses.
321 327
322Note these checks will not prevent developers from doing dumb things with eclass- 328Note these checks will not prevent developers from doing dumb things with eclass-
323these checks would only be capable of doing basic sanity checks, such as syntax checks. 329these checks would only be capable of doing basic sanity checks, such as syntax checks.
324There is no way to prevent people from doing dumb things (exempting perhaps repeated 330There is no way to prevent people from doing dumb things (exempting perhaps repeated
330--------------------------------------------------------- 336---------------------------------------------------------
331 337
332As clarified above, new eclasses will exist in a separate directory that will be 338As clarified above, new eclasses will exist in a separate directory that will be
333intentionally inaccessible to the inherit function. As such, users of older 339intentionally inaccessible to the inherit function. As such, users of older
334portage versions *will* have to upgrade to merge any ebuild that uses elibs/new 340portage versions *will* have to upgrade to merge any ebuild that uses elibs/new
335eclasses. A depend on the next major portage version would transparently handle 341eclasses. A depend on the next major portage version would transparently handle
336this for rsync users. 342this for rsync users.
337 343
338There still is the issue of users who haven't upgraded to the required portage 344There still is the issue of users who haven't upgraded to the required portage
339version. This is a minor concern frankly- portage releases include new 345version. This is a minor concern frankly- portage releases include new
340functionality, and bug fixes. If they won't upgrade, it's assumed they have 346functionality, and bug fixes. If they won't upgrade, it's assumed they have
341their reasons and are big boys, thus able to handle the complications themselves. 347their reasons and are big boys, thus able to handle the complications themselves.
342 348
343The real issue is broken envs, whether in binpkgs, or for installed packages. 349The real issue is broken envs, whether in binpkgs, or for installed packages.
344Two options exist- either the old eclasses are left in the tree indefinitely, or 350Two options exist- either the old eclasses are left in the tree indefinitely, or
380 386
381Note for this to happen requires either rather... unwise uses of root, or significant 387Note for this to happen requires either rather... unwise uses of root, or significant
382fs corruption. Regardless of the cause, it's quite likely for this to even become an 388fs corruption. Regardless of the cause, it's quite likely for this to even become an
383issue, the system's vdb is completely unusable. It's a moot issue at that point. 389issue, the system's vdb is completely unusable. It's a moot issue at that point.
384If you lose your vdb, or it gets seriously damaged, it's akin to lobotomizing portage- 390If you lose your vdb, or it gets seriously damaged, it's akin to lobotomizing portage-
385it doesn't know what's installed, it doesn't know of it's own files, and in general, 391it doesn't know what's installed, it doesn't know of its own files, and in general,
386a rebuilding of the system is about the only sane course of action. The missing env is 392a rebuilding of the system is about the only sane course of action. The missing env is
387truly the least of the users concern in such a case. 393truly the least of the users concern in such a case.
388 394
389Continuing with the more likely scenario, users unwilling to upgrade portage will 395Continuing with the more likely scenario, users unwilling to upgrade portage will
390*not* be left out in the rain. Merging the old eclass compat ebuild will provide 396*not* be left out in the rain. Merging the old eclass compat ebuild will provide
391the missing eclasses, thus providing that lost functionality . 397the missing eclasses, thus providing that lost functionality.
392 398
393Note the intention isn't to force them to upgrade, hence the ability to restore the 399Note the intention isn't to force them to upgrade, hence the ability to restore the
394lost functionality. The intention is to clean up the existing mess, and allow us 400lost functionality. The intention is to clean up the existing mess, and allow us
395to move forward. The saying "you've got to break a few eggs to make an omelet" 401to move forward. The saying "you've got to break a few eggs to make an omelet"
396is akin, exempting the fact we're providing a way to make the eggs whole again 402is akin, exempting the fact we're providing a way to make the eggs whole again
397(the king's men would've loved such an option). 403(the king's men would've loved such an option).
398 404
399 405
400Migrating to the new setup 406Migrating to the new setup
416infrastructure server that generates the cache for rsync users will have to 422infrastructure server that generates the cache for rsync users will have to
417either be upgraded to a version of portage supporting new eclasses, or patched. 423either be upgraded to a version of portage supporting new eclasses, or patched.
418The former being much more preferable then the latter for the portage devs. 424The former being much more preferable then the latter for the portage devs.
419 425
420Beyond that, an appropriate window for old eclasses to exist in the tree must be 426Beyond that, an appropriate window for old eclasses to exist in the tree must be
421determined, and prior to that window passing an ebuild must be added to the tree 427determined, and prior to that window passing, an ebuild must be added to the tree
422so users can get the old eclasses if needed. 428so users can get the old eclasses if needed.
423 429
424For eclass devs to migrate from old to new, it is possible for them to just 430For eclass devs to migrate from old to new, it is possible for them to just
425transfer the old eclass into an appropriate grouping in the new eclass directory, 431transfer the old eclass into an appropriate grouping in the new eclass directory,
426although it's advisable they cleanse all cruft out of the eclass. You can 432although it's advisable they cleanse all cruft out of the eclass. You can
427migrate ebuilds gradually over to the new eclass, and don't have to worry about 433migrate ebuilds gradually over to the new eclass, and don't have to worry about
428having to support ebuilds from X years back. 434having to support ebuilds from X years back.
429 435
430Essentially, you have a chance to nail the design perfectly/cleanly, and have a 436Essentially, you have a chance to nail the design perfectly/cleanly, and have a
431window in which to redesign it. It's humbly suggested eclass devs take 437window in which to redesign it. It's humbly suggested eclass devs take
441a more in depth discussion of the issue, along with a more extensive explanation 447a more in depth discussion of the issue, along with a more extensive explanation
442of the potential solutions, and reasons for the chosen solution. 448of the potential solutions, and reasons for the chosen solution.
443 449
444To recap: 450To recap:
445:: 451::
452
446 New eclasses and elib functionality will be tied to a specific portage 453 New eclasses and elib functionality will be tied to a specific portage
447 version. A DEPENDs on said portage version should address this for rsync 454 version. A DEPENDs on said portage version should address this for rsync
448 users who refuse to upgrade to a portage version that supports the new 455 users who refuse to upgrade to a portage version that supports the new
449 eclasses/elibs and will gradually be unable to merge ebuilds that use said 456 eclasses/elibs and will gradually be unable to merge ebuilds that use said
450 functionality. It is their choice to upgrade, as such, the gradual 457 functionality. It is their choice to upgrade, as such, the gradual
451 'thinning' of available ebuilds should they block the portage upgrade is 458 'thinning' of available ebuilds should they block the portage upgrade is
452 their responsibility. 459 their responsibility.
453 460
454 Old eclasses at some point in the future should be removed from the tree, 461 Old eclasses at some point in the future should be removed from the tree,
455 and released in a tarball/ebuild. This will cause installed ebuilds that 462 and released in a tarball/ebuild. This will cause installed ebuilds that
456 rely on the old eclass to be unable to unmerge, with the same applying for 463 rely on the old eclass to be unable to unmerge, with the same applying for
457 merging of binpkgs dependent on the following paragraph. 464 merging of binpkgs dependent on the following paragraph.
458 465
459 The old eclass-compat is only required for users who do not upgrade their 466 The old eclass-compat is only required for users who do not upgrade their
460 portage installation, and one further exemption- if the user has somehow 467 portage installation, and one further exemption- if the user has somehow

Legend:
Removed from v.1.2  
changed lines
  Added in v.1.6

  ViewVC Help
Powered by ViewVC 1.1.20