--- xml/htdocs/proj/en/glep/glep-0033.html 2005/09/15 21:04:13 1.4 +++ xml/htdocs/proj/en/glep/glep-0033.html 2010/04/07 22:04:22 1.10 @@ -1,17 +1,12 @@ - + - + GLEP 33 -- Eclass Restructure/Redesign - - + @@ -22,7 +17,7 @@ @@ -32,68 +27,69 @@ - + - + - + - + - +
Title:Eclass Restructure/Redesign
Version:1.5
Version:1.8
Last-Modified:2005/09/15 21:02:11
Last-Modified:2010/04/07 22:04:02
Author:Brian Harring <ferringb at gentoo.org>, John Mylchreest <johnm at gentoo.org>
Status:Approved
Status:Moribund
Type:Standards Track
Content-Type:text/x-rst
Content-Type:text/x-rst
Created:29-Jan-2005
Post-History:29-Jan-2005 6-Mar-2005 15-Sep-2005
Post-History:29-Jan-2005 6-Mar-2005 15-Sep-2005 5-Sep-2006

-

Contents

+

Contents

-

Status

-

Approved by the Gentoo Council on 15 September 2005.

+

Status

+

Approved by the Gentoo Council on 15 September 2005. As of Sept. 2006 +this GLEP is on hold, pending future revisions.

-

Abstract

-

For any design, the transition from theoretical to applied exposes inadequacies -in the original design. This document is intended to document, and propose a +

Abstract

+

For any design, the transition from theoretical to applied exposes inadequacies +in the original design. This document is intended to document, and propose a revision of the current eclass setup to address current eclass inadequacies.

-

This document proposes several things- the creation of ebuild libraries, 'elibs', -a narrowing of the focus of eclasses, a move of eclasses w/in the tree, the +

This document proposes several things- the creation of ebuild libraries, 'elibs', +a narrowing of the focus of eclasses, a move of eclasses w/in the tree, the addition of changelogs, and a way to allow for simple eclass gpg signing. In general, a large scale restructuring of what eclasses are and how they're implemented. Essentially version two of the eclass setup.

-

Terminology

+

Terminology

From this point on, the proposed eclass setup will be called 'new eclasses', the existing crop (as of this writing) will be referenced as 'old eclasses'. The distinction is elaborated on within this document.

-

Motivation and Rationale

+

Motivation and Rationale

Eclasses within the tree currently are a bit of a mess- they're forced to maintain backwards compatibility w/ all previous functionality. In effect, their api is constant, and can only be added to- never changing the existing @@ -114,28 +110,28 @@

Addressing up front why a collection of eclass refinements are being rolled into a single set of changes, parts of this proposal -could- be split into multiple phases. Why do it though? It's simpler for developers to know that the first -eclass specification was this, and that the second specification is that, -rather then requiring them to be aware of what phase of eclass changes is in +eclass specification was this, and that the second specification is that, +rather then requiring them to be aware of what phase of eclass changes is in progress.

By rolling all changes into one large change, a line is intentionally drawn in the sand. Old eclasses allowed for this, behaved this way. New eclasses allow for that, and behave this way. This should reduce misconceptions about what is allowed/possible with eclasses, thus reducing bugs that result from said misconceptions.

-

A few words on elibs- think of them as a clear definition between behavioral -functionality of an eclass, and the library functionality. Eclass's modify -template data, and are the basis for other ebuilds- elibs, however are just +

A few words on elibs- think of them as a clear definition between behavioral +functionality of an eclass, and the library functionality. Eclass's modify +template data, and are the basis for other ebuilds- elibs, however are just common bash functionality.

-

Consider the majority of the portage bin/* scripts- these all are candidates for +

Consider the majority of the portage bin/* scripts- these all are candidates for being added to the tree as elibs, as is the bulk of eutils.

-

Specification

+

Specification

The various parts of this proposal are broken down into a set of changes and elaborations on why a proposed change is preferable. It's advisable to the reader that this be read serially, rather then jumping around.

-

Ebuild Libraries (elibs for short)

+

Ebuild Libraries (elibs for short)

As briefly touched upon in Motivation and Rationale, the original eclass design allowed for the eclass to modify the metadata of an ebuild, metadata being the DEPENDS, RDEPENDS, SRC_URI, IUSE, etc, vars that are required to be constant, @@ -147,12 +143,12 @@

A new directory named elib should be added to the top level of the tree to serve as a repository of ebuild function libraries. Rather then relying on using the source command, an 'elib' function should be added to portage to import that -libraries functionality. The reason for the indirection via the function is -mostly related to portage internals, but it does serve as an abstraction such +libraries functionality. The reason for the indirection via the function is +mostly related to portage internals, but it does serve as an abstraction such that (for example) zsh compatibility hacks could be hidden in the elib function.

Elib's will be collections of bash functions- they're not allowed to do anything in the global scope aside from function definition, and any -minimal- -initialization of the library that is absolutely needed. Additionally, they +initialization of the library that is absolutely needed. Additionally, they cannot modify any ebuild template functions- src_compile, src_unpack. Since they are required to not modify the metadata keys, nor in any way affect the ebuild aside from providing functionality, they can be conditionally pulled in. They also @@ -163,22 +159,22 @@ forced to be regenerated/marked stale when changed (this is more of an infra benefit, although regen's that take too long due to eclass changes have been known to cause rsync issues due to missing timestamps).

-

Elibs will not be available in the global scope of an eclass, or ebuild- nor during the -depends phase (basically a phase that sources the ebuild, to get its metadata). Elib +

Elibs will not be available in the global scope of an eclass, or ebuild- nor during the +depends phase (basically a phase that sources the ebuild, to get its metadata). Elib calls in the global scope will be tracked, but the elib will not be loaded till just before -the setup phase (pkg_setup). There are two reasons for this- first, it ensures elibs are +the setup phase (pkg_setup). There are two reasons for this- first, it ensures elibs are completely incapable of modifying metadata. There is no room for confusion, late loading -of elibs gives you the functionality for all phases, except for depends- depends being the -only phase that is capable of specifying metadata. Second, as an added bonus, late +of elibs gives you the functionality for all phases, except for depends- depends being the +only phase that is capable of specifying metadata. Second, as an added bonus, late loading reduces the amount of bash sourced for a regen- faster regens. This however is minor, and is an ancillary benefit of the first reason.

There are a few further restrictions with elibs--mainly, elibs to load can only be specified -in either global scope, or in the setup, unpack, compile, test, and install phases. You can -not load elibs in prerm, postrm, preinst, and postinst. The reason being, for *rm phases, -installed pkgs will have to look to the tree for the elib, which allows for api drift to cause +in either global scope, or in the setup, unpack, compile, test, and install phases. You can +not load elibs in prerm, postrm, preinst, and postinst. The reason being, for *rm phases, +installed pkgs will have to look to the tree for the elib, which allows for api drift to cause breakage. For *inst phases, same thing, except the culprit is binpkgs.

-

There is a final restriction--elibs cannot change their exported api dependent on the api -(as some eclass do for example). The reason mainly being that elibs are loaded once--not +

There is a final restriction--elibs cannot change their exported api dependent on the api +(as some eclass do for example). The reason mainly being that elibs are loaded once--not multiple times, as eclasses are.

To clarify, for example this is invalid.

@@ -199,13 +195,13 @@
 needs them.  The reasons for this are explained below.

Structuring of the elibs directory will be exactly the same as that of the new eclass directory (detailed below), sans a different extension.

-

As to why their are so many restrictions, the answer is simple- the definition of -what elibs are, what they are capable of, and how to use them is nailed down as much as +

As to why their are so many restrictions, the answer is simple- the definition of +what elibs are, what they are capable of, and how to use them is nailed down as much as possible to avoid any ambiguity related to them. The intention is to make it clear, such that no misconceptions occur, resulting in bugs.

-

The reduced role of Eclasses, and a clarification of existing Eclass requirements

+

The reduced role of Eclasses, and a clarification of existing Eclass requirements

Since elibs are now intended on holding common bash functionality, the focus of eclasses should be in defining an appropriate template for ebuilds. For example, defining common DEPENDS, RDEPENDS, src_compile functions, src_unpack, etc. @@ -228,8 +224,8 @@ package, the parsing of the metadata is changed to address that. Cases where the constant requirement is violated are known, and a select few are allowed- these are exceptions to the rule that are required due to inadequacies in -portage. Any case where it's determined the constant requirement may need to be -violated the dev must make it aware to the majority of devs, along with the portage +portage. Any case where it's determined the constant requirement may need to be +violated the dev must make it aware to the majority of devs, along with the portage devs. This should be done prior to committing.

It's quite likely there is a way to allow what you're attempting- if you just go and do it, the rsync users (our user base) suffer the results of compilation @@ -244,11 +240,11 @@ longer in use.

-

The end of backwards compatibility...

+

The end of backwards compatibility...

With current eclasses, once the eclass is in use, its api can no longer be changed, nor can the eclass ever be removed from the tree. This is why we still have ancient eclasses that are completely unused sitting in the tree, for -example inherit.eclass. The reason for this, not surprisingly, is a portage +example inherit.eclass. The reason for this, not surprisingly, is a portage deficiency: on unmerging an installed ebuild, portage used the eclass from the current tree.

For a real world example of this, if you merged a glibc 2 years back, whatever @@ -267,15 +263,15 @@ they'll use the 'saved' version of the eclass they were built/merged with.

So the backwards compatibility requirement for users of the next major portage version (and beyond) isn't required. All the cruft can be dropped.

-

The problem is that there will be users using older versions of portage that don't -support this functionality- these older installations cannot use the -new eclasses, due to the fact that their portage version is incapable of +

The problem is that there will be users using older versions of portage that don't +support this functionality- these older installations cannot use the +new eclasses, due to the fact that their portage version is incapable of properly relying on the env- in other words, the varying api of the eclass will result in user-visible failures during unmerging.

-

So we're able to do a clean break of all old eclasses, and api cruft, but we need -a means to basically disallow access to the new eclasses for all portage versions +

So we're able to do a clean break of all old eclasses, and api cruft, but we need +a means to basically disallow access to the new eclasses for all portage versions incapable of properly handling the env requirements.

-

Unfortunately, we cannot just rely on a different grouping/naming convention within +

Unfortunately, we cannot just rely on a different grouping/naming convention within the old eclass directory. The new eclasses must be inaccessible, and portage throws a snag into this- the existing inherit function that is used to handle existing eclasses. Basically, whatever it's passed (inherit kernel or inherit @@ -287,7 +283,7 @@ portage versions could still access them.

-

Tree restructuring

+

Tree restructuring

There are only two way to block the existing (as of this writing) inherit functionality from accessing the new eclasses- either change the extension of eclasses to something other then 'eclass', or to have them stored in a separate @@ -296,7 +292,7 @@ eclass directory is already overgrown. Structuring of the new eclass dir (clarified below) will allow for easier signing, ChangeLogs, and grouping of eclasses. New eclasses allow for something akin to a clean break and have new -capabilities/requirements, thus it's advisable to start with a clean directory, +capabilities/requirements, thus it's advisable to start with a clean directory, devoid of all cruft from the old eclass implementation.

If it's unclear as to why the old inherit function cannot access the new eclasses, please reread the previous section. It's unfortunately a requirement @@ -330,18 +326,18 @@

Repoman will have to be extended to work within new eclass and elib groups, and to handle signing and committing. This is intentional, and a good thing. This gives repoman the possibility of doing sanity checks on elibs/new eclasses.

-

Note these checks will not prevent developers from doing dumb things with eclass- +

Note these checks will not prevent developers from doing dumb things with eclass- these checks would only be capable of doing basic sanity checks, such as syntax checks. -There is no way to prevent people from doing dumb things (exempting perhaps repeated +There is no way to prevent people from doing dumb things (exempting perhaps repeated applications of a cattle prod)- these are strictly automatic checks, akin to repoman's dependency checks.

-

The start of a different phase of backwards compatibility

+

The start of a different phase of backwards compatibility

As clarified above, new eclasses will exist in a separate directory that will be intentionally inaccessible to the inherit function. As such, users of older portage versions will have to upgrade to merge any ebuild that uses elibs/new -eclasses. A depend on the next major portage version would transparently handle +eclasses. A depend on the next major portage version would transparently handle this for rsync users.

There still is the issue of users who haven't upgraded to the required portage version. This is a minor concern frankly- portage releases include new @@ -365,29 +361,29 @@ If they specifically block the new portage version, as the ebuilds in the tree migrate to the new eclasses, they will have less and less ebuilds available to them. If they tried injecting the new portage version (lying to portage, -essentially), portage would bail since it cannot find the new eclass. -For ebuilds that use the new eclasses, there really isn't any way to sidestep +essentially), portage would bail since it cannot find the new eclass. +For ebuilds that use the new eclasses, there really isn't any way to sidestep the portage version requirement- same as it has been for other portage features.

What is a bit more annoying is that once the old eclasses are out of the tree, -if a user has not upgraded to a portage version supporting env processing, they +if a user has not upgraded to a portage version supporting env processing, they will lose the ability to unmerge any installed ebuild that used an old -eclass. Same cause, different symptom being they will lose the ability to merge +eclass. Same cause, different symptom being they will lose the ability to merge any tbz2 that uses old eclasses also.

-

There is one additional case that is a rarity, but should be noted- if a user -has suffered significant corruption of their installed package database (vdb). This is +

There is one additional case that is a rarity, but should be noted- if a user +has suffered significant corruption of their installed package database (vdb). This is ignoring the question of whether the vdb is even usable at this point, but the possibility exists for the saved envs to be non usable due to either A) missing, or B) corrupted. In such a case, even with the new portage capabilities, they would need the old eclass compat ebuild.

-

Note for this to happen requires either rather... unwise uses of root, or significant -fs corruption. Regardless of the cause, it's quite likely for this to even become an +

Note for this to happen requires either rather... unwise uses of root, or significant +fs corruption. Regardless of the cause, it's quite likely for this to even become an issue, the system's vdb is completely unusable. It's a moot issue at that point. -If you lose your vdb, or it gets seriously damaged, it's akin to lobotomizing portage- -it doesn't know what's installed, it doesn't know of its own files, and in general, -a rebuilding of the system is about the only sane course of action. The missing env is +If you lose your vdb, or it gets seriously damaged, it's akin to lobotomizing portage- +it doesn't know what's installed, it doesn't know of its own files, and in general, +a rebuilding of the system is about the only sane course of action. The missing env is truly the least of the users concern in such a case.

Continuing with the more likely scenario, users unwilling to upgrade portage will -not be left out in the rain. Merging the old eclass compat ebuild will provide +not be left out in the rain. Merging the old eclass compat ebuild will provide the missing eclasses, thus providing that lost functionality.

Note the intention isn't to force them to upgrade, hence the ability to restore the lost functionality. The intention is to clean up the existing mess, and allow us @@ -396,7 +392,7 @@ (the king's men would've loved such an option).

-

Migrating to the new setup

+

Migrating to the new setup

As has been done in the past whenever a change in the tree results in ebuilds requiring a specific version of portage, as ebuilds migrate to the new eclasses, they should depend on a version of portage that supports it. From the users @@ -425,7 +421,7 @@

-

Backwards Compatibility

+

Backwards Compatibility

All backwards compatibility issues are addressed in line, but a recap is offered- it's suggested that if the a particular compatibility issue is questioned/worried over, the reader read the relevant section. There should be @@ -443,32 +439,31 @@ Old eclasses at some point in the future should be removed from the tree, and released in a tarball/ebuild. This will cause installed ebuilds that -rely on the old eclass to be unable to unmerge, with the same applying for +rely on the old eclass to be unable to unmerge, with the same applying for merging of binpkgs dependent on the following paragraph. - -The old eclass-compat is only required for users who do not upgrade their + +The old eclass-compat is only required for users who do not upgrade their portage installation, and one further exemption- if the user has somehow -corrupted/destroyed their installed pkgs database (/var/db/pkg currently), +corrupted/destroyed their installed pkgs database (/var/db/pkg currently), in the process, they've lost their saved environments. The eclass-compat -ebuild would be required for ebuilds that required older eclasses in such a -case. Note, this case is rare also- as clarified above, it's mentioned +ebuild would be required for ebuilds that required older eclasses in such a +case. Note, this case is rare also- as clarified above, it's mentioned strictly to be complete, it's not much of a real world scenario as elaborated above.

-