1 | GLEP: 33 |
1 | GLEP: 33 |
2 | Title: Eclass Restructure/Redesign |
2 | Title: Eclass Restructure/Redesign |
3 | Version: $Revision: 1.3 $ |
3 | Version: $Revision: 1.4 $ |
4 | Last-Modified: $Date: 2005/03/06 20:39:19 $ |
4 | Last-Modified: $Date: 2005/09/15 02:37:38 $ |
5 | Author: Brian Harring <ferringb@gentoo.org>, John Mylchreest <johnm@gentoo.org> |
5 | Author: Brian Harring <ferringb@gentoo.org>, John Mylchreest <johnm@gentoo.org> |
6 | Status: Draft |
6 | Status: Draft |
7 | Type: Standards Track |
7 | Type: Standards Track |
8 | Content-Type: text/x-rst |
8 | Content-Type: text/x-rst |
9 | Created: 29-Jan-2005 |
9 | Created: 29-Jan-2005 |
… | |
… | |
36 | ======================== |
36 | ======================== |
37 | |
37 | |
38 | Eclasses within the tree currently are a bit of a mess- they're forced to |
38 | Eclasses within the tree currently are a bit of a mess- they're forced to |
39 | maintain backwards compatibility w/ all previous functionality. In effect, |
39 | maintain backwards compatibility w/ all previous functionality. In effect, |
40 | their api is constant, and can only be added to- never changing the existing |
40 | their api is constant, and can only be added to- never changing the existing |
41 | functionality. This obviously is quite limiting, and leads to cruft accruing in |
41 | functionality. This obviously is quite limiting, and leads to cruft accruing in |
42 | eclasses as a eclasses design is refined. This needs to be dealt with prior to |
42 | eclasses as a eclasses design is refined. This needs to be dealt with prior to |
43 | eclass code reaching a critical mass where they become unmanageable/fragile |
43 | eclass code reaching a critical mass where they become unmanageable/fragile |
44 | (recent pushes for eclass versioning could be interpreted as proof of this). |
44 | (recent pushes for eclass versioning could be interpreted as proof of this). |
45 | |
45 | |
46 | Beyond that, eclasses were originally intended as a method to allow for ebuilds |
46 | Beyond that, eclasses were originally intended as a method to allow for ebuilds |
47 | to use a pre-existing block of code, rather then having to duplicate the code in |
47 | to use a pre-existing block of code, rather then having to duplicate the code in |
48 | each ebuild. This is a good thing, but there are ill effects that result from |
48 | each ebuild. This is a good thing, but there are ill effects that result from |
49 | the current design. Eclasses inherit other eclasses to get a single function- in |
49 | the current design. Eclasses inherit other eclasses to get a single function- in |
50 | doing so, modifying the the exported 'template' (default src_compile, default |
50 | doing so, modifying the the exported 'template' (default src_compile, default |
51 | src_unpack, various vars, etc). All the eclass designer was after was reusing a |
51 | src_unpack, various vars, etc). All the eclass designer was after was reusing a |
52 | function, not making their eclass sensitive to changes in the template of the |
52 | function, not making their eclass sensitive to changes in the template of the |
53 | eclass it's inheriting. The eclass designer -should- be aware of changes in the |
53 | eclass it's inheriting. The eclass designer -should- be aware of changes in the |
54 | function they're using, but shouldn't have to worry about their default src_* |
54 | function they're using, but shouldn't have to worry about their default src_* |
55 | and pkg_* functions being overwritten, let alone the env changes. |
55 | and pkg_* functions being overwritten, let alone the env changes. |
56 | |
56 | |
… | |
… | |
61 | rather then requiring them to be aware of what phase of eclass changes is in |
61 | rather then requiring them to be aware of what phase of eclass changes is in |
62 | progress. |
62 | progress. |
63 | |
63 | |
64 | By rolling all changes into one large change, a line is intentionally drawn in |
64 | By rolling all changes into one large change, a line is intentionally drawn in |
65 | the sand. Old eclasses allowed for this, behaved this way. New eclasses allow |
65 | the sand. Old eclasses allowed for this, behaved this way. New eclasses allow |
66 | for that, and behave this way. This should reduce misconceptions about what is |
66 | for that, and behave this way. This should reduce misconceptions about what is |
67 | allowed/possible with eclasses, thus reducing bugs that result from said |
67 | allowed/possible with eclasses, thus reducing bugs that result from said |
68 | misconceptions. |
68 | misconceptions. |
69 | |
69 | |
70 | A few words on elibs- think of them as a clear definition between behavioral |
70 | A few words on elibs- think of them as a clear definition between behavioral |
71 | functionality of an eclass, and the library functionality. Eclass's modify |
71 | functionality of an eclass, and the library functionality. Eclass's modify |
… | |
… | |
74 | |
74 | |
75 | Consider the majority of the portage bin/* scripts- these all are candidates for |
75 | Consider the majority of the portage bin/* scripts- these all are candidates for |
76 | being added to the tree as elibs, as is the bulk of eutils. |
76 | being added to the tree as elibs, as is the bulk of eutils. |
77 | |
77 | |
78 | |
78 | |
79 | Specification. |
79 | Specification |
80 | ============== |
80 | ============= |
81 | |
81 | |
82 | The various parts of this proposal are broken down into a set of changes and |
82 | The various parts of this proposal are broken down into a set of changes and |
83 | elaborations on why a proposed change is preferable. It's advisable to the |
83 | elaborations on why a proposed change is preferable. It's advisable to the |
84 | reader that this be read serially, rather then jumping around. |
84 | reader that this be read serially, rather then jumping around. |
85 | |
85 | |
… | |
… | |
108 | initialization of the library that is absolutely needed. Additionally, they |
108 | initialization of the library that is absolutely needed. Additionally, they |
109 | cannot modify any ebuild template functions- src_compile, src_unpack. Since they are |
109 | cannot modify any ebuild template functions- src_compile, src_unpack. Since they are |
110 | required to not modify the metadata keys, nor in any way affect the ebuild aside |
110 | required to not modify the metadata keys, nor in any way affect the ebuild aside |
111 | from providing functionality, they can be conditionally pulled in. They also |
111 | from providing functionality, they can be conditionally pulled in. They also |
112 | are allowed to pull in other elibs, but strictly just elibs- no eclasses, just |
112 | are allowed to pull in other elibs, but strictly just elibs- no eclasses, just |
113 | other elibs. A real world example would be the eutils eclass. |
113 | other elibs. A real world example would be the eutils eclass. |
114 | |
114 | |
115 | Portage, since the elib's don't modify metadata, isn't required to track elibs |
115 | Portage, since the elib's don't modify metadata, isn't required to track elibs |
116 | as it tracks eclasses. Thus a change in an elib doesn't result in half the tree |
116 | as it tracks eclasses. Thus a change in an elib doesn't result in half the tree |
117 | forced to be regenerated/marked stale when changed (this is more of an infra |
117 | forced to be regenerated/marked stale when changed (this is more of an infra |
118 | benefit, although regen's that take too long due to eclass changes have been |
118 | benefit, although regen's that take too long due to eclass changes have been |
119 | known to cause rsync issues due to missing timestamps). |
119 | known to cause rsync issues due to missing timestamps). |
120 | |
120 | |
121 | Elibs will not be available in the global scope of an eclass, or ebuild- nor during the |
121 | Elibs will not be available in the global scope of an eclass, or ebuild- nor during the |
122 | depends phase (basically a phase that sources the ebuild, to get it's metadata). Elib |
122 | depends phase (basically a phase that sources the ebuild, to get its metadata). Elib |
123 | calls in the global scope will be tracked, but the elib will not be loaded till just before |
123 | calls in the global scope will be tracked, but the elib will not be loaded till just before |
124 | the setup phase (pkg_setup). There are two reasons for this- first, it ensures elibs are |
124 | the setup phase (pkg_setup). There are two reasons for this- first, it ensures elibs are |
125 | completely incapable of modifying metadata. There is no room for confusion, late loading |
125 | completely incapable of modifying metadata. There is no room for confusion, late loading |
126 | of elibs gives you the functionality for all phases, except for depends- depends being the |
126 | of elibs gives you the functionality for all phases, except for depends- depends being the |
127 | only phase that is capable of specifying metadata. Second, as an added bonus, late |
127 | only phase that is capable of specifying metadata. Second, as an added bonus, late |
128 | loading reduces the amount of bash sourced for a regen- faster regens. This however is minor, |
128 | loading reduces the amount of bash sourced for a regen- faster regens. This however is minor, |
129 | and is an ancillary benefit of the first reason. |
129 | and is an ancillary benefit of the first reason. |
… | |
… | |
148 | fi |
148 | fi |
149 | |
149 | |
150 | |
150 | |
151 | Regarding maintainability of elibs, it should be a less of a load then old |
151 | Regarding maintainability of elibs, it should be a less of a load then old |
152 | eclasses. One of the major issues with old eclasses is that their functions are |
152 | eclasses. One of the major issues with old eclasses is that their functions are |
153 | quite incestuous- they're bound tightly to the env they're defined in. This |
153 | quite incestuous- they're bound tightly to the env they're defined in. This |
154 | makes eclass functions a bit fragile- the restrictions on what can, and cannot |
154 | makes eclass functions a bit fragile- the restrictions on what can, and cannot |
155 | be done in elibs will address this, making functionality less fragile (thus a |
155 | be done in elibs will address this, making functionality less fragile (thus a |
156 | bit more maintainable). |
156 | bit more maintainable). |
157 | |
157 | |
158 | There is no need for backwards compatibility with elibs- they just must work |
158 | There is no need for backwards compatibility with elibs- they just must work |
… | |
… | |
169 | |
169 | |
170 | The reduced role of Eclasses, and a clarification of existing Eclass requirements |
170 | The reduced role of Eclasses, and a clarification of existing Eclass requirements |
171 | --------------------------------------------------------------------------------- |
171 | --------------------------------------------------------------------------------- |
172 | |
172 | |
173 | Since elibs are now intended on holding common bash functionality, the focus of |
173 | Since elibs are now intended on holding common bash functionality, the focus of |
174 | eclasses should be in defining an appropriate template for ebuilds. For example, |
174 | eclasses should be in defining an appropriate template for ebuilds. For example, |
175 | defining common DEPENDS, RDEPENDS, src_compile functions, src_unpack, etc. |
175 | defining common DEPENDS, RDEPENDS, src_compile functions, src_unpack, etc. |
176 | Additionally, eclasses should pull in any elibs they need for functionality. |
176 | Additionally, eclasses should pull in any elibs they need for functionality. |
177 | |
177 | |
178 | Eclass functionality that isn't directly related to the metadata, or src_* and |
178 | Eclass functionality that isn't directly related to the metadata, or src_* and |
179 | pkg_* funcs should be shifted into elibs to allow for maximal code reuse. This |
179 | pkg_* funcs should be shifted into elibs to allow for maximal code reuse. This |
… | |
… | |
216 | |
216 | |
217 | |
217 | |
218 | The end of backwards compatibility... |
218 | The end of backwards compatibility... |
219 | ------------------------------------- |
219 | ------------------------------------- |
220 | |
220 | |
221 | With current eclasses, once the eclass is in use, it's api can no longer be |
221 | With current eclasses, once the eclass is in use, its api can no longer be |
222 | changed, nor can the eclass ever be removed from the tree. This is why we still |
222 | changed, nor can the eclass ever be removed from the tree. This is why we still |
223 | have *ancient* eclasses that are completely unused sitting in the tree, for |
223 | have *ancient* eclasses that are completely unused sitting in the tree, for |
224 | example inherit.eclass . The reason for this, not surprisingly is a portage |
224 | example inherit.eclass. The reason for this, not surprisingly, is a portage |
225 | deficiency- on unmerging an installed ebuild, portage used the eclass from the |
225 | deficiency: on unmerging an installed ebuild, portage used the eclass from the |
226 | current tree. |
226 | current tree. |
227 | |
227 | |
228 | For a real world example of this, if you merged a glibc 2 years back, whatever |
228 | For a real world example of this, if you merged a glibc 2 years back, whatever |
229 | eclasses it used must still be compatible, or you may not be able to unmerge the |
229 | eclasses it used must still be compatible, or you may not be able to unmerge the |
230 | older glibc version during an upgrade to a newer version. So either the glibc |
230 | older glibc version during an upgrade to a newer version. So either the glibc |
… | |
… | |
266 | |
266 | |
267 | In other words, these new eclasses would in effect, be old eclasses since older |
267 | In other words, these new eclasses would in effect, be old eclasses since older |
268 | portage versions could still access them. |
268 | portage versions could still access them. |
269 | |
269 | |
270 | |
270 | |
271 | Tree restructuring. |
271 | Tree restructuring |
272 | ------------------- |
272 | ------------------ |
273 | |
273 | |
274 | There are only two way to block the existing (as of this writing) inherit |
274 | There are only two way to block the existing (as of this writing) inherit |
275 | functionality from accessing the new eclasses- either change the extension of |
275 | functionality from accessing the new eclasses- either change the extension of |
276 | eclasses to something other then 'eclass', or to have them stored in a separate |
276 | eclasses to something other then 'eclass', or to have them stored in a separate |
277 | subdirectory of the tree then eclass. |
277 | subdirectory of the tree then eclass. |
… | |
… | |
287 | eclasses, please reread the previous section. It's unfortunately a requirement |
287 | eclasses, please reread the previous section. It's unfortunately a requirement |
288 | to take advantage of all that the next major portage release will allow. |
288 | to take advantage of all that the next major portage release will allow. |
289 | |
289 | |
290 | The proposed directory structure is ${PORTDIR}/include/{eclass,elib}. |
290 | The proposed directory structure is ${PORTDIR}/include/{eclass,elib}. |
291 | Something like ${PORTDIR}/new-eclass, or ${PORTDIR}/eclass-ng could be used |
291 | Something like ${PORTDIR}/new-eclass, or ${PORTDIR}/eclass-ng could be used |
292 | (although many would cringe at the -ng), but such a name is unwise. Consider the |
292 | (although many would cringe at the -ng), but such a name is unwise. Consider the |
293 | possibility (likely a fact) that new eclasses someday may be found lacking, and |
293 | possibility (likely a fact) that new eclasses someday may be found lacking, and |
294 | refined further (version three as it were). Or perhaps we want to add yet more |
294 | refined further (version three as it were). Or perhaps we want to add yet more |
295 | functionality with direct relation to sourcing new files, and we would then need |
295 | functionality with direct relation to sourcing new files, and we would then need |
296 | to further populate ${PORTDIR}. |
296 | to further populate ${PORTDIR}. |
297 | |
297 | |
298 | The new-eclass directory will be (at least) 2 levels deep- for example: |
298 | The new-eclass directory will be (at least) 2 levels deep- for example: |
299 | |
299 | |
… | |
… | |
315 | to ensure no files are missing, and that nothing has been tainted. |
315 | to ensure no files are missing, and that nothing has been tainted. |
316 | |
316 | |
317 | The elib directory will be structured in the same way, for the same reasons. |
317 | The elib directory will be structured in the same way, for the same reasons. |
318 | |
318 | |
319 | Repoman will have to be extended to work within new eclass and elib groups, and |
319 | Repoman will have to be extended to work within new eclass and elib groups, and |
320 | to handle signing and committing. This is intentional, and a good thing. This |
320 | to handle signing and committing. This is intentional, and a good thing. This |
321 | gives repoman the possibility of doing sanity checks on elibs/new eclasses. |
321 | gives repoman the possibility of doing sanity checks on elibs/new eclasses. |
322 | |
322 | |
323 | Note these checks will not prevent developers from doing dumb things with eclass- |
323 | Note these checks will not prevent developers from doing dumb things with eclass- |
324 | these checks would only be capable of doing basic sanity checks, such as syntax checks. |
324 | these checks would only be capable of doing basic sanity checks, such as syntax checks. |
325 | There is no way to prevent people from doing dumb things (exempting perhaps repeated |
325 | There is no way to prevent people from doing dumb things (exempting perhaps repeated |
… | |
… | |
331 | --------------------------------------------------------- |
331 | --------------------------------------------------------- |
332 | |
332 | |
333 | As clarified above, new eclasses will exist in a separate directory that will be |
333 | As clarified above, new eclasses will exist in a separate directory that will be |
334 | intentionally inaccessible to the inherit function. As such, users of older |
334 | intentionally inaccessible to the inherit function. As such, users of older |
335 | portage versions *will* have to upgrade to merge any ebuild that uses elibs/new |
335 | portage versions *will* have to upgrade to merge any ebuild that uses elibs/new |
336 | eclasses. A depend on the next major portage version would transparently handle |
336 | eclasses. A depend on the next major portage version would transparently handle |
337 | this for rsync users. |
337 | this for rsync users. |
338 | |
338 | |
339 | There still is the issue of users who haven't upgraded to the required portage |
339 | There still is the issue of users who haven't upgraded to the required portage |
340 | version. This is a minor concern frankly- portage releases include new |
340 | version. This is a minor concern frankly- portage releases include new |
341 | functionality, and bug fixes. If they won't upgrade, it's assumed they have |
341 | functionality, and bug fixes. If they won't upgrade, it's assumed they have |
342 | their reasons and are big boys, thus able to handle the complications themselves. |
342 | their reasons and are big boys, thus able to handle the complications themselves. |
343 | |
343 | |
344 | The real issue is broken envs, whether in binpkgs, or for installed packages. |
344 | The real issue is broken envs, whether in binpkgs, or for installed packages. |
345 | Two options exist- either the old eclasses are left in the tree indefinitely, or |
345 | Two options exist- either the old eclasses are left in the tree indefinitely, or |
… | |
… | |
381 | |
381 | |
382 | Note for this to happen requires either rather... unwise uses of root, or significant |
382 | Note for this to happen requires either rather... unwise uses of root, or significant |
383 | fs corruption. Regardless of the cause, it's quite likely for this to even become an |
383 | fs corruption. Regardless of the cause, it's quite likely for this to even become an |
384 | issue, the system's vdb is completely unusable. It's a moot issue at that point. |
384 | issue, the system's vdb is completely unusable. It's a moot issue at that point. |
385 | If you lose your vdb, or it gets seriously damaged, it's akin to lobotomizing portage- |
385 | If you lose your vdb, or it gets seriously damaged, it's akin to lobotomizing portage- |
386 | it doesn't know what's installed, it doesn't know of it's own files, and in general, |
386 | it doesn't know what's installed, it doesn't know of its own files, and in general, |
387 | a rebuilding of the system is about the only sane course of action. The missing env is |
387 | a rebuilding of the system is about the only sane course of action. The missing env is |
388 | truly the least of the users concern in such a case. |
388 | truly the least of the users concern in such a case. |
389 | |
389 | |
390 | Continuing with the more likely scenario, users unwilling to upgrade portage will |
390 | Continuing with the more likely scenario, users unwilling to upgrade portage will |
391 | *not* be left out in the rain. Merging the old eclass compat ebuild will provide |
391 | *not* be left out in the rain. Merging the old eclass compat ebuild will provide |
392 | the missing eclasses, thus providing that lost functionality . |
392 | the missing eclasses, thus providing that lost functionality. |
393 | |
393 | |
394 | Note the intention isn't to force them to upgrade, hence the ability to restore the |
394 | Note the intention isn't to force them to upgrade, hence the ability to restore the |
395 | lost functionality. The intention is to clean up the existing mess, and allow us |
395 | lost functionality. The intention is to clean up the existing mess, and allow us |
396 | to move forward. The saying "you've got to break a few eggs to make an omelet" |
396 | to move forward. The saying "you've got to break a few eggs to make an omelet" |
397 | is akin, exempting the fact we're providing a way to make the eggs whole again |
397 | is akin, exempting the fact we're providing a way to make the eggs whole again |
398 | (the king's men would've loved such an option). |
398 | (the king's men would've loved such an option). |
399 | |
399 | |
400 | |
400 | |
401 | Migrating to the new setup |
401 | Migrating to the new setup |
… | |
… | |
417 | infrastructure server that generates the cache for rsync users will have to |
417 | infrastructure server that generates the cache for rsync users will have to |
418 | either be upgraded to a version of portage supporting new eclasses, or patched. |
418 | either be upgraded to a version of portage supporting new eclasses, or patched. |
419 | The former being much more preferable then the latter for the portage devs. |
419 | The former being much more preferable then the latter for the portage devs. |
420 | |
420 | |
421 | Beyond that, an appropriate window for old eclasses to exist in the tree must be |
421 | Beyond that, an appropriate window for old eclasses to exist in the tree must be |
422 | determined, and prior to that window passing an ebuild must be added to the tree |
422 | determined, and prior to that window passing, an ebuild must be added to the tree |
423 | so users can get the old eclasses if needed. |
423 | so users can get the old eclasses if needed. |
424 | |
424 | |
425 | For eclass devs to migrate from old to new, it is possible for them to just |
425 | For eclass devs to migrate from old to new, it is possible for them to just |
426 | transfer the old eclass into an appropriate grouping in the new eclass directory, |
426 | transfer the old eclass into an appropriate grouping in the new eclass directory, |
427 | although it's advisable they cleanse all cruft out of the eclass. You can |
427 | although it's advisable they cleanse all cruft out of the eclass. You can |
428 | migrate ebuilds gradually over to the new eclass, and don't have to worry about |
428 | migrate ebuilds gradually over to the new eclass, and don't have to worry about |
429 | having to support ebuilds from X years back. |
429 | having to support ebuilds from X years back. |
430 | |
430 | |
431 | Essentially, you have a chance to nail the design perfectly/cleanly, and have a |
431 | Essentially, you have a chance to nail the design perfectly/cleanly, and have a |
432 | window in which to redesign it. It's humbly suggested eclass devs take |
432 | window in which to redesign it. It's humbly suggested eclass devs take |
… | |
… | |
444 | |
444 | |
445 | To recap: |
445 | To recap: |
446 | :: |
446 | :: |
447 | |
447 | |
448 | New eclasses and elib functionality will be tied to a specific portage |
448 | New eclasses and elib functionality will be tied to a specific portage |
449 | version. A DEPENDs on said portage version should address this for rsync |
449 | version. A DEPENDs on said portage version should address this for rsync |
450 | users who refuse to upgrade to a portage version that supports the new |
450 | users who refuse to upgrade to a portage version that supports the new |
451 | eclasses/elibs and will gradually be unable to merge ebuilds that use said |
451 | eclasses/elibs and will gradually be unable to merge ebuilds that use said |
452 | functionality. It is their choice to upgrade, as such, the gradual |
452 | functionality. It is their choice to upgrade, as such, the gradual |
453 | 'thinning' of available ebuilds should they block the portage upgrade is |
453 | 'thinning' of available ebuilds should they block the portage upgrade is |
454 | their responsibility. |
454 | their responsibility. |
455 | |
455 | |
456 | Old eclasses at some point in the future should be removed from the tree, |
456 | Old eclasses at some point in the future should be removed from the tree, |
457 | and released in a tarball/ebuild. This will cause installed ebuilds that |
457 | and released in a tarball/ebuild. This will cause installed ebuilds that |
458 | rely on the old eclass to be unable to unmerge, with the same applying for |
458 | rely on the old eclass to be unable to unmerge, with the same applying for |
459 | merging of binpkgs dependent on the following paragraph. |
459 | merging of binpkgs dependent on the following paragraph. |
460 | |
460 | |
461 | The old eclass-compat is only required for users who do not upgrade their |
461 | The old eclass-compat is only required for users who do not upgrade their |
462 | portage installation, and one further exemption- if the user has somehow |
462 | portage installation, and one further exemption- if the user has somehow |