| 1 | GLEP: 33 |
1 | GLEP: 33 |
| 2 | Title: Eclass Restructure/Redesign |
2 | Title: Eclass Restructure/Redesign |
| 3 | Version: $Revision: 1.3 $ |
3 | Version: $Revision: 1.4 $ |
| 4 | Last-Modified: $Date: 2005/03/06 20:39:19 $ |
4 | Last-Modified: $Date: 2005/09/15 02:37:38 $ |
| 5 | Author: Brian Harring <ferringb@gentoo.org>, John Mylchreest <johnm@gentoo.org> |
5 | Author: Brian Harring <ferringb@gentoo.org>, John Mylchreest <johnm@gentoo.org> |
| 6 | Status: Draft |
6 | Status: Draft |
| 7 | Type: Standards Track |
7 | Type: Standards Track |
| 8 | Content-Type: text/x-rst |
8 | Content-Type: text/x-rst |
| 9 | Created: 29-Jan-2005 |
9 | Created: 29-Jan-2005 |
| … | |
… | |
| 36 | ======================== |
36 | ======================== |
| 37 | |
37 | |
| 38 | Eclasses within the tree currently are a bit of a mess- they're forced to |
38 | Eclasses within the tree currently are a bit of a mess- they're forced to |
| 39 | maintain backwards compatibility w/ all previous functionality. In effect, |
39 | maintain backwards compatibility w/ all previous functionality. In effect, |
| 40 | their api is constant, and can only be added to- never changing the existing |
40 | their api is constant, and can only be added to- never changing the existing |
| 41 | functionality. This obviously is quite limiting, and leads to cruft accruing in |
41 | functionality. This obviously is quite limiting, and leads to cruft accruing in |
| 42 | eclasses as a eclasses design is refined. This needs to be dealt with prior to |
42 | eclasses as a eclasses design is refined. This needs to be dealt with prior to |
| 43 | eclass code reaching a critical mass where they become unmanageable/fragile |
43 | eclass code reaching a critical mass where they become unmanageable/fragile |
| 44 | (recent pushes for eclass versioning could be interpreted as proof of this). |
44 | (recent pushes for eclass versioning could be interpreted as proof of this). |
| 45 | |
45 | |
| 46 | Beyond that, eclasses were originally intended as a method to allow for ebuilds |
46 | Beyond that, eclasses were originally intended as a method to allow for ebuilds |
| 47 | to use a pre-existing block of code, rather then having to duplicate the code in |
47 | to use a pre-existing block of code, rather then having to duplicate the code in |
| 48 | each ebuild. This is a good thing, but there are ill effects that result from |
48 | each ebuild. This is a good thing, but there are ill effects that result from |
| 49 | the current design. Eclasses inherit other eclasses to get a single function- in |
49 | the current design. Eclasses inherit other eclasses to get a single function- in |
| 50 | doing so, modifying the the exported 'template' (default src_compile, default |
50 | doing so, modifying the the exported 'template' (default src_compile, default |
| 51 | src_unpack, various vars, etc). All the eclass designer was after was reusing a |
51 | src_unpack, various vars, etc). All the eclass designer was after was reusing a |
| 52 | function, not making their eclass sensitive to changes in the template of the |
52 | function, not making their eclass sensitive to changes in the template of the |
| 53 | eclass it's inheriting. The eclass designer -should- be aware of changes in the |
53 | eclass it's inheriting. The eclass designer -should- be aware of changes in the |
| 54 | function they're using, but shouldn't have to worry about their default src_* |
54 | function they're using, but shouldn't have to worry about their default src_* |
| 55 | and pkg_* functions being overwritten, let alone the env changes. |
55 | and pkg_* functions being overwritten, let alone the env changes. |
| 56 | |
56 | |
| … | |
… | |
| 61 | rather then requiring them to be aware of what phase of eclass changes is in |
61 | rather then requiring them to be aware of what phase of eclass changes is in |
| 62 | progress. |
62 | progress. |
| 63 | |
63 | |
| 64 | By rolling all changes into one large change, a line is intentionally drawn in |
64 | By rolling all changes into one large change, a line is intentionally drawn in |
| 65 | the sand. Old eclasses allowed for this, behaved this way. New eclasses allow |
65 | the sand. Old eclasses allowed for this, behaved this way. New eclasses allow |
| 66 | for that, and behave this way. This should reduce misconceptions about what is |
66 | for that, and behave this way. This should reduce misconceptions about what is |
| 67 | allowed/possible with eclasses, thus reducing bugs that result from said |
67 | allowed/possible with eclasses, thus reducing bugs that result from said |
| 68 | misconceptions. |
68 | misconceptions. |
| 69 | |
69 | |
| 70 | A few words on elibs- think of them as a clear definition between behavioral |
70 | A few words on elibs- think of them as a clear definition between behavioral |
| 71 | functionality of an eclass, and the library functionality. Eclass's modify |
71 | functionality of an eclass, and the library functionality. Eclass's modify |
| … | |
… | |
| 74 | |
74 | |
| 75 | Consider the majority of the portage bin/* scripts- these all are candidates for |
75 | Consider the majority of the portage bin/* scripts- these all are candidates for |
| 76 | being added to the tree as elibs, as is the bulk of eutils. |
76 | being added to the tree as elibs, as is the bulk of eutils. |
| 77 | |
77 | |
| 78 | |
78 | |
| 79 | Specification. |
79 | Specification |
| 80 | ============== |
80 | ============= |
| 81 | |
81 | |
| 82 | The various parts of this proposal are broken down into a set of changes and |
82 | The various parts of this proposal are broken down into a set of changes and |
| 83 | elaborations on why a proposed change is preferable. It's advisable to the |
83 | elaborations on why a proposed change is preferable. It's advisable to the |
| 84 | reader that this be read serially, rather then jumping around. |
84 | reader that this be read serially, rather then jumping around. |
| 85 | |
85 | |
| … | |
… | |
| 108 | initialization of the library that is absolutely needed. Additionally, they |
108 | initialization of the library that is absolutely needed. Additionally, they |
| 109 | cannot modify any ebuild template functions- src_compile, src_unpack. Since they are |
109 | cannot modify any ebuild template functions- src_compile, src_unpack. Since they are |
| 110 | required to not modify the metadata keys, nor in any way affect the ebuild aside |
110 | required to not modify the metadata keys, nor in any way affect the ebuild aside |
| 111 | from providing functionality, they can be conditionally pulled in. They also |
111 | from providing functionality, they can be conditionally pulled in. They also |
| 112 | are allowed to pull in other elibs, but strictly just elibs- no eclasses, just |
112 | are allowed to pull in other elibs, but strictly just elibs- no eclasses, just |
| 113 | other elibs. A real world example would be the eutils eclass. |
113 | other elibs. A real world example would be the eutils eclass. |
| 114 | |
114 | |
| 115 | Portage, since the elib's don't modify metadata, isn't required to track elibs |
115 | Portage, since the elib's don't modify metadata, isn't required to track elibs |
| 116 | as it tracks eclasses. Thus a change in an elib doesn't result in half the tree |
116 | as it tracks eclasses. Thus a change in an elib doesn't result in half the tree |
| 117 | forced to be regenerated/marked stale when changed (this is more of an infra |
117 | forced to be regenerated/marked stale when changed (this is more of an infra |
| 118 | benefit, although regen's that take too long due to eclass changes have been |
118 | benefit, although regen's that take too long due to eclass changes have been |
| 119 | known to cause rsync issues due to missing timestamps). |
119 | known to cause rsync issues due to missing timestamps). |
| 120 | |
120 | |
| 121 | Elibs will not be available in the global scope of an eclass, or ebuild- nor during the |
121 | Elibs will not be available in the global scope of an eclass, or ebuild- nor during the |
| 122 | depends phase (basically a phase that sources the ebuild, to get it's metadata). Elib |
122 | depends phase (basically a phase that sources the ebuild, to get its metadata). Elib |
| 123 | calls in the global scope will be tracked, but the elib will not be loaded till just before |
123 | calls in the global scope will be tracked, but the elib will not be loaded till just before |
| 124 | the setup phase (pkg_setup). There are two reasons for this- first, it ensures elibs are |
124 | the setup phase (pkg_setup). There are two reasons for this- first, it ensures elibs are |
| 125 | completely incapable of modifying metadata. There is no room for confusion, late loading |
125 | completely incapable of modifying metadata. There is no room for confusion, late loading |
| 126 | of elibs gives you the functionality for all phases, except for depends- depends being the |
126 | of elibs gives you the functionality for all phases, except for depends- depends being the |
| 127 | only phase that is capable of specifying metadata. Second, as an added bonus, late |
127 | only phase that is capable of specifying metadata. Second, as an added bonus, late |
| 128 | loading reduces the amount of bash sourced for a regen- faster regens. This however is minor, |
128 | loading reduces the amount of bash sourced for a regen- faster regens. This however is minor, |
| 129 | and is an ancillary benefit of the first reason. |
129 | and is an ancillary benefit of the first reason. |
| … | |
… | |
| 148 | fi |
148 | fi |
| 149 | |
149 | |
| 150 | |
150 | |
| 151 | Regarding maintainability of elibs, it should be a less of a load then old |
151 | Regarding maintainability of elibs, it should be a less of a load then old |
| 152 | eclasses. One of the major issues with old eclasses is that their functions are |
152 | eclasses. One of the major issues with old eclasses is that their functions are |
| 153 | quite incestuous- they're bound tightly to the env they're defined in. This |
153 | quite incestuous- they're bound tightly to the env they're defined in. This |
| 154 | makes eclass functions a bit fragile- the restrictions on what can, and cannot |
154 | makes eclass functions a bit fragile- the restrictions on what can, and cannot |
| 155 | be done in elibs will address this, making functionality less fragile (thus a |
155 | be done in elibs will address this, making functionality less fragile (thus a |
| 156 | bit more maintainable). |
156 | bit more maintainable). |
| 157 | |
157 | |
| 158 | There is no need for backwards compatibility with elibs- they just must work |
158 | There is no need for backwards compatibility with elibs- they just must work |
| … | |
… | |
| 169 | |
169 | |
| 170 | The reduced role of Eclasses, and a clarification of existing Eclass requirements |
170 | The reduced role of Eclasses, and a clarification of existing Eclass requirements |
| 171 | --------------------------------------------------------------------------------- |
171 | --------------------------------------------------------------------------------- |
| 172 | |
172 | |
| 173 | Since elibs are now intended on holding common bash functionality, the focus of |
173 | Since elibs are now intended on holding common bash functionality, the focus of |
| 174 | eclasses should be in defining an appropriate template for ebuilds. For example, |
174 | eclasses should be in defining an appropriate template for ebuilds. For example, |
| 175 | defining common DEPENDS, RDEPENDS, src_compile functions, src_unpack, etc. |
175 | defining common DEPENDS, RDEPENDS, src_compile functions, src_unpack, etc. |
| 176 | Additionally, eclasses should pull in any elibs they need for functionality. |
176 | Additionally, eclasses should pull in any elibs they need for functionality. |
| 177 | |
177 | |
| 178 | Eclass functionality that isn't directly related to the metadata, or src_* and |
178 | Eclass functionality that isn't directly related to the metadata, or src_* and |
| 179 | pkg_* funcs should be shifted into elibs to allow for maximal code reuse. This |
179 | pkg_* funcs should be shifted into elibs to allow for maximal code reuse. This |
| … | |
… | |
| 216 | |
216 | |
| 217 | |
217 | |
| 218 | The end of backwards compatibility... |
218 | The end of backwards compatibility... |
| 219 | ------------------------------------- |
219 | ------------------------------------- |
| 220 | |
220 | |
| 221 | With current eclasses, once the eclass is in use, it's api can no longer be |
221 | With current eclasses, once the eclass is in use, its api can no longer be |
| 222 | changed, nor can the eclass ever be removed from the tree. This is why we still |
222 | changed, nor can the eclass ever be removed from the tree. This is why we still |
| 223 | have *ancient* eclasses that are completely unused sitting in the tree, for |
223 | have *ancient* eclasses that are completely unused sitting in the tree, for |
| 224 | example inherit.eclass . The reason for this, not surprisingly is a portage |
224 | example inherit.eclass. The reason for this, not surprisingly, is a portage |
| 225 | deficiency- on unmerging an installed ebuild, portage used the eclass from the |
225 | deficiency: on unmerging an installed ebuild, portage used the eclass from the |
| 226 | current tree. |
226 | current tree. |
| 227 | |
227 | |
| 228 | For a real world example of this, if you merged a glibc 2 years back, whatever |
228 | For a real world example of this, if you merged a glibc 2 years back, whatever |
| 229 | eclasses it used must still be compatible, or you may not be able to unmerge the |
229 | eclasses it used must still be compatible, or you may not be able to unmerge the |
| 230 | older glibc version during an upgrade to a newer version. So either the glibc |
230 | older glibc version during an upgrade to a newer version. So either the glibc |
| … | |
… | |
| 266 | |
266 | |
| 267 | In other words, these new eclasses would in effect, be old eclasses since older |
267 | In other words, these new eclasses would in effect, be old eclasses since older |
| 268 | portage versions could still access them. |
268 | portage versions could still access them. |
| 269 | |
269 | |
| 270 | |
270 | |
| 271 | Tree restructuring. |
271 | Tree restructuring |
| 272 | ------------------- |
272 | ------------------ |
| 273 | |
273 | |
| 274 | There are only two way to block the existing (as of this writing) inherit |
274 | There are only two way to block the existing (as of this writing) inherit |
| 275 | functionality from accessing the new eclasses- either change the extension of |
275 | functionality from accessing the new eclasses- either change the extension of |
| 276 | eclasses to something other then 'eclass', or to have them stored in a separate |
276 | eclasses to something other then 'eclass', or to have them stored in a separate |
| 277 | subdirectory of the tree then eclass. |
277 | subdirectory of the tree then eclass. |
| … | |
… | |
| 287 | eclasses, please reread the previous section. It's unfortunately a requirement |
287 | eclasses, please reread the previous section. It's unfortunately a requirement |
| 288 | to take advantage of all that the next major portage release will allow. |
288 | to take advantage of all that the next major portage release will allow. |
| 289 | |
289 | |
| 290 | The proposed directory structure is ${PORTDIR}/include/{eclass,elib}. |
290 | The proposed directory structure is ${PORTDIR}/include/{eclass,elib}. |
| 291 | Something like ${PORTDIR}/new-eclass, or ${PORTDIR}/eclass-ng could be used |
291 | Something like ${PORTDIR}/new-eclass, or ${PORTDIR}/eclass-ng could be used |
| 292 | (although many would cringe at the -ng), but such a name is unwise. Consider the |
292 | (although many would cringe at the -ng), but such a name is unwise. Consider the |
| 293 | possibility (likely a fact) that new eclasses someday may be found lacking, and |
293 | possibility (likely a fact) that new eclasses someday may be found lacking, and |
| 294 | refined further (version three as it were). Or perhaps we want to add yet more |
294 | refined further (version three as it were). Or perhaps we want to add yet more |
| 295 | functionality with direct relation to sourcing new files, and we would then need |
295 | functionality with direct relation to sourcing new files, and we would then need |
| 296 | to further populate ${PORTDIR}. |
296 | to further populate ${PORTDIR}. |
| 297 | |
297 | |
| 298 | The new-eclass directory will be (at least) 2 levels deep- for example: |
298 | The new-eclass directory will be (at least) 2 levels deep- for example: |
| 299 | |
299 | |
| … | |
… | |
| 315 | to ensure no files are missing, and that nothing has been tainted. |
315 | to ensure no files are missing, and that nothing has been tainted. |
| 316 | |
316 | |
| 317 | The elib directory will be structured in the same way, for the same reasons. |
317 | The elib directory will be structured in the same way, for the same reasons. |
| 318 | |
318 | |
| 319 | Repoman will have to be extended to work within new eclass and elib groups, and |
319 | Repoman will have to be extended to work within new eclass and elib groups, and |
| 320 | to handle signing and committing. This is intentional, and a good thing. This |
320 | to handle signing and committing. This is intentional, and a good thing. This |
| 321 | gives repoman the possibility of doing sanity checks on elibs/new eclasses. |
321 | gives repoman the possibility of doing sanity checks on elibs/new eclasses. |
| 322 | |
322 | |
| 323 | Note these checks will not prevent developers from doing dumb things with eclass- |
323 | Note these checks will not prevent developers from doing dumb things with eclass- |
| 324 | these checks would only be capable of doing basic sanity checks, such as syntax checks. |
324 | these checks would only be capable of doing basic sanity checks, such as syntax checks. |
| 325 | There is no way to prevent people from doing dumb things (exempting perhaps repeated |
325 | There is no way to prevent people from doing dumb things (exempting perhaps repeated |
| … | |
… | |
| 331 | --------------------------------------------------------- |
331 | --------------------------------------------------------- |
| 332 | |
332 | |
| 333 | As clarified above, new eclasses will exist in a separate directory that will be |
333 | As clarified above, new eclasses will exist in a separate directory that will be |
| 334 | intentionally inaccessible to the inherit function. As such, users of older |
334 | intentionally inaccessible to the inherit function. As such, users of older |
| 335 | portage versions *will* have to upgrade to merge any ebuild that uses elibs/new |
335 | portage versions *will* have to upgrade to merge any ebuild that uses elibs/new |
| 336 | eclasses. A depend on the next major portage version would transparently handle |
336 | eclasses. A depend on the next major portage version would transparently handle |
| 337 | this for rsync users. |
337 | this for rsync users. |
| 338 | |
338 | |
| 339 | There still is the issue of users who haven't upgraded to the required portage |
339 | There still is the issue of users who haven't upgraded to the required portage |
| 340 | version. This is a minor concern frankly- portage releases include new |
340 | version. This is a minor concern frankly- portage releases include new |
| 341 | functionality, and bug fixes. If they won't upgrade, it's assumed they have |
341 | functionality, and bug fixes. If they won't upgrade, it's assumed they have |
| 342 | their reasons and are big boys, thus able to handle the complications themselves. |
342 | their reasons and are big boys, thus able to handle the complications themselves. |
| 343 | |
343 | |
| 344 | The real issue is broken envs, whether in binpkgs, or for installed packages. |
344 | The real issue is broken envs, whether in binpkgs, or for installed packages. |
| 345 | Two options exist- either the old eclasses are left in the tree indefinitely, or |
345 | Two options exist- either the old eclasses are left in the tree indefinitely, or |
| … | |
… | |
| 381 | |
381 | |
| 382 | Note for this to happen requires either rather... unwise uses of root, or significant |
382 | Note for this to happen requires either rather... unwise uses of root, or significant |
| 383 | fs corruption. Regardless of the cause, it's quite likely for this to even become an |
383 | fs corruption. Regardless of the cause, it's quite likely for this to even become an |
| 384 | issue, the system's vdb is completely unusable. It's a moot issue at that point. |
384 | issue, the system's vdb is completely unusable. It's a moot issue at that point. |
| 385 | If you lose your vdb, or it gets seriously damaged, it's akin to lobotomizing portage- |
385 | If you lose your vdb, or it gets seriously damaged, it's akin to lobotomizing portage- |
| 386 | it doesn't know what's installed, it doesn't know of it's own files, and in general, |
386 | it doesn't know what's installed, it doesn't know of its own files, and in general, |
| 387 | a rebuilding of the system is about the only sane course of action. The missing env is |
387 | a rebuilding of the system is about the only sane course of action. The missing env is |
| 388 | truly the least of the users concern in such a case. |
388 | truly the least of the users concern in such a case. |
| 389 | |
389 | |
| 390 | Continuing with the more likely scenario, users unwilling to upgrade portage will |
390 | Continuing with the more likely scenario, users unwilling to upgrade portage will |
| 391 | *not* be left out in the rain. Merging the old eclass compat ebuild will provide |
391 | *not* be left out in the rain. Merging the old eclass compat ebuild will provide |
| 392 | the missing eclasses, thus providing that lost functionality . |
392 | the missing eclasses, thus providing that lost functionality. |
| 393 | |
393 | |
| 394 | Note the intention isn't to force them to upgrade, hence the ability to restore the |
394 | Note the intention isn't to force them to upgrade, hence the ability to restore the |
| 395 | lost functionality. The intention is to clean up the existing mess, and allow us |
395 | lost functionality. The intention is to clean up the existing mess, and allow us |
| 396 | to move forward. The saying "you've got to break a few eggs to make an omelet" |
396 | to move forward. The saying "you've got to break a few eggs to make an omelet" |
| 397 | is akin, exempting the fact we're providing a way to make the eggs whole again |
397 | is akin, exempting the fact we're providing a way to make the eggs whole again |
| 398 | (the king's men would've loved such an option). |
398 | (the king's men would've loved such an option). |
| 399 | |
399 | |
| 400 | |
400 | |
| 401 | Migrating to the new setup |
401 | Migrating to the new setup |
| … | |
… | |
| 417 | infrastructure server that generates the cache for rsync users will have to |
417 | infrastructure server that generates the cache for rsync users will have to |
| 418 | either be upgraded to a version of portage supporting new eclasses, or patched. |
418 | either be upgraded to a version of portage supporting new eclasses, or patched. |
| 419 | The former being much more preferable then the latter for the portage devs. |
419 | The former being much more preferable then the latter for the portage devs. |
| 420 | |
420 | |
| 421 | Beyond that, an appropriate window for old eclasses to exist in the tree must be |
421 | Beyond that, an appropriate window for old eclasses to exist in the tree must be |
| 422 | determined, and prior to that window passing an ebuild must be added to the tree |
422 | determined, and prior to that window passing, an ebuild must be added to the tree |
| 423 | so users can get the old eclasses if needed. |
423 | so users can get the old eclasses if needed. |
| 424 | |
424 | |
| 425 | For eclass devs to migrate from old to new, it is possible for them to just |
425 | For eclass devs to migrate from old to new, it is possible for them to just |
| 426 | transfer the old eclass into an appropriate grouping in the new eclass directory, |
426 | transfer the old eclass into an appropriate grouping in the new eclass directory, |
| 427 | although it's advisable they cleanse all cruft out of the eclass. You can |
427 | although it's advisable they cleanse all cruft out of the eclass. You can |
| 428 | migrate ebuilds gradually over to the new eclass, and don't have to worry about |
428 | migrate ebuilds gradually over to the new eclass, and don't have to worry about |
| 429 | having to support ebuilds from X years back. |
429 | having to support ebuilds from X years back. |
| 430 | |
430 | |
| 431 | Essentially, you have a chance to nail the design perfectly/cleanly, and have a |
431 | Essentially, you have a chance to nail the design perfectly/cleanly, and have a |
| 432 | window in which to redesign it. It's humbly suggested eclass devs take |
432 | window in which to redesign it. It's humbly suggested eclass devs take |
| … | |
… | |
| 444 | |
444 | |
| 445 | To recap: |
445 | To recap: |
| 446 | :: |
446 | :: |
| 447 | |
447 | |
| 448 | New eclasses and elib functionality will be tied to a specific portage |
448 | New eclasses and elib functionality will be tied to a specific portage |
| 449 | version. A DEPENDs on said portage version should address this for rsync |
449 | version. A DEPENDs on said portage version should address this for rsync |
| 450 | users who refuse to upgrade to a portage version that supports the new |
450 | users who refuse to upgrade to a portage version that supports the new |
| 451 | eclasses/elibs and will gradually be unable to merge ebuilds that use said |
451 | eclasses/elibs and will gradually be unable to merge ebuilds that use said |
| 452 | functionality. It is their choice to upgrade, as such, the gradual |
452 | functionality. It is their choice to upgrade, as such, the gradual |
| 453 | 'thinning' of available ebuilds should they block the portage upgrade is |
453 | 'thinning' of available ebuilds should they block the portage upgrade is |
| 454 | their responsibility. |
454 | their responsibility. |
| 455 | |
455 | |
| 456 | Old eclasses at some point in the future should be removed from the tree, |
456 | Old eclasses at some point in the future should be removed from the tree, |
| 457 | and released in a tarball/ebuild. This will cause installed ebuilds that |
457 | and released in a tarball/ebuild. This will cause installed ebuilds that |
| 458 | rely on the old eclass to be unable to unmerge, with the same applying for |
458 | rely on the old eclass to be unable to unmerge, with the same applying for |
| 459 | merging of binpkgs dependent on the following paragraph. |
459 | merging of binpkgs dependent on the following paragraph. |
| 460 | |
460 | |
| 461 | The old eclass-compat is only required for users who do not upgrade their |
461 | The old eclass-compat is only required for users who do not upgrade their |
| 462 | portage installation, and one further exemption- if the user has somehow |
462 | portage installation, and one further exemption- if the user has somehow |
Parent Directory
| 
Revision Log
| 
Patch