--- xml/htdocs/proj/en/glep/glep-0025.html 2005/04/01 01:32:29 1.3 +++ xml/htdocs/proj/en/glep/glep-0025.html 2006/10/10 20:25:14 1.4 @@ -8,9 +8,252 @@ -->- +
|Author:||Brian Harring <ferringb at gentoo.org>|
The intention of this GLEP is to propose the creation of patching support for portage, and iron out the implementation details.
Reduce the bandwidth load placed on our mirrors by decreasing the amount of bytes transferred when upgrading between versions. Side benefit of this is to significantly decrease the download requirements for users lacking broadband.
Most people are familiar with diff patches (unified diff for example)- this glep is specifically proposing the use of an actual binary differencer. The reason for this is that diff patches are line based- you change a single @@ -118,19 +361,19 @@ to say (with a fair amount of work) it couldn't be extended to support standard diffs.
The difference between source releases typically isn't very large, especially for minor releases. As an example, kdelibs-3.1.4.tar.bz2 is 10.53 MB, and kdelibs-3.1.5.tar.bz2 is 10.54 MB. A bzip2'ed patch between those versions is 75.6 kb , less then 1% the size of 3.1.5's tbz2.
Quite a few sections of gentoo are affected- mirroring, the portage tree, and portage itself.-
For adding patch info into the tree, this glep proposes a global patch list (stored in profiles as patches.global), and individual patch lists stored in relevant package directories (named patches). Using the kernel packages as an @@ -158,12 +401,12 @@ kdelibs-3.1.4-3.1.5 can only be used to upgrade from 3.1.4 to 3.1.5, not in reverse (originally explained in Binary patches vs GNUDiff patches).
This glep proposes the patching support should be (at this stage) optional- specifically, enabled via FEATURES="patching".-
When patching is enabled, the global patch list is read, and the packages patch list is read. From there, portage determines what files could be used as a base for patching to the desired file- further, determining if it's @@ -171,8 +414,8 @@ less then the sum of the patches needed). Any patches to be used are fetched, and md5 verified.
Upon fetching and md5 verification of patch(es), the desired file is reconstructed. Assuming reconstruction didn't return any errors, the target file has its uncompressed md5sum calculated and verified, then is recompressed @@ -183,14 +426,14 @@ md5 database is updated with new compressed md5. Details of this database (and the issue it addresses) follow.
There will be instances where a file is reconstructed perfectly, recompressed, and the recompressed md5sum differs from what is stored in the tree- the problem is that the md5sum of a compressed file is inherently tied to the compressor version/options used to compress the original source.-
A good example of this problem is related to bzip2 versions used for compression. Between bzip2 0.9x and bzip2 1.x, there was a subtle change in the compressor resulting in a slightly better compression result- end result @@ -207,8 +450,8 @@ the sake of recreating a compressed md5sum even though the uncompressed source's md5 has already been verified.
One issue of contention is where these files will actually be stored. As of the writing of this glep, a full distfiles mirror is roughly around 40 gb- a rough estimate by the author places the space requirements for patches for @@ -274,8 +517,8 @@ could be made to work. Feedback and any possible alternatives would be greatly appreciated.
Maintenance of patch lists, and the actual patch creation ought to be managed by a high level script- essentally a dev says "I want a patch between this version, and that version: make it so", the script churns away @@ -291,8 +534,8 @@ the transition and offer patches for people to test out.
As noted in The Proposed Solution, a system using patching and sharing out it's distfiles must share out it's alternate md5 db. Any system that uses the distfiles share must support the alternate md5 db also. If this is considered @@ -302,12 +545,12 @@
Also note that Distfile Mirror Additions may add additional backwards compatibility issues, depending on what solution is accepted.
|||(1, 2) kdelibs-3.1.4-3.1.5.patch.bz2, switching format patch, created via diffball-0.4_pre4 (diffball is available at http://sourceforge.net/projects/diffball) +|
|||(1, 2) kdelibs-3.1.4-3.1.5.patch.bz2, switching format patch, created via diffball-0.4_pre4 (diffball is available at http://sourceforge.net/projects/diffball) Bzip2 -9 compressed, the patch is 75,687 bytes, uncompressed it is 337,649 bytes. The patch is available at http://dev.gentoo.org/~ferringb/kdelibs-3.1.4-3.1.5.patch.bz2 for those curious.|
|||Glep9, 'Gentoo Package Update System' +|
|||Glep9, 'Gentoo Package Update System' (http://glep.gentoo.org/glep-0009.html)|