1 |
cardoe |
1.1 |
GLEP: 61 |
2 |
|
|
Title: Manifest2 compression |
3 |
robbat2 |
1.4 |
Version: $Revision: 1.3 $ |
4 |
|
|
Last-Modified: $Date: 2010/01/13 01:14:35 $ |
5 |
cardoe |
1.1 |
Author: Robin Hugh Johnson <robbat2@gentoo.org> |
6 |
|
|
Status: Draft |
7 |
|
|
Type: Standards Track |
8 |
|
|
Content-Type: text/x-rst |
9 |
|
|
Requires: 44 |
10 |
|
|
Created: July 2008 |
11 |
robbat2 |
1.3 |
Updated: October 2008, January 2010 |
12 |
cardoe |
1.1 |
Updates: 44 |
13 |
robbat2 |
1.3 |
Post-History: December 2009 |
14 |
cardoe |
1.1 |
|
15 |
|
|
Abstract |
16 |
|
|
======== |
17 |
|
|
Deals with compression of large Manifest2 files. |
18 |
|
|
|
19 |
|
|
Motivation |
20 |
|
|
========== |
21 |
|
|
With the introduction of MetaManifest, and full-tree Manifest coverage, |
22 |
|
|
we are faced with the possibility of having very large Manifests. |
23 |
|
|
|
24 |
|
|
Preliminary experiments with MetaManifest, show that with just the |
25 |
|
|
existing per-package Manifests, the full MetaManifest, for a tree |
26 |
|
|
including metadata/, exceeds 8MiB in size. Applying common compression |
27 |
|
|
can achieve a 50-60% reduction in this size. |
28 |
|
|
|
29 |
|
|
Specification |
30 |
|
|
============= |
31 |
|
|
When searching for a Manifest2 file, if the basename form does not |
32 |
|
|
exist, the package manager should search in the same location using |
33 |
|
|
common compressed suffixes, and use the compressed file in place of the |
34 |
|
|
Manifest2. |
35 |
|
|
|
36 |
robbat2 |
1.3 |
gzip, bzip2, lzma, xz should all be supported if available on the given |
37 |
cardoe |
1.1 |
platform. In the case that multiple versions exist, the package manager |
38 |
|
|
should simply pick one - they should be identical, differing only in |
39 |
|
|
compression. |
40 |
|
|
|
41 |
|
|
The Manifest generation process is required to ensure that inconsistent |
42 |
|
|
compressed versions do not exist. |
43 |
|
|
|
44 |
|
|
Backwards Compatibility |
45 |
|
|
======================= |
46 |
|
|
The package Manifests should also be maintained as ONLY uncompressed in |
47 |
|
|
CVS. |
48 |
|
|
|
49 |
|
|
For processing of all existing per-package Manifests, if compression is |
50 |
|
|
used, it should be done in parallel to the existing Manifests, to |
51 |
|
|
provide for a changeover period. Newer versions of Portage may later |
52 |
|
|
choose to exclude all non-compressed Manifests during emerge --sync if |
53 |
|
|
compressed versions are guaranteed to exist on the servers. |
54 |
|
|
|
55 |
|
|
MetaManifests may come into existence as compressed from the start, as |
56 |
|
|
do not have an backwards compatibility issues. |
57 |
|
|
|
58 |
|
|
As a side note, this breaks all manual interaction with Manifests |
59 |
|
|
such as grep, and so should only be applied to large Manifest2 files, |
60 |
robbat2 |
1.4 |
such as the MetaManifest. 384KiB is suggested as a arbitrary cut-off |
61 |
cardoe |
1.1 |
point to start generating compressed Manifest2 files. |
62 |
|
|
|
63 |
|
|
References |
64 |
|
|
========== |
65 |
|
|
.. [#GLEP44] Mauch, M. (2005) GLEP44 - Manifest2 format. |
66 |
|
|
http://www.gentoo.org/proj/en/glep/glep-0044.html |
67 |
|
|
|
68 |
|
|
Copyright |
69 |
|
|
========= |
70 |
robbat2 |
1.4 |
Copyright (c) 2008-2010 by Robin Hugh Johnson. This material may be |
71 |
cardoe |
1.1 |
distributed only subject to the terms and conditions set forth in the |
72 |
|
|
Open Publication License, v1.0. |
73 |
|
|
|
74 |
|
|
vim: tw=72 ts=2 expandtab: |