/[gentoo]/xml/htdocs/proj/en/glep/glep-0044.html
Gentoo

Diff of /xml/htdocs/proj/en/glep/glep-0044.html

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

Revision 1.9 Revision 1.11
1<?xml version="1.0" encoding="utf-8" ?> 1<?xml version="1.0" encoding="utf-8" ?>
2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
3<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> 3<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
4<!-- 4
5This HTML is auto-generated. DO NOT EDIT THIS FILE! If you are writing a new
6PEP, see http://www.python.org/peps/pep-0001.html for instructions and links
7to templates. DO NOT USE THIS HTML FILE AS YOUR TEMPLATE!
8-->
9<head> 5<head>
10 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> 6 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
11 <meta name="generator" content="Docutils 0.4: http://docutils.sourceforge.net/" /> 7 <meta name="generator" content="Docutils 0.5: http://docutils.sourceforge.net/" />
12 <title>GLEP 44 -- Manifest2 format</title> 8 <title>GLEP 44 -- Manifest2 format</title>
13 <style type="text/css"> 9 <link rel="stylesheet" href="tools/glep.css" type="text/css" /></head>
14
15/*
16:Author: David Goodger
17:Contact: goodger@users.sourceforge.net
18:date: $Date: 2007/01/25 03:26:26 $
19:version: $Revision: 1.9 $
20:copyright: This stylesheet has been placed in the public domain.
21
22Default cascading style sheet for the PEP HTML output of Docutils.
23*/
24
25.first {
26 margin-top: 0 }
27
28.last {
29 margin-bottom: 0 }
30
31.navigation {
32 width: 100% ;
33 background: #cc99ff ;
34 margin-top: 0px ;
35 margin-bottom: 0px }
36
37.navigation .navicon {
38 width: 150px ;
39 height: 35px }
40
41.navigation .textlinks {
42 padding-left: 1em ;
43 text-align: left }
44
45.navigation td, .navigation th {
46 padding-left: 0em ;
47 padding-right: 0em ;
48 vertical-align: middle }
49
50.rfc2822 {
51 margin-top: 0.5em ;
52 margin-left: 0.5em ;
53 margin-right: 0.5em ;
54 margin-bottom: 0em }
55
56.rfc2822 td {
57 text-align: left }
58
59.rfc2822 th.field-name {
60 text-align: right ;
61 font-family: sans-serif ;
62 padding-right: 0.5em ;
63 font-weight: bold ;
64 margin-bottom: 0em }
65
66a.toc-backref {
67 text-decoration: none ;
68 color: black }
69
70body {
71 margin: 0px ;
72 margin-bottom: 1em ;
73 padding: 0px }
74
75dd {
76 margin-bottom: 0.5em }
77
78div.section {
79 margin-left: 1em ;
80 margin-right: 1em ;
81 margin-bottom: 1.5em }
82
83div.section div.section {
84 margin-left: 0em ;
85 margin-right: 0em ;
86 margin-top: 1.5em }
87
88div.abstract {
89 margin: 2em 5em }
90
91div.abstract p.topic-title {
92 font-weight: bold ;
93 text-align: center }
94
95div.attention, div.caution, div.danger, div.error, div.hint,
96div.important, div.note, div.tip, div.warning {
97 margin: 2em ;
98 border: medium outset ;
99 padding: 1em }
100
101div.attention p.admonition-title, div.caution p.admonition-title,
102div.danger p.admonition-title, div.error p.admonition-title,
103div.warning p.admonition-title {
104 color: red ;
105 font-weight: bold ;
106 font-family: sans-serif }
107
108div.hint p.admonition-title, div.important p.admonition-title,
109div.note p.admonition-title, div.tip p.admonition-title {
110 font-weight: bold ;
111 font-family: sans-serif }
112
113div.figure {
114 margin-left: 2em }
115
116div.footer, div.header {
117 font-size: smaller }
118
119div.footer {
120 margin-left: 1em ;
121 margin-right: 1em }
122
123div.system-messages {
124 margin: 5em }
125
126div.system-messages h1 {
127 color: red }
128
129div.system-message {
130 border: medium outset ;
131 padding: 1em }
132
133div.system-message p.system-message-title {
134 color: red ;
135 font-weight: bold }
136
137div.topic {
138 margin: 2em }
139
140h1 {
141 font-family: sans-serif ;
142 font-size: large }
143
144h2 {
145 font-family: sans-serif ;
146 font-size: medium }
147
148h3 {
149 font-family: sans-serif ;
150 font-size: small }
151
152h4 {
153 font-family: sans-serif ;
154 font-style: italic ;
155 font-size: small }
156
157h5 {
158 font-family: sans-serif;
159 font-size: x-small }
160
161h6 {
162 font-family: sans-serif;
163 font-style: italic ;
164 font-size: x-small }
165
166.section hr {
167 width: 75% }
168
169ol.simple, ul.simple {
170 margin-bottom: 1em }
171
172ol.arabic {
173 list-style: decimal }
174
175ol.loweralpha {
176 list-style: lower-alpha }
177
178ol.upperalpha {
179 list-style: upper-alpha }
180
181ol.lowerroman {
182 list-style: lower-roman }
183
184ol.upperroman {
185 list-style: upper-roman }
186
187p.caption {
188 font-style: italic }
189
190p.credits {
191 font-style: italic ;
192 font-size: smaller }
193
194p.label {
195 white-space: nowrap }
196
197p.topic-title {
198 font-family: sans-serif ;
199 font-weight: bold }
200
201pre.line-block {
202 font-family: serif ;
203 font-size: 100% }
204
205pre.literal-block, pre.doctest-block {
206 margin-left: 2em ;
207 margin-right: 2em ;
208 background-color: #eeeeee }
209
210span.classifier {
211 font-family: sans-serif ;
212 font-style: oblique }
213
214span.classifier-delimiter {
215 font-family: sans-serif ;
216 font-weight: bold }
217
218span.interpreted {
219 font-family: sans-serif }
220
221span.option-argument {
222 font-style: italic }
223
224span.pre {
225 white-space: pre }
226
227span.problematic {
228 color: red }
229
230table {
231 margin-top: 0.5em ;
232 margin-bottom: 0.5em }
233
234td, th {
235 padding-left: 0.5em ;
236 padding-right: 0.5em ;
237 vertical-align: top }
238
239td.num {
240 text-align: right }
241
242th.field-name {
243 font-weight: bold ;
244 text-align: left ;
245 white-space: nowrap }
246
247h1 tt, h2 tt, h3 tt, h4 tt, h5 tt, h6 tt {
248 font-size: 100% }
249
250tt {
251 background-color: #eeeeee }
252
253ul.auto-toc {
254 list-style-type: none }
255
256</style>
257</head>
258<body bgcolor="white"> 10<body bgcolor="white">
259<table class="navigation" cellpadding="0" cellspacing="0" 11<table class="navigation" cellpadding="0" cellspacing="0"
260 width="100%" border="0"> 12 width="100%" border="0">
261<tr><td class="navicon" width="150" height="35"> 13<tr><td class="navicon" width="150" height="35">
262<a href="http://www.gentoo.org/" title="Gentoo Linux Home Page"> 14<a href="http://www.gentoo.org/" title="Gentoo Linux Home Page">
263<img src="http://www.gentoo.org/images/gentoo-new.gif" alt="[Gentoo]" 15<img src="http://www.gentoo.org/images/gentoo-new.gif" alt="[Gentoo]"
264 border="0" width="150" height="35" /></a></td> 16 border="0" width="150" height="35" /></a></td>
265<td class="textlinks" align="left"> 17<td class="textlinks" align="left">
266[<b><a href="http://www.gentoo.org/">Gentoo Linux Home</a></b>] 18[<b><a href="http://www.gentoo.org/">Gentoo Linux Home</a></b>]
267[<b><a href="http://www.gentoo.org/proj/en/glep/">GLEP Index</a></b>] 19[<b><a href="http://www.gentoo.org/proj/en/glep">GLEP Index</a></b>]
268[<b><a href="http://www.gentoo.org/proj/en/glep/glep-0044.txt">GLEP Source</a></b>] 20[<b><a href="http://www.gentoo.org/proj/en/glep/glep-0044.txt">GLEP Source</a></b>]
269</td></tr></table> 21</td></tr></table>
270<table class="rfc2822 docutils field-list" frame="void" rules="none"> 22<table class="rfc2822 docutils field-list" frame="void" rules="none">
271<col class="field-name" /> 23<col class="field-name" />
272<col class="field-body" /> 24<col class="field-body" />
273<tbody valign="top"> 25<tbody valign="top">
274<tr class="field"><th class="field-name">GLEP:</th><td class="field-body">44</td> 26<tr class="field"><th class="field-name">GLEP:</th><td class="field-body">44</td>
275</tr> 27</tr>
276<tr class="field"><th class="field-name">Title:</th><td class="field-body">Manifest2 format</td> 28<tr class="field"><th class="field-name">Title:</th><td class="field-body">Manifest2 format</td>
277</tr> 29</tr>
278<tr class="field"><th class="field-name">Version:</th><td class="field-body">1.6</td> 30<tr class="field"><th class="field-name">Version:</th><td class="field-body">1.7</td>
279</tr> 31</tr>
280<tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference" href="http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/proj/en/glep/glep-0044.txt?cvsroot=gentoo">2006/09/04 03:06:32</a></td> 32<tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference external" href="http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/proj/en/glep/glep-0044.txt?cvsroot=gentoo">2006/10/14 02:55:39</a></td>
281</tr> 33</tr>
282<tr class="field"><th class="field-name">Author:</th><td class="field-body">Marius Mauch &lt;genone&#32;&#97;t&#32;gentoo.org&gt;,</td> 34<tr class="field"><th class="field-name">Author:</th><td class="field-body">Marius Mauch &lt;genone&#32;&#97;t&#32;gentoo.org&gt;,</td>
283</tr> 35</tr>
284<tr class="field"><th class="field-name">Status:</th><td class="field-body">Accepted</td> 36<tr class="field"><th class="field-name">Status:</th><td class="field-body">Final</td>
285</tr> 37</tr>
286<tr class="field"><th class="field-name">Type:</th><td class="field-body">Standards Track</td> 38<tr class="field"><th class="field-name">Type:</th><td class="field-body">Standards Track</td>
287</tr> 39</tr>
288<tr class="field"><th class="field-name">Content-Type:</th><td class="field-body"><a class="reference" href="glep-0002.html">text/x-rst</a></td> 40<tr class="field"><th class="field-name">Content-Type:</th><td class="field-body"><a class="reference external" href="glep-0002.html">text/x-rst</a></td>
289</tr> 41</tr>
290<tr class="field"><th class="field-name">Created:</th><td class="field-body">04-Dec-2005</td> 42<tr class="field"><th class="field-name">Created:</th><td class="field-body">04-Dec-2005</td>
291</tr> 43</tr>
292<tr class="field"><th class="field-name">Post-History:</th><td class="field-body">06-Dec-2005, 23-Jan-2006, 3-Sep-2006</td> 44<tr class="field"><th class="field-name">Post-History:</th><td class="field-body">06-Dec-2005, 23-Jan-2006, 3-Sep-2006</td>
293</tr> 45</tr>
294</tbody> 46</tbody>
295</table> 47</table>
296<hr /> 48<hr />
297<div class="contents topic"> 49<div class="contents topic" id="contents">
298<p class="topic-title first"><a id="contents" name="contents">Contents</a></p> 50<p class="topic-title first">Contents</p>
299<ul class="simple"> 51<ul class="simple">
300<li><a class="reference" href="#abstract" id="id9" name="id9">Abstract</a></li> 52<li><a class="reference internal" href="#abstract" id="id9">Abstract</a></li>
301<li><a class="reference" href="#motivation" id="id10" name="id10">Motivation</a></li> 53<li><a class="reference internal" href="#motivation" id="id10">Motivation</a></li>
302<li><a class="reference" href="#specification" id="id11" name="id11">Specification</a><ul> 54<li><a class="reference internal" href="#specification" id="id11">Specification</a><ul>
303<li><a class="reference" href="#compability-entries" id="id12" name="id12">Compability Entries</a></li> 55<li><a class="reference internal" href="#compability-entries" id="id12">Compability Entries</a></li>
304<li><a class="reference" href="#scope" id="id13" name="id13">Scope</a></li> 56<li><a class="reference internal" href="#scope" id="id13">Scope</a></li>
305<li><a class="reference" href="#number-of-hashes" id="id14" name="id14">Number of hashes</a></li> 57<li><a class="reference internal" href="#number-of-hashes" id="id14">Number of hashes</a></li>
306</ul>
307</li> 58</ul>
59</li>
308<li><a class="reference" href="#rationale" id="id15" name="id15">Rationale</a><ul> 60<li><a class="reference internal" href="#rationale" id="id15">Rationale</a><ul>
309<li><a class="reference" href="#removal-of-digest-files" id="id16" name="id16">Removal of digest files</a></li> 61<li><a class="reference internal" href="#removal-of-digest-files" id="id16">Removal of digest files</a></li>
310<li><a class="reference" href="#reducing-redundancy" id="id17" name="id17">Reducing redundancy</a></li> 62<li><a class="reference internal" href="#reducing-redundancy" id="id17">Reducing redundancy</a></li>
311<li><a class="reference" href="#removal-of-checksum-collisions" id="id18" name="id18">Removal of checksum collisions</a></li> 63<li><a class="reference internal" href="#removal-of-checksum-collisions" id="id18">Removal of checksum collisions</a></li>
312<li><a class="reference" href="#flexible-verification-system" id="id19" name="id19">Flexible verification system</a></li> 64<li><a class="reference internal" href="#flexible-verification-system" id="id19">Flexible verification system</a></li>
313</ul>
314</li> 65</ul>
66</li>
315<li><a class="reference" href="#backwards-compatibility" id="id20" name="id20">Backwards Compatibility</a></li> 67<li><a class="reference internal" href="#backwards-compatibility" id="id20">Backwards Compatibility</a></li>
316<li><a class="reference" href="#other-problems" id="id21" name="id21">Other problems</a><ul> 68<li><a class="reference internal" href="#other-problems" id="id21">Other problems</a><ul>
317<li><a class="reference" href="#impacts-on-infrastructure" id="id22" name="id22">Impacts on infrastructure</a></li> 69<li><a class="reference internal" href="#impacts-on-infrastructure" id="id22">Impacts on infrastructure</a></li>
318</ul>
319</li> 70</ul>
71</li>
320<li><a class="reference" href="#reference-implementation" id="id23" name="id23">Reference Implementation</a></li> 72<li><a class="reference internal" href="#reference-implementation" id="id23">Reference Implementation</a></li>
321<li><a class="reference" href="#options" id="id24" name="id24">Options</a></li> 73<li><a class="reference internal" href="#options" id="id24">Options</a></li>
322<li><a class="reference" href="#credits" id="id25" name="id25">Credits</a></li> 74<li><a class="reference internal" href="#credits" id="id25">Credits</a></li>
323<li><a class="reference" href="#references" id="id26" name="id26">References</a></li> 75<li><a class="reference internal" href="#references" id="id26">References</a></li>
324<li><a class="reference" href="#copyright" id="id27" name="id27">Copyright</a></li> 76<li><a class="reference internal" href="#copyright" id="id27">Copyright</a></li>
325</ul> 77</ul>
326</div> 78</div>
327<div class="section"> 79<div class="section" id="abstract">
328<h1><a class="toc-backref" href="#id9" id="abstract" name="abstract">Abstract</a></h1> 80<h1><a class="toc-backref" href="#id9">Abstract</a></h1>
329<p>This GLEP proposes a new format for the Portage Manifest and digest file system 81<p>This GLEP proposes a new format for the Portage Manifest and digest file system
330by unifying both filetypes into one to improve functional and non-functional 82by unifying both filetypes into one to improve functional and non-functional
331aspects of the Portage Tree.</p> 83aspects of the Portage Tree.</p>
332</div> 84</div>
333<div class="section"> 85<div class="section" id="motivation">
334<h1><a class="toc-backref" href="#id10" id="motivation" name="motivation">Motivation</a></h1> 86<h1><a class="toc-backref" href="#id10">Motivation</a></h1>
335<p>Please see <a class="footnote-reference" href="#reorg-thread" id="id1" name="id1">[1]</a> for a general overview. 87<p>Please see <a class="footnote-reference" href="#reorg-thread" id="id1">[1]</a> for a general overview.
336The main long term goals of this proposal are to:</p> 88The main long term goals of this proposal are to:</p>
337<ul class="simple"> 89<ul class="simple">
338<li>Remove the tiny digest files from the tree. They are a major annoyance as on a 90<li>Remove the tiny digest files from the tree. They are a major annoyance as on a
339typical configuration they waste a lot of disk space and the simple transmission 91typical configuration they waste a lot of disk space and the simple transmission
340of the names for all digest files during a <tt class="docutils literal"><span class="pre">emerge</span> <span class="pre">--sync</span></tt> needs a substantial 92of the names for all digest files during a <tt class="docutils literal"><span class="pre">emerge</span> <span class="pre">--sync</span></tt> needs a substantial
… …
343<li>Remove potential for checksum collisions if a file is recorded in more than one 95<li>Remove potential for checksum collisions if a file is recorded in more than one
344digest file</li> 96digest file</li>
345<li>Difference between filetypes for a more flexible verification system</li> 97<li>Difference between filetypes for a more flexible verification system</li>
346</ul> 98</ul>
347</div> 99</div>
348<div class="section"> 100<div class="section" id="specification">
349<h1><a class="toc-backref" href="#id11" id="specification" name="specification">Specification</a></h1> 101<h1><a class="toc-backref" href="#id11">Specification</a></h1>
350<p>The new Manifest format would change the existing format in the following ways:</p> 102<p>The new Manifest format would change the existing format in the following ways:</p>
351<ul> 103<ul>
352<li><p class="first">Addition of a filetype specifier, currently planned are</p> 104<li><p class="first">Addition of a filetype specifier, currently planned are</p>
353<ul class="simple"> 105<ul class="simple">
354<li><tt class="docutils literal"><span class="pre">AUX</span></tt> for files directly used by ebuilds (e.g. patches or initscripts), 106<li><tt class="docutils literal"><span class="pre">AUX</span></tt> for files directly used by ebuilds (e.g. patches or initscripts),
… …
371<p>Each line in the new format has the following format:</p> 123<p>Each line in the new format has the following format:</p>
372<pre class="literal-block"> 124<pre class="literal-block">
373&lt;filetype&gt; &lt;filename&gt; &lt;filesize&gt; &lt;chksumtype1&gt; &lt;chksum1&gt; ... &lt;chksumtypen&gt; &lt;chksumn&gt; 125&lt;filetype&gt; &lt;filename&gt; &lt;filesize&gt; &lt;chksumtype1&gt; &lt;chksum1&gt; ... &lt;chksumtypen&gt; &lt;chksumn&gt;
374</pre> 126</pre>
375<p>However theses entries will be stored in the existing Manifest files.</p> 127<p>However theses entries will be stored in the existing Manifest files.</p>
376<p>An <a class="reference" href="glep-0044-extras/manifest2-example.txt">actual example</a> <a class="footnote-reference" href="#id7" id="id8" name="id8">[6]</a> for a (pure) Manifest2 file..</p> 128<p>An <a class="reference external" href="glep-0044-extras/manifest2-example.txt">actual example</a> <a class="footnote-reference" href="#id7" id="id8">[6]</a> for a (pure) Manifest2 file..</p>
377<div class="section"> 129<div class="section" id="compability-entries">
378<h2><a class="toc-backref" href="#id12" id="compability-entries" name="compability-entries">Compability Entries</a></h2> 130<h2><a class="toc-backref" href="#id12">Compability Entries</a></h2>
379<p>To maintain compability with existing portage versions a transition period after 131<p>To maintain compability with existing portage versions a transition period after
380is the introduction of the Manifest2 format is required during which portage 132is the introduction of the Manifest2 format is required during which portage
381will not only have to be capable of using existing Manifest and digest files but 133will not only have to be capable of using existing Manifest and digest files but
382also generate them in addition to the new entries. 134also generate them in addition to the new entries.
383Fortunately this can be accomplished by simply mixing old and new style entries 135Fortunately this can be accomplished by simply mixing old and new style entries
384in one file for the Manifest files, existing portage versions will simply ignore 136in one file for the Manifest files, existing portage versions will simply ignore
385the new style entries. For the digest files there are no new entries to care 137the new style entries. For the digest files there are no new entries to care
386about.</p> 138about.</p>
387</div> 139</div>
388<div class="section"> 140<div class="section" id="scope">
389<h2><a class="toc-backref" href="#id13" id="scope" name="scope">Scope</a></h2> 141<h2><a class="toc-backref" href="#id13">Scope</a></h2>
390<p>It is important to note that this proposal only deals with a change of the 142<p>It is important to note that this proposal only deals with a change of the
391format of the digest and Manifest system.</p> 143format of the digest and Manifest system.</p>
392<p>It does not expand the scope of it to cover eclasses, profiles or anything 144<p>It does not expand the scope of it to cover eclasses, profiles or anything
393else not already covered by the Manifest system, it also doesn't affect 145else not already covered by the Manifest system, it also doesn't affect
394the Manifest signing efforts in any way (though the implementations of both 146the Manifest signing efforts in any way (though the implementations of both
395might be coupled).</p> 147might be coupled).</p>
396<p>Also while multiple hash functions will become standard with the proposed 148<p>Also while multiple hash functions will become standard with the proposed
397implementation they are not a specific feature of this format <a class="footnote-reference" href="#multi-hash-thread" id="id3" name="id3">[2]</a>.</p> 149implementation they are not a specific feature of this format <a class="footnote-reference" href="#multi-hash-thread" id="id3">[2]</a>.</p>
398</div> 150</div>
399<div class="section"> 151<div class="section" id="number-of-hashes">
400<h2><a class="toc-backref" href="#id14" id="number-of-hashes" name="number-of-hashes">Number of hashes</a></h2> 152<h2><a class="toc-backref" href="#id14">Number of hashes</a></h2>
401<p>While using multiple hashes for each file is a major feature of this proposal 153<p>While using multiple hashes for each file is a major feature of this proposal
402we have to make sure that the number of hashes listed is limited to avoid 154we have to make sure that the number of hashes listed is limited to avoid
403an explosion of the Manifest size that would revert the main benefit of this proposal 155an explosion of the Manifest size that would revert the main benefit of this proposal
404(reduzing tree size). Therefore the number of hashes that will be generated 156(reduzing tree size). Therefore the number of hashes that will be generated
405will be limited to three different hash functions. For compability though we 157will be limited to three different hash functions. For compability though we
… …
407suggest to use SHA1 for this purpose (as it is supposed to be more secure than MD5 159suggest to use SHA1 for this purpose (as it is supposed to be more secure than MD5
408and currently only SHA1 and MD5 are directly available in python, also MD5 doesn't 160and currently only SHA1 and MD5 are directly available in python, also MD5 doesn't
409have any benefit in terms of compability).</p> 161have any benefit in terms of compability).</p>
410</div> 162</div>
411</div> 163</div>
412<div class="section"> 164<div class="section" id="rationale">
413<h1><a class="toc-backref" href="#id15" id="rationale" name="rationale">Rationale</a></h1> 165<h1><a class="toc-backref" href="#id15">Rationale</a></h1>
414<p>The main goals of the proposal have been listed in the <a class="reference" href="#motivation">Motivation</a>, here now 166<p>The main goals of the proposal have been listed in the <a class="reference internal" href="#motivation">Motivation</a>, here now
415the explanation why they are improvements and how the proposed format will 167the explanation why they are improvements and how the proposed format will
416accomplish them.</p> 168accomplish them.</p>
417<div class="section"> 169<div class="section" id="removal-of-digest-files">
418<h2><a class="toc-backref" href="#id16" id="removal-of-digest-files" name="removal-of-digest-files">Removal of digest files</a></h2> 170<h2><a class="toc-backref" href="#id16">Removal of digest files</a></h2>
419<p>Normal users that don't use a &quot;tuned&quot; filesystem for the portage tree are 171<p>Normal users that don't use a &quot;tuned&quot; filesystem for the portage tree are
420wasting several dozen to a few hundred megabytes of disk space with the current 172wasting several dozen to a few hundred megabytes of disk space with the current
421system, largely caused by the digest files. 173system, largely caused by the digest files.
422This is due to the filesystem overhead present in most filesystem that 174This is due to the filesystem overhead present in most filesystem that
423have a standard blocksize of four kilobytes while most digest files are under 175have a standard blocksize of four kilobytes while most digest files are under
… …
430verified yet.</p> 182verified yet.</p>
431<p>By unifying the digest files with the Manifest these tiny files are eliminated 183<p>By unifying the digest files with the Manifest these tiny files are eliminated
432(in the long run), reducing the apparent tree size by about 20%, benefitting 184(in the long run), reducing the apparent tree size by about 20%, benefitting
433both users and the Gentoo infrastructure.</p> 185both users and the Gentoo infrastructure.</p>
434</div> 186</div>
435<div class="section"> 187<div class="section" id="reducing-redundancy">
436<h2><a class="toc-backref" href="#id17" id="reducing-redundancy" name="reducing-redundancy">Reducing redundancy</a></h2> 188<h2><a class="toc-backref" href="#id17">Reducing redundancy</a></h2>
437<p>When multiple hashes are used with the current system 189<p>When multiple hashes are used with the current system
438both the filename and filesize are repeated for every checksum type used as each 190both the filename and filesize are repeated for every checksum type used as each
439checksum is standalone. However this doesn't add any functionality and is 191checksum is standalone. However this doesn't add any functionality and is
440therefore useless, so the new format removes this redundancy. 192therefore useless, so the new format removes this redundancy.
441This is a theoretical improvement at this moment as only one hash function is in 193This is a theoretical improvement at this moment as only one hash function is in
442use, but expected to change soon (see <a class="footnote-reference" href="#multi-hash-thread" id="id4" name="id4">[2]</a>).</p> 194use, but expected to change soon (see <a class="footnote-reference" href="#multi-hash-thread" id="id4">[2]</a>).</p>
443</div> 195</div>
444<div class="section"> 196<div class="section" id="removal-of-checksum-collisions">
445<h2><a class="toc-backref" href="#id18" id="removal-of-checksum-collisions" name="removal-of-checksum-collisions">Removal of checksum collisions</a></h2> 197<h2><a class="toc-backref" href="#id18">Removal of checksum collisions</a></h2>
446<p>The current system theoretically allows for a <tt class="docutils literal"><span class="pre">DIST</span></tt> type file to be recorded 198<p>The current system theoretically allows for a <tt class="docutils literal"><span class="pre">DIST</span></tt> type file to be recorded
447in multiple digest files with different sizes and/or checksums. In such a case 199in multiple digest files with different sizes and/or checksums. In such a case
448one version of a package would report a checksum violation while another one 200one version of a package would report a checksum violation while another one
449would not. This could create confusion and uncertainity among users. 201would not. This could create confusion and uncertainity among users.
450So far this case hasn't been observed, but it can't be ruled out with the 202So far this case hasn't been observed, but it can't be ruled out with the
451existing system. 203existing system.
452As the new format lists each file exactly once this would be no longer possible.</p> 204As the new format lists each file exactly once this would be no longer possible.</p>
453</div> 205</div>
454<div class="section"> 206<div class="section" id="flexible-verification-system">
455<h2><a class="toc-backref" href="#id19" id="flexible-verification-system" name="flexible-verification-system">Flexible verification system</a></h2> 207<h2><a class="toc-backref" href="#id19">Flexible verification system</a></h2>
456<p>Right now portage verifies the checksum of every file listed in the Manifest 208<p>Right now portage verifies the checksum of every file listed in the Manifest
457before using any file of the package and all <tt class="docutils literal"><span class="pre">DIST</span></tt> files of an ebuild 209before using any file of the package and all <tt class="docutils literal"><span class="pre">DIST</span></tt> files of an ebuild
458before using that ebuild. This is unnecessary in many cases:</p> 210before using that ebuild. This is unnecessary in many cases:</p>
459<ul class="simple"> 211<ul class="simple">
460<li>During the &quot;depend&quot; phase (when the ebuild metadata is generated) only 212<li>During the &quot;depend&quot; phase (when the ebuild metadata is generated) only
… …
471before unpacking them (which often will be one step), not every time their 223before unpacking them (which often will be one step), not every time their
472associated ebuild is used.</li> 224associated ebuild is used.</li>
473</ul> 225</ul>
474</div> 226</div>
475</div> 227</div>
476<div class="section"> 228<div class="section" id="backwards-compatibility">
477<h1><a class="toc-backref" href="#id20" id="backwards-compatibility" name="backwards-compatibility">Backwards Compatibility</a></h1> 229<h1><a class="toc-backref" href="#id20">Backwards Compatibility</a></h1>
478<p>Switching the Manifest system is a task that will need a long transition period 230<p>Switching the Manifest system is a task that will need a long transition period
479like most changes affecting both portage and the tree. In this case the 231like most changes affecting both portage and the tree. In this case the
480implementation will be rolled out in several phases:</p> 232implementation will be rolled out in several phases:</p>
481<ol class="arabic simple"> 233<ol class="arabic simple">
482<li>Add support for verification of Manifest2 entries in portage</li> 234<li>Add support for verification of Manifest2 entries in portage</li>
… …
514on the completion of each step.</p> 266on the completion of each step.</p>
515<p>In summary it can be said that while a full conversion will take over a year 267<p>In summary it can be said that while a full conversion will take over a year
516to be completed due to compability issues mentioned above some benefits of the 268to be completed due to compability issues mentioned above some benefits of the
517system can selectively be used as soon as step 2) is completed.</p> 269system can selectively be used as soon as step 2) is completed.</p>
518</div> 270</div>
519<div class="section"> 271<div class="section" id="other-problems">
520<h1><a class="toc-backref" href="#id21" id="other-problems" name="other-problems">Other problems</a></h1> 272<h1><a class="toc-backref" href="#id21">Other problems</a></h1>
521<div class="section"> 273<div class="section" id="impacts-on-infrastructure">
522<h2><a class="toc-backref" href="#id22" id="impacts-on-infrastructure" name="impacts-on-infrastructure">Impacts on infrastructure</a></h2> 274<h2><a class="toc-backref" href="#id22">Impacts on infrastructure</a></h2>
523<p>While one long term goal of this proposal is to reduce the size of the tree 275<p>While one long term goal of this proposal is to reduce the size of the tree
524and therefore make life for the Gentoo Infrastructure easier this will only 276and therefore make life for the Gentoo Infrastructure easier this will only
525take effect once the implementation is rolled out completely. In the meantime 277take effect once the implementation is rolled out completely. In the meantime
526however it will increase the tree size due to keeping checksums in both formats. 278however it will increase the tree size due to keeping checksums in both formats.
527It's not possible to give a usable estimate on the degree of the increase as 279It's not possible to give a usable estimate on the degree of the increase as
528it depends on many variables such as the exact implementation timeframe, 280it depends on many variables such as the exact implementation timeframe,
529propagation of Manifest2 capable portage versions among devs or the update 281propagation of Manifest2 capable portage versions among devs or the update
530rate of the tree. It has been suggested that Manifest files that are not gpg 282rate of the tree. It has been suggested that Manifest files that are not gpg
531signed could be mass converted in one step, this could certainly help but only 283signed could be mass converted in one step, this could certainly help but only
532to some degree (according to a recent research <a class="footnote-reference" href="#gpg-numbers" id="id5" name="id5">[3]</a> about 40% of 284to some degree (according to a recent research <a class="footnote-reference" href="#gpg-numbers" id="id5">[3]</a> about 40% of
533all Manifests in the tree are signed, but this number hasn't been verified).</p> 285all Manifests in the tree are signed, but this number hasn't been verified).</p>
534</div> 286</div>
535</div> 287</div>
536<div class="section"> 288<div class="section" id="reference-implementation">
537<h1><a class="toc-backref" href="#id23" id="reference-implementation" name="reference-implementation">Reference Implementation</a></h1> 289<h1><a class="toc-backref" href="#id23">Reference Implementation</a></h1>
538<p>A patch for a prototype implementation of Manifest2 verification and partial 290<p>A patch for a prototype implementation of Manifest2 verification and partial
539generation has been posted at <a class="footnote-reference" href="#manifest2-patch" id="id6" name="id6">[4]</a>, it will be reworked before 291generation has been posted at <a class="footnote-reference" href="#manifest2-patch" id="id6">[4]</a>, it will be reworked before
540being considered for inclusion in portage. However it shows that adding support 292being considered for inclusion in portage. However it shows that adding support
541for verification is quite simple, but generation is a bit tricky and will 293for verification is quite simple, but generation is a bit tricky and will
542therefore be implemented later.</p> 294therefore be implemented later.</p>
543</div> 295</div>
544<div class="section"> 296<div class="section" id="options">
545<h1><a class="toc-backref" href="#id24" id="options" name="options">Options</a></h1> 297<h1><a class="toc-backref" href="#id24">Options</a></h1>
546<p>Some things have been considered for this GLEP but aren't part of the proposal 298<p>Some things have been considered for this GLEP but aren't part of the proposal
547yet for various reasons:</p> 299yet for various reasons:</p>
548<ul class="simple"> 300<ul class="simple">
549<li>timestamp field: the author has considered adding a timestamp field for 301<li>timestamp field: the author has considered adding a timestamp field for
550each entry to list the time the entry was created. However so far no practical 302each entry to list the time the entry was created. However so far no practical
… …
562or packages directly they can contain viable information for users, so 314or packages directly they can contain viable information for users, so
563the author has the opinion that at least the option for integrity checks 315the author has the opinion that at least the option for integrity checks
564should be kept.</li> 316should be kept.</li>
565</ul> 317</ul>
566</div> 318</div>
567<div class="section"> 319<div class="section" id="credits">
568<h1><a class="toc-backref" href="#id25" id="credits" name="credits">Credits</a></h1> 320<h1><a class="toc-backref" href="#id25">Credits</a></h1>
569<p>Thanks to the following persons for their input on or related to this GLEP 321<p>Thanks to the following persons for their input on or related to this GLEP
570(even though they might not have known it): 322(even though they might not have known it):
571Ned Ludd (solar), Brian Harring (ferringb), Jason Stubbs (jstubbs), 323Ned Ludd (solar), Brian Harring (ferringb), Jason Stubbs (jstubbs),
572Robin H. Johnson (robbat2), Aron Griffis (agriffis)</p> 324Robin H. Johnson (robbat2), Aron Griffis (agriffis)</p>
573<p>Also thanks to Nicholas Jones (carpaski) to make the current Manifest system 325<p>Also thanks to Nicholas Jones (carpaski) to make the current Manifest system
574resistent enough to be able to handle this change without too many transition 326resistent enough to be able to handle this change without too many transition
575problems.</p> 327problems.</p>
576</div> 328</div>
577<div class="section"> 329<div class="section" id="references">
578<h1><a class="toc-backref" href="#id26" id="references" name="references">References</a></h1> 330<h1><a class="toc-backref" href="#id26">References</a></h1>
579<table class="docutils footnote" frame="void" id="reorg-thread" rules="none"> 331<table class="docutils footnote" frame="void" id="reorg-thread" rules="none">
580<colgroup><col class="label" /><col /></colgroup> 332<colgroup><col class="label" /><col /></colgroup>
581<tbody valign="top"> 333<tbody valign="top">
582<tr><td class="label"><a class="fn-backref" href="#id1" name="reorg-thread">[1]</a></td><td><a class="reference" href="http://thread.gmane.org/gmane.linux.gentoo.devel/21920">http://thread.gmane.org/gmane.linux.gentoo.devel/21920</a></td></tr> 334<tr><td class="label"><a class="fn-backref" href="#id1">[1]</a></td><td><a class="reference external" href="http://thread.gmane.org/gmane.linux.gentoo.devel/21920">http://thread.gmane.org/gmane.linux.gentoo.devel/21920</a></td></tr>
583</tbody> 335</tbody>
584</table> 336</table>
585<table class="docutils footnote" frame="void" id="multi-hash-thread" rules="none"> 337<table class="docutils footnote" frame="void" id="multi-hash-thread" rules="none">
586<colgroup><col class="label" /><col /></colgroup> 338<colgroup><col class="label" /><col /></colgroup>
587<tbody valign="top"> 339<tbody valign="top">
588<tr><td class="label"><a name="multi-hash-thread">[2]</a></td><td><em>(<a class="fn-backref" href="#id3">1</a>, <a class="fn-backref" href="#id4">2</a>)</em> <a class="reference" href="http://thread.gmane.org/gmane.linux.gentoo.devel/33434">http://thread.gmane.org/gmane.linux.gentoo.devel/33434</a></td></tr> 340<tr><td class="label">[2]</td><td><em>(<a class="fn-backref" href="#id3">1</a>, <a class="fn-backref" href="#id4">2</a>)</em> <a class="reference external" href="http://thread.gmane.org/gmane.linux.gentoo.devel/33434">http://thread.gmane.org/gmane.linux.gentoo.devel/33434</a></td></tr>
589</tbody> 341</tbody>
590</table> 342</table>
591<table class="docutils footnote" frame="void" id="gpg-numbers" rules="none"> 343<table class="docutils footnote" frame="void" id="gpg-numbers" rules="none">
592<colgroup><col class="label" /><col /></colgroup> 344<colgroup><col class="label" /><col /></colgroup>
593<tbody valign="top"> 345<tbody valign="top">
594<tr><td class="label"><a class="fn-backref" href="#id5" name="gpg-numbers">[3]</a></td><td>gentoo-core mailing list, topic &quot;Gentoo key signing practices 346<tr><td class="label"><a class="fn-backref" href="#id5">[3]</a></td><td>gentoo-core mailing list, topic &quot;Gentoo key signing practices
595and official Gentoo keyring&quot;, Message-ID &lt;<a class="reference" href="mailto:20051117075838.GB15734&#64;curie-int.vc.shawcable.net">20051117075838.GB15734&#64;curie-int.vc.shawcable.net</a>&gt;</td></tr> 347and official Gentoo keyring&quot;, Message-ID &lt;<a class="reference external" href="mailto:20051117075838.GB15734&#64;curie-int.vc.shawcable.net">20051117075838.GB15734&#64;curie-int.vc.shawcable.net</a>&gt;</td></tr>
596</tbody> 348</tbody>
597</table> 349</table>
598<table class="docutils footnote" frame="void" id="manifest2-patch" rules="none"> 350<table class="docutils footnote" frame="void" id="manifest2-patch" rules="none">
599<colgroup><col class="label" /><col /></colgroup> 351<colgroup><col class="label" /><col /></colgroup>
600<tbody valign="top"> 352<tbody valign="top">
601<tr><td class="label"><a class="fn-backref" href="#id6" name="manifest2-patch">[4]</a></td><td><a class="reference" href="http://thread.gmane.org/gmane.linux.gentoo.portage.devel/1374">http://thread.gmane.org/gmane.linux.gentoo.portage.devel/1374</a></td></tr> 353<tr><td class="label"><a class="fn-backref" href="#id6">[4]</a></td><td><a class="reference external" href="http://thread.gmane.org/gmane.linux.gentoo.portage.devel/1374">http://thread.gmane.org/gmane.linux.gentoo.portage.devel/1374</a></td></tr>
602</tbody> 354</tbody>
603</table> 355</table>
604<table class="docutils footnote" frame="void" id="manifest2-example" rules="none"> 356<table class="docutils footnote" frame="void" id="manifest2-example" rules="none">
605<colgroup><col class="label" /><col /></colgroup> 357<colgroup><col class="label" /><col /></colgroup>
606<tbody valign="top"> 358<tbody valign="top">
607<tr><td class="label"><a name="manifest2-example">[5]</a></td><td><a class="reference" href="http://www.gentoo.org/proj/en/glep/glep-0044-extras/manifest2-example">http://www.gentoo.org/proj/en/glep/glep-0044-extras/manifest2-example</a></td></tr> 359<tr><td class="label">[5]</td><td><a class="reference external" href="http://www.gentoo.org/proj/en/glep/glep-0044-extras/manifest2-example">http://www.gentoo.org/proj/en/glep/glep-0044-extras/manifest2-example</a></td></tr>
608</tbody> 360</tbody>
609</table> 361</table>
610<table class="docutils footnote" frame="void" id="id7" rules="none"> 362<table class="docutils footnote" frame="void" id="id7" rules="none">
611<colgroup><col class="label" /><col /></colgroup> 363<colgroup><col class="label" /><col /></colgroup>
612<tbody valign="top"> 364<tbody valign="top">
613<tr><td class="label"><a class="fn-backref" href="#id8" name="id7">[6]</a></td><td><a class="reference" href="glep-0044-extras/manifest2-example.txt">glep-0044-extras/manifest2-example.txt</a></td></tr> 365<tr><td class="label"><a class="fn-backref" href="#id8">[6]</a></td><td><a class="reference external" href="glep-0044-extras/manifest2-example.txt">glep-0044-extras/manifest2-example.txt</a></td></tr>
614</tbody> 366</tbody>
615</table> 367</table>
616</div> 368</div>
617<div class="section"> 369<div class="section" id="copyright">
618<h1><a class="toc-backref" href="#id27" id="copyright" name="copyright">Copyright</a></h1> 370<h1><a class="toc-backref" href="#id27">Copyright</a></h1>
619<p>This document has been placed in the public domain.</p> 371<p>This document has been placed in the public domain.</p>
620</div> 372</div>
621 373
622</div> 374</div>
623<div class="footer"> 375<div class="footer">
624<hr class="footer" /> 376<hr class="footer" />
625<a class="reference" href="glep-0044.txt">View document source</a>. 377<a class="reference external" href="glep-0044.txt">View document source</a>.
626Generated on: 2006-10-14 03:01 UTC. 378Generated on: 2009-01-11 19:35 UTC.
627Generated by <a class="reference" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source. 379Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.
628 380
629</div> 381</div>
630</body> 382</body>
631</html> 383</html>
632
Legend:
Removed from v.1.9  
changed lines
  Added in v.1.11
  ViewVC Help
Powered by ViewVC 1.1.20