/[gentoo]/xml/htdocs/proj/en/glep/glep-0044.html
Gentoo

Diff of /xml/htdocs/proj/en/glep/glep-0044.html

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

Revision 1.2 Revision 1.4
… …
30<tbody valign="top"> 30<tbody valign="top">
31<tr class="field"><th class="field-name">GLEP:</th><td class="field-body">44</td> 31<tr class="field"><th class="field-name">GLEP:</th><td class="field-body">44</td>
32</tr> 32</tr>
33<tr class="field"><th class="field-name">Title:</th><td class="field-body">Manifest2 format</td> 33<tr class="field"><th class="field-name">Title:</th><td class="field-body">Manifest2 format</td>
34</tr> 34</tr>
35<tr class="field"><th class="field-name">Version:</th><td class="field-body">1.1</td> 35<tr class="field"><th class="field-name">Version:</th><td class="field-body">1.3</td>
36</tr> 36</tr>
37<tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference" href="http://www.gentoo.org/cgi-bin/viewcvs/xml/htdocs/proj/en/glep/glep-0044.txt?cvsroot=gentoo">2005/12/06 03:34:21</a></td> 37<tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference" href="http://www.gentoo.org/cgi-bin/viewcvs/xml/htdocs/proj/en/glep/glep-0044.txt?cvsroot=gentoo">2006/01/23 10:24:24</a></td>
38</tr> 38</tr>
39<tr class="field"><th class="field-name">Author:</th><td class="field-body">Marius Mauch &lt;genone&#32;&#97;t&#32;gentoo.org&gt;,</td> 39<tr class="field"><th class="field-name">Author:</th><td class="field-body">Marius Mauch &lt;genone&#32;&#97;t&#32;gentoo.org&gt;,</td>
40</tr> 40</tr>
41<tr class="field"><th class="field-name">Status:</th><td class="field-body">Draft</td> 41<tr class="field"><th class="field-name">Status:</th><td class="field-body">Draft</td>
42</tr> 42</tr>
… …
44</tr> 44</tr>
45<tr class="field"><th class="field-name">Content-Type:</th><td class="field-body"><a class="reference" href="http://www.python.org/peps/glep-0012.html">text/x-rst</a></td> 45<tr class="field"><th class="field-name">Content-Type:</th><td class="field-body"><a class="reference" href="http://www.python.org/peps/glep-0012.html">text/x-rst</a></td>
46</tr> 46</tr>
47<tr class="field"><th class="field-name">Created:</th><td class="field-body">04-Dec-2005</td> 47<tr class="field"><th class="field-name">Created:</th><td class="field-body">04-Dec-2005</td>
48</tr> 48</tr>
49<tr class="field"><th class="field-name">Post-History:</th><td class="field-body">06-Dec-2005</td> 49<tr class="field"><th class="field-name">Post-History:</th><td class="field-body">06-Dec-2005, 23-Jan-2006</td>
50</tr> 50</tr>
51</tbody> 51</tbody>
52</table> 52</table>
53<hr /> 53<hr />
54<div class="contents topic" id="contents"> 54<div class="contents topic" id="contents">
55<p class="topic-title first"><a name="contents">Contents</a></p> 55<p class="topic-title first"><a name="contents">Contents</a></p>
56<ul class="simple"> 56<ul class="simple">
57<li><a class="reference" href="#abstract" id="id6" name="id6">Abstract</a></li> 57<li><a class="reference" href="#abstract" id="id9" name="id9">Abstract</a></li>
58<li><a class="reference" href="#motivation" id="id7" name="id7">Motivation</a></li> 58<li><a class="reference" href="#motivation" id="id10" name="id10">Motivation</a></li>
59<li><a class="reference" href="#specification" id="id8" name="id8">Specification</a><ul> 59<li><a class="reference" href="#specification" id="id11" name="id11">Specification</a><ul>
60<li><a class="reference" href="#compability-entries" id="id9" name="id9">Compability Entries</a></li> 60<li><a class="reference" href="#compability-entries" id="id12" name="id12">Compability Entries</a></li>
61<li><a class="reference" href="#scope" id="id10" name="id10">Scope</a></li> 61<li><a class="reference" href="#scope" id="id13" name="id13">Scope</a></li>
62</ul> 62<li><a class="reference" href="#number-of-hashes" id="id14" name="id14">Number of hashes</a></li>
63</li> 63</ul>
64</li>
64<li><a class="reference" href="#rationale" id="id11" name="id11">Rationale</a><ul> 65<li><a class="reference" href="#rationale" id="id15" name="id15">Rationale</a><ul>
65<li><a class="reference" href="#removal-of-digest-files" id="id12" name="id12">Removal of digest files</a></li> 66<li><a class="reference" href="#removal-of-digest-files" id="id16" name="id16">Removal of digest files</a></li>
66<li><a class="reference" href="#reducing-redundancy" id="id13" name="id13">Reducing redundancy</a></li> 67<li><a class="reference" href="#reducing-redundancy" id="id17" name="id17">Reducing redundancy</a></li>
67<li><a class="reference" href="#removal-of-checksum-collisions" id="id14" name="id14">Removal of checksum collisions</a></li> 68<li><a class="reference" href="#removal-of-checksum-collisions" id="id18" name="id18">Removal of checksum collisions</a></li>
68<li><a class="reference" href="#flexible-verification-system" id="id15" name="id15">Flexible verification system</a></li> 69<li><a class="reference" href="#flexible-verification-system" id="id19" name="id19">Flexible verification system</a></li>
69</ul>
70</li> 70</ul>
71</li>
71<li><a class="reference" href="#backwards-compatibility" id="id16" name="id16">Backwards Compatibility</a></li> 72<li><a class="reference" href="#backwards-compatibility" id="id20" name="id20">Backwards Compatibility</a></li>
72<li><a class="reference" href="#other-problems" id="id17" name="id17">Other problems</a><ul> 73<li><a class="reference" href="#other-problems" id="id21" name="id21">Other problems</a><ul>
73<li><a class="reference" href="#impacts-on-infrastructure" id="id18" name="id18">Impacts on infrastructure</a></li> 74<li><a class="reference" href="#impacts-on-infrastructure" id="id22" name="id22">Impacts on infrastructure</a></li>
74</ul>
75</li> 75</ul>
76</li>
76<li><a class="reference" href="#reference-implementation" id="id19" name="id19">Reference Implementation</a></li> 77<li><a class="reference" href="#reference-implementation" id="id23" name="id23">Reference Implementation</a></li>
77<li><a class="reference" href="#options" id="id20" name="id20">Options</a></li> 78<li><a class="reference" href="#options" id="id24" name="id24">Options</a></li>
78<li><a class="reference" href="#credits" id="id21" name="id21">Credits</a></li> 79<li><a class="reference" href="#credits" id="id25" name="id25">Credits</a></li>
79<li><a class="reference" href="#references" id="id22" name="id22">References</a></li> 80<li><a class="reference" href="#references" id="id26" name="id26">References</a></li>
80<li><a class="reference" href="#copyright" id="id23" name="id23">Copyright</a></li> 81<li><a class="reference" href="#copyright" id="id27" name="id27">Copyright</a></li>
81</ul> 82</ul>
82</div> 83</div>
83<div class="section" id="abstract"> 84<div class="section" id="abstract">
84<h1><a class="toc-backref" href="#id6" name="abstract">Abstract</a></h1> 85<h1><a class="toc-backref" href="#id9" name="abstract">Abstract</a></h1>
85<p>This GLEP proposes a new format for the Portage Manifest and digest file system 86<p>This GLEP proposes a new format for the Portage Manifest and digest file system
86by unifying both filetypes into one to improve functional and non-functional 87by unifying both filetypes into one to improve functional and non-functional
87aspects of the Portage Tree.</p> 88aspects of the Portage Tree.</p>
88</div> 89</div>
89<div class="section" id="motivation"> 90<div class="section" id="motivation">
90<h1><a class="toc-backref" href="#id7" name="motivation">Motivation</a></h1> 91<h1><a class="toc-backref" href="#id10" name="motivation">Motivation</a></h1>
91<p>Please see <a class="footnote-reference" href="#reorg-thread" id="id1" name="id1">[1]</a> for a general overview. 92<p>Please see <a class="footnote-reference" href="#reorg-thread" id="id1" name="id1">[1]</a> for a general overview.
92The main long term goals of this proposal are to:</p> 93The main long term goals of this proposal are to:</p>
93<ul class="simple"> 94<ul class="simple">
94<li>Remove the tiny digest files from the tree. They are a major annoyance as on a 95<li>Remove the tiny digest files from the tree. They are a major annoyance as on a
95typical configuration they waste a lot of discspace and the simple transmission 96typical configuration they waste a lot of disk space and the simple transmission
96of the names for all digest files during a <tt class="docutils literal"><span class="pre">emerge</span> <span class="pre">--sync</span></tt> needs a substantial 97of the names for all digest files during a <tt class="docutils literal"><span class="pre">emerge</span> <span class="pre">--sync</span></tt> needs a substantial
97amount of bandwidth.</li> 98amount of bandwidth.</li>
98<li>Reduce redundancy when multiple hash functions are used</li> 99<li>Reduce redundancy when multiple hash functions are used</li>
99<li>Remove potential for checksum collisions if a file is recorded in more than one 100<li>Remove potential for checksum collisions if a file is recorded in more than one
100digest file</li> 101digest file</li>
101<li>Difference between filetypes for a more flexible verification system</li> 102<li>Difference between filetypes for a more flexible verification system</li>
102</ul> 103</ul>
103</div> 104</div>
104<div class="section" id="specification"> 105<div class="section" id="specification">
105<h1><a class="toc-backref" href="#id8" name="specification">Specification</a></h1> 106<h1><a class="toc-backref" href="#id11" name="specification">Specification</a></h1>
106<p>The new Manifest format would change the existing format in the following ways:</p> 107<p>The new Manifest format would change the existing format in the following ways:</p>
107<ul> 108<ul>
108<li><p class="first">Addition of a filetype specifier, currently planned are</p> 109<li><p class="first">Addition of a filetype specifier, currently planned are</p>
109<ul class="simple"> 110<ul class="simple">
110<li><tt class="docutils literal"><span class="pre">AUXFILE</span></tt> for files directly used by ebuilds (e.g. patches or initscripts), 111<li><tt class="docutils literal"><span class="pre">AUX</span></tt> for files directly used by ebuilds (e.g. patches or initscripts),
111located in the <tt class="docutils literal"><span class="pre">files/</span></tt> subdirectory</li> 112located in the <tt class="docutils literal"><span class="pre">files/</span></tt> subdirectory</li>
112<li><tt class="docutils literal"><span class="pre">EBUILD</span></tt> for all ebuilds</li> 113<li><tt class="docutils literal"><span class="pre">EBUILD</span></tt> for all ebuilds</li>
113<li><tt class="docutils literal"><span class="pre">MISCFILE</span></tt> for files not directly used by ebuilds like <tt class="docutils literal"><span class="pre">ChangeLog</span></tt> or 114<li><tt class="docutils literal"><span class="pre">MISC</span></tt> for files not directly used by ebuilds like <tt class="docutils literal"><span class="pre">ChangeLog</span></tt> or
114<tt class="docutils literal"><span class="pre">metadata.xml</span></tt> files</li> 115<tt class="docutils literal"><span class="pre">metadata.xml</span></tt> files</li>
115<li><tt class="docutils literal"><span class="pre">SRCURI</span></tt> for release tarballs recorded in the <tt class="docutils literal"><span class="pre">SRC_URI</span></tt> variable of an ebuild, 116<li><tt class="docutils literal"><span class="pre">DIST</span></tt> for release tarballs recorded in the <tt class="docutils literal"><span class="pre">SRC_URI</span></tt> variable of an ebuild,
116these were previously recorded in the digest files</li> 117these were previously recorded in the digest files</li>
117</ul> 118</ul>
118<p>Future portage improvements might extend this list (for example with types 119<p>Future portage improvements might extend this list (for example with types
119relevant for eclasses or profiles)</p> 120relevant for eclasses or profiles)</p>
120</li> 121</li>
… …
127<p>Each line in the new format has the following format:</p> 128<p>Each line in the new format has the following format:</p>
128<pre class="literal-block"> 129<pre class="literal-block">
129&lt;filetype&gt; &lt;filename&gt; &lt;filesize&gt; &lt;chksumtype1&gt; &lt;chksum1&gt; ... &lt;chksumtypen&gt; &lt;chksumn&gt; 130&lt;filetype&gt; &lt;filename&gt; &lt;filesize&gt; &lt;chksumtype1&gt; &lt;chksum1&gt; ... &lt;chksumtypen&gt; &lt;chksumn&gt;
130</pre> 131</pre>
131<p>However theses entries will be stored in the existing Manifest files.</p> 132<p>However theses entries will be stored in the existing Manifest files.</p>
132<p>An actual example for a (pure) Manifest2 file could look like this (using 133<p>An <a class="reference" href="glep-0044-extras/manifest2-example">actual example</a> <a class="footnote-reference" href="#id7" id="id8" name="id8">[6]</a> for a (pure) Manifest2 file..</p>
133indentation to indicate line continuation):</p>
134<pre class="literal-block">
135AUXFILE ldif-buffer-overflow-fix.diff 5007 RMD160 1354a6bd2687430b628b78aaf43f5c793d2f0704
136 SHA1 424e1dfca06488f605b9611160020227ecdd03ac MD5 06d23c04b3d6ddfb1431c22ecc5b28f6
137AUXFILE procmime.patch 977 RMD160 39a51a4d654759b15d1644a79fb6e8921130df3c
138 SHA1 d76929f6dfc2179281f7ccee5789aab4e970ba9e MD5 bf4c9cd9cb7cdc6ece7d4d327910f0cf
139EBUILD sylpheed-claws-1.0.5-r1.ebuild 3906 RMD160 cdd546c128db2dea7044437de01ec96e12b4f5bf
140 SHA1 a84b49e76961d7a9100852b64c2bfbf9b053d45e MD5 b9fe79135a475458ef1b2240ee302ebd
141EBUILD sylpheed-claws-1.9.100.ebuild 4444 RMD160 89326038bfc694dafd22f10400a08d3f930fb2bd
142 SHA1 8895342f3f0cc6fcbdd0fdada2ad8e23ce539d23 MD5 0643de736b42d8c0e1673e86ae0b7f80
143EBUILD sylpheed-claws-1.9.15.ebuild 4821 RMD160 ec0ff811b893084459fe5b17b8ba8d6b35a55687
144 SHA1 358278a43da244e1f4803ec4b04d6fa45c41ab4d MD5 15b5c9348ba0b0a416892588256b4cbc
145MISCFILE ChangeLog 25770 RMD160 0e69dd7425add1560d630dd3367342418e9be776
146 SHA1 1210160f7baf0319de3b1b58dc80d7680d316d28 MD5 732cdc3b41403a115970d497a9ec257e
147MISCFILE metadata.xml 269 RMD160 39d775de55f9963f8946feaf088aa0324770bacb
148 SHA1 4fd7b285049d0e587f89e86becf06c0fd77bae6d MD5 82e806ed62f0596fb7bef493d225712f
149SRCURI sylpheed-claws-1.0.5.tar.bz2 3268626 RMD160 f2708b5d69bc9a5025812511fde04eca7782e367
150 SHA1 d351d7043eef7a875df18a8c4b9464be49e2164b MD5 ef4a1a7beb407dc7c31b4799bc48f12e
151SRCURI sylpheed-claws-1.9.100.tar.bz2 3480063 RMD160 72fbcbcc05d966f34897efcc1c96377420dc5544
152 SHA1 47465662b5470af5711493ce4eaad764c5bf02ca MD5 863c314557f90f17c2f6d6a0ab57e6c2
153SRCURI sylpheed-claws-1.9.15.tar.bz2 3481018 RMD160 b01d1af2df55806a8a8275102b10e389e0d98e94
154 SHA1 a17fc64b8dcc5b56432e5beb5c826913cb3ad79e MD5 0d187526e0eca23b87ffa4981f7e1765
155</pre>
156<div class="section" id="compability-entries"> 134<div class="section" id="compability-entries">
157<h2><a class="toc-backref" href="#id9" name="compability-entries">Compability Entries</a></h2> 135<h2><a class="toc-backref" href="#id12" name="compability-entries">Compability Entries</a></h2>
158<p>To maintain compability with existing portage versions a transition period after 136<p>To maintain compability with existing portage versions a transition period after
159is the introduction of the Manifest2 format is required during which portage 137is the introduction of the Manifest2 format is required during which portage
160will not only have to be capable of using existing Manifest and digest files but 138will not only have to be capable of using existing Manifest and digest files but
161also generate them in addition to the new entries. 139also generate them in addition to the new entries.
162Fortunately this can be accomplished by simply mixing old and new style entries 140Fortunately this can be accomplished by simply mixing old and new style entries
163in one file for the Manifest files, existing portage versions will simply ignore 141in one file for the Manifest files, existing portage versions will simply ignore
164the new style entries. For the digest files there are no new entries to care 142the new style entries. For the digest files there are no new entries to care
165about.</p> 143about.</p>
166</div> 144</div>
167<div class="section" id="scope"> 145<div class="section" id="scope">
168<h2><a class="toc-backref" href="#id10" name="scope">Scope</a></h2> 146<h2><a class="toc-backref" href="#id13" name="scope">Scope</a></h2>
169<p>It is important to note that this proposal only deals with a change of the 147<p>It is important to note that this proposal only deals with a change of the
170format of the digest and Manifest system.</p> 148format of the digest and Manifest system.</p>
171<p>It does not expand the scope of it to cover eclasses, profiles or anything 149<p>It does not expand the scope of it to cover eclasses, profiles or anything
172else not already covered by the Manifest system, it also doesn't affect 150else not already covered by the Manifest system, it also doesn't affect
173the Manifest signing efforts in any way (though the implementations of both 151the Manifest signing efforts in any way (though the implementations of both
174might be coupled).</p> 152might be coupled).</p>
175<p>Also while multiple hash functions will become standard with the proposed 153<p>Also while multiple hash functions will become standard with the proposed
176implementation they are not a specific feature of this format <a class="footnote-reference" href="#multi-hash-thread" id="id2" name="id2">[2]</a>.</p> 154implementation they are not a specific feature of this format <a class="footnote-reference" href="#multi-hash-thread" id="id3" name="id3">[2]</a>.</p>
155</div>
156<div class="section" id="number-of-hashes">
157<h2><a class="toc-backref" href="#id14" name="number-of-hashes">Number of hashes</a></h2>
158<p>While using multiple hashes for each file is a major feature of this proposal
159we have to make sure that the number of hashes listed is limited to avoid
160an explosion of the Manifest size that would revert the main benefit of this proposal
161(reduzing tree size). Therefore the number of hashes that will be generated
162will be limited to three different hash functions. For compability though we
163have to rely on at least one hash function to always be present, this proposal
164suggest to use SHA1 for this purpose (as it is supposed to be more secure than MD5
165and currently only SHA1 and MD5 are directly available in python, also MD5 doesn't
166have any benefit in terms of compability).</p>
177</div> 167</div>
178</div> 168</div>
179<div class="section" id="rationale"> 169<div class="section" id="rationale">
180<h1><a class="toc-backref" href="#id11" name="rationale">Rationale</a></h1> 170<h1><a class="toc-backref" href="#id15" name="rationale">Rationale</a></h1>
181<p>The main goals of the proposal have been listed in the <a class="reference" href="#motivation">Motivation</a>, here now 171<p>The main goals of the proposal have been listed in the <a class="reference" href="#motivation">Motivation</a>, here now
182the explanation why they are improvements and how the proposed format will 172the explanation why they are improvements and how the proposed format will
183accomplish them.</p> 173accomplish them.</p>
184<div class="section" id="removal-of-digest-files"> 174<div class="section" id="removal-of-digest-files">
185<h2><a class="toc-backref" href="#id12" name="removal-of-digest-files">Removal of digest files</a></h2> 175<h2><a class="toc-backref" href="#id16" name="removal-of-digest-files">Removal of digest files</a></h2>
186<p>Normal users that don't use a &quot;tuned&quot; filesystem for the portage tree are 176<p>Normal users that don't use a &quot;tuned&quot; filesystem for the portage tree are
187wasting several dozen to a few hundred megabytes of discspace with the current 177wasting several dozen to a few hundred megabytes of disk space with the current
188system, largely caused by the digest files. 178system, largely caused by the digest files.
189This is due to the filesystem overhead present in most filesystem that 179This is due to the filesystem overhead present in most filesystem that
190have a standard blocksize of four kilobytes while most digest files are under 180have a standard blocksize of four kilobytes while most digest files are under
191one kilobyte in size, so this results in approximately a waste of three kilobytes 181one kilobyte in size, so this results in approximately a waste of three kilobytes
192per digest file (likely even more). At the time of this writing the tree contains 182per digest file (likely even more). At the time of this writing the tree contains
193roughly 22.000 digest files, so the overall waste caused by digest files is 183roughly 22.000 digest files, so the overall waste caused by digest files is
194estimated at about 70-100 megabytes. 184estimated at about 70-100 megabytes.
195Furthermore it is assumed that this will also reduce the discspace wasted by 185Furthermore it is assumed that this will also reduce the disk space wasted by
196the Manifest files as they now contain more content, but this hasn't been 186the Manifest files as they now contain more content, but this hasn't been
197verified yet.</p> 187verified yet.</p>
198<p>By unifying the digest files with the Manifest these tiny files are eliminated 188<p>By unifying the digest files with the Manifest these tiny files are eliminated
199(in the long run), reducing the apparent tree size by about 20%, benefitting 189(in the long run), reducing the apparent tree size by about 20%, benefitting
200both users and the Gentoo infrastructure.</p> 190both users and the Gentoo infrastructure.</p>
201</div> 191</div>
202<div class="section" id="reducing-redundancy"> 192<div class="section" id="reducing-redundancy">
203<h2><a class="toc-backref" href="#id13" name="reducing-redundancy">Reducing redundancy</a></h2> 193<h2><a class="toc-backref" href="#id17" name="reducing-redundancy">Reducing redundancy</a></h2>
204<p>When multiple hashes are used with the current system 194<p>When multiple hashes are used with the current system
205both the filename and filesize are repeated for every checksum type used as each 195both the filename and filesize are repeated for every checksum type used as each
206checksum is standalone. However this doesn't add any functionality and is 196checksum is standalone. However this doesn't add any functionality and is
207therefore useless, so the new format removes this redundancy. 197therefore useless, so the new format removes this redundancy.
208This is a theoretical improvement at this moment as only one hash function is in 198This is a theoretical improvement at this moment as only one hash function is in
209use, but expected to change soon (see <a class="footnote-reference" href="#multi-hash-thread" id="id3" name="id3">[2]</a>).</p> 199use, but expected to change soon (see <a class="footnote-reference" href="#multi-hash-thread" id="id4" name="id4">[2]</a>).</p>
210</div> 200</div>
211<div class="section" id="removal-of-checksum-collisions"> 201<div class="section" id="removal-of-checksum-collisions">
212<h2><a class="toc-backref" href="#id14" name="removal-of-checksum-collisions">Removal of checksum collisions</a></h2> 202<h2><a class="toc-backref" href="#id18" name="removal-of-checksum-collisions">Removal of checksum collisions</a></h2>
213<p>The current system theoretically allows for a <tt class="docutils literal"><span class="pre">SRCURI</span></tt> type file to be recorded 203<p>The current system theoretically allows for a <tt class="docutils literal"><span class="pre">DIST</span></tt> type file to be recorded
214in multiple digest files with different sizes and/or checksums. In such a case 204in multiple digest files with different sizes and/or checksums. In such a case
215one version of a package would report a checksum violation while another one 205one version of a package would report a checksum violation while another one
216would not. This could create confusion and uncertainity among users. 206would not. This could create confusion and uncertainity among users.
217So far this case hasn't been observed, but it can't be ruled out with the 207So far this case hasn't been observed, but it can't be ruled out with the
218existing system. 208existing system.
219As the new format lists each file exactly once this would be no longer possible.</p> 209As the new format lists each file exactly once this would be no longer possible.</p>
220</div> 210</div>
221<div class="section" id="flexible-verification-system"> 211<div class="section" id="flexible-verification-system">
222<h2><a class="toc-backref" href="#id15" name="flexible-verification-system">Flexible verification system</a></h2> 212<h2><a class="toc-backref" href="#id19" name="flexible-verification-system">Flexible verification system</a></h2>
223<p>Right now portage verifies the checksum of every file listed in the Manifest 213<p>Right now portage verifies the checksum of every file listed in the Manifest
224before using any file of the package and all <tt class="docutils literal"><span class="pre">SRCURI</span></tt> files of an ebuild 214before using any file of the package and all <tt class="docutils literal"><span class="pre">DIST</span></tt> files of an ebuild
225before using that ebuild. This is unnecessary in many cases:</p> 215before using that ebuild. This is unnecessary in many cases:</p>
226<ul class="simple"> 216<ul class="simple">
227<li>During the &quot;depend&quot; phase (when the ebuild metadata is generated) only 217<li>During the &quot;depend&quot; phase (when the ebuild metadata is generated) only
228files of type <tt class="docutils literal"><span class="pre">EBUILD</span></tt> are used, so verifying the other types isn't 218files of type <tt class="docutils literal"><span class="pre">EBUILD</span></tt> are used, so verifying the other types isn't
229necessary. Theoretically it is possible for an ebuild to include other 219necessary. Theoretically it is possible for an ebuild to include other
230files like those of type <tt class="docutils literal"><span class="pre">AUXFILE</span></tt> at this phase, but that would be a 220files like those of type <tt class="docutils literal"><span class="pre">AUX</span></tt> at this phase, but that would be a
231major QA violation and should never occur, so it can be ignored here. 221major QA violation and should never occur, so it can be ignored here.
232It is also not a security concern as the ebuild is verified before parsing 222It is also not a security concern as the ebuild is verified before parsing
233it, so each manipulation would show up.</li> 223it, so each manipulation would show up.</li>
234<li>Generally files of type <tt class="docutils literal"><span class="pre">MISCFILE</span></tt> don't need to be verified as they are 224<li>Generally files of type <tt class="docutils literal"><span class="pre">MISC</span></tt> don't need to be verified as they are
235only used in very specific situations, aren't executed (just parsed at most) 225only used in very specific situations, aren't executed (just parsed at most)
236and don't affect the package build process.</li> 226and don't affect the package build process.</li>
237<li>Files of type <tt class="docutils literal"><span class="pre">SRCURI</span></tt> only need to be verified directly after fetching and 227<li>Files of type <tt class="docutils literal"><span class="pre">DIST</span></tt> only need to be verified directly after fetching and
238before unpacking them (which often will be one step), not every time their 228before unpacking them (which often will be one step), not every time their
239associated ebuild is used.</li> 229associated ebuild is used.</li>
240</ul> 230</ul>
241</div> 231</div>
242</div> 232</div>
243<div class="section" id="backwards-compatibility"> 233<div class="section" id="backwards-compatibility">
244<h1><a class="toc-backref" href="#id16" name="backwards-compatibility">Backwards Compatibility</a></h1> 234<h1><a class="toc-backref" href="#id20" name="backwards-compatibility">Backwards Compatibility</a></h1>
245<p>Switching the Manifest system is a task that will need a long transition period 235<p>Switching the Manifest system is a task that will need a long transition period
246like most changes affecting both portage and the tree. In this case the 236like most changes affecting both portage and the tree. In this case the
247implementation will be rolled out in several phases:</p> 237implementation will be rolled out in several phases:</p>
248<ol class="arabic simple"> 238<ol class="arabic simple">
249<li>Add support for verification of Manifest2 entries in portage</li> 239<li>Add support for verification of Manifest2 entries in portage</li>
… …
282<p>In summary it can be said that while a full conversion will take over a year 272<p>In summary it can be said that while a full conversion will take over a year
283to be completed due to compability issues mentioned above some benefits of the 273to be completed due to compability issues mentioned above some benefits of the
284system can selectively be used as soon as step 2) is completed.</p> 274system can selectively be used as soon as step 2) is completed.</p>
285</div> 275</div>
286<div class="section" id="other-problems"> 276<div class="section" id="other-problems">
287<h1><a class="toc-backref" href="#id17" name="other-problems">Other problems</a></h1> 277<h1><a class="toc-backref" href="#id21" name="other-problems">Other problems</a></h1>
288<div class="section" id="impacts-on-infrastructure"> 278<div class="section" id="impacts-on-infrastructure">
289<h2><a class="toc-backref" href="#id18" name="impacts-on-infrastructure">Impacts on infrastructure</a></h2> 279<h2><a class="toc-backref" href="#id22" name="impacts-on-infrastructure">Impacts on infrastructure</a></h2>
290<p>While one long term goal of this proposal is to reduce the size of the tree 280<p>While one long term goal of this proposal is to reduce the size of the tree
291and therefore make life for the Gentoo Infrastructure easier this will only 281and therefore make life for the Gentoo Infrastructure easier this will only
292take effect once the implementation is rolled out completely. In the meantime 282take effect once the implementation is rolled out completely. In the meantime
293however it will increase the tree size due to keeping checksums in both formats. 283however it will increase the tree size due to keeping checksums in both formats.
294It's not possible to give a usable estimate on the degree of the increase as 284It's not possible to give a usable estimate on the degree of the increase as
295it depends on many variables such as the exact implementation timeframe, 285it depends on many variables such as the exact implementation timeframe,
296propagation of Manifest2 capable portage versions among devs or the update 286propagation of Manifest2 capable portage versions among devs or the update
297rate of the tree. It has been suggested that Manifest files that are not gpg 287rate of the tree. It has been suggested that Manifest files that are not gpg
298signed could be mass converted in one step, this could certainly help but only 288signed could be mass converted in one step, this could certainly help but only
299to some degree (according to a recent research <a class="footnote-reference" href="#gpg-numbers" id="id4" name="id4">[3]</a> about 40% of 289to some degree (according to a recent research <a class="footnote-reference" href="#gpg-numbers" id="id5" name="id5">[3]</a> about 40% of
300all Manifests in the tree are signed, but this number hasn't been verified).</p> 290all Manifests in the tree are signed, but this number hasn't been verified).</p>
301</div> 291</div>
302</div> 292</div>
303<div class="section" id="reference-implementation"> 293<div class="section" id="reference-implementation">
304<h1><a class="toc-backref" href="#id19" name="reference-implementation">Reference Implementation</a></h1> 294<h1><a class="toc-backref" href="#id23" name="reference-implementation">Reference Implementation</a></h1>
305<p>A patch for a prototype implementation of Manifest2 verification and partial 295<p>A patch for a prototype implementation of Manifest2 verification and partial
306generation has been posted at <a class="footnote-reference" href="#manifest2-patch" id="id5" name="id5">[4]</a>, it will be reworked before 296generation has been posted at <a class="footnote-reference" href="#manifest2-patch" id="id6" name="id6">[4]</a>, it will be reworked before
307being considered for inclusion in portage. However it shows that adding support 297being considered for inclusion in portage. However it shows that adding support
308for verification is quite simple, but generation is a bit tricky and will 298for verification is quite simple, but generation is a bit tricky and will
309therefore be implemented later.</p> 299therefore be implemented later.</p>
310</div> 300</div>
311<div class="section" id="options"> 301<div class="section" id="options">
312<h1><a class="toc-backref" href="#id20" name="options">Options</a></h1> 302<h1><a class="toc-backref" href="#id24" name="options">Options</a></h1>
313<p>Some things have been considered for this GLEP but aren't part of the proposal 303<p>Some things have been considered for this GLEP but aren't part of the proposal
314yet for various reasons:</p> 304yet for various reasons:</p>
315<ul class="simple"> 305<ul class="simple">
316<li>timestamp field: the author has considered adding a timestamp field for 306<li>timestamp field: the author has considered adding a timestamp field for
317each entry to list the time the entry was created. However so far no practical 307each entry to list the time the entry was created. However so far no practical
318use for such a feature has been found.</li> 308use for such a feature has been found.</li>
319<li>convert size field into checksum: Another idea was to treat the size field 309<li>convert size field into checksum: Another idea was to treat the size field
320like any other checksum. But so far no real benefit (other than a slightly 310like any other checksum. But so far no real benefit (other than a slightly
321more modular implementation) for this has been seen while it has several 311more modular implementation) for this has been seen while it has several
322drawbacks: For once, unlike checksums, the size field is definitely required 312drawbacks: For once, unlike checksums, the size field is definitely required
323for all <tt class="docutils literal"><span class="pre">SRCURI</span></tt> files, also it would slightly increase the length of 313for all <tt class="docutils literal"><span class="pre">DIST</span></tt> files, also it would slightly increase the length of
324each entry by adding a <tt class="docutils literal"><span class="pre">SIZE</span></tt> keyword.</li> 314each entry by adding a <tt class="docutils literal"><span class="pre">SIZE</span></tt> keyword.</li>
325<li>removal of the <tt class="docutils literal"><span class="pre">MISCFILE</span></tt> type: It has been suggested to completely drop 315<li>removal of the <tt class="docutils literal"><span class="pre">MISC</span></tt> type: It has been suggested to completely drop
326entries of type <tt class="docutils literal"><span class="pre">MISCFILE</span></tt>. This would result in a minor space reduction 316entries of type <tt class="docutils literal"><span class="pre">MISC</span></tt>. This would result in a minor space reduction
327(its rather unlikely to free any blocks) but completely remove the ability 317(its rather unlikely to free any blocks) but completely remove the ability
328to check these files for integrity. While they don't influence portage 318to check these files for integrity. While they don't influence portage
329or packages directly they can contain viable information for users, so 319or packages directly they can contain viable information for users, so
330the author has the opinion that at least the option for integrity checks 320the author has the opinion that at least the option for integrity checks
331should be kept.</li> 321should be kept.</li>
332</ul> 322</ul>
333</div> 323</div>
334<div class="section" id="credits"> 324<div class="section" id="credits">
335<h1><a class="toc-backref" href="#id21" name="credits">Credits</a></h1> 325<h1><a class="toc-backref" href="#id25" name="credits">Credits</a></h1>
336<p>Thanks to the following persons for their input on or related to this GLEP 326<p>Thanks to the following persons for their input on or related to this GLEP
337(even though they might not have known it): 327(even though they might not have known it):
338Ned Ludd (solar), Brian Harring (ferringb), Jason Stubbs (jstubbs), 328Ned Ludd (solar), Brian Harring (ferringb), Jason Stubbs (jstubbs),
339Robin H. Johnson (robbat2), Aron Griffis (agriffis)</p> 329Robin H. Johnson (robbat2), Aron Griffis (agriffis)</p>
340<p>Also thanks to Nicholas Jones (carpaski) to make the current Manifest system 330<p>Also thanks to Nicholas Jones (carpaski) to make the current Manifest system
341resistent enough to be able to handle this change without too many transition 331resistent enough to be able to handle this change without too many transition
342problems.</p> 332problems.</p>
343</div> 333</div>
344<div class="section" id="references"> 334<div class="section" id="references">
345<h1><a class="toc-backref" href="#id22" name="references">References</a></h1> 335<h1><a class="toc-backref" href="#id26" name="references">References</a></h1>
346<table class="docutils footnote" frame="void" id="reorg-thread" rules="none"> 336<table class="docutils footnote" frame="void" id="reorg-thread" rules="none">
347<colgroup><col class="label" /><col /></colgroup> 337<colgroup><col class="label" /><col /></colgroup>
348<tbody valign="top"> 338<tbody valign="top">
349<tr><td class="label"><a class="fn-backref" href="#id1" name="reorg-thread">[1]</a></td><td><a class="reference" href="http://thread.gmane.org/gmane.linux.gentoo.devel/21920">http://thread.gmane.org/gmane.linux.gentoo.devel/21920</a></td></tr> 339<tr><td class="label"><a class="fn-backref" href="#id1" name="reorg-thread">[1]</a></td><td><a class="reference" href="http://thread.gmane.org/gmane.linux.gentoo.devel/21920">http://thread.gmane.org/gmane.linux.gentoo.devel/21920</a></td></tr>
350</tbody> 340</tbody>
351</table> 341</table>
352<table class="docutils footnote" frame="void" id="multi-hash-thread" rules="none"> 342<table class="docutils footnote" frame="void" id="multi-hash-thread" rules="none">
353<colgroup><col class="label" /><col /></colgroup> 343<colgroup><col class="label" /><col /></colgroup>
354<tbody valign="top"> 344<tbody valign="top">
355<tr><td class="label"><a name="multi-hash-thread">[2]</a></td><td><em>(<a class="fn-backref" href="#id2">1</a>, <a class="fn-backref" href="#id3">2</a>)</em> <a class="reference" href="http://thread.gmane.org/gmane.linux.gentoo.devel/33434">http://thread.gmane.org/gmane.linux.gentoo.devel/33434</a></td></tr> 345<tr><td class="label"><a name="multi-hash-thread">[2]</a></td><td><em>(<a class="fn-backref" href="#id3">1</a>, <a class="fn-backref" href="#id4">2</a>)</em> <a class="reference" href="http://thread.gmane.org/gmane.linux.gentoo.devel/33434">http://thread.gmane.org/gmane.linux.gentoo.devel/33434</a></td></tr>
356</tbody> 346</tbody>
357</table> 347</table>
358<table class="docutils footnote" frame="void" id="gpg-numbers" rules="none"> 348<table class="docutils footnote" frame="void" id="gpg-numbers" rules="none">
359<colgroup><col class="label" /><col /></colgroup> 349<colgroup><col class="label" /><col /></colgroup>
360<tbody valign="top"> 350<tbody valign="top">
361<tr><td class="label"><a class="fn-backref" href="#id4" name="gpg-numbers">[3]</a></td><td>gentoo-core mailing list, topic &quot;Gentoo key signing practices 351<tr><td class="label"><a class="fn-backref" href="#id5" name="gpg-numbers">[3]</a></td><td>gentoo-core mailing list, topic &quot;Gentoo key signing practices
362and official Gentoo keyring&quot;, Message-ID &lt;<a class="reference" href="mailto:20051117075838.GB15734&#64;curie-int.vc.shawcable.net">20051117075838.GB15734&#64;curie-int.vc.shawcable.net</a>&gt;</td></tr> 352and official Gentoo keyring&quot;, Message-ID &lt;<a class="reference" href="mailto:20051117075838.GB15734&#64;curie-int.vc.shawcable.net">20051117075838.GB15734&#64;curie-int.vc.shawcable.net</a>&gt;</td></tr>
363</tbody> 353</tbody>
364</table> 354</table>
365<table class="docutils footnote" frame="void" id="manifest2-patch" rules="none"> 355<table class="docutils footnote" frame="void" id="manifest2-patch" rules="none">
366<colgroup><col class="label" /><col /></colgroup> 356<colgroup><col class="label" /><col /></colgroup>
367<tbody valign="top"> 357<tbody valign="top">
368<tr><td class="label"><a class="fn-backref" href="#id5" name="manifest2-patch">[4]</a></td><td><a class="reference" href="http://thread.gmane.org/gmane.linux.gentoo.portage.devel/1374">http://thread.gmane.org/gmane.linux.gentoo.portage.devel/1374</a></td></tr> 358<tr><td class="label"><a class="fn-backref" href="#id6" name="manifest2-patch">[4]</a></td><td><a class="reference" href="http://thread.gmane.org/gmane.linux.gentoo.portage.devel/1374">http://thread.gmane.org/gmane.linux.gentoo.portage.devel/1374</a></td></tr>
359</tbody>
360</table>
361<table class="docutils footnote" frame="void" id="manifest2-example" rules="none">
362<colgroup><col class="label" /><col /></colgroup>
363<tbody valign="top">
364<tr><td class="label"><a name="manifest2-example">[5]</a></td><td><a class="reference" href="http://www.gentoo.org/proj/en/glep/glep-0044-extras/manifest2-example">http://www.gentoo.org/proj/en/glep/glep-0044-extras/manifest2-example</a></td></tr>
365</tbody>
366</table>
367<table class="docutils footnote" frame="void" id="id7" rules="none">
368<colgroup><col class="label" /><col /></colgroup>
369<tbody valign="top">
370<tr><td class="label"><a class="fn-backref" href="#id8" name="id7">[6]</a></td><td><a class="reference" href="glep-0044-extras/manifest2-example">glep-0044-extras/manifest2-example</a></td></tr>
369</tbody> 371</tbody>
370</table> 372</table>
371</div> 373</div>
372<div class="section" id="copyright"> 374<div class="section" id="copyright">
373<h1><a class="toc-backref" href="#id23" name="copyright">Copyright</a></h1> 375<h1><a class="toc-backref" href="#id27" name="copyright">Copyright</a></h1>
374<p>This document has been placed in the public domain.</p> 376<p>This document has been placed in the public domain.</p>
375</div> 377</div>
376 378
377</div> 379</div>
378<div class="footer"> 380<div class="footer">
379<hr class="footer" /> 381<hr class="footer" />
380<a class="reference" href="glep-0044.txt">View document source</a>. 382<a class="reference" href="glep-0044.txt">View document source</a>.
381Generated on: 2005-12-06 16:19 UTC. 383Generated on: 2006-02-10 23:25 UTC.
382Generated by <a class="reference" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source. 384Generated by <a class="reference" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.
383 385
384</div> 386</div>
385</body> 387</body>
386</html> 388</html>
Legend:
Removed from v.1.2  
changed lines
  Added in v.1.4
  ViewVC Help
Powered by ViewVC 1.1.13