/[gentoo]/xml/htdocs/proj/en/glep/glep-0044.html
Gentoo

Diff of /xml/htdocs/proj/en/glep/glep-0044.html

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

Revision 1.2 Revision 1.3
30<tbody valign="top"> 30<tbody valign="top">
31<tr class="field"><th class="field-name">GLEP:</th><td class="field-body">44</td> 31<tr class="field"><th class="field-name">GLEP:</th><td class="field-body">44</td>
32</tr> 32</tr>
33<tr class="field"><th class="field-name">Title:</th><td class="field-body">Manifest2 format</td> 33<tr class="field"><th class="field-name">Title:</th><td class="field-body">Manifest2 format</td>
34</tr> 34</tr>
35<tr class="field"><th class="field-name">Version:</th><td class="field-body">1.1</td> 35<tr class="field"><th class="field-name">Version:</th><td class="field-body">1.2</td>
36</tr> 36</tr>
37<tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference" href="http://www.gentoo.org/cgi-bin/viewcvs/xml/htdocs/proj/en/glep/glep-0044.txt?cvsroot=gentoo">2005/12/06 03:34:21</a></td> 37<tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference" href="http://www.gentoo.org/cgi-bin/viewcvs/xml/htdocs/proj/en/glep/glep-0044.txt?cvsroot=gentoo">2005/12/06 16:19:37</a></td>
38</tr> 38</tr>
39<tr class="field"><th class="field-name">Author:</th><td class="field-body">Marius Mauch &lt;genone&#32;&#97;t&#32;gentoo.org&gt;,</td> 39<tr class="field"><th class="field-name">Author:</th><td class="field-body">Marius Mauch &lt;genone&#32;&#97;t&#32;gentoo.org&gt;,</td>
40</tr> 40</tr>
41<tr class="field"><th class="field-name">Status:</th><td class="field-body">Draft</td> 41<tr class="field"><th class="field-name">Status:</th><td class="field-body">Draft</td>
42</tr> 42</tr>
44</tr> 44</tr>
45<tr class="field"><th class="field-name">Content-Type:</th><td class="field-body"><a class="reference" href="http://www.python.org/peps/glep-0012.html">text/x-rst</a></td> 45<tr class="field"><th class="field-name">Content-Type:</th><td class="field-body"><a class="reference" href="http://www.python.org/peps/glep-0012.html">text/x-rst</a></td>
46</tr> 46</tr>
47<tr class="field"><th class="field-name">Created:</th><td class="field-body">04-Dec-2005</td> 47<tr class="field"><th class="field-name">Created:</th><td class="field-body">04-Dec-2005</td>
48</tr> 48</tr>
49<tr class="field"><th class="field-name">Post-History:</th><td class="field-body">06-Dec-2005</td> 49<tr class="field"><th class="field-name">Post-History:</th><td class="field-body">06-Dec-2005, 23-Jan-2006</td>
50</tr> 50</tr>
51</tbody> 51</tbody>
52</table> 52</table>
53<hr /> 53<hr />
54<div class="contents topic" id="contents"> 54<div class="contents topic" id="contents">
90<h1><a class="toc-backref" href="#id7" name="motivation">Motivation</a></h1> 90<h1><a class="toc-backref" href="#id7" name="motivation">Motivation</a></h1>
91<p>Please see <a class="footnote-reference" href="#reorg-thread" id="id1" name="id1">[1]</a> for a general overview. 91<p>Please see <a class="footnote-reference" href="#reorg-thread" id="id1" name="id1">[1]</a> for a general overview.
92The main long term goals of this proposal are to:</p> 92The main long term goals of this proposal are to:</p>
93<ul class="simple"> 93<ul class="simple">
94<li>Remove the tiny digest files from the tree. They are a major annoyance as on a 94<li>Remove the tiny digest files from the tree. They are a major annoyance as on a
95typical configuration they waste a lot of discspace and the simple transmission 95typical configuration they waste a lot of disk space and the simple transmission
96of the names for all digest files during a <tt class="docutils literal"><span class="pre">emerge</span> <span class="pre">--sync</span></tt> needs a substantial 96of the names for all digest files during a <tt class="docutils literal"><span class="pre">emerge</span> <span class="pre">--sync</span></tt> needs a substantial
97amount of bandwidth.</li> 97amount of bandwidth.</li>
98<li>Reduce redundancy when multiple hash functions are used</li> 98<li>Reduce redundancy when multiple hash functions are used</li>
99<li>Remove potential for checksum collisions if a file is recorded in more than one 99<li>Remove potential for checksum collisions if a file is recorded in more than one
100digest file</li> 100digest file</li>
110<li><tt class="docutils literal"><span class="pre">AUXFILE</span></tt> for files directly used by ebuilds (e.g. patches or initscripts), 110<li><tt class="docutils literal"><span class="pre">AUXFILE</span></tt> for files directly used by ebuilds (e.g. patches or initscripts),
111located in the <tt class="docutils literal"><span class="pre">files/</span></tt> subdirectory</li> 111located in the <tt class="docutils literal"><span class="pre">files/</span></tt> subdirectory</li>
112<li><tt class="docutils literal"><span class="pre">EBUILD</span></tt> for all ebuilds</li> 112<li><tt class="docutils literal"><span class="pre">EBUILD</span></tt> for all ebuilds</li>
113<li><tt class="docutils literal"><span class="pre">MISCFILE</span></tt> for files not directly used by ebuilds like <tt class="docutils literal"><span class="pre">ChangeLog</span></tt> or 113<li><tt class="docutils literal"><span class="pre">MISCFILE</span></tt> for files not directly used by ebuilds like <tt class="docutils literal"><span class="pre">ChangeLog</span></tt> or
114<tt class="docutils literal"><span class="pre">metadata.xml</span></tt> files</li> 114<tt class="docutils literal"><span class="pre">metadata.xml</span></tt> files</li>
115<li><tt class="docutils literal"><span class="pre">SRCURI</span></tt> for release tarballs recorded in the <tt class="docutils literal"><span class="pre">SRC_URI</span></tt> variable of an ebuild, 115<li><tt class="docutils literal"><span class="pre">DISTFILE</span></tt> for release tarballs recorded in the <tt class="docutils literal"><span class="pre">SRC_URI</span></tt> variable of an ebuild,
116these were previously recorded in the digest files</li> 116these were previously recorded in the digest files</li>
117</ul> 117</ul>
118<p>Future portage improvements might extend this list (for example with types 118<p>Future portage improvements might extend this list (for example with types
119relevant for eclasses or profiles)</p> 119relevant for eclasses or profiles)</p>
120</li> 120</li>
144 SHA1 358278a43da244e1f4803ec4b04d6fa45c41ab4d MD5 15b5c9348ba0b0a416892588256b4cbc 144 SHA1 358278a43da244e1f4803ec4b04d6fa45c41ab4d MD5 15b5c9348ba0b0a416892588256b4cbc
145MISCFILE ChangeLog 25770 RMD160 0e69dd7425add1560d630dd3367342418e9be776 145MISCFILE ChangeLog 25770 RMD160 0e69dd7425add1560d630dd3367342418e9be776
146 SHA1 1210160f7baf0319de3b1b58dc80d7680d316d28 MD5 732cdc3b41403a115970d497a9ec257e 146 SHA1 1210160f7baf0319de3b1b58dc80d7680d316d28 MD5 732cdc3b41403a115970d497a9ec257e
147MISCFILE metadata.xml 269 RMD160 39d775de55f9963f8946feaf088aa0324770bacb 147MISCFILE metadata.xml 269 RMD160 39d775de55f9963f8946feaf088aa0324770bacb
148 SHA1 4fd7b285049d0e587f89e86becf06c0fd77bae6d MD5 82e806ed62f0596fb7bef493d225712f 148 SHA1 4fd7b285049d0e587f89e86becf06c0fd77bae6d MD5 82e806ed62f0596fb7bef493d225712f
149SRCURI sylpheed-claws-1.0.5.tar.bz2 3268626 RMD160 f2708b5d69bc9a5025812511fde04eca7782e367 149DISTFILE sylpheed-claws-1.0.5.tar.bz2 3268626 RMD160 f2708b5d69bc9a5025812511fde04eca7782e367
150 SHA1 d351d7043eef7a875df18a8c4b9464be49e2164b MD5 ef4a1a7beb407dc7c31b4799bc48f12e 150 SHA1 d351d7043eef7a875df18a8c4b9464be49e2164b MD5 ef4a1a7beb407dc7c31b4799bc48f12e
151SRCURI sylpheed-claws-1.9.100.tar.bz2 3480063 RMD160 72fbcbcc05d966f34897efcc1c96377420dc5544 151DISTFILE sylpheed-claws-1.9.100.tar.bz2 3480063 RMD160 72fbcbcc05d966f34897efcc1c96377420dc5544
152 SHA1 47465662b5470af5711493ce4eaad764c5bf02ca MD5 863c314557f90f17c2f6d6a0ab57e6c2 152 SHA1 47465662b5470af5711493ce4eaad764c5bf02ca MD5 863c314557f90f17c2f6d6a0ab57e6c2
153SRCURI sylpheed-claws-1.9.15.tar.bz2 3481018 RMD160 b01d1af2df55806a8a8275102b10e389e0d98e94 153DISTFILE sylpheed-claws-1.9.15.tar.bz2 3481018 RMD160 b01d1af2df55806a8a8275102b10e389e0d98e94
154 SHA1 a17fc64b8dcc5b56432e5beb5c826913cb3ad79e MD5 0d187526e0eca23b87ffa4981f7e1765 154 SHA1 a17fc64b8dcc5b56432e5beb5c826913cb3ad79e MD5 0d187526e0eca23b87ffa4981f7e1765
155</pre> 155</pre>
156<div class="section" id="compability-entries"> 156<div class="section" id="compability-entries">
157<h2><a class="toc-backref" href="#id9" name="compability-entries">Compability Entries</a></h2> 157<h2><a class="toc-backref" href="#id9" name="compability-entries">Compability Entries</a></h2>
158<p>To maintain compability with existing portage versions a transition period after 158<p>To maintain compability with existing portage versions a transition period after
182the explanation why they are improvements and how the proposed format will 182the explanation why they are improvements and how the proposed format will
183accomplish them.</p> 183accomplish them.</p>
184<div class="section" id="removal-of-digest-files"> 184<div class="section" id="removal-of-digest-files">
185<h2><a class="toc-backref" href="#id12" name="removal-of-digest-files">Removal of digest files</a></h2> 185<h2><a class="toc-backref" href="#id12" name="removal-of-digest-files">Removal of digest files</a></h2>
186<p>Normal users that don't use a &quot;tuned&quot; filesystem for the portage tree are 186<p>Normal users that don't use a &quot;tuned&quot; filesystem for the portage tree are
187wasting several dozen to a few hundred megabytes of discspace with the current 187wasting several dozen to a few hundred megabytes of disk space with the current
188system, largely caused by the digest files. 188system, largely caused by the digest files.
189This is due to the filesystem overhead present in most filesystem that 189This is due to the filesystem overhead present in most filesystem that
190have a standard blocksize of four kilobytes while most digest files are under 190have a standard blocksize of four kilobytes while most digest files are under
191one kilobyte in size, so this results in approximately a waste of three kilobytes 191one kilobyte in size, so this results in approximately a waste of three kilobytes
192per digest file (likely even more). At the time of this writing the tree contains 192per digest file (likely even more). At the time of this writing the tree contains
193roughly 22.000 digest files, so the overall waste caused by digest files is 193roughly 22.000 digest files, so the overall waste caused by digest files is
194estimated at about 70-100 megabytes. 194estimated at about 70-100 megabytes.
195Furthermore it is assumed that this will also reduce the discspace wasted by 195Furthermore it is assumed that this will also reduce the disk space wasted by
196the Manifest files as they now contain more content, but this hasn't been 196the Manifest files as they now contain more content, but this hasn't been
197verified yet.</p> 197verified yet.</p>
198<p>By unifying the digest files with the Manifest these tiny files are eliminated 198<p>By unifying the digest files with the Manifest these tiny files are eliminated
199(in the long run), reducing the apparent tree size by about 20%, benefitting 199(in the long run), reducing the apparent tree size by about 20%, benefitting
200both users and the Gentoo infrastructure.</p> 200both users and the Gentoo infrastructure.</p>
208This is a theoretical improvement at this moment as only one hash function is in 208This is a theoretical improvement at this moment as only one hash function is in
209use, but expected to change soon (see <a class="footnote-reference" href="#multi-hash-thread" id="id3" name="id3">[2]</a>).</p> 209use, but expected to change soon (see <a class="footnote-reference" href="#multi-hash-thread" id="id3" name="id3">[2]</a>).</p>
210</div> 210</div>
211<div class="section" id="removal-of-checksum-collisions"> 211<div class="section" id="removal-of-checksum-collisions">
212<h2><a class="toc-backref" href="#id14" name="removal-of-checksum-collisions">Removal of checksum collisions</a></h2> 212<h2><a class="toc-backref" href="#id14" name="removal-of-checksum-collisions">Removal of checksum collisions</a></h2>
213<p>The current system theoretically allows for a <tt class="docutils literal"><span class="pre">SRCURI</span></tt> type file to be recorded 213<p>The current system theoretically allows for a <tt class="docutils literal"><span class="pre">DISTFILE</span></tt> type file to be recorded
214in multiple digest files with different sizes and/or checksums. In such a case 214in multiple digest files with different sizes and/or checksums. In such a case
215one version of a package would report a checksum violation while another one 215one version of a package would report a checksum violation while another one
216would not. This could create confusion and uncertainity among users. 216would not. This could create confusion and uncertainity among users.
217So far this case hasn't been observed, but it can't be ruled out with the 217So far this case hasn't been observed, but it can't be ruled out with the
218existing system. 218existing system.
219As the new format lists each file exactly once this would be no longer possible.</p> 219As the new format lists each file exactly once this would be no longer possible.</p>
220</div> 220</div>
221<div class="section" id="flexible-verification-system"> 221<div class="section" id="flexible-verification-system">
222<h2><a class="toc-backref" href="#id15" name="flexible-verification-system">Flexible verification system</a></h2> 222<h2><a class="toc-backref" href="#id15" name="flexible-verification-system">Flexible verification system</a></h2>
223<p>Right now portage verifies the checksum of every file listed in the Manifest 223<p>Right now portage verifies the checksum of every file listed in the Manifest
224before using any file of the package and all <tt class="docutils literal"><span class="pre">SRCURI</span></tt> files of an ebuild 224before using any file of the package and all <tt class="docutils literal"><span class="pre">DISTFILE</span></tt> files of an ebuild
225before using that ebuild. This is unnecessary in many cases:</p> 225before using that ebuild. This is unnecessary in many cases:</p>
226<ul class="simple"> 226<ul class="simple">
227<li>During the &quot;depend&quot; phase (when the ebuild metadata is generated) only 227<li>During the &quot;depend&quot; phase (when the ebuild metadata is generated) only
228files of type <tt class="docutils literal"><span class="pre">EBUILD</span></tt> are used, so verifying the other types isn't 228files of type <tt class="docutils literal"><span class="pre">EBUILD</span></tt> are used, so verifying the other types isn't
229necessary. Theoretically it is possible for an ebuild to include other 229necessary. Theoretically it is possible for an ebuild to include other
232It is also not a security concern as the ebuild is verified before parsing 232It is also not a security concern as the ebuild is verified before parsing
233it, so each manipulation would show up.</li> 233it, so each manipulation would show up.</li>
234<li>Generally files of type <tt class="docutils literal"><span class="pre">MISCFILE</span></tt> don't need to be verified as they are 234<li>Generally files of type <tt class="docutils literal"><span class="pre">MISCFILE</span></tt> don't need to be verified as they are
235only used in very specific situations, aren't executed (just parsed at most) 235only used in very specific situations, aren't executed (just parsed at most)
236and don't affect the package build process.</li> 236and don't affect the package build process.</li>
237<li>Files of type <tt class="docutils literal"><span class="pre">SRCURI</span></tt> only need to be verified directly after fetching and 237<li>Files of type <tt class="docutils literal"><span class="pre">DISTFILE</span></tt> only need to be verified directly after fetching and
238before unpacking them (which often will be one step), not every time their 238before unpacking them (which often will be one step), not every time their
239associated ebuild is used.</li> 239associated ebuild is used.</li>
240</ul> 240</ul>
241</div> 241</div>
242</div> 242</div>
318use for such a feature has been found.</li> 318use for such a feature has been found.</li>
319<li>convert size field into checksum: Another idea was to treat the size field 319<li>convert size field into checksum: Another idea was to treat the size field
320like any other checksum. But so far no real benefit (other than a slightly 320like any other checksum. But so far no real benefit (other than a slightly
321more modular implementation) for this has been seen while it has several 321more modular implementation) for this has been seen while it has several
322drawbacks: For once, unlike checksums, the size field is definitely required 322drawbacks: For once, unlike checksums, the size field is definitely required
323for all <tt class="docutils literal"><span class="pre">SRCURI</span></tt> files, also it would slightly increase the length of 323for all <tt class="docutils literal"><span class="pre">DISTFILE</span></tt> files, also it would slightly increase the length of
324each entry by adding a <tt class="docutils literal"><span class="pre">SIZE</span></tt> keyword.</li> 324each entry by adding a <tt class="docutils literal"><span class="pre">SIZE</span></tt> keyword.</li>
325<li>removal of the <tt class="docutils literal"><span class="pre">MISCFILE</span></tt> type: It has been suggested to completely drop 325<li>removal of the <tt class="docutils literal"><span class="pre">MISCFILE</span></tt> type: It has been suggested to completely drop
326entries of type <tt class="docutils literal"><span class="pre">MISCFILE</span></tt>. This would result in a minor space reduction 326entries of type <tt class="docutils literal"><span class="pre">MISCFILE</span></tt>. This would result in a minor space reduction
327(its rather unlikely to free any blocks) but completely remove the ability 327(its rather unlikely to free any blocks) but completely remove the ability
328to check these files for integrity. While they don't influence portage 328to check these files for integrity. While they don't influence portage
376 376
377</div> 377</div>
378<div class="footer"> 378<div class="footer">
379<hr class="footer" /> 379<hr class="footer" />
380<a class="reference" href="glep-0044.txt">View document source</a>. 380<a class="reference" href="glep-0044.txt">View document source</a>.
381Generated on: 2005-12-06 16:19 UTC. 381Generated on: 2006-01-23 10:22 UTC.
382Generated by <a class="reference" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source. 382Generated by <a class="reference" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.
383 383
384</div> 384</div>
385</body> 385</body>
386</html> 386</html>

Legend:
Removed from v.1.2  
changed lines
  Added in v.1.3

  ViewVC Help
Powered by ViewVC 1.1.20