/[gentoo]/xml/htdocs/proj/en/glep/glep-0044.html
Gentoo

Diff of /xml/htdocs/proj/en/glep/glep-0044.html

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

Revision 1.11 Revision 1.12
2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> 2<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
3<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> 3<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
4 4
5<head> 5<head>
6 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> 6 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
7 <meta name="generator" content="Docutils 0.5: http://docutils.sourceforge.net/" /> 7 <meta name="generator" content="Docutils 0.6: http://docutils.sourceforge.net/" />
8 <title>GLEP 44 -- Manifest2 format</title> 8 <title>GLEP 44 -- Manifest2 format</title>
9 <link rel="stylesheet" href="tools/glep.css" type="text/css" /></head> 9 <link rel="stylesheet" href="tools/glep.css" type="text/css" /></head>
10<body bgcolor="white"> 10<body bgcolor="white">
11<table class="navigation" cellpadding="0" cellspacing="0" 11<table class="navigation" cellpadding="0" cellspacing="0"
12 width="100%" border="0"> 12 width="100%" border="0">
25<tbody valign="top"> 25<tbody valign="top">
26<tr class="field"><th class="field-name">GLEP:</th><td class="field-body">44</td> 26<tr class="field"><th class="field-name">GLEP:</th><td class="field-body">44</td>
27</tr> 27</tr>
28<tr class="field"><th class="field-name">Title:</th><td class="field-body">Manifest2 format</td> 28<tr class="field"><th class="field-name">Title:</th><td class="field-body">Manifest2 format</td>
29</tr> 29</tr>
30<tr class="field"><th class="field-name">Version:</th><td class="field-body">1.7</td> 30<tr class="field"><th class="field-name">Version:</th><td class="field-body">1.8</td>
31</tr> 31</tr>
32<tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference external" href="http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/proj/en/glep/glep-0044.txt?cvsroot=gentoo">2006/10/14 02:55:39</a></td> 32<tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference external" href="http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/proj/en/glep/glep-0044.txt?cvsroot=gentoo">2009/01/11 19:40:56</a></td>
33</tr> 33</tr>
34<tr class="field"><th class="field-name">Author:</th><td class="field-body">Marius Mauch &lt;genone&#32;&#97;t&#32;gentoo.org&gt;,</td> 34<tr class="field"><th class="field-name">Author:</th><td class="field-body">Marius Mauch &lt;genone&#32;&#97;t&#32;gentoo.org&gt;,</td>
35</tr> 35</tr>
36<tr class="field"><th class="field-name">Status:</th><td class="field-body">Final</td> 36<tr class="field"><th class="field-name">Status:</th><td class="field-body">Final</td>
37</tr> 37</tr>
87<p>Please see <a class="footnote-reference" href="#reorg-thread" id="id1">[1]</a> for a general overview. 87<p>Please see <a class="footnote-reference" href="#reorg-thread" id="id1">[1]</a> for a general overview.
88The main long term goals of this proposal are to:</p> 88The main long term goals of this proposal are to:</p>
89<ul class="simple"> 89<ul class="simple">
90<li>Remove the tiny digest files from the tree. They are a major annoyance as on a 90<li>Remove the tiny digest files from the tree. They are a major annoyance as on a
91typical configuration they waste a lot of disk space and the simple transmission 91typical configuration they waste a lot of disk space and the simple transmission
92of the names for all digest files during a <tt class="docutils literal"><span class="pre">emerge</span> <span class="pre">--sync</span></tt> needs a substantial 92of the names for all digest files during a <tt class="docutils literal">emerge <span class="pre">--sync</span></tt> needs a substantial
93amount of bandwidth.</li> 93amount of bandwidth.</li>
94<li>Reduce redundancy when multiple hash functions are used</li> 94<li>Reduce redundancy when multiple hash functions are used</li>
95<li>Remove potential for checksum collisions if a file is recorded in more than one 95<li>Remove potential for checksum collisions if a file is recorded in more than one
96digest file</li> 96digest file</li>
97<li>Difference between filetypes for a more flexible verification system</li> 97<li>Difference between filetypes for a more flexible verification system</li>
101<h1><a class="toc-backref" href="#id11">Specification</a></h1> 101<h1><a class="toc-backref" href="#id11">Specification</a></h1>
102<p>The new Manifest format would change the existing format in the following ways:</p> 102<p>The new Manifest format would change the existing format in the following ways:</p>
103<ul> 103<ul>
104<li><p class="first">Addition of a filetype specifier, currently planned are</p> 104<li><p class="first">Addition of a filetype specifier, currently planned are</p>
105<ul class="simple"> 105<ul class="simple">
106<li><tt class="docutils literal"><span class="pre">AUX</span></tt> for files directly used by ebuilds (e.g. patches or initscripts), 106<li><tt class="docutils literal">AUX</tt> for files directly used by ebuilds (e.g. patches or initscripts),
107located in the <tt class="docutils literal"><span class="pre">files/</span></tt> subdirectory</li> 107located in the <tt class="docutils literal">files/</tt> subdirectory</li>
108<li><tt class="docutils literal"><span class="pre">EBUILD</span></tt> for all ebuilds</li> 108<li><tt class="docutils literal">EBUILD</tt> for all ebuilds</li>
109<li><tt class="docutils literal"><span class="pre">MISC</span></tt> for files not directly used by ebuilds like <tt class="docutils literal"><span class="pre">ChangeLog</span></tt> or 109<li><tt class="docutils literal">MISC</tt> for files not directly used by ebuilds like <tt class="docutils literal">ChangeLog</tt> or
110<tt class="docutils literal"><span class="pre">metadata.xml</span></tt> files</li> 110<tt class="docutils literal">metadata.xml</tt> files</li>
111<li><tt class="docutils literal"><span class="pre">DIST</span></tt> for release tarballs recorded in the <tt class="docutils literal"><span class="pre">SRC_URI</span></tt> variable of an ebuild, 111<li><tt class="docutils literal">DIST</tt> for release tarballs recorded in the <tt class="docutils literal">SRC_URI</tt> variable of an ebuild,
112these were previously recorded in the digest files</li> 112these were previously recorded in the digest files</li>
113</ul> 113</ul>
114<p>Future portage improvements might extend this list (for example with types 114<p>Future portage improvements might extend this list (for example with types
115relevant for eclasses or profiles)</p> 115relevant for eclasses or profiles)</p>
116</li> 116</li>
117<li><p class="first">Only have one line per file listing all information instead of one line per 117<li><p class="first">Only have one line per file listing all information instead of one line per
118file and checksum type</p> 118file and checksum type</p>
119</li> 119</li>
120<li><p class="first">Remove the separated digest-* files in the <tt class="docutils literal"><span class="pre">files/</span></tt> subdirectory</p> 120<li><p class="first">Remove the separated digest-* files in the <tt class="docutils literal">files/</tt> subdirectory</p>
121</li> 121</li>
122</ul> 122</ul>
123<p>Each line in the new format has the following format:</p> 123<p>Each line in the new format has the following format:</p>
124<pre class="literal-block"> 124<pre class="literal-block">
125&lt;filetype&gt; &lt;filename&gt; &lt;filesize&gt; &lt;chksumtype1&gt; &lt;chksum1&gt; ... &lt;chksumtypen&gt; &lt;chksumn&gt; 125&lt;filetype&gt; &lt;filename&gt; &lt;filesize&gt; &lt;chksumtype1&gt; &lt;chksum1&gt; ... &lt;chksumtypen&gt; &lt;chksumn&gt;
193This is a theoretical improvement at this moment as only one hash function is in 193This is a theoretical improvement at this moment as only one hash function is in
194use, but expected to change soon (see <a class="footnote-reference" href="#multi-hash-thread" id="id4">[2]</a>).</p> 194use, but expected to change soon (see <a class="footnote-reference" href="#multi-hash-thread" id="id4">[2]</a>).</p>
195</div> 195</div>
196<div class="section" id="removal-of-checksum-collisions"> 196<div class="section" id="removal-of-checksum-collisions">
197<h2><a class="toc-backref" href="#id18">Removal of checksum collisions</a></h2> 197<h2><a class="toc-backref" href="#id18">Removal of checksum collisions</a></h2>
198<p>The current system theoretically allows for a <tt class="docutils literal"><span class="pre">DIST</span></tt> type file to be recorded 198<p>The current system theoretically allows for a <tt class="docutils literal">DIST</tt> type file to be recorded
199in multiple digest files with different sizes and/or checksums. In such a case 199in multiple digest files with different sizes and/or checksums. In such a case
200one version of a package would report a checksum violation while another one 200one version of a package would report a checksum violation while another one
201would not. This could create confusion and uncertainity among users. 201would not. This could create confusion and uncertainity among users.
202So far this case hasn't been observed, but it can't be ruled out with the 202So far this case hasn't been observed, but it can't be ruled out with the
203existing system. 203existing system.
204As the new format lists each file exactly once this would be no longer possible.</p> 204As the new format lists each file exactly once this would be no longer possible.</p>
205</div> 205</div>
206<div class="section" id="flexible-verification-system"> 206<div class="section" id="flexible-verification-system">
207<h2><a class="toc-backref" href="#id19">Flexible verification system</a></h2> 207<h2><a class="toc-backref" href="#id19">Flexible verification system</a></h2>
208<p>Right now portage verifies the checksum of every file listed in the Manifest 208<p>Right now portage verifies the checksum of every file listed in the Manifest
209before using any file of the package and all <tt class="docutils literal"><span class="pre">DIST</span></tt> files of an ebuild 209before using any file of the package and all <tt class="docutils literal">DIST</tt> files of an ebuild
210before using that ebuild. This is unnecessary in many cases:</p> 210before using that ebuild. This is unnecessary in many cases:</p>
211<ul class="simple"> 211<ul class="simple">
212<li>During the &quot;depend&quot; phase (when the ebuild metadata is generated) only 212<li>During the &quot;depend&quot; phase (when the ebuild metadata is generated) only
213files of type <tt class="docutils literal"><span class="pre">EBUILD</span></tt> are used, so verifying the other types isn't 213files of type <tt class="docutils literal">EBUILD</tt> are used, so verifying the other types isn't
214necessary. Theoretically it is possible for an ebuild to include other 214necessary. Theoretically it is possible for an ebuild to include other
215files like those of type <tt class="docutils literal"><span class="pre">AUX</span></tt> at this phase, but that would be a 215files like those of type <tt class="docutils literal">AUX</tt> at this phase, but that would be a
216major QA violation and should never occur, so it can be ignored here. 216major QA violation and should never occur, so it can be ignored here.
217It is also not a security concern as the ebuild is verified before parsing 217It is also not a security concern as the ebuild is verified before parsing
218it, so each manipulation would show up.</li> 218it, so each manipulation would show up.</li>
219<li>Generally files of type <tt class="docutils literal"><span class="pre">MISC</span></tt> don't need to be verified as they are 219<li>Generally files of type <tt class="docutils literal">MISC</tt> don't need to be verified as they are
220only used in very specific situations, aren't executed (just parsed at most) 220only used in very specific situations, aren't executed (just parsed at most)
221and don't affect the package build process.</li> 221and don't affect the package build process.</li>
222<li>Files of type <tt class="docutils literal"><span class="pre">DIST</span></tt> only need to be verified directly after fetching and 222<li>Files of type <tt class="docutils literal">DIST</tt> only need to be verified directly after fetching and
223before unpacking them (which often will be one step), not every time their 223before unpacking them (which often will be one step), not every time their
224associated ebuild is used.</li> 224associated ebuild is used.</li>
225</ul> 225</ul>
226</div> 226</div>
227</div> 227</div>
231like most changes affecting both portage and the tree. In this case the 231like most changes affecting both portage and the tree. In this case the
232implementation will be rolled out in several phases:</p> 232implementation will be rolled out in several phases:</p>
233<ol class="arabic simple"> 233<ol class="arabic simple">
234<li>Add support for verification of Manifest2 entries in portage</li> 234<li>Add support for verification of Manifest2 entries in portage</li>
235<li>Enable generation of Manifest2 entries in addition to the current system</li> 235<li>Enable generation of Manifest2 entries in addition to the current system</li>
236<li>Ignore digests during <tt class="docutils literal"><span class="pre">emerge</span> <span class="pre">--sync</span></tt> to get the size-benefit clientside. 236<li>Ignore digests during <tt class="docutils literal">emerge <span class="pre">--sync</span></tt> to get the size-benefit clientside.
237This step may be ommitted if the following steps are expected to follow soon.</li> 237This step may be ommitted if the following steps are expected to follow soon.</li>
238<li>Disable generation of entries for the current system</li> 238<li>Disable generation of entries for the current system</li>
239<li>Remove all traces of the current system from the tree (serverside)</li> 239<li>Remove all traces of the current system from the tree (serverside)</li>
240</ol> 240</ol>
241<p>Each step has its own issues. While 1) and 2) can be implemented without any 241<p>Each step has its own issues. While 1) and 2) can be implemented without any
303use for such a feature has been found.</li> 303use for such a feature has been found.</li>
304<li>convert size field into checksum: Another idea was to treat the size field 304<li>convert size field into checksum: Another idea was to treat the size field
305like any other checksum. But so far no real benefit (other than a slightly 305like any other checksum. But so far no real benefit (other than a slightly
306more modular implementation) for this has been seen while it has several 306more modular implementation) for this has been seen while it has several
307drawbacks: For once, unlike checksums, the size field is definitely required 307drawbacks: For once, unlike checksums, the size field is definitely required
308for all <tt class="docutils literal"><span class="pre">DIST</span></tt> files, also it would slightly increase the length of 308for all <tt class="docutils literal">DIST</tt> files, also it would slightly increase the length of
309each entry by adding a <tt class="docutils literal"><span class="pre">SIZE</span></tt> keyword.</li> 309each entry by adding a <tt class="docutils literal">SIZE</tt> keyword.</li>
310<li>removal of the <tt class="docutils literal"><span class="pre">MISC</span></tt> type: It has been suggested to completely drop 310<li>removal of the <tt class="docutils literal">MISC</tt> type: It has been suggested to completely drop
311entries of type <tt class="docutils literal"><span class="pre">MISC</span></tt>. This would result in a minor space reduction 311entries of type <tt class="docutils literal">MISC</tt>. This would result in a minor space reduction
312(its rather unlikely to free any blocks) but completely remove the ability 312(its rather unlikely to free any blocks) but completely remove the ability
313to check these files for integrity. While they don't influence portage 313to check these files for integrity. While they don't influence portage
314or packages directly they can contain viable information for users, so 314or packages directly they can contain viable information for users, so
315the author has the opinion that at least the option for integrity checks 315the author has the opinion that at least the option for integrity checks
316should be kept.</li> 316should be kept.</li>
373 373
374</div> 374</div>
375<div class="footer"> 375<div class="footer">
376<hr class="footer" /> 376<hr class="footer" />
377<a class="reference external" href="glep-0044.txt">View document source</a>. 377<a class="reference external" href="glep-0044.txt">View document source</a>.
378Generated on: 2009-01-11 19:35 UTC. 378Generated on: 2010-04-07 22:08 UTC.
379Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source. 379Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.
380 380
381</div> 381</div>
382</body> 382</body>
383</html> 383</html>

Legend:
Removed from v.1.11  
changed lines
  Added in v.1.12

  ViewVC Help
Powered by ViewVC 1.1.20