| … | |
… | |
| 2 | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> |
2 | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> |
| 3 | <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
3 | <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
| 4 | |
4 | |
| 5 | <head> |
5 | <head> |
| 6 | <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> |
6 | <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> |
| 7 | <meta name="generator" content="Docutils 0.5: http://docutils.sourceforge.net/" /> |
7 | <meta name="generator" content="Docutils 0.6: http://docutils.sourceforge.net/" /> |
| 8 | <title>GLEP 44 -- Manifest2 format</title> |
8 | <title>GLEP 44 -- Manifest2 format</title> |
| 9 | <link rel="stylesheet" href="tools/glep.css" type="text/css" /></head> |
9 | <link rel="stylesheet" href="tools/glep.css" type="text/css" /></head> |
| 10 | <body bgcolor="white"> |
10 | <body bgcolor="white"> |
| 11 | <table class="navigation" cellpadding="0" cellspacing="0" |
11 | <table class="navigation" cellpadding="0" cellspacing="0" |
| 12 | width="100%" border="0"> |
12 | width="100%" border="0"> |
| … | |
… | |
| 25 | <tbody valign="top"> |
25 | <tbody valign="top"> |
| 26 | <tr class="field"><th class="field-name">GLEP:</th><td class="field-body">44</td> |
26 | <tr class="field"><th class="field-name">GLEP:</th><td class="field-body">44</td> |
| 27 | </tr> |
27 | </tr> |
| 28 | <tr class="field"><th class="field-name">Title:</th><td class="field-body">Manifest2 format</td> |
28 | <tr class="field"><th class="field-name">Title:</th><td class="field-body">Manifest2 format</td> |
| 29 | </tr> |
29 | </tr> |
| 30 | <tr class="field"><th class="field-name">Version:</th><td class="field-body">1.7</td> |
30 | <tr class="field"><th class="field-name">Version:</th><td class="field-body">1.8</td> |
| 31 | </tr> |
31 | </tr> |
| 32 | <tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference external" href="http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/proj/en/glep/glep-0044.txt?cvsroot=gentoo">2006/10/14 02:55:39</a></td> |
32 | <tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference external" href="http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/proj/en/glep/glep-0044.txt?cvsroot=gentoo">2009/01/11 19:40:56</a></td> |
| 33 | </tr> |
33 | </tr> |
| 34 | <tr class="field"><th class="field-name">Author:</th><td class="field-body">Marius Mauch <genone at gentoo.org>,</td> |
34 | <tr class="field"><th class="field-name">Author:</th><td class="field-body">Marius Mauch <genone at gentoo.org>,</td> |
| 35 | </tr> |
35 | </tr> |
| 36 | <tr class="field"><th class="field-name">Status:</th><td class="field-body">Final</td> |
36 | <tr class="field"><th class="field-name">Status:</th><td class="field-body">Final</td> |
| 37 | </tr> |
37 | </tr> |
| … | |
… | |
| 87 | <p>Please see <a class="footnote-reference" href="#reorg-thread" id="id1">[1]</a> for a general overview. |
87 | <p>Please see <a class="footnote-reference" href="#reorg-thread" id="id1">[1]</a> for a general overview. |
| 88 | The main long term goals of this proposal are to:</p> |
88 | The main long term goals of this proposal are to:</p> |
| 89 | <ul class="simple"> |
89 | <ul class="simple"> |
| 90 | <li>Remove the tiny digest files from the tree. They are a major annoyance as on a |
90 | <li>Remove the tiny digest files from the tree. They are a major annoyance as on a |
| 91 | typical configuration they waste a lot of disk space and the simple transmission |
91 | typical configuration they waste a lot of disk space and the simple transmission |
| 92 | of the names for all digest files during a <tt class="docutils literal"><span class="pre">emerge</span> <span class="pre">--sync</span></tt> needs a substantial |
92 | of the names for all digest files during a <tt class="docutils literal">emerge <span class="pre">--sync</span></tt> needs a substantial |
| 93 | amount of bandwidth.</li> |
93 | amount of bandwidth.</li> |
| 94 | <li>Reduce redundancy when multiple hash functions are used</li> |
94 | <li>Reduce redundancy when multiple hash functions are used</li> |
| 95 | <li>Remove potential for checksum collisions if a file is recorded in more than one |
95 | <li>Remove potential for checksum collisions if a file is recorded in more than one |
| 96 | digest file</li> |
96 | digest file</li> |
| 97 | <li>Difference between filetypes for a more flexible verification system</li> |
97 | <li>Difference between filetypes for a more flexible verification system</li> |
| … | |
… | |
| 101 | <h1><a class="toc-backref" href="#id11">Specification</a></h1> |
101 | <h1><a class="toc-backref" href="#id11">Specification</a></h1> |
| 102 | <p>The new Manifest format would change the existing format in the following ways:</p> |
102 | <p>The new Manifest format would change the existing format in the following ways:</p> |
| 103 | <ul> |
103 | <ul> |
| 104 | <li><p class="first">Addition of a filetype specifier, currently planned are</p> |
104 | <li><p class="first">Addition of a filetype specifier, currently planned are</p> |
| 105 | <ul class="simple"> |
105 | <ul class="simple"> |
| 106 | <li><tt class="docutils literal"><span class="pre">AUX</span></tt> for files directly used by ebuilds (e.g. patches or initscripts), |
106 | <li><tt class="docutils literal">AUX</tt> for files directly used by ebuilds (e.g. patches or initscripts), |
| 107 | located in the <tt class="docutils literal"><span class="pre">files/</span></tt> subdirectory</li> |
107 | located in the <tt class="docutils literal">files/</tt> subdirectory</li> |
| 108 | <li><tt class="docutils literal"><span class="pre">EBUILD</span></tt> for all ebuilds</li> |
108 | <li><tt class="docutils literal">EBUILD</tt> for all ebuilds</li> |
| 109 | <li><tt class="docutils literal"><span class="pre">MISC</span></tt> for files not directly used by ebuilds like <tt class="docutils literal"><span class="pre">ChangeLog</span></tt> or |
109 | <li><tt class="docutils literal">MISC</tt> for files not directly used by ebuilds like <tt class="docutils literal">ChangeLog</tt> or |
| 110 | <tt class="docutils literal"><span class="pre">metadata.xml</span></tt> files</li> |
110 | <tt class="docutils literal">metadata.xml</tt> files</li> |
| 111 | <li><tt class="docutils literal"><span class="pre">DIST</span></tt> for release tarballs recorded in the <tt class="docutils literal"><span class="pre">SRC_URI</span></tt> variable of an ebuild, |
111 | <li><tt class="docutils literal">DIST</tt> for release tarballs recorded in the <tt class="docutils literal">SRC_URI</tt> variable of an ebuild, |
| 112 | these were previously recorded in the digest files</li> |
112 | these were previously recorded in the digest files</li> |
| 113 | </ul> |
113 | </ul> |
| 114 | <p>Future portage improvements might extend this list (for example with types |
114 | <p>Future portage improvements might extend this list (for example with types |
| 115 | relevant for eclasses or profiles)</p> |
115 | relevant for eclasses or profiles)</p> |
| 116 | </li> |
116 | </li> |
| 117 | <li><p class="first">Only have one line per file listing all information instead of one line per |
117 | <li><p class="first">Only have one line per file listing all information instead of one line per |
| 118 | file and checksum type</p> |
118 | file and checksum type</p> |
| 119 | </li> |
119 | </li> |
| 120 | <li><p class="first">Remove the separated digest-* files in the <tt class="docutils literal"><span class="pre">files/</span></tt> subdirectory</p> |
120 | <li><p class="first">Remove the separated digest-* files in the <tt class="docutils literal">files/</tt> subdirectory</p> |
| 121 | </li> |
121 | </li> |
| 122 | </ul> |
122 | </ul> |
| 123 | <p>Each line in the new format has the following format:</p> |
123 | <p>Each line in the new format has the following format:</p> |
| 124 | <pre class="literal-block"> |
124 | <pre class="literal-block"> |
| 125 | <filetype> <filename> <filesize> <chksumtype1> <chksum1> ... <chksumtypen> <chksumn> |
125 | <filetype> <filename> <filesize> <chksumtype1> <chksum1> ... <chksumtypen> <chksumn> |
| … | |
… | |
| 193 | This is a theoretical improvement at this moment as only one hash function is in |
193 | This is a theoretical improvement at this moment as only one hash function is in |
| 194 | use, but expected to change soon (see <a class="footnote-reference" href="#multi-hash-thread" id="id4">[2]</a>).</p> |
194 | use, but expected to change soon (see <a class="footnote-reference" href="#multi-hash-thread" id="id4">[2]</a>).</p> |
| 195 | </div> |
195 | </div> |
| 196 | <div class="section" id="removal-of-checksum-collisions"> |
196 | <div class="section" id="removal-of-checksum-collisions"> |
| 197 | <h2><a class="toc-backref" href="#id18">Removal of checksum collisions</a></h2> |
197 | <h2><a class="toc-backref" href="#id18">Removal of checksum collisions</a></h2> |
| 198 | <p>The current system theoretically allows for a <tt class="docutils literal"><span class="pre">DIST</span></tt> type file to be recorded |
198 | <p>The current system theoretically allows for a <tt class="docutils literal">DIST</tt> type file to be recorded |
| 199 | in multiple digest files with different sizes and/or checksums. In such a case |
199 | in multiple digest files with different sizes and/or checksums. In such a case |
| 200 | one version of a package would report a checksum violation while another one |
200 | one version of a package would report a checksum violation while another one |
| 201 | would not. This could create confusion and uncertainity among users. |
201 | would not. This could create confusion and uncertainity among users. |
| 202 | So far this case hasn't been observed, but it can't be ruled out with the |
202 | So far this case hasn't been observed, but it can't be ruled out with the |
| 203 | existing system. |
203 | existing system. |
| 204 | As the new format lists each file exactly once this would be no longer possible.</p> |
204 | As the new format lists each file exactly once this would be no longer possible.</p> |
| 205 | </div> |
205 | </div> |
| 206 | <div class="section" id="flexible-verification-system"> |
206 | <div class="section" id="flexible-verification-system"> |
| 207 | <h2><a class="toc-backref" href="#id19">Flexible verification system</a></h2> |
207 | <h2><a class="toc-backref" href="#id19">Flexible verification system</a></h2> |
| 208 | <p>Right now portage verifies the checksum of every file listed in the Manifest |
208 | <p>Right now portage verifies the checksum of every file listed in the Manifest |
| 209 | before using any file of the package and all <tt class="docutils literal"><span class="pre">DIST</span></tt> files of an ebuild |
209 | before using any file of the package and all <tt class="docutils literal">DIST</tt> files of an ebuild |
| 210 | before using that ebuild. This is unnecessary in many cases:</p> |
210 | before using that ebuild. This is unnecessary in many cases:</p> |
| 211 | <ul class="simple"> |
211 | <ul class="simple"> |
| 212 | <li>During the "depend" phase (when the ebuild metadata is generated) only |
212 | <li>During the "depend" phase (when the ebuild metadata is generated) only |
| 213 | files of type <tt class="docutils literal"><span class="pre">EBUILD</span></tt> are used, so verifying the other types isn't |
213 | files of type <tt class="docutils literal">EBUILD</tt> are used, so verifying the other types isn't |
| 214 | necessary. Theoretically it is possible for an ebuild to include other |
214 | necessary. Theoretically it is possible for an ebuild to include other |
| 215 | files like those of type <tt class="docutils literal"><span class="pre">AUX</span></tt> at this phase, but that would be a |
215 | files like those of type <tt class="docutils literal">AUX</tt> at this phase, but that would be a |
| 216 | major QA violation and should never occur, so it can be ignored here. |
216 | major QA violation and should never occur, so it can be ignored here. |
| 217 | It is also not a security concern as the ebuild is verified before parsing |
217 | It is also not a security concern as the ebuild is verified before parsing |
| 218 | it, so each manipulation would show up.</li> |
218 | it, so each manipulation would show up.</li> |
| 219 | <li>Generally files of type <tt class="docutils literal"><span class="pre">MISC</span></tt> don't need to be verified as they are |
219 | <li>Generally files of type <tt class="docutils literal">MISC</tt> don't need to be verified as they are |
| 220 | only used in very specific situations, aren't executed (just parsed at most) |
220 | only used in very specific situations, aren't executed (just parsed at most) |
| 221 | and don't affect the package build process.</li> |
221 | and don't affect the package build process.</li> |
| 222 | <li>Files of type <tt class="docutils literal"><span class="pre">DIST</span></tt> only need to be verified directly after fetching and |
222 | <li>Files of type <tt class="docutils literal">DIST</tt> only need to be verified directly after fetching and |
| 223 | before unpacking them (which often will be one step), not every time their |
223 | before unpacking them (which often will be one step), not every time their |
| 224 | associated ebuild is used.</li> |
224 | associated ebuild is used.</li> |
| 225 | </ul> |
225 | </ul> |
| 226 | </div> |
226 | </div> |
| 227 | </div> |
227 | </div> |
| … | |
… | |
| 231 | like most changes affecting both portage and the tree. In this case the |
231 | like most changes affecting both portage and the tree. In this case the |
| 232 | implementation will be rolled out in several phases:</p> |
232 | implementation will be rolled out in several phases:</p> |
| 233 | <ol class="arabic simple"> |
233 | <ol class="arabic simple"> |
| 234 | <li>Add support for verification of Manifest2 entries in portage</li> |
234 | <li>Add support for verification of Manifest2 entries in portage</li> |
| 235 | <li>Enable generation of Manifest2 entries in addition to the current system</li> |
235 | <li>Enable generation of Manifest2 entries in addition to the current system</li> |
| 236 | <li>Ignore digests during <tt class="docutils literal"><span class="pre">emerge</span> <span class="pre">--sync</span></tt> to get the size-benefit clientside. |
236 | <li>Ignore digests during <tt class="docutils literal">emerge <span class="pre">--sync</span></tt> to get the size-benefit clientside. |
| 237 | This step may be ommitted if the following steps are expected to follow soon.</li> |
237 | This step may be ommitted if the following steps are expected to follow soon.</li> |
| 238 | <li>Disable generation of entries for the current system</li> |
238 | <li>Disable generation of entries for the current system</li> |
| 239 | <li>Remove all traces of the current system from the tree (serverside)</li> |
239 | <li>Remove all traces of the current system from the tree (serverside)</li> |
| 240 | </ol> |
240 | </ol> |
| 241 | <p>Each step has its own issues. While 1) and 2) can be implemented without any |
241 | <p>Each step has its own issues. While 1) and 2) can be implemented without any |
| … | |
… | |
| 303 | use for such a feature has been found.</li> |
303 | use for such a feature has been found.</li> |
| 304 | <li>convert size field into checksum: Another idea was to treat the size field |
304 | <li>convert size field into checksum: Another idea was to treat the size field |
| 305 | like any other checksum. But so far no real benefit (other than a slightly |
305 | like any other checksum. But so far no real benefit (other than a slightly |
| 306 | more modular implementation) for this has been seen while it has several |
306 | more modular implementation) for this has been seen while it has several |
| 307 | drawbacks: For once, unlike checksums, the size field is definitely required |
307 | drawbacks: For once, unlike checksums, the size field is definitely required |
| 308 | for all <tt class="docutils literal"><span class="pre">DIST</span></tt> files, also it would slightly increase the length of |
308 | for all <tt class="docutils literal">DIST</tt> files, also it would slightly increase the length of |
| 309 | each entry by adding a <tt class="docutils literal"><span class="pre">SIZE</span></tt> keyword.</li> |
309 | each entry by adding a <tt class="docutils literal">SIZE</tt> keyword.</li> |
| 310 | <li>removal of the <tt class="docutils literal"><span class="pre">MISC</span></tt> type: It has been suggested to completely drop |
310 | <li>removal of the <tt class="docutils literal">MISC</tt> type: It has been suggested to completely drop |
| 311 | entries of type <tt class="docutils literal"><span class="pre">MISC</span></tt>. This would result in a minor space reduction |
311 | entries of type <tt class="docutils literal">MISC</tt>. This would result in a minor space reduction |
| 312 | (its rather unlikely to free any blocks) but completely remove the ability |
312 | (its rather unlikely to free any blocks) but completely remove the ability |
| 313 | to check these files for integrity. While they don't influence portage |
313 | to check these files for integrity. While they don't influence portage |
| 314 | or packages directly they can contain viable information for users, so |
314 | or packages directly they can contain viable information for users, so |
| 315 | the author has the opinion that at least the option for integrity checks |
315 | the author has the opinion that at least the option for integrity checks |
| 316 | should be kept.</li> |
316 | should be kept.</li> |
| … | |
… | |
| 373 | |
373 | |
| 374 | </div> |
374 | </div> |
| 375 | <div class="footer"> |
375 | <div class="footer"> |
| 376 | <hr class="footer" /> |
376 | <hr class="footer" /> |
| 377 | <a class="reference external" href="glep-0044.txt">View document source</a>. |
377 | <a class="reference external" href="glep-0044.txt">View document source</a>. |
| 378 | Generated on: 2009-01-11 19:35 UTC. |
378 | Generated on: 2010-04-07 22:08 UTC. |
| 379 | Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source. |
379 | Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source. |
| 380 | |
380 | |
| 381 | </div> |
381 | </div> |
| 382 | </body> |
382 | </body> |
| 383 | </html> |
383 | </html> |
Parent Directory
| 
Revision Log
| 
Patch