| … | |
… | |
| 30 | <tbody valign="top"> |
30 | <tbody valign="top"> |
| 31 | <tr class="field"><th class="field-name">GLEP:</th><td class="field-body">44</td> |
31 | <tr class="field"><th class="field-name">GLEP:</th><td class="field-body">44</td> |
| 32 | </tr> |
32 | </tr> |
| 33 | <tr class="field"><th class="field-name">Title:</th><td class="field-body">Manifest2 format</td> |
33 | <tr class="field"><th class="field-name">Title:</th><td class="field-body">Manifest2 format</td> |
| 34 | </tr> |
34 | </tr> |
| 35 | <tr class="field"><th class="field-name">Version:</th><td class="field-body">1.1</td> |
35 | <tr class="field"><th class="field-name">Version:</th><td class="field-body">1.2</td> |
| 36 | </tr> |
36 | </tr> |
| 37 | <tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference" href="http://www.gentoo.org/cgi-bin/viewcvs/xml/htdocs/proj/en/glep/glep-0044.txt?cvsroot=gentoo">2005/12/06 03:34:21</a></td> |
37 | <tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference" href="http://www.gentoo.org/cgi-bin/viewcvs/xml/htdocs/proj/en/glep/glep-0044.txt?cvsroot=gentoo">2005/12/06 16:19:37</a></td> |
| 38 | </tr> |
38 | </tr> |
| 39 | <tr class="field"><th class="field-name">Author:</th><td class="field-body">Marius Mauch <genone at gentoo.org>,</td> |
39 | <tr class="field"><th class="field-name">Author:</th><td class="field-body">Marius Mauch <genone at gentoo.org>,</td> |
| 40 | </tr> |
40 | </tr> |
| 41 | <tr class="field"><th class="field-name">Status:</th><td class="field-body">Draft</td> |
41 | <tr class="field"><th class="field-name">Status:</th><td class="field-body">Draft</td> |
| 42 | </tr> |
42 | </tr> |
| … | |
… | |
| 44 | </tr> |
44 | </tr> |
| 45 | <tr class="field"><th class="field-name">Content-Type:</th><td class="field-body"><a class="reference" href="http://www.python.org/peps/glep-0012.html">text/x-rst</a></td> |
45 | <tr class="field"><th class="field-name">Content-Type:</th><td class="field-body"><a class="reference" href="http://www.python.org/peps/glep-0012.html">text/x-rst</a></td> |
| 46 | </tr> |
46 | </tr> |
| 47 | <tr class="field"><th class="field-name">Created:</th><td class="field-body">04-Dec-2005</td> |
47 | <tr class="field"><th class="field-name">Created:</th><td class="field-body">04-Dec-2005</td> |
| 48 | </tr> |
48 | </tr> |
| 49 | <tr class="field"><th class="field-name">Post-History:</th><td class="field-body">06-Dec-2005</td> |
49 | <tr class="field"><th class="field-name">Post-History:</th><td class="field-body">06-Dec-2005, 23-Jan-2006</td> |
| 50 | </tr> |
50 | </tr> |
| 51 | </tbody> |
51 | </tbody> |
| 52 | </table> |
52 | </table> |
| 53 | <hr /> |
53 | <hr /> |
| 54 | <div class="contents topic" id="contents"> |
54 | <div class="contents topic" id="contents"> |
| … | |
… | |
| 90 | <h1><a class="toc-backref" href="#id7" name="motivation">Motivation</a></h1> |
90 | <h1><a class="toc-backref" href="#id7" name="motivation">Motivation</a></h1> |
| 91 | <p>Please see <a class="footnote-reference" href="#reorg-thread" id="id1" name="id1">[1]</a> for a general overview. |
91 | <p>Please see <a class="footnote-reference" href="#reorg-thread" id="id1" name="id1">[1]</a> for a general overview. |
| 92 | The main long term goals of this proposal are to:</p> |
92 | The main long term goals of this proposal are to:</p> |
| 93 | <ul class="simple"> |
93 | <ul class="simple"> |
| 94 | <li>Remove the tiny digest files from the tree. They are a major annoyance as on a |
94 | <li>Remove the tiny digest files from the tree. They are a major annoyance as on a |
| 95 | typical configuration they waste a lot of discspace and the simple transmission |
95 | typical configuration they waste a lot of disk space and the simple transmission |
| 96 | of the names for all digest files during a <tt class="docutils literal"><span class="pre">emerge</span> <span class="pre">--sync</span></tt> needs a substantial |
96 | of the names for all digest files during a <tt class="docutils literal"><span class="pre">emerge</span> <span class="pre">--sync</span></tt> needs a substantial |
| 97 | amount of bandwidth.</li> |
97 | amount of bandwidth.</li> |
| 98 | <li>Reduce redundancy when multiple hash functions are used</li> |
98 | <li>Reduce redundancy when multiple hash functions are used</li> |
| 99 | <li>Remove potential for checksum collisions if a file is recorded in more than one |
99 | <li>Remove potential for checksum collisions if a file is recorded in more than one |
| 100 | digest file</li> |
100 | digest file</li> |
| … | |
… | |
| 110 | <li><tt class="docutils literal"><span class="pre">AUXFILE</span></tt> for files directly used by ebuilds (e.g. patches or initscripts), |
110 | <li><tt class="docutils literal"><span class="pre">AUXFILE</span></tt> for files directly used by ebuilds (e.g. patches or initscripts), |
| 111 | located in the <tt class="docutils literal"><span class="pre">files/</span></tt> subdirectory</li> |
111 | located in the <tt class="docutils literal"><span class="pre">files/</span></tt> subdirectory</li> |
| 112 | <li><tt class="docutils literal"><span class="pre">EBUILD</span></tt> for all ebuilds</li> |
112 | <li><tt class="docutils literal"><span class="pre">EBUILD</span></tt> for all ebuilds</li> |
| 113 | <li><tt class="docutils literal"><span class="pre">MISCFILE</span></tt> for files not directly used by ebuilds like <tt class="docutils literal"><span class="pre">ChangeLog</span></tt> or |
113 | <li><tt class="docutils literal"><span class="pre">MISCFILE</span></tt> for files not directly used by ebuilds like <tt class="docutils literal"><span class="pre">ChangeLog</span></tt> or |
| 114 | <tt class="docutils literal"><span class="pre">metadata.xml</span></tt> files</li> |
114 | <tt class="docutils literal"><span class="pre">metadata.xml</span></tt> files</li> |
| 115 | <li><tt class="docutils literal"><span class="pre">SRCURI</span></tt> for release tarballs recorded in the <tt class="docutils literal"><span class="pre">SRC_URI</span></tt> variable of an ebuild, |
115 | <li><tt class="docutils literal"><span class="pre">DISTFILE</span></tt> for release tarballs recorded in the <tt class="docutils literal"><span class="pre">SRC_URI</span></tt> variable of an ebuild, |
| 116 | these were previously recorded in the digest files</li> |
116 | these were previously recorded in the digest files</li> |
| 117 | </ul> |
117 | </ul> |
| 118 | <p>Future portage improvements might extend this list (for example with types |
118 | <p>Future portage improvements might extend this list (for example with types |
| 119 | relevant for eclasses or profiles)</p> |
119 | relevant for eclasses or profiles)</p> |
| 120 | </li> |
120 | </li> |
| … | |
… | |
| 144 | SHA1 358278a43da244e1f4803ec4b04d6fa45c41ab4d MD5 15b5c9348ba0b0a416892588256b4cbc |
144 | SHA1 358278a43da244e1f4803ec4b04d6fa45c41ab4d MD5 15b5c9348ba0b0a416892588256b4cbc |
| 145 | MISCFILE ChangeLog 25770 RMD160 0e69dd7425add1560d630dd3367342418e9be776 |
145 | MISCFILE ChangeLog 25770 RMD160 0e69dd7425add1560d630dd3367342418e9be776 |
| 146 | SHA1 1210160f7baf0319de3b1b58dc80d7680d316d28 MD5 732cdc3b41403a115970d497a9ec257e |
146 | SHA1 1210160f7baf0319de3b1b58dc80d7680d316d28 MD5 732cdc3b41403a115970d497a9ec257e |
| 147 | MISCFILE metadata.xml 269 RMD160 39d775de55f9963f8946feaf088aa0324770bacb |
147 | MISCFILE metadata.xml 269 RMD160 39d775de55f9963f8946feaf088aa0324770bacb |
| 148 | SHA1 4fd7b285049d0e587f89e86becf06c0fd77bae6d MD5 82e806ed62f0596fb7bef493d225712f |
148 | SHA1 4fd7b285049d0e587f89e86becf06c0fd77bae6d MD5 82e806ed62f0596fb7bef493d225712f |
| 149 | SRCURI sylpheed-claws-1.0.5.tar.bz2 3268626 RMD160 f2708b5d69bc9a5025812511fde04eca7782e367 |
149 | DISTFILE sylpheed-claws-1.0.5.tar.bz2 3268626 RMD160 f2708b5d69bc9a5025812511fde04eca7782e367 |
| 150 | SHA1 d351d7043eef7a875df18a8c4b9464be49e2164b MD5 ef4a1a7beb407dc7c31b4799bc48f12e |
150 | SHA1 d351d7043eef7a875df18a8c4b9464be49e2164b MD5 ef4a1a7beb407dc7c31b4799bc48f12e |
| 151 | SRCURI sylpheed-claws-1.9.100.tar.bz2 3480063 RMD160 72fbcbcc05d966f34897efcc1c96377420dc5544 |
151 | DISTFILE sylpheed-claws-1.9.100.tar.bz2 3480063 RMD160 72fbcbcc05d966f34897efcc1c96377420dc5544 |
| 152 | SHA1 47465662b5470af5711493ce4eaad764c5bf02ca MD5 863c314557f90f17c2f6d6a0ab57e6c2 |
152 | SHA1 47465662b5470af5711493ce4eaad764c5bf02ca MD5 863c314557f90f17c2f6d6a0ab57e6c2 |
| 153 | SRCURI sylpheed-claws-1.9.15.tar.bz2 3481018 RMD160 b01d1af2df55806a8a8275102b10e389e0d98e94 |
153 | DISTFILE sylpheed-claws-1.9.15.tar.bz2 3481018 RMD160 b01d1af2df55806a8a8275102b10e389e0d98e94 |
| 154 | SHA1 a17fc64b8dcc5b56432e5beb5c826913cb3ad79e MD5 0d187526e0eca23b87ffa4981f7e1765 |
154 | SHA1 a17fc64b8dcc5b56432e5beb5c826913cb3ad79e MD5 0d187526e0eca23b87ffa4981f7e1765 |
| 155 | </pre> |
155 | </pre> |
| 156 | <div class="section" id="compability-entries"> |
156 | <div class="section" id="compability-entries"> |
| 157 | <h2><a class="toc-backref" href="#id9" name="compability-entries">Compability Entries</a></h2> |
157 | <h2><a class="toc-backref" href="#id9" name="compability-entries">Compability Entries</a></h2> |
| 158 | <p>To maintain compability with existing portage versions a transition period after |
158 | <p>To maintain compability with existing portage versions a transition period after |
| … | |
… | |
| 182 | the explanation why they are improvements and how the proposed format will |
182 | the explanation why they are improvements and how the proposed format will |
| 183 | accomplish them.</p> |
183 | accomplish them.</p> |
| 184 | <div class="section" id="removal-of-digest-files"> |
184 | <div class="section" id="removal-of-digest-files"> |
| 185 | <h2><a class="toc-backref" href="#id12" name="removal-of-digest-files">Removal of digest files</a></h2> |
185 | <h2><a class="toc-backref" href="#id12" name="removal-of-digest-files">Removal of digest files</a></h2> |
| 186 | <p>Normal users that don't use a "tuned" filesystem for the portage tree are |
186 | <p>Normal users that don't use a "tuned" filesystem for the portage tree are |
| 187 | wasting several dozen to a few hundred megabytes of discspace with the current |
187 | wasting several dozen to a few hundred megabytes of disk space with the current |
| 188 | system, largely caused by the digest files. |
188 | system, largely caused by the digest files. |
| 189 | This is due to the filesystem overhead present in most filesystem that |
189 | This is due to the filesystem overhead present in most filesystem that |
| 190 | have a standard blocksize of four kilobytes while most digest files are under |
190 | have a standard blocksize of four kilobytes while most digest files are under |
| 191 | one kilobyte in size, so this results in approximately a waste of three kilobytes |
191 | one kilobyte in size, so this results in approximately a waste of three kilobytes |
| 192 | per digest file (likely even more). At the time of this writing the tree contains |
192 | per digest file (likely even more). At the time of this writing the tree contains |
| 193 | roughly 22.000 digest files, so the overall waste caused by digest files is |
193 | roughly 22.000 digest files, so the overall waste caused by digest files is |
| 194 | estimated at about 70-100 megabytes. |
194 | estimated at about 70-100 megabytes. |
| 195 | Furthermore it is assumed that this will also reduce the discspace wasted by |
195 | Furthermore it is assumed that this will also reduce the disk space wasted by |
| 196 | the Manifest files as they now contain more content, but this hasn't been |
196 | the Manifest files as they now contain more content, but this hasn't been |
| 197 | verified yet.</p> |
197 | verified yet.</p> |
| 198 | <p>By unifying the digest files with the Manifest these tiny files are eliminated |
198 | <p>By unifying the digest files with the Manifest these tiny files are eliminated |
| 199 | (in the long run), reducing the apparent tree size by about 20%, benefitting |
199 | (in the long run), reducing the apparent tree size by about 20%, benefitting |
| 200 | both users and the Gentoo infrastructure.</p> |
200 | both users and the Gentoo infrastructure.</p> |
| … | |
… | |
| 208 | This is a theoretical improvement at this moment as only one hash function is in |
208 | This is a theoretical improvement at this moment as only one hash function is in |
| 209 | use, but expected to change soon (see <a class="footnote-reference" href="#multi-hash-thread" id="id3" name="id3">[2]</a>).</p> |
209 | use, but expected to change soon (see <a class="footnote-reference" href="#multi-hash-thread" id="id3" name="id3">[2]</a>).</p> |
| 210 | </div> |
210 | </div> |
| 211 | <div class="section" id="removal-of-checksum-collisions"> |
211 | <div class="section" id="removal-of-checksum-collisions"> |
| 212 | <h2><a class="toc-backref" href="#id14" name="removal-of-checksum-collisions">Removal of checksum collisions</a></h2> |
212 | <h2><a class="toc-backref" href="#id14" name="removal-of-checksum-collisions">Removal of checksum collisions</a></h2> |
| 213 | <p>The current system theoretically allows for a <tt class="docutils literal"><span class="pre">SRCURI</span></tt> type file to be recorded |
213 | <p>The current system theoretically allows for a <tt class="docutils literal"><span class="pre">DISTFILE</span></tt> type file to be recorded |
| 214 | in multiple digest files with different sizes and/or checksums. In such a case |
214 | in multiple digest files with different sizes and/or checksums. In such a case |
| 215 | one version of a package would report a checksum violation while another one |
215 | one version of a package would report a checksum violation while another one |
| 216 | would not. This could create confusion and uncertainity among users. |
216 | would not. This could create confusion and uncertainity among users. |
| 217 | So far this case hasn't been observed, but it can't be ruled out with the |
217 | So far this case hasn't been observed, but it can't be ruled out with the |
| 218 | existing system. |
218 | existing system. |
| 219 | As the new format lists each file exactly once this would be no longer possible.</p> |
219 | As the new format lists each file exactly once this would be no longer possible.</p> |
| 220 | </div> |
220 | </div> |
| 221 | <div class="section" id="flexible-verification-system"> |
221 | <div class="section" id="flexible-verification-system"> |
| 222 | <h2><a class="toc-backref" href="#id15" name="flexible-verification-system">Flexible verification system</a></h2> |
222 | <h2><a class="toc-backref" href="#id15" name="flexible-verification-system">Flexible verification system</a></h2> |
| 223 | <p>Right now portage verifies the checksum of every file listed in the Manifest |
223 | <p>Right now portage verifies the checksum of every file listed in the Manifest |
| 224 | before using any file of the package and all <tt class="docutils literal"><span class="pre">SRCURI</span></tt> files of an ebuild |
224 | before using any file of the package and all <tt class="docutils literal"><span class="pre">DISTFILE</span></tt> files of an ebuild |
| 225 | before using that ebuild. This is unnecessary in many cases:</p> |
225 | before using that ebuild. This is unnecessary in many cases:</p> |
| 226 | <ul class="simple"> |
226 | <ul class="simple"> |
| 227 | <li>During the "depend" phase (when the ebuild metadata is generated) only |
227 | <li>During the "depend" phase (when the ebuild metadata is generated) only |
| 228 | files of type <tt class="docutils literal"><span class="pre">EBUILD</span></tt> are used, so verifying the other types isn't |
228 | files of type <tt class="docutils literal"><span class="pre">EBUILD</span></tt> are used, so verifying the other types isn't |
| 229 | necessary. Theoretically it is possible for an ebuild to include other |
229 | necessary. Theoretically it is possible for an ebuild to include other |
| … | |
… | |
| 232 | It is also not a security concern as the ebuild is verified before parsing |
232 | It is also not a security concern as the ebuild is verified before parsing |
| 233 | it, so each manipulation would show up.</li> |
233 | it, so each manipulation would show up.</li> |
| 234 | <li>Generally files of type <tt class="docutils literal"><span class="pre">MISCFILE</span></tt> don't need to be verified as they are |
234 | <li>Generally files of type <tt class="docutils literal"><span class="pre">MISCFILE</span></tt> don't need to be verified as they are |
| 235 | only used in very specific situations, aren't executed (just parsed at most) |
235 | only used in very specific situations, aren't executed (just parsed at most) |
| 236 | and don't affect the package build process.</li> |
236 | and don't affect the package build process.</li> |
| 237 | <li>Files of type <tt class="docutils literal"><span class="pre">SRCURI</span></tt> only need to be verified directly after fetching and |
237 | <li>Files of type <tt class="docutils literal"><span class="pre">DISTFILE</span></tt> only need to be verified directly after fetching and |
| 238 | before unpacking them (which often will be one step), not every time their |
238 | before unpacking them (which often will be one step), not every time their |
| 239 | associated ebuild is used.</li> |
239 | associated ebuild is used.</li> |
| 240 | </ul> |
240 | </ul> |
| 241 | </div> |
241 | </div> |
| 242 | </div> |
242 | </div> |
| … | |
… | |
| 318 | use for such a feature has been found.</li> |
318 | use for such a feature has been found.</li> |
| 319 | <li>convert size field into checksum: Another idea was to treat the size field |
319 | <li>convert size field into checksum: Another idea was to treat the size field |
| 320 | like any other checksum. But so far no real benefit (other than a slightly |
320 | like any other checksum. But so far no real benefit (other than a slightly |
| 321 | more modular implementation) for this has been seen while it has several |
321 | more modular implementation) for this has been seen while it has several |
| 322 | drawbacks: For once, unlike checksums, the size field is definitely required |
322 | drawbacks: For once, unlike checksums, the size field is definitely required |
| 323 | for all <tt class="docutils literal"><span class="pre">SRCURI</span></tt> files, also it would slightly increase the length of |
323 | for all <tt class="docutils literal"><span class="pre">DISTFILE</span></tt> files, also it would slightly increase the length of |
| 324 | each entry by adding a <tt class="docutils literal"><span class="pre">SIZE</span></tt> keyword.</li> |
324 | each entry by adding a <tt class="docutils literal"><span class="pre">SIZE</span></tt> keyword.</li> |
| 325 | <li>removal of the <tt class="docutils literal"><span class="pre">MISCFILE</span></tt> type: It has been suggested to completely drop |
325 | <li>removal of the <tt class="docutils literal"><span class="pre">MISCFILE</span></tt> type: It has been suggested to completely drop |
| 326 | entries of type <tt class="docutils literal"><span class="pre">MISCFILE</span></tt>. This would result in a minor space reduction |
326 | entries of type <tt class="docutils literal"><span class="pre">MISCFILE</span></tt>. This would result in a minor space reduction |
| 327 | (its rather unlikely to free any blocks) but completely remove the ability |
327 | (its rather unlikely to free any blocks) but completely remove the ability |
| 328 | to check these files for integrity. While they don't influence portage |
328 | to check these files for integrity. While they don't influence portage |
| … | |
… | |
| 376 | |
376 | |
| 377 | </div> |
377 | </div> |
| 378 | <div class="footer"> |
378 | <div class="footer"> |
| 379 | <hr class="footer" /> |
379 | <hr class="footer" /> |
| 380 | <a class="reference" href="glep-0044.txt">View document source</a>. |
380 | <a class="reference" href="glep-0044.txt">View document source</a>. |
| 381 | Generated on: 2005-12-06 16:19 UTC. |
381 | Generated on: 2006-01-23 10:22 UTC. |
| 382 | Generated by <a class="reference" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source. |
382 | Generated by <a class="reference" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source. |
| 383 | |
383 | |
| 384 | </div> |
384 | </div> |
| 385 | </body> |
385 | </body> |
| 386 | </html> |
386 | </html> |
Parent Directory
| 
Revision Log
| 
Patch