/[gentoo]/xml/htdocs/proj/en/glep/glep-0059.html
Gentoo

Diff of /xml/htdocs/proj/en/glep/glep-0059.html

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

Revision 1.8 Revision 1.11
25<tbody valign="top"> 25<tbody valign="top">
26<tr class="field"><th class="field-name">GLEP:</th><td class="field-body">59</td> 26<tr class="field"><th class="field-name">GLEP:</th><td class="field-body">59</td>
27</tr> 27</tr>
28<tr class="field"><th class="field-name">Title:</th><td class="field-body">Manifest2 hash policies and security implications</td> 28<tr class="field"><th class="field-name">Title:</th><td class="field-body">Manifest2 hash policies and security implications</td>
29</tr> 29</tr>
30<tr class="field"><th class="field-name">Version:</th><td class="field-body">1.6</td> 30<tr class="field"><th class="field-name">Version:</th><td class="field-body">1.9</td>
31</tr> 31</tr>
32<tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference external" href="http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/proj/en/glep/glep-0059.txt?cvsroot=gentoo">2010/01/31 09:55:43</a></td> 32<tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference external" href="http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/proj/en/glep/glep-0059.txt?cvsroot=gentoo">2010/04/07 21:34:24</a></td>
33</tr> 33</tr>
34<tr class="field"><th class="field-name">Author:</th><td class="field-body">Robin Hugh Johnson &lt;robbat2&#32;&#97;t&#32;gentoo.org&gt;,</td> 34<tr class="field"><th class="field-name">Author:</th><td class="field-body">Robin Hugh Johnson &lt;robbat2&#32;&#97;t&#32;gentoo.org&gt;,</td>
35</tr> 35</tr>
36<tr class="field"><th class="field-name">Status:</th><td class="field-body">Draft</td> 36<tr class="field"><th class="field-name">Status:</th><td class="field-body">Draft</td>
37</tr> 37</tr>
53</table> 53</table>
54<hr /> 54<hr />
55<div class="contents topic" id="contents"> 55<div class="contents topic" id="contents">
56<p class="topic-title first">Contents</p> 56<p class="topic-title first">Contents</p>
57<ul class="simple"> 57<ul class="simple">
58<li><a class="reference internal" href="#abstract" id="id1">Abstract</a></li> 58<li><a class="reference internal" href="#abstract" id="id2">Abstract</a></li>
59<li><a class="reference internal" href="#motivation" id="id2">Motivation</a></li> 59<li><a class="reference internal" href="#motivation" id="id3">Motivation</a></li>
60<li><a class="reference internal" href="#specification" id="id3">Specification</a><ul> 60<li><a class="reference internal" href="#specification" id="id4">Specification</a><ul>
61<li><a class="reference internal" href="#the-bad-news" id="id4">The bad news</a></li> 61<li><a class="reference internal" href="#the-bad-news" id="id5">The bad news</a></li>
62<li><a class="reference internal" href="#how-fast-can-md5-be-broken" id="id5">How fast can MD5 be broken?</a></li> 62<li><a class="reference internal" href="#how-fast-can-md5-be-broken" id="id6">How fast can MD5 be broken?</a></li>
63<li><a class="reference internal" href="#the-good-news" id="id6">The good news</a></li> 63<li><a class="reference internal" href="#the-good-news" id="id7">The good news</a></li>
64<li><a class="reference internal" href="#what-should-be-done" id="id7">What should be done</a></li> 64<li><a class="reference internal" href="#what-should-be-done" id="id8">What should be done</a></li>
65<li><a class="reference internal" href="#checksum-depreciation-timing" id="id8">Checksum depreciation timing</a></li> 65<li><a class="reference internal" href="#checksum-depreciation-timing" id="id9">Checksum depreciation timing</a><ul>
66<li><a class="reference internal" href="#general-principles" id="id10">General principles:</a></li>
67<li><a class="reference internal" href="#immediate-plans" id="id11">Immediate plans:</a></li>
66</ul> 68</ul>
67</li> 69</li>
70</ul>
71</li>
68<li><a class="reference internal" href="#backwards-compatibility" id="id9">Backwards Compatibility</a></li> 72<li><a class="reference internal" href="#backwards-compatibility" id="id12">Backwards Compatibility</a></li>
69<li><a class="reference internal" href="#references" id="id10">References</a></li> 73<li><a class="reference internal" href="#references" id="id13">References</a></li>
70<li><a class="reference internal" href="#thanks-to" id="id11">Thanks to</a></li> 74<li><a class="reference internal" href="#thanks-to" id="id14">Thanks to</a></li>
75<li><a class="reference internal" href="#id1" id="id15">References</a></li>
71<li><a class="reference internal" href="#copyright" id="id12">Copyright</a></li> 76<li><a class="reference internal" href="#copyright" id="id16">Copyright</a></li>
72</ul> 77</ul>
73</div> 78</div>
74<div class="section" id="abstract"> 79<div class="section" id="abstract">
75<h1><a class="toc-backref" href="#id1">Abstract</a></h1> 80<h1><a class="toc-backref" href="#id2">Abstract</a></h1>
76<p>While Manifest2 format allows multiple hashes, the question of which 81<p>While Manifest2 format allows multiple hashes, the question of which
77checksums should be present, why, and the security implications of such 82checksums should be present, why, and the security implications of such
78have never been resolved. This GLEP covers all of these issues, and 83have never been resolved. This GLEP covers all of these issues, and
79makes recommendations as to how to handle checksums both now, and in 84makes recommendations as to how to handle checksums both now, and in
80future.</p> 85future.</p>
81</div> 86</div>
82<div class="section" id="motivation"> 87<div class="section" id="motivation">
83<h1><a class="toc-backref" href="#id2">Motivation</a></h1> 88<h1><a class="toc-backref" href="#id3">Motivation</a></h1>
84<p>This GLEP is being written as part of the work on signing the Portage 89<p>This GLEP is being written as part of the work on signing the Portage
85tree, but is only tangentially related to the actual signing of 90tree, but is only tangentially related to the actual signing of
86Manifests. Checksums present one possible weak point in the overall 91Manifests. Checksums present one possible weak point in the overall
87security of the tree - and a comprehensive security plan is needed.</p> 92security of the tree - and a comprehensive security plan is needed.</p>
88<p>This GLEP is not mandatory for the tree-signing specification, but 93<p>This GLEP is not mandatory for the tree-signing specification, but
89instead aims to improve the security of the hashes used in Manifest2. 94instead aims to improve the security of the hashes used in Manifest2
90As such, it is also able to stand on it's own.</p> 95[GLEP44]. As such, it is also able to stand on it's own.</p>
91</div> 96</div>
92<div class="section" id="specification"> 97<div class="section" id="specification">
93<h1><a class="toc-backref" href="#id3">Specification</a></h1> 98<h1><a class="toc-backref" href="#id4">Specification</a></h1>
94<div class="section" id="the-bad-news"> 99<div class="section" id="the-bad-news">
95<h2><a class="toc-backref" href="#id4">The bad news</a></h2> 100<h2><a class="toc-backref" href="#id5">The bad news</a></h2>
96<p>First of all, I'd like to cover the bad news in checksum security. 101<p>First of all, I'd like to cover the bad news in checksum security.
97A much discussed point, as been the simple question: What is the 102A much discussed point, as been the simple question: What is the
98security of multiple independent checksums on the same data? 103security of multiple independent checksums on the same data?
99The most common position (and indeed the one previously held by myself), 104The most common position (and indeed the one previously held by myself),
100is that multiple checksums would be an increase in security, but we 105is that multiple checksums would be an increase in security, but we
106checksum.</p> 111checksum.</p>
107<p>Wang et al [W04] extended Joux's [J04] work on SHA-0 to cover MD4, MD5, 112<p>Wang et al [W04] extended Joux's [J04] work on SHA-0 to cover MD4, MD5,
108HAVAL-128 and RIPEMD families of hashes.</p> 113HAVAL-128 and RIPEMD families of hashes.</p>
109</div> 114</div>
110<div class="section" id="how-fast-can-md5-be-broken"> 115<div class="section" id="how-fast-can-md5-be-broken">
111<h2><a class="toc-backref" href="#id5">How fast can MD5 be broken?</a></h2> 116<h2><a class="toc-backref" href="#id6">How fast can MD5 be broken?</a></h2>
112<p>For a general collision, not a pre-image attack, since the announcement 117<p>For a general collision, not a pre-image attack, since the announcement
113by Wang et al [W04], the time required to break MD5 has been massively 118by Wang et al [W04], the time required to break MD5 has been massively
114reduced. Originally at 1 hour on a near-supercomputer (IBM P690) and 119reduced. Originally at 1 hour on a near-supercomputer (IBM P690) and
115estimated at 64 hours with a Pentium-3 1.7Ghz. This has gone down to 120estimated at 64 hours with a Pentium-3 1.7Ghz. This has gone down to
116less than in two years, to 17 seconds [K06a].</p> 121less than in two years, to 17 seconds [K06a].</p>
122<ul class="simple">
117<p>08/2004 - 1 hour, IBM pSeries 690 (32x 1.7Ghz POWER4+) = 54.4 GHz-Hours 123<li>08/2004 - 1 hour, IBM pSeries 690 (32x 1.7Ghz POWER4+) = 54.4 GHz-Hours</li>
11803/2005 - 8 hours, Pentium-M 1.6Ghz = 12.8 Ghz-Hours 124<li>03/2005 - 8 hours, Pentium-M 1.6Ghz = 12.8 Ghz-Hours</li>
11911/2005 - 5 hours, Pentium-4 1.7Ghz = 8.5 Ghz-Hours 125<li>11/2005 - 5 hours, Pentium-4 1.7Ghz = 8.5 Ghz-Hours</li>
12003/2006 - 1 minute, Pentium-4 3.2Ghz = .05 Ghz-Hours 126<li>03/2006 - 1 minute, Pentium-4 3.2Ghz = .05 Ghz-Hours</li>
12104/2006 - 17 seconds, Pentium-4 3.2Ghz = .01 Ghz-Hours</p> 127<li>04/2006 - 17 seconds, Pentium-4 3.2Ghz = .01 Ghz-Hours</li>
128</ul>
122<p>If we accept a factor of 800x as a sample of how much faster a checksum 129<p>If we accept a factor of 800x as a sample of how much faster a checksum
123may be broken over the course of 2 years (MD5 using the above data is 130may be broken over the course of 2 years (MD5 using the above data is
124&gt;2000x), then existing checksums do not stand a significant chance of 131&gt;2000x), then existing checksums do not stand a significant chance of
125survival in the future. We should thus accept that whatever checksums we 132survival in the future. We should thus accept that whatever checksums we
126are using today, will be broken in the near future, and plan as best as 133are using today, will be broken in the near future, and plan as best as
130feasible, see [K06b] for an application that can produce two 137feasible, see [K06b] for an application that can produce two
131self-extracting EXE files, with identical MD5s, and whatever payload you 138self-extracting EXE files, with identical MD5s, and whatever payload you
132want.</p> 139want.</p>
133</div> 140</div>
134<div class="section" id="the-good-news"> 141<div class="section" id="the-good-news">
135<h2><a class="toc-backref" href="#id6">The good news</a></h2> 142<h2><a class="toc-backref" href="#id7">The good news</a></h2>
136<p>Of the checksums presently used by Manifest2 (SHA1, SHA256, RIPEMD160), 143<p>Of the checksums presently used by Manifest2 (SHA1, SHA256, RIPEMD160),
137one stands close to being completely broken: SHA1; and another is 144one stands close to being completely broken: SHA1; and another is
138significantly weakened: RIPEMD160. The SHA2 series has suffered some 145significantly weakened: RIPEMD160. The SHA2 series has suffered some
139attacks, but still remains reasonably solid [G07],[K08].</p> 146attacks, but still remains reasonably solid [G07],[K08].</p>
140<p>To reduce the potential for future problems and any single checksum 147<p>To reduce the potential for future problems and any single checksum
142strongest hash available from each family of checksums, and be prepared 149strongest hash available from each family of checksums, and be prepared
143to retire old checksums actively, unless there is a overriding reason to 150to retire old checksums actively, unless there is a overriding reason to
144keep a specific checksum, such as part of a migration plan.</p> 151keep a specific checksum, such as part of a migration plan.</p>
145</div> 152</div>
146<div class="section" id="what-should-be-done"> 153<div class="section" id="what-should-be-done">
147<h2><a class="toc-backref" href="#id7">What should be done</a></h2> 154<h2><a class="toc-backref" href="#id8">What should be done</a></h2>
148<p>Portage should always try to verify all supported hashes that are 155<p>Portage should always try to verify all supported hashes that are
149available in a Manifest2, starting with the strongest ones as maintained 156available in a Manifest2, starting with the strongest ones as maintained
150by a preference list. Over time, the weaker checksums should be removed 157by a preference list. Over time, the weaker checksums should be removed
151from Manifest2 files, once all old Portage installations have had 158from Manifest2 files, once all old Portage installations have had
152sufficient time to upgrade. We should be prepared to add stronger 159sufficient time to upgrade. Stronger checksums shall be added as soon as
153checksums wherever possible, and to remove those that have been 160an implementation is available in Portage. Weak checksums may be removed
154defeated.</p> 161as long as the depreciation process is followed (see below).</p>
155<p>As soon as feasible, we should add the SHA512 and WHIRLPOOL algorithms. 162<p>As soon as feasible, we should add the SHA512 and WHIRLPOOL algorithms.
156In future, as stream-based checksums are developed (in response to the 163In future, as stream-based checksums are developed (in response to the
157development by NIST [AHS]), they should be considered and used.</p> 164development by NIST [AHS]), they should be considered and used.</p>
158<p>The SHA512 algorithm is available in Python 2.5, which has been a 165<p>The SHA512 algorithm is available in Python 2.5, which has been a
159dependency of Portage since approximately Python 2.1.6.13.</p> 166dependency of Portage since approximately Portage 2.1.6.13.</p>
160<p>The WHIRLPOOL checksum is not available within the PyCrypto library or 167<p>The WHIRLPOOL checksum is not available within the PyCrypto library or
161hashlib that is part of Python 2.5, but there are multiple alternative 168hashlib that is part of Python 2.5, but there are multiple alternative
162Python implementations available, ranging from pure Python to C-based 169Python implementations available, ranging from pure Python to C-based
163(python-mhash).</p> 170(python-mhash).</p>
164<p>The existence unsupported hash is not considered to be a failure unless 171<p>The existence unsupported hash is not considered to be a failure unless
165no supported hashes are available for a given Manifest entry.</p> 172no supported hashes are available for a given Manifest entry.</p>
166</div> 173</div>
167<div class="section" id="checksum-depreciation-timing"> 174<div class="section" id="checksum-depreciation-timing">
168<h2><a class="toc-backref" href="#id8">Checksum depreciation timing</a></h2> 175<h2><a class="toc-backref" href="#id9">Checksum depreciation timing</a></h2>
176<div class="section" id="general-principles">
177<h3><a class="toc-backref" href="#id10">General principles:</a></h3>
178<p>A minimum set of depreciated checksums shall be maintained only to
179support old package manager versions where needed by historically used
180trees:</p>
181<ul class="simple">
182<li>New package manager versions should NOT use depreciated checksums in</li>
183<li>New trees with that have never used the depreciated checksums may omit
184them for reasons of size, but are still strongly suggested to include
185them.</li>
186<li>Removal of depreciated checksums shall happen after no less than 18
187months or one major Portage version cycle, whichever is greater.</li>
188</ul>
189</div>
190<div class="section" id="immediate-plans">
191<h3><a class="toc-backref" href="#id11">Immediate plans:</a></h3>
169<p>For the current Portage, both SHA1 and RIPEMD160 should be immediately 192<p>For the current Portage, both SHA1 and RIPEMD160 should be immediately
170removed, as they present no advantages over the already present SHA256. 193removed, as they present no advantages over the already present SHA256.
171SHA256 cannot be replaced immediately with SHA512, as existing Portage 194SHA256 cannot be replaced immediately with SHA512, as existing Portage
172versions need at least one supported algorithm present (SHA256 support 195versions need at least one supported algorithm present (SHA256 support
173was added in June 2006), so it must be retained for some while.</p> 196was added in June 2006), so it must be retained for some while.</p>
174<p>Immediately: 197<p>Immediately:</p>
198<ul class="simple">
175- Add WHIRLPOOL and SHA512. 199<li>Add WHIRLPOOL and SHA512.</li>
176- Remove SHA1 and RIPEMD160.</p> 200<li>Remove SHA1 and RIPEMD160.</li>
201</ul>
177<p>After the majority of Portage installations include SHA512 support: 202<p>After the majority of Portage installations include SHA512 support:</p>
203<ul class="simple">
178- Remove SHA256.</p> 204<li>Remove SHA256.</li>
205</ul>
206</div>
179</div> 207</div>
180</div> 208</div>
181<div class="section" id="backwards-compatibility"> 209<div class="section" id="backwards-compatibility">
182<h1><a class="toc-backref" href="#id9">Backwards Compatibility</a></h1> 210<h1><a class="toc-backref" href="#id12">Backwards Compatibility</a></h1>
183<p>Old versions of Portage may support and expect only specific checksums. 211<p>Old versions of Portage may support and expect only specific checksums.
184This is accounted for in the checksum depreciation discussion.</p> 212This is accounted for in the checksum depreciation discussion.</p>
213<p>For maximum compatiability, we should only have to include each of the
214old algorithms that we are officially still supporting, as well as the
215new ones that we prefer.</p>
185</div> 216</div>
186<div class="section" id="references"> 217<div class="section" id="references">
187<h1><a class="toc-backref" href="#id10">References</a></h1> 218<h1><a class="toc-backref" href="#id13">References</a></h1>
188<dl class="docutils"> 219<dl class="docutils">
189<dt>[AHS] NIST (2007). &quot;NIST's Plan for New Cryptographic Hash Functions&quot;,</dt> 220<dt>[AHS] NIST (2007). &quot;NIST's Plan for New Cryptographic Hash Functions&quot;,</dt>
190<dd>(Advanced Hash Standard). <a class="reference external" href="http://csrc.nist.gov/pki/HashWorkshop/">http://csrc.nist.gov/pki/HashWorkshop/</a></dd> 221<dd>(Advanced Hash Standard). <a class="reference external" href="http://csrc.nist.gov/pki/HashWorkshop/">http://csrc.nist.gov/pki/HashWorkshop/</a></dd>
191<dt>[BOBO06] Boneh, D. and Boyen, X. (2006). &quot;On the Impossibility of</dt> 222<dt>[BOBO06] Boneh, D. and Boyen, X. (2006). &quot;On the Impossibility of</dt>
192<dd>Efficiently Combining Collision Resistant Hash Functions&quot;; Proceedings 223<dd>Efficiently Combining Collision Resistant Hash Functions&quot;; Proceedings
220version (August 17, 2004). Available online from: 251version (August 17, 2004). Available online from:
221<a class="reference external" href="http://eprint.iacr.org/2004/199.pdf">http://eprint.iacr.org/2004/199.pdf</a></dd> 252<a class="reference external" href="http://eprint.iacr.org/2004/199.pdf">http://eprint.iacr.org/2004/199.pdf</a></dd>
222</dl> 253</dl>
223</div> 254</div>
224<div class="section" id="thanks-to"> 255<div class="section" id="thanks-to">
225<h1><a class="toc-backref" href="#id11">Thanks to</a></h1> 256<h1><a class="toc-backref" href="#id14">Thanks to</a></h1>
226<dl class="docutils"> 257<dl class="docutils">
227<dt>I'd like to thank the following folks, in no specific order:</dt> 258<dt>I'd like to thank the following folks, in no specific order:</dt>
228<dd><ul class="first last simple"> 259<dd><ul class="first last simple">
229<li>Ciaran McCreesh (ciaranm) - for pointing out the Joux (2004) paper, 260<li>Ciaran McCreesh (ciaranm) - for pointing out the Joux (2004) paper,
230and also being stubborn enough in not accepting a partial solution.</li> 261and also being stubborn enough in not accepting a partial solution.</li>
233codebase.</li> 264codebase.</li>
234</ul> 265</ul>
235</dd> 266</dd>
236</dl> 267</dl>
237</div> 268</div>
269<div class="section" id="id1">
270<h1><a class="toc-backref" href="#id15">References</a></h1>
271<table class="docutils citation" frame="void" id="glep44" rules="none">
272<colgroup><col class="label" /><col /></colgroup>
273<tbody valign="top">
274<tr><td class="label">[GLEP44]</td><td>Mauch, M. (2005) GLEP44 - Manifest2 format.
275<a class="reference external" href="http://www.gentoo.org/proj/en/glep/glep-0044.html">http://www.gentoo.org/proj/en/glep/glep-0044.html</a></td></tr>
276</tbody>
277</table>
278</div>
238<div class="section" id="copyright"> 279<div class="section" id="copyright">
239<h1><a class="toc-backref" href="#id12">Copyright</a></h1> 280<h1><a class="toc-backref" href="#id16">Copyright</a></h1>
240<p>Copyright (c) 2006-2010 by Robin Hugh Johnson. This material may be 281<p>Copyright (c) 2006-2010 by Robin Hugh Johnson. This material may be
241distributed only subject to the terms and conditions set forth in the 282distributed only subject to the terms and conditions set forth in the
242Open Publication License, v1.0.</p> 283Open Publication License, v1.0.</p>
243<p>vim: tw=72 ts=2 expandtab:</p> 284<!-- vim: tw=72 ts=2 expandtab: -->
244</div> 285</div>
245 286
246</div> 287</div>
247<div class="footer"> 288<div class="footer">
248<hr class="footer" /> 289<hr class="footer" />
249<a class="reference external" href="glep-0059.txt">View document source</a>. 290<a class="reference external" href="glep-0059.txt">View document source</a>.
250Generated on: 2010-01-31 09:56 UTC. 291Generated on: 2010-04-07 21:54 UTC.
251Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source. 292Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.
252 293
253</div> 294</div>
254</body> 295</body>
255</html> 296</html>

Legend:
Removed from v.1.8  
changed lines
  Added in v.1.11

  ViewVC Help
Powered by ViewVC 1.1.20