Contents of /xml/htdocs/proj/en/glep/glep-0058.html

Parent Directory Parent Directory | Revision Log Revision Log

Revision 1.1 - (show annotations) (download) (as text)
Tue Oct 28 07:47:52 2008 UTC (10 years ago) by robbat2
Branch: MAIN
File MIME type: text/html
Regen HTML.

1 <?xml version="1.0" encoding="utf-8" ?>
2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
3 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
5 <head>
6 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
7 <meta name="generator" content="Docutils 0.5: http://docutils.sourceforge.net/" />
8 <title>GLEP 58 -- Security of distribution of Gentoo software - Infrastructure to User distribution - MetaManifest</title>
9 <link rel="stylesheet" href="tools/glep.css" type="text/css" /></head>
10 <body bgcolor="white">
11 <table class="navigation" cellpadding="0" cellspacing="0"
12 width="100%" border="0">
13 <tr><td class="navicon" width="150" height="35">
14 <a href="http://www.gentoo.org/" title="Gentoo Linux Home Page">
15 <img src="http://www.gentoo.org/images/gentoo-new.gif" alt="[Gentoo]"
16 border="0" width="150" height="35" /></a></td>
17 <td class="textlinks" align="left">
18 [<b><a href="http://www.gentoo.org/">Gentoo Linux Home</a></b>]
19 [<b><a href="http://www.gentoo.org/proj/en/glep">GLEP Index</a></b>]
20 [<b><a href="http://www.gentoo.org/proj/en/glep/glep-0058.txt">GLEP Source</a></b>]
21 </td></tr></table>
22 <table class="rfc2822 docutils field-list" frame="void" rules="none">
23 <col class="field-name" />
24 <col class="field-body" />
25 <tbody valign="top">
26 <tr class="field"><th class="field-name">GLEP:</th><td class="field-body">58</td>
27 </tr>
28 <tr class="field"><th class="field-name">Title:</th><td class="field-body">Security of distribution of Gentoo software - Infrastructure to User distribution - MetaManifest</td>
29 </tr>
30 <tr class="field"><th class="field-name">Version:</th><td class="field-body">1.4</td>
31 </tr>
32 <tr class="field"><th class="field-name">Last-Modified:</th><td class="field-body"><a class="reference external" href="http://www.gentoo.org/cgi-bin/viewcvs.cgi/xml/htdocs/proj/en/glep/glep-0058.txt?cvsroot=gentoo">2008/10/28 07:45:27</a></td>
33 </tr>
34 <tr class="field"><th class="field-name">Author:</th><td class="field-body">Robin Hugh Johnson &lt;robbat2&#32;&#97;t&#32;gentoo.org&gt;,</td>
35 </tr>
36 <tr class="field"><th class="field-name">Status:</th><td class="field-body">Draft</td>
37 </tr>
38 <tr class="field"><th class="field-name">Type:</th><td class="field-body">Standards Track</td>
39 </tr>
40 <tr class="field"><th class="field-name">Content-Type:</th><td class="field-body"><a class="reference external" href="glep-0002.html">text/x-rst</a></td>
41 </tr>
42 <tr class="field"><th class="field-name">Requires:</th><td class="field-body"><a class="reference external" href="http://www.gentoo.org/proj/en/glepglep-0044.html">44</a> <a class="reference external" href="http://www.gentoo.org/proj/en/glepglep-0060.html">60</a></td>
43 </tr>
44 <tr class="field"><th class="field-name">Created:</th><td class="field-body">October 2006</td>
45 </tr>
46 <tr class="field"><th class="field-name">Updated:</th><td class="field-body">November 2007, June 2008, July 2008, October 2008</td>
47 </tr>
48 <tr class="field"><th class="field-name">Post-History:</th><td class="field-body"></td>
49 </tr>
50 </tbody>
51 </table>
52 <hr />
53 <div class="contents topic" id="contents">
54 <p class="topic-title first">Contents</p>
55 <ul class="simple">
56 <li><a class="reference internal" href="#abstract" id="id1">Abstract</a></li>
57 <li><a class="reference internal" href="#motivation" id="id2">Motivation</a></li>
58 <li><a class="reference internal" href="#specification" id="id3">Specification</a><ul>
59 <li><a class="reference internal" href="#procedure-for-creating-the-metamanifest-file" id="id4">Procedure for creating the MetaManifest file:</a></li>
60 <li><a class="reference internal" href="#verification-of-one-or-more-items-from-the-metamanifest" id="id5">Verification of one or more items from the MetaManifest:</a></li>
61 <li><a class="reference internal" href="#procedure-for-verifying-an-item-in-the-metamanifest" id="id6">Procedure for verifying an item in the MetaManifest:</a><ul>
62 <li><a class="reference internal" href="#notes" id="id7">Notes:</a></li>
63 </ul>
64 </li>
65 </ul>
66 </li>
67 <li><a class="reference internal" href="#implementation-notes" id="id8">Implementation Notes</a><ul>
68 <li><a class="reference internal" href="#metamanifest-and-the-new-manifest2-filetypes" id="id9">MetaManifest and the new Manifest2 filetypes</a></li>
69 <li><a class="reference internal" href="#timestamps-additional-distribution-of-metamanifest" id="id10">Timestamps &amp; Additional distribution of MetaManifest</a></li>
70 <li><a class="reference internal" href="#metamanifest-size-considerations" id="id11">MetaManifest size considerations</a></li>
71 </ul>
72 </li>
73 <li><a class="reference internal" href="#backwards-compatibility" id="id12">Backwards Compatibility</a></li>
74 <li><a class="reference internal" href="#thanks" id="id13">Thanks</a></li>
75 <li><a class="reference internal" href="#references" id="id14">References</a></li>
76 <li><a class="reference internal" href="#copyright" id="id15">Copyright</a></li>
77 </ul>
78 </div>
79 <div class="section" id="abstract">
80 <h1><a class="toc-backref" href="#id1">Abstract</a></h1>
81 <p>MetaManifest provides a means of verifiable distribution from Gentoo
82 Infrastructure to a user system, while data is conveyed over completely
83 untrusted networks and system, by extending the Manifest2 specification,
84 and adding a top-level Manifest file, with support for other nested
85 Manifests.</p>
86 </div>
87 <div class="section" id="motivation">
88 <h1><a class="toc-backref" href="#id2">Motivation</a></h1>
89 <p>As part of a comprehensive security plan, we need a way to prove that
90 something originating from Gentoo as an organization (read Gentoo-owned
91 hardware, run by infrastructure), has not been tampered with. This
92 allows the usage of third-party rsync mirrors, without worrying that
93 they have modified something critical (e.g. eclasses, which are still
94 unsigned).</p>
95 <p>Securing the untrusted distribution is one of the easier tasks in the
96 security plan - in short, all that is required is having a hash of every
97 item in the tree, and signing that hash to prove it came from Gentoo.</p>
98 <p>Ironically we have a hashed and signed distribution (it's just not used
99 by most users, due to it's drawbacks): Our tree snapshot tarballs have
100 hashes and signatures.</p>
101 <p>So now we want to add the same verification to our material that is
102 distributed by rsync. We already provide hashes of subsets of the tree -
103 our Manifests protect individual packages. However metadata, eclasses
104 and profiles are not protected at this time. The directories of
105 packages and distfiles are NOT covered by this, as they are not
106 distributed by rsync.</p>
107 <p>This portion of the tree-signing work provides only the following
108 guarantee: A user can prove that the tree from the Gentoo infrastructure
109 has not been tampered with since leaving the Gentoo infrastructure.
110 No other guarantees, either implicit or explicit are made.</p>
111 <p>Additionally, distributing a set of the most recent MetaManifests from a
112 trusted source allows validation of trees that come from community
113 mirrors, and allows detection of all cases of malicious mirrors (either
114 by deliberate delay, replay [C08a, C08b] or alteration).</p>
115 </div>
116 <div class="section" id="specification">
117 <h1><a class="toc-backref" href="#id3">Specification</a></h1>
118 <p>For lack of a better name, the following solution should be known as the
119 MetaManifest. Those responsible for the name have already been sacked.</p>
120 <p>MetaManifest basically contains hashes of every file in the tree, either
121 directly or indirectly. The direct case applies to ANY file that does
122 not appear in an existing Manifest file (e.g. eclasses, Manifest files
123 themselves). The indirect case is covered by the CONTENTS of existing
124 Manifest files. If the Manifest itself is correct, we know that by
125 tracking the hash of the Manifest, we can be assured that the contents
126 are protected.</p>
127 <p>In the following, the MetaManifest file is a file named 'Manifest',
128 located at the root of a repository.</p>
129 <div class="section" id="procedure-for-creating-the-metamanifest-file">
130 <h2><a class="toc-backref" href="#id4">Procedure for creating the MetaManifest file:</a></h2>
131 <ol class="arabic simple">
132 <li>Start at the root of the Gentoo Portage tree (gentoo-x86, although
133 this procedure applies to overlays as well).</li>
134 <li>Initialize two unordered sets: COVERED, ALL.<ol class="arabic">
135 <li>'ALL' will contain every file in the tree.</li>
136 <li>'COVERED' will contain every file that is mentioned in an existing
137 Manifest2.</li>
138 </ol>
139 </li>
140 <li>Traverse the tree, depth-first.<ol class="arabic">
141 <li>At the top level only, ignore the following directories: distfiles,
142 packages, local</li>
143 <li>If a directory contains a Manifest file, extract all relevant local
144 files from it (presently: AUX, MISC, EBUILD; but should follow the
145 evolution of Manifest2 entry types per [#GLEP60]), and place them
146 into the COVERED set.</li>
147 <li>Recursively add every file in the directory to the ALL set,
148 pursusant to the exclusion list as mentioned in [#GLEP60].</li>
149 </ol>
150 </li>
151 <li>Produce a new set, UNCOVERED, as the set-difference (ALL)-(COVERED).
152 This is every item that is not covered by another Manifest, or part
153 of an exclusion list.</li>
154 <li>If an existing MetaManifest file is present, remove it.</li>
155 <li>For each file in UNCOVERED, assign a Manifest2 type, produce the
156 hashes, and add with the filetype to the MetaManifest file.</li>
157 <li>For unique identification of the MetaManifest, a header line should
158 be included, using the exact contents of the metadata/timestamp.x
159 file, so that a MetaManifest may be tied back to a tree as
160 distributed by the rsync mirror system. The string of
161 'metadata/timestamp.x' should be included to identify this revision
162 of MetaManifest generation. Eg:
163 &quot;Timestamp: metadata/timestamp.x: 1215722461 Thu Jul 10 20:41:01 2008 UTC&quot;
164 The package manager MUST not use the identifying string as a filename.</li>
165 <li>The MetaManifest must ultimately be GnuPG-signed.<ol class="arabic">
166 <li>For the initial implementation, the same key as used for snapshot
167 tarball signing is sufficient.</li>
168 <li>For the future, the key used for fully automated signing by infra
169 should not be on the same keyring as developer keys. See [#GLEPxx+3
170 for further notes].</li>
171 </ol>
172 </li>
173 </ol>
174 <p>The above does not conflict the proposal contained in GLEP33, which
175 restructure eclasses to include subdirectories and Manifest files, as
176 the Manifest rules above still provide indirect verification for all
177 files after the GLEP33 restructuring if it comes to pass.</p>
178 <p>If other Manifests are added (such as per-category, or protecting
179 versioned eclases), the size of the MetaManifest will be greatly
180 reduced, and this specification was written with such a possible future
181 addition in mind.</p>
182 <p>MetaManifest generation will take place as part of the existing process
183 by infrastructure that takes the contents of CVS and prepares it for
184 distribution via rsync, which includes generating metadata. In-tree
185 Manifest files are not checked at this point, as they are assumed to be
186 correct.</p>
187 </div>
188 <div class="section" id="verification-of-one-or-more-items-from-the-metamanifest">
189 <h2><a class="toc-backref" href="#id5">Verification of one or more items from the MetaManifest:</a></h2>
190 <p>There are two times that this may happen: firstly, immediately after the
191 rsync has completed - this has the advantage that the kernel file cache
192 is hot, and checking the entire tree can be accomplished quickly.
193 Secondly, the MetaManifest should be checked during installation of a
194 package.</p>
195 </div>
196 <div class="section" id="procedure-for-verifying-an-item-in-the-metamanifest">
197 <h2><a class="toc-backref" href="#id6">Procedure for verifying an item in the MetaManifest:</a></h2>
198 <p>In the following, I've used term 'M2-verify' to note following the hash
199 verification procedures as defined by the Manifest2 format - which
200 compromise checking the file length, and that the hashes match. Which
201 filetypes may be ignored on missing is discussed in [#GLEP60].</p>
202 <ol class="arabic simple">
203 <li>Check the GnuPG signature on the MetaManifest against the keyring of
204 automated Gentoo keys. See [#GLEPxx+3] for full details regarding
205 verification of GnuPG signatures.
206 1. Abort if the signature check fails.</li>
207 <li>Check the Timestamp header. If it is significently out of date
208 compared to the local clock or a trusted source, halt or require
209 manual intervention from the user.</li>
210 <li>For a verification of the tree following an rsync:<ol class="arabic">
211 <li>Build a set 'ALL' of every file covered by the rsync. (exclude
212 distfiles/, packages/, local/)</li>
213 <li>M2-verify every entry in the MetaManifest, descending into inferior
214 Manifests as needed. Place the relative path of every checked item
215 into a set 'COVERED'.</li>
216 <li>Construct the set 'UNCOVERED' by set-difference between the ALL and
217 COVERED sets.</li>
218 <li>For each file in the UNCOVERED set, assign a Manifest2 filetype.</li>
219 <li>If the filetype for any file in the UNCOVERED set requires a halt
220 on error, abort and display a suitable error.</li>
221 <li>Completed verification</li>
222 </ol>
223 </li>
224 <li>If checking at the installation of a package:<ol class="arabic">
225 <li>M2-verify the entry in MetaManifest for the Manifest</li>
226 <li>M2-verify all relevant metadata/ contents if metadata/ is being
227 used in any way (optionally done before dependancy checking).</li>
228 <li>M2-verifying the contents of the Manifest.</li>
229 <li>Perform M2-verification of all eclasses and profiles used (both
230 directly and indirectly) by the ebuild.</li>
231 </ol>
232 </li>
233 </ol>
234 <div class="section" id="notes">
235 <h3><a class="toc-backref" href="#id7">Notes:</a></h3>
236 <ol class="arabic simple">
237 <li>For initial implementations, it is acceptable to check EVERY item in
238 the eclass and profiles directory, rather than tracking the exact
239 files used by every eclass (see note #2). Later implementations
240 should strive to only verify individual eclasses and profiles as
241 needed.</li>
242 <li>Tracking of exact files is of specific significance to the libtool
243 eclass, as it stores patches under eclass/ELT-patches, and as such
244 that would not be picked up by any tracing of the inherit function.
245 This may be alleviated by a later eclass and ebuild variable that
246 explicitly declares what files from the tree are used by a package.</li>
247 </ol>
248 </div>
249 </div>
250 </div>
251 <div class="section" id="implementation-notes">
252 <h1><a class="toc-backref" href="#id8">Implementation Notes</a></h1>
253 <p>For this portion of the tree-signing work, no actions are required of
254 the individual Gentoo developers. They will continue to develop and
255 commit as they do presently, and the MetaManifest is added by
256 Infrastructure during the tree generation process, and distributed to
257 users.</p>
258 <div class="section" id="metamanifest-and-the-new-manifest2-filetypes">
259 <h2><a class="toc-backref" href="#id9">MetaManifest and the new Manifest2 filetypes</a></h2>
260 <p>While [#GLEP60] describes the addition of new filetypes, these are NOT
261 needed for implementation of the MetaManifest proposal. Without the new
262 filetypes, all entries in the MetaManifest would be of type 'MISC'.</p>
263 </div>
264 <div class="section" id="timestamps-additional-distribution-of-metamanifest">
265 <h2><a class="toc-backref" href="#id10">Timestamps &amp; Additional distribution of MetaManifest</a></h2>
266 <p>As discussed by [C08a,C08b], malicious third-party mirrors may use the
267 principles of exclusion and replay to deny an update to clients, while
268 at the same time recording the identity of clients to attack.</p>
269 <p>This should be guarded against by including a timestamp in the header of
270 the MetaManifest, as well as distributing the latest MetaManifests by a
271 trusted channel.</p>
272 <p>On all rsync mirrors directly maintained by the Gentoo infrastructure,
273 and not on community mirrors, there should be a new module
274 'gentoo-portage-metamanifests'. Within this module, all MetaManifests
275 for a recent time frame (eg one week) should be kept, named as
276 &quot;MetaManifest.$TS&quot;, where $TS is the timestamp from inside the file.
277 The most recent MetaManifest should always be symlinked as
278 MetaManifest.current. The possibility of serving the recent
279 MetaManifests via HTTPS should also be explored to mitigate MitM
280 attacks.</p>
281 <p>The package manager should obtain MetaManifest.current and use it to
282 decide is the tree is too out of date per operation #2 of the
283 verification process. The decision about freshness should be a
284 user-configuration setting, with the ability to override.</p>
285 </div>
286 <div class="section" id="metamanifest-size-considerations">
287 <h2><a class="toc-backref" href="#id11">MetaManifest size considerations</a></h2>
288 <p>With only two levels of Manifests (per-package and top-level), every
289 rsync will cause a lot of traffic transfering the modified top-level
290 MetaManifest. To reduce this, per-category Manifests are strongly
291 recommended. Alternatively, if the distribution method efficently
292 handles small patch-like changes in an existing file, using an
293 uncompressed MetaManifest may be acceptable (this would primarily be
294 distributed version control systems). Other suggestions in reducing this
295 traffic are welcomed.</p>
296 </div>
297 </div>
298 <div class="section" id="backwards-compatibility">
299 <h1><a class="toc-backref" href="#id12">Backwards Compatibility</a></h1>
300 <ul class="simple">
301 <li>There are no backwards compatibility issues, as old versions of
302 Portage do not look for a Manifest file at the top level of the tree.</li>
303 <li>Manifest2-aware versions of Portage ignore all entries that they are
304 not certain how to handle. Enabling headers and PGP signing to be
305 conducted easily.</li>
306 </ul>
307 </div>
308 <div class="section" id="thanks">
309 <h1><a class="toc-backref" href="#id13">Thanks</a></h1>
310 <p>I'd like to thank the following people for input on this GLEP.</p>
311 <ul class="simple">
312 <li>Patrick Lauer (patrick): Prodding me to get all of the tree-signing
313 work finished, and helping to edit.</li>
314 <li>Ciaran McCreesh (ciaranm): Paludis Manifest2</li>
315 <li>Brian Harring (ferringb): pkgcore Manifest2</li>
316 <li>Marius Mauch (genone) &amp; Zac Medico (zmedico): Portage Manifest2</li>
317 <li>Ned Ludd (solar) - Security concept review</li>
318 </ul>
319 </div>
320 <div class="section" id="references">
321 <h1><a class="toc-backref" href="#id14">References</a></h1>
322 <dl class="docutils">
323 <dt>[C08a] Cappos, J et al. (2008). &quot;Package Management Security&quot;.</dt>
324 <dd>University of Arizona Technical Report TR08-02. Available online
325 from: <a class="reference external" href="ftp://ftp.cs.arizona.edu/reports/2008/TR08-02.pdf">ftp://ftp.cs.arizona.edu/reports/2008/TR08-02.pdf</a></dd>
326 <dt>[C08b] Cappos, J et al. (2008). &quot;Attacks on Package Managers&quot;</dt>
327 <dd>Available online at:
328 <a class="reference external" href="http://www.cs.arizona.edu/people/justin/packagemanagersecurity/">http://www.cs.arizona.edu/people/justin/packagemanagersecurity/</a></dd>
329 </dl>
330 </div>
331 <div class="section" id="copyright">
332 <h1><a class="toc-backref" href="#id15">Copyright</a></h1>
333 <p>Copyright (c) 2006 by Robin Hugh Johnson. This material may be
334 distributed only subject to the terms and conditions set forth in the
335 Open Publication License, v1.0.</p>
336 <p>vim: tw=72 ts=2 expandtab:</p>
337 </div>
339 </div>
340 <div class="footer">
341 <hr class="footer" />
342 <a class="reference external" href="glep-0058.txt">View document source</a>.
343 Generated on: 2008-10-28 07:47 UTC.
344 Generated by <a class="reference external" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference external" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.
346 </div>
347 </body>
348 </html>

  ViewVC Help
Powered by ViewVC 1.1.20