/[gentoo]/xml/htdocs/proj/en/glep/glep-0058.txt
Gentoo

Contents of /xml/htdocs/proj/en/glep/glep-0058.txt

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.2 - (hide annotations) (download)
Wed Oct 22 17:59:43 2008 UTC (5 years, 6 months ago) by robbat2
Branch: MAIN
Changes since 1.1: +25 -20 lines
File MIME type: text/plain
Fix RST validation for tree-signing GLEPS.

1 cardoe 1.1 GLEP: 58
2     Title: Security of distribution of Gentoo software - Infrastructure to User distribution - MetaManifest
3 robbat2 1.2 Version: $Revision: 1.1 $
4     Last-Modified: $Date: 2008/10/21 23:30:47 $
5 cardoe 1.1 Author: Robin Hugh Johnson <robbat2@gentoo.org>,
6     Status: Draft
7     Type: Standards Track
8     Content-Type: text/x-rst
9     Requires: GLEP44, GLEP60
10     Created: October 2006
11     Updated: November 2007, June 2008, July 2008, October 2008
12     Post-History: ...
13    
14     ========
15     Abstract
16     ========
17     MetaManifest provides a means of verifiable distribution from Gentoo
18     Infrastructure to a user system, while data is conveyed over completely
19     untrusted networks and system, by extending the Manifest2 specification,
20     and adding a top-level Manifest file, with support for other nested
21     Manifests.
22    
23     ==========
24     Motivation
25     ==========
26     As part of a comprehensive security plan, we need a way to prove that
27     something originating from Gentoo as an organization (read Gentoo-owned
28     hardware, run by infrastructure), has not been tampered with. This
29     allows the usage of third-party rsync mirrors, without worrying that
30     they have modified something critical (e.g. eclasses, which are still
31     unsigned).
32    
33     Securing the untrusted distribution is one of the easier tasks in the
34     security plan - in short, all that is required is having a hash of every
35     item in the tree, and signing that hash to prove it came from Gentoo.
36    
37     Ironically we have a hashed and signed distribution (it's just not used
38     by most users, due to it's drawbacks): Our tree snapshot tarballs have
39     hashes and signatures.
40    
41     So now we want to add the same verification to our material that is
42     distributed by rsync. We already provide hashes of subsets of the tree -
43     our Manifests protect individual packages. However metadata, eclasses
44     and profiles are not protected at this time. The directories of
45     packages and distfiles are NOT covered by this, as they are not
46     distributed by rsync.
47    
48     This portion of the tree-signing work provides only the following
49     guarantee: A user can prove that the tree from the Gentoo infrastructure
50     has not been tampered with since leaving the Gentoo infrastructure.
51     No other guarantees, either implicit or explicit are made.
52    
53     Additionally, distributing a set of the most recent MetaManifests from a
54     trusted source allows validation of trees that come from community
55     mirrors, and allows detection of all cases of malicious mirrors (either
56     by deliberate delay, replay [C08a, C08b] or alteration).
57    
58     =============
59     Specification
60     =============
61     For lack of a better name, the following solution should be known as the
62     MetaManifest. Those responsible for the name have already been sacked.
63    
64     MetaManifest basically contains hashes of every file in the tree, either
65     directly or indirectly. The direct case applies to ANY file that does
66     not appear in an existing Manifest file (e.g. eclasses, Manifest files
67     themselves). The indirect case is covered by the CONTENTS of existing
68     Manifest files. If the Manifest itself is correct, we know that by
69     tracking the hash of the Manifest, we can be assured that the contents
70     are protected.
71    
72     In the following, the MetaManifest file is a file named 'Manifest',
73     located at the root of a repository.
74    
75     ---------------------------------------------
76     Procedure for creating the MetaManifest file:
77     ---------------------------------------------
78     1. Start at the root of the Gentoo Portage tree (gentoo-x86, although
79     this procedure applies to overlays as well).
80    
81     2. Initialize two unordered sets: COVERED, ALL.
82 robbat2 1.2
83     1. 'ALL' will contain every file in the tree.
84     2. 'COVERED' will contain every file that is mentioned in an existing
85 cardoe 1.1 Manifest2.
86    
87     3. Traverse the tree, depth-first.
88 robbat2 1.2
89     1. At the top level only, ignore the following directories: distfiles,
90 cardoe 1.1 packages, local
91 robbat2 1.2 2. If a directory contains a Manifest file, extract all relevant local
92 cardoe 1.1 files from it (presently: AUX, MISC, EBUILD; but should follow the
93     evolution of Manifest2 entry types per [GLEPxx+5]), and place them
94     into the COVERED set.
95 robbat2 1.2 3. Recursively add every file in the directory to the ALL set,
96 cardoe 1.1 pursusant to the exclusion list as mentioned in [GLEPxx+5].
97    
98     4. Produce a new set, UNCOVERED, as the set-difference (ALL)-(COVERED).
99     This is every item that is not covered by another Manifest, or part
100     of an exclusion list.
101    
102     5. If an existing MetaManifest file is present, remove it.
103    
104     6. For each file in UNCOVERED, assign a Manifest2 type, produce the
105     hashes, and add with the filetype to the MetaManifest file.
106    
107     7. For unique identification of the MetaManifest, a header line should
108     be included, using the exact contents of the metadata/timestamp.x
109     file, so that a MetaManifest may be tied back to a tree as
110     distributed by the rsync mirror system. The string of
111     'metadata/timestamp.x' should be included to identify this revision
112     of MetaManifest generation. Eg:
113     "Timestamp: metadata/timestamp.x: 1215722461 Thu Jul 10 20:41:01 2008 UTC"
114     The package manager MUST not use the identifying string as a filename.
115    
116     8. The MetaManifest must ultimately be GnuPG-signed.
117 robbat2 1.2
118     1. For the initial implementation, the same key as used for snapshot
119 cardoe 1.1 tarball signing is sufficient.
120 robbat2 1.2 2. For the future, the key used for fully automated signing by infra
121 cardoe 1.1 should not be on the same keyring as developer keys. See [GLEPxx+3
122     for further notes].
123    
124     The above does not conflict the proposal contained in GLEP33, which
125     restructure eclasses to include subdirectories and Manifest files, as
126     the Manifest rules above still provide indirect verification for all
127     files after the GLEP33 restructuring if it comes to pass.
128    
129     If other Manifests are added (such as per-category, or protecting
130     versioned eclases), the size of the MetaManifest will be greatly
131     reduced, and this specification was written with such a possible future
132     addition in mind.
133    
134     MetaManifest generation will take place as part of the existing process
135     by infrastructure that takes the contents of CVS and prepares it for
136     distribution via rsync, which includes generating metadata. In-tree
137     Manifest files are not checked at this point, as they are assumed to be
138     correct.
139    
140     --------------------------------------------------------
141     Verification of one or more items from the MetaManifest:
142     --------------------------------------------------------
143     There are two times that this may happen: firstly, immediately after the
144     rsync has completed - this has the advantage that the kernel file cache
145     is hot, and checking the entire tree can be accomplished quickly.
146     Secondly, the MetaManifest should be checked during installation of a
147     package.
148    
149     ----------------------------------------------------
150     Procedure for verifying an item in the MetaManifest:
151     ----------------------------------------------------
152     In the following, I've used term 'M2-verify' to note following the hash
153     verification procedures as defined by the Manifest2 format - which
154     compromise checking the file length, and that the hashes match. Which
155     filetypes may be ignored on missing is discussed in [GLEPxx+5].
156    
157     1. Check the GnuPG signature on the MetaManifest against the keyring of
158     automated Gentoo keys. See [GLEPxx+3] for full details regarding
159     verification of GnuPG signatures.
160 robbat2 1.2 1. Abort if the signature check fails.
161 cardoe 1.1
162     2. Check the Timestamp header. If it is significently out of date
163     compared to the local clock or a trusted source, halt or require
164     manual intervention from the user.
165    
166     3. For a verification of the tree following an rsync:
167 robbat2 1.2
168     1. Build a set 'ALL' of every file covered by the rsync. (exclude
169 cardoe 1.1 distfiles/, packages/, local/)
170 robbat2 1.2 2. M2-verify every entry in the MetaManifest, descending into inferior
171 cardoe 1.1 Manifests as needed. Place the relative path of every checked item
172     into a set 'COVERED'.
173 robbat2 1.2 3. Construct the set 'UNCOVERED' by set-difference between the ALL and
174 cardoe 1.1 COVERED sets.
175 robbat2 1.2 4. For each file in the UNCOVERED set, assign a Manifest2 filetype.
176     5. If the filetype for any file in the UNCOVERED set requires a halt
177 cardoe 1.1 on error, abort and display a suitable error.
178 robbat2 1.2 6. Completed verification
179 cardoe 1.1
180     4. If checking at the installation of a package:
181 robbat2 1.2
182     1. M2-verify the entry in MetaManifest for the Manifest
183     2. M2-verify all relevant metadata/ contents if metadata/ is being
184 cardoe 1.1 used in any way (optionally done before dependancy checking).
185 robbat2 1.2 3. M2-verifying the contents of the Manifest.
186     4. Perform M2-verification of all eclasses and profiles used (both
187 cardoe 1.1 directly and indirectly) by the ebuild.
188    
189     Notes:
190     ======
191     1. For initial implementations, it is acceptable to check EVERY item in
192     the eclass and profiles directory, rather than tracking the exact
193     files used by every eclass (see note #2). Later implementations
194     should strive to only verify individual eclasses and profiles as
195     needed.
196     2. Tracking of exact files is of specific significance to the libtool
197     eclass, as it stores patches under eclass/ELT-patches, and as such
198     that would not be picked up by any tracing of the inherit function.
199     This may be alleviated by a later eclass and ebuild variable that
200     explicitly declares what files from the tree are used by a package.
201    
202     ====================
203     Implementation Notes
204     ====================
205     For this portion of the tree-signing work, no actions are required of
206     the individual Gentoo developers. They will continue to develop and
207     commit as they do presently, and the MetaManifest is added by
208     Infrastructure during the tree generation process, and distributed to
209     users.
210    
211     --------------------------------------------
212     MetaManifest and the new Manifest2 filetypes
213     --------------------------------------------
214     While [GLEPxx+5] describes the addition of new filetypes, these are NOT
215     needed for implementation of the MetaManifest proposal. Without the new
216     filetypes, all entries in the MetaManifest would be of type 'MISC'.
217    
218     ----------------------------------------------------
219     Timestamps & Additional distribution of MetaManifest
220     ----------------------------------------------------
221     As discussed by [C08a,C08b], malicious third-party mirrors may use the
222     principles of exclusion and replay to deny an update to clients, while
223     at the same time recording the identity of clients to attack.
224    
225     This should be guarded against by including a timestamp in the header of
226     the MetaManifest, as well as distributing the latest MetaManifests by a
227     trusted channel.
228    
229     On all rsync mirrors directly maintained by the Gentoo infrastructure,
230     and not on community mirrors, there should be a new module
231     'gentoo-portage-metamanifests'. Within this module, all MetaManifests
232     for a recent time frame (eg one week) should be kept, named as
233     "MetaManifest.$TS", where $TS is the timestamp from inside the file.
234     The most recent MetaManifest should always be symlinked as
235     MetaManifest.current. The possibility of serving the recent
236     MetaManifests via HTTPS should also be explored to mitigate MitM
237     attacks.
238    
239     The package manager should obtain MetaManifest.current and use it to
240     decide is the tree is too out of date per operation #2 of the
241     verification process. The decision about freshness should be a
242     user-configuration setting, with the ability to override.
243    
244     --------------------------------
245     MetaManifest size considerations
246     --------------------------------
247     With only two levels of Manifests (per-package and top-level), every
248     rsync will cause a lot of traffic transfering the modified top-level
249     MetaManifest. To reduce this, per-category Manifests are strongly
250     recommended. Alternatively, if the distribution method efficently
251     handles small patch-like changes in an existing file, using an
252     uncompressed MetaManifest may be acceptable (this would primarily be
253     distributed version control systems). Other suggestions in reducing this
254     traffic are welcomed.
255    
256     =======================
257     Backwards Compatibility
258     =======================
259     - There are no backwards compatibility issues, as old versions of
260     Portage do not look for a Manifest file at the top level of the tree.
261     - Manifest2-aware versions of Portage ignore all entries that they are
262     not certain how to handle. Enabling headers and PGP signing to be
263     conducted easily.
264    
265     ======
266     Thanks
267     ======
268     I'd like to thank the following people for input on this GLEP.
269     - Patrick Lauer (patrick): Prodding me to get all of the tree-signing
270     work finished, and helping to edit.
271     - Ciaran McCreesh (ciaranm): Paludis Manifest2
272     - Brian Harring (ferringb): pkgcore Manifest2
273     - Marius Mauch (genone) & Zac Medico (zmedico): Portage Manifest2
274     - Ned Ludd (solar) - Security concept review
275    
276     ==========
277     References
278     ==========
279    
280     [C08a] Cappos, J et al. (2008). "Package Management Security".
281     University of Arizona Technical Report TR08-02. Available online
282     from: ftp://ftp.cs.arizona.edu/reports/2008/TR08-02.pdf
283     [C08b] Cappos, J et al. (2008). "Attacks on Package Managers"
284     Available online at:
285     http://www.cs.arizona.edu/people/justin/packagemanagersecurity/
286    
287     =========
288     Copyright
289     =========
290     Copyright (c) 2006 by Robin Hugh Johnson. This material may be
291     distributed only subject to the terms and conditions set forth in the
292     Open Publication License, v1.0.
293    
294     vim: tw=72 ts=2 expandtab:

  ViewVC Help
Powered by ViewVC 1.1.20