Contents of /xml/htdocs/proj/en/glep/glep-0058.txt

Parent Directory Parent Directory | Revision Log Revision Log

Revision 1.1 - (show annotations) (download)
Tue Oct 21 23:30:47 2008 UTC (9 years, 9 months ago) by cardoe
Branch: MAIN
File MIME type: text/plain
add Robin's tree signing gleps. They still need lots of editing love (some won't glep-ify) but at least they're here and have glep #s reserved

1 GLEP: 58
2 Title: Security of distribution of Gentoo software - Infrastructure to User distribution - MetaManifest
3 Version: $Revision: 1.24 $
4 Last-Modified: $Date: 2008/10/09 23:23:12 $
5 Author: Robin Hugh Johnson <robbat2@gentoo.org>,
6 Status: Draft
7 Type: Standards Track
8 Content-Type: text/x-rst
9 Requires: GLEP44, GLEP60
10 Created: October 2006
11 Updated: November 2007, June 2008, July 2008, October 2008
12 Post-History: ...
14 ========
15 Abstract
16 ========
17 MetaManifest provides a means of verifiable distribution from Gentoo
18 Infrastructure to a user system, while data is conveyed over completely
19 untrusted networks and system, by extending the Manifest2 specification,
20 and adding a top-level Manifest file, with support for other nested
21 Manifests.
23 ==========
24 Motivation
25 ==========
26 As part of a comprehensive security plan, we need a way to prove that
27 something originating from Gentoo as an organization (read Gentoo-owned
28 hardware, run by infrastructure), has not been tampered with. This
29 allows the usage of third-party rsync mirrors, without worrying that
30 they have modified something critical (e.g. eclasses, which are still
31 unsigned).
33 Securing the untrusted distribution is one of the easier tasks in the
34 security plan - in short, all that is required is having a hash of every
35 item in the tree, and signing that hash to prove it came from Gentoo.
37 Ironically we have a hashed and signed distribution (it's just not used
38 by most users, due to it's drawbacks): Our tree snapshot tarballs have
39 hashes and signatures.
41 So now we want to add the same verification to our material that is
42 distributed by rsync. We already provide hashes of subsets of the tree -
43 our Manifests protect individual packages. However metadata, eclasses
44 and profiles are not protected at this time. The directories of
45 packages and distfiles are NOT covered by this, as they are not
46 distributed by rsync.
48 This portion of the tree-signing work provides only the following
49 guarantee: A user can prove that the tree from the Gentoo infrastructure
50 has not been tampered with since leaving the Gentoo infrastructure.
51 No other guarantees, either implicit or explicit are made.
53 Additionally, distributing a set of the most recent MetaManifests from a
54 trusted source allows validation of trees that come from community
55 mirrors, and allows detection of all cases of malicious mirrors (either
56 by deliberate delay, replay [C08a, C08b] or alteration).
58 =============
59 Specification
60 =============
61 For lack of a better name, the following solution should be known as the
62 MetaManifest. Those responsible for the name have already been sacked.
64 MetaManifest basically contains hashes of every file in the tree, either
65 directly or indirectly. The direct case applies to ANY file that does
66 not appear in an existing Manifest file (e.g. eclasses, Manifest files
67 themselves). The indirect case is covered by the CONTENTS of existing
68 Manifest files. If the Manifest itself is correct, we know that by
69 tracking the hash of the Manifest, we can be assured that the contents
70 are protected.
72 In the following, the MetaManifest file is a file named 'Manifest',
73 located at the root of a repository.
75 ---------------------------------------------
76 Procedure for creating the MetaManifest file:
77 ---------------------------------------------
78 1. Start at the root of the Gentoo Portage tree (gentoo-x86, although
79 this procedure applies to overlays as well).
81 2. Initialize two unordered sets: COVERED, ALL.
82 2.1. 'ALL' will contain every file in the tree.
83 2.2. 'COVERED' will contain every file that is mentioned in an existing
84 Manifest2.
86 3. Traverse the tree, depth-first.
87 3.1. At the top level only, ignore the following directories: distfiles,
88 packages, local
89 3.2. If a directory contains a Manifest file, extract all relevant local
90 files from it (presently: AUX, MISC, EBUILD; but should follow the
91 evolution of Manifest2 entry types per [GLEPxx+5]), and place them
92 into the COVERED set.
93 3.3. Recursively add every file in the directory to the ALL set,
94 pursusant to the exclusion list as mentioned in [GLEPxx+5].
96 4. Produce a new set, UNCOVERED, as the set-difference (ALL)-(COVERED).
97 This is every item that is not covered by another Manifest, or part
98 of an exclusion list.
100 5. If an existing MetaManifest file is present, remove it.
102 6. For each file in UNCOVERED, assign a Manifest2 type, produce the
103 hashes, and add with the filetype to the MetaManifest file.
105 7. For unique identification of the MetaManifest, a header line should
106 be included, using the exact contents of the metadata/timestamp.x
107 file, so that a MetaManifest may be tied back to a tree as
108 distributed by the rsync mirror system. The string of
109 'metadata/timestamp.x' should be included to identify this revision
110 of MetaManifest generation. Eg:
111 "Timestamp: metadata/timestamp.x: 1215722461 Thu Jul 10 20:41:01 2008 UTC"
112 The package manager MUST not use the identifying string as a filename.
114 8. The MetaManifest must ultimately be GnuPG-signed.
115 8.1. For the initial implementation, the same key as used for snapshot
116 tarball signing is sufficient.
117 8.2. For the future, the key used for fully automated signing by infra
118 should not be on the same keyring as developer keys. See [GLEPxx+3
119 for further notes].
121 The above does not conflict the proposal contained in GLEP33, which
122 restructure eclasses to include subdirectories and Manifest files, as
123 the Manifest rules above still provide indirect verification for all
124 files after the GLEP33 restructuring if it comes to pass.
126 If other Manifests are added (such as per-category, or protecting
127 versioned eclases), the size of the MetaManifest will be greatly
128 reduced, and this specification was written with such a possible future
129 addition in mind.
131 MetaManifest generation will take place as part of the existing process
132 by infrastructure that takes the contents of CVS and prepares it for
133 distribution via rsync, which includes generating metadata. In-tree
134 Manifest files are not checked at this point, as they are assumed to be
135 correct.
137 --------------------------------------------------------
138 Verification of one or more items from the MetaManifest:
139 --------------------------------------------------------
140 There are two times that this may happen: firstly, immediately after the
141 rsync has completed - this has the advantage that the kernel file cache
142 is hot, and checking the entire tree can be accomplished quickly.
143 Secondly, the MetaManifest should be checked during installation of a
144 package.
146 ----------------------------------------------------
147 Procedure for verifying an item in the MetaManifest:
148 ----------------------------------------------------
149 In the following, I've used term 'M2-verify' to note following the hash
150 verification procedures as defined by the Manifest2 format - which
151 compromise checking the file length, and that the hashes match. Which
152 filetypes may be ignored on missing is discussed in [GLEPxx+5].
154 1. Check the GnuPG signature on the MetaManifest against the keyring of
155 automated Gentoo keys. See [GLEPxx+3] for full details regarding
156 verification of GnuPG signatures.
157 1.1. Abort if the signature check fails.
159 2. Check the Timestamp header. If it is significently out of date
160 compared to the local clock or a trusted source, halt or require
161 manual intervention from the user.
163 3. For a verification of the tree following an rsync:
164 3.1. Build a set 'ALL' of every file covered by the rsync. (exclude
165 distfiles/, packages/, local/)
166 3.2. M2-verify every entry in the MetaManifest, descending into inferior
167 Manifests as needed. Place the relative path of every checked item
168 into a set 'COVERED'.
169 3.3. Construct the set 'UNCOVERED' by set-difference between the ALL and
170 COVERED sets.
171 3.4. For each file in the UNCOVERED set, assign a Manifest2 filetype.
172 3.5. If the filetype for any file in the UNCOVERED set requires a halt
173 on error, abort and display a suitable error.
174 3.6. Completed verification
176 4. If checking at the installation of a package:
177 4.1. M2-verify the entry in MetaManifest for the Manifest
178 4.2. M2-verify all relevant metadata/ contents if metadata/ is being
179 used in any way (optionally done before dependancy checking).
180 4.3. M2-verifying the contents of the Manifest.
181 4.4. Perform M2-verification of all eclasses and profiles used (both
182 directly and indirectly) by the ebuild.
184 Notes:
185 ======
186 1. For initial implementations, it is acceptable to check EVERY item in
187 the eclass and profiles directory, rather than tracking the exact
188 files used by every eclass (see note #2). Later implementations
189 should strive to only verify individual eclasses and profiles as
190 needed.
191 2. Tracking of exact files is of specific significance to the libtool
192 eclass, as it stores patches under eclass/ELT-patches, and as such
193 that would not be picked up by any tracing of the inherit function.
194 This may be alleviated by a later eclass and ebuild variable that
195 explicitly declares what files from the tree are used by a package.
197 ====================
198 Implementation Notes
199 ====================
200 For this portion of the tree-signing work, no actions are required of
201 the individual Gentoo developers. They will continue to develop and
202 commit as they do presently, and the MetaManifest is added by
203 Infrastructure during the tree generation process, and distributed to
204 users.
206 --------------------------------------------
207 MetaManifest and the new Manifest2 filetypes
208 --------------------------------------------
209 While [GLEPxx+5] describes the addition of new filetypes, these are NOT
210 needed for implementation of the MetaManifest proposal. Without the new
211 filetypes, all entries in the MetaManifest would be of type 'MISC'.
213 ----------------------------------------------------
214 Timestamps & Additional distribution of MetaManifest
215 ----------------------------------------------------
216 As discussed by [C08a,C08b], malicious third-party mirrors may use the
217 principles of exclusion and replay to deny an update to clients, while
218 at the same time recording the identity of clients to attack.
220 This should be guarded against by including a timestamp in the header of
221 the MetaManifest, as well as distributing the latest MetaManifests by a
222 trusted channel.
224 On all rsync mirrors directly maintained by the Gentoo infrastructure,
225 and not on community mirrors, there should be a new module
226 'gentoo-portage-metamanifests'. Within this module, all MetaManifests
227 for a recent time frame (eg one week) should be kept, named as
228 "MetaManifest.$TS", where $TS is the timestamp from inside the file.
229 The most recent MetaManifest should always be symlinked as
230 MetaManifest.current. The possibility of serving the recent
231 MetaManifests via HTTPS should also be explored to mitigate MitM
232 attacks.
234 The package manager should obtain MetaManifest.current and use it to
235 decide is the tree is too out of date per operation #2 of the
236 verification process. The decision about freshness should be a
237 user-configuration setting, with the ability to override.
239 --------------------------------
240 MetaManifest size considerations
241 --------------------------------
242 With only two levels of Manifests (per-package and top-level), every
243 rsync will cause a lot of traffic transfering the modified top-level
244 MetaManifest. To reduce this, per-category Manifests are strongly
245 recommended. Alternatively, if the distribution method efficently
246 handles small patch-like changes in an existing file, using an
247 uncompressed MetaManifest may be acceptable (this would primarily be
248 distributed version control systems). Other suggestions in reducing this
249 traffic are welcomed.
251 =======================
252 Backwards Compatibility
253 =======================
254 - There are no backwards compatibility issues, as old versions of
255 Portage do not look for a Manifest file at the top level of the tree.
256 - Manifest2-aware versions of Portage ignore all entries that they are
257 not certain how to handle. Enabling headers and PGP signing to be
258 conducted easily.
260 ======
261 Thanks
262 ======
263 I'd like to thank the following people for input on this GLEP.
264 - Patrick Lauer (patrick): Prodding me to get all of the tree-signing
265 work finished, and helping to edit.
266 - Ciaran McCreesh (ciaranm): Paludis Manifest2
267 - Brian Harring (ferringb): pkgcore Manifest2
268 - Marius Mauch (genone) & Zac Medico (zmedico): Portage Manifest2
269 - Ned Ludd (solar) - Security concept review
271 ==========
272 References
273 ==========
275 [C08a] Cappos, J et al. (2008). "Package Management Security".
276 University of Arizona Technical Report TR08-02. Available online
277 from: ftp://ftp.cs.arizona.edu/reports/2008/TR08-02.pdf
278 [C08b] Cappos, J et al. (2008). "Attacks on Package Managers"
279 Available online at:
280 http://www.cs.arizona.edu/people/justin/packagemanagersecurity/
282 =========
283 Copyright
284 =========
285 Copyright (c) 2006 by Robin Hugh Johnson. This material may be
286 distributed only subject to the terms and conditions set forth in the
287 Open Publication License, v1.0.
289 vim: tw=72 ts=2 expandtab:

  ViewVC Help
Powered by ViewVC 1.1.20