1 |
GLEP: 58 |
2 |
Title: Security of distribution of Gentoo software - Infrastructure to User distribution - MetaManifest |
3 |
Version: $Revision: 1.7 $ |
4 |
Last-Modified: $Date: 2010/01/31 07:53:30 $ |
5 |
Author: Robin Hugh Johnson <robbat2@gentoo.org>, |
6 |
Status: Draft |
7 |
Type: Standards Track |
8 |
Content-Type: text/x-rst |
9 |
Requires: 44, 60 |
10 |
Created: October 2006 |
11 |
Updated: November 2007, June 2008, July 2008, October 2008, January 2010 |
12 |
Post-History: December 2009, January 2010 |
13 |
|
14 |
======== |
15 |
Abstract |
16 |
======== |
17 |
MetaManifest provides a means of verifiable distribution from Gentoo |
18 |
Infrastructure to a user system, while data is conveyed over completely |
19 |
untrusted networks and system, by extending the Manifest2 specification, |
20 |
and adding a top-level Manifest file, with support for other nested |
21 |
Manifests. |
22 |
|
23 |
========== |
24 |
Motivation |
25 |
========== |
26 |
As part of a comprehensive security plan, we need a way to prove that |
27 |
something originating from Gentoo as an organization (read Gentoo-owned |
28 |
hardware, run by infrastructure), has not been tampered with. This |
29 |
allows the usage of third-party rsync mirrors, without worrying that |
30 |
they have modified something critical (e.g. eclasses, which are still |
31 |
unsigned). |
32 |
|
33 |
Securing the untrusted distribution is one of the easier tasks in the |
34 |
security plan - in short, all that is required is having a hash of every |
35 |
item in the tree, and signing that hash to prove it came from Gentoo. |
36 |
|
37 |
Ironically we have a hashed and signed distribution (it's just not used |
38 |
by most users, due to it's drawbacks): Our tree snapshot tarballs have |
39 |
hashes and signatures. |
40 |
|
41 |
So now we want to add the same verification to our material that is |
42 |
distributed by rsync. We already provide hashes of subsets of the tree - |
43 |
our Manifests protect individual packages. However metadata, eclasses |
44 |
and profiles are not protected at this time. The directories of |
45 |
packages and distfiles are NOT covered by this, as they are not |
46 |
distributed by rsync. |
47 |
|
48 |
This portion of the tree-signing work provides only the following |
49 |
guarantee: A user can prove that the tree from the Gentoo infrastructure |
50 |
has not been tampered with since leaving the Gentoo infrastructure. |
51 |
No other guarantees, either implicit or explicit are made. |
52 |
|
53 |
Additionally, distributing a set of the most recent MetaManifests from a |
54 |
trusted source allows validation of trees that come from community |
55 |
mirrors, and allows detection of all cases of malicious mirrors (either |
56 |
by deliberate delay, replay [C08a, C08b] or alteration). |
57 |
|
58 |
============= |
59 |
Specification |
60 |
============= |
61 |
For lack of a better name, the following solution should be known as the |
62 |
MetaManifest. Those responsible for the name have already been sacked. |
63 |
|
64 |
MetaManifest basically contains hashes of every file in the tree, either |
65 |
directly or indirectly. The direct case applies to ANY file that does |
66 |
not appear in an existing Manifest file (e.g. eclasses, Manifest files |
67 |
themselves). The indirect case is covered by the CONTENTS of existing |
68 |
Manifest files. If the Manifest itself is correct, we know that by |
69 |
tracking the hash of the Manifest, we can be assured that the contents |
70 |
are protected. |
71 |
|
72 |
In the following, the MetaManifest file is a file named 'Manifest', |
73 |
located at the root of a repository. |
74 |
|
75 |
--------------------------------------------- |
76 |
Procedure for creating the MetaManifest file: |
77 |
--------------------------------------------- |
78 |
Summary: |
79 |
======== |
80 |
The objective of creating the MetaManifest file(s) is to ensure that |
81 |
every single file in the tree occurs in at least one Manifest. |
82 |
|
83 |
Process: |
84 |
======== |
85 |
1. Start at the root of the Gentoo Portage tree (gentoo-x86, although |
86 |
this procedure applies to overlays as well). |
87 |
|
88 |
2. Initialize two unordered sets: COVERED, ALL. |
89 |
|
90 |
1. 'ALL' shall contain every file that exists in the present tree. |
91 |
2. 'COVERED' shall contain EVERY file that is mentioned in an existing |
92 |
Manifest2. If a file is mentioned in a Manifest2, but does not |
93 |
exist, it must still be included. No files should be excluded. |
94 |
|
95 |
3. Traverse the tree, depth-first. |
96 |
|
97 |
1. At the top level only, ignore the following directories: distfiles, |
98 |
packages, local. |
99 |
2. If a directory contains a Manifest file, extract all relevant local |
100 |
files from it (presently: AUX, MISC, EBUILD; but should follow the |
101 |
evolution of Manifest2 entry types per [#GLEP60]), and place them |
102 |
into the COVERED set. |
103 |
3. Recursively add every file in the directory to the ALL set, |
104 |
pursuant to the exclusion list as mentioned in [#GLEP60]. |
105 |
|
106 |
4. Produce a new set, UNCOVERED, as the set-difference (ALL)-(COVERED). |
107 |
This is every item that is not covered by another Manifest, or part |
108 |
of an exclusion list. |
109 |
|
110 |
5. If an existing MetaManifest file is present, remove it. |
111 |
|
112 |
6. For each file in UNCOVERED, assign a Manifest2 type, produce the |
113 |
hashes, and add with the filetype to the MetaManifest file. |
114 |
|
115 |
7. For unique identification of the MetaManifest, a header line should |
116 |
be included, using the exact contents of the metadata/timestamp.x |
117 |
file, so that a MetaManifest may be tied back to a tree as |
118 |
distributed by the rsync mirror system. The string of |
119 |
'metadata/timestamp.x' should be included to identify this revision |
120 |
of MetaManifest generation. e.g.: |
121 |
"Timestamp: metadata/timestamp.x: 1215722461 Thu Jul 10 20:41:01 2008 UTC" |
122 |
The package manager MUST not use the identifying string as a filename. |
123 |
|
124 |
8. The MetaManifest must ultimately be GnuPG-signed. |
125 |
|
126 |
1. For the initial implementation, the same key as used for snapshot |
127 |
tarball signing is sufficient. |
128 |
2. For the future, the key used for fully automated signing by infra |
129 |
should not be on the same keyring as developer keys. See [#GLEPxx+3 |
130 |
for further notes]. |
131 |
|
132 |
Notes: |
133 |
====== |
134 |
The above does not conflict the proposal contained in GLEP33, which |
135 |
restructure eclasses to include subdirectories and Manifest files, as |
136 |
the Manifest rules above still provide indirect verification for all |
137 |
files after the GLEP33 restructuring if it comes to pass. |
138 |
|
139 |
Additional levels of Manifests are required, such as per-category, and |
140 |
in the eclasses, profiles and metadata directories. This ensures that a |
141 |
change to a singular file causes the smallest possible overall change in |
142 |
the Manifests as propagated. Creation of the additional levels of |
143 |
Manifests uses the same process as described above, simply starting at a |
144 |
different root point. |
145 |
|
146 |
MetaManifest generation will take place as part of the existing process |
147 |
by infrastructure that takes the contents of CVS and prepares it for |
148 |
distribution via rsync, which includes generating metadata. In-tree |
149 |
Manifest files are not validated at this point, as they are assumed to |
150 |
be correct. |
151 |
|
152 |
-------------------------------------------------------- |
153 |
Verification of one or more items from the MetaManifest: |
154 |
-------------------------------------------------------- |
155 |
There are two times that this may happen: firstly, immediately after the |
156 |
rsync has completed - this has the advantage that the kernel file cache |
157 |
is hot, and checking the entire tree can be accomplished quickly. |
158 |
Secondly, the MetaManifest should be checked during installation of a |
159 |
package. |
160 |
|
161 |
---------------------------------------------------- |
162 |
Procedure for verifying an item in the MetaManifest: |
163 |
---------------------------------------------------- |
164 |
In the following, I've used term 'M2-verify' to note following the hash |
165 |
verification procedures as defined by the Manifest2 format - which |
166 |
compromise checking the file length, and that the hashes match. Which |
167 |
filetypes may be ignored on missing is discussed in [#GLEP60]. |
168 |
|
169 |
1. Check the GnuPG signature on the MetaManifest against the keyring of |
170 |
automated Gentoo keys. See [#GLEPxx+3] for full details regarding |
171 |
verification of GnuPG signatures. |
172 |
1. Abort if the signature check fails. |
173 |
|
174 |
2. Check the Timestamp header. If it is significantly out of date |
175 |
compared to the local clock or a trusted source, halt or require |
176 |
manual intervention from the user. |
177 |
|
178 |
3. For a verification of the tree following an rsync: |
179 |
|
180 |
1. Build a set 'ALL' of every file covered by the rsync. (exclude |
181 |
distfiles/, packages/, local/) |
182 |
2. M2-verify every entry in the MetaManifest, descending into inferior |
183 |
Manifests as needed. Place the relative path of every checked item |
184 |
into a set 'COVERED'. |
185 |
3. Construct the set 'UNCOVERED' by set-difference between the ALL and |
186 |
COVERED sets. |
187 |
4. For each file in the UNCOVERED set, assign a Manifest2 filetype. |
188 |
5. If the filetype for any file in the UNCOVERED set requires a halt |
189 |
on error, abort and display a suitable error. |
190 |
6. Completed verification |
191 |
|
192 |
4. If checking at the installation of a package: |
193 |
|
194 |
1. M2-verify the entry in MetaManifest for the Manifest |
195 |
2. M2-verify all relevant metadata/ contents if metadata/ is being |
196 |
used in any way (optionally done before dependency checking). |
197 |
3. M2-verifying the contents of the Manifest. |
198 |
4. Perform M2-verification of all eclasses and profiles used (both |
199 |
directly and indirectly) by the ebuild. |
200 |
|
201 |
Notes: |
202 |
====== |
203 |
1. For initial implementations, it is acceptable to check EVERY item in |
204 |
the eclass and profiles directory, rather than tracking the exact |
205 |
files used by every eclass (see note #2). Later implementations |
206 |
should strive to only verify individual eclasses and profiles as |
207 |
needed. |
208 |
2. Tracking of exact files is of specific significance to the libtool |
209 |
eclass, as it stores patches under eclass/ELT-patches, and as such |
210 |
that would not be picked up by any tracing of the inherit function. |
211 |
This may be alleviated by a later eclass and ebuild variable that |
212 |
explicitly declares what files from the tree are used by a package. |
213 |
|
214 |
==================== |
215 |
Implementation Notes |
216 |
==================== |
217 |
For this portion of the tree-signing work, no actions are required of |
218 |
the individual Gentoo developers. They will continue to develop and |
219 |
commit as they do presently, and the MetaManifest is added by |
220 |
Infrastructure during the tree generation process, and distributed to |
221 |
users. |
222 |
|
223 |
Any scripts generating Manifests and the MetaManifest may find it useful |
224 |
to generate multiple levels of Manifests in parallel, and this is |
225 |
explicitly permitted, provided that every file in the tree is covered by |
226 |
at least one Manifest or the MetaManifest file. The uppermost |
227 |
Manifest (MetaManifest) is the only item that does not occur in any |
228 |
other Manifest file, but is instead GPG-signed to enable it's |
229 |
validation. |
230 |
|
231 |
-------------------------------------------- |
232 |
MetaManifest and the new Manifest2 filetypes |
233 |
-------------------------------------------- |
234 |
While [#GLEP60] describes the addition of new filetypes, these are NOT |
235 |
needed for implementation of the MetaManifest proposal. Without the new |
236 |
filetypes, all entries in the MetaManifest would be of type 'MISC'. |
237 |
|
238 |
---------------------------------------------------- |
239 |
Timestamps & Additional distribution of MetaManifest |
240 |
---------------------------------------------------- |
241 |
As discussed by [C08a,C08b], malicious third-party mirrors may use the |
242 |
principles of exclusion and replay to deny an update to clients, while |
243 |
at the same time recording the identity of clients to attack. |
244 |
|
245 |
This should be guarded against by including a timestamp in the header of |
246 |
the MetaManifest, as well as distributing the latest MetaManifests by a |
247 |
trusted channel. |
248 |
|
249 |
On all rsync mirrors directly maintained by the Gentoo infrastructure, |
250 |
and not on community mirrors, there should be a new module |
251 |
'gentoo-portage-metamanifests'. Within this module, all MetaManifests |
252 |
for a recent time frame (e.g. one week) should be kept, named as |
253 |
"MetaManifest.$TS", where $TS is the timestamp from inside the file. |
254 |
The most recent MetaManifest should always be symlinked as |
255 |
MetaManifest.current. The possibility of serving the recent |
256 |
MetaManifests via HTTPS should also be explored to mitigate |
257 |
man-in-the-middle attacks. |
258 |
|
259 |
The package manager should obtain MetaManifest.current and use it to |
260 |
decide is the tree is too out of date per operation #2 of the |
261 |
verification process. The decision about freshness should be a |
262 |
user-configuration setting, with the ability to override. |
263 |
|
264 |
-------------------------------- |
265 |
MetaManifest size considerations |
266 |
-------------------------------- |
267 |
With only two levels of Manifests (per-package and top-level), every |
268 |
rsync will cause a lot of traffic transferring the modified top-level |
269 |
MetaManifest. To reduce this, first-level directory Manifests are |
270 |
required. Alternatively, if the distribution method efficiently handles |
271 |
small patch-like changes in an existing file, using an uncompressed |
272 |
MetaManifest may be acceptable (this would primarily be distributed |
273 |
version control systems). Other suggestions in reducing this traffic are |
274 |
welcomed. |
275 |
|
276 |
======================= |
277 |
Backwards Compatibility |
278 |
======================= |
279 |
- There are no backwards compatibility issues, as old versions of |
280 |
Portage do not look for a Manifest file at the top level of the tree. |
281 |
- Manifest2-aware versions of Portage ignore all entries that they are |
282 |
not certain how to handle. Enabling headers and PGP signing to be |
283 |
conducted easily. |
284 |
|
285 |
====== |
286 |
Thanks |
287 |
====== |
288 |
I'd like to thank the following people for input on this GLEP. |
289 |
|
290 |
- Patrick Lauer (patrick): Prodding me to get all of the tree-signing |
291 |
work finished, and helping to edit. |
292 |
- Ciaran McCreesh (ciaranm): Paludis Manifest2 |
293 |
- Brian Harring (ferringb): pkgcore Manifest2 |
294 |
- Marius Mauch (genone) & Zac Medico (zmedico): Portage Manifest2 |
295 |
- Ned Ludd (solar) - Security concept review |
296 |
|
297 |
========== |
298 |
References |
299 |
========== |
300 |
|
301 |
[C08a] Cappos, J et al. (2008). "Package Management Security". |
302 |
University of Arizona Technical Report TR08-02. Available online |
303 |
from: ftp://ftp.cs.arizona.edu/reports/2008/TR08-02.pdf |
304 |
[C08b] Cappos, J et al. (2008). "Attacks on Package Managers" |
305 |
Available online at: |
306 |
http://www.cs.arizona.edu/people/justin/packagemanagersecurity/ |
307 |
[#GLEPxx+2] Future GLEP on Developer Process security. |
308 |
[#GLEPxx+3] Future GLEP on GnuPG Policies and Handling. |
309 |
|
310 |
========= |
311 |
Copyright |
312 |
========= |
313 |
Copyright (c) 2006-2010 by Robin Hugh Johnson. This material may be |
314 |
distributed only subject to the terms and conditions set forth in the |
315 |
Open Publication License, v1.0. |
316 |
|
317 |
vim: tw=72 ts=2 expandtab: |