1 | GLEP: 58 |
1 | GLEP: 58 |
2 | Title: Security of distribution of Gentoo software - Infrastructure to User distribution - MetaManifest |
2 | Title: Security of distribution of Gentoo software - Infrastructure to User distribution - MetaManifest |
3 | Version: $Revision: 1.4 $ |
3 | Version: $Revision: 1.10 $ |
4 | Last-Modified: $Date: 2008/10/28 07:45:27 $ |
4 | Last-Modified: $Date: 2010/04/07 21:34:24 $ |
5 | Author: Robin Hugh Johnson <robbat2@gentoo.org>, |
5 | Author: Robin Hugh Johnson <robbat2@gentoo.org>, |
6 | Status: Draft |
6 | Status: Draft |
7 | Type: Standards Track |
7 | Type: Standards Track |
8 | Content-Type: text/x-rst |
8 | Content-Type: text/x-rst |
9 | Requires: 44, 60 |
9 | Requires: 44, 60 |
10 | Created: October 2006 |
10 | Created: October 2006 |
11 | Updated: November 2007, June 2008, July 2008, October 2008 |
11 | Updated: November 2007, June 2008, July 2008, October 2008, January 2010 |
12 | Post-History: |
12 | Post-History: December 2009, January 2010 |
13 | |
13 | |
14 | ======== |
14 | ======== |
15 | Abstract |
15 | Abstract |
16 | ======== |
16 | ======== |
17 | MetaManifest provides a means of verifiable distribution from Gentoo |
17 | MetaManifest provides a means of verifiable distribution from Gentoo |
… | |
… | |
73 | located at the root of a repository. |
73 | located at the root of a repository. |
74 | |
74 | |
75 | --------------------------------------------- |
75 | --------------------------------------------- |
76 | Procedure for creating the MetaManifest file: |
76 | Procedure for creating the MetaManifest file: |
77 | --------------------------------------------- |
77 | --------------------------------------------- |
|
|
78 | Summary: |
|
|
79 | ======== |
|
|
80 | The objective of creating the MetaManifest file(s) is to ensure that |
|
|
81 | every single file in the tree occurs in at least one Manifest. |
|
|
82 | |
|
|
83 | Process: |
|
|
84 | ======== |
78 | 1. Start at the root of the Gentoo Portage tree (gentoo-x86, although |
85 | 1. Start at the root of the Gentoo Portage tree (gentoo-x86, although |
79 | this procedure applies to overlays as well). |
86 | this procedure applies to overlays as well). |
80 | |
87 | |
81 | 2. Initialize two unordered sets: COVERED, ALL. |
88 | 2. Initialize two unordered sets: COVERED, ALL. |
82 | |
89 | |
83 | 1. 'ALL' will contain every file in the tree. |
90 | 1. 'ALL' shall contain every file that exists in the present tree. |
84 | 2. 'COVERED' will contain every file that is mentioned in an existing |
91 | 2. 'COVERED' shall contain EVERY file that is mentioned in an existing |
85 | Manifest2. |
92 | Manifest2. If a file is mentioned in a Manifest2, but does not |
|
|
93 | exist, it must still be included. No files should be excluded. |
86 | |
94 | |
87 | 3. Traverse the tree, depth-first. |
95 | 3. Traverse the tree, depth-first. |
88 | |
96 | |
89 | 1. At the top level only, ignore the following directories: distfiles, |
97 | 1. At the top level only, ignore the following directories: distfiles, |
90 | packages, local |
98 | packages, local. |
91 | 2. If a directory contains a Manifest file, extract all relevant local |
99 | 2. If a directory contains a Manifest file, extract all relevant local |
92 | files from it (presently: AUX, MISC, EBUILD; but should follow the |
100 | files from it (presently: AUX, MISC, EBUILD; but should follow the |
93 | evolution of Manifest2 entry types per [#GLEP60]), and place them |
101 | evolution of Manifest2 entry types per [GLEP60]), and place them |
94 | into the COVERED set. |
102 | into the COVERED set. |
95 | 3. Recursively add every file in the directory to the ALL set, |
103 | 3. Recursively add every file in the directory to the ALL set, |
96 | pursusant to the exclusion list as mentioned in [#GLEP60]. |
104 | pursuant to the exclusion list as mentioned in [GLEP60]. |
97 | |
105 | |
98 | 4. Produce a new set, UNCOVERED, as the set-difference (ALL)-(COVERED). |
106 | 4. Produce a new set, UNCOVERED, as the set-difference (ALL)-(COVERED). |
99 | This is every item that is not covered by another Manifest, or part |
107 | This is every item that is not covered by another Manifest, or part |
100 | of an exclusion list. |
108 | of an exclusion list. |
101 | |
109 | |
… | |
… | |
107 | 7. For unique identification of the MetaManifest, a header line should |
115 | 7. For unique identification of the MetaManifest, a header line should |
108 | be included, using the exact contents of the metadata/timestamp.x |
116 | be included, using the exact contents of the metadata/timestamp.x |
109 | file, so that a MetaManifest may be tied back to a tree as |
117 | file, so that a MetaManifest may be tied back to a tree as |
110 | distributed by the rsync mirror system. The string of |
118 | distributed by the rsync mirror system. The string of |
111 | 'metadata/timestamp.x' should be included to identify this revision |
119 | 'metadata/timestamp.x' should be included to identify this revision |
112 | of MetaManifest generation. Eg: |
120 | of MetaManifest generation. e.g.: |
113 | "Timestamp: metadata/timestamp.x: 1215722461 Thu Jul 10 20:41:01 2008 UTC" |
121 | "Timestamp: metadata/timestamp.x: 1215722461 Thu Jul 10 20:41:01 2008 UTC" |
114 | The package manager MUST not use the identifying string as a filename. |
122 | The package manager MUST not use the identifying string as a filename. |
115 | |
123 | |
116 | 8. The MetaManifest must ultimately be GnuPG-signed. |
124 | 8. The MetaManifest must ultimately be GnuPG-signed. |
117 | |
125 | |
118 | 1. For the initial implementation, the same key as used for snapshot |
126 | 1. For the initial implementation, the same key as used for snapshot |
119 | tarball signing is sufficient. |
127 | tarball signing is sufficient. |
120 | 2. For the future, the key used for fully automated signing by infra |
128 | 2. For the future, the key used for fully automated signing by infra |
121 | should not be on the same keyring as developer keys. See [#GLEPxx+3 |
129 | should not be on the same keyring as developer keys. See |
122 | for further notes]. |
130 | [GLEPxx3] for further notes. |
123 | |
131 | |
|
|
132 | Notes: |
|
|
133 | ====== |
124 | The above does not conflict the proposal contained in GLEP33, which |
134 | The above does not conflict the proposal contained in [GLEP33], which |
125 | restructure eclasses to include subdirectories and Manifest files, as |
135 | restructure eclasses to include subdirectories and Manifest files, as |
126 | the Manifest rules above still provide indirect verification for all |
136 | the Manifest rules above still provide indirect verification for all |
127 | files after the GLEP33 restructuring if it comes to pass. |
137 | files after the [GLEP33] restructuring if it comes to pass. |
128 | |
138 | |
129 | If other Manifests are added (such as per-category, or protecting |
139 | Additional levels of Manifests are required, such as per-category, and |
130 | versioned eclases), the size of the MetaManifest will be greatly |
140 | in the eclasses, profiles and metadata directories. This ensures that a |
131 | reduced, and this specification was written with such a possible future |
141 | change to a singular file causes the smallest possible overall change in |
132 | addition in mind. |
142 | the Manifests as propagated. Creation of the additional levels of |
|
|
143 | Manifests uses the same process as described above, simply starting at a |
|
|
144 | different root point. |
133 | |
145 | |
134 | MetaManifest generation will take place as part of the existing process |
146 | MetaManifest generation will take place as part of the existing process |
135 | by infrastructure that takes the contents of CVS and prepares it for |
147 | by infrastructure that takes the contents of CVS and prepares it for |
136 | distribution via rsync, which includes generating metadata. In-tree |
148 | distribution via rsync, which includes generating metadata. In-tree |
137 | Manifest files are not checked at this point, as they are assumed to be |
149 | Manifest files are not validated at this point, as they are assumed to |
138 | correct. |
150 | be correct. |
139 | |
151 | |
140 | -------------------------------------------------------- |
152 | -------------------------------------------------------- |
141 | Verification of one or more items from the MetaManifest: |
153 | Verification of one or more items from the MetaManifest: |
142 | -------------------------------------------------------- |
154 | -------------------------------------------------------- |
143 | There are two times that this may happen: firstly, immediately after the |
155 | There are two times that this may happen: firstly, immediately after the |
… | |
… | |
150 | Procedure for verifying an item in the MetaManifest: |
162 | Procedure for verifying an item in the MetaManifest: |
151 | ---------------------------------------------------- |
163 | ---------------------------------------------------- |
152 | In the following, I've used term 'M2-verify' to note following the hash |
164 | In the following, I've used term 'M2-verify' to note following the hash |
153 | verification procedures as defined by the Manifest2 format - which |
165 | verification procedures as defined by the Manifest2 format - which |
154 | compromise checking the file length, and that the hashes match. Which |
166 | compromise checking the file length, and that the hashes match. Which |
155 | filetypes may be ignored on missing is discussed in [#GLEP60]. |
167 | filetypes may be ignored on missing is discussed in [GLEP60]. |
156 | |
168 | |
157 | 1. Check the GnuPG signature on the MetaManifest against the keyring of |
169 | 1. Check the GnuPG signature on the MetaManifest against the keyring of |
158 | automated Gentoo keys. See [#GLEPxx+3] for full details regarding |
170 | automated Gentoo keys. See [GLEPxx3] for full details regarding |
159 | verification of GnuPG signatures. |
171 | verification of GnuPG signatures. |
160 | 1. Abort if the signature check fails. |
172 | 1. Abort if the signature check fails. |
161 | |
173 | |
162 | 2. Check the Timestamp header. If it is significently out of date |
174 | 2. Check the Timestamp header. If it is significantly out of date |
163 | compared to the local clock or a trusted source, halt or require |
175 | compared to the local clock or a trusted source, halt or require |
164 | manual intervention from the user. |
176 | manual intervention from the user. |
165 | |
177 | |
166 | 3. For a verification of the tree following an rsync: |
178 | 3. For a verification of the tree following an rsync: |
167 | |
179 | |
… | |
… | |
179 | |
191 | |
180 | 4. If checking at the installation of a package: |
192 | 4. If checking at the installation of a package: |
181 | |
193 | |
182 | 1. M2-verify the entry in MetaManifest for the Manifest |
194 | 1. M2-verify the entry in MetaManifest for the Manifest |
183 | 2. M2-verify all relevant metadata/ contents if metadata/ is being |
195 | 2. M2-verify all relevant metadata/ contents if metadata/ is being |
184 | used in any way (optionally done before dependancy checking). |
196 | used in any way (optionally done before dependency checking). |
185 | 3. M2-verifying the contents of the Manifest. |
197 | 3. M2-verifying the contents of the Manifest. |
186 | 4. Perform M2-verification of all eclasses and profiles used (both |
198 | 4. Perform M2-verification of all eclasses and profiles used (both |
187 | directly and indirectly) by the ebuild. |
199 | directly and indirectly) by the ebuild. |
188 | |
200 | |
189 | Notes: |
201 | Notes: |
… | |
… | |
206 | the individual Gentoo developers. They will continue to develop and |
218 | the individual Gentoo developers. They will continue to develop and |
207 | commit as they do presently, and the MetaManifest is added by |
219 | commit as they do presently, and the MetaManifest is added by |
208 | Infrastructure during the tree generation process, and distributed to |
220 | Infrastructure during the tree generation process, and distributed to |
209 | users. |
221 | users. |
210 | |
222 | |
|
|
223 | Any scripts generating Manifests and the MetaManifest may find it useful |
|
|
224 | to generate multiple levels of Manifests in parallel, and this is |
|
|
225 | explicitly permitted, provided that every file in the tree is covered by |
|
|
226 | at least one Manifest or the MetaManifest file. The uppermost |
|
|
227 | Manifest (MetaManifest) is the only item that does not occur in any |
|
|
228 | other Manifest file, but is instead GPG-signed to enable it's |
|
|
229 | validation. |
|
|
230 | |
211 | -------------------------------------------- |
231 | -------------------------------------------- |
212 | MetaManifest and the new Manifest2 filetypes |
232 | MetaManifest and the new Manifest2 filetypes |
213 | -------------------------------------------- |
233 | -------------------------------------------- |
214 | While [#GLEP60] describes the addition of new filetypes, these are NOT |
234 | While [GLEP60] describes the addition of new filetypes, these are NOT |
215 | needed for implementation of the MetaManifest proposal. Without the new |
235 | needed for implementation of the MetaManifest proposal. Without the new |
216 | filetypes, all entries in the MetaManifest would be of type 'MISC'. |
236 | filetypes, all entries in the MetaManifest would be of type 'MISC'. |
217 | |
237 | |
218 | ---------------------------------------------------- |
238 | ---------------------------------------------------- |
219 | Timestamps & Additional distribution of MetaManifest |
239 | Timestamps & Additional distribution of MetaManifest |
… | |
… | |
227 | trusted channel. |
247 | trusted channel. |
228 | |
248 | |
229 | On all rsync mirrors directly maintained by the Gentoo infrastructure, |
249 | On all rsync mirrors directly maintained by the Gentoo infrastructure, |
230 | and not on community mirrors, there should be a new module |
250 | and not on community mirrors, there should be a new module |
231 | 'gentoo-portage-metamanifests'. Within this module, all MetaManifests |
251 | 'gentoo-portage-metamanifests'. Within this module, all MetaManifests |
232 | for a recent time frame (eg one week) should be kept, named as |
252 | for a recent time frame (e.g. one week) should be kept, named as |
233 | "MetaManifest.$TS", where $TS is the timestamp from inside the file. |
253 | "MetaManifest.$TS", where $TS is the timestamp from inside the file. |
234 | The most recent MetaManifest should always be symlinked as |
254 | The most recent MetaManifest should always be symlinked as |
235 | MetaManifest.current. The possibility of serving the recent |
255 | MetaManifest.current. The possibility of serving the recent |
236 | MetaManifests via HTTPS should also be explored to mitigate MitM |
256 | MetaManifests via HTTPS should also be explored to mitigate |
237 | attacks. |
257 | man-in-the-middle attacks. |
238 | |
258 | |
239 | The package manager should obtain MetaManifest.current and use it to |
259 | The package manager should obtain MetaManifest.current and use it to |
240 | decide is the tree is too out of date per operation #2 of the |
260 | decide is the tree is too out of date per operation #2 of the |
241 | verification process. The decision about freshness should be a |
261 | verification process. The decision about freshness should be a |
242 | user-configuration setting, with the ability to override. |
262 | user-configuration setting, with the ability to override. |
243 | |
263 | |
244 | -------------------------------- |
264 | -------------------------------- |
245 | MetaManifest size considerations |
265 | MetaManifest size considerations |
246 | -------------------------------- |
266 | -------------------------------- |
247 | With only two levels of Manifests (per-package and top-level), every |
267 | With only two levels of Manifests (per-package and top-level), every |
248 | rsync will cause a lot of traffic transfering the modified top-level |
268 | rsync will cause a lot of traffic transferring the modified top-level |
249 | MetaManifest. To reduce this, per-category Manifests are strongly |
269 | MetaManifest. To reduce this, first-level directory Manifests are |
250 | recommended. Alternatively, if the distribution method efficently |
270 | required. Alternatively, if the distribution method efficiently handles |
251 | handles small patch-like changes in an existing file, using an |
271 | small patch-like changes in an existing file, using an uncompressed |
252 | uncompressed MetaManifest may be acceptable (this would primarily be |
272 | MetaManifest may be acceptable (this would primarily be distributed |
253 | distributed version control systems). Other suggestions in reducing this |
273 | version control systems). Other suggestions in reducing this traffic are |
254 | traffic are welcomed. |
274 | welcomed. |
255 | |
275 | |
256 | ======================= |
276 | ======================= |
257 | Backwards Compatibility |
277 | Backwards Compatibility |
258 | ======================= |
278 | ======================= |
259 | - There are no backwards compatibility issues, as old versions of |
279 | - There are no backwards compatibility issues, as old versions of |
… | |
… | |
276 | |
296 | |
277 | ========== |
297 | ========== |
278 | References |
298 | References |
279 | ========== |
299 | ========== |
280 | |
300 | |
281 | [C08a] Cappos, J et al. (2008). "Package Management Security". |
301 | .. [C08a] Cappos, J et al. (2008). "Package Management Security". |
282 | University of Arizona Technical Report TR08-02. Available online |
302 | University of Arizona Technical Report TR08-02. Available online |
283 | from: ftp://ftp.cs.arizona.edu/reports/2008/TR08-02.pdf |
303 | from: ftp://ftp.cs.arizona.edu/reports/2008/TR08-02.pdf |
|
|
304 | |
284 | [C08b] Cappos, J et al. (2008). "Attacks on Package Managers" |
305 | .. [C08b] Cappos, J et al. (2008). "Attacks on Package Managers" |
285 | Available online at: |
306 | Available online at: |
286 | http://www.cs.arizona.edu/people/justin/packagemanagersecurity/ |
307 | http://www.cs.arizona.edu/people/justin/packagemanagersecurity/ |
|
|
308 | |
|
|
309 | .. [GLEP33] Eclass Restructure/Redesign |
|
|
310 | http://www.gentoo.org/proj/en/glep/glep-0033.html |
|
|
311 | |
|
|
312 | .. [GLEP60] Manifest2 filetypes |
|
|
313 | http://www.gentoo.org/proj/en/glep/glep-0044.html |
|
|
314 | |
|
|
315 | .. [GLEPxx2] Future GLEP on Developer Process security. |
|
|
316 | |
|
|
317 | .. [GLEPxx3] Future GLEP on GnuPG Policies and Handling. |
287 | |
318 | |
288 | ========= |
319 | ========= |
289 | Copyright |
320 | Copyright |
290 | ========= |
321 | ========= |
291 | Copyright (c) 2006 by Robin Hugh Johnson. This material may be |
322 | Copyright (c) 2006-2010 by Robin Hugh Johnson. This material may be |
292 | distributed only subject to the terms and conditions set forth in the |
323 | distributed only subject to the terms and conditions set forth in the |
293 | Open Publication License, v1.0. |
324 | Open Publication License, v1.0. |
294 | |
325 | |
295 | vim: tw=72 ts=2 expandtab: |
326 | .. vim: tw=72 ts=2 expandtab: |