/[gentoo]/xml/htdocs/proj/en/glep/glep-0058.txt
Gentoo

Diff of /xml/htdocs/proj/en/glep/glep-0058.txt

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

Revision 1.4 Revision 1.10
1GLEP: 58 1GLEP: 58
2Title: Security of distribution of Gentoo software - Infrastructure to User distribution - MetaManifest 2Title: Security of distribution of Gentoo software - Infrastructure to User distribution - MetaManifest
3Version: $Revision: 1.4 $ 3Version: $Revision: 1.10 $
4Last-Modified: $Date: 2008/10/28 07:45:27 $ 4Last-Modified: $Date: 2010/04/07 21:34:24 $
5Author: Robin Hugh Johnson <robbat2@gentoo.org>, 5Author: Robin Hugh Johnson <robbat2@gentoo.org>,
6Status: Draft 6Status: Draft
7Type: Standards Track 7Type: Standards Track
8Content-Type: text/x-rst 8Content-Type: text/x-rst
9Requires: 44, 60 9Requires: 44, 60
10Created: October 2006 10Created: October 2006
11Updated: November 2007, June 2008, July 2008, October 2008 11Updated: November 2007, June 2008, July 2008, October 2008, January 2010
12Post-History: 12Post-History: December 2009, January 2010
13 13
14======== 14========
15Abstract 15Abstract
16======== 16========
17MetaManifest provides a means of verifiable distribution from Gentoo 17MetaManifest provides a means of verifiable distribution from Gentoo
… …
73located at the root of a repository. 73located at the root of a repository.
74 74
75--------------------------------------------- 75---------------------------------------------
76Procedure for creating the MetaManifest file: 76Procedure for creating the MetaManifest file:
77--------------------------------------------- 77---------------------------------------------
78Summary:
79========
80The objective of creating the MetaManifest file(s) is to ensure that
81every single file in the tree occurs in at least one Manifest.
82
83Process:
84========
781. Start at the root of the Gentoo Portage tree (gentoo-x86, although 851. Start at the root of the Gentoo Portage tree (gentoo-x86, although
79 this procedure applies to overlays as well). 86 this procedure applies to overlays as well).
80 87
812. Initialize two unordered sets: COVERED, ALL. 882. Initialize two unordered sets: COVERED, ALL.
82 89
83 1. 'ALL' will contain every file in the tree. 90 1. 'ALL' shall contain every file that exists in the present tree.
84 2. 'COVERED' will contain every file that is mentioned in an existing 91 2. 'COVERED' shall contain EVERY file that is mentioned in an existing
85 Manifest2. 92 Manifest2. If a file is mentioned in a Manifest2, but does not
93 exist, it must still be included. No files should be excluded.
86 94
873. Traverse the tree, depth-first. 953. Traverse the tree, depth-first.
88 96
89 1. At the top level only, ignore the following directories: distfiles, 97 1. At the top level only, ignore the following directories: distfiles,
90 packages, local 98 packages, local.
91 2. If a directory contains a Manifest file, extract all relevant local 99 2. If a directory contains a Manifest file, extract all relevant local
92 files from it (presently: AUX, MISC, EBUILD; but should follow the 100 files from it (presently: AUX, MISC, EBUILD; but should follow the
93 evolution of Manifest2 entry types per [#GLEP60]), and place them 101 evolution of Manifest2 entry types per [GLEP60]), and place them
94 into the COVERED set. 102 into the COVERED set.
95 3. Recursively add every file in the directory to the ALL set, 103 3. Recursively add every file in the directory to the ALL set,
96 pursusant to the exclusion list as mentioned in [#GLEP60]. 104 pursuant to the exclusion list as mentioned in [GLEP60].
97 105
984. Produce a new set, UNCOVERED, as the set-difference (ALL)-(COVERED). 1064. Produce a new set, UNCOVERED, as the set-difference (ALL)-(COVERED).
99 This is every item that is not covered by another Manifest, or part 107 This is every item that is not covered by another Manifest, or part
100 of an exclusion list. 108 of an exclusion list.
101 109
… …
1077. For unique identification of the MetaManifest, a header line should 1157. For unique identification of the MetaManifest, a header line should
108 be included, using the exact contents of the metadata/timestamp.x 116 be included, using the exact contents of the metadata/timestamp.x
109 file, so that a MetaManifest may be tied back to a tree as 117 file, so that a MetaManifest may be tied back to a tree as
110 distributed by the rsync mirror system. The string of 118 distributed by the rsync mirror system. The string of
111 'metadata/timestamp.x' should be included to identify this revision 119 'metadata/timestamp.x' should be included to identify this revision
112 of MetaManifest generation. Eg: 120 of MetaManifest generation. e.g.:
113 "Timestamp: metadata/timestamp.x: 1215722461 Thu Jul 10 20:41:01 2008 UTC" 121 "Timestamp: metadata/timestamp.x: 1215722461 Thu Jul 10 20:41:01 2008 UTC"
114 The package manager MUST not use the identifying string as a filename. 122 The package manager MUST not use the identifying string as a filename.
115 123
1168. The MetaManifest must ultimately be GnuPG-signed. 1248. The MetaManifest must ultimately be GnuPG-signed.
117 125
118 1. For the initial implementation, the same key as used for snapshot 126 1. For the initial implementation, the same key as used for snapshot
119 tarball signing is sufficient. 127 tarball signing is sufficient.
120 2. For the future, the key used for fully automated signing by infra 128 2. For the future, the key used for fully automated signing by infra
121 should not be on the same keyring as developer keys. See [#GLEPxx+3 129 should not be on the same keyring as developer keys. See
122 for further notes]. 130 [GLEPxx3] for further notes.
123 131
132Notes:
133======
124The above does not conflict the proposal contained in GLEP33, which 134The above does not conflict the proposal contained in [GLEP33], which
125restructure eclasses to include subdirectories and Manifest files, as 135restructure eclasses to include subdirectories and Manifest files, as
126the Manifest rules above still provide indirect verification for all 136the Manifest rules above still provide indirect verification for all
127files after the GLEP33 restructuring if it comes to pass. 137files after the [GLEP33] restructuring if it comes to pass.
128 138
129If other Manifests are added (such as per-category, or protecting 139Additional levels of Manifests are required, such as per-category, and
130versioned eclases), the size of the MetaManifest will be greatly 140in the eclasses, profiles and metadata directories. This ensures that a
131reduced, and this specification was written with such a possible future 141change to a singular file causes the smallest possible overall change in
132addition in mind. 142the Manifests as propagated. Creation of the additional levels of
143Manifests uses the same process as described above, simply starting at a
144different root point.
133 145
134MetaManifest generation will take place as part of the existing process 146MetaManifest generation will take place as part of the existing process
135by infrastructure that takes the contents of CVS and prepares it for 147by infrastructure that takes the contents of CVS and prepares it for
136distribution via rsync, which includes generating metadata. In-tree 148distribution via rsync, which includes generating metadata. In-tree
137Manifest files are not checked at this point, as they are assumed to be 149Manifest files are not validated at this point, as they are assumed to
138correct. 150be correct.
139 151
140-------------------------------------------------------- 152--------------------------------------------------------
141Verification of one or more items from the MetaManifest: 153Verification of one or more items from the MetaManifest:
142-------------------------------------------------------- 154--------------------------------------------------------
143There are two times that this may happen: firstly, immediately after the 155There are two times that this may happen: firstly, immediately after the
… …
150Procedure for verifying an item in the MetaManifest: 162Procedure for verifying an item in the MetaManifest:
151---------------------------------------------------- 163----------------------------------------------------
152In the following, I've used term 'M2-verify' to note following the hash 164In the following, I've used term 'M2-verify' to note following the hash
153verification procedures as defined by the Manifest2 format - which 165verification procedures as defined by the Manifest2 format - which
154compromise checking the file length, and that the hashes match. Which 166compromise checking the file length, and that the hashes match. Which
155filetypes may be ignored on missing is discussed in [#GLEP60]. 167filetypes may be ignored on missing is discussed in [GLEP60].
156 168
1571. Check the GnuPG signature on the MetaManifest against the keyring of 1691. Check the GnuPG signature on the MetaManifest against the keyring of
158 automated Gentoo keys. See [#GLEPxx+3] for full details regarding 170 automated Gentoo keys. See [GLEPxx3] for full details regarding
159 verification of GnuPG signatures. 171 verification of GnuPG signatures.
160 1. Abort if the signature check fails. 172 1. Abort if the signature check fails.
161 173
1622. Check the Timestamp header. If it is significently out of date 1742. Check the Timestamp header. If it is significantly out of date
163 compared to the local clock or a trusted source, halt or require 175 compared to the local clock or a trusted source, halt or require
164 manual intervention from the user. 176 manual intervention from the user.
165 177
1663. For a verification of the tree following an rsync: 1783. For a verification of the tree following an rsync:
167 179
… …
179 191
1804. If checking at the installation of a package: 1924. If checking at the installation of a package:
181 193
182 1. M2-verify the entry in MetaManifest for the Manifest 194 1. M2-verify the entry in MetaManifest for the Manifest
183 2. M2-verify all relevant metadata/ contents if metadata/ is being 195 2. M2-verify all relevant metadata/ contents if metadata/ is being
184 used in any way (optionally done before dependancy checking). 196 used in any way (optionally done before dependency checking).
185 3. M2-verifying the contents of the Manifest. 197 3. M2-verifying the contents of the Manifest.
186 4. Perform M2-verification of all eclasses and profiles used (both 198 4. Perform M2-verification of all eclasses and profiles used (both
187 directly and indirectly) by the ebuild. 199 directly and indirectly) by the ebuild.
188 200
189Notes: 201Notes:
… …
206the individual Gentoo developers. They will continue to develop and 218the individual Gentoo developers. They will continue to develop and
207commit as they do presently, and the MetaManifest is added by 219commit as they do presently, and the MetaManifest is added by
208Infrastructure during the tree generation process, and distributed to 220Infrastructure during the tree generation process, and distributed to
209users. 221users.
210 222
223Any scripts generating Manifests and the MetaManifest may find it useful
224to generate multiple levels of Manifests in parallel, and this is
225explicitly permitted, provided that every file in the tree is covered by
226at least one Manifest or the MetaManifest file. The uppermost
227Manifest (MetaManifest) is the only item that does not occur in any
228other Manifest file, but is instead GPG-signed to enable it's
229validation.
230
211-------------------------------------------- 231--------------------------------------------
212MetaManifest and the new Manifest2 filetypes 232MetaManifest and the new Manifest2 filetypes
213-------------------------------------------- 233--------------------------------------------
214While [#GLEP60] describes the addition of new filetypes, these are NOT 234While [GLEP60] describes the addition of new filetypes, these are NOT
215needed for implementation of the MetaManifest proposal. Without the new 235needed for implementation of the MetaManifest proposal. Without the new
216filetypes, all entries in the MetaManifest would be of type 'MISC'. 236filetypes, all entries in the MetaManifest would be of type 'MISC'.
217 237
218---------------------------------------------------- 238----------------------------------------------------
219Timestamps & Additional distribution of MetaManifest 239Timestamps & Additional distribution of MetaManifest
… …
227trusted channel. 247trusted channel.
228 248
229On all rsync mirrors directly maintained by the Gentoo infrastructure, 249On all rsync mirrors directly maintained by the Gentoo infrastructure,
230and not on community mirrors, there should be a new module 250and not on community mirrors, there should be a new module
231'gentoo-portage-metamanifests'. Within this module, all MetaManifests 251'gentoo-portage-metamanifests'. Within this module, all MetaManifests
232for a recent time frame (eg one week) should be kept, named as 252for a recent time frame (e.g. one week) should be kept, named as
233"MetaManifest.$TS", where $TS is the timestamp from inside the file. 253"MetaManifest.$TS", where $TS is the timestamp from inside the file.
234The most recent MetaManifest should always be symlinked as 254The most recent MetaManifest should always be symlinked as
235MetaManifest.current. The possibility of serving the recent 255MetaManifest.current. The possibility of serving the recent
236MetaManifests via HTTPS should also be explored to mitigate MitM 256MetaManifests via HTTPS should also be explored to mitigate
237attacks. 257man-in-the-middle attacks.
238 258
239The package manager should obtain MetaManifest.current and use it to 259The package manager should obtain MetaManifest.current and use it to
240decide is the tree is too out of date per operation #2 of the 260decide is the tree is too out of date per operation #2 of the
241verification process. The decision about freshness should be a 261verification process. The decision about freshness should be a
242user-configuration setting, with the ability to override. 262user-configuration setting, with the ability to override.
243 263
244-------------------------------- 264--------------------------------
245MetaManifest size considerations 265MetaManifest size considerations
246-------------------------------- 266--------------------------------
247With only two levels of Manifests (per-package and top-level), every 267With only two levels of Manifests (per-package and top-level), every
248rsync will cause a lot of traffic transfering the modified top-level 268rsync will cause a lot of traffic transferring the modified top-level
249MetaManifest. To reduce this, per-category Manifests are strongly 269MetaManifest. To reduce this, first-level directory Manifests are
250recommended. Alternatively, if the distribution method efficently 270required. Alternatively, if the distribution method efficiently handles
251handles small patch-like changes in an existing file, using an 271small patch-like changes in an existing file, using an uncompressed
252uncompressed MetaManifest may be acceptable (this would primarily be 272MetaManifest may be acceptable (this would primarily be distributed
253distributed version control systems). Other suggestions in reducing this 273version control systems). Other suggestions in reducing this traffic are
254traffic are welcomed. 274welcomed.
255 275
256======================= 276=======================
257Backwards Compatibility 277Backwards Compatibility
258======================= 278=======================
259- There are no backwards compatibility issues, as old versions of 279- There are no backwards compatibility issues, as old versions of
… …
276 296
277========== 297==========
278References 298References
279========== 299==========
280 300
281[C08a] Cappos, J et al. (2008). "Package Management Security". 301.. [C08a] Cappos, J et al. (2008). "Package Management Security".
282 University of Arizona Technical Report TR08-02. Available online 302 University of Arizona Technical Report TR08-02. Available online
283 from: ftp://ftp.cs.arizona.edu/reports/2008/TR08-02.pdf 303 from: ftp://ftp.cs.arizona.edu/reports/2008/TR08-02.pdf
304
284[C08b] Cappos, J et al. (2008). "Attacks on Package Managers" 305.. [C08b] Cappos, J et al. (2008). "Attacks on Package Managers"
285 Available online at: 306 Available online at:
286 http://www.cs.arizona.edu/people/justin/packagemanagersecurity/ 307 http://www.cs.arizona.edu/people/justin/packagemanagersecurity/
308
309.. [GLEP33] Eclass Restructure/Redesign
310 http://www.gentoo.org/proj/en/glep/glep-0033.html
311
312.. [GLEP60] Manifest2 filetypes
313 http://www.gentoo.org/proj/en/glep/glep-0044.html
314
315.. [GLEPxx2] Future GLEP on Developer Process security.
316
317.. [GLEPxx3] Future GLEP on GnuPG Policies and Handling.
287 318
288========= 319=========
289Copyright 320Copyright
290========= 321=========
291Copyright (c) 2006 by Robin Hugh Johnson. This material may be 322Copyright (c) 2006-2010 by Robin Hugh Johnson. This material may be
292distributed only subject to the terms and conditions set forth in the 323distributed only subject to the terms and conditions set forth in the
293Open Publication License, v1.0. 324Open Publication License, v1.0.
294 325
295vim: tw=72 ts=2 expandtab: 326.. vim: tw=72 ts=2 expandtab:
Legend:
Removed from v.1.4  
changed lines
  Added in v.1.10
  ViewVC Help
Powered by ViewVC 1.1.20