/[gentoo-src]/portage/portage.txt
Gentoo

Contents of /portage/portage.txt

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.3 - (hide annotations) (download)
Wed May 4 01:07:04 2005 UTC (9 years, 6 months ago) by vapier
Branch: MAIN
CVS Tags: HEAD
Branch point for: portage_2_1
Changes since 1.2: +1 -1 lines
File MIME type: text/plain
rename BUILDDIR to PORTAGE_BUILDDIR #68269

1 g2boojum 1.1 ------------------------------
2     My poor Portage deconstruction
3     ------------------------------
4    
5     Here's what I have so far in my "walk" through the Portage code (specifically,
6     portage HEAD sometime around Feb. 2005). Right now I've enumerated ebuild_,
7     the effects of `importing portage`_, emerge_, the Portage
8     `portage.doebuild()`_ function, and the `ebuild.sh`_ script.
9    
10     ebuild
11     ------
12    
13     The ebuild script is a simple interface to portage.py. It's a rather simple
14     python script:
15    
16     * Import needed modules
17     * Add ``/usr/lib/portage/pym`` to the python path so that portage can be
18     found.
19     * Define the ``getroot()`` function to ensure that ``ROOT`` is set correctly.
20     * Set the ``PORTAGE_CALLER`` environment variable to ``ebuild``.
21     * Parse options using getopt:
22     * Look for ``--debug``, set ``debug=1`` if found
23     * Look for ``merge``, add ``-noauto`` to ``FEATURES`` if found.
24     * Import portage (and doing all of the setup that entails).
25    
26    
27     ::
28    
29     Example: ebuild --debug /path/to/foo.ebuild fetch unpack
30    
31    
32     Script loops over arguments after the ebuild name:
33     - store (clone) portage.config to be passed to doebuild
34     - Set ``cleanup=1`` if running either ``clean`` or ``config``
35     - execute ``a=portage.doebuild(ebuild name, argument, ROOT, config, debug=debug,cleanup=cleanup)``
36     - fail (exit) if ``a`` exists (or user interrupt or I/O error)
37    
38     importing portage
39     -----------------
40    
41     Quite a few things happen when portage is imported. Most of portage.py
42     goes about defining a boatload of classes and functions, but there's
43     also a fair amount of information that is assembled and made available
44     by the import process. Here's a (probably incomplete) list of those
45     items:
46    
47 g2boojum 1.2 * Define constants: portage version number, a list of variables that
48     are "incremental" (USE, FEATURES, etc.), and a list of variables that
49     are "sticky" (KEYWORDS_ACCEPT, USE, ...).
50     * Import a lot of python modules that will be used by portage. Core
51     python modules are imported first, followed by various portage modules.
52     Along the way, ``catpkgsplit``, ``pkgsplit``, and ``pkgcmp`` from
53     ``portage_versions`` are added to the portage namespace.
54     * Establish a signal handler to catch SIGCHLD, SIG_DFL, SIGINT, and
55     SIGTERM.
56     * Define a *lot* of functions and classes.
57    
58 g2boojum 1.1 * Define the "VDB" path (traditionally ``/var/db/pkg``)
59     * Identify ostype from uname, set some os-related defaults,
60     export USERLAND to the environment
61     * Check to make sure user is in the portage group (or root)
62     * List of "incrementals" and "stickies" keys
63     * List possible ebuild version suffixes ("pre", "alpha", ...)
64     * List ebuild keys ("DEPEND", "LICENSE", ...)
65     * Check environment for ROOT, use "/" if not found
66     * Create tmp and var/tmp directories (below ROOT)
67     * Set 022 umask
68     * Check to see if the current profile is deprecated
69     * Set some global settings (PID, bash env, cache dir)
70     * Read use.defaults and set variables appropriately
71     * Get OVERLAY
72     * Lock global settings
73     * Check for selinux
74     * Set up some cachedir stuff
75     * mtime db stuff???
76     * Get FEATURES
77     * Set up porttree and bintree dbs
78     * Get thirdpartymirrors
79     * Perform PORTAGE_CACHEDIR and PORTAGE_TMPDIR sanity checks
80     * Get category list from file
81     * Get package.mask, /etc/portage/package.*
82     * Get packages file
83     * Get unmask list, mask list, "revmask"?
84     * Get ACCEPT_KEYWORDS
85     * Check that /etc/make.profile is a symlink
86     * Check for prelink
87    
88     emerge
89     ------
90    
91     This script is much more complicated than the ebuild script (portage.py
92     is just a bit more than 2x as many lines as this script).
93    
94     * Set environment variable ``PORTAGE_CALLER="emerge"``
95     * Import portage <-- portage.settings and other vars now exposed
96     * Check if output should be colored (if not, call ``nocolor()`` ).
97     * Check portage.settings for ``PORTAGE_NICENESS``; if so, nice portage
98     * Freeze portdb api (don't know what this does just yet)
99     * Turn off "noauto" if set in ``FEATURES``
100     * Set # of merged ebuilds = 0
101     * Set up lists of valid emerge parameters (params),
102     actions, and options (all hard-coded lists); also
103     map short-form of options to long-form versions.
104     * Process command line (ugly!)
105     - process short-form to convert to long-form
106     - Process long-form arguments
107     - Create list of options (myopts), actions (myactions), & files (myfiles)
108     - minor fixes ( ``-U`` implies ``-u``, etcetera)
109     - handle ``--noconfmem`` and ``--debug`` flags
110     * Define log, exit, countdown, signal handling, help, and "emerge info"
111     helper functions.
112     * Check permissions; exit if invalid.
113     * Start logging (unless ``--pretend`` was set)
114     * Decide what type of dependencies to use
115     (self, selective, recurse, deep, or empty)
116     * Define spinner func
117     * Define the search class
118     * Build package digraph
119     - Define getlist function for world or system
120     - Define genericdict function
121     - Define depgraph class (huge)
122     * Define unmerge function
123     * Define post_emerge function (finish logging, handle info stuff,
124     config protection)
125     * If --debug set, edebug=1 (not sure why two different debug flags set)
126     * Handle ``emerge sync`` (big if, elif, else construction)
127     * Handle ``emerge regen``
128     * Handle ``emerge config``
129     * Handle ``emerge info``
130     * Handle ``emerge search``
131     * Handle ``emerge inject``
132     * Handle ``emerge unmerge``, ``emerge prune``, and ``emerge clean``
133     - Call ``unmerge(myaction, myfiles)``
134     * Handle ``emerge depclean``
135     * Else: Handle update, system, or just process files
136     - Do some setup stuff if ``--pretend`` set
137     - If ``--resume`` and portage.mtimedb has a "resume" key, then update opts, create a "resume" depgraph
138     - Else, create depgraph, error out if something is wrong
139     - If ``--pretend``, show depgraph
140     - Else
141     + If ``--buildpkgonly``, check that deps are satisfied
142     + If --resume, do merge using ``mydepgraph.merge(portage.mtimedb["resume"]["mergelist"])``
143     + Else, handle ``--digest`` or perform merge, the latter with ``mydepgraph.merge(mydepgraph.altlist())``
144     - Run autoclean
145     - Run ``post_emerge()``
146    
147     portage.doebuild()
148     ------------------
149    
150     Much of what happens during the actual merge of a package happens
151     in portage's doebuild function. The interface is::
152    
153     doebuild(myebuild, mydo, myroot, debug=0, listonly=0, fetchonly=0)
154    
155     where ``myebuild`` is the path to the ebuild to be merged,
156     ``mydo`` is the action to be performed (one of the commands that
157     follows ``ebuild foo.ebuild ...``), and ``debug``, ``listonly``,
158     and ``fetchonly`` are flags that can be true (1) or false (0).
159    
160     Here's what happens:
161    
162     * Check to make sure the input is valid
163     * Set a boatload of keys in the ``settings`` instance of the config class
164     - PORTAGE_DEBUG, ROOT, STARTDIR, EBUILD, O, CATEGORY, FILESDIR,
165     PF, ECLASSDIR, SANDBOX_LOG, P, PN, PV, PR, PVR, SLOT, BUILD_PREFIX,
166 vapier 1.3 PKG_TMPDIR, PORTAGE_BUILDDIR, KV, KVERS
167 g2boojum 1.1 * If the action is ``depend``, then return dependency info (eclasses?).
168     If ``--debug`` was also set, then the deps are printed on the cmd line.
169     * If necessary (meaning not ``fetch``, ``digest``, or ``manifest``),
170     create the build directory (``/var/tmp/portage/whatever``) and the
171     temp directory (and add T to ``settings``). Also create a ccache
172     directory, if necessary, along with WORKDIR and D.
173     * If ``unmerge``, call ``unmerge()`` and return.
174     * Setup logging if requested
175     * If ``help``, ``clean``, ``setup``, ``prerm``, ``postrm``, ``preinst``,
176     ``postinst``, or ``config``, then pass the action to ``ebuild.sh``
177     (using "``ebuild.sh mydo,debug,free=1``") and return.
178     * Generate A from SRC_URI
179     * Ensure that DISTDIR exists; change permissions for ${DISTDIR}/cvs-src
180     if necessary.
181     * Run fetch(); return error on failure.
182     * Generate digest if ``digest`` in FEATURES or called explicitly (return
183     in the latter case).
184     * If ``manifest`` then generate manifest and return.
185     * Check manifest and die on failure if ``strict`` in FEATURES
186     * If ``fetch``, return (since already done above, and it must have worked
187     if we're here)
188     * Set up an "``actionmap``" of dependencies (i.e., ``unpack`` requires
189     ``dep`` and ``setup``).
190     * If ``package``, ensure that PKGDIR is created, call
191     ``spawnebuild`` with the current action and the actionmap, and return.
192     * If ``qmerge``, call merge w/o checking for dependencies, and return.
193     * If ``merge``, call ``spawnebuild`` with "``install``" and the
194     ``actionmap``, and then call ``merge`` and return.
195     * Else, die because the action didn't match any of the above. Oops!
196    
197     ebuild.sh
198     ---------
199    
200     The ``ebuild.sh`` bash script predominantly gets called by the
201     ``portage.doebuild`` function to handle the bash functions in an
202     ebuild. Following the bouncing ball....
203    
204     The first block of code is initialization stuff:
205    
206     * If not ``depend`` or ``clean``, then
207     - clean the "successful" file from ${T} if it exists
208     - Set up logging if desired (including punching a hole in
209     sandbox for the log file)
210     - If ${T}/environment exists, source it
211     * unalias everything
212     * Set "expand_aliases" and set up ``die`` and ``assert`` aliaes
213     * Source ``/etc/profile`` on GNU userland systems
214     * Export proper CC and CXX variables.
215     * Set up the a hardcoded PATH, tack ${ROOTPATH} to the end of it,
216     and add ${PREROOTPATH} to the beginning of the PATH if it exists.
217     * Source helper functions files (``functions.sh`` and
218     ``extra_functions.sh``).
219    
220     Then a great number of functions are defined:
221    
222     * Define a do-nothing ``esyslog()`` override function.
223     * Define a variety of USE flag functions:
224     ``use``, ``has``, ``has_version``, ``best_version``,
225     ``use_with``, and ``use_enable`` functions.
226     * Define ``diefunc`` function.
227     * Set umask and a number of installation defaults.
228     * Define ``check_KV`` function.
229     * Define the ``keepdir`` function to put ".keep" files in directories
230     that portage should not automatically remove if empty.
231     * Define a series of sandbox helper functions.
232     * Define the ``unpack`` function.
233     * Define the ``econf`` convenience function. Notice that ``econf``
234     automatically dies on error(s).
235     * Define the ``einstall`` convenience function.
236     * Define ``pkg_setup`` as a placebo function to be overridden.
237     * Define ``pkg_nofetch``.
238     * Define minimal, but functional, ``src_unpack`` and ``src_compile``
239     functions.
240     * Define ``src_install``, ``pkg_preinst``, ``pkg_postinst``,
241     ``pkg_prerm``, and ``pkg_postrm`` skeleton functions that, when
242     needed, should be overridden.
243     * Define the deprecated ``try`` function.
244     * Define the ``gen_wrapper`` function to generate ``lib/cpp`` and
245     ``/usr/bin/cc`` wrappers.
246     * Define some of the ``dyn-*`` functions: ``dyn_setup``, ``dyn_unpack``,
247     ``dyn_clean``.
248     * Define some ``install`` helper functions: ``into``, ``insinto``,
249     ``exeinto``, ``docinto``, ``insopts``, ``diropts``, ``exeopts``,
250     and ``libopts``. Note that a number of additional helper
251     functions, such as the ``do*`` functions, are actual executable
252     scripts in ``/usr/lib/portage/bin``.
253     * Define a number of abort handler functions: ``abort_handler``,
254     ``abort_compile``, ``abort_unpack``, ``abort_package``, and
255     ``abort_install``.
256     * Define the rest of the ``dyn_*`` functions: ``dyn_compile``,
257     ``dyn_package``, ``dyn_install``, ``dyn_spec``, ``dyn_rpm``,
258     and ``dyn_help``.
259     * Define some eclass debugging functions: ``debug-print``,
260     ``debug-print-function``, and ``debug-print-section``.
261     * Define the eclass ``inherit`` function.
262     * Define some eclass helper functions: ``EXPORT_FUNCTIONS``,
263     ``newdepend``, ``newrdepend``, ``newcdepend``, ``newpdepend``,
264     and ``do_newdepend``.
265    
266     Finally, the main part of the script is reached:
267    
268     * If not ``depend`` or ``clean``, then
269     - cd into the ``/var/tmp/portage/whatever`` directory
270     - export USER=portage if the effective user ID is "portage"
271     - Set up distcc and ccache PATHs, environment variables, and
272     sandbox holes
273     * Turn on sandbox flag
274     * Set the S default and unset a handful of variables
275     * Source the ebuild
276     * Set the S default if S not defined in the ebuild. (redundant?)
277     * Set TMP and TMPDIR to be ${T} (otherwise sandbox might error out)
278     * Set RDEPEND is not set, set it to be DEPEND.
279     * Add eclass deps to {R,C,P,}DEPEND.
280     * Loop over arguments passed to ``ebuild.sh``, using a big case statement
281     - ``nofetch`` -- call ``pkg_nofetch``
282     - ``prerm``, ``postrm``, ``preinst``, ``postinst``, ``config`` --
283     turn off sandbox, handle debugging, and call the eponymous
284     ``pkg_*`` function.
285     - ``unpack``, ``compile``, ``clean``, ``install`` -- handle sandbox
286     and debugging, then call the eponymous ``dyn_*`` function.
287     - ``help``, ``clean``, ``setup`` -- Turn off sandbox, handle
288     debugging, and call the appropriate ``dyn_*`` function.
289     - ``package``, ``rpm`` -- Turn off sandbox, handle debugging,
290     and call the eponymous ``dyn_*`` function.
291     - ``depend`` -- Turn off sandbox and write dependency info
292     to ``/var/edb/cache/dep/CATEGORY/PF``.
293     - Else, error that the command wasn't valid.
294     * If ``clean``, clean that package's temp directory
295     * Finish by touching "successful" in the package's temp directory.

  ViewVC Help
Powered by ViewVC 1.1.20