| 1 |
------------------------------
|
| 2 |
My poor Portage deconstruction
|
| 3 |
------------------------------
|
| 4 |
|
| 5 |
Here's what I have so far in my "walk" through the Portage code (specifically,
|
| 6 |
portage HEAD sometime around Feb. 2005). Right now I've enumerated ebuild_,
|
| 7 |
the effects of `importing portage`_, emerge_, the Portage
|
| 8 |
`portage.doebuild()`_ function, and the `ebuild.sh`_ script.
|
| 9 |
|
| 10 |
ebuild
|
| 11 |
------
|
| 12 |
|
| 13 |
The ebuild script is a simple interface to portage.py. It's a rather simple
|
| 14 |
python script:
|
| 15 |
|
| 16 |
* Import needed modules
|
| 17 |
* Add ``/usr/lib/portage/pym`` to the python path so that portage can be
|
| 18 |
found.
|
| 19 |
* Define the ``getroot()`` function to ensure that ``ROOT`` is set correctly.
|
| 20 |
* Set the ``PORTAGE_CALLER`` environment variable to ``ebuild``.
|
| 21 |
* Parse options using getopt:
|
| 22 |
* Look for ``--debug``, set ``debug=1`` if found
|
| 23 |
* Look for ``merge``, add ``-noauto`` to ``FEATURES`` if found.
|
| 24 |
* Import portage (and doing all of the setup that entails).
|
| 25 |
|
| 26 |
|
| 27 |
::
|
| 28 |
|
| 29 |
Example: ebuild --debug /path/to/foo.ebuild fetch unpack
|
| 30 |
|
| 31 |
|
| 32 |
Script loops over arguments after the ebuild name:
|
| 33 |
- store (clone) portage.config to be passed to doebuild
|
| 34 |
- Set ``cleanup=1`` if running either ``clean`` or ``config``
|
| 35 |
- execute ``a=portage.doebuild(ebuild name, argument, ROOT, config, debug=debug,cleanup=cleanup)``
|
| 36 |
- fail (exit) if ``a`` exists (or user interrupt or I/O error)
|
| 37 |
|
| 38 |
importing portage
|
| 39 |
-----------------
|
| 40 |
|
| 41 |
Quite a few things happen when portage is imported. Most of portage.py
|
| 42 |
goes about defining a boatload of classes and functions, but there's
|
| 43 |
also a fair amount of information that is assembled and made available
|
| 44 |
by the import process. Here's a (probably incomplete) list of those
|
| 45 |
items:
|
| 46 |
|
| 47 |
* Define constants: portage version number, a list of variables that
|
| 48 |
are "incremental" (USE, FEATURES, etc.), and a list of variables that
|
| 49 |
are "sticky" (KEYWORDS_ACCEPT, USE, ...).
|
| 50 |
* Import a lot of python modules that will be used by portage. Core
|
| 51 |
python modules are imported first, followed by various portage modules.
|
| 52 |
Along the way, ``catpkgsplit``, ``pkgsplit``, and ``pkgcmp`` from
|
| 53 |
``portage_versions`` are added to the portage namespace.
|
| 54 |
* Establish a signal handler to catch SIGCHLD, SIG_DFL, SIGINT, and
|
| 55 |
SIGTERM.
|
| 56 |
* Define a *lot* of functions and classes.
|
| 57 |
|
| 58 |
* Define the "VDB" path (traditionally ``/var/db/pkg``)
|
| 59 |
* Identify ostype from uname, set some os-related defaults,
|
| 60 |
export USERLAND to the environment
|
| 61 |
* Check to make sure user is in the portage group (or root)
|
| 62 |
* List of "incrementals" and "stickies" keys
|
| 63 |
* List possible ebuild version suffixes ("pre", "alpha", ...)
|
| 64 |
* List ebuild keys ("DEPEND", "LICENSE", ...)
|
| 65 |
* Check environment for ROOT, use "/" if not found
|
| 66 |
* Create tmp and var/tmp directories (below ROOT)
|
| 67 |
* Set 022 umask
|
| 68 |
* Check to see if the current profile is deprecated
|
| 69 |
* Set some global settings (PID, bash env, cache dir)
|
| 70 |
* Read use.defaults and set variables appropriately
|
| 71 |
* Get OVERLAY
|
| 72 |
* Lock global settings
|
| 73 |
* Check for selinux
|
| 74 |
* Set up some cachedir stuff
|
| 75 |
* mtime db stuff???
|
| 76 |
* Get FEATURES
|
| 77 |
* Set up porttree and bintree dbs
|
| 78 |
* Get thirdpartymirrors
|
| 79 |
* Perform PORTAGE_CACHEDIR and PORTAGE_TMPDIR sanity checks
|
| 80 |
* Get category list from file
|
| 81 |
* Get package.mask, /etc/portage/package.*
|
| 82 |
* Get packages file
|
| 83 |
* Get unmask list, mask list, "revmask"?
|
| 84 |
* Get ACCEPT_KEYWORDS
|
| 85 |
* Check that /etc/make.profile is a symlink
|
| 86 |
* Check for prelink
|
| 87 |
|
| 88 |
emerge
|
| 89 |
------
|
| 90 |
|
| 91 |
This script is much more complicated than the ebuild script (portage.py
|
| 92 |
is just a bit more than 2x as many lines as this script).
|
| 93 |
|
| 94 |
* Set environment variable ``PORTAGE_CALLER="emerge"``
|
| 95 |
* Import portage <-- portage.settings and other vars now exposed
|
| 96 |
* Check if output should be colored (if not, call ``nocolor()`` ).
|
| 97 |
* Check portage.settings for ``PORTAGE_NICENESS``; if so, nice portage
|
| 98 |
* Freeze portdb api (don't know what this does just yet)
|
| 99 |
* Turn off "noauto" if set in ``FEATURES``
|
| 100 |
* Set # of merged ebuilds = 0
|
| 101 |
* Set up lists of valid emerge parameters (params),
|
| 102 |
actions, and options (all hard-coded lists); also
|
| 103 |
map short-form of options to long-form versions.
|
| 104 |
* Process command line (ugly!)
|
| 105 |
- process short-form to convert to long-form
|
| 106 |
- Process long-form arguments
|
| 107 |
- Create list of options (myopts), actions (myactions), & files (myfiles)
|
| 108 |
- minor fixes ( ``-U`` implies ``-u``, etcetera)
|
| 109 |
- handle ``--noconfmem`` and ``--debug`` flags
|
| 110 |
* Define log, exit, countdown, signal handling, help, and "emerge info"
|
| 111 |
helper functions.
|
| 112 |
* Check permissions; exit if invalid.
|
| 113 |
* Start logging (unless ``--pretend`` was set)
|
| 114 |
* Decide what type of dependencies to use
|
| 115 |
(self, selective, recurse, deep, or empty)
|
| 116 |
* Define spinner func
|
| 117 |
* Define the search class
|
| 118 |
* Build package digraph
|
| 119 |
- Define getlist function for world or system
|
| 120 |
- Define genericdict function
|
| 121 |
- Define depgraph class (huge)
|
| 122 |
* Define unmerge function
|
| 123 |
* Define post_emerge function (finish logging, handle info stuff,
|
| 124 |
config protection)
|
| 125 |
* If --debug set, edebug=1 (not sure why two different debug flags set)
|
| 126 |
* Handle ``emerge sync`` (big if, elif, else construction)
|
| 127 |
* Handle ``emerge regen``
|
| 128 |
* Handle ``emerge config``
|
| 129 |
* Handle ``emerge info``
|
| 130 |
* Handle ``emerge search``
|
| 131 |
* Handle ``emerge inject``
|
| 132 |
* Handle ``emerge unmerge``, ``emerge prune``, and ``emerge clean``
|
| 133 |
- Call ``unmerge(myaction, myfiles)``
|
| 134 |
* Handle ``emerge depclean``
|
| 135 |
* Else: Handle update, system, or just process files
|
| 136 |
- Do some setup stuff if ``--pretend`` set
|
| 137 |
- If ``--resume`` and portage.mtimedb has a "resume" key, then update opts, create a "resume" depgraph
|
| 138 |
- Else, create depgraph, error out if something is wrong
|
| 139 |
- If ``--pretend``, show depgraph
|
| 140 |
- Else
|
| 141 |
+ If ``--buildpkgonly``, check that deps are satisfied
|
| 142 |
+ If --resume, do merge using ``mydepgraph.merge(portage.mtimedb["resume"]["mergelist"])``
|
| 143 |
+ Else, handle ``--digest`` or perform merge, the latter with ``mydepgraph.merge(mydepgraph.altlist())``
|
| 144 |
- Run autoclean
|
| 145 |
- Run ``post_emerge()``
|
| 146 |
|
| 147 |
portage.doebuild()
|
| 148 |
------------------
|
| 149 |
|
| 150 |
Much of what happens during the actual merge of a package happens
|
| 151 |
in portage's doebuild function. The interface is::
|
| 152 |
|
| 153 |
doebuild(myebuild, mydo, myroot, debug=0, listonly=0, fetchonly=0)
|
| 154 |
|
| 155 |
where ``myebuild`` is the path to the ebuild to be merged,
|
| 156 |
``mydo`` is the action to be performed (one of the commands that
|
| 157 |
follows ``ebuild foo.ebuild ...``), and ``debug``, ``listonly``,
|
| 158 |
and ``fetchonly`` are flags that can be true (1) or false (0).
|
| 159 |
|
| 160 |
Here's what happens:
|
| 161 |
|
| 162 |
* Check to make sure the input is valid
|
| 163 |
* Set a boatload of keys in the ``settings`` instance of the config class
|
| 164 |
- PORTAGE_DEBUG, ROOT, STARTDIR, EBUILD, O, CATEGORY, FILESDIR,
|
| 165 |
PF, ECLASSDIR, SANDBOX_LOG, P, PN, PV, PR, PVR, SLOT, BUILD_PREFIX,
|
| 166 |
PKG_TMPDIR, PORTAGE_BUILDDIR, KV, KVERS
|
| 167 |
* If the action is ``depend``, then return dependency info (eclasses?).
|
| 168 |
If ``--debug`` was also set, then the deps are printed on the cmd line.
|
| 169 |
* If necessary (meaning not ``fetch``, ``digest``, or ``manifest``),
|
| 170 |
create the build directory (``/var/tmp/portage/whatever``) and the
|
| 171 |
temp directory (and add T to ``settings``). Also create a ccache
|
| 172 |
directory, if necessary, along with WORKDIR and D.
|
| 173 |
* If ``unmerge``, call ``unmerge()`` and return.
|
| 174 |
* Setup logging if requested
|
| 175 |
* If ``help``, ``clean``, ``setup``, ``prerm``, ``postrm``, ``preinst``,
|
| 176 |
``postinst``, or ``config``, then pass the action to ``ebuild.sh``
|
| 177 |
(using "``ebuild.sh mydo,debug,free=1``") and return.
|
| 178 |
* Generate A from SRC_URI
|
| 179 |
* Ensure that DISTDIR exists; change permissions for ${DISTDIR}/cvs-src
|
| 180 |
if necessary.
|
| 181 |
* Run fetch(); return error on failure.
|
| 182 |
* Generate digest if ``digest`` in FEATURES or called explicitly (return
|
| 183 |
in the latter case).
|
| 184 |
* If ``manifest`` then generate manifest and return.
|
| 185 |
* Check manifest and die on failure if ``strict`` in FEATURES
|
| 186 |
* If ``fetch``, return (since already done above, and it must have worked
|
| 187 |
if we're here)
|
| 188 |
* Set up an "``actionmap``" of dependencies (i.e., ``unpack`` requires
|
| 189 |
``dep`` and ``setup``).
|
| 190 |
* If ``package``, ensure that PKGDIR is created, call
|
| 191 |
``spawnebuild`` with the current action and the actionmap, and return.
|
| 192 |
* If ``qmerge``, call merge w/o checking for dependencies, and return.
|
| 193 |
* If ``merge``, call ``spawnebuild`` with "``install``" and the
|
| 194 |
``actionmap``, and then call ``merge`` and return.
|
| 195 |
* Else, die because the action didn't match any of the above. Oops!
|
| 196 |
|
| 197 |
ebuild.sh
|
| 198 |
---------
|
| 199 |
|
| 200 |
The ``ebuild.sh`` bash script predominantly gets called by the
|
| 201 |
``portage.doebuild`` function to handle the bash functions in an
|
| 202 |
ebuild. Following the bouncing ball....
|
| 203 |
|
| 204 |
The first block of code is initialization stuff:
|
| 205 |
|
| 206 |
* If not ``depend`` or ``clean``, then
|
| 207 |
- clean the "successful" file from ${T} if it exists
|
| 208 |
- Set up logging if desired (including punching a hole in
|
| 209 |
sandbox for the log file)
|
| 210 |
- If ${T}/environment exists, source it
|
| 211 |
* unalias everything
|
| 212 |
* Set "expand_aliases" and set up ``die`` and ``assert`` aliaes
|
| 213 |
* Source ``/etc/profile`` on GNU userland systems
|
| 214 |
* Export proper CC and CXX variables.
|
| 215 |
* Set up the a hardcoded PATH, tack ${ROOTPATH} to the end of it,
|
| 216 |
and add ${PREROOTPATH} to the beginning of the PATH if it exists.
|
| 217 |
* Source helper functions files (``functions.sh`` and
|
| 218 |
``extra_functions.sh``).
|
| 219 |
|
| 220 |
Then a great number of functions are defined:
|
| 221 |
|
| 222 |
* Define a do-nothing ``esyslog()`` override function.
|
| 223 |
* Define a variety of USE flag functions:
|
| 224 |
``use``, ``has``, ``has_version``, ``best_version``,
|
| 225 |
``use_with``, and ``use_enable`` functions.
|
| 226 |
* Define ``diefunc`` function.
|
| 227 |
* Set umask and a number of installation defaults.
|
| 228 |
* Define ``check_KV`` function.
|
| 229 |
* Define the ``keepdir`` function to put ".keep" files in directories
|
| 230 |
that portage should not automatically remove if empty.
|
| 231 |
* Define a series of sandbox helper functions.
|
| 232 |
* Define the ``unpack`` function.
|
| 233 |
* Define the ``econf`` convenience function. Notice that ``econf``
|
| 234 |
automatically dies on error(s).
|
| 235 |
* Define the ``einstall`` convenience function.
|
| 236 |
* Define ``pkg_setup`` as a placebo function to be overridden.
|
| 237 |
* Define ``pkg_nofetch``.
|
| 238 |
* Define minimal, but functional, ``src_unpack`` and ``src_compile``
|
| 239 |
functions.
|
| 240 |
* Define ``src_install``, ``pkg_preinst``, ``pkg_postinst``,
|
| 241 |
``pkg_prerm``, and ``pkg_postrm`` skeleton functions that, when
|
| 242 |
needed, should be overridden.
|
| 243 |
* Define the deprecated ``try`` function.
|
| 244 |
* Define the ``gen_wrapper`` function to generate ``lib/cpp`` and
|
| 245 |
``/usr/bin/cc`` wrappers.
|
| 246 |
* Define some of the ``dyn-*`` functions: ``dyn_setup``, ``dyn_unpack``,
|
| 247 |
``dyn_clean``.
|
| 248 |
* Define some ``install`` helper functions: ``into``, ``insinto``,
|
| 249 |
``exeinto``, ``docinto``, ``insopts``, ``diropts``, ``exeopts``,
|
| 250 |
and ``libopts``. Note that a number of additional helper
|
| 251 |
functions, such as the ``do*`` functions, are actual executable
|
| 252 |
scripts in ``/usr/lib/portage/bin``.
|
| 253 |
* Define a number of abort handler functions: ``abort_handler``,
|
| 254 |
``abort_compile``, ``abort_unpack``, ``abort_package``, and
|
| 255 |
``abort_install``.
|
| 256 |
* Define the rest of the ``dyn_*`` functions: ``dyn_compile``,
|
| 257 |
``dyn_package``, ``dyn_install``, ``dyn_spec``, ``dyn_rpm``,
|
| 258 |
and ``dyn_help``.
|
| 259 |
* Define some eclass debugging functions: ``debug-print``,
|
| 260 |
``debug-print-function``, and ``debug-print-section``.
|
| 261 |
* Define the eclass ``inherit`` function.
|
| 262 |
* Define some eclass helper functions: ``EXPORT_FUNCTIONS``,
|
| 263 |
``newdepend``, ``newrdepend``, ``newcdepend``, ``newpdepend``,
|
| 264 |
and ``do_newdepend``.
|
| 265 |
|
| 266 |
Finally, the main part of the script is reached:
|
| 267 |
|
| 268 |
* If not ``depend`` or ``clean``, then
|
| 269 |
- cd into the ``/var/tmp/portage/whatever`` directory
|
| 270 |
- export USER=portage if the effective user ID is "portage"
|
| 271 |
- Set up distcc and ccache PATHs, environment variables, and
|
| 272 |
sandbox holes
|
| 273 |
* Turn on sandbox flag
|
| 274 |
* Set the S default and unset a handful of variables
|
| 275 |
* Source the ebuild
|
| 276 |
* Set the S default if S not defined in the ebuild. (redundant?)
|
| 277 |
* Set TMP and TMPDIR to be ${T} (otherwise sandbox might error out)
|
| 278 |
* Set RDEPEND is not set, set it to be DEPEND.
|
| 279 |
* Add eclass deps to {R,C,P,}DEPEND.
|
| 280 |
* Loop over arguments passed to ``ebuild.sh``, using a big case statement
|
| 281 |
- ``nofetch`` -- call ``pkg_nofetch``
|
| 282 |
- ``prerm``, ``postrm``, ``preinst``, ``postinst``, ``config`` --
|
| 283 |
turn off sandbox, handle debugging, and call the eponymous
|
| 284 |
``pkg_*`` function.
|
| 285 |
- ``unpack``, ``compile``, ``clean``, ``install`` -- handle sandbox
|
| 286 |
and debugging, then call the eponymous ``dyn_*`` function.
|
| 287 |
- ``help``, ``clean``, ``setup`` -- Turn off sandbox, handle
|
| 288 |
debugging, and call the appropriate ``dyn_*`` function.
|
| 289 |
- ``package``, ``rpm`` -- Turn off sandbox, handle debugging,
|
| 290 |
and call the eponymous ``dyn_*`` function.
|
| 291 |
- ``depend`` -- Turn off sandbox and write dependency info
|
| 292 |
to ``/var/edb/cache/dep/CATEGORY/PF``.
|
| 293 |
- Else, error that the command wasn't valid.
|
| 294 |
* If ``clean``, clean that package's temp directory
|
| 295 |
* Finish by touching "successful" in the package's temp directory.
|
| 296 |
|
Parent Directory
| 
Revision Log