| 1 |
g2boojum |
1.1 |
------------------------------
|
| 2 |
|
|
My poor Portage deconstruction
|
| 3 |
|
|
------------------------------
|
| 4 |
|
|
|
| 5 |
|
|
Here's what I have so far in my "walk" through the Portage code (specifically,
|
| 6 |
|
|
portage HEAD sometime around Feb. 2005). Right now I've enumerated ebuild_,
|
| 7 |
|
|
the effects of `importing portage`_, emerge_, the Portage
|
| 8 |
|
|
`portage.doebuild()`_ function, and the `ebuild.sh`_ script.
|
| 9 |
|
|
|
| 10 |
|
|
ebuild
|
| 11 |
|
|
------
|
| 12 |
|
|
|
| 13 |
|
|
The ebuild script is a simple interface to portage.py. It's a rather simple
|
| 14 |
|
|
python script:
|
| 15 |
|
|
|
| 16 |
|
|
* Import needed modules
|
| 17 |
|
|
* Add ``/usr/lib/portage/pym`` to the python path so that portage can be
|
| 18 |
|
|
found.
|
| 19 |
|
|
* Define the ``getroot()`` function to ensure that ``ROOT`` is set correctly.
|
| 20 |
|
|
* Set the ``PORTAGE_CALLER`` environment variable to ``ebuild``.
|
| 21 |
|
|
* Parse options using getopt:
|
| 22 |
|
|
* Look for ``--debug``, set ``debug=1`` if found
|
| 23 |
|
|
* Look for ``merge``, add ``-noauto`` to ``FEATURES`` if found.
|
| 24 |
|
|
* Import portage (and doing all of the setup that entails).
|
| 25 |
|
|
|
| 26 |
|
|
|
| 27 |
|
|
::
|
| 28 |
|
|
|
| 29 |
|
|
Example: ebuild --debug /path/to/foo.ebuild fetch unpack
|
| 30 |
|
|
|
| 31 |
|
|
|
| 32 |
|
|
Script loops over arguments after the ebuild name:
|
| 33 |
|
|
- store (clone) portage.config to be passed to doebuild
|
| 34 |
|
|
- Set ``cleanup=1`` if running either ``clean`` or ``config``
|
| 35 |
|
|
- execute ``a=portage.doebuild(ebuild name, argument, ROOT, config, debug=debug,cleanup=cleanup)``
|
| 36 |
|
|
- fail (exit) if ``a`` exists (or user interrupt or I/O error)
|
| 37 |
|
|
|
| 38 |
|
|
importing portage
|
| 39 |
|
|
-----------------
|
| 40 |
|
|
|
| 41 |
|
|
Quite a few things happen when portage is imported. Most of portage.py
|
| 42 |
|
|
goes about defining a boatload of classes and functions, but there's
|
| 43 |
|
|
also a fair amount of information that is assembled and made available
|
| 44 |
|
|
by the import process. Here's a (probably incomplete) list of those
|
| 45 |
|
|
items:
|
| 46 |
|
|
|
| 47 |
|
|
* Define the "VDB" path (traditionally ``/var/db/pkg``)
|
| 48 |
|
|
* Set up a signal handler to catch SIGCHLD, SIG_DFL
|
| 49 |
|
|
* Identify ostype from uname, set some os-related defaults,
|
| 50 |
|
|
export USERLAND to the environment
|
| 51 |
|
|
* Check to make sure user is in the portage group (or root)
|
| 52 |
|
|
* List of "incrementals" and "stickies" keys
|
| 53 |
|
|
* List possible ebuild version suffixes ("pre", "alpha", ...)
|
| 54 |
|
|
* List ebuild keys ("DEPEND", "LICENSE", ...)
|
| 55 |
|
|
* Check environment for ROOT, use "/" if not found
|
| 56 |
|
|
* Create tmp and var/tmp directories (below ROOT)
|
| 57 |
|
|
* Set 022 umask
|
| 58 |
|
|
* Check to see if the current profile is deprecated
|
| 59 |
|
|
* Set some global settings (PID, bash env, cache dir)
|
| 60 |
|
|
* Read use.defaults and set variables appropriately
|
| 61 |
|
|
* Get OVERLAY
|
| 62 |
|
|
* Lock global settings
|
| 63 |
|
|
* Check for selinux
|
| 64 |
|
|
* Set up some cachedir stuff
|
| 65 |
|
|
* mtime db stuff???
|
| 66 |
|
|
* Get FEATURES
|
| 67 |
|
|
* Set up porttree and bintree dbs
|
| 68 |
|
|
* Get thirdpartymirrors
|
| 69 |
|
|
* Perform PORTAGE_CACHEDIR and PORTAGE_TMPDIR sanity checks
|
| 70 |
|
|
* Get category list from file
|
| 71 |
|
|
* Get package.mask, /etc/portage/package.*
|
| 72 |
|
|
* Get packages file
|
| 73 |
|
|
* Get unmask list, mask list, "revmask"?
|
| 74 |
|
|
* Get ACCEPT_KEYWORDS
|
| 75 |
|
|
* Check that /etc/make.profile is a symlink
|
| 76 |
|
|
* Check for prelink
|
| 77 |
|
|
|
| 78 |
|
|
emerge
|
| 79 |
|
|
------
|
| 80 |
|
|
|
| 81 |
|
|
This script is much more complicated than the ebuild script (portage.py
|
| 82 |
|
|
is just a bit more than 2x as many lines as this script).
|
| 83 |
|
|
|
| 84 |
|
|
* Set environment variable ``PORTAGE_CALLER="emerge"``
|
| 85 |
|
|
* Import portage <-- portage.settings and other vars now exposed
|
| 86 |
|
|
* Check if output should be colored (if not, call ``nocolor()`` ).
|
| 87 |
|
|
* Check portage.settings for ``PORTAGE_NICENESS``; if so, nice portage
|
| 88 |
|
|
* Freeze portdb api (don't know what this does just yet)
|
| 89 |
|
|
* Turn off "noauto" if set in ``FEATURES``
|
| 90 |
|
|
* Set # of merged ebuilds = 0
|
| 91 |
|
|
* Set up lists of valid emerge parameters (params),
|
| 92 |
|
|
actions, and options (all hard-coded lists); also
|
| 93 |
|
|
map short-form of options to long-form versions.
|
| 94 |
|
|
* Process command line (ugly!)
|
| 95 |
|
|
- process short-form to convert to long-form
|
| 96 |
|
|
- Process long-form arguments
|
| 97 |
|
|
- Create list of options (myopts), actions (myactions), & files (myfiles)
|
| 98 |
|
|
- minor fixes ( ``-U`` implies ``-u``, etcetera)
|
| 99 |
|
|
- handle ``--noconfmem`` and ``--debug`` flags
|
| 100 |
|
|
* Define log, exit, countdown, signal handling, help, and "emerge info"
|
| 101 |
|
|
helper functions.
|
| 102 |
|
|
* Check permissions; exit if invalid.
|
| 103 |
|
|
* Start logging (unless ``--pretend`` was set)
|
| 104 |
|
|
* Decide what type of dependencies to use
|
| 105 |
|
|
(self, selective, recurse, deep, or empty)
|
| 106 |
|
|
* Define spinner func
|
| 107 |
|
|
* Define the search class
|
| 108 |
|
|
* Build package digraph
|
| 109 |
|
|
- Define getlist function for world or system
|
| 110 |
|
|
- Define genericdict function
|
| 111 |
|
|
- Define depgraph class (huge)
|
| 112 |
|
|
* Define unmerge function
|
| 113 |
|
|
* Define post_emerge function (finish logging, handle info stuff,
|
| 114 |
|
|
config protection)
|
| 115 |
|
|
* If --debug set, edebug=1 (not sure why two different debug flags set)
|
| 116 |
|
|
* Handle ``emerge sync`` (big if, elif, else construction)
|
| 117 |
|
|
* Handle ``emerge regen``
|
| 118 |
|
|
* Handle ``emerge config``
|
| 119 |
|
|
* Handle ``emerge info``
|
| 120 |
|
|
* Handle ``emerge search``
|
| 121 |
|
|
* Handle ``emerge inject``
|
| 122 |
|
|
* Handle ``emerge unmerge``, ``emerge prune``, and ``emerge clean``
|
| 123 |
|
|
- Call ``unmerge(myaction, myfiles)``
|
| 124 |
|
|
* Handle ``emerge depclean``
|
| 125 |
|
|
* Else: Handle update, system, or just process files
|
| 126 |
|
|
- Do some setup stuff if ``--pretend`` set
|
| 127 |
|
|
- If ``--resume`` and portage.mtimedb has a "resume" key, then update opts, create a "resume" depgraph
|
| 128 |
|
|
- Else, create depgraph, error out if something is wrong
|
| 129 |
|
|
- If ``--pretend``, show depgraph
|
| 130 |
|
|
- Else
|
| 131 |
|
|
+ If ``--buildpkgonly``, check that deps are satisfied
|
| 132 |
|
|
+ If --resume, do merge using ``mydepgraph.merge(portage.mtimedb["resume"]["mergelist"])``
|
| 133 |
|
|
+ Else, handle ``--digest`` or perform merge, the latter with ``mydepgraph.merge(mydepgraph.altlist())``
|
| 134 |
|
|
- Run autoclean
|
| 135 |
|
|
- Run ``post_emerge()``
|
| 136 |
|
|
|
| 137 |
|
|
portage.doebuild()
|
| 138 |
|
|
------------------
|
| 139 |
|
|
|
| 140 |
|
|
Much of what happens during the actual merge of a package happens
|
| 141 |
|
|
in portage's doebuild function. The interface is::
|
| 142 |
|
|
|
| 143 |
|
|
doebuild(myebuild, mydo, myroot, debug=0, listonly=0, fetchonly=0)
|
| 144 |
|
|
|
| 145 |
|
|
where ``myebuild`` is the path to the ebuild to be merged,
|
| 146 |
|
|
``mydo`` is the action to be performed (one of the commands that
|
| 147 |
|
|
follows ``ebuild foo.ebuild ...``), and ``debug``, ``listonly``,
|
| 148 |
|
|
and ``fetchonly`` are flags that can be true (1) or false (0).
|
| 149 |
|
|
|
| 150 |
|
|
Here's what happens:
|
| 151 |
|
|
|
| 152 |
|
|
* Check to make sure the input is valid
|
| 153 |
|
|
* Set a boatload of keys in the ``settings`` instance of the config class
|
| 154 |
|
|
- PORTAGE_DEBUG, ROOT, STARTDIR, EBUILD, O, CATEGORY, FILESDIR,
|
| 155 |
|
|
PF, ECLASSDIR, SANDBOX_LOG, P, PN, PV, PR, PVR, SLOT, BUILD_PREFIX,
|
| 156 |
|
|
PKG_TMPDIR, BUILDDIR, KV, KVERS
|
| 157 |
|
|
* If the action is ``depend``, then return dependency info (eclasses?).
|
| 158 |
|
|
If ``--debug`` was also set, then the deps are printed on the cmd line.
|
| 159 |
|
|
* If necessary (meaning not ``fetch``, ``digest``, or ``manifest``),
|
| 160 |
|
|
create the build directory (``/var/tmp/portage/whatever``) and the
|
| 161 |
|
|
temp directory (and add T to ``settings``). Also create a ccache
|
| 162 |
|
|
directory, if necessary, along with WORKDIR and D.
|
| 163 |
|
|
* If ``unmerge``, call ``unmerge()`` and return.
|
| 164 |
|
|
* Setup logging if requested
|
| 165 |
|
|
* If ``help``, ``clean``, ``setup``, ``prerm``, ``postrm``, ``preinst``,
|
| 166 |
|
|
``postinst``, or ``config``, then pass the action to ``ebuild.sh``
|
| 167 |
|
|
(using "``ebuild.sh mydo,debug,free=1``") and return.
|
| 168 |
|
|
* Generate A from SRC_URI
|
| 169 |
|
|
* Ensure that DISTDIR exists; change permissions for ${DISTDIR}/cvs-src
|
| 170 |
|
|
if necessary.
|
| 171 |
|
|
* Run fetch(); return error on failure.
|
| 172 |
|
|
* Generate digest if ``digest`` in FEATURES or called explicitly (return
|
| 173 |
|
|
in the latter case).
|
| 174 |
|
|
* If ``manifest`` then generate manifest and return.
|
| 175 |
|
|
* Check manifest and die on failure if ``strict`` in FEATURES
|
| 176 |
|
|
* If ``fetch``, return (since already done above, and it must have worked
|
| 177 |
|
|
if we're here)
|
| 178 |
|
|
* Set up an "``actionmap``" of dependencies (i.e., ``unpack`` requires
|
| 179 |
|
|
``dep`` and ``setup``).
|
| 180 |
|
|
* If ``package``, ensure that PKGDIR is created, call
|
| 181 |
|
|
``spawnebuild`` with the current action and the actionmap, and return.
|
| 182 |
|
|
* If ``qmerge``, call merge w/o checking for dependencies, and return.
|
| 183 |
|
|
* If ``merge``, call ``spawnebuild`` with "``install``" and the
|
| 184 |
|
|
``actionmap``, and then call ``merge`` and return.
|
| 185 |
|
|
* Else, die because the action didn't match any of the above. Oops!
|
| 186 |
|
|
|
| 187 |
|
|
ebuild.sh
|
| 188 |
|
|
---------
|
| 189 |
|
|
|
| 190 |
|
|
The ``ebuild.sh`` bash script predominantly gets called by the
|
| 191 |
|
|
``portage.doebuild`` function to handle the bash functions in an
|
| 192 |
|
|
ebuild. Following the bouncing ball....
|
| 193 |
|
|
|
| 194 |
|
|
The first block of code is initialization stuff:
|
| 195 |
|
|
|
| 196 |
|
|
* If not ``depend`` or ``clean``, then
|
| 197 |
|
|
- clean the "successful" file from ${T} if it exists
|
| 198 |
|
|
- Set up logging if desired (including punching a hole in
|
| 199 |
|
|
sandbox for the log file)
|
| 200 |
|
|
- If ${T}/environment exists, source it
|
| 201 |
|
|
* unalias everything
|
| 202 |
|
|
* Set "expand_aliases" and set up ``die`` and ``assert`` aliaes
|
| 203 |
|
|
* Source ``/etc/profile`` on GNU userland systems
|
| 204 |
|
|
* Export proper CC and CXX variables.
|
| 205 |
|
|
* Set up the a hardcoded PATH, tack ${ROOTPATH} to the end of it,
|
| 206 |
|
|
and add ${PREROOTPATH} to the beginning of the PATH if it exists.
|
| 207 |
|
|
* Source helper functions files (``functions.sh`` and
|
| 208 |
|
|
``extra_functions.sh``).
|
| 209 |
|
|
|
| 210 |
|
|
Then a great number of functions are defined:
|
| 211 |
|
|
|
| 212 |
|
|
* Define a do-nothing ``esyslog()`` override function.
|
| 213 |
|
|
* Define a variety of USE flag functions:
|
| 214 |
|
|
``use``, ``has``, ``has_version``, ``best_version``,
|
| 215 |
|
|
``use_with``, and ``use_enable`` functions.
|
| 216 |
|
|
* Define ``diefunc`` function.
|
| 217 |
|
|
* Set umask and a number of installation defaults.
|
| 218 |
|
|
* Define ``check_KV`` function.
|
| 219 |
|
|
* Define the ``keepdir`` function to put ".keep" files in directories
|
| 220 |
|
|
that portage should not automatically remove if empty.
|
| 221 |
|
|
* Define a series of sandbox helper functions.
|
| 222 |
|
|
* Define the ``unpack`` function.
|
| 223 |
|
|
* Define the ``econf`` convenience function. Notice that ``econf``
|
| 224 |
|
|
automatically dies on error(s).
|
| 225 |
|
|
* Define the ``einstall`` convenience function.
|
| 226 |
|
|
* Define ``pkg_setup`` as a placebo function to be overridden.
|
| 227 |
|
|
* Define ``pkg_nofetch``.
|
| 228 |
|
|
* Define minimal, but functional, ``src_unpack`` and ``src_compile``
|
| 229 |
|
|
functions.
|
| 230 |
|
|
* Define ``src_install``, ``pkg_preinst``, ``pkg_postinst``,
|
| 231 |
|
|
``pkg_prerm``, and ``pkg_postrm`` skeleton functions that, when
|
| 232 |
|
|
needed, should be overridden.
|
| 233 |
|
|
* Define the deprecated ``try`` function.
|
| 234 |
|
|
* Define the ``gen_wrapper`` function to generate ``lib/cpp`` and
|
| 235 |
|
|
``/usr/bin/cc`` wrappers.
|
| 236 |
|
|
* Define some of the ``dyn-*`` functions: ``dyn_setup``, ``dyn_unpack``,
|
| 237 |
|
|
``dyn_clean``.
|
| 238 |
|
|
* Define some ``install`` helper functions: ``into``, ``insinto``,
|
| 239 |
|
|
``exeinto``, ``docinto``, ``insopts``, ``diropts``, ``exeopts``,
|
| 240 |
|
|
and ``libopts``. Note that a number of additional helper
|
| 241 |
|
|
functions, such as the ``do*`` functions, are actual executable
|
| 242 |
|
|
scripts in ``/usr/lib/portage/bin``.
|
| 243 |
|
|
* Define a number of abort handler functions: ``abort_handler``,
|
| 244 |
|
|
``abort_compile``, ``abort_unpack``, ``abort_package``, and
|
| 245 |
|
|
``abort_install``.
|
| 246 |
|
|
* Define the rest of the ``dyn_*`` functions: ``dyn_compile``,
|
| 247 |
|
|
``dyn_package``, ``dyn_install``, ``dyn_spec``, ``dyn_rpm``,
|
| 248 |
|
|
and ``dyn_help``.
|
| 249 |
|
|
* Define some eclass debugging functions: ``debug-print``,
|
| 250 |
|
|
``debug-print-function``, and ``debug-print-section``.
|
| 251 |
|
|
* Define the eclass ``inherit`` function.
|
| 252 |
|
|
* Define some eclass helper functions: ``EXPORT_FUNCTIONS``,
|
| 253 |
|
|
``newdepend``, ``newrdepend``, ``newcdepend``, ``newpdepend``,
|
| 254 |
|
|
and ``do_newdepend``.
|
| 255 |
|
|
|
| 256 |
|
|
Finally, the main part of the script is reached:
|
| 257 |
|
|
|
| 258 |
|
|
* If not ``depend`` or ``clean``, then
|
| 259 |
|
|
- cd into the ``/var/tmp/portage/whatever`` directory
|
| 260 |
|
|
- export USER=portage if the effective user ID is "portage"
|
| 261 |
|
|
- Set up distcc and ccache PATHs, environment variables, and
|
| 262 |
|
|
sandbox holes
|
| 263 |
|
|
* Turn on sandbox flag
|
| 264 |
|
|
* Set the S default and unset a handful of variables
|
| 265 |
|
|
* Source the ebuild
|
| 266 |
|
|
* Set the S default if S not defined in the ebuild. (redundant?)
|
| 267 |
|
|
* Set TMP and TMPDIR to be ${T} (otherwise sandbox might error out)
|
| 268 |
|
|
* Set RDEPEND is not set, set it to be DEPEND.
|
| 269 |
|
|
* Add eclass deps to {R,C,P,}DEPEND.
|
| 270 |
|
|
* Loop over arguments passed to ``ebuild.sh``, using a big case statement
|
| 271 |
|
|
- ``nofetch`` -- call ``pkg_nofetch``
|
| 272 |
|
|
- ``prerm``, ``postrm``, ``preinst``, ``postinst``, ``config`` --
|
| 273 |
|
|
turn off sandbox, handle debugging, and call the eponymous
|
| 274 |
|
|
``pkg_*`` function.
|
| 275 |
|
|
- ``unpack``, ``compile``, ``clean``, ``install`` -- handle sandbox
|
| 276 |
|
|
and debugging, then call the eponymous ``dyn_*`` function.
|
| 277 |
|
|
- ``help``, ``clean``, ``setup`` -- Turn off sandbox, handle
|
| 278 |
|
|
debugging, and call the appropriate ``dyn_*`` function.
|
| 279 |
|
|
- ``package``, ``rpm`` -- Turn off sandbox, handle debugging,
|
| 280 |
|
|
and call the eponymous ``dyn_*`` function.
|
| 281 |
|
|
- ``depend`` -- Turn off sandbox and write dependency info
|
| 282 |
|
|
to ``/var/edb/cache/dep/CATEGORY/PF``.
|
| 283 |
|
|
- Else, error that the command wasn't valid.
|
| 284 |
|
|
* If ``clean``, clean that package's temp directory
|
| 285 |
|
|
* Finish by touching "successful" in the package's temp directory.
|
| 286 |
|
|
|