en/glep/glep-0062.txt

GLEP: 62
Title: Optional runtime dependencies via runtime-switchable USE flags
Version: $Revision: 1.1 $
Last-Modified: $Date: 2012/07/11 20:21:27 $
Author: Michał Górny <mgorny@gentoo.org>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 17 Jun 2012
Post-History: 11 Jul 2012


Abstract
========

This GLEP addresses the issue of referencing optional runtime
dependencies in Gentoo packages and ebuilds. It does introduce
a concept of runtime-switchable USE flags to achieve that goal.


Motivation
==========

Optional runtime dependencies are often found in packages installing
various scripts (shell, python, perl). These are not strictly required
for the particular package to work but installing them enables
additional functionality.

Unlike in compiled programs, enabling or disabling those features
(dependencies) does not affect the files installed by the package.
They can be installed and uninstalled independently of the package,
resulting in changes of functionality without a need to rebuild
the package.

Currently such dependencies are usually expressed only through
``pkg_postinst()`` messages. This forces user to manually install
the necessary dependencies, and uninstall them when they are no longer
necessary.

Another solution is to use regular USE flags. Those flags do not strictly
follow the principles of USE flags because they do not affect files
installed by the package and are not entirely effective to the package
(a disabled feature will still be available if necessary dependency is
installed). Additionally, it requires unnecessary rebuilds
of the package in order to change the dependencies.


Specification
=============

The ebuilds aiming to provide features enabled through optional runtime
dependencies should:

1. create regular USE flags for all those features, following
   appropriate specifications for Gentoo ebuilds, and including
   the flags in the ``IUSE`` variable;
2. introduce additional ``IUSE_RUNTIME`` variable listing names of USE
   flags related to optional runtime dependencies (without prefixes
   related to IUSE defaults).

Additionally, the ebuilds must obey the following rules:

1. all flags listed in ``IUSE_RUNTIME`` have to be listed in ``IUSE``
   as well,
2. flags listed in ``IUSE_RUNTIME`` can be referenced in ``RDEPEND``,
   ``PDEPEND`` and ``REQUIRED_USE`` variables,
3. flags listed in ``IUSE_RUNTIME`` must not be referenced in phase
   functions, ``DEPEND``, ``LICENSE`` or ``SRC_URI``,
4. flags listed in ``IUSE_RUNTIME`` can be referenced through USE
   dependencies by other packages' ``DEPEND``, ``RDEPEND``
   and ``PDEPEND`` variables but it is unallowed to request disabling
   those flags (only ``[flag]`` and ``[flag?]`` forms are allowed),
5. flags listed in ``IUSE_RUNTIME`` can be referenced through
   ``has_version`` and ``best_version`` yet the caller must not rely
   upon those flags being disabled.

The package manager should treat flags listed in ``IUSE_RUNTIME``
as regular USE flags, except for the following:

1. enabling or disabling any of the flags must not involve rebuilding
   the package,
2. it should be possible for a package manager to change those flags
   on a installed package without using the original ebuild [1]_,
3. when queried on a installed package, the package manager must
   consider a particular flag enabled only if its dependencies
   are satisfied already [2]_,
4. the flags may be listed in the visual output in a distinct way
   to inform the user that they affect runtime dependencies only.

.. [1] The package manager has to ensure that all relevant information
       is stored in the installed package metadata.
.. [2] The result of this check can be cached when updating the metadata
       of installed package, and it is not strictly required that
       a package manager must ensure that the dependency graph is still
       consistent afterwards.


Rationale
=========

The proposed solution tries to solve the issue of handling runtime
dependencies while reusing the existing infrastructure. Most
importantly, users will be able to reuse the existing tools
and configuration files to enable and disable optional runtime
and build-time dependencies alike.

The remaining reused features include:

- dependency syntax (USE-conditionals),
- ability to use ``REQUIRED_USE``, USE dependencies,
- ability to describe flags in `metadata.xml`,
- global flag names (and descriptions).

Alternative proposed solution involved creating additional ``SDEPEND``
variable. That proposition had the following disadvantages:

- being package-oriented rather than feature-oriented,
- lack of ability to express multiple packages required by a single
  feature,
- lack of ability to express cross-feature dependencies,
- lack of ability to describe features provided by enabled packages,
- necessity of implementing a new user interface parts to control
  the dependencies,
- lack of backwards compatibility.

Those disadvantages could be fixed by either extending dependency
syntax alike Exherbo (grouping, annotations) or using USE flags.
The latter is practically equivalent to this solution yet introduces
another variable unnecessarily.


Reference implementation
========================

In order to support runtime-switchable USE flag changes on installed
packages, a package manager should store the following information
in the installed package metadata:

1. a list of runtime-switchable flags (``IUSE_RUNTIME``),
2. a list of runtime dependencies conditional to runtime-switchable
   flags, not subjected to USE expansion (relevant part of ``RDEPEND``),
3. a list of ``REQUIRED_USE`` constraints relevant to runtime-switchable
   USE flags.

The package manager should be also able to update the list of effective
USE flags in installed package metadata (``USE``) without rebuilding
the package.

The following procedure should apply when a dependency atom is
considered (either as a package dependency or user-requested atom):

1. if the dependency is not satisfied with an installed package,
   perform the dependency resolution on ebuilds as usual (in order to
   install a new package);
2. if runtime-switchable USE updates are disabled, continue to
   the next dependency (with this one satisfied);
3. ``current_use`` = installed package metadata . ``USE``;
4. ``effective_use`` = requested ``USE`` for the package;
5. ``use_changes`` = ``current_use`` xor ``effective_use``;
6. if ``use_changes`` is empty, continue to the next dependency;
7. ``iuse_runtime`` = installed package metadata . ``IUSE_RUNTIME``;
8. ``reg_use_changes`` = ``use_changes`` and not ``iuse_runtime``;
9. if ``reg_use_changes`` is not empty and regular USE updates
   are enabled, perform the dependency resolution on ebuilds (in order
   to rebuild the package);
10. ``run_use_changes`` = ``use_changes`` and ``iuse_runtime``;
11. if ``run_use_changes`` is empty, continue to the next dependency;
12. ``required_use`` = installed package metadata . ``REQUIRED_USE``;
13. perform a check for ``required_use`` constraints being satisfied
    by ``effective_use``;
14. ``run_rdepend`` = installed package metadata . ``RDEPEND`` (with
    unexpanded ``IUSE_RUNTIME`` conditionals);
15. perform the dependency resolution for ``run_rdepend`` with
    ``effective_use``,
16. queue the package for runtime-switchable USE update.

Furthermore, after installing all dependencies introduced by the above
procedure (``run_rdepend`` resolution), if a package was queued for
runtime-switchable USE update, the package manager should write a new
value for ``USE`` key in installed package metadata.


Backwards compatibility
=======================

Package managers not implementing this GLEP will consider
the ``IUSE_RUNTIME`` variable as an irrelevant bash variable and treat
runtime-switchable USE flags as regular USE flags. The dependency tree
will still be consistent yet packages may be rebuilt unnecessarily.


Copyright
=========

This document has been placed in the public domain.
1	antarus	1.2	GLEP: 62
2	antarus	1.1	Title: Optional runtime dependencies via runtime-switchable USE flags
3	antarus	1.2	Version: $Revision: 1.1 $
4			Last-Modified: $Date: 2012/07/11 20:21:27 $
5	antarus	1.1	Author: Michał Górny <mgorny@gentoo.org>
6			Status: Draft
7			Type: Standards Track
8			Content-Type: text/x-rst
9			Created: 17 Jun 2012
10	antarus	1.2	Post-History: 11 Jul 2012
11	antarus	1.1
12
13			Abstract
14			========
15
16			This GLEP addresses the issue of referencing optional runtime
17			dependencies in Gentoo packages and ebuilds. It does introduce
18			a concept of runtime-switchable USE flags to achieve that goal.
19
20
21			Motivation
22			==========
23
24			Optional runtime dependencies are often found in packages installing
25			various scripts (shell, python, perl). These are not strictly required
26			for the particular package to work but installing them enables
27			additional functionality.
28
29			Unlike in compiled programs, enabling or disabling those features
30			(dependencies) does not affect the files installed by the package.
31			They can be installed and uninstalled independently of the package,
32			resulting in changes of functionality without a need to rebuild
33			the package.
34
35			Currently such dependencies are usually expressed only through
36			``pkg_postinst()`` messages. This forces user to manually install
37			the necessary dependencies, and uninstall them when they are no longer
38			necessary.
39
40			Another solution is to use regular USE flags. Those flags do not strictly
41			follow the principles of USE flags because they do not affect files
42			installed by the package and are not entirely effective to the package
43			(a disabled feature will still be available if necessary dependency is
44			installed). Additionally, it requires unnecessary rebuilds
45			of the package in order to change the dependencies.
46
47
48			Specification
49			=============
50
51			The ebuilds aiming to provide features enabled through optional runtime
52			dependencies should:
53
54			1. create regular USE flags for all those features, following
55			appropriate specifications for Gentoo ebuilds, and including
56			the flags in the ``IUSE`` variable;
57			2. introduce additional ``IUSE_RUNTIME`` variable listing names of USE
58			flags related to optional runtime dependencies (without prefixes
59			related to IUSE defaults).
60
61			Additionally, the ebuilds must obey the following rules:
62
63			1. all flags listed in ``IUSE_RUNTIME`` have to be listed in ``IUSE``
64			as well,
65			2. flags listed in ``IUSE_RUNTIME`` can be referenced in ``RDEPEND``,
66			``PDEPEND`` and ``REQUIRED_USE`` variables,
67			3. flags listed in ``IUSE_RUNTIME`` must not be referenced in phase
68			functions, ``DEPEND``, ``LICENSE`` or ``SRC_URI``,
69			4. flags listed in ``IUSE_RUNTIME`` can be referenced through USE
70			dependencies by other packages' ``DEPEND``, ``RDEPEND``
71			and ``PDEPEND`` variables but it is unallowed to request disabling
72			those flags (only ``[flag]`` and ``[flag?]`` forms are allowed),
73			5. flags listed in ``IUSE_RUNTIME`` can be referenced through
74			``has_version`` and ``best_version`` yet the caller must not rely
75			upon those flags being disabled.
76
77			The package manager should treat flags listed in ``IUSE_RUNTIME``
78			as regular USE flags, except for the following:
79
80			1. enabling or disabling any of the flags must not involve rebuilding
81			the package,
82			2. it should be possible for a package manager to change those flags
83			on a installed package without using the original ebuild [1]_,
84			3. when queried on a installed package, the package manager must
85			consider a particular flag enabled only if its dependencies
86			are satisfied already [2]_,
87			4. the flags may be listed in the visual output in a distinct way
88			to inform the user that they affect runtime dependencies only.
89
90			.. [1] The package manager has to ensure that all relevant information
91			is stored in the installed package metadata.
92			.. [2] The result of this check can be cached when updating the metadata
93			of installed package, and it is not strictly required that
94			a package manager must ensure that the dependency graph is still
95			consistent afterwards.
96
97
98			Rationale
99			=========
100
101			The proposed solution tries to solve the issue of handling runtime
102			dependencies while reusing the existing infrastructure. Most
103			importantly, users will be able to reuse the existing tools
104			and configuration files to enable and disable optional runtime
105			and build-time dependencies alike.
106
107			The remaining reused features include:
108
109			- dependency syntax (USE-conditionals),
110			- ability to use ``REQUIRED_USE``, USE dependencies,
111			- ability to describe flags in `metadata.xml`,
112			- global flag names (and descriptions).
113
114			Alternative proposed solution involved creating additional ``SDEPEND``
115			variable. That proposition had the following disadvantages:
116
117			- being package-oriented rather than feature-oriented,
118			- lack of ability to express multiple packages required by a single
119			feature,
120			- lack of ability to express cross-feature dependencies,
121			- lack of ability to describe features provided by enabled packages,
122			- necessity of implementing a new user interface parts to control
123			the dependencies,
124			- lack of backwards compatibility.
125
126			Those disadvantages could be fixed by either extending dependency
127			syntax alike Exherbo (grouping, annotations) or using USE flags.
128			The latter is practically equivalent to this solution yet introduces
129			another variable unnecessarily.
130
131
132			Reference implementation
133			========================
134
135			In order to support runtime-switchable USE flag changes on installed
136			packages, a package manager should store the following information
137			in the installed package metadata:
138
139			1. a list of runtime-switchable flags (``IUSE_RUNTIME``),
140			2. a list of runtime dependencies conditional to runtime-switchable
141			flags, not subjected to USE expansion (relevant part of ``RDEPEND``),
142			3. a list of ``REQUIRED_USE`` constraints relevant to runtime-switchable
143			USE flags.
144
145			The package manager should be also able to update the list of effective
146			USE flags in installed package metadata (``USE``) without rebuilding
147			the package.
148
149			The following procedure should apply when a dependency atom is
150			considered (either as a package dependency or user-requested atom):
151
152			1. if the dependency is not satisfied with an installed package,
153			perform the dependency resolution on ebuilds as usual (in order to
154			install a new package);
155			2. if runtime-switchable USE updates are disabled, continue to
156			the next dependency (with this one satisfied);
157			3. ``current_use`` = installed package metadata . ``USE``;
158			4. ``effective_use`` = requested ``USE`` for the package;
159			5. ``use_changes`` = ``current_use`` xor ``effective_use``;
160			6. if ``use_changes`` is empty, continue to the next dependency;
161			7. ``iuse_runtime`` = installed package metadata . ``IUSE_RUNTIME``;
162			8. ``reg_use_changes`` = ``use_changes`` and not ``iuse_runtime``;
163			9. if ``reg_use_changes`` is not empty and regular USE updates
164			are enabled, perform the dependency resolution on ebuilds (in order
165			to rebuild the package);
166			10. ``run_use_changes`` = ``use_changes`` and ``iuse_runtime``;
167			11. if ``run_use_changes`` is empty, continue to the next dependency;
168			12. ``required_use`` = installed package metadata . ``REQUIRED_USE``;
169			13. perform a check for ``required_use`` constraints being satisfied
170			by ``effective_use``;
171			14. ``run_rdepend`` = installed package metadata . ``RDEPEND`` (with
172			unexpanded ``IUSE_RUNTIME`` conditionals);
173			15. perform the dependency resolution for ``run_rdepend`` with
174			``effective_use``,
175			16. queue the package for runtime-switchable USE update.
176
177			Furthermore, after installing all dependencies introduced by the above
178			procedure (``run_rdepend`` resolution), if a package was queued for
179			runtime-switchable USE update, the package manager should write a new
180			value for ``USE`` key in installed package metadata.
181
182
183			Backwards compatibility
184			=======================
185
186			Package managers not implementing this GLEP will consider
187			the ``IUSE_RUNTIME`` variable as an irrelevant bash variable and treat
188			runtime-switchable USE flags as regular USE flags. The dependency tree
189			will still be consistent yet packages may be rebuilt unnecessarily.
190
191
192			Copyright
193			=========
194
195			This document has been placed in the public domain.