en/glep/glep-0035.txt

GLEP: 35
Title: Automated consistency check for ebuilds
Version: $Revision: 1.1 $
Last-Modified: $Date: 2005/03/12 20:26:01 $
Author: Adrian Lambeck <adrian@basicsedv.de>,
Status: Deferred
Type: Standards Track
Content-Type: text/x-rst
Created: 12-Mar-2005
Post-History: 12-Mar-2005


Abstract
========

This proposal is meant to enhance productivity for Gentoo developers.
It aims to reduce the number of trivial bugs by automatically detecting them
through a consistency check that is performed before checking and on a regular
basis through the whole tree.
Why bother with trivial bugs when automated tests find them ?
Save time and improve quality !


Motivation
==========

When browsing `bugs.gentoo.org`_ you will find Bugs that take away a good
amount of scarce developing time that could be used otherwise.  These are
trivial bugs, i.e. wrong SRC_URI or cycles in DEPEND. Even worst - these bugs
are sometimes reported several times so that they need to be marked as
dublicates.  Bugs of that kind are easy to find and easy to fix. By using
automatic checks on a regular schedule these bugs can be found.  Users have to
be asked NOT to commit these bugs to `bugs.gentoo.org`_.  So there will
(hopefully) be fewer bugs that need to be checked and assigned and they might
get fixed faster.

.. _bugs.gentoo.org: http://bugs.gentoo.org

The Bugs found should be kept in an automatically generated list so that users
can see that the problem has been caught and that it is being worked on.


Specification
=============

Checks need to be performed for every ebuild.

A report needs to be generated

        - links to the specific problem need to be included
        - reports need to be send to the groups responsible
        
Checks could be:

        - cycles within DEPEND
        - invalid SRC_URI
        - "non-official" USE Flags
        - Packages within DEPEND that are "*" for the arch specified
        - broken shell scripts with invalid or missing commands
        - inheritance of eclasses
        - ...

There might be other checks and tests that should be run
that have not come to my mind yet. Also I might have suggested something that
is not useful at all.

If there are major problems (needs to be defined) within an ebuild a possible
action could be to disable the ebuild (with ``"-*"``,)  perhaps, and send a
mail to the maintainer.

These kind of errors are not always the fault of the developers.

There should be no compilation or something like that. If an ebuild fails to
build somewhere then the user should file it as a bug as usual.


Implementation
==============

The functionality described could be implemented in three ways:

        1. On the developers machine ("client") where it is run before checking
                only for the ebuilds that changed. (client does not fit here because 
                the server and client should not communicate with each other at all)
        
        2. On the server where checks are run, i.e. once a week.
        
        3. On the "client" AND server 


        Of course there are cons and pros (what came to my mind so far)
        
        1.
                pro: 
                        - the tree can not become inconsistent in the first place (? see contra)
                        - once an ebuild is checked there is no need to do this again
                        - no dedicated machine necessary
                        - generate traffic only once on one machine
                        - errors that are caught here do not bother later on
                
                contra:
                        - the consistency is based on the tool installed 
                                (what happens when different devs use different versions ?)
                        - what happens when the ebuild layout changes and some ebuilds
                                do not get updated ?
                         
        2.
                pro:
                        - Properties of other ebuilds might change that fit while writing an ebuild
                
                contra:
                        - the errors are found when the ebuild is already in CVS
                        - the whole tree needs to be checked
                        - possibly creates a lot of traffic on every run
                                (-> is there an FTP equivalent to HTTP`s HEAD ?)
                                
        3. see 1. and 2.

My favorite is 3 . All properties are checked before check-in and 
the properties that change might be checked on a regular basis on the server.
Only solution 3 brings the best from 1 and 2 together while delivering the best result.

I never had a look at portage source but I can imagine that there is a library
that has everything that a developer needs to "query" ebuilds. If not, this
would be a reason for another GLEP (?).

For performance I would use a database (on the server) to store the whole tree before
running the checks. This is not necessary for the "client".


Backwards Compatibility
=======================

Not a problem for this GLEP.
1	GLEP: 35
2	Title: Automated consistency check for ebuilds
3	Version: $Revision: 1.1 $
4	Last-Modified: $Date: 2005/03/12 20:26:01 $
5	Author: Adrian Lambeck <adrian@basicsedv.de>,
6	Status: Deferred
7	Type: Standards Track
8	Content-Type: text/x-rst
9	Created: 12-Mar-2005
10	Post-History: 12-Mar-2005
11
12
13	Abstract
14	========
15
16	This proposal is meant to enhance productivity for Gentoo developers.
17	It aims to reduce the number of trivial bugs by automatically detecting them
18	through a consistency check that is performed before checking and on a regular
19	basis through the whole tree.
20	Why bother with trivial bugs when automated tests find them ?
21	Save time and improve quality !
22
23
24	Motivation
25	==========
26
27	When browsing `bugs.gentoo.org`_ you will find Bugs that take away a good
28	amount of scarce developing time that could be used otherwise. These are
29	trivial bugs, i.e. wrong SRC_URI or cycles in DEPEND. Even worst - these bugs
30	are sometimes reported several times so that they need to be marked as
31	dublicates. Bugs of that kind are easy to find and easy to fix. By using
32	automatic checks on a regular schedule these bugs can be found. Users have to
33	be asked NOT to commit these bugs to `bugs.gentoo.org`_. So there will
34	(hopefully) be fewer bugs that need to be checked and assigned and they might
35	get fixed faster.
36
37	.. _bugs.gentoo.org: http://bugs.gentoo.org
38
39	The Bugs found should be kept in an automatically generated list so that users
40	can see that the problem has been caught and that it is being worked on.
41
42
43	Specification
44	=============
45
46	Checks need to be performed for every ebuild.
47
48	A report needs to be generated
49
50	- links to the specific problem need to be included
51	- reports need to be send to the groups responsible
52
53	Checks could be:
54
55	- cycles within DEPEND
56	- invalid SRC_URI
57	- "non-official" USE Flags
58	- Packages within DEPEND that are "*" for the arch specified
59	- broken shell scripts with invalid or missing commands
60	- inheritance of eclasses
61	- ...
62
63	There might be other checks and tests that should be run
64	that have not come to my mind yet. Also I might have suggested something that
65	is not useful at all.
66
67	If there are major problems (needs to be defined) within an ebuild a possible
68	action could be to disable the ebuild (with ``"-*"``,) perhaps, and send a
69	mail to the maintainer.
70
71	These kind of errors are not always the fault of the developers.
72
73	There should be no compilation or something like that. If an ebuild fails to
74	build somewhere then the user should file it as a bug as usual.
75
76
77	Implementation
78	==============
79
80	The functionality described could be implemented in three ways:
81
82	1. On the developers machine ("client") where it is run before checking
83	only for the ebuilds that changed. (client does not fit here because
84	the server and client should not communicate with each other at all)
85
86	2. On the server where checks are run, i.e. once a week.
87
88	3. On the "client" AND server
89
90
91	Of course there are cons and pros (what came to my mind so far)
92
93	1.
94	pro:
95	- the tree can not become inconsistent in the first place (? see contra)
96	- once an ebuild is checked there is no need to do this again
97	- no dedicated machine necessary
98	- generate traffic only once on one machine
99	- errors that are caught here do not bother later on
100
101	contra:
102	- the consistency is based on the tool installed
103	(what happens when different devs use different versions ?)
104	- what happens when the ebuild layout changes and some ebuilds
105	do not get updated ?
106
107	2.
108	pro:
109	- Properties of other ebuilds might change that fit while writing an ebuild
110
111	contra:
112	- the errors are found when the ebuild is already in CVS
113	- the whole tree needs to be checked
114	- possibly creates a lot of traffic on every run
115	(-> is there an FTP equivalent to HTTP`s HEAD ?)
116
117	3. see 1. and 2.
118
119	My favorite is 3 . All properties are checked before check-in and
120	the properties that change might be checked on a regular basis on the server.
121	Only solution 3 brings the best from 1 and 2 together while delivering the best result.
122
123	I never had a look at portage source but I can imagine that there is a library
124	that has everything that a developer needs to "query" ebuilds. If not, this
125	would be a reason for another GLEP (?).
126
127	For performance I would use a database (on the server) to store the whole tree before
128	running the checks. This is not necessary for the "client".
129
130
131	Backwards Compatibility
132	=======================
133
134	Not a problem for this GLEP.