/[gentoo]/xml/htdocs/doc/en/articles/autotools-practices.xml
Gentoo

Contents of /xml/htdocs/doc/en/articles/autotools-practices.xml

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.4 - (hide annotations) (download) (as text)
Thu Feb 23 17:52:06 2012 UTC (2 years, 10 months ago) by swift
Branch: MAIN
CVS Tags: HEAD
Changes since 1.3: +3 -3 lines
File MIME type: application/xml
Mark article as... article. Going against policy and not bumping date as that would confuse users. Thanks to Agostino Sarubbo for noticing/notifying

1 rane 1.1 <?xml version='1.0' encoding="UTF-8"?>
2     <!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
3 swift 1.4 <!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/articles/autotools-practices.xml,v 1.3 2011/09/04 17:53:41 swift Exp $ -->
4 rane 1.1
5 swift 1.4 <guide disclaimer="articles">
6 rane 1.1 <title>Best practices with autotools</title>
7    
8     <author title="Author">
9     <mail link="flameeyes@gentoo.org">Diego Pettenò</mail>
10     </author>
11    
12     <abstract>
13     This article covers some of the most common errors people make when using
14     autotools and ways to achieve better results.
15     </abstract>
16    
17     <!-- The content of this document is licensed under the CC-BY-SA license -->
18     <!-- See http://creativecommons.org/licenses/by-sa/2.5 -->
19     <license/>
20    
21 swift 1.4 <version>2</version>
22 rane 1.1 <date>2005-12-16</date>
23    
24     <chapter>
25     <title>Best practices with autotools</title>
26     <section>
27     <body>
28    
29     <p>
30     The core of GNU's compile chain -- the set of tools used to build GNU software
31     packages -- is the so-called "autotools," a term that refers to the autoconf
32     and automake programs, as well as libtool, autoheader, pkg-config, and
33     sometimes gettext. These tools let you compile GNU software on a wide variety
34     of platforms and Unix and Unix-like operating systems, providing developers a
35     framework to check for the presence of the libraries, functions, and tools that
36     they want to use. While autotools are great in the hands of an experienced
37     developer, they can be quite a handful for the first-time user, and it's not so
38     rare that packages are shipped with working-but-broken autotools support. This
39     article will cover some of the most common errors people make when using
40     autotools and ways to achieve better results.
41     </p>
42    
43     <p>
44     Regardless of anyone's opinion about them, we currently have no valid
45     alternative for autotools. Projects such as Scons are not as portable as
46     autotools, and they don't embody enough knowledge to be useful yet. We have
47     tons of automatic checks with autotools, and a lot of libraries come with an m4
48     library with macros to check for their presence.
49     </p>
50    
51     <p>
52     The basic structure of an autotooled project is simple. Autoconf takes help of
53     an <path>aclocal.m4</path> file (created by aclocal using the m4 libraries on
54     its search path and <path>acinclude.m4</path> file) to parse the
55     <path>configure.ac</path> file (formerly <path>configure.in</path>) and
56     transform it into a "configure" script. For every directory there should exist
57     an <path>Makefile.am</path>, which automake uses to create Makefile.in
58     templates. The Makefile.in templates are then processed and transformed by the
59     <path>configure</path> script into real Makefiles. You can avoid avoid using
60     automake and just write your own Makefile.in files, but this is quite complex,
61     and you lose a few features of autotools.
62     </p>
63    
64     <p>
65     In a <path>configure.ac</path> file you can use macros you define yourself, the
66     default ones provided by autoconf and aclocal, or external macros provided, for
67     instance, by other packages. In such a case aclocal will create the
68     <path>aclocal.m4</path> file adding the library files it finds on the system's
69     library with the defined macros; this is a critical step to have a working
70     autotooled project, as we'll see in a moment.
71     </p>
72    
73     <p>
74     A <path>Makefile.am</path> is mainly a declaration of intents: you can fill
75     some targets variables with the name of the targets you want to build. These
76     variables are structured in a format like placetoinstall_TYPEOFTARGET. The
77     place is the location in a hierarchical Unix filesystem (bin, lib, include,
78     ...), a non-used keyword that can be defined with an arbitrary path (using the
79     keyworddir variable), or the special keyword noinst that marks the targets that
80     need not to be installed (for example private headers, or static libraries used
81     during build). After naming the target, you can use the name (replacing dots
82     with underscores) as the prefix for the variables that affects its build. In
83     this way you can provide special CFLAGS, LDFLAGS, and LDADD variables used
84     during the build of a single target, instead of changing them for all the
85     targets. You can also use variables collected during configure phase, if you
86     passed them to the AC_SUBST macro in <path>configure.ac</path>, so that they
87     are replaced inside makefiles. Also, though defining CFLAGS and LDFLAGS on a
88     per-target basis seems useful, adding static flags in Makefile.am is a bad
89     thing for portability, as you can't tell if the compiler you're using supports
90     them, or if you really need them (-ldl put in LDFLAGS is a good example of a
91     flag needed on Linux but not on FreeBSD); in such cases you should use
92     <path>configure.ac</path> to add these flags.
93     </p>
94    
95     <p>
96     The most commonly used macros in configure.ac are AC_CHECK_HEADERS,
97 nightmorph 1.2 AC_CHECK_FUNCS, and AC_CHECK_LIB, which test for the presence of, respectively,
98     some header files, some library functions, and a given library (with a specific
99     function in it). They are important for portability as they provides a way to
100     check which headers are present and which are not (for example system headers
101     that have different locations in different operating systems), and to check
102     whether a function is present in the system library (asprintf() is missing in
103     OpenBSD for example, while it's present on GNU C library and FreeBSD), and
104     finally to check for the presence of some third-party library or to see if a
105     specific link to a library is needed to get some functions (for example
106     dlopen() function is in libdl library on GNU systems, while it's provided by
107     the system's C library on FreeBSD).
108 rane 1.1 </p>
109    
110     <p>
111     Along with testing for the presence or absence of functions or headers (and
112     sometimes libraries) you usually need to change the code's path (for example to
113     avoid the use of missing functions, or to define a drop-in replacement for
114     them). Autoconf is commonly coupled with another tool, autoheader, which
115     creates a <path>config.h.in</path> template, used by configure script to create
116     a <path>config.h</path> header in which are defined a few preprocessor macros
117     in form of HAVE_givenfunction or HAVE_givenheader_H which can be tested with
118     #ifdef/#ifndef directives inside a C or C++ source file to change the code
119     according to the features present.
120     </p>
121    
122     <p>
123     Here are some practices to keep in mind to help you use autotools to create the
124     most portable code possible.
125     </p>
126    
127     <p>
128     <b>The config.h header file should be considered to be an internal header
129     file</b>, so it should be used just by the single package in which it's
130     created. You should avoid editing the <path>config.h.in</path> template to add
131     your own code there, as this requires you to manually update it according to
132     the <path>configure.ac</path> you're writing.
133     </p>
134    
135     <p>
136     Unfortunately a few projects, such as Net-SNMP, export this header file with
137     other libraries' headers, which requires any projects that use their libraries
138     to include them (or provide their own copy of the internal Net-SNMP
139     structures). This is a bad thing, as the autotools structure of a library
140     project should be invisible to software using it (which might not use autotools
141     at all). Also, changes in autotools behavior are anything but rare, so you can
142     have two identical checks with different results due to changes in the way they
143     are executed. If you need to define your own wrappers or replacements in case
144     something is not in the environment you're compiling for, you should do that in
145     private headers that do not get installed (declared as noinst_HEADERS in
146     <path>Makefile.am</path> files).
147     </p>
148    
149     <p>
150     <b>Always provide the m4 files you used</b>. As autotools have been in use for
151     years, many packages (for example libraries) that can be reused by other
152     programs provide an m4 library file in <path>/usr/share/aclocal</path> that
153     makes it possible to check for their presence (for example using the -config
154     scripts) with a simple macro call. These files are used by aclocal to create
155     the <path>aclocal.m4</path> file, and they usually are present on the
156     developers' systems where aclocal is executed to create the release, but when
157     they are for optional dependencies, they can be missing on users' systems.
158     While this is usually not a problem, because users rarely executes aclocal,
159     it's a problem for source distributions, such as Gentoo, where sometimes you
160     need to patch a Makefile.am or the <path>configure.ac</path> and then re-run
161     autoconf without having all the optional dependencies installed (or having
162     different versions, which can be incompatible or bugged, of the same m4 file).
163     </p>
164    
165     <p>
166     To avoid this problem, you should create an m4 subdirectory in your package's
167     directory and then put there the m4 library files you are using. You must then
168     call aclocal with aclocal -I m4 options to search in that directory before the
169     system library. You can then choose whether to put that directory under
170     revision control (CVS, SVN, or whatever else you are using) or just create it
171     for the releases. The latter case is the bare minimum requirement for a
172     package. It minimizes the amount of revision-controlled code and ensures that
173     you're always using the latest m4 version, but has the drawback that anyone who
174     checks out your repository won't be able to execute autoconf without having to
175     look on a release tarball to take the m4 from (and that might not work, as you
176     can have updated the configure.ac to suit a newer macro or added more
177     dependencies). On the other hand, putting the m4 directory under revision
178     control sometimes tempts the developers to change the macros to suit their
179     needs. Although this seems logical, as the m4 files are under your revision
180     control, it will upset many package maintainers, as sometimes new versions of
181     m4 files fix bugs or support newer options and installation paths (for example
182     multilib setups), and having the m4 files modified makes it impossible to just
183     replace them with updated versions. It also mean that when you're going to
184     update an m4 file you must redo the modification against the original.
185     </p>
186    
187     <p>
188     m4 files are always a problem to work with. They must replicate almost the same
189     code from library to library (depending on the way you need to provide
190     CFLAGS/LDFLAGS: with tests or with a -config script). To avoid this problem,
191     the GNOME and FreeDesktop projects developed a tool called pkg-config, which
192     provides both an executable binary and an m4 file to include in configure.ac
193     files, and lets developers check for the presence of a given library (and/or
194     package), provided that the package itself installed a pkg-config .pc data
195     file. This approach simplifies the work of maintaining configure.ac scripts,
196     and requires a lot less time to be processed during execution of configure
197     script, as it uses the information provided by the installed package itself
198     instead of just trying if it's present. On the other hand, this approach means
199     that an error the developers make concerning a dependency can break the user
200     program, as they just hardcode the compiler and linker flags in the data file
201     and the configure script doesn't actually check whether the library works.
202     Fortunately, this doesn't happen too often.
203     </p>
204    
205     <p>
206     To create the configure file, you need PKG_CHECK_MODULES, contained in the
207     <path>pkg.m4</path> library. You should add that file to your m4 directory. If
208     pkg-config dependency is mandatory (as the tool is run by the configure script)
209     you can't be sure that the m4 file you are using is the same as one on users'
210     systems, nor you can be sure that it does not include extra bugs, as it can be
211     older than yours.
212     </p>
213    
214     <p>
215     <b>Always check for the libraries you're going to link to</b>, if you have them
216     as mandatory dependencies. Usually autoconf macros or pkg-config data files
217     define prerequisite libraries that you need to successfully link to your
218     library. Also, some functions that are in extra libraries in some systems
219     (like dlopen() in libdl on Linux and Mac OS X) can be in the libc of another
220     system (the same function is in libc on FreeBSD). In these cases you need to
221     check whether the function can be found without linking to anything, or if you
222     need to use a specific library (for example to avoid linking to a non-existent
223     libdl that would fail where it's not needed).
224     </p>
225    
226     <p>
227     <b>Be careful with GNU extensions</b>. One of the things that makes portability
228     a big pain is the use of extension functions, which are provided by GNU libc
229     but aren't present on other C libraries like BSD's or uClibc. When you use such
230     functions, you should always provide a "drop-in replacement," a function that
231     can provide the same functionality as the library function, maybe with less
232     performance or security, which can be used when the extension function is not
233     present on system's C library. Those functions must be protected by a #ifdef
234     HAVE_function ... #endif block, so that they don't get duplicated when they are
235     already present. Make sure that these functions are not exported by the library
236     to the external users; they should be declared inside an internal header, to
237     avoid breaking other libraries that may be doing similar tricks.
238     </p>
239    
240     <p>
241     <b>Avoid compiling OS-specific code when not needed</b>. When a program
242     optionally supports specific libraries or specific operating systems, it's not
243     rare to have entire source files that are specific to that code path. To avoid
244     compiling them when they're not needed, use the AM_CONDITIONAL macro inside a
245     configure.ac file. This automake macro (usable only if you're using automake to
246     build the project) allows you to define if .. endif blocks inside a Makefile.am
247     file, inside which you can set special variables. You can, for example, add a
248     "platformsrcs" variable that you set to the right source file for the platform
249     to build for, then use in a _SOURCES variable.
250     </p>
251    
252     <p>
253     However, there are two common errors developers make when using AM_CONDITIONAL.
254     The first is the use of AM_CONDITIONAL in an already conditional branch (for
255     example under an info or in a case switch), which leads to automake complaining
256     about a conditional defined only conditionally (AM_CONDITIONAL must be called
257     on global scope, out of every if block, so you must define a variable to
258     contain the status of the conditions and then test against when calling the
259     AM_CONDITIONAL). The other one is that you can't change the targets' variables
260     directly, and you must define "commodity" variables, whose results empty out of
261     the conditional, to add or remove source files and targets.
262     </p>
263    
264     <p>
265     Many projects, to avoid compiling code for specific code paths, add the entire
266     files in #ifdef ... #endif preprocessor conditionals. While this usually works,
267     it makes the code ugly and error-prone, as a single statement out of the
268     conditional block can be compiled where the source file is not needed. It also
269     misleads users sometimes, as the source files seem to be compiled in situations
270     where they don't make sense.
271     </p>
272    
273     <p>
274     <b>Be smart in looking for operating system or hardware platform</b>. Sometimes
275     you need to search for a specific operating system or hardware platform. The
276     right way to do this depends on where you need to know this. If you must know
277     it to enable extra tests on configure, or you must add extra targets on
278     makefiles, you must do the check in configure.ac. On the other hand, if the
279     difference must be known in a source file, for example to enable an optional
280     asm-coded function, you should rely directly on the compiler/preprocessor, so
281     you should use #ifdef directives with the default macros enabled on the target
282     platform (for example __linux__, __i386__, _ARC_PPC, __sparc__, _FreeBSD_ and
283     __APPLE__).
284     </p>
285    
286     <p>
287     <b>Don't run commands in configure.ac</b>. If you need to check for hardware or
288     operating system in a configure.ac, you should avoid using the uname command,
289     despite this being one of the most common way to do such a test. This is
290     actually an error, as this breaks crosscompilation. Autotools supports
291     crosscompile projects from one machine to another using hosts definitions:
292     strings in the form "hardware-vendor-os" (actually, "hardware-vendor-os-libc"
293     when GNU libc is used), such as i686-pc-linux-gnu and
294     x86_64-unknown-freebsd5.4. CHOST is the host definition for the system you're
295     compiling the software for, CBUILD is the host definition for the system you're
296     compiling on; when CHOST and CBUILD differ, you're crosscompiling.
297     </p>
298    
299     <p>
300     In the examples above, the first host definition shows an x86-like system, with
301     a pentium2-equivalent (or later) processor, running a Linux kernel with a GNU
302     libc (usually this refers to a GNU/Linux system). The second refers to an AMD64
303     system with a FreeBSD 5.4 operating system. (For a GNU/kFreeBSD system, which
304     uses FreeBSD kernel and GNU libc, the host definition is
305     hw-unknown-freebsd-gnu, while for a Gentoo/FreeBSD, using FreeBSD's kernel and
306     libc, but with Gentoo framework, the host definition is hw-gentoo-freebsd5.4.)
307     By using $host and $build variables inside a configure.ac script you can enable
308     or disable specific features based on the operating system or on the hardware
309     platform you're compiling to or on.
310     </p>
311    
312     <p>
313     <b>Don't abuse "automagic" dependencies</b>. One of the most useful features of
314     autotools are the automatic checks for the presence of a library, which are
315     often used to automatically enable support for extra dependencies and such.
316     However, abusing this feature makes the build of a package a bit of a problem.
317     While this is quite useful for first-time users, and although most of the
318     projects having complex dependencies (such as multimedia programs like xine and
319     VLC) use a plugin-based framework that allows them to avoid most of the
320     breakages, "automagic" dependencies are a great pain for packagers, especially
321     ones working on source-based distributions such as Gentoo and ports-like
322     frameworks. When you build something with automagical dependencies you enable
323     the functions supported by the libraries found on the system on which the
324     configure script is run. This means that the output binaries might not work on
325     a system that shares the same base packages but misses one extra library, for
326     example. Also, you can't tell the exact dependencies of a package, as some
327     might be optional and not be built when the libraries are not present.
328     </p>
329    
330     <p>
331     To avoid this, autoconf allows you to add --enable/--disable and
332     --with/--without options to configure scripts. With such options you can
333     forcefully enable or disable a specific option (such as the support for an
334     extra library or for a specific feature), and leave the default to automatic
335     tests.
336     </p>
337    
338     <p>
339     Unfortunately, many developers misunderstand the meaning of the two parameters
340     of the functions used to add those options (AC_ARG_ENABLE and AC_ARG_WITH).
341     They represent the code to execute when a parameter is passed and when one is
342     not. Many developers mistakenly think that the two parameters define the code
343     to execute when the feature is enabled and when is disabled. While this usually
344     works when you pass a parameter just to change the default behavior, many
345     source-based distributions pass parameters also to confirm the default
346     behavior, which leads to errors (features explicitly requested missing). Being
347     able to disable optional features if they don't add dependencies (think of OSS
348     audio support on Linux) is always a good thing for users, who can avoid
349     building extra code if they don't plan to use it, and prevents maintainers from
350     doing dirty caching tricks to enable or disable features as their users
351     request.
352     </p>
353    
354     <p>
355     While autotools were a big problem for both developers and maintainers because
356     there are different incompatible versions that do not get along well together
357     (since they install in the same places, with the same names) and which are used
358     in different combinations, the use of autotools saves maintainers from doing
359     all sorts of dirty tricks to compile software. If you look at ebuild from
360     Gentoo's portage, the few that do not use autotools are the more complex ones,
361     as they need to check variables on very different setups (we can or not have
362     NPTL support; we can be on Linux, FreeBSD, or Mac OS X; we can be using GLIBC
363     or another libc; and so on), while autotools usually take care of that on their
364     own. It's also true that many patches applied by maintainers are to fix broken
365     autotools script in upstream sources, but this is just a little problem
366     compared to the chaos of using special build systems that don't work at all
367     with little environmental changes.
368     </p>
369    
370     <p>
371     Autotools can be quite tricky for newcomers, but when you start using them on a
372     daily basis you find it's a lot easier than having to deal with manual
373     makefiles or other strange build tools such as imake or qmake, or even worse,
374     special autotools-like build scripts that try to recognize the system they are
375     building on. Autotools makes it simple to support new OSes and new hardware
376     platforms, and saves maintainers and porters from having to learn how to
377     custom-build a system to fix compilation. By carefully writing a script,
378     developers can support new platforms without any changes at all.
379     </p>
380    
381     </body>
382     </section>
383     </chapter>
384     </guide>

  ViewVC Help
Powered by ViewVC 1.1.20