en/draft/bootstrapping-guide.xml

<?xml version='1.0' encoding="UTF-8"?>

<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/draft/bootstrapping-guide.xml,v 1.2 2005/11/30 05:14:15 swift Exp $ -->

<!DOCTYPE guide SYSTEM "/dtd/guide.dtd">

<guide link="/doc/en/draft/bootstrapping-guide.xml">
<title>Gentoo Bootstrapping Guide</title>

<author title="Author">
  <mail link="swift@gentoo.org">Sven Vermeulen</mail>
</author>

<abstract>
Bootstrapping means to build a toolchain so that it is ready to build the rest
of your system. Gentoo is a perfect operating system to perform such 
installation while retaining support from the software management system, 
Portage.
</abstract>

<!-- The content of this document is licensed under the CC-BY-SA license -->
<!-- See http://creativecommons.org/licenses/by-sa/2.5 -->
<license/>

<version>0.1</version>
<date>2005-07-25</date>

<chapter>
<title>What is Bootstrapping?</title>
<section>
<title>Definition</title>
<body>

<warn>
Not only is this guide still in its early stage, it is also built on theoretical
information found on the Internet and not from experience. It should be taken
with a big grain of salt until the steps in it are verified and accepted by more
experienced people.
</warn>

<p>
If we were to believe the stories, <e>bootstrapping</e> - the term - originates
from a German legend about Baron Münchhausen who was able to save himself from
drowning in a swamp by pulling himself up by his hairs.
</p>

<p>
In computer theory, bootstrapping has several meanings. All of them boil down to
building more complex systems from simple ones. This document will discuss
bootstrapping a toolchain: building a full cross-compilation environment able to
build software for the target system, followed by a rebuild of the system to the
native environment.
</p>

</body>
</section>
<section>
<title>Toolchain Bootstrapping</title>
<body>

<p>
The process of bootstrapping a toolchain is three-fold.
</p>

<p>
At first, you use an existing toolchain to create a cross-compilation
environment, a toolchain capable of running on one system but building software
for a different one. The second step is to use the cross-compilation toolchain 
to rebuild itself so that it builds code native to the system it is booted on.
The third step uses the native compiler to (re)build all packages (including
itself) so that every tool is built on the target system, for the target system.
</p>

<p>
There are three important terms we use in this definition:
</p>

<ul>
  <li>the <e>host</e> system is the system on which the programs are ran, </li>
  <li>
    the <e>build</e> system is the system on which a particular package is being
    built, and
  </li>
  <li>
    the <e>target</e> system is the system for which the software generates
    output (like the compiler)
  </li>
</ul>

</body>
</section>
</chapter>

<chapter>
<title>Installing Gentoo on an Unsupported Platform</title>
<section>
<title>Creating the Cross-Compilation Environment</title>
<body>

<p>
We first reserve some space in the home directory to install the
cross-compilation environment in. We advise to perform the next steps as a
regular, unprivileged user, so that you can not harm the current system. After
all, we are going to rebuild core system packages on the system, ready for use
on a different system, and we don't want to overwrite the ones on the current
system ;)
</p>

<pre caption="Creating a destination for the cross-compilation environment">
$ <i>mkdir ~/cd ~/cd/src</i>
</pre>

<p>
We'll store all source code <e>and</e> binaries inside <path>~/cd</path> because
we will need to use those files to boot the new system when the first two phases
are complete.
</p>

<p>
At first, extract the source code for the following packages (or similar ones,
depending on your setup) inside the <path>~/cd/src</path> directory:
</p>

<dl>
  <dt><c>gcc</c></dt>
  <dd>The GNU Compiler Collection</dd>
  <dt><c>glibc</c></dt>
  <dd>The GNU C Library</dd>
  <dt><c>binutils</c></dt>
  <dd>The tools needed to build programs</dd>
  <dt><c>vanilla-sources</c></dt>
  <dd>The Linux kernel tree</dd>
</dl>

<p>
These are just examples that are well known, but you can also try using the
Intel compiler with the ucLibc library, etc.
</p>

<p>
Copy over the header files from the kernel to the build root, allowing the other
tools to use the architecture-specific settings of the target architecture:
</p>

<pre caption="Copying over the header files">
$ <i>mkdir -p ~/cd/usr/include</i>
$ <i>cp -a /usr/include/asm* /usr/include/linux ~/cd/usr/include</i>
</pre>

<p>
The next step is to build the <c>binutils</c> package suitable for
cross-compiling. It is recommended that you read the documentation of
<c>binutils</c> for precise instructions how to do this (just like you should
for the next packages, <c>gcc</c>, <c>glibc</c> and the Linux kernel). 
</p>

<pre caption="Building the binutils package">
$ <i>cd ~/cd/src/binutils-*</i>
$ <i>./configure --prefix=/usr --target=&lt;target&gt; --with-sysroot=~/cd</i>
$ <i>make all &amp;&amp; make install</i>
</pre>

<p>
Now that the <c>binutils</c> are available in the cross-compilation environment,
we install the <c>glibc</c> headers (the function &amp; constant definitions of
the c library). Lucky for us, the fine folks at GNU have made this step easier
by adding a <c>install-headers</c> directive for <c>make</c>:
</p>

<pre caption="Installing the glibc headers">
$ <i>cd ~/cd/src/glibc*</i>
$ <i>./configure --prefix=/usr --build=&lt;build&gt; --host=&lt;target&gt; \
  --with-headers=~/cd/usr/include --without-cvs --disable-profile \
  --disable-debug --without-gd --enable-add-ons=nptl --with-tls \
  --without-__thread --enable-kernel=2.6</i>
$ <i>make cross-compiling=yes install-headers install_root=~/cd</i>
$ <i>cp -r include/* ~/cd/usr/include</i>
</pre>

<p>
Our next step is to build the cross-compiler:
</p>

<pre caption="Installing the cross-compiler">
$ <i>cd ~/cd/src/gcc*</i>
$ <i>./configure --prefix=/usr --target=&lt;target&gt; --with-sysroot=~/cd \
  --with-headers=~/cd/usr/include --disable-threads --disable-shared \
  --enable-language=c</i>
$ <i>make &amp;&amp; make install</i>
</pre>

<p>
Our almost-final step is to build the <c>glibc</c> package (previously, we just
used the header files):
</p>

<pre caption="Building the glibc package">
$ <i>cd ~/cd/src/glibc*</i>
$ <i>./configure --prefix=/usr --libdir=/usr/lib --build=&lt;build&gt; \
  --host=&lt;target&gt; --with-headers=/usr/include --without-cvs \
  --disable-profile --disable-debug --without-gd \
  --enable-add-ons=nptl --with-tls --without-__thread --enable-kernel=2.6</i>
$ <i>make &amp;&amp; make install install_root=~/cd</i>
</pre>

<p>
In our final step, we build the <c>gcc</c> package again, but now we enable
support for C++ and shared libraries (which wasn't possible at first):
</p>

<pre caption="Building gcc">
$ <i>cd ~/cd/src/gcc*</i>
$ <i>./configure --prefix=/usr --target=&lt;target&gt; --with-sysroot=~/cd \
  --with-headers=/usr/include --enable-threads=posix --enable-languages=c,++</i>
$ <i>make &amp;&amp; make install</i>
</pre>

</body>
</section>
<section>
<title>Filling the Environment</title>
<body>

<p>
The <path>~/cd</path> location now contains a minimal environment with the
cross-compiling toolchain. The next step is to build the core system packages so
that you are able to boot into the minimal environment later on.
</p>

<p>
Our first task is to build a Linux kernel. 
</p>

<pre caption="Building the Linux kernel">
$ <i>cd ~/cd/src/linux-*</i>
$ <i>make menuconfig</i>
$ <i>make dep boot CROSS_COMPILE=&lt;target&gt;</i>
</pre>

</body>
</section>
<section>
<title>Bootstrapping the Toolchain</title>
<body>

</body>
</section>
<section>
<title>Building the Core System Packages</title>
<body>

</body>
</section>
<section>
<title>Porting Portage</title>
<body>

</body>
</section>
<section>
<title>Creating a Stage1 Tarball</title>
<body>

</body>
</section>
<section>
<title>Creating a Bootable Environment</title>
<body>

</body>
</section>
<section>
<title>Finishing Off</title>
<body>

</body>
</section>
</chapter>

<chapter>
<title>Bootstrapping the System</title>
<section>
<title>Installing Gentoo</title>
<body>

<p>
With the bootable environment at your disposal, you can now boot the target
system into a small Linux environment. Once booted, follow the installation
instructions inside the <uri link="/doc/en/handbook">Gentoo Handbook</uri> to
the point where you chroot into your Gentoo environment. Of course, since you
only have a stage1 tarball at your disposal, you should use that one instead of
the stage3 used in the installation instructions.
</p>

<p>
After chrooting the system, you should update the Portage tree.
</p>

<pre caption="Updating the Portage tree">
# <i>emerge --sync</i>
</pre>

</body>
</section>
<section>
<title>Using the Bootstrap Script</title>
<body>

<p>
Next, we'll rebuild the toolchain provided by the stage1 tarball natively.
Gentoo provides a script that does this for you.
</p>

<pre caption="Rebuilding the toolchain">
# <i>/usr/portage/scripts/bootstrap.sh</i>
</pre>

</body>
</section>
<section>
<title>Building the Core System</title>
<body>

<p>
With the toolchain rebuild and ready for general usage, we'll build the core
system packages for the system:
</p>

<pre caption="Building the core system packages">
# <i>emerge --emptytree system</i>
</pre>

</body>
</section>
<section>
<title>Finishing the Installation</title>
<body>

<p>
Now that the core system packages are built, you can continue using the
installation instructions in the Gentoo Handbook. You will probably get a few
complaints by Portage telling you certain packages are masked. This is because
your architecture isn't supported by Gentoo yet, in which case you need to
unmask the packages in <path>/etc/portage/package.keywords</path> like you did
previously.
</p>

</body>
</section>
</chapter>

<chapter>
<title>Frequently Asked Questions</title>
<section>
<title>
  Should I bootstrap when I want my entire system to use changed CFLAGS,
  CXXFLAGS, USE settings and profile changes?
</title>
<body>

<p>
No. After your changes, you should rebuild the toolchain first, after which you
can rebuild the entire system using the new toolchain. When your system suffers
from circular dependencies, you'll need to rebuild the participants in that
circle. For instance, if <c>openssl</c> depends on <c>python</c> which depends
on <c>perl</c> which depends on <c>openssl</c> again (yes, this is a fictuous
example), rebuild all those packages too.
</p>

<pre caption="Rebuilding the system">
# <i>emerge --oneshot --emptytree glibc binutils glibc</i>
# <i>emerge --emptytree world</i>
</pre>

<p>
You don't need to bootstrap here because your architecture still remains the
same, as is the target system.
</p>

</body>
</section>
<section>
<title>
  Should I bootstrap when I want my entire system to use changed CHOST settings?
</title>
<body>

<p>
Not if the system itself supports the new CHOST setting too (for instance,
i386-pc-linux-gnu and i686-pc-linux-gnu on a Pentium IV system). Otherwise, yes,
but then we are really interested in hearing how you managed to install Gentoo
using the current - wrong - CHOST settings in the first place ;)
</p>

<p>
If your system supports both CHOST settings, you can follow the same
instructions as given in the previous FAQ.
</p>

</body>
</section>
</chapter>

</guide>
1	swift	1.1	<?xml version='1.0' encoding="UTF-8"?>
2
3	swift	1.3	<!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/draft/bootstrapping-guide.xml,v 1.2 2005/11/30 05:14:15 swift Exp $ -->
4	swift	1.1
5			<!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
6
7			<guide link="/doc/en/draft/bootstrapping-guide.xml">
8			<title>Gentoo Bootstrapping Guide</title>
9
10			<author title="Author">
11			<mail link="swift@gentoo.org">Sven Vermeulen</mail>
12			</author>
13
14			<abstract>
15			Bootstrapping means to build a toolchain so that it is ready to build the rest
16			of your system. Gentoo is a perfect operating system to perform such
17			installation while retaining support from the software management system,
18			Portage.
19			</abstract>
20
21			<!-- The content of this document is licensed under the CC-BY-SA license -->
22			<!-- See http://creativecommons.org/licenses/by-sa/2.5 -->
23			<license/>
24
25			<version>0.1</version>
26			<date>2005-07-25</date>
27
28			<chapter>
29			<title>What is Bootstrapping?</title>
30			<section>
31			<title>Definition</title>
32			<body>
33
34			<warn>
35			Not only is this guide still in its early stage, it is also built on theoretical
36			information found on the Internet and not from experience. It should be taken
37			with a big grain of salt until the steps in it are verified and accepted by more
38			experienced people.
39			</warn>
40
41			<p>
42			If we were to believe the stories, <e>bootstrapping</e> - the term - originates
43			from a German legend about Baron Münchhausen who was able to save himself from
44			drowning in a swamp by pulling himself up by his hairs.
45			</p>
46
47			<p>
48			In computer theory, bootstrapping has several meanings. All of them boil down to
49			building more complex systems from simple ones. This document will discuss
50			bootstrapping a toolchain: building a full cross-compilation environment able to
51	swift	1.3	build software for the target system, followed by a rebuild of the system to the
52			native environment.
53	swift	1.1	</p>
54
55			</body>
56			</section>
57			<section>
58			<title>Toolchain Bootstrapping</title>
59			<body>
60
61			<p>
62	swift	1.3	The process of bootstrapping a toolchain is three-fold.
63	swift	1.1	</p>
64
65			<p>
66			At first, you use an existing toolchain to create a cross-compilation
67			environment, a toolchain capable of running on one system but building software
68			for a different one. The second step is to use the cross-compilation toolchain
69			to rebuild itself so that it builds code native to the system it is booted on.
70	swift	1.3	The third step uses the native compiler to (re)build all packages (including
71			itself) so that every tool is built on the target system, for the target system.
72	swift	1.1	</p>
73
74			<p>
75			There are three important terms we use in this definition:
76			</p>
77
78			<ul>
79			<li>the <e>host</e> system is the system on which the programs are ran, </li>
80			<li>
81			the <e>build</e> system is the system on which a particular package is being
82			built, and
83			</li>
84			<li>
85			the <e>target</e> system is the system for which the software generates
86			output (like the compiler)
87			</li>
88			</ul>
89
90			</body>
91			</section>
92			</chapter>
93
94			<chapter>
95			<title>Installing Gentoo on an Unsupported Platform</title>
96			<section>
97			<title>Creating the Cross-Compilation Environment</title>
98			<body>
99
100	swift	1.3	<p>
101			We first reserve some space in the home directory to install the
102			cross-compilation environment in. We advise to perform the next steps as a
103			regular, unprivileged user, so that you can not harm the current system. After
104			all, we are going to rebuild core system packages on the system, ready for use
105			on a different system, and we don't want to overwrite the ones on the current
106			system ;)
107			</p>
108
109			<pre caption="Creating a destination for the cross-compilation environment">
110			$ <i>mkdir ~/cd ~/cd/src</i>
111			</pre>
112
113			<p>
114			We'll store all source code <e>and</e> binaries inside <path>~/cd</path> because
115			we will need to use those files to boot the new system when the first two phases
116			are complete.
117			</p>
118
119			<p>
120			At first, extract the source code for the following packages (or similar ones,
121			depending on your setup) inside the <path>~/cd/src</path> directory:
122			</p>
123
124			<dl>
125			<dt><c>gcc</c></dt>
126			<dd>The GNU Compiler Collection</dd>
127			<dt><c>glibc</c></dt>
128			<dd>The GNU C Library</dd>
129			<dt><c>binutils</c></dt>
130			<dd>The tools needed to build programs</dd>
131			<dt><c>vanilla-sources</c></dt>
132			<dd>The Linux kernel tree</dd>
133			</dl>
134
135			<p>
136			These are just examples that are well known, but you can also try using the
137			Intel compiler with the ucLibc library, etc.
138			</p>
139
140			<p>
141			Copy over the header files from the kernel to the build root, allowing the other
142			tools to use the architecture-specific settings of the target architecture:
143			</p>
144
145			<pre caption="Copying over the header files">
146			$ <i>mkdir -p ~/cd/usr/include</i>
147			$ <i>cp -a /usr/include/asm* /usr/include/linux ~/cd/usr/include</i>
148			</pre>
149
150			<p>
151			The next step is to build the <c>binutils</c> package suitable for
152			cross-compiling. It is recommended that you read the documentation of
153			<c>binutils</c> for precise instructions how to do this (just like you should
154			for the next packages, <c>gcc</c>, <c>glibc</c> and the Linux kernel).
155			</p>
156
157			<pre caption="Building the binutils package">
158			$ <i>cd ~/cd/src/binutils-*</i>
159			$ <i>./configure --prefix=/usr --target=<target> --with-sysroot=~/cd</i>
160			$ <i>make all && make install</i>
161			</pre>
162
163			<p>
164			Now that the <c>binutils</c> are available in the cross-compilation environment,
165			we install the <c>glibc</c> headers (the function & constant definitions of
166			the c library). Lucky for us, the fine folks at GNU have made this step easier
167			by adding a <c>install-headers</c> directive for <c>make</c>:
168			</p>
169
170			<pre caption="Installing the glibc headers">
171			$ <i>cd ~/cd/src/glibc*</i>
172			$ <i>./configure --prefix=/usr --build=<build> --host=<target> \
173			--with-headers=~/cd/usr/include --without-cvs --disable-profile \
174			--disable-debug --without-gd --enable-add-ons=nptl --with-tls \
175			--without-__thread --enable-kernel=2.6</i>
176			$ <i>make cross-compiling=yes install-headers install_root=~/cd</i>
177			$ <i>cp -r include/* ~/cd/usr/include</i>
178			</pre>
179
180			<p>
181			Our next step is to build the cross-compiler:
182			</p>
183
184			<pre caption="Installing the cross-compiler">
185			$ <i>cd ~/cd/src/gcc*</i>
186			$ <i>./configure --prefix=/usr --target=<target> --with-sysroot=~/cd \
187			--with-headers=~/cd/usr/include --disable-threads --disable-shared \
188			--enable-language=c</i>
189			$ <i>make && make install</i>
190			</pre>
191
192			<p>
193			Our almost-final step is to build the <c>glibc</c> package (previously, we just
194			used the header files):
195			</p>
196
197			<pre caption="Building the glibc package">
198			$ <i>cd ~/cd/src/glibc*</i>
199			$ <i>./configure --prefix=/usr --libdir=/usr/lib --build=<build> \
200			--host=<target> --with-headers=/usr/include --without-cvs \
201			--disable-profile --disable-debug --without-gd \
202			--enable-add-ons=nptl --with-tls --without-__thread --enable-kernel=2.6</i>
203			$ <i>make && make install install_root=~/cd</i>
204			</pre>
205
206			<p>
207			In our final step, we build the <c>gcc</c> package again, but now we enable
208			support for C++ and shared libraries (which wasn't possible at first):
209			</p>
210
211			<pre caption="Building gcc">
212			$ <i>cd ~/cd/src/gcc*</i>
213			$ <i>./configure --prefix=/usr --target=<target> --with-sysroot=~/cd \
214			--with-headers=/usr/include --enable-threads=posix --enable-languages=c,++</i>
215			$ <i>make && make install</i>
216			</pre>
217
218	swift	1.1	</body>
219			</section>
220			<section>
221			<title>Filling the Environment</title>
222			<body>
223
224	swift	1.3	<p>
225			The <path>~/cd</path> location now contains a minimal environment with the
226			cross-compiling toolchain. The next step is to build the core system packages so
227			that you are able to boot into the minimal environment later on.
228			</p>
229
230			<p>
231			Our first task is to build a Linux kernel.
232			</p>
233
234			<pre caption="Building the Linux kernel">
235			$ <i>cd ~/cd/src/linux-*</i>
236			$ <i>make menuconfig</i>
237			$ <i>make dep boot CROSS_COMPILE=<target></i>
238			</pre>
239
240	swift	1.1	</body>
241			</section>
242			<section>
243			<title>Bootstrapping the Toolchain</title>
244			<body>
245
246			</body>
247			</section>
248			<section>
249			<title>Building the Core System Packages</title>
250			<body>
251
252			</body>
253			</section>
254			<section>
255			<title>Porting Portage</title>
256			<body>
257
258			</body>
259			</section>
260			<section>
261			<title>Creating a Stage1 Tarball</title>
262			<body>
263
264			</body>
265			</section>
266			<section>
267			<title>Creating a Bootable Environment</title>
268			<body>
269
270			</body>
271			</section>
272			<section>
273			<title>Finishing Off</title>
274			<body>
275
276			</body>
277			</section>
278			</chapter>
279
280			<chapter>
281			<title>Bootstrapping the System</title>
282			<section>
283			<title>Installing Gentoo</title>
284			<body>
285
286	swift	1.2	<p>
287			With the bootable environment at your disposal, you can now boot the target
288			system into a small Linux environment. Once booted, follow the installation
289			instructions inside the <uri link="/doc/en/handbook">Gentoo Handbook</uri> to
290			the point where you chroot into your Gentoo environment. Of course, since you
291			only have a stage1 tarball at your disposal, you should use that one instead of
292			the stage3 used in the installation instructions.
293			</p>
294
295			<p>
296			After chrooting the system, you should update the Portage tree.
297			</p>
298
299			<pre caption="Updating the Portage tree">
300			# <i>emerge --sync</i>
301			</pre>
302
303	swift	1.1	</body>
304			</section>
305			<section>
306			<title>Using the Bootstrap Script</title>
307			<body>
308
309	swift	1.2	<p>
310			Next, we'll rebuild the toolchain provided by the stage1 tarball natively.
311			Gentoo provides a script that does this for you.
312			</p>
313
314			<pre caption="Rebuilding the toolchain">
315			# <i>/usr/portage/scripts/bootstrap.sh</i>
316			</pre>
317
318	swift	1.1	</body>
319			</section>
320			<section>
321			<title>Building the Core System</title>
322			<body>
323
324	swift	1.2	<p>
325			With the toolchain rebuild and ready for general usage, we'll build the core
326			system packages for the system:
327			</p>
328
329			<pre caption="Building the core system packages">
330			# <i>emerge --emptytree system</i>
331			</pre>
332
333	swift	1.1	</body>
334			</section>
335			<section>
336			<title>Finishing the Installation</title>
337			<body>
338
339	swift	1.2	<p>
340			Now that the core system packages are built, you can continue using the
341			installation instructions in the Gentoo Handbook. You will probably get a few
342			complaints by Portage telling you certain packages are masked. This is because
343			your architecture isn't supported by Gentoo yet, in which case you need to
344			unmask the packages in <path>/etc/portage/package.keywords</path> like you did
345			previously.
346			</p>
347
348	swift	1.1	</body>
349			</section>
350			</chapter>
351
352			<chapter>
353			<title>Frequently Asked Questions</title>
354			<section>
355			<title>
356			Should I bootstrap when I want my entire system to use changed CFLAGS,
357			CXXFLAGS, USE settings and profile changes?
358			</title>
359			<body>
360
361	swift	1.2	<p>
362			No. After your changes, you should rebuild the toolchain first, after which you
363			can rebuild the entire system using the new toolchain. When your system suffers
364			from circular dependencies, you'll need to rebuild the participants in that
365			circle. For instance, if <c>openssl</c> depends on <c>python</c> which depends
366			on <c>perl</c> which depends on <c>openssl</c> again (yes, this is a fictuous
367			example), rebuild all those packages too.
368			</p>
369
370			<pre caption="Rebuilding the system">
371			# <i>emerge --oneshot --emptytree glibc binutils glibc</i>
372			# <i>emerge --emptytree world</i>
373			</pre>
374
375			<p>
376			You don't need to bootstrap here because your architecture still remains the
377			same, as is the target system.
378			</p>
379
380	swift	1.1	</body>
381			</section>
382			<section>
383			<title>
384			Should I bootstrap when I want my entire system to use changed CHOST settings?
385			</title>
386			<body>
387
388	swift	1.2	<p>
389			Not if the system itself supports the new CHOST setting too (for instance,
390			i386-pc-linux-gnu and i686-pc-linux-gnu on a Pentium IV system). Otherwise, yes,
391			but then we are really interested in hearing how you managed to install Gentoo
392			using the current - wrong - CHOST settings in the first place ;)
393			</p>
394
395			<p>
396			If your system supports both CHOST settings, you can follow the same
397			instructions as given in the previous FAQ.
398			</p>
399
400	swift	1.1	</body>
401			</section>
402			</chapter>
403
404			</guide>