/[gentoo]/xml/htdocs/doc/en/draft/bootstrapping-guide.xml
Gentoo

Contents of /xml/htdocs/doc/en/draft/bootstrapping-guide.xml

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.7 - (hide annotations) (download) (as text)
Sun Sep 4 17:53:42 2011 UTC (2 years, 7 months ago) by swift
Branch: MAIN
CVS Tags: HEAD
Changes since 1.6: +2 -2 lines
File MIME type: application/xml
#379883 - Removing link attribute from guide element as it is not used anymore. Next step will be to remove it from DTD

1 swift 1.1 <?xml version='1.0' encoding="UTF-8"?>
2    
3 swift 1.7 <!-- $Header: /var/cvsroot/gentoo/xml/htdocs/doc/en/draft/bootstrapping-guide.xml,v 1.6 2006/01/06 19:50:25 swift Exp $ -->
4 swift 1.1
5     <!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
6    
7 swift 1.7 <guide disclaimer="draft">
8 swift 1.1 <title>Gentoo Bootstrapping Guide</title>
9    
10     <author title="Author">
11     <mail link="swift@gentoo.org">Sven Vermeulen</mail>
12     </author>
13    
14     <abstract>
15     Bootstrapping means to build a toolchain so that it is ready to build the rest
16     of your system. Gentoo is a perfect operating system to perform such
17     installation while retaining support from the software management system,
18     Portage.
19     </abstract>
20    
21     <!-- The content of this document is licensed under the CC-BY-SA license -->
22     <!-- See http://creativecommons.org/licenses/by-sa/2.5 -->
23     <license/>
24    
25     <version>0.1</version>
26     <date>2005-07-25</date>
27    
28     <chapter>
29     <title>What is Bootstrapping?</title>
30     <section>
31     <title>Definition</title>
32     <body>
33    
34     <warn>
35     Not only is this guide still in its early stage, it is also built on theoretical
36     information found on the Internet and not from experience. It should be taken
37     with a big grain of salt until the steps in it are verified and accepted by more
38     experienced people.
39     </warn>
40    
41     <p>
42     If we were to believe the stories, <e>bootstrapping</e> - the term - originates
43     from a German legend about Baron M√ľnchhausen who was able to save himself from
44     drowning in a swamp by pulling himself up by his hairs.
45     </p>
46    
47     <p>
48     In computer theory, bootstrapping has several meanings. All of them boil down to
49     building more complex systems from simple ones. This document will discuss
50     bootstrapping a toolchain: building a full cross-compilation environment able to
51 swift 1.3 build software for the target system, followed by a rebuild of the system to the
52     native environment.
53 swift 1.1 </p>
54    
55 swift 1.6 <p>
56     Sounds strange to you? Don't despair, we'll discuss all that in the rest of this
57     document...
58     </p>
59    
60 swift 1.1 </body>
61     </section>
62     <section>
63     <title>Toolchain Bootstrapping</title>
64     <body>
65    
66     <p>
67 swift 1.3 The process of bootstrapping a toolchain is three-fold.
68 swift 1.1 </p>
69    
70     <p>
71     At first, you use an existing toolchain to create a cross-compilation
72     environment, a toolchain capable of running on one system but building software
73     for a different one. The second step is to use the cross-compilation toolchain
74     to rebuild itself so that it builds code native to the system it is booted on.
75 swift 1.3 The third step uses the native compiler to (re)build all packages (including
76     itself) so that every tool is built on the target system, for the target system.
77 swift 1.1 </p>
78    
79     <p>
80     There are three important terms we use in this definition:
81     </p>
82    
83     <ul>
84     <li>the <e>host</e> system is the system on which the programs are ran, </li>
85     <li>
86     the <e>build</e> system is the system on which a particular package is being
87     built, and
88     </li>
89     <li>
90     the <e>target</e> system is the system for which the software generates
91     output (like the compiler)
92     </li>
93     </ul>
94    
95 swift 1.6 <p>
96     Each of those terms has a certain syntax used by the GNU compiler collection. It
97     is therefore adviseable to consult the <uri
98     link="http://gcc.gnu.org/onlinedocs">online GCC documentation</uri> for more
99     information.
100     </p>
101    
102 swift 1.1 </body>
103     </section>
104     </chapter>
105    
106     <chapter>
107     <title>Installing Gentoo on an Unsupported Platform</title>
108     <section>
109     <title>Creating the Cross-Compilation Environment</title>
110     <body>
111    
112 swift 1.3 <p>
113     We first reserve some space in the home directory to install the
114     cross-compilation environment in. We advise to perform the next steps as a
115     regular, unprivileged user, so that you can not harm the current system. After
116     all, we are going to rebuild core system packages on the system, ready for use
117     on a different system, and we don't want to overwrite the ones on the current
118     system ;)
119     </p>
120    
121     <pre caption="Creating a destination for the cross-compilation environment">
122     $ <i>mkdir ~/cd ~/cd/src</i>
123     </pre>
124    
125     <p>
126     We'll store all source code <e>and</e> binaries inside <path>~/cd</path> because
127     we will need to use those files to boot the new system when the first two phases
128     are complete.
129     </p>
130    
131     <p>
132     At first, extract the source code for the following packages (or similar ones,
133     depending on your setup) inside the <path>~/cd/src</path> directory:
134     </p>
135    
136     <dl>
137     <dt><c>gcc</c></dt>
138     <dd>The GNU Compiler Collection</dd>
139     <dt><c>glibc</c></dt>
140     <dd>The GNU C Library</dd>
141     <dt><c>binutils</c></dt>
142     <dd>The tools needed to build programs</dd>
143     <dt><c>vanilla-sources</c></dt>
144     <dd>The Linux kernel tree</dd>
145     </dl>
146    
147     <p>
148     These are just examples that are well known, but you can also try using the
149     Intel compiler with the ucLibc library, etc.
150     </p>
151    
152     <p>
153     Copy over the header files from the kernel to the build root, allowing the other
154     tools to use the architecture-specific settings of the target architecture:
155     </p>
156    
157     <pre caption="Copying over the header files">
158     $ <i>mkdir -p ~/cd/usr/include</i>
159     $ <i>cp -a /usr/include/asm* /usr/include/linux ~/cd/usr/include</i>
160     </pre>
161    
162     <p>
163     The next step is to build the <c>binutils</c> package suitable for
164     cross-compiling. It is recommended that you read the documentation of
165     <c>binutils</c> for precise instructions how to do this (just like you should
166     for the next packages, <c>gcc</c>, <c>glibc</c> and the Linux kernel).
167     </p>
168    
169     <pre caption="Building the binutils package">
170     $ <i>cd ~/cd/src/binutils-*</i>
171     $ <i>./configure --prefix=/usr --target=&lt;target&gt; --with-sysroot=~/cd</i>
172     $ <i>make all &amp;&amp; make install</i>
173     </pre>
174    
175     <p>
176     Now that the <c>binutils</c> are available in the cross-compilation environment,
177     we install the <c>glibc</c> headers (the function &amp; constant definitions of
178     the c library). Lucky for us, the fine folks at GNU have made this step easier
179     by adding a <c>install-headers</c> directive for <c>make</c>:
180     </p>
181    
182     <pre caption="Installing the glibc headers">
183     $ <i>cd ~/cd/src/glibc*</i>
184     $ <i>./configure --prefix=/usr --build=&lt;build&gt; --host=&lt;target&gt; \
185     --with-headers=~/cd/usr/include --without-cvs --disable-profile \
186     --disable-debug --without-gd --enable-add-ons=nptl --with-tls \
187     --without-__thread --enable-kernel=2.6</i>
188     $ <i>make cross-compiling=yes install-headers install_root=~/cd</i>
189     $ <i>cp -r include/* ~/cd/usr/include</i>
190     </pre>
191    
192     <p>
193     Our next step is to build the cross-compiler:
194     </p>
195    
196     <pre caption="Installing the cross-compiler">
197     $ <i>cd ~/cd/src/gcc*</i>
198     $ <i>./configure --prefix=/usr --target=&lt;target&gt; --with-sysroot=~/cd \
199     --with-headers=~/cd/usr/include --disable-threads --disable-shared \
200     --enable-language=c</i>
201     $ <i>make &amp;&amp; make install</i>
202     </pre>
203    
204     <p>
205     Our almost-final step is to build the <c>glibc</c> package (previously, we just
206     used the header files):
207     </p>
208    
209     <pre caption="Building the glibc package">
210     $ <i>cd ~/cd/src/glibc*</i>
211     $ <i>./configure --prefix=/usr --libdir=/usr/lib --build=&lt;build&gt; \
212     --host=&lt;target&gt; --with-headers=/usr/include --without-cvs \
213     --disable-profile --disable-debug --without-gd \
214     --enable-add-ons=nptl --with-tls --without-__thread --enable-kernel=2.6</i>
215     $ <i>make &amp;&amp; make install install_root=~/cd</i>
216     </pre>
217    
218     <p>
219     In our final step, we build the <c>gcc</c> package again, but now we enable
220     support for C++ and shared libraries (which wasn't possible at first):
221     </p>
222    
223     <pre caption="Building gcc">
224     $ <i>cd ~/cd/src/gcc*</i>
225     $ <i>./configure --prefix=/usr --target=&lt;target&gt; --with-sysroot=~/cd \
226     --with-headers=/usr/include --enable-threads=posix --enable-languages=c,++</i>
227     $ <i>make &amp;&amp; make install</i>
228     </pre>
229    
230 swift 1.1 </body>
231     </section>
232     <section>
233     <title>Filling the Environment</title>
234     <body>
235    
236 swift 1.3 <p>
237     The <path>~/cd</path> location now contains a minimal environment with the
238     cross-compiling toolchain. The next step is to build the core system packages so
239     that you are able to boot into the minimal environment later on.
240     </p>
241    
242     <p>
243 swift 1.6 Our first task is to build a Linux kernel:
244 swift 1.3 </p>
245    
246     <pre caption="Building the Linux kernel">
247     $ <i>cd ~/cd/src/linux-*</i>
248     $ <i>make menuconfig</i>
249     $ <i>make dep boot CROSS_COMPILE=&lt;target&gt;</i>
250     </pre>
251    
252 swift 1.6 <p>
253     Next, build the core packages required to succesfully boot the system. The set
254     of packages you'll need are:
255     </p>
256    
257     <dl>
258     <dt><c>coreutils</c></dt>
259     <dd>Standard set of (POSIX) commands</dd>
260     <dt><c>diffutils</c></dt>
261     <dd>Tools for displaying the differences between files</dd>
262     <dt><c>grep</c></dt>
263     <dd>Text pattern searcher</dd>
264     <dt><c>sed</c></dt>
265     <dd>Streaming text editor</dd>
266     <dt><c>make</c></dt>
267     <dd>Makefile parser</dd>
268     <dt><c>tar</c></dt>
269     <dd>Tape Archive tool</dd>
270     <dt><c>gzip</c></dt>
271     <dd>Compression tool</dd>
272     <dt><c>util-linux</c></dt>
273     <dd>Linux utilities</dd>
274     </dl>
275    
276     <p>
277     All these tools should be fairly easy to build. Generally, you can set the
278     following variables to declare your platform; the <c>configure</c> and
279     <c>make</c> steps will then use those variables to make their platform-dependant
280     decisions:
281     </p>
282    
283     <pre caption="Setting platform environment variables">
284     $ <i>export CC=</i><comment>your-platform</comment><i>-gcc</i>
285     $ <i>export AR=</i><comment>your-platform</comment><i>-ar</i>
286     $ <i>export RANLIB=</i><comment>your-platform</comment><i>-ranlib</i>
287     $ <i>export LD=</i><comment>your-platform</comment><i>-ld</i>
288     $ <i>export BUILD_CC=cc</i>
289     $ <i>export HOST_CC=cc</i>
290     $ <i>export CFLAGS=</i><comment>sane cflags</comment><i> -I~/cd/usr/include</i>
291     </pre>
292    
293     <p>
294     The build steps are then quite simple:
295     </p>
296    
297     <pre caption="Build steps for the core packages">
298     $ <i>./configure --prefix=/usr</i>
299     $ <i>make</i>
300     $ <i>make install prefix=~/cd/final</i>
301     </pre>
302    
303     <p>
304     The last thing you'll need is a statically linked version of a shell, for
305     instance, <c>bash</c>:
306     </p>
307    
308     <pre caption="Building a statically-linked bash">
309     $ <comment>TODO</comment>
310     </pre>
311    
312 swift 1.1 </body>
313     </section>
314     <section>
315     <title>Bootstrapping the Toolchain</title>
316     <body>
317    
318 swift 1.6 <p>
319     Now that you have build a minimal environment inside <path>~/cd</path>, we'll
320     rebuild the toolchain so that it not only builds for the target platform (which
321     it does already) but also builds <e>on</e> the target platform (it currently
322     only works on the current system).
323     </p>
324    
325     <p>
326     First, we rebuild the <c>glibc</c> package:
327     </p>
328    
329     <pre caption="Rebuilding glibc">
330     $ <i>./configure --prefix=/usr --libdir=~/cd/usr/lib --build=${BUILD} \
331     --host=${TARGET} --with-headers=~/cd/usr/include --without-cvs \
332     --disable-profile --disable-debug --without-gd --enable-add-ons=nptl \
333     --with-tls --enable-kernel=2.6 --enable-shared</i>
334     $ <i>make</i>
335     $ <i>make install install_root=~/cd/final</i>
336     </pre>
337    
338     <p>
339     Next, we rebuild the <c>binutils</c> package:
340     </p>
341    
342     <pre caption="Rebuilding binutils">
343     $ <i>./configure --prefix=/usr --target=${TARGET} --with-sysroot=~/cd/final \
344     --libdir=~/cd/usr/lib --with-headers=~/cd/usr/include</i>
345     $ <i>make all</i>
346     $ <i>make install</i>
347     </pre>
348    
349     <p>
350     Finally, we rebuild the <c>gcc</c> package:
351     </p>
352    
353     <pre caption="Rebuilding gcc">
354     $ <i>./configure --prefix=/usr --target=${TARGET} --with-sysroot=~/cd/final \
355     --with-headers=~/cd/usr/include --enable-threads=posix \
356     --enable-languages=c,c++</i>
357     $ <i>make</i>
358     $ <i>make install</i>
359     </pre>
360    
361     <p>
362     Now the directory <path>~/cd/final</path> contains everything needed to
363     continue.
364     </p>
365    
366 swift 1.1 </body>
367     </section>
368     <section>
369 swift 1.6 <title>Booting the System</title>
370 swift 1.1 <body>
371    
372 swift 1.6 <p>
373     The next step is to try and boot the system. The easiest approach is to
374     NFS-mount <path>~/cd/final</path> and boot the target platform using the kernel
375     built at the beginning. Don't forget to set <c>init=/bin/sh</c> since the entire
376     bootup sequence stuff (like <c>init</c>) isn't available yet.
377     </p>
378    
379     <p>
380     <brite>TODO</brite> inform how to boot from the CD and use an NFS-mounted root
381     file system.
382     </p>
383    
384 swift 1.1 </body>
385     </section>
386     <section>
387     <title>Porting Portage</title>
388     <body>
389    
390 swift 1.6 <p>
391     To be able to use Portage, we need to be able to use Python. Download the
392     sources in <path>~/cd/final/tmp</path> (so that it is available for the booted
393     platform) and build it:
394     </p>
395    
396     <pre caption="Building python">
397     <comment>TODO</comment>
398     </pre>
399    
400     <p>
401     Next, download a Portage rescue set and install it. As Portage is a set of
402     Python scripts with bash scripts, this should have no further requirements after
403     installation:
404     </p>
405    
406     <pre caption="Installing Portage">
407     <comment>TODO</comment>
408     </pre>
409    
410     <p>
411     Once installed, try to run <c>emerge</c> and <c>ebuild</c> to find out if they
412     appear to work:
413     </p>
414    
415     <pre caption="Checking emerge">
416     $ <i>emerge --info</i>
417     </pre>
418    
419 swift 1.1 </body>
420     </section>
421     <section>
422     <title>Creating a Stage1 Tarball</title>
423     <body>
424    
425 swift 1.6 <p>
426     Booted in the platform and with a working Portage, you are now ready to create a
427     stage1 tarball. Create a snapshot of your environment using <c>tar</c>:
428     </p>
429    
430     <pre caption="Creating a stage1 tarball">
431     <comment>TODO</comment>
432     Don't forget to talk about unmasking all packages...
433     </pre>
434    
435 swift 1.1 </body>
436     </section>
437     <section>
438     <title>Creating a Bootable Environment</title>
439     <body>
440    
441 swift 1.6 <p>
442     Next, to make sure that you'll always be able to boot the system (and help
443     others as well), we'll create a bootable environment for the platform. Assuming
444     that all platforms have a CD-ROM drive they can boot from, we'll focus on such
445     environment.
446     </p>
447    
448     <p>
449     <brite>TODO</brite> talk about creating bootable CD.
450     </p>
451    
452 swift 1.1 </body>
453     </section>
454     <section>
455     <title>Finishing Off</title>
456     <body>
457    
458 swift 1.6 <p>
459     All set. Right? Nope, but almost :-)
460     </p>
461    
462     <p>
463     The most important step now is to inform the Gentoo community about what you've
464     accomplished. Make sure you pay a visit at <c>#gentoo-dev</c> on
465     <c>irc.freenode.net</c> and use our <uri link="http://forums.gentoo.org">Gentoo
466     Forums</uri> to tell about the up and downfalls of your expedition. The most
467     difficult steps are finished!
468     </p>
469    
470 swift 1.1 </body>
471     </section>
472     </chapter>
473    
474     <chapter>
475     <title>Bootstrapping the System</title>
476     <section>
477     <title>Installing Gentoo</title>
478     <body>
479    
480 swift 1.2 <p>
481     With the bootable environment at your disposal, you can now boot the target
482     system into a small Linux environment. Once booted, follow the installation
483     instructions inside the <uri link="/doc/en/handbook">Gentoo Handbook</uri> to
484     the point where you chroot into your Gentoo environment. Of course, since you
485     only have a stage1 tarball at your disposal, you should use that one instead of
486     the stage3 used in the installation instructions.
487     </p>
488    
489     <p>
490     After chrooting the system, you should update the Portage tree.
491     </p>
492    
493     <pre caption="Updating the Portage tree">
494     # <i>emerge --sync</i>
495     </pre>
496    
497 swift 1.1 </body>
498     </section>
499     <section>
500     <title>Using the Bootstrap Script</title>
501     <body>
502    
503 swift 1.2 <p>
504     Next, we'll rebuild the toolchain provided by the stage1 tarball natively.
505     Gentoo provides a script that does this for you.
506     </p>
507    
508     <pre caption="Rebuilding the toolchain">
509     # <i>/usr/portage/scripts/bootstrap.sh</i>
510     </pre>
511    
512 swift 1.1 </body>
513     </section>
514     <section>
515     <title>Building the Core System</title>
516     <body>
517    
518 swift 1.2 <p>
519     With the toolchain rebuild and ready for general usage, we'll build the core
520     system packages for the system:
521     </p>
522    
523     <pre caption="Building the core system packages">
524     # <i>emerge --emptytree system</i>
525     </pre>
526    
527 swift 1.1 </body>
528     </section>
529     <section>
530     <title>Finishing the Installation</title>
531     <body>
532    
533 swift 1.2 <p>
534     Now that the core system packages are built, you can continue using the
535     installation instructions in the Gentoo Handbook. You will probably get a few
536     complaints by Portage telling you certain packages are masked. This is because
537     your architecture isn't supported by Gentoo yet, in which case you need to
538     unmask the packages in <path>/etc/portage/package.keywords</path> like you did
539     previously.
540     </p>
541    
542 swift 1.1 </body>
543     </section>
544 swift 1.6 <section>
545     <title>Creating a Fully Working Installation CD</title>
546     <body>
547    
548     <p>
549     If your entire installation has succeeded it is best to try and create an
550     installation CD for your platform using <c>catalyst</c>. Not only will this
551     require an additional profile (to support the new platform) but also some help
552     from the Gentoo developers themselves. On the other hand, if you've succeeded in
553     following all I've written until this part, you're probably already on your way
554     to become a developer yourself :)
555     </p>
556    
557     <p>
558     The major benefit of using <c>catalyst</c> is that Gentoo is then able to create
559     official support for the platform. Not only will there be a fully functional
560     profile and keyword setting, but the core packages will be accepted for your
561     platform, stages will be build and a working installation CD, just like those
562     for the other architectures, will be available.
563     </p>
564    
565     <p>
566     <brite>TODO</brite> Talk about using <c>catalyst</c> to create all needed stuff.
567     </p>
568    
569     </body>
570     </section>
571 swift 1.1 </chapter>
572    
573     <chapter>
574     <title>Frequently Asked Questions</title>
575     <section>
576     <title>
577     Should I bootstrap when I want my entire system to use changed CFLAGS,
578     CXXFLAGS, USE settings and profile changes?
579     </title>
580     <body>
581    
582 swift 1.2 <p>
583     No. After your changes, you should rebuild the toolchain first, after which you
584     can rebuild the entire system using the new toolchain. When your system suffers
585     from circular dependencies, you'll need to rebuild the participants in that
586     circle. For instance, if <c>openssl</c> depends on <c>python</c> which depends
587     on <c>perl</c> which depends on <c>openssl</c> again (yes, this is a fictuous
588     example), rebuild all those packages too.
589     </p>
590    
591     <pre caption="Rebuilding the system">
592 swift 1.5 # <i>emerge --oneshot --emptytree glibc binutils gcc</i>
593 swift 1.2 # <i>emerge --emptytree world</i>
594     </pre>
595    
596     <p>
597     You don't need to bootstrap here because your architecture still remains the
598     same, as is the target system.
599     </p>
600    
601 swift 1.1 </body>
602     </section>
603     <section>
604     <title>
605     Should I bootstrap when I want my entire system to use changed CHOST settings?
606     </title>
607     <body>
608    
609 swift 1.2 <p>
610     Not if the system itself supports the new CHOST setting too (for instance,
611     i386-pc-linux-gnu and i686-pc-linux-gnu on a Pentium IV system). Otherwise, yes,
612     but then we are really interested in hearing how you managed to install Gentoo
613     using the current - wrong - CHOST settings in the first place ;)
614     </p>
615    
616     <p>
617     If your system supports both CHOST settings, you can follow the same
618     instructions as given in the previous FAQ.
619     </p>
620    
621 swift 1.1 </body>
622     </section>
623     </chapter>
624    
625     </guide>

  ViewVC Help
Powered by ViewVC 1.1.20