The Xen technology allows you to run multiple operating systems on a single physical system, govern resource consumption and even migrate domains (which are the virtual environments in which a guest operating system runs) from one Xen-powered system to another. Xen requires the host operating system to support Xen (which, in this case, will be a Linux kernel) but guest operating systems can run unmodified if your hardware supports Intel Virtualization Technology (VT-x) or AMD Virtualization Technology (SVM). Otherwise your guest operating systems must also support Xen.

This guide will talk you through the configuration steps necessary to get Xen up and running on Gentoo Linux. We will not discuss Xen itself (the Xen project has decent documentation available) nor will we talk about specialized setups that might be very interesting for Xen setups but are not Xen-related (like exporting Portage through NFS, booting Linux using PXE, etc.)

Domain0 is the primary domain under Xen, hosting the host operating system which governs all other domains. In this chapter we will prepare an existing Gentoo installation to become the host operating system in this domain and build the Xen-powered kernel so that Gentoo is ready to host other Xen domains.

A dramatic change that might be necessary is to rebuild the entire Gentoo installation with a different CFLAGS setting. Guest operating systems running under Xen might otherwise see major performance degradation. If you, however, are planning on checking out Xen rather than installing it for production use and are not terribly fond of rebuilding all programs, you can skip this step. In this case you will notice performance degradation but you will still be able to use Xen.

~# nano -w /etc/make.conf
(Add -mno-tls-direct-seg-refs)
CFLAGS="-O2 -march=pentium4 -pipe -mno-tls-direct-seg-refs"

~# emerge -e world

If you boot your system using an initial ramdisk (initrd) you need to rebuild the initrd as well (which is best done by running all steps you would do when you rebuild your kernel).

Xen actually contains many components, so you'll need to install a couple of packages. Because it is still ~arch masked you first need to unmask it by adding the necessary lines to /etc/portage/package.keywords and then install them.

~# nano -w /etc/portage/package.keywords
app-emulation/xen
app-emulation/xen-tools
sys-kernel/xen-sources

~# emerge xen xen-tools xen-sources

Next we'll build the Linux kernel with Xen support. This kernel, whose sources are available at /usr/src/linux-2.6.x.z-xen, will be our main running kernel (i.e. the one running domain 0). In the XEN section you'll find drivers for all kinds of input/output, each driver having a backend and frontend implementation available. For the domain 0 kernel you need to select the backend implementation: these are used by the other domains (who use the frontend drivers) to communicate directly with the hardware.

Of course, don't forget to select Xen-compatible at Processor type and features. If you're wondering about networking: each interface in a domain has a point-to-point link to an interface on domain 0 (called vifX.Y where X is the domain number and Y the Yth interface of that domain), so you can configure your network the way you want (bridging, NAT, etc.)

Processor type and features  --->
      Subarchitecture Type (Xen-compatible)

Processor type and features  --->
      Subarchitecture Type (PC-compatible)
  [*] Enable Xen compatible kernel

Bus options (PCI etc.)  --->
  [*] PCI support
  [ ]   Xen PCI Frontend Debugging

Networking  --->
  Networking options  --->
    <*> 802.1d Ethernet Bridging
    Only required by bridged networking.

XEN  --->
  [*] Privileged Guest (domain 0)
  <*> Backend driver support
  <*>   Block-device backend driver
  <*>   Network-device backend driver
  <*>   PCI-device backend driver
           PCI Backend Mode (Virtual PCI) --->
  [*] Scrub memory before freeing it to Xen
  [*] Disable serial port drivers
      Xen version compatibility (3.0.4 and later)

Bus options (PCI etc.)  --->
  [ ] PCI support

Device Drivers  --->
  SCSI device support  --->
    < > SCSI device support
    Disabling SCSI support frees up the /dev/sd* device names
    for use as Xen virtual block devices.

XEN  --->
  [ ] Privileged Guest (domain 0)
  <*> Block-device frontend driver
  <*> Network-device frontend driver
  [*] Scrub memory before freeing it to Xen
  [*] Disable serial port drivers
      Xen version compatibility (3.0.4 and later)

A nice hint is to have the kernel make process store its intermediate object files elsewhere so that you can reuse the same kernel tree to build different configurations:

~# mkdir -p ~/build/dom0 ~/build/domU
~# make O=~/build/dom0 menuconfig
(Configure the kernel)
~# make O=~/build/dom0 && make O=~/build/dom0 modules_install

Once the kernel is built you'll find the kernel image immediately in the build directory (not inside arch/ or any other directory) called vmlinuz. Copy it to /boot and then configure your bootloader to use the Xen hypervisor (one of the components installed previously) which is stored as /boot/xen.gz. In the bootloader configuration, add your newly built kernel as the kernel that Xen should boot. For instance, for GRUB:

title Xen 3.0 / Gentoo Linux 2.6.x.y
root (hd0,0)
kernel /boot/xen.gz
module /boot/kernel-2.6.x.y-xen0 root=/dev/hda3

Now reboot your system into Xen. Once you are booted, you need to load the Xen daemon:

~# /etc/init.d/xend start

Now check if you can do whatever you normally do on your system. If this is the case, you can edit your bootloader configuration to always boot into Xen and add the Xen deamon to the default runlevel so that it is started automatically next time you boot.

Go to the Xen-powered Linux kernel source and update the configuration. It is wise to keep as many topics as possible similar to the main kernel except the XEN settings where drivers should now have their frontend implementation selected instead of the backend. Then build the kernel and place the resulting vmlinuz file where you want (we assume this is /mnt/data/xen/kernel):

~# make O=~/build/domU
~# cp ~/build/domU/vmlinuz /mnt/data/xen/kernel/kernel-2.6.x.y-xen

It is also possible to create a single kernel image for both the administrative domain and the unpriviledged domain. More information about this can be found in the Xen user manual.

For best performance, it is best to dedicate a partition (or logical volume) to a domain rather than a file based filesystem. However, if you are going to use Xen primarily for tests using a file based filesystem does have its advantages (especially regarding maintenance).

You can create a file based filesystem using dd and mke2fs (or any other file system creation tool). For instance, to create a 2Gbyte ext3 filesystem:

~# dd if=/dev/zero of=/mnt/data/xen/disks/ext3root.img bs=1M count=2048
~# mke2fs -j /mnt/data/xen/disks/ext3root.img

Next we create a Xen configuration file for a domain. You can store these configuration files where you want, for instance at /mnt/data/xen/configs. As an example, we create a configuration file for a small Gentoo environment which uses the disk image we created previously:

~# nano -w /mnt/data/xen/configs/gentoo

kernel = "/mnt/data/xen/kernel/kernel-2.6.x.y-xen"
memory = 512
name   = "gentoo"
(Map the disk image to the virtual /dev/sda1)
disk   = ['file:/mnt/data/xen/disks/ext3root.img,sda1,w']
root   = "/dev/sda1 ro"

If you are using a block device (such as an lvm volume or partition) for the disk use 'phy:' instead of 'file:' and leave off /dev. For example:

(LVM Volume)
disk = [ 'phy:lvm/xen-guest-root,sda1,w' ]

(Physical Partition)
disk = [ 'phy:sdb6,sda1,w' ]

You can find example configuration files in /etc/xen.

Now we're all set and we can launch the new domain. If the disk image contained an operating system, we could just create and attach the domain using the xm command (Xen manager):

~# xm create /mnt/data/xen/configs/gentoo -c

The domain would be booted inside the terminal in which you executed the command. However, in our case, the disk image is empty so the domain won't boot up in anything useful. To fix this, you can loop-mount the image and install Gentoo as you're used to.

If you want to disconnect from the domain, press Ctrl+]. You can always reconnect to the domains' console using xm console gentoo. However, there is only one console per domain, so only use it when you can't access the domain otherwise (for instance, through SSH).

Xen supports at least two ways of configuring your (virtual) network: routed and bridged.

When selecting the routed approach, the interface inside your unpriviledged domain is connected to the virtual interface on your administrative domain. On your administrative domain (domain 0), the virtual interface is linked together with eth0. The interface inside your unpriviledged domain should have an IP address on the same network as the interface on the administrative domain. Any communication to that IP address can only occur from the administrative domain, unless you set up specific routing rules.

When selecting the bridged approach, your default network interface on the administrative domain becomes a bridge which accepts connections to the virtual domains as well as to the IP address your administrative domain has.

Before you set up the interface on your unpriviledged domain, make sure that Xen's netloop and netbk drivers are loaded. A quick hint: if you have netloop as a module, load it with nloopbacks=0 so that it doesn't create pointless interfaces to the loopback device. Then, edit your domain configuration file and add a vif instruction to it.

~# nano -w /mnt/data/xen/configs/gentoo

(Add the vif instruction)
vif    = [ 'ip=192.168.1.101, vifname=veth1' ]

In the above example, the interface will be created for the unpriviledged domain (in which it will be called eth0) and Xen will ensure that address 192.168.1.101 will be reachable from the administrative domain through interface veth1.

This doesn't mean that the virtual eth0 interface will automatically have IP 192.168.1.101 assigned to it, but rather that, if you don't give it that IP, it will not be connected with the administrative domain and thus cannot be reached.

Now edit /etc/xen/xend-config.xsp as follows to select routed network configuration:

~# nano -w /etc/xen/xend-config.xsp

(Comment out the following lines)
#(network-script network-bridge)
#(vif-script vif-bridge)

(Enable the following lines)
(network-script network-route)
(vif-script vif-route)

Unlike the routed interfaces you now need to load the netloop driver with nloopbacks=1 (or higher) as the additional loopback devices are used to create the bridge. For the other modules you still need the netbk module as well as briding functionality (bridge module if build as such).

Now edit your virtual domain and add the vif construct:

~# nano -w /mnt/data/xen/configs/gentoo

(Add the vif instruction)
vif    = [ 'ip=192.168.1.101, vifname=veth0' ]

Next edit /etc/xen/xend-config.xsp as follows to select bridged network configuration:

~# nano -w /etc/xen/xend-config.xsp

(Enable the following lines)
(network-script network-bridge)
(vif-script vif-bridge)

(Comment out the following lines if not done already)
# (network-script network-route)
# (vif-script vif-route)

By default, the bridge will contain whatever interface is configured to be the default interface (the device that is listed under the default route through ip route list). If you want to alter this behavior, edit the xend-config.xsp as follows:

~# nano -w /etc/xen/xend-config.xsp

(Edit the network-script line)
(network-script 'network-bridge netdev=eth0 bridge=xenbr0 vifnum=0')

Once the configuration is done, restart the xend init script to have Xen build the bridge:

~# /etc/init.d/xend restart

• Xen Users' Manual
• Xen Community Documentation
• Xen Wiki

• app-emulation/virt-manager is a graphical tool for administering virtual machines