Setting up Stable Xen Dom0 with Fedora: Xen 3.4.3 with Xenified Linux Kernel 2.6.32.13 in Fedora 12

This is the latest stable and recommended stable Xen Dom0 solution on Fedora 12. No serious bug found till now and we will fix the bugs by ourselves if some appears. It also works on Fedora 14 as well. It should not be hard to use this solution on other versions of Fedora or other Linux distribution.

How to set up Xen Dom0 with Xenified Linux kernel in Fedora 12 will be introduced in this post. We use Xen 3.4.3 from xen.org and Xenified Linux kernel 2.6.32.13. This is a very stable Dom0 solution for Fedora 12. Lot’s DomUs have been tested on this platform.

Hardware:

Dom0’s hardware platform:

Motherboard: INTEL S5500BC S5500 Quad Core Xeon Server Board
CPU: 2 x Intel Quad Core Xeon E5520 2.26G (5.86GT/sec,8M,Socket 1366)
Memory: 8 x Kingston DDR-3 1333MHz 4GB ECC REG. CL9 DIMM w/Parity & Thermal Sensor
HD: 4 x WD WD10EARS 1 TB, SATA II 3Gb/s, 64 MB Cache

Linux system:

Fedora 12 x86_64

SELinux is disabled. Please refer here for detail: Disabled SELinux on Fedora.

ext3 is recommended for the file system of disk partition for /boot.

Update the system:

# yum update

The Xen and libvirt packages in Fedora should not be installed to avoid conflict.

# yum erase xen* libvirt

Build and install Xen hypervisor and tools

Download Xen 3.4.3

$ wget http://bits.xensource.com/oss-xen/release/3.4.3/xen-3.4.3.tar.gz
$ tar xf xen-3.4.3.tar.gz

Build Xen and tools

You may need to install packages depended by this. You can try this for solving the dependencies:

# yum install make gcc -y;
# yum groupinstall "Development Libraries" -y;
# yum groupinstall "Development Tools" -y;
# yum install transfig texi2html \
dev86 glibc-devel glibc-devel.i686 \
e2fsprogs-devel gitk mkinitrd \
iasl xz-devel bzip2-devel \
pciutils-libs pciutils-devel \
SDL-devel libX11-devel gtk2-devel \
bridge-utils PyXML qemu-common \
qemu-img mercurial ed -y

The we can make Xen and Xen tools:

$ make xen
$ make tools

Install Xen and tools

$ make install-xen
$ make install-tools

Build and install xenified Linux kernel

Download Linux kernel 2.6.32.13

$ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.32.13.tar.bz2
$ tar xf linux-2.6.32.13.tar.bz2

Download 2.6.32 Xen patches v2

$ wget http://gentoo-xen-kernel.googlecode.com/files/xen-patches-2.6.32-2.tar.bz2
$ mkdir xen-patches-2.6.32-2
$ tar xf xen-patches-2.6.32-2.tar.bz2 -C xen-patches-2.6.32-2

Apply Xen patches

Apply all the patches downloaded above following the patch number.

You may need to install patch if it isn’t installed:

# yum install patch

This patch.sh script can be used (we assume the patch and the kernel are in the same directory):

patch.sh:

#!/bin/bash
for P in `ls ../xen-patches-2.6.32-2/6*.patch1 | sort`
do
    patch -p1 -s -i $P
    if [ $? = 0 ]; then
        echo $P applied
    else
        echo "Error processing "$P
        exit 1
    fi
done

Put this script into Linux source directory and execute:

$ sh ./patch.sh

The structure of these directories are as follows:

|
|-linux-2.6.32.13
|    |
|    `- patch.sh
` xen-patches-2.6.32-2
      |
      `- 6*.patch1

Configure Xenified Linux kernel

A working configuration file that I used can be downloaded directly from here:

config-for-xenified-linux-2.6.32.13

Just download this file, put it into the kernel source code file directory and rename it to .config .

Other than use my configuration file, you can also configure it by yourself by using “make menuconfig”.

Make sure you build the kernel with these components enabled:

    Processor type and features  --->
     [*] Symmetric multi-processing support
     [*] Support sparse irq numbering
     [*] Enable Xen compatible kernel 

    Device Drivers  --->
     XEN  --->
     [*] Privileged Guest (domain 0) 
     <*> Backend driver support (NEW)
     <*>   Block-device backend driver (NEW)
     <*>   Block-device tap backend driver (NEW)
     < > Block-device tap backend driver 2 (NEW) 
     <*> Network-device backend driver (NEW) 
     (8)     Maximum simultaneous transmit requests (as a power of 2) (NEW)
     [ ]     Pipelined transmitter (DANGEROUS) (NEW)
     < >     Network-device loopback driver (NEW)
     < > PCI-device backend driver (NEW) 
     < >   TPM-device backend driver (NEW)
     <M>   SCSI backend driver (NEW)
     <M>   USB backend driver (NEW)
     <M> Block-device frontend driver
     <M> Network-device frontend driver
     <M>   Network-device frontend driver acceleration for Solarflare NICs (NEW)
     <M> SCSI frontend driver (NEW)
     <M> USB frontend driver (NEW)
     [*]   Taking the HCD statistics (for debug) (NEW)
     [ ]   HCD suspend/resume support (DO NOT USE) (NEW)
     <*> User-space granted page access driver (NEW)
     <*> Framebuffer-device frontend driver (NEW)
     <*>   Keyboard-device frontend driver (NEW)
     [*] Disable serial port drivers (NEW)
     <*> Export Xen attributes in sysfs (NEW)
     (256) Number of guest devices (NEW)
     Xen version compatibility (no compatibility code) ---> 
     [*] Place shared vCPU info in per-CPU storage (NEW)

Build kernel

$ make -jX

X can be two times of the number of the processor. We use this to let make invoke compilation work in X-way parallel.

Install modules and kernel

# make modules_install install

Configure grub

Add one entry for Xen in /boot/grub/grub.conf. This is an example entry:

title Xen 3.4.3 - Xenified Linux 2.6.32.13
  root (hd0,0)
  kernel /xen-3.4.3.gz console=vga vga=ask noreboot
  module /vmlinuz-2.6.32.13 ro root=/dev/mapper/VolGroup-LogVol_root noiswmd LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us
  module /initramfs-2.6.32.13.img

The root and other parameters may be different depending on the configuration.

Make Xend and Xendomains services automatically start when system boots

# cd /etc/init.d/
# chkconfig --add xend
# chkconfig --add xendomains</pre>

Check whether Xend and Xendomains services are automatically started in level 3-5:

# chkconfig --list | grep xend

It should be like this:

xend               0:off    1:off    2:off    3:on    4:on    5:on    6:off
xendomains         0:off    1:off    2:off    3:on    4:on    5:on    6:off

Enjoy the fun now!

After booting the system, you can try to use xm to check xen info

# xm info

Then xm command can be used to start up DomUs.

Making the performance more stable

Allocating dedicated CPU core and memory for Dom0 may provide more stable performance for the Xen platform. Please refer to Managing Xen Dom0’s CPU and Memory for detailed instruction.

Problems

A list of common problems and tips can be found in Problems during Installing Xen Dom0 in Fedora.

Please refer to https://www.systutorials.com/xen-solutions for the DomU solution and more Xen Dom0 solution.