Installing Xen on Fedora as Domain-0 (Fedora 17)

The new development of Xen and Linux kernel make it easy to install Xen on Fedora as the Domain-0 now. This post uses Fedora 17 as an example platform to introduce how to set up Domain-0 on Fedora Linux. Compared to our old method (Setting up Stable Xen Dom0 with Fedora: Xen 3.4.3 with Xenified Linux Kernel 2.6.32.13 in Fedora 12) which requires manually compiled Xen and patched kernel, the current packages and support to Xen in Fedora and Linux kernel make the system administrators life much easier.

On the other hand, Xen has changed its tool stack from xm/xend to xl and the network configuration should be done manually on different platforms.

Installing Xen

First, install the xen pacakges:

# yum install xen

The Linux kernel is already ready to run in Domain-0 with the pv_ops technology enabled. The xen and kernel versions I used is listed as follows.

xen-4.1.3-5.fc17.x86_64
kernel-3.6.6-1.fc17.x86_64

Run grub2-mkconfig

Run this command to make config for grub2 again:

# grub2-mkconfig -o /boot/grub2/grub.cfg

Otherwise, the kernel will fail to boot and print messages like “swap not trainted…”.

Then, we can boot the Fedora to run on Xen as Domain-0 by selecting the Fedora, with Xen hypervisor in Grub2 during booting the system. We can also Setting Default Entry in Grub2 and Grub.

After booting Linux on Xen as Domain-0, we can list the information by ‘xl info’ as follow.

# xl info
host                   : office.zhiqiangma.com
release                : 3.6.6-1.fc17.x86_64
version                : #1 SMP Mon Nov 5 21:59:35 UTC 2012
machine                : x86_64
nr_cpus                : 4
nr_nodes               : 1
cores_per_socket       : 4
threads_per_core       : 1
cpu_mhz                : 2809
hw_caps                : bfebfbff:28100800:00000000:00003b40:0098e3fd:00000000:00000001:00000000
virt_caps              : hvm
total_memory           : 2039
free_memory            : 126
free_cpus              : 0
xen_major              : 4
xen_minor              : 1
xen_extra              : .3
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : unavailable
xen_commandline        : placeholder
cc_compiler            : gcc version 4.7.2 20120921 (Red Hat 4.7.2-2) (GCC) 
cc_compile_by          : mockbuild
cc_compile_domain      : [unknown]
cc_compile_date        : Sun Oct 28 23:08:22 UTC 2012
xend_config_format     : 4

Configuring the network

Method 1: The VMs and the host form a private network

If the computer has no wired NIC, such as a laptop that only has wireless network, a dummy device may be helpful. I also show the script I used on my laptop here.

#!/bin/bash

# set up a bridge as the backed device for xen
brctl addbr xenbr0

# set up a dummy device
modprobe dummy
ip link set name xendummy0 dev dummy0
brctl addif xenbr0 xendummy0

# give the bridge an IP
ifconfig xenbr0 10.0.0.2

# set its netmask
ifconfig xenbr0 netmask 255.255.0.0

# set up NAT and make the Domain-0 acts as a gateway
#    You may change this rule for better security
iptables -I FORWARD -j ACCEPT
iptables -t nat -I POSTROUTING --out-interface wlan0 -j MASQUERADE
echo 1 > /proc/sys/net/ipv4/ip_forward

Method 2: bridged networking

NetworkManager does not work with bridge currently. To use bridge-based network for Xen. We should change the network management service from NetworkManager to network. The networking configuration is stored in ‘/etc/sysconfig/network-scripts/’.

To disable NetworkManager and start network, do the following:

# systemctl disable NetworkManager.service
# systemctl restart network.service

Make sure that the network service is automatically started by:

# chkconfig network on

Then, we can create the configuration file for the bridge. Let’s call the bridge ‘xenbr0’.

Edit ‘/etc/sysconfig/network-scripts/ifcfg-xenbr0’ (we assume dhcp here. You can also give the bridge a static IP as for the other network devices.):

DEVICE=xenbr0
TYPE=Bridge
ONBOOT=yes
DELAY=0
NM_CONTROLLED=no
BOOTPROTO=dhcp

Then find the configuration file for your existing network adaptor (e.g. ifcfg-em1) and edit it as follows.

NM_CONTROLLED=no
BRIDGE=xenbr0

Finally, make the network configuration take effect by:

# systemctl restart network.service

As I have stated, the network configuration should be done by the administrator according to the Linux distro’s method. On Fedora, we use NetworkManager as it is. In this tutorial, we set up bridge-based network for xen. This method works well not only for wired network but also wireless network.

I make all the steps together into one script as follows:

#!/bin/bash

# set up a bridge as the backed device for xen
brctl addbr xenbr0

# bridge to eth0
brctl addif xenbr0 eth0

# bring up the bridge
ifconfig xenbr0 up

In this script, we first set up the bridge xenbr0 for xen and bridge it to eth0 which is the real NIC device. Then, we can set up Domain-U to use this bridge as the backed to set up its network.


You can make Linux run this this script each time the system is booted by adding ‘@reboot /path/to/the/script.sh’ to cron jobs through ‘crontab -e’ or setting up a new service to invoke this script.

One example Dom-U configuration file

I give one example Domain-U configuration file here as follows.

name="10.1.0.114"
vcpus=2
memory=2048
disk=['phy:/dev/vg_xen/vm-10.1.0.114,xvda,w']
vif=['bridge=xenbr0']
bootloader="/usr/bin/pygrub"
on_reboot="restart"
on_crash="restart"
# extra="single"

Most of the old ‘xm’ commands for Domain-U management work under ‘xl’ by just simply replacing ‘xm’ with ‘xl’.

Eric Zhiqiang Ma

Eric is interested in building high-performance and scalable distributed systems and related technologies. The views or opinions expressed here are solely Eric's own and do not necessarily represent those of any third parties.

3 comments:

  1. Hi, Eric,

    I use ” Method 2: bridged networking”.

    When I logon on DomU and want to yum install glibc-devel , there is an error:

    Error: Cann’t retrieve repository metadata.

    How can I do to solve this ?

    Thank you.

    1. There are many possible reasons that can cause this problem. Try to understand how the overall system works. For example, the problem you described would be for `yum tries to download the metadata`:

      1. yum send network packets to the DomU
      2. the network requests are passed to the Dom0
      3. the Dom0 sends out the packets
      4. the metadata server receives the packets
      5. the responding packets are sent back to Dom0 then to DomU

      You can use common tools to verify whether each step’s needed function is working. For example, verify DomU can use Internet (1, 2, 3, 5), verify the metadata server is working from another server/host (4). Or there are other possible reasons.

      Simply asking how to solve one problem you found will not help the others solve you problem as they do not know many details. You may either narrow down the problem and give the other more details.

Leave a Reply

Your email address will not be published. Required fields are marked *