Automatically Backing Up Xen File-backed DomU

A script for backing up file-backed Xen DomU is introduced in this post. This script can be changed to similar platform.

In our cluster, virtual machines are stored under /lhome/xen/. Virtual machine with id vmid is stored in directory vmvmid. The raw image disk file name can also be derived from vmid. Some more details and the configuration file can be found from Unified Xen DomU configuration file. The set up process of Xen platform can be found Setting up Stable Xen Dom0 with Fedora: Xen 3.4.3 with Xenified Linux Kernel 2.6.32.13 in Fedora 12 in Xen Solutions. Some introduction to Xen management command xm can be found from Create and Manage Virtual Machines on Xen.

This script is called by cron from every one week. We want to backup the virtual machines for every other week. mark_file is used to make the backup command run every other time the script is called. More details please refer to How to Run a cron Job Every Two Weeks / Months / Days. If this script is used manually, please delete the “# check whether bak runned last week” part.

Two copies of backup is made in the script: one in local directory in another hard disk, another one in hard disk of another machine. The raw image file are compress for save disk space and network usage. In our virtual machines, the raw image file size is 20G, while the compressed one is only about 3G to 4G depends on the usage of the virtual machine’s hard disk space. ssh tunnel is used for data transmission.

The backup consist of several steps:

1. For every virtual machine:
    1.1 Check whether VM is running
    1.2 Shutdown the virtual machine
    1.3 Copy the image file to local directory in another hard disk
    1.4 Start the virtual machine
2. Compress the copied image file in local directory
3. Copy these compressed image file to remote directory

As we use scp to transfer backup files to remote server, we need to Enabling Password-less ssh Login first.

This is the script:

#!/bin/bash

# Author: Zhiqiang Ma (http://www.ericzma.com/)
# Dec. 27, 2010

mark_file=/tmp/xen-bak-marker
log_file=/lhome/xen/bak-111.log
err_log_file=/lhome/xen/bak_err-111.log
xen_dir=/lhome/xen
local_bak_dir=/lhome/xen-bak-tmp
bak_dir=xenbak@backup_host1:/lhome/xenbak
xen_name_list="10.10.10.111"

# check whether bak runned last week
if [ -e $mark_file ] ; then
  rm -f $mark_file
else
  touch $mark_file
  # exit 0
fi

# set std and stderr to log file
mv $log_file $log_file.old
mv $err_log_file $err_log_file.old
exec 2> $err_log_file
exec > $log_file 

# check whether the VM is running
# We only backup running VMs

echo "*** Check alive VMs"

xen_name_list_tmp=""

for i in $xen_name_list
do
  /usr/sbin/xm list > /tmp/tmp-xen-list
  grepinlist=`grep $i" " /tmp/tmp-xen-list`
  if [[ $grepinlist == "" ]]
  then
    echo $i is not alive.
  else
    echo $i is alive.
    xen_name_list_tmp=$xen_name_list_tmp" "$i
  fi
done

xen_name_list=$xen_name_list_tmp

echo "Alive VM list:"

for i in $xen_name_list
do
   echo $i
done

echo "End alive VM list."

###############################
date 
echo "*** Backup starts" 

###############################
date
echo "*** Copy VMs to local disk"

for i in $xen_name_list
do
  date 
  echo "Copy $i" 
  echo "Shutdown $i" 
  /usr/sbin/xm shutdown $i 
  sleep 30
  echo "Copy to local_bak_dir: $local_bak_dir" 
  cp $xen_dir/vm-$i/vmdisk0 $local_bak_dir/vmdisk0-$i 
  cp $xen_dir/vm-$i/vm.run $local_bak_dir/vm.run-$i 
  date 
  echo "Create $i"
  # with vmmem=1024" 
  # /usr/sbin/xm create $xen_dir/vm.run vmid=$i vmmem=1024 
  /usr/sbin/xm create $xen_dir/vm-$i/vm.run
  scp $log_file $bak_dir/bak-111.log
  cp $log_file $local_bak_dir/bak-111.log
done

####################
date
echo "*** Compress local bak vmdisks"

for i in $xen_name_list
do
  date 
  echo "Compress $i" 
  tar -z -cf $local_bak_dir/vmdisk0-$i.tar.gz $local_bak_dir/vmdisk0-$i $local_bak_dir/vm.run-$i
  rm -f $local_bak_dir/vmdisk0-$i $local_bak_dir/vm.run-$i
done

####################
date
echo "*** Copy local bak vmdisks to remote machines"

for i in $xen_name_list
do
  date 
  echo "Copy to remote: vm$i" 
  scp $local_bak_dir/vmdisk0-$i.tar.gz $bak_dir/vmdisk0-$i.tar.gz
done

#####################
date 
echo "Backup finishes" 
scp $log_file $bak_dir/bak-111.log

Eric Zhiqiang Ma

Eric is interested in building high-performance and scalable distributed systems and related technologies. The views or opinions expressed here are solely Eric's own and do not necessarily represent those of any third parties.

4 comments:

  1. Hello. Is it possible to do that the script would delete old backups? I mean keeping the last 2 last. Backup once per day.

      1. I do not really know much about scripts. Could you tell me how to do it, I will be very grateful.

        1. Sorry I do not have the time to modify the script and debug it currently.

          On the other hand, I am afraid that, even with the script modified with the function that you need, you still need to learn a little bit of bash/ssh/scp if you really want to use this script. You need to test, debug and maintain the script in your environment as time goes…

Leave a Reply

Your email address will not be published. Required fields are marked *