How to backup Xen with Logical Volume Mounts ; Works with HyperVM, SolusVM, FluidVM and More

Through our research and implementation of many Xen environments, it has become necessary to develop a reliable and secure method for backing up our Xen instances that are mounted on Logical Volumes (LVM).

The underlying problem is that the logical volume is usually a live file system that cannot be directly mounted / backed up or imaged safely.

We have written a script that processes all running Xen logical volumes, creates a snapshot of the volume and through that snapshot , uses dd to image the snapshot to another server over ssh.

You would be surprised at how well these dd images compress. Piping dd to bzip2 then to ssh to receive the image produces a very substantial compression ratio.

The initial trouble was writing the logic in the script to properly go through each Xen LV , create the snapshot, image and then remove the snapshot. Obviously extensive testing had to be completed to ensure reliability and proper error reporting.

This script should work with any 3rd party Xen control panel implementation (HyperVM, FluidVM, SolusVM to name a few). They all use the same underlying technology / framework. Since our script is a simple bash / shell script, it will run on any linux based system with little modification.

If you are using a LV for another purpose on the same box, it is probably a good idea to modify the script to ignore that so it doesn’t inadvertently get backed up.

Before implementing the script, it is probably a good idea to go through the motions manually just to see how it performs :

lvcreate -s -L 5G -n vm101_img_snapshot /dev/vps/vm101_img
dd if=/dev/vps/vm101_img_snapshot | bzip2 | ssh xenbackup@x.x.x.x "dd of=vm101_img.bz2"

One thing that you cant get around is space — you need to leave as much room as the largest Xen image on your logical volume — otherwise the script will fail at the snapshot creation process.

Find the script below. Hopefully it will help make your life easier (as well as being able to sleep at night) :

# XEN Backup script
# Written by Star Dot Hosting

todaysdate=`date "+%Y-%m-%d"`

echo "XEN Backup Log: " $currentmonth > /var/log/backup.log
echo -e "------------------------------------" >> /var/log/backup.log
echo -e "" >> /var/log/backup.log

for obj0 in $(lvs --noheadings --separator ',' -o lv_name,lv_size | grep -v "swap" | awk -F "," '{printf "%sn", $1}');

#grab the snapshot size
snapsize=`lvs --noheadings --separator ',' -o lv_name,lv_size | grep -v "swap" | grep $obj0 | awk -F "," '{printf "%s", $2}'`

#create the snapshot
lvcreate -s -L $snapsize -n $obj0_snapshot /dev/xenlvm/$obj0 >> /var/log/backup.log 2>&1

#dd piped to bzip2 to compress the stream before piping it over the network via ssh to the destination box
dd if=/dev/xenlvm/$obj0_snapshot | bzip2 | ssh xenbackup@ "dd of=/home/xenbackup/xen-backups/$obj0.$" >> /var/log/backup.log 2>&1

if [ "$?" -eq 1 ]
        echo -e "***SCRIPT FAILED, THERE WERE ERRORS***" >> /var/log/backup.log 2>&1
        cat /var/log/backup.log | mail -s "XEN Backup Job failed"
        lvremove -f /dev/xenlvm/$obj0_snapshot
        exit 1
        echo -e "Backup of $obj0 Completed Successfully!" >> /var/log/backup.log 2>&1

# remove the snapshot
lvremove -f /dev/xenlvm/$obj0_snapshot


cat /var/log/backup.log | mail -s "XEN Backup Job Completed"

If you plan on automating this script in a cronjob, it may be a good idea to utilize ssh key authentication between your destination server and your Xen server.

Migrate FreeBSD to Xen

There seems to be a lot of tutorials with respect to how you can dump/restore FreeBSD implementations. However, none of them appear to be all encompassing what is actually required from start to finish during the entire process.

The one thing that I think is lacking in proper documentation is utilizing FreeBSD in a LiveCD scenario (LiveFS) within a network capacity (necessary for migration).

We decided to write this tutorial so that people could have one place to establish all the necessary things required for this type of migration from start to finish.

In this scenario we actually migrated a FreeBSD implementation on VMWARE to XEN HVM. In the end, there were no technical problems with FreeBSD actually running after it was migrated — it ran beautifully actually.

I should note that this was tested with FreeBSD 7.2-RELEASE disc images.

Please find the guide below :

Prepare OLD Instance

1. Boot into old operating system

2. Take note of partition slices / slice names / sizes / etc

3. Reboot with FreeBSD LiveFS disc

Prepare NEW Xen

1. Boot Xen instance with FreeBSD Disc 1 ISO

2. Partition / install boot loader exactly the same slices as the old instance. To be extra careful, give your slices a bit more disc space than the old implementation.

3. Write changes & reboot with FreeBSD LiveFS disc

Establish FreeBSD LiveFS environment

You need to establish a few things to get SSH / DUMP / RESTORE to work properly on both the ”’old”’ and ”’new”’ instances

1. Boot into FreeBSD LiveFS (Fixit > livefs)

2. Create the following folders :


3. Copy the following files :

cp /mnt2/bin/ps /bin
cp /mnt2/sbin/sysctl /sbin
cp /mnt2/etc/ssh/* /etc/ssh
cp /mnt2/bin/csh /bin
cp /mnt2/bin/cat > /bin
cp /mnt2/sbin/restore > /sbin

4. Set an IP address on both old and new instances:

new :

ifconfig eth0 netmask

old :

ifconfig eth0 netmask

5. Start sshd :

/mnt2/etc/rc.d/sshd forcestart

Start transferring slices

1. To allow for transferring of partitions properly, the /tmp partition should be mounted on the new Xen instance :

mount -t ufs /dev/ad0s1e /tmp

2. For the first partition you wish to transfer, mount the empty slice on the new xen instance :

mount -t ufs /dev/ad0s1a /mnt/ufs.1

Sometimes you have to fsck mark the filesystem clean to mount it :

fsck /dev/ad0s1a

3. On the old instance :

dump -0aLf - /dev/ad0s1a | ssh "cd /mnt/ufs.1 && cat | restore -rf -"

That should dump/restore the slice from old > new.

Final things on the new Xen instance

Dont forget to boot the new instance in single user mode and modify ”’fstab”’ to reflect the new slice names (if applicable), as well as ”’rc.conf”’ for any hard coded interface names, etc. FreeBSD won’t boot if the right slice names / interface names aren’t present. Or at least cause problems.

You can mount the /etc slice while still in the LiveFS for the new FreeBSD instance.

Hopefully this was helpful! Obviously this has nothing to do with Xen, other than the fact that we were migrating the FreeBSD vmware instance to Xen.

You can do this on “real” machines, or from xen to vmware or anywhere. As long as the hardware is compatible.