SEEK_HOLE and SEEK_DATA: efficiently archive/copy large sparse files

Posted on

How to efficiently archive a very large sparse file, say 1TB? The sparse file may contains a small amount of data, say 32MB. SEEK_HOLE and SEEK_DATA The SEEK_HOLE/SEEK_DATA functionalities play the trick and makes `tar` and `cp` handle the large sparse file very efficiently. `lseek` with `SEEK_HOLE` returns the offset of the start of the
Read more

Managing LVM

Posted on

Any tutorials or tips on how to manage LVM? LVM provides a flexible and easy way to management storage disks on Linux. It also provides a set of tools for management. Here are some tutorials and references for managing LVM: LVM man pages: https://www.systutorials.com/docs/linux/man/8-lvm/#lbAL Manage Disk Volume in Linux: http://www.hongkedavid.com/blog/disk_volume.html LVM with Xen: https://www.systutorials.com/b/linux/tag/lvm/ Remove missing phyiscal volumes
Read more

How to Install Paravirtualized CentOS 7 DomU on Xen

Posted on

This post introduces how to install a paravirtualized CentOS 7 DomU on Xen. The very common way of installing DomU does not work for CentOS 7. A little trick to set the repository and the network used by the VM should be used by adding a setting strings to the “extra=” field for this VM.
Read more

How to Create Fedora 20 Domain-U on Fedora 20 Domain-0

Posted on

In this post, creating a file-backed virtual block device (VBD) and installing Fedora 20 in the Xen DomU via internet will be introduced. This domain is created on a Fedora 20 Dom0 as introduced in https://www.systutorials.com/installing-xen-on-fedora-20-as-domain-0/. For better performance, you may consider using LVM backed VM. Create file-backed VBD The actual space of VBD will
Read more

Installing Xen on Fedora 20 as Domain-0

Posted on

I ever introduced [[installing-xen-domain-0-on-fedora-17]] as the first try to use the xen package delivered from Fedora and get away from manually compiling Xen and patching the kernel. In this post, I introduce installing Xen Dom0 on Fedora 20. Installing Xen First, install the xen pacakges: # yum install xen The Linux kernel is already ready
Read more

Managing Xen Dom0′s CPU and Memory

Posted on

The performance of Xen’s Dom0 is important for the overall system. The disk and network drivers are running on Dom0. I/O intensive guests’ workloads may consume lots Dom0′s CPU cycles. The Linux kernel calculates various network related parameters based on the amount of memory at boot time. The kernel also allocate memory for storing memory
Read more

Script: Shutting Down All Xen VMs on a Server

Posted on

Shutting down servers is a common operations for managing a cluster. However, if this server is configured to a Xen Dom0 and has Xen VMs (DomUs), the VMs should be shutdown first to avoid data lost on these VMs. xm supports a -a option to shutdown all VMs: # xm shutdown -a Add the -w
Read more

Installing Fedora 17 PV Domain-U on Xen with PXE Booting

Posted on

An introduction to the general method of installing Domain-U on Xen is introduced here: Setting Up Stable Xen DomU with Fedora: Unmodified Fedora 12 on top of Xenified Fedora 12 Dom0 with Xen (this is a general introduction, some details are changed, such as ‘xl’ replacing ‘xm’, LVM backing the disk for higher performance. But
Read more

How to Install Xen on Fedora as Domain-0 (Fedora 17)

Posted on

The new development of Xen and Linux kernel make it easy to install Xen on Fedora as the Domain-0 now. This post uses Fedora 17 as an example platform to introduce how to set up Domain-0 on Fedora Linux. Compared to our old method (https://www.systutorials.com/setting-up-stable-xen-dom0-with-fedora-xen-3-4-3-with-xenified-linux-kernel-2-6-32-13-in-fedora-12/) which requires manually compiled Xen and patched kernel, the current
Read more

How to Set Default Entry in Grub2 and Grub

Posted on

Linux booting is usually controlled by Grub or the new Grub2. Setting the default booting entry is a frequent operations. Here, we introduce how to set the default entry in Grub2 and Grub. Setting the default booting entry in grub2 Note1: With some version of grub2, the grub2-set-default method and the script below may not
Read more

Extending a LVM Volume Group

Posted on

We introduced how to create a LVM group with two hard disk partitions, which the LVM group can be used for [[setting-up-lvm-backed-xen-domu|installing LVM backed Xen DomUs]]. The LVM volume can have large capacity from several hard disks. However, we may use all the disk space after some time and we may need to extend the
Read more

Setting Up Ubuntu DomU on Xen: Ubuntu 10.10 on Fedora Xen Dom0

Posted on

Setting up Ubuntu 10.10 DomU on top of Fedora Xen Dom0 is introduced in this post. The process of setting up Ubuntu 10.10 DomU is the same as Setting Up Stable Xen DomU with Fedora: Unmodified Fedora 12 on top of Xenified Fedora 12 Dom0 with Xen 4.0 This post only show the difference which
Read more

Simple Introduction to paravirt_ops for Xen

Posted on

The is a simple introduction to paravirt_ops in Linux kernel for Xen, VMware, etc. We make this introduction from the view of code. We use the function raw_local_irq_disable() and raw_local_irq_enable() functions in Linux kernel to introduce paravirt_ops for Xen and Xenified kernel. Please download the introduction to paravirt_ops pdf file: introduction-to-pv-ops-v3.pdf

Xen with LVM

Posted on

LVM volumes as backing for DomU’s file system is an appealing solution to Xen VBD. LVM volumes can dynamically grow/shrink and snapshot. These features make it simple and fast to duplicate DomU and adding storage to DomU. This post is a summary of tutorials related Xen DomU and LVM. Setting Up LVM Backed Xen DomU
Read more

Creating LVM Volume Group

Posted on

How to create a LVM group from two hard disk partitions is introduced in this post. Assume we have installed two hard disks /dev/sdb and /dev/sdc to the server, our task now is to create one LVM volume group vg_xen for installing LVM backed Xen DomUs from these two disks. We will do this by
Read more

Duplicating and Backing Up LVM Backed Xen DomU from a Remote Server

Posted on

LVM’s snapshot feature enables us to duplicate an LVM backed Xen DomU in seconds inside of the same volume group. When we want to duplicate LVM backed Xen DomU from a remote server, we need to make use of LVM’s snapshot function and tools like scp and dd. Backing up the DomU is only part
Read more

Setting Up LVM Backed Xen DomU

Posted on

LVM volumes as backing for DomU’s file system is an appealing solution to Xen VBD. LVM volumes can dynamically grow/shrink and snapshot. These features make it simple and fast to duplicate DomU and adding storage to DomU. Creating LVM-backed Xen DomU is introduced in this post. Create LVM-backed VBD Suppose we have a LVM volume
Read more

Duplicating LVM Backed Xen DomU

Posted on

LVM’s snapshot feature enables us to duplicate an LVM backed Xen DomU in seconds rather than minutes. We no longer need to copy the entire file system image like backing up file backed Xen DomU. We just need to make a snapshot of the current Xen DomU in seconds. When there are changes to the
Read more

An Introduction to Xen Source Code Structure and Disk in Xen

Posted on

Please note these slides on Xen was made in around 2013 and the Xen source code structure might have already changed a lot. Please check this as a reference to the Xen source structure only. I have created slides from introducing Xen’s source code structure and backend and frontend drivers in Xen. Please find the
Read more

Xen DomU’s I/O Performance of LVM and loopback Backed VBDs

Posted on

This posts list benchmark (using bonnie++) result of I/O performance of Xen LVM and loopback backed VBDs. The configuration of machines Dom0 VCPU: 2 (Intel(R) Xeon(R) CPU E5520  @ 2.27GHz) Memory: 2GB Xen and Linux kernel: Xen 3.4.3 with Xenified 2.6.32.13 kernel DomU VCPU: 2 Memory: 2GB Linux kernel: Fedora (2.6.32.19-163.fc12.x86_64) DomU’s profile: name=”10.0.1.200″ vcps=2
Read more