Xen Project Comprehensive Cheatsheet

1. Installation and Setup (Dom0 Configuration)

Prerequisites

Bash

1
2
3
4
5
6
7
8
9
10
# Update system packages
sudo apt-get update && sudo apt-get upgrade

# Install Xen hypervisor and tools (Xen 4.19 is the current stable series)
sudo apt-get install xen-hypervisor-amd64 xen-tools xen-utils

# Verify Xen installation
sudo xl list
sudo xl info

Bootloader Configuration (GRUB)

Bash

1
2
3
4
5
6
7
8
9
10
# Edit GRUB configuration
sudo nano /etc/default/grub

# Add Xen parameters (dom0 memory ballooning is generally discouraged in production)
GRUB_DEFAULT="Xen 4.19"
GRUB_CMDLINE_XEN_DEFAULT="dom0_mem=4G,max:4G dom0_max_vcpus=4 dom0_vcpus_pin"

# Update GRUB
sudo update-grub

2. Xen Hypervisor Architecture Overview

Key Components

Hypervisor (Xen): Lightweight, type-1 bare-metal virtualization layer
Dom0: Privileged control domain with direct hardware access (PV or PVH)
DomU: Unprivileged guest domains (virtual machines)
Driver/Stub Domains: Optional disaggregated service domains for isolation

Virtualization Types

PV (Paravirtualized): Modified guest OS, direct hypercall communication. 32-bit PV guests are deprecated/removed; 64-bit PV remains supported but is no longer the recommended default
HVM (Hardware-Assisted): Full virtualization, requires Intel VT-x or AMD-V
PVH: Default recommended mode — lightweight HVM container running PV-aware guests with no QEMU device model required for most workloads

3. Domain Management (DomU Creation and Configuration)

Create Configuration File

Bash

1
2
3
4
5
6
7
8
9
10
11
# Sample PVH domain configuration (/etc/xen/example-pvh.cfg)
name = "example-pvh"
type = "pvh"
kernel = "/boot/vmlinuz-linux"
ramdisk = "/boot/initramfs-linux.img"
cmdline = "root=/dev/xvda1 ro console=hvc0"
memory = 2048
vcpus = 2
disk = ['phy:/dev/vg0/example-pvh,xvda,w']
vif = ['bridge=xenbr0']

Create HVM Domain Configuration

Bash

1
2
3
4
5
6
7
8
9
10
11
# Sample HVM domain configuration (/etc/xen/example-hvm.cfg)
name = "example-hvm"
type = "hvm"
firmware = "bios"          # or "uefi" for OVMF
memory = 4096
vcpus = 4
disk = ['file:/var/lib/xen/images/example-hvm.img,xvda,w']
vif = ['bridge=xenbr0,model=virtio-net-pci']
vnc = 1
vnclisten = "127.0.0.1"

4. xl Toolstack Commands

Domain Lifecycle Management

Bash

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# Create and start a domain
xl create /etc/xen/example-pvh.cfg

# List running domains
xl list

# Pause / unpause
xl pause example-pvh
xl unpause example-pvh

# Graceful shutdown
xl shutdown example-pvh

# Force stop a domain
xl destroy example-pvh

# Attach to a domain console
xl console example-pvh

Note: the legacy xm toolstack and xend daemon have been removed for many releases. Use xl exclusively.

5. Virtual Machine Lifecycle Management

Common Lifecycle Operations

Bash

1
2
3
4
5
6
7
8
9
10
11
12
# Save domain state (suspend to file)
xl save example-pvh /var/lib/xen/save/example-pvh.save

# Restore domain state
xl restore /var/lib/xen/save/example-pvh.save

# Reboot domain
xl reboot example-pvh

# Create a live snapshot via libvirt (optional)
virsh snapshot-create-as --domain example-pvh snap1

6. Storage Configuration

File-Based Storage

Bash

1
2
3
4
5
6
7
8
9
# Create raw disk image
dd if=/dev/zero of=/var/lib/xen/images/example.img bs=1M count=20480

# Or use qemu-img for sparse/qcow2 (HVM only for qcow2)
qemu-img create -f qcow2 /var/lib/xen/images/example.qcow2 20G

# LVM Storage (recommended for performance)
sudo lvcreate -L 20G -n example-lv vg0

Storage Configuration in Domain Config

disk = [
    'file:/var/lib/xen/images/example.img,xvda,w',
    'phy:/dev/vg0/example-lv,xvdb,w',
    'format=qcow2,vdev=xvdc,access=rw,target=/var/lib/xen/images/data.qcow2'
]

Supported backends include phy:, file: (raw via blktap2/loop), and qdisk (QEMU-backed qcow2/raw). The legacy tap2: alias is deprecated.

7. Network Configuration

Bridge Setup (systemd-networkd / Netplan / ifupdown all supported)

Bash

1
2
3
4
5
6
7
8
9
10
11
12
# Install bridge utilities
sudo apt-get install bridge-utils

# Configure bridge (/etc/network/interfaces)
auto xenbr0
iface xenbr0 inet static
    address 192.168.1.10/24
    gateway 192.168.1.1
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0

VLAN, Open vSwitch and SR-IOV

Bash

1
2
3
4
5
6
7
8
9
# VLAN tagged vif
vif = ['bridge=xenbr0,vlan=10']

# Open vSwitch bridge
vif = ['bridge=ovsbr0,script=vif-openvswitch']

# PCI passthrough / SR-IOV virtual function
pci = ['0000:02:00.1']

8. PV vs HVM vs PVH Domain Types

PV

Requires PV-aware guest kernel
Lower overhead historically, but 32-bit PV has been removed and PV is no longer the recommended default
Direct hypercall communication

HVM

Full hardware virtualization, runs unmodified OSes (Windows, BSD, etc.)
Requires Intel VT-x or AMD-V; nested EPT/NPT recommended
Uses QEMU device model for emulated devices; virtio and PV drivers strongly recommended

PVH (recommended default)

Lightweight HVM container with PV interfaces
No QEMU device model needed for typical workloads → smaller attack surface
Supported as Dom0 mode (dom0=pvh on the Xen command line)

9. Resource Allocation

CPU Pinning and CPU Pools

Bash

1
2
3
4
5
6
7
8
9
10
# Pin vCPUs to specific physical CPUs
xl vcpu-pin example-pvh 0 0
xl vcpu-pin example-pvh 1 1

# CPU pools (cpupools) for partitioning
xl cpupool-list
xl cpupool-create name=\"pool1\" sched=\"credit2\"
xl cpupool-cpu-add pool1 2-5
xl cpupool-migrate example-pvh pool1

Memory Management

Bash

1
2
3
4
# Dynamic memory allocation (requires maxmem >= target)
xl mem-set example-pvh 4096
xl mem-max example-pvh 8192

10. Live Migration

Prepare Migration

Bash

1
2
3
4
5
6
7
8
9
# Cold/live migration to another host (SSH transport by default)
xl migrate example-pvh destination-host

# Localhost migration (testing)
xl migrate --debug example-pvh localhost

# Remus / COLO high-availability replication (where available)
xl remus -i 100 example-pvh backup-host

Note: xl migrate-live is deprecated/aliased; xl migrate performs a live migration by default.

11. Configuration Files

Key Configuration Locations

/etc/xen/xl.conf: Global xl toolstack configuration
/etc/xen/*.cfg: Individual domain configurations
/etc/default/xen and /etc/default/xendomains: Distro defaults and auto-start lists
/etc/xen/scripts/: Helper scripts (vif-bridge, block-*, etc.)

Configuration Syntax Example

# /etc/xen/xl.conf
autoballoon="off"
lockfile="/var/lock/xl"
vif.default.script="vif-bridge"

12. Security and Isolation

Mandatory Access Control

XSM/FLASK (Xen Security Modules) for fine-grained domain policy
SELinux or AppArmor inside Dom0/DomU
Stub domains and driver domains for service disaggregation
IOMMU (Intel VT-d / AMD-Vi) is required for safe PCI passthrough

Hardening Recommendations

Bash

1
2
3
4
5
6
7
8
# Enable XSM/FLASK on the hypervisor command line
GRUB_CMDLINE_XEN_DEFAULT="... flask=enforcing"

# Run Dom0 as PVH for a smaller attack surface
GRUB_CMDLINE_XEN_DEFAULT="... dom0=pvh,verbose"

# Subscribe to Xen Security Advisories (XSAs) and apply promptly

13. Performance Tuning

Optimization Techniques

Prefer PVH guests; for HVM use virtio-net / virtio-blk / virtio-scsi PV drivers
Use credit2 scheduler (default) and isolate Dom0 with dom0_vcpus_pin
Enable IOMMU and use PCI passthrough/SR-IOV for I/O-bound workloads
Pin Dom0 and high-priority guests to dedicated cpupools
Disable unused emulated devices in HVM configs
For Intel/AMD, ensure microcode is current and relevant speculative-execution mitigations are configured (spec-ctrl= Xen options)

14. Monitoring and Logging

Monitoring Tools

Bash

1
2
3
4
5
6
7
8
9
10
11
12
# Xen-specific monitoring
xl top              # alias for xentop
xentop
xl info
xl dmesg
xenstore-ls

# Per-domain stats
xl vcpu-list
xl network-list example-pvh
xl block-list example-pvh

Logging

Bash

1
2
3
4
5
6
# Xen hypervisor and toolstack logs
/var/log/xen/hypervisor.log
/var/log/xen/xl-<domain>.log
/var/log/xen/qemu-dm-<domain>.log
journalctl -u xen-*           # systemd units on modern distros

Note: xend.log no longer exists; xend was removed long ago in favor of libxl/xl.

15. Troubleshooting

Common Issues

Verify VT-x/AMD-V and IOMMU are enabled in firmware
Check xl dmesg for hypervisor errors and XSA mitigation messages
Validate kernel/initrd paths and console settings (console=hvc0)
Confirm bridge/OVS configuration with ip link and brctl show

Debugging Commands

Bash

1
2
3
4
5
6
7
8
9
# Xen version and build info
xl info
xl dmesg | head -40

# Detailed domain info
xl list -l example-pvh
xl uptime
xenstore-ls -f /local/domain/<domid>

16. Cloud and Orchestration Integration

Cloud / Management Stacks

libvirt (virsh -c xen:///) for libvirt-based tooling
OpenStack via the libvirt+Xen driver
Apache CloudStack (Xen/XCP-ng hypervisor support)
XCP-ng and Citrix Hypervisor (XenServer) for turnkey Xen distributions
Xen Orchestra for management of XCP-ng/XenServer pools

Pro Tips:

Track the Xen Project release cadence (current stable series: 4.19; 4.20 in development) and security advisories at xenproject.org
Prefer PVH over classic PV for new deployments
Use configuration management (Ansible/Salt) for /etc/xen/*.cfg and bridge config
Regularly test save/restore and live migration as part of disaster-recovery drills
Keep CPU microcode and Xen patched against published XSAs

Xen Project

Continue Learning