Managing Xen Dom0′s CPU and Memory

The performance of Xen’s Dom0 is important for the overall system. The disk and network drivers are running on Dom0. I/O intensive guests’ workloads may consume lots Dom0′s CPU cycles. The Linux kernel calculates various network related parameters based on the amount of memory at boot time. The kernel also allocate memory for storing memory metadata (per page info structures) is also based on the boot time amount of memory. After ballooning down Dom0′s memory, the network related parameters will not be correct. Ballooning down busy Dom0′s memory sometimes cause SSH to die from our observation, which is a nightmare for the administrator since SSH is usually the only way for remote control of the server. Another bed effect is that it’s a waste of memory with a large memory metadata for a smaller memory amount.

Please refer to Xen solution for the more latest stable Xen Dom0 solution.

Now let’s look at how to menage Xen Dom0′s CPU and memory in a better way.

Dedicate a CPU core for Dom0

Dom0 will have free CPU time to process the I/O requests from the DomUs if it has dedicated CPU core(s). Better performance may be achieved by this since there are less CPU context switches to do in Dom0.

We can dedicate CPU core for Dom0 by passing “dom0maxvcpus=X dom0vcpuspin” options to Xen hypervisor (xen.gz) in /boot/grub/grub.conf. X is the number of vcpus dedicated to Dom0.

As hyperthreading technology is enabled in most modern CPUs, we need to specify two processors to dedicate one CPU core. So the “X” above should usually be 2 for one CPU core.

kernel /xen.gz console=vga vga=ask noreboot dom0maxvcpus=2 dom0vcpuspin

After booting the system, the VCPU list can be got on Dom0 by this command:

# xm vcpu-list

Even after booting the system, the VCPU number can be configured by xm command. We can set Domain-0 have two VCPUs and processor 0 and 1 to be dedicated to Dom0 by these commands:

# xm vcpu-set Domin-0 2
# xm vcpu-pin Domain-0 0
# xm vcpu-pin Domain-0 1

Dedicate memory for Dom0

We should always dedicate fixed amount of memory for Xen Dom0.

We can set the initial memory size of Dom0 by passing “dom0_mem=xxx” (in KB) option to Xen hypervisor (gen.gz) in /boot/grub/grub.conf. “xxx” is the amount of memory for Dom0 in KB.

If we set the initial memory size of Dom0 to 2GB, just change the entry in grub.conf to:

kernel /xen.gz console=vga vga=ask noreboot dom0maxvcpus=2 dom0vcpuspin dom0_mem=2097152

Set lowest permissible memory for Dom0

The option dom0-min-mem in Xend configuration file /etc/xen/xend-config.sxp is used to specify the lowest permissible memory for Dom0.

The value of dom0-min-mem (in MB) is the lowest permissible memory level for Dom0. The default value is 256. If we limit the memory size of Dom0 to 2G, just set:

(dom0-min-mem 2048)

Preventing dom0 memory ballooning

The “enable-dom0-ballooning” option in Xend configuration file is used to specify whether Dom0′s memory can be ballooned out. Setting “enable-dom0-ballooning” to “no” will make sure Xend never takes any memory away from Dom0:

(enable-dom0-ballooning no)