True static partitioning with Xen Dom0-less

The Xen Project hypervisor has relied on a special virtual machine, Dom0, to perform privileged operations since the early days of the project. Dom0 has always been the very first environment to come up at boot time, providing a familiar Linux command line interface for the users. With Dom0-less, we introduced a radical new way of using the Xen hypervisor aimed at embedded Arm systems: it makes it possible to have a Xen deployment with multiple virtual machines, without any Dom0, and without any Xen userspace tools.

The main reason for using Xen Dom0-less is to shorten the boot-time of critical applications. In a traditional Xen and Dom0-based system, one has to wait for Dom0 to be fully booted before creating any other virtual machines. Xen was originally written with the datacenter in mind, where short boot-times are nice to have but not a strict requirement. Without Dom0less, in the best of cases, it will take several seconds to boot a VM measured from system bring-up: Xen needs to boot first, then the Dom0 kernel, then Dom0 userspace, finally `xl’ becomes available, and other VMs can be started. At Xilinx, we focus on embedded deployments: often, our users need to be able to have several VMs up and running in less than a second, which is an order of magnitude less than what we were able to do until now.

With Dom0-less, Xen boots selected VMs in parallel on different physical CPU cores directly from the hypervisor at boot time. That means that a VM carrying a motor control application has only to wait for Xen to be ready before starting. It drastically reduces boot times — booting in less than a second is certainly possible now.

Hardware resources can be directly assigned to Dom0-less VMs. Xen Dom0-less is a natural fit for static partitioning, where a user splits the platform into multiple isolated domains and runs different operating systems on each domain. The configuration (both the domains to boot and device assignments) is done via device tree. U-Boot is responsible for loading all the required binaries, such as the domains kernels and ramdisks. It is possible to issue the U-Boot commands and add any device tree configurations by hand, but it is best to have it done automatically for you. A set of tools that were recently published by the community (see gitlab.com/ViryaOS/imagebuilder and wiki.xenproject.org/wiki/ImageBuilder) can be used to generate a U-Boot script that loads everything necessary and modifies an existing device tree automatically at boot time to make the system boot under Xen without any manual intervention. They make the task of configuring a Xen Dom0-less system quick and easy.

Despite the name “Dom0-less”, Xen still starts a Dom0 VM at boot, but the hypervisor creates additional VMs in parallel without any interactions or help from Dom0. Having a Dom0 environment can be convenient because it allows users to monitor the system, start/stop additional VMs, reboot the platform, etc. However, since Dom0 is not required to start or run VMs, it becomes possible to get rid of it, leading to “true Dom0-less” systems where only regular unprivileged guests are running. True Dom0-less is desirable for security reasons, to reduce the surface of attack, to build a system without any privileged VMs, or simply to save on resource utilization.

One interesting side effect of not having Dom0 is that the Xen userspace tools are not necessary anymore. Especially in cross-building environments, it is not easy to compile the Xen tools: typically a fully-featured build system such as Yocto is required. With Dom0-less, only the Xen hypervisor binary is necessary to start multiple VMs. Thus, users just build the hypervisor binary, which takes far less time and can be accomplished with a single `make’. In addition to shortening the actual boot time, Dom0-less also reduces the overall time to build, configure, and assemble a multi-domains system.

Dom0-less, including device assignment, is fully upstream in the upcoming Xen v4.13 release. User feedback started to seep through and has been very positive so far. I want to encourage the readers to give it a try and explore this new way of bringing virtualization to embedded systems.

Read more

Say hello to our new website
12/05/2024

Hello Xen Community, You may have noticed something different... We've refreshed our existing website! Why did we do this? Well, all these new changes are part of an ongoing effort to increase our visibility and make it easier to find information on pages. We know how important it

Xen Project Announces Performance and Security Advancements with Release of 4.19
08/05/2024

New release marks significant enhancements in performance, security, and versatility across various architectures.  SAN FRANCISCO – July 31st, 2024 – The Xen Project, an open source project under the Linux Foundation, is proud to announce the release of Xen Project 4.19. This release marks a significant milestone in enhancing performance, security,

Upcoming Closure of Xen Project Colo Facility
07/10/2024

Dear Xen Community, We regret to inform you that the Xen Project is currently experiencing unexpected changes due to the sudden shutdown of our colocated (colo) data center facility by Synoptek. This incident is beyond our control and will impact the continuity of OSSTest (the gating Xen Project CI loop)

Xen Summit Talks Now Live on YouTube!
06/18/2024

Hello Xen Community! We have some thrilling news to share with you all. The highly anticipated talks from this year’s Xen Summit are now live on YouTube! Whether you attended the summit in person or couldn’t make it this time, you can now access all the insightful presentations