One of the challenges of using Xen in embedded environments is the need for core components to meet critical timing requirements. In traditional implementations engineers use real-time operating systems (RTOS) to ensure, for example, that an automobile’s brakes engage within a reasonable amount of time after the driver presses the brake pedal. It would clearly be bad if such a command were to be delayed unduly due to the car’s navigation or entertainment systems.
Over the last year, Galois has been trying to simplify one aspect of this challenge by porting an open-source RTOS, FreeRTOS, to Xen. This allows engineers to implement Xen domains that meet their own independent timing requirements. To be fully effective, this must be combined with a real-time scheduler for Xen itself, allowing for a top-to-bottom real time system. For now, we have had some success getting real-time behavior by simply pinning real-time domains to one CPU on a multi-CPU system.
In this article I’ll show you how you can get the code and how you can build your own FreeRTOS-based kernels for use on Xen/ARM systems. I’ll also provide links to technical documentation for those interested in learning more about the implementation.
As part of the community’s commitment to improving Xen for non-datacenter uses, Galois is also a member of the Xen project’s Embedded and Automotive team. Because of its key isolation and security features, flexible virtualization mode and architecture, driver disaggregation and ARM support (only 90K lines of code), Xen is a perfect fit for embedded applications.
Getting the code
The source is on GitHub and can be obtained with git:
git clone https://github.com/GaloisInc/FreeRTOS-Xen.git
Setting up
For the purposes of this article I’ll assume an Ubuntu Linux development system. I’ll cover how to cross-compile FreeRTOS but I’ll assume you already have Xen deployed on an ARM system. To build FreeRTOS, you’ll need:
- An installed collection of Xen development headers corresponding to the version of Xen running on your ARM system
- An ARM cross compiler toolchain (installable on Ubuntu under the package name
arm-none-eabi-gcc)
Building FreeRTOS
The repository includes an example application in the Example/ directory. This application starts up some simple tasks and shuts down, but most of what happens in the application takes place before main() runs while the Xen services such as the console are configured.
Before we can build the application itself, we’ll need to build the FreeRTOS library. To do that, from the FreeRTOS-Xen repository, run
$ make -C Demo/CORTEX_A15_Xen_GCC
This builds FreeRTOS.a, which is everything but your application – which we’ll build and link in the next step.
NOTE: if your cross compiler name prefix isn’t arm-none-eabi-, adjust the CROSS_COMPILE variable in the Makefile accordingly. The same goes for the location of your Xen headers, which is configured by XEN_PREFIX.
Building the example application
Now we can build the application and link it against the FreeRTOS library. To do so, from the FreeRTOS-Xen repository, run
$ make -C Example/
This will build two binaries:
Example/Example.elf: the ELF version of your FreeRTOS application, suitable for debugging and disassembly with GDB and other toolsExample/Example.bin: the binary version of your FreeRTOS application, suitable for deploying as the kernel image of a Xen VM
Starting a FreeRTOS Xen guest
Once we’ve built Example.bin, we’ll need to place it somewhere on the filesystem for our ARM system and create a Xen guest configuration file, say Example.cfg, as follows (with the path to the kernel adjusted accordingly):
kernel = ".../path/to/Example.bin" memory = 16 name = "Example" vcpus = 1 disk = []
This port of FreeRTOS does not use multiple CPUs, so we assign only one. The application we’ve built could probably run with even less memory; the default configurable heap size is 1 MB, so even smaller memory sizes are realistic.
We can launch the VM with xl as usual:
xl create -f Example.cfg
Once the domain is up and has established a Xen console connection, we should see the application’s example tasks running and printing messages to the Xen emergency console:
Output from the example program included in the distribution of FreeRTOS for Xen on ARM
README for information about using the standard Xen console.)
Future Work & Contributions
This port is intended for research use, and we would love help with some of its missing or incomplete features. You can read a list of those in the readme in the repository and submit GitHub pull requests to contribute!
Learning More
Many more details of the port, including source code layout, Xen services, configuration parameters, memory layout, etc., can be found in the readme file in the repository. For information on FreeRTOS itself, the FreeRTOS web site provides excellent documentation on both FreeRTOS concepts and APIs.
Need help?
Contact me (Jonathan Daugherty) at jtd AT galois DOT com if you have questions and feel free to open tickets or submit pull requests on GitHub!