It all started with pvgrub2: it was March 2015 and I wanted to add grub2 to the Xen build system. We were already building grub-legacy as part of the Xen build, so that we could produce a pvgrub binary to be used to boot PV guests. After Vladimir ‘phcoder‘ Serbinenko’s good work on grub2, the latest and greatest upstream grub2 could be built with Xen support and used to boot PV guests. It made perfect sense to add grub2 to the Xen build system too, right? Maybe not. Unexpectedly some key Xen Project contributors pushed back: “there doesn’t seem to be a good reason for cloning and building yet another third-party project as part of the Xen build”, wrote David Vrabel.
Conflicting requirements
It was then that I realized that we have two contrasting set of requirements: on one hand we want to support users that clone xen-unstable, build everything from source, and expect the system to be fully ready after typing ./configure; make; make install. On the other hand, we also want to support distros and product groups that take Xen releases and integrate them into their Linux distros or enterprise build systems. The former want things like grub2 to be part of the xen-unstable build, because the grub2 package provided by their distro doesn’t necessarily comes with Xen support enabled. While the distro packagers are already building a grub2 package and certainly don’t want xen-unstable to go and clone grub2 again. They probably abhor the whole idea of xen-unstable git cloning external trees without their explicit assent. In fact they had been carrying patches to make sure xen-unstable doesn’t clone anything else “behind their back”, until we provided build options to disable all the third-party builds.
Raisin: Xen’s DevStack
How to find a solution that would make both camps happy? Surely others must have had the same issue. Is there another open source project that has to build several separate components in order to be fully functional? Yes, of course, there are many. One of them is OpenStack and it solves the problem by providing a set of scripts called DevStack, which build and setup the system from scratch.
This is where “Raisin” comes from. I announced the new project on the 31st of March 2015. The idea is that Raisin takes care of building Xen and all the other components, which are required to have a fully functional Xen system, but that don’t belong to xen-unstable. For example QEMU, SeaBIOS, and, of course, grub2. Users that build everything from source will clone Raisin to find a single place where they can build all the latest and greatest Xen stuff with a single command. Raisin can be very useful to setup a development environment too. Distro people can refer to Raisin as the most common way to build, install and configure Xen and related components, but they are unlikely to actually use it to build their packages. Raisin helps Xen developers improve the boundaries and interfaces between Xen and external components, by making such boundaries clearer and more explicit. Things like QEMU and SeaBIOS, currently cloned and built by xen-unstable, will be moved out to Raisin, making both Xen maintainers and distro packagers happier. Other Xen related components, that are good to have but not actually required, such as libvirt, will find their place in Raisin too.
Raisin: where we are, what’s next
After few busy months of development, we now have a set of bash scripts that can install dependencies and build Xen, QEMU, qemu-traditional, SeaBIOS, OVMF, Grub2, Libvirt and Linux with a single command. All you need to do is edit the config file, type raise -y build, go get a coffee, and everything will be ready when you come back. Raisin is not tied to a specific version of Xen. In fact, one can choose any git tags or commit ids newer than Xen 4.5 (RELEASE-4.5.0 is the git tag for the Xen 4.5 release) and Raisin will build it. Other commands are available to install and configure the system with the most common settings. Give a look at the README for an introduction on how to use the command line tool.
During the last few weeks I have been working on integrating Raisin in OSSTest, the automated testing framework run by Xen Project. I am currently adding a new test to validate Raisin itself, but going forward it makes sense to actually use Raisin to build Xen, QEMU and anything else OSSTest needs, similarly to what DevStack does for the OpenStack gate.
Making testing easier and accessible to everybody
Talking about tests, this is another area where Raisin can help greatly. I always liked the idea of providing a set of unit and functional tests, quick and simple to run, that can be executed by any Xen contributors to validate their changes before sending a patch to xen-devel. However we didn’t really have place to put them. OSSTest is too big and tightly coupled to the Xen Project Test Lab infrastructure for this use case, and the last thing xen-unstable needs is more scripts. On the other hand, Raisin is at the right abstraction level to run functional tests for the components it already builds. I introduced a few simple tests, which can stack on top of each other, to create busybox based PV and HVM guests. I plan to continue adding more tests, then expose them to OSSTest via Raisin, so that they will be continuously run by the Xen Project Test Lab. But, at the same time, anybody can still manually execute them on their test box with a single raise test command. I am hoping to be able to start asking contributors to run Raisin tests before submitting patches early in the next release cycle. If you use Xen and know bash scripting, you should consider writing a Raisin test to validate your favourite functionality today!
Raisin, you didn’t know you needed it, you can’t live without it 😉
Resources
The Raisin git repository is available here. The README is up to date and describes the command line interface. We also have quickstart guide on our wiki. Raisin patches are discussed on xen-devel and follow the regular Xen development process.