Taking XenExpress for a Spin
According to the XenSource About page, "XenSource plays the dual role of leading the open source Xen(tm) community, while simultaneously selling value-added enterprise solutions based on Xen technology." The first part of that leads to various Linux distro makers integrating Xen into their distributions (like SUSE, Red Hat/clones, and Fedora). For the second part of that, XenSource currently offers a product line which includes XenExpress, XenServer and XenEnterprise. Of the three offerings, XenExpress is designed to be the entry level product and is free. I recently downloaded XenExpress and gave it a try.
During the course of this article I will describe the basic design of XenExpress, its installation, installation and use of the Administrator Console client application, creation, monitoring and management of Xen virtual machines... and then I'll try to contrast how XenSource's product line stacks up to Xen as offered by Red Hat and clones. Feel free to jump directly to the XenExpress photo gallery if desired.
What is XenExpress?
XenExpress is a custom Linux distro that actually uses CentOS as a base. The XenSource folks take CentOS, replace the stock kernel with one with Xen support, strip down the install package set to a fairly barebones level (approximately 190 rpms) and then add in a few custom Xen server components.
The current install
.iso image for XenExpress is approximately 671MB and includes everything needed to install the XenExpress on a dedicated server machine (no dual-booting here). There are a few other things on the CD including an installer for the Administrative Console as well as an
.iso image containing drivers for virtual Windows machines to boost their performance.
As you would imagine, you boot from the CD just like you would to install any other Linux distribution. The install process is very simple and only a few questions are asked. XenExpress assumes you are installing to a dedicated machine and offers no options for dual-booting nor customizing the hard disk partitioning. It will also check to see if your CPU has hardware virtualization support (aka VT) or not. VT is needed if you want to create virtual machines running operating systems that are not Xen aware (ie Microsoft Windows). It will prompt you for network settings and a root password and that is about it. Install time depends on the speed of your hardware but 10-15 minutes is a good guess.
Booting your new XenExpress Server
After the install is done it'll prompt you to reboot. The grub screen will appear and in a few seconds the Xen kernel will boot and you'll get the virtual console login screen containing a message similiar to:
XenServer Host 3.1.0-1332
System Booted: 2007-03-28 11:20
Your XenServer Host has now finished booting. To manage this server please use the XenServer Administrator Console application. You can install the XenServer Administrator Console for Windows and Linux from the XenServer install media.
You can connect to this system using one of the following network addresses:
Unlike most Linux distributions that have seven virtual consoles, XenExpress has two... which will save a little resources by not running five additional copies of
/sbin/mingetty. If you feel compelled to have seven virtual consoles, feel free to edit the
Installing the Administrator Console
On the XenExpress CD is a directory named
client_install which contains the following files:
As you can see, there's a Windows version and a Linux version. The Linux version is in three RPMs. If you are on an RPM based distro, it is pretty easy. Just
rpm -ivh *.rpm and it will install everything you need including the Java JRE package needed to run the Java based Administrator Console. Once installed run the following to start the Administrator Console.
The first time you run the Administrator Console, it will ask you to set a master password and then it will ask for the IP address and root password of the Xen server you want to connect to. After you put in the requested information you should get a window that looks something like...
Installing a Virtual Machine
XenExpress will allow you to install Red Hat Enterprise Linux, Debian Sarge, Windows Server 2003, and Windows XP SP2. As previously stated, in order to install Windows virtual machines, your CPU has to have hardware VT support.
Notice the row of buttons running horizontally across the center of the screen. To install a virtual machine click on Install XenVM button... at which point the lower half of the Administrator Console window will change like so:
Debian Sarge is the easiest to install because XenExpress comes with a pre-built OS template for it... which is another way of saying disk image... or that everything you need to do the install is already on the XenExpress server's hard drive somewhere. In the case of Red Hat Enterprise Linux, it just has a disk image of the network install media... so you can do an RHEL install but only over the network. In my tests I did several Debian installs since they were painless. Just select Debian, give it a name (debian1 for example), plop in a description (debian test 1 for example)... and then click the Install button in the lower right corner which will start the install.
There isn't much to see during the install... except in the top left of the window, under the host server line, will be a new line representing the installing virtual machine. After a few minutes or so the *** Installing will change to a green triagle and the word On. Once the machine is On you can click on that line and the buttons that run horizontally across the center of the screen will change to reflect your options for the selected virtual machine. Click on the Text Console button and the virtual console for that virtual machine will appear in the lower half of the window.
Although it might be hard to read in the resized image above, the first time Debian boots, it prompts you to set a root password and then prompts for an VNC password... and finally you end up at the login prompt.
Oddly enough their template for Debian Sarge automatically starts up VNC server so you can connect to it by clicking on the Graphical Console button which cause a graphical password prompt to appear in the lower half of the screen. After typing in the password and submitting, the graphical login screen will appear in the lower half of the window. Perhaps you'd like to see it the whole thing rather than having to scroll the display around in the lower half... just click on the Undock button and a new window will pop open like so:
Feel free to login as root if you didn't create a new account while in the Text Console. The Debian Sarge template they provide has XFCE as the default desktop which is a good idea because it is a lot lighter-weight than GNOME or KDE. Here's a screenshot with Firefox viewing my favorite website:
I don't recommend running the graphical console. In fact, I prefer just to ssh into the machine the old regular way and do everything from there.
My test machine had 2GB of RAM and plenty of disk space, and since XenExpress allows for upto four virtual machines to be installed, I installed Debian Sarge four times with each machine having 256MB of RAM, a 5GB root partition, and a 512MB swap partition. For virtual machine disk storage, XenExpress creates LVM partitions for all of the virtual machines and installing to disk image files is not an option. My test machine didn't have VT hardware support so I couldn't do any Windows installs. That's a feature I think. :)
Since all of the machines I installed were of the para-virtualized variety, they were Xen aware and ran at near physical machine speed. I went as far as doing an
apt-get update and an
apt-get upgrade on all of the machines at the same time and the CPU usage on the machine was barely affected. Each machine seemed full speed. On one of the machines I installed
apache and browsed to it and the default
apache page appeared as expected.
Below is a screenshot showing all four of my virtual machines running. Perhaps you can make out the CPU and RAM usage stats in the top half of the window and notice the
sshd config screen that apt-get presented during the upgrade of each machine.
While it would have been nice to run benchmarking software on a single virtual machine and then all of them at once to see how they turned out, I wasn't really prepared to do so. I was definitely impressed with the performance of the virtual machines and would say that they were as fast as my favorite, lighter-weight virtualization solution, OpenVZ.
XenExpress, XenServer, and XenEnterprise
XenExpress allows both Windows and Linux virtual machines, supports upto 4GB of RAM on the physical machine, allows for upto four virtual machines and is free.
XenServer allows only Windows virtual machines, supports upto 8GB of RAM and two CPUs, allows for upto eight virtual machines and goes for $99 per physical machine.
XenEnterpise allows both Windows and Linux virtual machines, supports upto 16GB of RAM and four CPUs, doesn't have a set limit on virtual machines and starts at $488 for a single physical machine. Check the XenSource website for specific prices as they offer several packages including 1, 5 and 10 server packs, 2 or 4 CPUs, and annual subscription or perpetual licensing. For a 10 server pack with 4 CPUs per machine (40 CPUs total) the price for a perpetual license at time of writing was $12,000.00. If that figure sounds high, a co-worker just priced out a VMware ESX setup for 6 CPUs that was over $30,000.00. While there is a great difference in the feature sets between XenEnterprise and VMware ESX, overall virtual machine performance is supposed to be close to a dead-even tie according to those who have been running benchmarks.
Physical to Virtual
The XenSource product line includes an option on the boot CD to migrate a physical machine to a virtual machine aka P2V. This option is only for the conversion of Linux installations although there are (fairly expensive) third-party alternatives available for doing Windows P2V. If the option is chosen, a wizard is started to walk you through the process. The end result can be either a new virtual machine on your Xen server or you can choose to save it on an NFS mount as an OS install template. I gave this a try on a fairly complex Linux machine (software RAID 1 on multiple mounted partitions) and it errored out almost immediately... but in no way harmed anything.
What about Live and Offline Migration?
Oddly enough, migration features (both live and offline) are missing in this release. XenSource has a public beta release of the next version (3.2) but it too is missing migration features. One wonders why this is? They do offer an export and an import feature that is a form of offline migration. I haven't done it but according to what I've read in the documentation, it basically allows you to turn a virtual machine that isn't running, into an OS install template which can easily be transfered to other Xen servers and installed.
The Xen technology DOES support migration features... yes, both live and offline... but they aren't implimented, so far as I can tell, with any XenSource product release.
What about disk storage options?
XenSource's product offerings currently only support LVM partition based storage... although that does include fiber channel based SANs. It appears more storage options are going to become availble in the next release including iSCSI.
Xen from other sources
Novell/SUSE introduced Xen support on SLES with their last release. Red Hat only recently released RHEL 5 which includes Xen support. I can't comment on Novell's integration because I haven't used SLES before. I did read a blog posting written by a Xen developer (
although the URL alludes me currently Found it: Xen and the Theory of RHEL-ativity by Simon Crosby, CTO of XenSource) that said that Red Hat's Xen integration was significantly better than SUSE's because of how disk access was handled... something about SUSE using a loopback device whereas Red Hat used a blksomething access instead. I don't currently understand the difference so I'll not even try to fake it.
I also must admit that I haven't done a RHEL 5 install yet... so I obviously haven't done any Xen virtual machine work under it. I have read Red Hat's Virtualization Guide (also in PDF) and I did work with the Virtual Machine Manager application a bit under CentOS 5 Beta. Virtual Machine Manager is no where near as complete as XenSource's Administrator Console but Red Hat's overall Xen integration has several options that XenSource's product line does not offer. These include:
- Live and offline migration
- Storage in both LVM and disk image formats
- Greater than 16GB of RAM support on the physical host
Support for more than 4 CPUsXenEnterprise supports upto 32 CPUs
While not exactly part of Red Hat's Xen integration, RHEL 5 has added clustering (several types) and cluster storage (GFS) as part of the package (they used to be optional addons) so in theory they could also be used for virtual machines although there certainly would be a lot of planning and work involved.
Red Hat does offer maintenance of the underlying OSes they support (RHEL 3, 4 and 5). For XenSource, it isn't quite so clear. The XenSource documentation doesn't seem to mention how to keep the Xen server OS (which is installed on the physical host) updated and the system seems to depend on the CentOS mirror repos for updates (per the
/etc/yum.repos.ed/*.repo config files)... although how Xen specific software on the machine is updated is still a mystery to me at this time.
While I certainly haven't run XenExpress in a production environment nor done any benchmarking with it... heck, I've only been testing it for a few days... I can't really make any first-hand knowledge statements about how well it works... but I have skimmed through several benchmark papers/articles from various sources (here's one). As a result, it is my belief Xen is ready for production work and truely offers most if not all of the benefits that virtualization has to offer.
Xen isn't as feature rich as VMware's expensive product line but it appears to be equal to it with regards to virtual machine performance and stability.
The OpenVZ folks also have indirectly provided some benchmarks that say that OpenVZ is basically equal to VMware and Xen in performance. I have been using OpenVZ in a production environment for a while now and it hasn't let me down... although in several areas it isn't as flexible as Xen and VMware... yet OpenVZ is lighter-weight and seems to scale better (more virtual machines per physical machine).
Where XenSource's products do stand out is in ease of installation and virtual machine creation and management. It is basically as close to a turnkey solution or application appliance as you can get and is even accessible to the less technical user. Downloading and installing XenExpress is a no-brainer for anyone who wants to give Xen a try.
This summer I hope to get at least two new servers (dual, quad-core CPUs with VT technology and 16GB of RAM) on which to do much more extensive testing and eventually production work. I won't know which solution I prefer (XenSource or Red Hat) until after I've done more testing... including the P2V features from XenSource. I plan to do follow up postings when the time comes.
I certainly welcome comments, feedback, as well as blog postings on the subject from other montanalinux.org members. What are you waiting for?