Like any organization the Xen.org project has a mission statement. For those of you who have never seen it, I thought I would share it today. I am planning to add it to the Xen.org website and thought a little extra promotion via the blog would be useful.

• Build the industry standard open source hypervisor
  • Core “engine” that is incorporated into multiple vendors’ products
• Maintain Xen’s industry-leading performance
  • Be first to exploit new hardware acceleration features
  • Help OS vendors paravirtualize their OSes
• Maintain Xen’s reputation for stability and quality
  • Security must now be paramount
• Support multiple CPU types; big and small systems
  • From server to client to mobile phone
• Foster innovation
• Drive interoperability

The Xen Community has completed a discussion around the selection of the proper tree for future development activities;

from Keir Fraser on June 4th  (full thread)…

With 3.4 out the door it is time to revisit the state of our Linux repositories. Currently we have a number of trees in various states of maintenance: - linux-2.6.18-xen.hg: the ‘original’ tree. Still maintained, used and tested but increasingly long in the tooth. - ext/linux-2.6.27-xen.hg: a snapshot of opensuse’s kernel port. This clones tree is not maintained or tested. - XCI/linux-2.6.27.git: a forward port of the Xen patches to 2.6.27. Maintained as part of XCI project. - Jeremy’s pv_ops patches against kernel.org: maintained, (somewhat) tested, but incomplete.

It is probably time to kill the 2.6.18 tree, or at least stop active development within it. It is increasingly a kludged collection of backports of more recent kernel patches, and is also missing a lot of drivers for more modern hardware.

Our proposal is to move XCI’s linux-2.6.27 tree out of the XCI subproject and make it the main user tree. Development and automated testing would occur on that tree and of course on Jeremy’s pv_ops patchset (which we want to completely move onto at some point in the future).

The community has decided to move all development to “Jeremy’s pv_ops tree” as the new platform. Jeremy’s tree contains the 2.6.29 Linux kernel with associated patches and will end the Xen’s use of 2.6.18 on all future releases. The tree switch is anticipated to be complete by the middle of next week and I will post the information on this blog.

As I posted late last week, Xen is still engaged with the kernel.org team to move pv_ops Dom0 into upstream.

For those of you looking for more assistance with a particular Domain0 or DomainU guest operating system, I have created a new document that lists the OS and links to support sites.

I have also added a new link on the Xen.org Support pages to the Pre-Built DomU sites that currnently create images for your leverage.

As the document and Wiki page are “living” entities, please feel free to edit the Wiki page with more information or send me new information for the document.

The second release candidiate for 3.4.1 is tagged as ‘3.4.1-rc2′ in http://xenbits.xensource.com/xen-3.4-testing.hg. I plan to release this (and also 3.3.2) next week. Please test!

Xen Community:

The Xen.org merchandise store is now available at http://www.cafepress.com/xen_org for your Xen shirts, hats, jackets, dog bowls, etc. The store is set at CafePress cost so any items you order are paid directly to CafePress to cover their printing and shipping costs.

If you would like something currently not listed or would like me to adjust the design or logo size, please let me know and I can make the change.

Enjoy shopping for Xen items and promoting the community!

There has been a great deal of developer discussion lately around the proposed patches to extend the Linux kernel’s existing Xen support to include control domain capabilities (loosely known as “the dom0 patches”). These discussions are generating a great deal of interest in Xen and Linux so I thought I would add some perspective for people in the community looking to get a better understanding of the situation.

As a reminder, paravirtualized Xen guest support has been available in Linux kernels since 2.6.24 for 32 bit Guests and 2.6.27 for 64 bit Guests. This joint effort between Xen.org and the kernel.org communities now enable all Linux distributions to support para-virtualization “out of the box” with no locally-maintained patches, additional coding or porting efforts required by customers or Linux distribution vendors.

The second focus of our Linux integration efforts centers on the control Guest (Domain0) present in every Xen system. As far as the hypervisor is concerned, it is a normal paravirtualized domain which happens to have some additional privileges (such as direct hardware access, the ability to start other domains, and so on).  Its main claim to fame is that the hypervisor starts it automatically at boot time, akin to the “init” process in a Linux system. There’s no fundamental reason why there should be a unique, absolutely privileged domain.  The Xen architecture allows for multiple privileged domains, and splitting the initial domain’s responsibilities into a number of more special-purpose domains can make things more reliable and secure.

Patches exist today that allow users to provide these essential services for a variety of operating systems including Linux, NetBSD, and Solaris. Specifically for Linux users, the patches are currently not available in upstream Linux and require efforts by all Linux distribution vendors to ensure proper Xen support in their distribution; as they have to do with the many other features they ship which do not enjoy upstream Linux support.

The Xen.org community recognizes the additional work required by Linux distribution vendors to enable Xen in their solutions and is working directly with the kernel.org community to include these patches directly in the Linux kernel, thereby removing this effort for Linux distribution vendors. The current discussions within the Xen.org mailing lists center on the best way to achieve this integration and the feedback from the kernel.org community including Linus Torvalds is helping us understand the best method for integration.

The complete technical discussions on the merge is available in xen-users and xen-devel but I thought I would highlight some comments from Jeremy Fitzhardinge, the lead community developer from Xen.org on the merge:

The issue is about Dom0 support, a subset of Xen, which primarily relates to allowing Xen domains to have direct access to hardware.  It is technically challenging because it covers quite different set of functionality in different parts of the kernel - pci, dma, interrupts, etc.

In some cases, the dom0 changes are fairly uncontroversial because they’re just another user of existing interfaces (dma_ops) or slightly controversial because they need tweaks to an existing interface (swiotlb).

However, where the existing kernel code doesn’t have a suitable abstraction layer, or even particularly clean internal interfaces (like the apic code), working out how to make the appropriate Xen changes poses a tricky tradeoff: do I attempt to restructure a large complex subsystem with lots of subtle interactions with the rest of the kernel - not to mention subtle interactions with many types of quirky hardware - just to add my changes?  Or do I make some relatively small, low risk (but low beauty) changes to get the job done?

I went for the latter; the cost-benefit tradeoff just didn’t seem to justify a massive refactor.  But others have pretty pointedly had the opposite view, so I’m now investigating what its actually going to involve.

From Keir on xen-devel:

The first release candidate for Xen 3.4.1 is tagged as 3.4.1-rc1 at http://xenbits.xensource.com/xen-3.4-testing.hg

Quite a few important fixes have been made since 3.4.0. Please test and let’s get this one out quickly!

With my current access to some of the Xen.org developers here in Cambridge, UK I am planning to do a series of short (10 minutes) video interviews to learn more about the people behind Xen. I am putting together a series of questions for these interviews and would like to get questions from you as well.

Please post your questions to this blog post or email me directly.

The GlusterFS team is currently trying to understand more about GlusterFS deployments with  Xen. We are looking at feedback from users of the same.

We are interested in knowing the following:

1) How many servers (Real and Virtual) do you use in the deployment?
2) What versions of operating systems/distributions do you use?
3) What is the Xen version that you use?
4) What disk based file systems do you use?
5) What is the amount of storage in the deployment?
6) What is the network interconnect that you use?  (TCP/IP, Infiniband)

Any information that you can provide us on the above would be much appreciated. Also, if you could share a topology of your deployment with us, it would be great!

Please write to Vijay <vijay at gluster.com> with any information that you can share with us.

We promise that any  information that you share with us would be kept strictly confidential.

Thanks in Advance,
GlusterFS Team

The Japanese software project Karesansui has released the first version of its open source software. The Web console allows management of virtual systems under Xen.

Karesansui provides creation of virtual guests and starting, suspending and shutting down the VM. It also takes snapshots and assigns CPU and RAM resources.

Rest of Article

From xen-devel:

I am pleased to announce the release of GUK (Project Guest VM Microkernel), which is an enhanced version of Mini-OS that underpins the Guest VM virtual machine. Guest VM is an implementation of the Java platform, written almost completely in Java, running directly on Xen. Although GUK was developed specifically for Guest VM, it can be used stand-alone in a similar way to Mini-OS.

The main differences between Mini-OS, when originally forked, and GUK are:
•    Support for multiple VCPUS (SMP)
•    Pre-emptive thread scheduling
•    Memory management tailored to a language run time virtual machine
•    Memory ballooning
•    Block device support
•    Suspend/Resume support

N.B. GUK has not been updated with any of the changes to Mini-OS since the initial fork.

The source code, which is licensed under GPL v2, can be found at https://kenai.com/projects/guestvm.
E.g.,

% hg clone https://kenai.com/hg/guestvm~guk guk

I’d like to acknowledge the significant contributions made by Grzegorz Milos and Harald Roeck during summer internships at Sun.

Mick Jordan

From Keir in Xen-Devel:

‘3.3.2-rc1′ is now tagged in http://xenbits.xensource.com/xen-3.3-testing.hg

Please test!

After much thought and contemplation (and lots of VISA paperwork), I have decided to take a 2 month break from the hot, humid summers of South Florida for the anticipated lovely weather in the UK. I will continue to update the xen-users and xen-devel messages on the Xen Weekly Wiki site and other Xen.org activities but will now be done 5 hours earlier!

If you are in Europe or near Cambridge and would like to get together to discuss Xen.org please feel free to contact me as I will have time over the next two months to meet with community members in Europe. For those of you in Paris, I will be spending a few days in July (23-26) and will be there at the finish line of the Tour de France.  I will also be at Xen Directions in Berlin on June 27, 2009 and look forward to meeting with everyone at our first technical/marketing event for Xen.org.

In August, I will be at the Xen.org booth at CloudWorld in San Francisco and am considering attending LinuxCon in Portland, OR in September. Anyone going to LinuxCon?

Hopefully, I will get a chance to meet with more community members over the summer and fall to help promote the Xen.org project!

The recent Xen.org community survey showed that 23.6% of you are aware of my Twitter feed on all things Xen. As I feel that Twitter is a valuable tool for the community, I thought I would present some more information on how I am using Twitter and what I have found from other Twitter users. NOTE - you don’t even have to join Twitter to take advantage of its information.

For Twitter newbies, twitter is a real-time system where anyone can post a message containing at most 140 characters. A basic overview of twitter is here or here.  People can select users to follow, thereby creating a stream of messages on topics you are interested in. Twitter also contains a hashtag tool that allows people to mark a particular post as being associated with a given topic - for example, I add the hashtag #xen to all my twitter messages associated with the Xen hypervisor or Xen.org community.

So, what does this mean to me and how can I use Twitter?

Twitter Search
A great way to leverage the continuous stream of information on Twitter is to use the real-time search engine at http://search.twitter.com/.  You can add any topic or hashtag you like and obtain the latest set of messages on that topic. For example, searching on #xen gave me the following:

• jarlostensen: @zack_perry responded to my question about #virtualization, I’d never heard of #xen before. Looks interesting about 7 hours ago from web · Reply · View Tweet · Show ConversationHide Conversation
• charltonb: VMware, Xen driving Rich Cloud Services http://bit.ly/KGmgo (expand) #cloudcomputing #vmware #xen about 14 hours ago from Seesmic Desktop · Reply · View Tweet
• xen_com_mgr: The Case for Private Clouds - Bernard Golden http://bit.ly/14RRm9 (expand) #xen about 17 hours ago from TweetDeck · Reply · View Tweet
• ddeighton: RT @xen_com_mgr: Xen DomU Images for Everyone! http://bit.ly/14bnQK (expand) #xen [Looks promising!] about 18 hours ago from Spaz · Reply · View Tweet
• cloudmeme: VMware, Xen Heat Up the Cloud –serverwatch.com Amy Newman– http://bit.ly/KGmgo (expand) #cloudcomputing #vmware #xen about 19 hours ago from web · Reply · View Tweet
• xen_com_mgr: Xen.org Xen Users Commonly Asked Questions Guide http://bit.ly/15ey37 (expand) #xen about 20 hours ago from TweetDeck · Reply · View Tweet

This snapshot gives you the latest messages sent by anyone on Twitter who added the #xen hashtag to their message. As you can see, not only will you find myself posting information but other people from around the globe.

Event Promotion
I have also been using Twitter to promote our upcoming Xen Directions event in Berlin at LinuxTAG. I have created a new hashtag for the event, #xendirect09 and have also added the #linuxtag hashtag to ensure that all people interested in the LinuxTAG event are also aware of the Xen Directions event. Here is an example of a post from me:

Retweeting
Another interesting way that Twitter expands your message reach is the use of Retweeting a message or RT for short. Many times, I will see a message from someone that I think the people following me will be interested in so I resend the message with a RT in front. For example, here is a message that I sent yesterday that smoeone else thought their followers would like to see so they RT:

This feature allows more people on the Twitter system to find me as well as ensure that more people are made aware of the content of my message. It is through this networking of people that I am able to expand my followers list and find new people to follow.

I have made the #xen twitter search available to the community directly on the blog.xen.org home page on the right side of the screen where you can also sign up for the RSS feed and get all #xen messages delivered directly to you via http://search.twitter.com/search?q=%23xen.

If you decide to join Twitter and become an active participant, please let me know your user-id so I can follow you and learn from your postings. To follow me, go to http://twitter.com/xen_com_mgr.

The fist release of the xen-users commonly asked questions guide is now available on the Xen.org website at http://xen.org/files/Support/XenUsersCommonlyAskedQuestions.pdf. Of course, this is a document in continuous change so please send any updates to this blog post or to me for document update.

Thanks to Todd Deshane for linking me to Stacklet (http://stacklet.com), a repository for Xen DomU Images.  The site currently has images for CentOS 5.3, Debian (Lenny), Fedora 10, Gentoo, Mandriva, Slackware, and Ubuntu 9.04. There is also a forum associated with the site if you have any questions.

If you try any of these images, let the community know how they work so everyone can start leveraging the great efforts from the Stacklet team.

Thanks to the 72 people who responded to the Xen.org community survey over the past 2 weeks; the results of the survey are attached in a pdf file (minus the email addresses of anyone who wanted to have further discussions).

Xen.org Survey May 2009.pdf

The feedback has been incredibly useful and here are the 3 main takeaways from the results:

• Documentation and Support
  • Better technical guides for current/future developers
  • Better user guides; emphasis for new users
• Community Tools
  • High usage of Mailing Lists, Wiki, & Blog
  • Limited exposure to various Web 2.0 tools
• Open Source Comfort
  • Familiar with various open source communities
  • Consistent to slightly better rating than other open source projects

I have several action items based on these takeaways and will be in touch with the survey respondents to ensure that I get their feedback and follow-up on all open issues they have. I plan to run a similar survey in 6 months to continuously monitor the project to ensure that everything you need from Xen.org is available.

As is standard, feel free to contact me at any time with additional questions, comments, etc. I am always avaiable via email at stephen.spector@xen.org.

From VMBlog.com, you can now read Xen Virtualization Essentials on the Amazon Kindle - http://vmblog.com/archive/2009/05/25/xen-virtualization-essentials-kindle-edition.aspx.

I have created yet another Xen.org group in the newly launched Linux.com site - http://www.linux.com/community/groups/viewgroup/792-Xenorg.  That brings to 6 the number of Social Network sites that have Xen.org groups; that I am aware of. If you know of others please let me know. (Note - SlideShare is not considered for this blog post)

This brings me to today’s blog post - what are we doing with all these social networks? I am currently monitoring and adding information to the following groups:

• Facebook: Xen Virtualisation
• LinkedIn: Xen Masters
• Ohloh: Xen Project
• Xing: Open Source Xen
• Plaxo: Xen Virtualization
• Linux.com: Xen.org Group

So, who is in these groups? Based on the survey data that I am now compiling, it appears that our community is not really involved in these sites (N=69).  I have been posting weekly community updates in these groups as well as having this blog directly feed the sites to ensure that the latest Xen.org info is available. Other than that, I am not really sure what people are looking for on these sites? If you were one of the people who answered below - please let me know what you use these tools for? I want to ensure that I supply the information these groups are looking for and I am currently unsure.

Hope everyone had a productive week, here is the Xen.org weekly update:

•    Xen 3.4 Release Announcement - http://blog.xen.org/index.php/2009/05/18/xen-34-launch-announcement/
•    Xen Directions Europe 2009 at LinuxTag - http://blog.xen.org/index.php/2009/05/20/are-you-coming-to-xen-directions-europe-2009/
•    Xen.org Community Survey - http://blog.xen.org/index.php/2009/05/12/xenorg-community-survey-make-your-voice-heard/
•    Summary of Xen-Devel & Xen-Users mails for the week with MarkMail Search Links for Easy Review

Full Weekly Update at http://wiki.xensource.com/xenwiki/May_22%2C_2009

Xen Directions Europe 2009 is the first “techno-marketing” event for Xen.org presenting the Xen hypervisor and the products/solutions based on it as well as give university researchers a chance to show how Xen is being leveraged. This event is being co-located at the LinuxTAG conference in Berlin, Germany on June 27, 2009. All LinuxTAG attendees can participate in the Xen Directions event and people interested in just Xen Directions can register for a single day LinuxTAG pass which I believe is about 10 euros.

The agenda for Xen Directions is a nice mix of enterprise companies, university researchers, smaller companies, and Xen.org “staff”. The global nature of our community is also on display with speakers from South America, Europe, and North America.

Speaker                   Company/University
Ian Pratt                   Xen.org
Jerg Theurer           HP
Dirk Hohndel          Intel
Ruben Montero     University of Madrid
Christian Rothe     Oracle
Frank Kohler          Citrix Systems
Boris Quiroz            Substance
Bjorn Brundert      Lufthansa Systems
Chris Schlaeger     AMD
Stephen Spector   Xen.org

The complete agenda with topic information is available in English and German.

Additionally, the next Xen Summit event for the development community is scheduled for November 19-20, 2009 in Shanghai, China at Intel’s R&D facility. More information on this event will be made available later this summer.

I finished reading a great blog post from Michael Dehann at  http://michaeldehaan.net/2009/05/17/oss-pitfalls/ about open source communities and some common misconceptions. Well worth the few minutes to read and think about when considering how Xen.org works and how you can contribute. Feedback on what Xen.org can do to better support the community is always welcome!

Thanks to Henning Sprang for the German translation:

http://www.xen.org/files/Marketing/XenOverview_Q22009_ger.pdf

Xen.org is pleased to announce the latest release of the Xen hypervisor, the open source industry standard for virtualization. Xen.org is a global community of independent and industry developers, university researchers, users, and virtualization gurus who regularly contribute to the shared design, development, support, and improvement of the Xen hypervisor platform.

The new release, Xen 3.4, furthers the vision of creating a powerful, efficient, and ubiquitous virtualization hypervisor. As part of the Xen community’s commitment to continuous improvement, the new hypervisor offers significant enhancements in the following areas:

•    Xen Client Initiative (XCI) Enhancements –Xen.org continues develop industry virtualization standards for desktop and client devices. Xen 3.4 contains the initial XCI code release providing a base client hypervisor for the community to extend and improve. This new version of the Xen hypervisor expands the hardware options for the leading open source virtualization platform.

•    Reliability – Availability - Serviceability (RAS) – In addition, Xen now delivers a collection of features designed to avoid and detect system failures, provide maximum uptime by isolating system faults, and provide system failure notices to administrators to properly service the hardware/software. The combination of these services provide for a robust Xen hypervisor with fault-tolerant and back-up capabilities built-in.

•    Power Management - Xen 3.4 improves the power saving features with a host of new algorithms to better manage the processor including schedulers and timers optimized for peak power savings.

Xen 3.4 is currently available via free download to developers by visiting the Xen.org website at: http://www.xen.org/download

Momentum in the Xen Community

The Xen community remains strong and active.  On average, Xen.org receives more than 750 new code submissions to the source tree each month from developers across the world working on an array of solutions within the hypervisor. Industry leading companies such as Intel, AMD, HP, IBM, Oracle, Red Hat, Sun, Fujitsu, and Novell are part of the vibrant Xen.org community and contribute to the development of the hypervisor code. The Yankee Group’s Third Annual Virtualization Survey reports a significant increase of commercial Xen-based solutions which represent 17 percent of total market share. This includes a Citrix XenServer share of 11 percent, plus an additional six percent from other open source suppliers.

On an ongoing basis, university research and other high profile Xen projects   are regularly incorporated or run on the hypervisor. For example, Project Snowflock from the University of Toronto leverages the Xen hypervisor to instantaneously launch thousands of virtual machines for fast, efficient, scalable parallel processing and Project HXen extends the Type1 virtual machine monitor (VMM) functionality in Xen to a Type 2 VMM for a simplified method of deploying Xen to desktops, laptops, USB sticks and other devices where the base OS is left in place. For high availability, Project Kemari  and Project Remus provide transparent, comprehensive, high availability to ordinary virtual machines running on the Xen virtual machine monitor by maintaining a completely up-to-date copy of a running VM on a backup server, which automatically activates if the primary server fails. These are examples of the various contributions across academia and within the development community to Xen.org.

To date, thousands of companies and universities have chosen the power of open source Xen, making the Xen hypervisor their choice to provide virtualization in their IT environment. Last month, more than 100 attendees from the open source community participated in the Xen Summit sponsored by Oracle; similar events are planned in Europe and Asia this year to support the growing global community.

In addition, the Xen.org community is committed to providing more resources to its members:

•    Xen.org Solutions Search is a new online tool enabling customers to quickly find consultants, hosting providers, developers, and solutions built on the Xen hypervisor platform. This search system profiles the growing ecosystem for the Xen hypervisor.

•    Xen.org is committing resources to expand the global footprint for support and promotion of the open source Xen hypervisor. Materials are being translated into Spanish, German, Portuguese, Chinese, and Japanese with the community Wiki now available in English, Spanish, French, Russian, German, Chinese, Korean Japanese, and Italian. Support groups for customers are also available in English, Portuguese, Japanese, and Italian.

•    Finally, Xen.org is excited to offer an event for virtualization customers and prospects in Europe called Xen Directions, being held in conjunction with LinuxTAG on June 27, 2009 in Berlin, Germany.  A variety of hands-on Xen demonstrations will be offered to the European technical audience for the first time. In 2010, Xen.org will be hosting the first event in South America as part of the FISL event in Brazil.

It’s Friday – time flys in Xen world.

Highlights:
• Xen.org Community Survey - https://citrix.websurveyor.net/wsb.dll/s/92g666
• Check out the new and improved Community Blog Site - http://blog.xen.org
• Xen Directions Europe 2009 at LinuxTag – June 27, 2009 in Berlin – See you there
• Patches, Patches, Patches,…

Full Weekly Update - http://wiki.xensource.com/xenwiki/May_15%2C_2009

From Sang-bum at Samsung -

I am happy to announce that a patch for a new emulator platform Goldfish (QEMU 0.82 based Android emulator) is released at wiki.xensource.com/xenwiki/XenARM.

Xen Community:

The project formerely known as KXEN is now called HXEN (Hosted Xen). This project allows a user to run the Xen hypervisor as a Type 2 VMM instead of the usual Type 1 use of Xen. Project information and downloads are available at http://xen.org/community/projects.html.

Xen Community:

I joined the TWINE network (www.twine.com) and have signed up for an existing Xen Hypervisor Twine which I have begun loading with content. The TWINE system works best when lots of people join and add information in real-time to the system. If you have any quetsions, please let me know; otherwise join Twine and select the Xen Hypervisor twine. You will know I am there as my username is ohiocitrix. See you in Web 3.0 land…

Xen Community:

I have created a short survey (only take about 5 to 10 minutes to take) to gather feedback on the current set of tools available for Xen.org community members as well as determine the next set of tools that the community is looking for. The survey does not require identification so all answers are anonymous. Once the survey is complete, I will publish the results so everyone can get an idea of what your fellow community members think. I anticipate running this survey for 2 weeks and publishing results by the end of this month.

Survey Link - https://citrix.websurveyor.net/wsb.dll/s/92g666

Xen Community:

I have reviewed the past three months of xen-users emails and tried to create a document that has some common questions that were repeated or seemed very useful based on my review. Please feel free to add more questions/answers to this document or let me know if something significant is missing. The document can be viewed in open office format -  xen-users-guide_v11