desktop and application delivery alternatives

14
Red Hat Enterprise Linux Virtualization and Intel Itanium: Linux Open Source Virtualization with Enterprise-Class Features A FOCUS White Paper August 2008 FOCUS Consulting

Upload: cameroon45

Post on 01-Dec-2014

815 views

Category:

Technology


1 download

DESCRIPTION

 

TRANSCRIPT

Page 1: Desktop and Application Delivery Alternatives

FC Draft 8152008

RHEL and Xen on Itanium Open Source Virtualization with Enterprise Class

Features

2008 FOCUS Consulting wwwfocusonsystemscom Page 1

Red Hat Enterprise Linux Virtualization and Intel Itanium Linux Open Source Virtualization with Enterprise-Class Features

A FOCUS White Paper August 2008

FOCUS Consulting

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 1

Table of Contents

Executive Summary 2

State of the Industry 2

Drivers for Virtualization 2

Drivers for Enterprise-Class Features on Servers 3

Drivers for Open Source 4

Background and History of Xen 5

Solution Overview ndash Red Hat Enterprise Linux on Itanium 5

Xen and Red Hat Enterprise Linux 5

Key Features 6

Guest Operating Systems 6

Management 7

Licensing 8

Intel Itanium Overview 8

Features 9

Intel VT-i 10

Key Benefits and Challenges 11

Conclusions and Recommendations 11

FOCUS Assessment 12

References 12

Other Related FOCUS Reports 12

About FOCUS 13

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 2

Executive Summary

Server virtualization is a powerful enabling technology that offers a compelling business case for consolidating multiple server workloads onto one physical server using server virtualization software Leveraging this capability increases the criticality of the server hardware mdash if a server fails many applicationsworkloads are affected rather than just one As a result consolidating business-critical workloads drives a need for enterprise mainframe-like features into the server

In a virtual environment obtaining enterprise mainframe-class features requires tightly interconnected hardware firmware virtualization software and guest operating system solutions Red Hat Enterprise Linux Virtualization on Intel Itanium with VT-i extensions is a good example of a powerful combined solution The use of multiple technologies from the open source community including Linux Xen libvirt and others brings the work of the best and brightest together in this solution Additionally Red Hat leverages standards such as libvirt and common information model (CIM) to provide management based on open standards allowing organizations to select the tools that best fit their environment

This software combination coupled with Intelrsquos highly reliable available and serviceable Itanium processor with Machine Check Architecture (MCA) and Intel VT-i virtualization assist provides a foundation for mainframe-class computing without mainframe pricing and maintenance expenses

This white paper discusses the state of the industry detailing market drivers for mainframe-class features on a server and the benefits of open source technology solutions explaining the background and history of open source virtualization beginning with the Xen hypervisor It details Red Hat Enterprise Linux Virtualization for Itanium describing salient features and provides the architectural overview of both the hardware and software components of the solution Key benefits and challenges are covered as well as conclusions and recommendations

State of the Industry

The compelling business case around server consolidation using server virtualization has gained the attention of C-level executives around the world

Drivers for Virtualization

It is estimated that most servers today are utilizing less than 10 of available capacity Using server virtualization to consolidate these underutilized resources drives up utilization on the remaining systems This increases the return on the server investment and reduces the total cost of server ownership through reductions in space power cooling management and hardware maintenance

IDC estimates that 90 of all very large corporations (companies with more than 10000 employees) and between 70 and 80 of medium-to-large corporations (with 1000 to 10000 employees) have implemented server virtualization in one form or another However current

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 3

estimates indicate that only 10 of all servers are currently virtualized This means that there is still substantial growth yet to occur for server virtualization throughout corporate enterprises In addition to the early dominance of VMware and other niche offerings the market is continuing to mature with the entry of many new server virtualization solutions including Microsoft Hyper-V and a list of companies incorporating the Xen hypervisor including Red Hat Novell Citrix Virtual Iron Sun and Oracle

Server consolidation forces the requirement to make the system highly available This section describes the market drivers for mainframe-class servers and for open source server virtualization including a brief history of the Xen Open Source Hypervisor

Drivers for Enterprise-Class Features on Servers

The need for enterprise-class features in servers has been growing over the past 10 years starting with the explosive growth of the Internet Additionally the shift to a more global market has accelerated the need for cost-effective commodity servers that have to be up and running 24x7 The cost of downtime is high and constantly increasing More recently the popularity of consolidating servers using virtualization has raised the issue of ldquoputting all your eggs in one basketrdquo As a result the need for high reliability availability and serviceability of servers running virtual workloads has moved to the forefront

With the single application per server model used by most IT organizations in the past when a server failed only one application was at risk Today with many application workloads consolidated on a single server using server virtualization (see Figure 1) a failure at the server level can lead to disastrous results

Figure 1 Server virtualization consolidates physical servers and their application workloads onto one server

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 4

Configuring a redundant IO infrastructure is fairly standard but processor and memory redundancy usually comes at the cost of another physical system with high availability (HA) failover techniques employed

The combination of systems configured with the Intel Itanium chipset and Red Hat Enterprise Linux (RHEL) 52 running the Xen hypervisor native across a cluster of physical servers is a good example of a hardwaresoftware pairing that provides this mainframe-class reliability and resilience without incurring the complete costs of moving to a mainframe

Drivers for Open Source

Businesses are finding significant value using technologies developed by the open source community on a number of fronts The open source projects typically have some of the best and brightest minds in the industry from many different vendors contributing to make the best possible technologies available

In addition well-defined application programming interfaces (APIs) are established and published so that anyone can create added-value software conforming to the APIs such as management software

Furthermore once the core of the technology has been developed it undergoes great scrutiny from many people producing high-quality base solutions Thereafter the market greatly benefits from the breadth of add-on work produced (again contributed by the best and brightest) to further the capabilities of the technology rather than multiple vendors spending this time and energy working on duplicate separate efforts The base Linux code used by Red Hat and the Xen hypervisor are products of the open source community

Definition of Terms

ECC ndash Error Correcting Code memory tests the accuracy of the data passing in and out of memory

Full Virtualization ndash Allows guest operating systems to run unmodified in a virtual environment Rather than using the traditional emulation approach open source solutions such as Xen and KVM support full virtualization using a platform incorporating processor chipsets with virtualization hardware assist such as Intel VT-i

Hypercall ndash The interface a paravirtualized guest operating system uses to access hardware resources bypassing the emulation layer

Hypervisor ndash A virtualization layer loaded onto bare metal which interacts with the hardware directly and creates and manages virtual machines in which operating systems either paravirtualized or not are run The hypervisor virtualizes or abstracts the hardware resources such as CPU memory and IO devices for the virtual machine and its guest operating system Hypervisors are also called virtual machine monitors

Live Migration ndash The ability to very quickly move a virtual machine and its entire contents (OS and applications) from the system where it is running to another physical system without requiring a restart of the OS or applications When the move is complete the OS and applications resume execution This is performed fast enough to avoid applicationservice timeouts

Paravirtualization ndash A virtualization approach in which the guest OS is modified to know it is running in a virtualized environment Paravirtualization is used to increase the performance of a guest operating system when it is running in a virtual machine

Virtual Machine (VM) ndash A virtual or logical system (hardware) running under the control of virtualization software such as a hypervisor The virtual machine nomenclature is also used to describe the virtualization of processes In this paper virtual machine is used in the context of duplicating hardware

Virtual Machine Monitor ndash Another term for

hypervisor

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 5

Background and History of Xen

Xen is a hypervisor or virtual machine monitor which is a thin layer of software that is loaded onto bare-metal hardware and that enables multiple virtual machines (or domains) to run concurrently on the same system hardware The Xen hypervisor grew out of the open source community after it began as a research project at Englandrsquos University of Cambridge The first release became generally available in 2003 after the presentation and publication of ldquoXen and the Art of Virtualizationrdquo at the ACM Symposium of Operating System Principles The Xen virtual machine monitor was originally developed for the x86 architecture to be very efficient and to host up to 100 virtual machine instances simultaneously on both Windows and Linux with little-to-no performance overhead (as compared with an unvirtualized environment)

The Intel x86 and Itanium processors were not originally designed to support a virtualization environment This made creating a hypervisor difficult forcing increased complexity and requiring large amounts of code thereby reducing performance One way of overcoming these challenges was to modify the guest operating system to understand that it is running in a virtual environment (paravirtualization)

In late 2005 Intel delivered hardware-assists for virtualization known as Intel VT-x (x86) and Intel VT-i (Itanium) extensions to its chipsets These extensions improved performance and removed the need for modifiedparavirtualized guest operating systems The Xen 30 release (December 2005) took full advantage of these enhancements to the chipset enabling support for more guest operating systems

In 2003 with the first release Xen was placed in open source and the Xen community was formed Since that time some of the industryrsquos top technologists have been working together to continue to create update and improve releases of Xen

Solution Overview ndash Red Hat Enterprise Linux on Itanium

The 51 release of Red Hat Enterprise Linux comes with the Xen hypervisor integrated and has support for guest operating systems built using the Intel Itanium processor with the VT-i extensions This release combines the value of several open source technologies including Linux Xen libvirt and others into a server-based solution that offers many mainframe-class features such as high availability superior error handling and dynamic management

Xen and Red Hat Enterprise Linux

Combining two strong open source technology solutions mdash Linux and Xen mdash RHEL 51 and later versions ship with the Xen hypervisor included As shown in Figure 2 the Xen hypervisor is loaded on the bare metal and runs on the Intel Itanium chipset both with and without the Intel VT-i extensions However only the paravirtualized guest operating systems can run on the Itanium hardware without support for Intel VT-i

The Xen architecture defines the virtual machine monitor separate from the device drivers To gain access to the plethora of device drivers already available and to maintain high-performance

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 6

access to IO the device drivers run in ldquoDomain0rdquo (dom0) or host domain which runs RHEL This domain provides the IO infrastructure for all virtual machines that request IO using hypercalls All IO requests from the guest operating systems go through the parent partitionrsquos device drivers

Key Features

Red Hat supports both paravirtualized and fully virtualized (unmodified) guest operating systems To deliver the highest levels of performance and scalability the kernel of the guest virtual machine can be optimized to run in a virtual environment In this model the virtual machine ldquocooperatesrdquo with the hypervisor eliminating the overhead of emulation and delivering near bare-metal performance

With paravirtualization the guest operating system becomes ldquovirtualization-awarerdquo allowing the guest to benefit from advanced features not possible in traditional emulation-based solutions One example is dynamic resource allocation or allowing the hardware resources of the guest to be changed on the fly mdash adding or removing memory CPUs disks and even network interfaces mdash without requiring a reboot

Live migration allows an administrator to dynamically relocate a virtual machine from one host system to another without service interruption mdash the virtual machine continues to operate and service user requests during the migration This can be used to perform hardware maintenance without application downtime by moving the virtual machine to a new host during a maintenance window Additionally live migration is used to dynamically balance resources within a cluster to gain optimal utilization of the resources

Red Hat also provides high-availability clustering and failover to enable automated restart of the virtual machines and associated environments in the event of hardware failure

Guest Operating Systems

As shown in Figure 2 the guest operating systems are run in virtual machines also known as unprivileged domains (domU) Red Hat offers a paravirtualized version of RHEL 5 that can run on Intel Itanium and with the VT-i extensions This provides for the lowest overhead and highest performing operating system support for applications that require this level of performance

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 7

Figure 2 High-level RHEL 51 for Itanium architecture

Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported

Management

There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release

The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments

Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 8

information model provider for the Distributed Management Task Force (DMTF) virtualization schema

Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform

Red Hat delivers a number of management tools built on libvirt including

virsh ndash A command line shell enabling customers to script and automate virtualization management

Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management

Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring

Licensing

Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL

Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product

Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices

Intel Itanium Overview

The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 9

Features

All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture

A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously

RAS Feature Typical

Mainframe Intel Itanium

Platforms Intel Xeon

MP Platforms Intel Xeon Platforms

Cache ECC coverage

Memory single-device error correct

Memory retry on double-bit error

Error recovery on data bus (ECC)

Internal logic soft error checking

Badpoisoned data containment

Cache reliability

Memory sparing

Memory mirroring

Hot plug IO (PCI-X PCI Express)

Memory hot swap

Table 1 Dual-core Itanium processor error coverage

With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)

Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 10

Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs

The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware

Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture

Intel VT-i

The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 11

paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment

In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments

Key Benefits and Challenges

For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system

Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility

As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization

Conclusions and Recommendations

The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 12

FOCUS Assessment

Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices

References

ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf

ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance

Other Related FOCUS Reports

FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch

FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC

FOCUS Market Landscape Report Desktop and Application Delivery Alternatives

FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager

FOCUS Solution Profile Citrix XenServer

FOCUS Solution Profile VMware Desktop Solutions

All trademarks are the properties of their respective owners

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 13

About FOCUS

Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years

Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers

FOCUS delivers research analysis and consulting focused on systems software and storage

in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of

Technology (ROI TCO) wwwfocusonsystemscom

Page 2: Desktop and Application Delivery Alternatives

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 1

Table of Contents

Executive Summary 2

State of the Industry 2

Drivers for Virtualization 2

Drivers for Enterprise-Class Features on Servers 3

Drivers for Open Source 4

Background and History of Xen 5

Solution Overview ndash Red Hat Enterprise Linux on Itanium 5

Xen and Red Hat Enterprise Linux 5

Key Features 6

Guest Operating Systems 6

Management 7

Licensing 8

Intel Itanium Overview 8

Features 9

Intel VT-i 10

Key Benefits and Challenges 11

Conclusions and Recommendations 11

FOCUS Assessment 12

References 12

Other Related FOCUS Reports 12

About FOCUS 13

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 2

Executive Summary

Server virtualization is a powerful enabling technology that offers a compelling business case for consolidating multiple server workloads onto one physical server using server virtualization software Leveraging this capability increases the criticality of the server hardware mdash if a server fails many applicationsworkloads are affected rather than just one As a result consolidating business-critical workloads drives a need for enterprise mainframe-like features into the server

In a virtual environment obtaining enterprise mainframe-class features requires tightly interconnected hardware firmware virtualization software and guest operating system solutions Red Hat Enterprise Linux Virtualization on Intel Itanium with VT-i extensions is a good example of a powerful combined solution The use of multiple technologies from the open source community including Linux Xen libvirt and others brings the work of the best and brightest together in this solution Additionally Red Hat leverages standards such as libvirt and common information model (CIM) to provide management based on open standards allowing organizations to select the tools that best fit their environment

This software combination coupled with Intelrsquos highly reliable available and serviceable Itanium processor with Machine Check Architecture (MCA) and Intel VT-i virtualization assist provides a foundation for mainframe-class computing without mainframe pricing and maintenance expenses

This white paper discusses the state of the industry detailing market drivers for mainframe-class features on a server and the benefits of open source technology solutions explaining the background and history of open source virtualization beginning with the Xen hypervisor It details Red Hat Enterprise Linux Virtualization for Itanium describing salient features and provides the architectural overview of both the hardware and software components of the solution Key benefits and challenges are covered as well as conclusions and recommendations

State of the Industry

The compelling business case around server consolidation using server virtualization has gained the attention of C-level executives around the world

Drivers for Virtualization

It is estimated that most servers today are utilizing less than 10 of available capacity Using server virtualization to consolidate these underutilized resources drives up utilization on the remaining systems This increases the return on the server investment and reduces the total cost of server ownership through reductions in space power cooling management and hardware maintenance

IDC estimates that 90 of all very large corporations (companies with more than 10000 employees) and between 70 and 80 of medium-to-large corporations (with 1000 to 10000 employees) have implemented server virtualization in one form or another However current

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 3

estimates indicate that only 10 of all servers are currently virtualized This means that there is still substantial growth yet to occur for server virtualization throughout corporate enterprises In addition to the early dominance of VMware and other niche offerings the market is continuing to mature with the entry of many new server virtualization solutions including Microsoft Hyper-V and a list of companies incorporating the Xen hypervisor including Red Hat Novell Citrix Virtual Iron Sun and Oracle

Server consolidation forces the requirement to make the system highly available This section describes the market drivers for mainframe-class servers and for open source server virtualization including a brief history of the Xen Open Source Hypervisor

Drivers for Enterprise-Class Features on Servers

The need for enterprise-class features in servers has been growing over the past 10 years starting with the explosive growth of the Internet Additionally the shift to a more global market has accelerated the need for cost-effective commodity servers that have to be up and running 24x7 The cost of downtime is high and constantly increasing More recently the popularity of consolidating servers using virtualization has raised the issue of ldquoputting all your eggs in one basketrdquo As a result the need for high reliability availability and serviceability of servers running virtual workloads has moved to the forefront

With the single application per server model used by most IT organizations in the past when a server failed only one application was at risk Today with many application workloads consolidated on a single server using server virtualization (see Figure 1) a failure at the server level can lead to disastrous results

Figure 1 Server virtualization consolidates physical servers and their application workloads onto one server

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 4

Configuring a redundant IO infrastructure is fairly standard but processor and memory redundancy usually comes at the cost of another physical system with high availability (HA) failover techniques employed

The combination of systems configured with the Intel Itanium chipset and Red Hat Enterprise Linux (RHEL) 52 running the Xen hypervisor native across a cluster of physical servers is a good example of a hardwaresoftware pairing that provides this mainframe-class reliability and resilience without incurring the complete costs of moving to a mainframe

Drivers for Open Source

Businesses are finding significant value using technologies developed by the open source community on a number of fronts The open source projects typically have some of the best and brightest minds in the industry from many different vendors contributing to make the best possible technologies available

In addition well-defined application programming interfaces (APIs) are established and published so that anyone can create added-value software conforming to the APIs such as management software

Furthermore once the core of the technology has been developed it undergoes great scrutiny from many people producing high-quality base solutions Thereafter the market greatly benefits from the breadth of add-on work produced (again contributed by the best and brightest) to further the capabilities of the technology rather than multiple vendors spending this time and energy working on duplicate separate efforts The base Linux code used by Red Hat and the Xen hypervisor are products of the open source community

Definition of Terms

ECC ndash Error Correcting Code memory tests the accuracy of the data passing in and out of memory

Full Virtualization ndash Allows guest operating systems to run unmodified in a virtual environment Rather than using the traditional emulation approach open source solutions such as Xen and KVM support full virtualization using a platform incorporating processor chipsets with virtualization hardware assist such as Intel VT-i

Hypercall ndash The interface a paravirtualized guest operating system uses to access hardware resources bypassing the emulation layer

Hypervisor ndash A virtualization layer loaded onto bare metal which interacts with the hardware directly and creates and manages virtual machines in which operating systems either paravirtualized or not are run The hypervisor virtualizes or abstracts the hardware resources such as CPU memory and IO devices for the virtual machine and its guest operating system Hypervisors are also called virtual machine monitors

Live Migration ndash The ability to very quickly move a virtual machine and its entire contents (OS and applications) from the system where it is running to another physical system without requiring a restart of the OS or applications When the move is complete the OS and applications resume execution This is performed fast enough to avoid applicationservice timeouts

Paravirtualization ndash A virtualization approach in which the guest OS is modified to know it is running in a virtualized environment Paravirtualization is used to increase the performance of a guest operating system when it is running in a virtual machine

Virtual Machine (VM) ndash A virtual or logical system (hardware) running under the control of virtualization software such as a hypervisor The virtual machine nomenclature is also used to describe the virtualization of processes In this paper virtual machine is used in the context of duplicating hardware

Virtual Machine Monitor ndash Another term for

hypervisor

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 5

Background and History of Xen

Xen is a hypervisor or virtual machine monitor which is a thin layer of software that is loaded onto bare-metal hardware and that enables multiple virtual machines (or domains) to run concurrently on the same system hardware The Xen hypervisor grew out of the open source community after it began as a research project at Englandrsquos University of Cambridge The first release became generally available in 2003 after the presentation and publication of ldquoXen and the Art of Virtualizationrdquo at the ACM Symposium of Operating System Principles The Xen virtual machine monitor was originally developed for the x86 architecture to be very efficient and to host up to 100 virtual machine instances simultaneously on both Windows and Linux with little-to-no performance overhead (as compared with an unvirtualized environment)

The Intel x86 and Itanium processors were not originally designed to support a virtualization environment This made creating a hypervisor difficult forcing increased complexity and requiring large amounts of code thereby reducing performance One way of overcoming these challenges was to modify the guest operating system to understand that it is running in a virtual environment (paravirtualization)

In late 2005 Intel delivered hardware-assists for virtualization known as Intel VT-x (x86) and Intel VT-i (Itanium) extensions to its chipsets These extensions improved performance and removed the need for modifiedparavirtualized guest operating systems The Xen 30 release (December 2005) took full advantage of these enhancements to the chipset enabling support for more guest operating systems

In 2003 with the first release Xen was placed in open source and the Xen community was formed Since that time some of the industryrsquos top technologists have been working together to continue to create update and improve releases of Xen

Solution Overview ndash Red Hat Enterprise Linux on Itanium

The 51 release of Red Hat Enterprise Linux comes with the Xen hypervisor integrated and has support for guest operating systems built using the Intel Itanium processor with the VT-i extensions This release combines the value of several open source technologies including Linux Xen libvirt and others into a server-based solution that offers many mainframe-class features such as high availability superior error handling and dynamic management

Xen and Red Hat Enterprise Linux

Combining two strong open source technology solutions mdash Linux and Xen mdash RHEL 51 and later versions ship with the Xen hypervisor included As shown in Figure 2 the Xen hypervisor is loaded on the bare metal and runs on the Intel Itanium chipset both with and without the Intel VT-i extensions However only the paravirtualized guest operating systems can run on the Itanium hardware without support for Intel VT-i

The Xen architecture defines the virtual machine monitor separate from the device drivers To gain access to the plethora of device drivers already available and to maintain high-performance

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 6

access to IO the device drivers run in ldquoDomain0rdquo (dom0) or host domain which runs RHEL This domain provides the IO infrastructure for all virtual machines that request IO using hypercalls All IO requests from the guest operating systems go through the parent partitionrsquos device drivers

Key Features

Red Hat supports both paravirtualized and fully virtualized (unmodified) guest operating systems To deliver the highest levels of performance and scalability the kernel of the guest virtual machine can be optimized to run in a virtual environment In this model the virtual machine ldquocooperatesrdquo with the hypervisor eliminating the overhead of emulation and delivering near bare-metal performance

With paravirtualization the guest operating system becomes ldquovirtualization-awarerdquo allowing the guest to benefit from advanced features not possible in traditional emulation-based solutions One example is dynamic resource allocation or allowing the hardware resources of the guest to be changed on the fly mdash adding or removing memory CPUs disks and even network interfaces mdash without requiring a reboot

Live migration allows an administrator to dynamically relocate a virtual machine from one host system to another without service interruption mdash the virtual machine continues to operate and service user requests during the migration This can be used to perform hardware maintenance without application downtime by moving the virtual machine to a new host during a maintenance window Additionally live migration is used to dynamically balance resources within a cluster to gain optimal utilization of the resources

Red Hat also provides high-availability clustering and failover to enable automated restart of the virtual machines and associated environments in the event of hardware failure

Guest Operating Systems

As shown in Figure 2 the guest operating systems are run in virtual machines also known as unprivileged domains (domU) Red Hat offers a paravirtualized version of RHEL 5 that can run on Intel Itanium and with the VT-i extensions This provides for the lowest overhead and highest performing operating system support for applications that require this level of performance

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 7

Figure 2 High-level RHEL 51 for Itanium architecture

Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported

Management

There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release

The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments

Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 8

information model provider for the Distributed Management Task Force (DMTF) virtualization schema

Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform

Red Hat delivers a number of management tools built on libvirt including

virsh ndash A command line shell enabling customers to script and automate virtualization management

Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management

Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring

Licensing

Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL

Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product

Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices

Intel Itanium Overview

The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 9

Features

All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture

A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously

RAS Feature Typical

Mainframe Intel Itanium

Platforms Intel Xeon

MP Platforms Intel Xeon Platforms

Cache ECC coverage

Memory single-device error correct

Memory retry on double-bit error

Error recovery on data bus (ECC)

Internal logic soft error checking

Badpoisoned data containment

Cache reliability

Memory sparing

Memory mirroring

Hot plug IO (PCI-X PCI Express)

Memory hot swap

Table 1 Dual-core Itanium processor error coverage

With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)

Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 10

Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs

The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware

Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture

Intel VT-i

The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 11

paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment

In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments

Key Benefits and Challenges

For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system

Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility

As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization

Conclusions and Recommendations

The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 12

FOCUS Assessment

Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices

References

ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf

ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance

Other Related FOCUS Reports

FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch

FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC

FOCUS Market Landscape Report Desktop and Application Delivery Alternatives

FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager

FOCUS Solution Profile Citrix XenServer

FOCUS Solution Profile VMware Desktop Solutions

All trademarks are the properties of their respective owners

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 13

About FOCUS

Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years

Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers

FOCUS delivers research analysis and consulting focused on systems software and storage

in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of

Technology (ROI TCO) wwwfocusonsystemscom

Page 3: Desktop and Application Delivery Alternatives

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 2

Executive Summary

Server virtualization is a powerful enabling technology that offers a compelling business case for consolidating multiple server workloads onto one physical server using server virtualization software Leveraging this capability increases the criticality of the server hardware mdash if a server fails many applicationsworkloads are affected rather than just one As a result consolidating business-critical workloads drives a need for enterprise mainframe-like features into the server

In a virtual environment obtaining enterprise mainframe-class features requires tightly interconnected hardware firmware virtualization software and guest operating system solutions Red Hat Enterprise Linux Virtualization on Intel Itanium with VT-i extensions is a good example of a powerful combined solution The use of multiple technologies from the open source community including Linux Xen libvirt and others brings the work of the best and brightest together in this solution Additionally Red Hat leverages standards such as libvirt and common information model (CIM) to provide management based on open standards allowing organizations to select the tools that best fit their environment

This software combination coupled with Intelrsquos highly reliable available and serviceable Itanium processor with Machine Check Architecture (MCA) and Intel VT-i virtualization assist provides a foundation for mainframe-class computing without mainframe pricing and maintenance expenses

This white paper discusses the state of the industry detailing market drivers for mainframe-class features on a server and the benefits of open source technology solutions explaining the background and history of open source virtualization beginning with the Xen hypervisor It details Red Hat Enterprise Linux Virtualization for Itanium describing salient features and provides the architectural overview of both the hardware and software components of the solution Key benefits and challenges are covered as well as conclusions and recommendations

State of the Industry

The compelling business case around server consolidation using server virtualization has gained the attention of C-level executives around the world

Drivers for Virtualization

It is estimated that most servers today are utilizing less than 10 of available capacity Using server virtualization to consolidate these underutilized resources drives up utilization on the remaining systems This increases the return on the server investment and reduces the total cost of server ownership through reductions in space power cooling management and hardware maintenance

IDC estimates that 90 of all very large corporations (companies with more than 10000 employees) and between 70 and 80 of medium-to-large corporations (with 1000 to 10000 employees) have implemented server virtualization in one form or another However current

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 3

estimates indicate that only 10 of all servers are currently virtualized This means that there is still substantial growth yet to occur for server virtualization throughout corporate enterprises In addition to the early dominance of VMware and other niche offerings the market is continuing to mature with the entry of many new server virtualization solutions including Microsoft Hyper-V and a list of companies incorporating the Xen hypervisor including Red Hat Novell Citrix Virtual Iron Sun and Oracle

Server consolidation forces the requirement to make the system highly available This section describes the market drivers for mainframe-class servers and for open source server virtualization including a brief history of the Xen Open Source Hypervisor

Drivers for Enterprise-Class Features on Servers

The need for enterprise-class features in servers has been growing over the past 10 years starting with the explosive growth of the Internet Additionally the shift to a more global market has accelerated the need for cost-effective commodity servers that have to be up and running 24x7 The cost of downtime is high and constantly increasing More recently the popularity of consolidating servers using virtualization has raised the issue of ldquoputting all your eggs in one basketrdquo As a result the need for high reliability availability and serviceability of servers running virtual workloads has moved to the forefront

With the single application per server model used by most IT organizations in the past when a server failed only one application was at risk Today with many application workloads consolidated on a single server using server virtualization (see Figure 1) a failure at the server level can lead to disastrous results

Figure 1 Server virtualization consolidates physical servers and their application workloads onto one server

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 4

Configuring a redundant IO infrastructure is fairly standard but processor and memory redundancy usually comes at the cost of another physical system with high availability (HA) failover techniques employed

The combination of systems configured with the Intel Itanium chipset and Red Hat Enterprise Linux (RHEL) 52 running the Xen hypervisor native across a cluster of physical servers is a good example of a hardwaresoftware pairing that provides this mainframe-class reliability and resilience without incurring the complete costs of moving to a mainframe

Drivers for Open Source

Businesses are finding significant value using technologies developed by the open source community on a number of fronts The open source projects typically have some of the best and brightest minds in the industry from many different vendors contributing to make the best possible technologies available

In addition well-defined application programming interfaces (APIs) are established and published so that anyone can create added-value software conforming to the APIs such as management software

Furthermore once the core of the technology has been developed it undergoes great scrutiny from many people producing high-quality base solutions Thereafter the market greatly benefits from the breadth of add-on work produced (again contributed by the best and brightest) to further the capabilities of the technology rather than multiple vendors spending this time and energy working on duplicate separate efforts The base Linux code used by Red Hat and the Xen hypervisor are products of the open source community

Definition of Terms

ECC ndash Error Correcting Code memory tests the accuracy of the data passing in and out of memory

Full Virtualization ndash Allows guest operating systems to run unmodified in a virtual environment Rather than using the traditional emulation approach open source solutions such as Xen and KVM support full virtualization using a platform incorporating processor chipsets with virtualization hardware assist such as Intel VT-i

Hypercall ndash The interface a paravirtualized guest operating system uses to access hardware resources bypassing the emulation layer

Hypervisor ndash A virtualization layer loaded onto bare metal which interacts with the hardware directly and creates and manages virtual machines in which operating systems either paravirtualized or not are run The hypervisor virtualizes or abstracts the hardware resources such as CPU memory and IO devices for the virtual machine and its guest operating system Hypervisors are also called virtual machine monitors

Live Migration ndash The ability to very quickly move a virtual machine and its entire contents (OS and applications) from the system where it is running to another physical system without requiring a restart of the OS or applications When the move is complete the OS and applications resume execution This is performed fast enough to avoid applicationservice timeouts

Paravirtualization ndash A virtualization approach in which the guest OS is modified to know it is running in a virtualized environment Paravirtualization is used to increase the performance of a guest operating system when it is running in a virtual machine

Virtual Machine (VM) ndash A virtual or logical system (hardware) running under the control of virtualization software such as a hypervisor The virtual machine nomenclature is also used to describe the virtualization of processes In this paper virtual machine is used in the context of duplicating hardware

Virtual Machine Monitor ndash Another term for

hypervisor

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 5

Background and History of Xen

Xen is a hypervisor or virtual machine monitor which is a thin layer of software that is loaded onto bare-metal hardware and that enables multiple virtual machines (or domains) to run concurrently on the same system hardware The Xen hypervisor grew out of the open source community after it began as a research project at Englandrsquos University of Cambridge The first release became generally available in 2003 after the presentation and publication of ldquoXen and the Art of Virtualizationrdquo at the ACM Symposium of Operating System Principles The Xen virtual machine monitor was originally developed for the x86 architecture to be very efficient and to host up to 100 virtual machine instances simultaneously on both Windows and Linux with little-to-no performance overhead (as compared with an unvirtualized environment)

The Intel x86 and Itanium processors were not originally designed to support a virtualization environment This made creating a hypervisor difficult forcing increased complexity and requiring large amounts of code thereby reducing performance One way of overcoming these challenges was to modify the guest operating system to understand that it is running in a virtual environment (paravirtualization)

In late 2005 Intel delivered hardware-assists for virtualization known as Intel VT-x (x86) and Intel VT-i (Itanium) extensions to its chipsets These extensions improved performance and removed the need for modifiedparavirtualized guest operating systems The Xen 30 release (December 2005) took full advantage of these enhancements to the chipset enabling support for more guest operating systems

In 2003 with the first release Xen was placed in open source and the Xen community was formed Since that time some of the industryrsquos top technologists have been working together to continue to create update and improve releases of Xen

Solution Overview ndash Red Hat Enterprise Linux on Itanium

The 51 release of Red Hat Enterprise Linux comes with the Xen hypervisor integrated and has support for guest operating systems built using the Intel Itanium processor with the VT-i extensions This release combines the value of several open source technologies including Linux Xen libvirt and others into a server-based solution that offers many mainframe-class features such as high availability superior error handling and dynamic management

Xen and Red Hat Enterprise Linux

Combining two strong open source technology solutions mdash Linux and Xen mdash RHEL 51 and later versions ship with the Xen hypervisor included As shown in Figure 2 the Xen hypervisor is loaded on the bare metal and runs on the Intel Itanium chipset both with and without the Intel VT-i extensions However only the paravirtualized guest operating systems can run on the Itanium hardware without support for Intel VT-i

The Xen architecture defines the virtual machine monitor separate from the device drivers To gain access to the plethora of device drivers already available and to maintain high-performance

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 6

access to IO the device drivers run in ldquoDomain0rdquo (dom0) or host domain which runs RHEL This domain provides the IO infrastructure for all virtual machines that request IO using hypercalls All IO requests from the guest operating systems go through the parent partitionrsquos device drivers

Key Features

Red Hat supports both paravirtualized and fully virtualized (unmodified) guest operating systems To deliver the highest levels of performance and scalability the kernel of the guest virtual machine can be optimized to run in a virtual environment In this model the virtual machine ldquocooperatesrdquo with the hypervisor eliminating the overhead of emulation and delivering near bare-metal performance

With paravirtualization the guest operating system becomes ldquovirtualization-awarerdquo allowing the guest to benefit from advanced features not possible in traditional emulation-based solutions One example is dynamic resource allocation or allowing the hardware resources of the guest to be changed on the fly mdash adding or removing memory CPUs disks and even network interfaces mdash without requiring a reboot

Live migration allows an administrator to dynamically relocate a virtual machine from one host system to another without service interruption mdash the virtual machine continues to operate and service user requests during the migration This can be used to perform hardware maintenance without application downtime by moving the virtual machine to a new host during a maintenance window Additionally live migration is used to dynamically balance resources within a cluster to gain optimal utilization of the resources

Red Hat also provides high-availability clustering and failover to enable automated restart of the virtual machines and associated environments in the event of hardware failure

Guest Operating Systems

As shown in Figure 2 the guest operating systems are run in virtual machines also known as unprivileged domains (domU) Red Hat offers a paravirtualized version of RHEL 5 that can run on Intel Itanium and with the VT-i extensions This provides for the lowest overhead and highest performing operating system support for applications that require this level of performance

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 7

Figure 2 High-level RHEL 51 for Itanium architecture

Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported

Management

There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release

The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments

Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 8

information model provider for the Distributed Management Task Force (DMTF) virtualization schema

Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform

Red Hat delivers a number of management tools built on libvirt including

virsh ndash A command line shell enabling customers to script and automate virtualization management

Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management

Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring

Licensing

Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL

Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product

Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices

Intel Itanium Overview

The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 9

Features

All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture

A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously

RAS Feature Typical

Mainframe Intel Itanium

Platforms Intel Xeon

MP Platforms Intel Xeon Platforms

Cache ECC coverage

Memory single-device error correct

Memory retry on double-bit error

Error recovery on data bus (ECC)

Internal logic soft error checking

Badpoisoned data containment

Cache reliability

Memory sparing

Memory mirroring

Hot plug IO (PCI-X PCI Express)

Memory hot swap

Table 1 Dual-core Itanium processor error coverage

With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)

Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 10

Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs

The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware

Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture

Intel VT-i

The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 11

paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment

In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments

Key Benefits and Challenges

For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system

Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility

As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization

Conclusions and Recommendations

The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 12

FOCUS Assessment

Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices

References

ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf

ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance

Other Related FOCUS Reports

FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch

FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC

FOCUS Market Landscape Report Desktop and Application Delivery Alternatives

FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager

FOCUS Solution Profile Citrix XenServer

FOCUS Solution Profile VMware Desktop Solutions

All trademarks are the properties of their respective owners

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 13

About FOCUS

Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years

Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers

FOCUS delivers research analysis and consulting focused on systems software and storage

in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of

Technology (ROI TCO) wwwfocusonsystemscom

Page 4: Desktop and Application Delivery Alternatives

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 3

estimates indicate that only 10 of all servers are currently virtualized This means that there is still substantial growth yet to occur for server virtualization throughout corporate enterprises In addition to the early dominance of VMware and other niche offerings the market is continuing to mature with the entry of many new server virtualization solutions including Microsoft Hyper-V and a list of companies incorporating the Xen hypervisor including Red Hat Novell Citrix Virtual Iron Sun and Oracle

Server consolidation forces the requirement to make the system highly available This section describes the market drivers for mainframe-class servers and for open source server virtualization including a brief history of the Xen Open Source Hypervisor

Drivers for Enterprise-Class Features on Servers

The need for enterprise-class features in servers has been growing over the past 10 years starting with the explosive growth of the Internet Additionally the shift to a more global market has accelerated the need for cost-effective commodity servers that have to be up and running 24x7 The cost of downtime is high and constantly increasing More recently the popularity of consolidating servers using virtualization has raised the issue of ldquoputting all your eggs in one basketrdquo As a result the need for high reliability availability and serviceability of servers running virtual workloads has moved to the forefront

With the single application per server model used by most IT organizations in the past when a server failed only one application was at risk Today with many application workloads consolidated on a single server using server virtualization (see Figure 1) a failure at the server level can lead to disastrous results

Figure 1 Server virtualization consolidates physical servers and their application workloads onto one server

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 4

Configuring a redundant IO infrastructure is fairly standard but processor and memory redundancy usually comes at the cost of another physical system with high availability (HA) failover techniques employed

The combination of systems configured with the Intel Itanium chipset and Red Hat Enterprise Linux (RHEL) 52 running the Xen hypervisor native across a cluster of physical servers is a good example of a hardwaresoftware pairing that provides this mainframe-class reliability and resilience without incurring the complete costs of moving to a mainframe

Drivers for Open Source

Businesses are finding significant value using technologies developed by the open source community on a number of fronts The open source projects typically have some of the best and brightest minds in the industry from many different vendors contributing to make the best possible technologies available

In addition well-defined application programming interfaces (APIs) are established and published so that anyone can create added-value software conforming to the APIs such as management software

Furthermore once the core of the technology has been developed it undergoes great scrutiny from many people producing high-quality base solutions Thereafter the market greatly benefits from the breadth of add-on work produced (again contributed by the best and brightest) to further the capabilities of the technology rather than multiple vendors spending this time and energy working on duplicate separate efforts The base Linux code used by Red Hat and the Xen hypervisor are products of the open source community

Definition of Terms

ECC ndash Error Correcting Code memory tests the accuracy of the data passing in and out of memory

Full Virtualization ndash Allows guest operating systems to run unmodified in a virtual environment Rather than using the traditional emulation approach open source solutions such as Xen and KVM support full virtualization using a platform incorporating processor chipsets with virtualization hardware assist such as Intel VT-i

Hypercall ndash The interface a paravirtualized guest operating system uses to access hardware resources bypassing the emulation layer

Hypervisor ndash A virtualization layer loaded onto bare metal which interacts with the hardware directly and creates and manages virtual machines in which operating systems either paravirtualized or not are run The hypervisor virtualizes or abstracts the hardware resources such as CPU memory and IO devices for the virtual machine and its guest operating system Hypervisors are also called virtual machine monitors

Live Migration ndash The ability to very quickly move a virtual machine and its entire contents (OS and applications) from the system where it is running to another physical system without requiring a restart of the OS or applications When the move is complete the OS and applications resume execution This is performed fast enough to avoid applicationservice timeouts

Paravirtualization ndash A virtualization approach in which the guest OS is modified to know it is running in a virtualized environment Paravirtualization is used to increase the performance of a guest operating system when it is running in a virtual machine

Virtual Machine (VM) ndash A virtual or logical system (hardware) running under the control of virtualization software such as a hypervisor The virtual machine nomenclature is also used to describe the virtualization of processes In this paper virtual machine is used in the context of duplicating hardware

Virtual Machine Monitor ndash Another term for

hypervisor

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 5

Background and History of Xen

Xen is a hypervisor or virtual machine monitor which is a thin layer of software that is loaded onto bare-metal hardware and that enables multiple virtual machines (or domains) to run concurrently on the same system hardware The Xen hypervisor grew out of the open source community after it began as a research project at Englandrsquos University of Cambridge The first release became generally available in 2003 after the presentation and publication of ldquoXen and the Art of Virtualizationrdquo at the ACM Symposium of Operating System Principles The Xen virtual machine monitor was originally developed for the x86 architecture to be very efficient and to host up to 100 virtual machine instances simultaneously on both Windows and Linux with little-to-no performance overhead (as compared with an unvirtualized environment)

The Intel x86 and Itanium processors were not originally designed to support a virtualization environment This made creating a hypervisor difficult forcing increased complexity and requiring large amounts of code thereby reducing performance One way of overcoming these challenges was to modify the guest operating system to understand that it is running in a virtual environment (paravirtualization)

In late 2005 Intel delivered hardware-assists for virtualization known as Intel VT-x (x86) and Intel VT-i (Itanium) extensions to its chipsets These extensions improved performance and removed the need for modifiedparavirtualized guest operating systems The Xen 30 release (December 2005) took full advantage of these enhancements to the chipset enabling support for more guest operating systems

In 2003 with the first release Xen was placed in open source and the Xen community was formed Since that time some of the industryrsquos top technologists have been working together to continue to create update and improve releases of Xen

Solution Overview ndash Red Hat Enterprise Linux on Itanium

The 51 release of Red Hat Enterprise Linux comes with the Xen hypervisor integrated and has support for guest operating systems built using the Intel Itanium processor with the VT-i extensions This release combines the value of several open source technologies including Linux Xen libvirt and others into a server-based solution that offers many mainframe-class features such as high availability superior error handling and dynamic management

Xen and Red Hat Enterprise Linux

Combining two strong open source technology solutions mdash Linux and Xen mdash RHEL 51 and later versions ship with the Xen hypervisor included As shown in Figure 2 the Xen hypervisor is loaded on the bare metal and runs on the Intel Itanium chipset both with and without the Intel VT-i extensions However only the paravirtualized guest operating systems can run on the Itanium hardware without support for Intel VT-i

The Xen architecture defines the virtual machine monitor separate from the device drivers To gain access to the plethora of device drivers already available and to maintain high-performance

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 6

access to IO the device drivers run in ldquoDomain0rdquo (dom0) or host domain which runs RHEL This domain provides the IO infrastructure for all virtual machines that request IO using hypercalls All IO requests from the guest operating systems go through the parent partitionrsquos device drivers

Key Features

Red Hat supports both paravirtualized and fully virtualized (unmodified) guest operating systems To deliver the highest levels of performance and scalability the kernel of the guest virtual machine can be optimized to run in a virtual environment In this model the virtual machine ldquocooperatesrdquo with the hypervisor eliminating the overhead of emulation and delivering near bare-metal performance

With paravirtualization the guest operating system becomes ldquovirtualization-awarerdquo allowing the guest to benefit from advanced features not possible in traditional emulation-based solutions One example is dynamic resource allocation or allowing the hardware resources of the guest to be changed on the fly mdash adding or removing memory CPUs disks and even network interfaces mdash without requiring a reboot

Live migration allows an administrator to dynamically relocate a virtual machine from one host system to another without service interruption mdash the virtual machine continues to operate and service user requests during the migration This can be used to perform hardware maintenance without application downtime by moving the virtual machine to a new host during a maintenance window Additionally live migration is used to dynamically balance resources within a cluster to gain optimal utilization of the resources

Red Hat also provides high-availability clustering and failover to enable automated restart of the virtual machines and associated environments in the event of hardware failure

Guest Operating Systems

As shown in Figure 2 the guest operating systems are run in virtual machines also known as unprivileged domains (domU) Red Hat offers a paravirtualized version of RHEL 5 that can run on Intel Itanium and with the VT-i extensions This provides for the lowest overhead and highest performing operating system support for applications that require this level of performance

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 7

Figure 2 High-level RHEL 51 for Itanium architecture

Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported

Management

There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release

The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments

Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 8

information model provider for the Distributed Management Task Force (DMTF) virtualization schema

Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform

Red Hat delivers a number of management tools built on libvirt including

virsh ndash A command line shell enabling customers to script and automate virtualization management

Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management

Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring

Licensing

Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL

Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product

Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices

Intel Itanium Overview

The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 9

Features

All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture

A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously

RAS Feature Typical

Mainframe Intel Itanium

Platforms Intel Xeon

MP Platforms Intel Xeon Platforms

Cache ECC coverage

Memory single-device error correct

Memory retry on double-bit error

Error recovery on data bus (ECC)

Internal logic soft error checking

Badpoisoned data containment

Cache reliability

Memory sparing

Memory mirroring

Hot plug IO (PCI-X PCI Express)

Memory hot swap

Table 1 Dual-core Itanium processor error coverage

With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)

Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 10

Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs

The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware

Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture

Intel VT-i

The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 11

paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment

In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments

Key Benefits and Challenges

For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system

Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility

As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization

Conclusions and Recommendations

The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 12

FOCUS Assessment

Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices

References

ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf

ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance

Other Related FOCUS Reports

FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch

FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC

FOCUS Market Landscape Report Desktop and Application Delivery Alternatives

FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager

FOCUS Solution Profile Citrix XenServer

FOCUS Solution Profile VMware Desktop Solutions

All trademarks are the properties of their respective owners

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 13

About FOCUS

Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years

Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers

FOCUS delivers research analysis and consulting focused on systems software and storage

in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of

Technology (ROI TCO) wwwfocusonsystemscom

Page 5: Desktop and Application Delivery Alternatives

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 4

Configuring a redundant IO infrastructure is fairly standard but processor and memory redundancy usually comes at the cost of another physical system with high availability (HA) failover techniques employed

The combination of systems configured with the Intel Itanium chipset and Red Hat Enterprise Linux (RHEL) 52 running the Xen hypervisor native across a cluster of physical servers is a good example of a hardwaresoftware pairing that provides this mainframe-class reliability and resilience without incurring the complete costs of moving to a mainframe

Drivers for Open Source

Businesses are finding significant value using technologies developed by the open source community on a number of fronts The open source projects typically have some of the best and brightest minds in the industry from many different vendors contributing to make the best possible technologies available

In addition well-defined application programming interfaces (APIs) are established and published so that anyone can create added-value software conforming to the APIs such as management software

Furthermore once the core of the technology has been developed it undergoes great scrutiny from many people producing high-quality base solutions Thereafter the market greatly benefits from the breadth of add-on work produced (again contributed by the best and brightest) to further the capabilities of the technology rather than multiple vendors spending this time and energy working on duplicate separate efforts The base Linux code used by Red Hat and the Xen hypervisor are products of the open source community

Definition of Terms

ECC ndash Error Correcting Code memory tests the accuracy of the data passing in and out of memory

Full Virtualization ndash Allows guest operating systems to run unmodified in a virtual environment Rather than using the traditional emulation approach open source solutions such as Xen and KVM support full virtualization using a platform incorporating processor chipsets with virtualization hardware assist such as Intel VT-i

Hypercall ndash The interface a paravirtualized guest operating system uses to access hardware resources bypassing the emulation layer

Hypervisor ndash A virtualization layer loaded onto bare metal which interacts with the hardware directly and creates and manages virtual machines in which operating systems either paravirtualized or not are run The hypervisor virtualizes or abstracts the hardware resources such as CPU memory and IO devices for the virtual machine and its guest operating system Hypervisors are also called virtual machine monitors

Live Migration ndash The ability to very quickly move a virtual machine and its entire contents (OS and applications) from the system where it is running to another physical system without requiring a restart of the OS or applications When the move is complete the OS and applications resume execution This is performed fast enough to avoid applicationservice timeouts

Paravirtualization ndash A virtualization approach in which the guest OS is modified to know it is running in a virtualized environment Paravirtualization is used to increase the performance of a guest operating system when it is running in a virtual machine

Virtual Machine (VM) ndash A virtual or logical system (hardware) running under the control of virtualization software such as a hypervisor The virtual machine nomenclature is also used to describe the virtualization of processes In this paper virtual machine is used in the context of duplicating hardware

Virtual Machine Monitor ndash Another term for

hypervisor

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 5

Background and History of Xen

Xen is a hypervisor or virtual machine monitor which is a thin layer of software that is loaded onto bare-metal hardware and that enables multiple virtual machines (or domains) to run concurrently on the same system hardware The Xen hypervisor grew out of the open source community after it began as a research project at Englandrsquos University of Cambridge The first release became generally available in 2003 after the presentation and publication of ldquoXen and the Art of Virtualizationrdquo at the ACM Symposium of Operating System Principles The Xen virtual machine monitor was originally developed for the x86 architecture to be very efficient and to host up to 100 virtual machine instances simultaneously on both Windows and Linux with little-to-no performance overhead (as compared with an unvirtualized environment)

The Intel x86 and Itanium processors were not originally designed to support a virtualization environment This made creating a hypervisor difficult forcing increased complexity and requiring large amounts of code thereby reducing performance One way of overcoming these challenges was to modify the guest operating system to understand that it is running in a virtual environment (paravirtualization)

In late 2005 Intel delivered hardware-assists for virtualization known as Intel VT-x (x86) and Intel VT-i (Itanium) extensions to its chipsets These extensions improved performance and removed the need for modifiedparavirtualized guest operating systems The Xen 30 release (December 2005) took full advantage of these enhancements to the chipset enabling support for more guest operating systems

In 2003 with the first release Xen was placed in open source and the Xen community was formed Since that time some of the industryrsquos top technologists have been working together to continue to create update and improve releases of Xen

Solution Overview ndash Red Hat Enterprise Linux on Itanium

The 51 release of Red Hat Enterprise Linux comes with the Xen hypervisor integrated and has support for guest operating systems built using the Intel Itanium processor with the VT-i extensions This release combines the value of several open source technologies including Linux Xen libvirt and others into a server-based solution that offers many mainframe-class features such as high availability superior error handling and dynamic management

Xen and Red Hat Enterprise Linux

Combining two strong open source technology solutions mdash Linux and Xen mdash RHEL 51 and later versions ship with the Xen hypervisor included As shown in Figure 2 the Xen hypervisor is loaded on the bare metal and runs on the Intel Itanium chipset both with and without the Intel VT-i extensions However only the paravirtualized guest operating systems can run on the Itanium hardware without support for Intel VT-i

The Xen architecture defines the virtual machine monitor separate from the device drivers To gain access to the plethora of device drivers already available and to maintain high-performance

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 6

access to IO the device drivers run in ldquoDomain0rdquo (dom0) or host domain which runs RHEL This domain provides the IO infrastructure for all virtual machines that request IO using hypercalls All IO requests from the guest operating systems go through the parent partitionrsquos device drivers

Key Features

Red Hat supports both paravirtualized and fully virtualized (unmodified) guest operating systems To deliver the highest levels of performance and scalability the kernel of the guest virtual machine can be optimized to run in a virtual environment In this model the virtual machine ldquocooperatesrdquo with the hypervisor eliminating the overhead of emulation and delivering near bare-metal performance

With paravirtualization the guest operating system becomes ldquovirtualization-awarerdquo allowing the guest to benefit from advanced features not possible in traditional emulation-based solutions One example is dynamic resource allocation or allowing the hardware resources of the guest to be changed on the fly mdash adding or removing memory CPUs disks and even network interfaces mdash without requiring a reboot

Live migration allows an administrator to dynamically relocate a virtual machine from one host system to another without service interruption mdash the virtual machine continues to operate and service user requests during the migration This can be used to perform hardware maintenance without application downtime by moving the virtual machine to a new host during a maintenance window Additionally live migration is used to dynamically balance resources within a cluster to gain optimal utilization of the resources

Red Hat also provides high-availability clustering and failover to enable automated restart of the virtual machines and associated environments in the event of hardware failure

Guest Operating Systems

As shown in Figure 2 the guest operating systems are run in virtual machines also known as unprivileged domains (domU) Red Hat offers a paravirtualized version of RHEL 5 that can run on Intel Itanium and with the VT-i extensions This provides for the lowest overhead and highest performing operating system support for applications that require this level of performance

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 7

Figure 2 High-level RHEL 51 for Itanium architecture

Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported

Management

There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release

The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments

Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 8

information model provider for the Distributed Management Task Force (DMTF) virtualization schema

Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform

Red Hat delivers a number of management tools built on libvirt including

virsh ndash A command line shell enabling customers to script and automate virtualization management

Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management

Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring

Licensing

Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL

Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product

Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices

Intel Itanium Overview

The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 9

Features

All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture

A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously

RAS Feature Typical

Mainframe Intel Itanium

Platforms Intel Xeon

MP Platforms Intel Xeon Platforms

Cache ECC coverage

Memory single-device error correct

Memory retry on double-bit error

Error recovery on data bus (ECC)

Internal logic soft error checking

Badpoisoned data containment

Cache reliability

Memory sparing

Memory mirroring

Hot plug IO (PCI-X PCI Express)

Memory hot swap

Table 1 Dual-core Itanium processor error coverage

With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)

Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 10

Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs

The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware

Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture

Intel VT-i

The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 11

paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment

In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments

Key Benefits and Challenges

For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system

Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility

As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization

Conclusions and Recommendations

The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 12

FOCUS Assessment

Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices

References

ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf

ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance

Other Related FOCUS Reports

FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch

FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC

FOCUS Market Landscape Report Desktop and Application Delivery Alternatives

FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager

FOCUS Solution Profile Citrix XenServer

FOCUS Solution Profile VMware Desktop Solutions

All trademarks are the properties of their respective owners

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 13

About FOCUS

Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years

Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers

FOCUS delivers research analysis and consulting focused on systems software and storage

in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of

Technology (ROI TCO) wwwfocusonsystemscom

Page 6: Desktop and Application Delivery Alternatives

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 5

Background and History of Xen

Xen is a hypervisor or virtual machine monitor which is a thin layer of software that is loaded onto bare-metal hardware and that enables multiple virtual machines (or domains) to run concurrently on the same system hardware The Xen hypervisor grew out of the open source community after it began as a research project at Englandrsquos University of Cambridge The first release became generally available in 2003 after the presentation and publication of ldquoXen and the Art of Virtualizationrdquo at the ACM Symposium of Operating System Principles The Xen virtual machine monitor was originally developed for the x86 architecture to be very efficient and to host up to 100 virtual machine instances simultaneously on both Windows and Linux with little-to-no performance overhead (as compared with an unvirtualized environment)

The Intel x86 and Itanium processors were not originally designed to support a virtualization environment This made creating a hypervisor difficult forcing increased complexity and requiring large amounts of code thereby reducing performance One way of overcoming these challenges was to modify the guest operating system to understand that it is running in a virtual environment (paravirtualization)

In late 2005 Intel delivered hardware-assists for virtualization known as Intel VT-x (x86) and Intel VT-i (Itanium) extensions to its chipsets These extensions improved performance and removed the need for modifiedparavirtualized guest operating systems The Xen 30 release (December 2005) took full advantage of these enhancements to the chipset enabling support for more guest operating systems

In 2003 with the first release Xen was placed in open source and the Xen community was formed Since that time some of the industryrsquos top technologists have been working together to continue to create update and improve releases of Xen

Solution Overview ndash Red Hat Enterprise Linux on Itanium

The 51 release of Red Hat Enterprise Linux comes with the Xen hypervisor integrated and has support for guest operating systems built using the Intel Itanium processor with the VT-i extensions This release combines the value of several open source technologies including Linux Xen libvirt and others into a server-based solution that offers many mainframe-class features such as high availability superior error handling and dynamic management

Xen and Red Hat Enterprise Linux

Combining two strong open source technology solutions mdash Linux and Xen mdash RHEL 51 and later versions ship with the Xen hypervisor included As shown in Figure 2 the Xen hypervisor is loaded on the bare metal and runs on the Intel Itanium chipset both with and without the Intel VT-i extensions However only the paravirtualized guest operating systems can run on the Itanium hardware without support for Intel VT-i

The Xen architecture defines the virtual machine monitor separate from the device drivers To gain access to the plethora of device drivers already available and to maintain high-performance

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 6

access to IO the device drivers run in ldquoDomain0rdquo (dom0) or host domain which runs RHEL This domain provides the IO infrastructure for all virtual machines that request IO using hypercalls All IO requests from the guest operating systems go through the parent partitionrsquos device drivers

Key Features

Red Hat supports both paravirtualized and fully virtualized (unmodified) guest operating systems To deliver the highest levels of performance and scalability the kernel of the guest virtual machine can be optimized to run in a virtual environment In this model the virtual machine ldquocooperatesrdquo with the hypervisor eliminating the overhead of emulation and delivering near bare-metal performance

With paravirtualization the guest operating system becomes ldquovirtualization-awarerdquo allowing the guest to benefit from advanced features not possible in traditional emulation-based solutions One example is dynamic resource allocation or allowing the hardware resources of the guest to be changed on the fly mdash adding or removing memory CPUs disks and even network interfaces mdash without requiring a reboot

Live migration allows an administrator to dynamically relocate a virtual machine from one host system to another without service interruption mdash the virtual machine continues to operate and service user requests during the migration This can be used to perform hardware maintenance without application downtime by moving the virtual machine to a new host during a maintenance window Additionally live migration is used to dynamically balance resources within a cluster to gain optimal utilization of the resources

Red Hat also provides high-availability clustering and failover to enable automated restart of the virtual machines and associated environments in the event of hardware failure

Guest Operating Systems

As shown in Figure 2 the guest operating systems are run in virtual machines also known as unprivileged domains (domU) Red Hat offers a paravirtualized version of RHEL 5 that can run on Intel Itanium and with the VT-i extensions This provides for the lowest overhead and highest performing operating system support for applications that require this level of performance

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 7

Figure 2 High-level RHEL 51 for Itanium architecture

Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported

Management

There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release

The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments

Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 8

information model provider for the Distributed Management Task Force (DMTF) virtualization schema

Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform

Red Hat delivers a number of management tools built on libvirt including

virsh ndash A command line shell enabling customers to script and automate virtualization management

Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management

Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring

Licensing

Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL

Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product

Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices

Intel Itanium Overview

The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 9

Features

All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture

A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously

RAS Feature Typical

Mainframe Intel Itanium

Platforms Intel Xeon

MP Platforms Intel Xeon Platforms

Cache ECC coverage

Memory single-device error correct

Memory retry on double-bit error

Error recovery on data bus (ECC)

Internal logic soft error checking

Badpoisoned data containment

Cache reliability

Memory sparing

Memory mirroring

Hot plug IO (PCI-X PCI Express)

Memory hot swap

Table 1 Dual-core Itanium processor error coverage

With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)

Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 10

Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs

The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware

Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture

Intel VT-i

The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 11

paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment

In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments

Key Benefits and Challenges

For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system

Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility

As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization

Conclusions and Recommendations

The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 12

FOCUS Assessment

Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices

References

ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf

ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance

Other Related FOCUS Reports

FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch

FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC

FOCUS Market Landscape Report Desktop and Application Delivery Alternatives

FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager

FOCUS Solution Profile Citrix XenServer

FOCUS Solution Profile VMware Desktop Solutions

All trademarks are the properties of their respective owners

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 13

About FOCUS

Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years

Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers

FOCUS delivers research analysis and consulting focused on systems software and storage

in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of

Technology (ROI TCO) wwwfocusonsystemscom

Page 7: Desktop and Application Delivery Alternatives

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 6

access to IO the device drivers run in ldquoDomain0rdquo (dom0) or host domain which runs RHEL This domain provides the IO infrastructure for all virtual machines that request IO using hypercalls All IO requests from the guest operating systems go through the parent partitionrsquos device drivers

Key Features

Red Hat supports both paravirtualized and fully virtualized (unmodified) guest operating systems To deliver the highest levels of performance and scalability the kernel of the guest virtual machine can be optimized to run in a virtual environment In this model the virtual machine ldquocooperatesrdquo with the hypervisor eliminating the overhead of emulation and delivering near bare-metal performance

With paravirtualization the guest operating system becomes ldquovirtualization-awarerdquo allowing the guest to benefit from advanced features not possible in traditional emulation-based solutions One example is dynamic resource allocation or allowing the hardware resources of the guest to be changed on the fly mdash adding or removing memory CPUs disks and even network interfaces mdash without requiring a reboot

Live migration allows an administrator to dynamically relocate a virtual machine from one host system to another without service interruption mdash the virtual machine continues to operate and service user requests during the migration This can be used to perform hardware maintenance without application downtime by moving the virtual machine to a new host during a maintenance window Additionally live migration is used to dynamically balance resources within a cluster to gain optimal utilization of the resources

Red Hat also provides high-availability clustering and failover to enable automated restart of the virtual machines and associated environments in the event of hardware failure

Guest Operating Systems

As shown in Figure 2 the guest operating systems are run in virtual machines also known as unprivileged domains (domU) Red Hat offers a paravirtualized version of RHEL 5 that can run on Intel Itanium and with the VT-i extensions This provides for the lowest overhead and highest performing operating system support for applications that require this level of performance

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 7

Figure 2 High-level RHEL 51 for Itanium architecture

Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported

Management

There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release

The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments

Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 8

information model provider for the Distributed Management Task Force (DMTF) virtualization schema

Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform

Red Hat delivers a number of management tools built on libvirt including

virsh ndash A command line shell enabling customers to script and automate virtualization management

Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management

Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring

Licensing

Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL

Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product

Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices

Intel Itanium Overview

The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 9

Features

All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture

A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously

RAS Feature Typical

Mainframe Intel Itanium

Platforms Intel Xeon

MP Platforms Intel Xeon Platforms

Cache ECC coverage

Memory single-device error correct

Memory retry on double-bit error

Error recovery on data bus (ECC)

Internal logic soft error checking

Badpoisoned data containment

Cache reliability

Memory sparing

Memory mirroring

Hot plug IO (PCI-X PCI Express)

Memory hot swap

Table 1 Dual-core Itanium processor error coverage

With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)

Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 10

Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs

The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware

Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture

Intel VT-i

The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 11

paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment

In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments

Key Benefits and Challenges

For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system

Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility

As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization

Conclusions and Recommendations

The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 12

FOCUS Assessment

Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices

References

ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf

ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance

Other Related FOCUS Reports

FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch

FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC

FOCUS Market Landscape Report Desktop and Application Delivery Alternatives

FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager

FOCUS Solution Profile Citrix XenServer

FOCUS Solution Profile VMware Desktop Solutions

All trademarks are the properties of their respective owners

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 13

About FOCUS

Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years

Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers

FOCUS delivers research analysis and consulting focused on systems software and storage

in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of

Technology (ROI TCO) wwwfocusonsystemscom

Page 8: Desktop and Application Delivery Alternatives

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 7

Figure 2 High-level RHEL 51 for Itanium architecture

Red Hat also provides a number of supported guest operating systems that only run on Itanium chipsets with the Intel VT-i Itanium extensions Additionally the RHEL 4 and 5 guest operating systems can include paravirtualized IO improving IO performance running on the Intel Itanium with the VT-i extensions For customers who are already running versions of RHEL 3 4 or 5 and wish to consolidate using Red Hat Virtualization full virtualization of these unmodified guest operating systems is also supported Additionally Windows Server 2000 and Server 2003 are supported

Management

There are two important aspects to the Red Hat management offerings for RHEL 5 and virtualization that are worth discussing here One aspect is Red Hatrsquos time money and dedication in developing the libvirt virtualization API and then releasing it to open source The other is the management capabilities that are shipped with the RHEL 51 release

The libvirt API is a C toolkit that was developed by Red Hat to be a stable interface that interacts with the virtualization capabilities of a number of hypervisors It provides for local and remote management of the entire virtual environment including machines networks and storage Remote management is secured using transport layer security certificates authenticating with Kerberos and Simple Authentication and Security Layer (SASL) It was first released to open source by Red Hat in late 2005 providing basic management of existing Xen environments

Today libvirt supports Xen KVM QEMU Linux Containers (LXC) OpenVZ and Solaris Logical Domains (LDoms) It is available for free under the GNU Lesser General Public License (wwwlibvirtorg) and comes with a set of bindings for common languages and a common

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 8

information model provider for the Distributed Management Task Force (DMTF) virtualization schema

Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform

Red Hat delivers a number of management tools built on libvirt including

virsh ndash A command line shell enabling customers to script and automate virtualization management

Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management

Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring

Licensing

Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL

Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product

Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices

Intel Itanium Overview

The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 9

Features

All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture

A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously

RAS Feature Typical

Mainframe Intel Itanium

Platforms Intel Xeon

MP Platforms Intel Xeon Platforms

Cache ECC coverage

Memory single-device error correct

Memory retry on double-bit error

Error recovery on data bus (ECC)

Internal logic soft error checking

Badpoisoned data containment

Cache reliability

Memory sparing

Memory mirroring

Hot plug IO (PCI-X PCI Express)

Memory hot swap

Table 1 Dual-core Itanium processor error coverage

With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)

Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 10

Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs

The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware

Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture

Intel VT-i

The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 11

paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment

In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments

Key Benefits and Challenges

For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system

Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility

As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization

Conclusions and Recommendations

The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 12

FOCUS Assessment

Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices

References

ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf

ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance

Other Related FOCUS Reports

FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch

FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC

FOCUS Market Landscape Report Desktop and Application Delivery Alternatives

FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager

FOCUS Solution Profile Citrix XenServer

FOCUS Solution Profile VMware Desktop Solutions

All trademarks are the properties of their respective owners

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 13

About FOCUS

Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years

Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers

FOCUS delivers research analysis and consulting focused on systems software and storage

in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of

Technology (ROI TCO) wwwfocusonsystemscom

Page 9: Desktop and Application Delivery Alternatives

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 8

information model provider for the Distributed Management Task Force (DMTF) virtualization schema

Libvirt is shipped with all leading Linux distributions including Red Hat Enterprise Linux Novells SUSE Enterprise Linux Canonicals Ubuntu and many others Libvirt is also included within Sun Solaris to manage Logical Domains and Suns xVM platform

Red Hat delivers a number of management tools built on libvirt including

virsh ndash A command line shell enabling customers to script and automate virtualization management

Virtual Machine Manager (virt-manager) ndash A graphical tool for managing local and remote virtual hosts including network configuration storage configuration installation and full life cycle management

Red Hat Network which provides automated management capabilities for the entire Red Hat Enterprise from a centralized console has been extended to support virtualization providing complete life cycle management for both virtual and physical machines including provisioning patch management configuration management and monitoring

Licensing

Customers running previous releases of Red Hat Enterprise Linux (eg versions 21 3 and 4) can upgrade to Red Hat Enterprise Linux 5 for free Red Hat subscriptions allow the customer to run any release of RHEL

Using Red Hats virtualization with Red Hat Enterprise Linux Server customers can run four RHEL virtual machines for no additional cost these subscriptions are included with the base product

Red Hat Enterprise Linux Advanced Platform supports an unlimited number of guest environments constrained only by physical hardware resources Red Hat does not limit the number of virtual machines run on the host system but third-party operating system guests such as Windows must be licensed in compliance with the respective vendorrsquos licensing practices

Intel Itanium Overview

The need for reliable hardware platforms has never been greater As the number of virtual machines or domains being run on a system increases the damage a hardware failure can cause greatly increases Rather than affecting just one applicationuser hardware failures in virtual environments affect many applications and users To provide the level of high reliability required in a virtual environment tight integration of hardware software and firmware collaborating on a number of levels mdash including error detection correction and management as well as virtualization The Intel Itanium VT-I chipset incorporates these features to provide proactive high reliability for virtual platforms

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 9

Features

All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture

A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously

RAS Feature Typical

Mainframe Intel Itanium

Platforms Intel Xeon

MP Platforms Intel Xeon Platforms

Cache ECC coverage

Memory single-device error correct

Memory retry on double-bit error

Error recovery on data bus (ECC)

Internal logic soft error checking

Badpoisoned data containment

Cache reliability

Memory sparing

Memory mirroring

Hot plug IO (PCI-X PCI Express)

Memory hot swap

Table 1 Dual-core Itanium processor error coverage

With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)

Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 10

Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs

The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware

Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture

Intel VT-i

The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 11

paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment

In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments

Key Benefits and Challenges

For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system

Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility

As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization

Conclusions and Recommendations

The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 12

FOCUS Assessment

Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices

References

ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf

ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance

Other Related FOCUS Reports

FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch

FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC

FOCUS Market Landscape Report Desktop and Application Delivery Alternatives

FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager

FOCUS Solution Profile Citrix XenServer

FOCUS Solution Profile VMware Desktop Solutions

All trademarks are the properties of their respective owners

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 13

About FOCUS

Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years

Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers

FOCUS delivers research analysis and consulting focused on systems software and storage

in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of

Technology (ROI TCO) wwwfocusonsystemscom

Page 10: Desktop and Application Delivery Alternatives

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 9

Features

All processors have to perform some level of errorfault handling One of the critical differentiators between different processors is their ability to detect automatically correct and report these errors Intel Itanium processors were designed for high-end reliability availability and serviceability (RAS) of business-critical applications environments supported by a virtual infrastructure This is accomplished in the Itanium-based systems utilizing a sophisticated layered error handling architecture called Machine Check Architecture

A key component of the MCA is its firmware layer called the processor abstraction layer (PAL) PAL is capable of correcting and logging all one-bit errors (see Table 1) and handling over 9999 of all processor hardware errors potentially including multiple errors simultaneously

RAS Feature Typical

Mainframe Intel Itanium

Platforms Intel Xeon

MP Platforms Intel Xeon Platforms

Cache ECC coverage

Memory single-device error correct

Memory retry on double-bit error

Error recovery on data bus (ECC)

Internal logic soft error checking

Badpoisoned data containment

Cache reliability

Memory sparing

Memory mirroring

Hot plug IO (PCI-X PCI Express)

Memory hot swap

Table 1 Dual-core Itanium processor error coverage

With the Itanium MCA memory and processing errors can be resolved within the chipset (see Table 1) If the errors cannot be resolved within the chipset the errors are logged in the systems abstraction layer (SAL) to be processed by the Xen hypervisor (see Figure 3) Guest operating systems can also access the SAL tables through a Xen API called a hypercall (system call to the hypervisor)

Processors with fewer error handling capabilities than the Intel Itanium processor must send more errors directly to the operating system for handling Reducing the number of errors sent to the operating system both improves error handling performance (and system performance) and reduces the opportunities for system-level failures For cases where the PAL is not able to handle the error it is handed to the system abstraction layer for handling or to be passed to the

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 10

Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs

The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware

Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture

Intel VT-i

The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 11

paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment

In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments

Key Benefits and Challenges

For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system

Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility

As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization

Conclusions and Recommendations

The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 12

FOCUS Assessment

Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices

References

ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf

ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance

Other Related FOCUS Reports

FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch

FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC

FOCUS Market Landscape Report Desktop and Application Delivery Alternatives

FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager

FOCUS Solution Profile Citrix XenServer

FOCUS Solution Profile VMware Desktop Solutions

All trademarks are the properties of their respective owners

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 13

About FOCUS

Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years

Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers

FOCUS delivers research analysis and consulting focused on systems software and storage

in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of

Technology (ROI TCO) wwwfocusonsystemscom

Page 11: Desktop and Application Delivery Alternatives

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 10

Xen hypervisor The value of logging and handing off error information to the SAL is that the system is made aware of these errorsfaults and can react to faulting processor(s) or memory proactively before a downtime-inducing failure occurs

The ability to log and monitor these errors allows the system to be alerted to failing hardware prior to a hard failure providing systems administrators or policy-based software the ability to proactively move virtual machines (live migrate) off the failing hardware to perform preventative maintenance Once the failing part has been replaced the virtual machines can be moved back onto the system hardware

Figure 3 Red HatXen interaction with Itanium VT-i with machine check architecture

Intel VT-i

The Intel VT-i extensions bring virtualization hardware assist to the Itanium-based chipsets because they were originally developed to support one OS per server not multiple operating systems running on top of a hypervisor This new paradigm brought with it some difficulties around privileged access to the hardware Unmodified OS kernels are expected to run at the most privileged levels (Ring 0) to access hardware including the CPU memory and IO devices The Xen hypervisor as a bare-metal virtual machine monitor by definition had to run at the most privileged level This caused conflict between unmodified guest operating systems and the hypervisor Initially to solve this conflict operating systems vendors modified or

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 11

paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment

In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments

Key Benefits and Challenges

For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system

Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility

As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization

Conclusions and Recommendations

The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 12

FOCUS Assessment

Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices

References

ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf

ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance

Other Related FOCUS Reports

FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch

FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC

FOCUS Market Landscape Report Desktop and Application Delivery Alternatives

FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager

FOCUS Solution Profile Citrix XenServer

FOCUS Solution Profile VMware Desktop Solutions

All trademarks are the properties of their respective owners

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 13

About FOCUS

Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years

Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers

FOCUS delivers research analysis and consulting focused on systems software and storage

in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of

Technology (ROI TCO) wwwfocusonsystemscom

Page 12: Desktop and Application Delivery Alternatives

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 11

paravirtualized their operating system to know it was being run in a virtual environment and to use hypercalls to the hypervisor to gain privileged access to hardware andor services Unmodified guest operating systems did not work in this environment

In 2005 Intel introduced the Intel VT-i extensions to Itanium processor hardware and the PAL firmware The hardware extensions consist of the addition of a new bit that signals whether an instruction came from a hypervisor (privileged) or a guest (non-privileged) This allows the guest operating system to run unmodified at its expected privilege level giving the hypervisor the flexibility to use multiple privilege levels The processor abstraction layer extensions include a consistent programming interface to the hypervisor across multiple processor generations (and possibly different implementations) These extensions also define how the PAL interacts with the virtual machine environments

Key Benefits and Challenges

For Linux customers who need mainframe-class systems hardware to support a consolidated server infrastructure RHEL 51 with Xen running on Intel Itanium platforms combines the benefits of open source technology innovation with Itaniumrsquos advanced reliability availability and serviceability With the availability of Intelrsquos VT-i extensions to Itanium chipsets IT can run paravirtualized guest operating systems to realize greater gains in performance or they can run unmodified guest operating systems including Linux and Windows with minimal overhead to solve a breadth of computing needs all on one physical system

Applications on RHEL and Itanium continue to run unmodified with the addition of the Xen hypervisor Red Hat provides a consistent application binary interface (ABI) between bare-metal deployments and virtualized deployments guaranteeing application compatibility

As with all virtualization environments applications requiring very tight integration with the systems hardware may not be the best candidates for virtualization

Conclusions and Recommendations

The combination of Red Hat Enterprise Linux virtualization and Itanium brings high performance plus high reliability availability and serviceability (RAS) to the Linux server consolidation market For customers who are looking to consolidate servers and are running in a mixed or predominantly Linux environment with applications that support Itanium the combined solution of RHEL 51 on Intel Itanium with VT-i extensions is a very strong candidate for this environment By providing virtualization at no added cost and with guest virtual machine subscriptions for free an organization can save thousands of dollars in software license costs

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 12

FOCUS Assessment

Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices

References

ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf

ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance

Other Related FOCUS Reports

FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch

FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC

FOCUS Market Landscape Report Desktop and Application Delivery Alternatives

FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager

FOCUS Solution Profile Citrix XenServer

FOCUS Solution Profile VMware Desktop Solutions

All trademarks are the properties of their respective owners

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 13

About FOCUS

Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years

Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers

FOCUS delivers research analysis and consulting focused on systems software and storage

in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of

Technology (ROI TCO) wwwfocusonsystemscom

Page 13: Desktop and Application Delivery Alternatives

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 12

FOCUS Assessment

Integrating the Xen hypervisor with the market-leading Red Hat Enterprise Linux will contribute to virtualization becoming ubiquitous in the market For RHEL customers it makes the move to virtualization as easy as enabling a feature The consolidation of multiple server workloads onto one physical server should be a strong motivator for IT to look to platforms with higher performance plus higher reliability and availability to run the multiple workloads now being placed upon them A combined RHELXenItanium solution offers a high-end RAS choice for Linux workloads with mainframe features but without mainframe prices

References

ldquoXen and the Art of Virtualizationrdquo Paul Barham Boris Dragovic Keir Fraser Steven Hand Tim -Computer Laboratory Society of Operating Systems Principles (SOSP) 2003 http6720714065wp-contentuploads200802xen-and-art-of-virtualization-2003-paperpdf

ldquoOptimizing IT Value by Mixing and Matching Industry-Standard Server Platforms mdash Itaniumreg 2-based Solutions and the x86 Architecturerdquo Itanium Solutions Alliance

Other Related FOCUS Reports

FOCUS has published the following related reports For more information please see details at httpwwwfocusonsystemscomresearch

FOCUS White Paper Dynamic Hardware Partitioning ndash Intel Microsoft NEC

FOCUS Market Landscape Report Desktop and Application Delivery Alternatives

FOCUS Solution Profile Microsoft Hyper-V and System Center Virtual Machine Manager

FOCUS Solution Profile Citrix XenServer

FOCUS Solution Profile VMware Desktop Solutions

All trademarks are the properties of their respective owners

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 13

About FOCUS

Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years

Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers

FOCUS delivers research analysis and consulting focused on systems software and storage

in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of

Technology (ROI TCO) wwwfocusonsystemscom

Page 14: Desktop and Application Delivery Alternatives

FC 8152008

Red Hat Enterprise Linux Virtualization and Intel Itanium

2008 FOCUS Consulting wwwfocusonsystemscom Page 13

About FOCUS

Anne Skamarock Research Director with FOCUS has spent nearly 30 years in software engineering and technical marketing as an end-user vendor analyst and author with Sun SRI Solbourne StorageTek and Enterprise Management Associates (EMA) For the past several years she has focused on virtualization and blade systems as they have moved from servers to desktops As an analyst for the past decade she has covered systems software storage storage networking and storage management solutions and the intersection points between systems software and storage She has been a regular expert columnist for Network World and TechTarget and is co-author of the book Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs A frequent speaker at conferences she is co-chair of Interoprsquos Virtualization Track was co-creator and Program Manager for the ROI Planning Lab at the 2007 Server Blade Summit on Blades and Virtualization and chaired Interops Network Storage Track for several years

Barb Goldworm president and chief analyst of FOCUS has spent 30 years in technical development marketing sales senior management and industry analyst positions with IBM Novell StorageTek EMA and multiple startups Barb is virtualization chair for Interop and Blade Systems Insight chaired the 2007 Server Blade Summit on Blades and Virtualization created and chaired the Network Storage Track of Interop and has been one of the top ranked expert speakers at Data Center Decisions and SNW Barb is on the advisory board for several TechTarget sites and has been a regular expert columnist and speaker for TechTarget Ziff-Davis Computerworld Storage Networking World Online Network World and Virtual Strategy Magazine Co-author of Blade Servers and Virtualization Transforming Enterprise Computing While Cutting Costs she has published extensively including research reports market studies landscape reports and white papers

FOCUS delivers research analysis and consulting focused on systems software and storage

in the enterprise SMB and government markets FOCUS areas include Systems Storage and Enterprise Management (Physical and Virtual) Server Desktop and Application VirtualizationStreaming High Availability Disaster Recovery Business Continuity Backup Data Protection Storage Networking (NAS SAN Fibre Channel iSCSI) Storage Networks IO and File Virtualization Storage Technologies (Clustered File Systems data de-duplication VTL etc) Blade Systems (Server Workstation and PC Blades) and Business Benefits of

Technology (ROI TCO) wwwfocusonsystemscom