develop, consolidate and manage virtual environments entirely
TRANSCRIPT
OpenSolaris Virtualization Technologies
Aaron Dailey
Staff EngineerSolaris Storage Software
Agenda• Virtualization Overview• Zones
> BrandZ> lx
• Xen (xVM)• VirtualBox• Demo• Q&A
The Need for Virtualization• Driven by the need to consolidate multiple hosts
and services on a single machine• Leads to...
> Increased hardware utilization (currently average data center utilization is below 15%)
> Greater flexibility in resource allocation> Reduced power requirements> Minimize management costs> Lower the cost of ownership
Use Cases• Server Consolidation• Testing and Development• Fail-over through replication• Provisioning compute resources• Legacy Applications• Secure Execution
Types of Virtualization• Hard Partitions
> Ex: Sun Domains, IBM LPARS, HP nPARS• Virtual Machines
> Para-virtualization> Ex: Xen, User-Mode Linux (UML)
> Full Virtualization> Ex: LDOMS, VMWare, Parallels, Xen (VT or AMDV), Virtual
Box• OS level Virtualization
> Ex: Solaris Zones, Vservers, Jails
Types of VirtualizationHard Partitions Virtual Machines OS Virtualization Resource Mgmt.
Server
OS
App
Multiple OS's Single OS
Dynamic SystemDomains\
Solaris Containers(Zones + SRM)
BrandZ
Solaris ResourceManager (SRM)
Logical DomainsXen
Trend to flexibility Trend to isolation
CrossBow
Trend to observability
●OS level Virtualization
Solaris ZonesSingle OS instance – Many Virtual OSes
• Virtualize OS services to emulate an OS instance• Isolates applications from each other• Improve security by intrusion-isolation• Boot and bring down zones independent of the OS
instance• Compatible with existing applications• Complements existing resource management.• Almost arbitrary granularity in isolating and sharing
resources
Facilities Virtualized for Zones• Processes• File Systems• Networking• Identity• Devices• Packaging
Zones Block Diagram
network device(hme0)
storage complex
global zone (v880-room2-rack5-1; 129.76.1.12)
dns1 zone (dnsserver1)
zoneadmd
mail zone (mailserver)
network services(sendmail, IMAP)
remote admin/monitoring(SNMP, SunMC, WBEM)
platform administration(syseventd, devfsadm, ifconfig, metadb,...)
core services(inetd)
core services(inetd)
core services(inetd, rpcbind, sshd, ...)
zone root: /zone/dns1 zone root: /zone/mail1
network device(ce0)
zone management (zonecfg(1M), zoneadm(1M), zlogin(1), ...)
ce0:
3
ce1:
1
hme0
:1
zcon
s
zcon
s
zoneadmd
/usr
/usr
Appli
catio
nEn
viron
ment
Virtu
alPl
atfor
m
login services(SSH sshd)
network services(named)
zoneadmd
web1 zone (foo.org)
network services(Apache, Tomcat)
core services(inetd)
zone root: /zone/web1
hme0
:2
ce0:
1
zcon
s
/usr
zoneadmd
web2 zone (bar.net)
network services(IWS)
core services(inetd)
zone root: /zone/web2
hme0
:3
ce0:
2
zcon
s
/usr
pool2 (4 CPU)
network device(ce1)
login services(SSH sshd)
login services(SSH sshd)
login services(SSH sshd, telnetd)
10
pool1 (4 CPU), FSS
30 60
BrandZ: Branded Zones• Extends Zones model to support “non-native” zones
on a Solaris system> Only supports user-space environments> If you need a different kernel, see Xen
• Each distinct zone type is called a Brand• Possible uses:
> A Linux zone> A Solaris GNU zone (Nexenta/ShilliX/BeleniX)> Support for Solaris N-1 on Solaris N> A MacOS X zone
The lx Brand• Marketing Name: Solaris Containers for Linux
Applications• Enables Linux Binaries to run unmodified on Solaris • Creates a zone for Linux application execution
> Zone is populated only with Linux software> At boot, it runs the Linux init(1M), configuration scripts, and
applications> It all runs on a Solaris kernel.
• There is no Linux software delivered with BrandZ> This is not a new Linux distro> We install and run standard Linux distributions
Branded Zones Block Diagram
network device(hme0)
storage complex
global zone (v880-room2-rack5-1; 129.76.1.12)
dns1 zone (dnsserver1)
zoneadmd
Linux zone (linux)
remote admin/monitoring(SNMP, SunMC, WBEM)
platform administration(syseventd, devfsadm, ifconfig, metadb,...)
core services(inetd)
Linux core services(NIS, xinetd, autofs)
core services(inetd, rpcbind, sshd, ...)
zone root: /zone/dns1 zone root: /zone/lx
network device(ce0)
zone management (zonecfg(1M), zoneadm(1M), zlogin(1), ...)
ce0:
3
ce1:
1
hme0
:1
zcon
s
zcon
s
zoneadmd
/usr
/usr
Appli
catio
nEn
viron
ment
Virtu
alPl
atfor
m
login services(SSH sshd)
network services(named)
zoneadmd
web1 zone (foo.org)
network services(Apache, Tomcat)
core services(inetd)
zone root: /zone/web1
hme0
:2
ce0:
1
zcon
s
/usr
zoneadmd
web2 zone (bar.net)
network services(IWS)
core services(inetd)
zone root: /zone/web2
hme0
:3
ce0:
2
zcon
s
/usr
pool2 (4 CPU)
network device(ce1)
login services(SSH sshd)
login services(SSH sshd, telnetd)
10
pool1 (4 CPU), FSS
30 60
linux user apps (OpenSSH 3.6, acroread,MATLAB, yum, pandora)
BrandZ Use Cases• As a transition tool, reducing the Linux “barrier to exit”
> Customer would like to move to Solaris, but has legacy Linux applications
• Best of both worlds> Users familiar with Linux environment> Administrators want Solaris' enterprise-class features:
resource management, fault management, DTrace• Developer/ISV workload
> Solaris has strong development tools, let Linux developers leverage them.
> We want Solaris to be a better Linux development platform than Linux.
What BrandZ is Not• Not a full system emulator or virtualization layer
> No non-Solaris kernel code is ever executed.> You can't run any random Linux distribution.
• Doesn't support all Linux kernel functionality.> No support for Linux file systems, kernel modules, or
device drivers.> Not all system calls are fully supported.
• Not simply binary emulation (like lxrun, wine, etc.)> You can't just run the Linux version of acroread from your
Solaris shell prompt.
BrandZ Status• Available in Nevada (SX*) and s10u4• Zones running a Red Hat Enterprise Linux 3.x or
CentOS 3.x operating environment> Support for Linux 2.4.21 system call interface> Basic /proc and /dev support
• DTrace support for Linux applications> Linux syscall provider> PID provider
• Rapid deployment and teardown of Linux zones.> Perfect for building 'throwaway' zones for
development/QA
●Virtual Machines
Para- vs Full Virtualization• Para-virtualization
> Runs OS ported to virtual machine arch> Uses “virtual” device drivers to communicate between
guest and host operating systems> More efficient since it is hypervisor-aware> Xen, Logical Domains
• Full virtualization> Runs binary image of “metal” OS> Must emulate real I/O devices> Can be slow, needs help from hardware> May use trap and emulate or rewriting> VMware, Parallels, Xen, Virtual Box
Xen• Open source hypervisor technology developed at
the University of Cambridge> http://www.cl.cam.ac.uk/Research/SRG/netos/xen/
• Supports both para- and full virtualization• Runs on x86/x64, PowerPC, Itanium• Supports Solaris, Linux, FreeBSD• OpenSolaris on Xen (xVM) community
> http://www.opensolaris.org/os/community/xen
Xen 3.x Architecture
Event Channel Virtual MMUVirtual CPU Control IF
Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE)
NativeDeviceDriver
GuestOS(Solaris)
Device Manager & Control s/w
VM0
NativeDeviceDriver
GuestOS(XenLinux )
VM1
Front -EndDevice Drivers
GuestOS(Solaris)
UnmodifiedUser
Software
VM2
Front -EndDevice Drivers
UnmodifiedGuestOS(WinXP ))
UnmodifiedUser
Software
VM3
Safe HW IF
Xen Virtual Machine Monitor
Back -End Back -End
VT-xAMDV
32/64bit
AGPACPIPCI
SMP
dom0 domU1 domU2 domU3
Key Xen Capabilities• Multiple OSes running simultaneously
> Linux, Solaris, Windows XP> No longer a boot-time decision
• Special purpose kernels within DomUs for paravirtualized> Drivers, filesystems
• Checkpoint/Restart and Live Migration> Provisioning> Grid operations
OpenSolaris on Xen Port• Platform rather than arch port
> Able to leverage most of the Solaris x86 code base> Areas of greatest difference from “i86pc”
> Privileged CPU operations turned into hypercalls> Low-level MMU, segmentation, exceptions> Xen “event” model of interrupts
• New virtual device drivers for paravirtualization> net, disk, console
• Dom0 infrastructure and tools• Paravirtualized DomU
Why Solaris Domain 0• Observability, debugging tools• ZFS• FMA• Containers and TX• CrossBow (virtualized network support)• HW support
OpenSolaris on Xen Status• OpenSolaris domU and dom0
> 32/64-bit, UP, MP (virtual 32-way!)> Virtual disks, network, bridge> CPU and Memory Hot plug support
• Currently available:> OpenSolaris build 75 and onwards> Xen 3.1> PV drivers for Solaris and Windows
Sun xVM
Complete Virtualization and Management Solution
Sun xVM Server• Hypervisor family• Consolidates Windows,
Linux and Solaris• Implementations for x86
and SPARC
Sun xVM Ops Center• Physical and virtual
resource management• Manage thousands
of hardware andsoftware entities
VirtualBox• Full x86 PC virtualization• Free and open source• Easy to Use• Host OS: Solaris, Windows, Linux, Mac (beta)• Guest OS: Solaris, Windows, Most Linux, *BSD,
DOS, OS/2, others• Develop on VirtualBox, deploy on xVM Server• http://virtualbox.org
Join Us...• Our communities and projects are open on
OpenSolaris.org:> Zones: http://opensolaris.org/os/community/zones> BrandZ: http://opensolaris.org/os/community/brandz> Xen: http://opensolaris.org/os/community/xen> CrossBow: http://opensolaris.org/os/project/crossbow
• Where you will find:> Lively discussions, design docs, FAQs, source code
drops, preliminary binary releases, etc...
●Backup Slides
Solaris Zones vs Hypervisors• Zones
> Scalable, fast, virtual platform, platform agnostic> Emphasis on sharing, simpler administration> Improved fault isolation over “single system.”> Alternate brands
• Hardware Virtualization> Emphasis on separation> Fault isolation, (Xen: SPOFs remain)> Live Migration> Foreign OSes
Zones and Resource Management• RM configured within zonecfg
> New 'dedicated-cpu' and 'capped-memory' resources> All RM configuration performed when zone boots> RM configuration migrates with the zone
• Temporary Pools• rcapd can run in global zone and cap zones
> Improved RSS accounting• New zone.max-swap rctl• Simplified rctl syntax within zonecfg• Persistent RM configuration for global zone
●Network Virtualization
The Need for Network Virtualization• ISP offering web and e-mail services
> Consolidate multiple hosts on a single machine> Users expect minimal performance level per virtual host
• Financial services> Consolidate multiple services on a single machine> Some services have minimum performance
requirements, or higher priority
Crossbow• Building blocks for network virtualization and
resource control• Virtualizes: stack, services, protocols or virtual
machine.• Each virtual stack can be assigned its own priority
and bandwidth.• Built into the architecture• Better defense against denial-of-service attacks
Functional Components
• Virtual Network Interface Cards (VNICs)• Flow Management• Hardware Support of flow processing• Adminstration model
> dladm(1M)> flowadm(1M)
CrossBow Virtual NICs
• Carve up 1Gb/s and 10Gb/s hardware NIC into multiple virtual NICs
• Implemented as a Nemo/GLDv3 MAC driver.• Assign NIC hardware resources (interrupts, rings,
etc) to virtual NICs• Rely on hardware-based flow classification to steer
traffic to VNICs and maximize performance• Assign VNICs to Zones or Xen domains
CrossBow Virtual NICs Example
Zone 1VirtualSqueue
Zone 'n'VirtualSqueue
Zone 2VirtualSqueue . . .
Zone 2 Virtual SQUEUE
All Traffic
Compute Resources
NIC
Zone 1HTTPRing
Zone 2All TrafficRing
Zone 1HTTPSRing
Flow Classifier.. .
Zone 1Default Ring .. . .
Zone 1 Virtual SQUEUE
HTTPSqueue
HTTPSSqueue
Default Squeue.. .
VNIC1 VNIC2
CrossBow Virtual NICs for XenSolaris Guest OS 2
Guest OS 2 Virtual SQUEUE
All Traffic
Guest OS 2 VNIC
Solaris Guest OS 1
Guest 1 Virtual SQUEUE
HTTPSqueue
HTTPSSqueue
Default Squeue.. .
Solaris Guest OS 2 VNIC
NIC
HOST OSAll traffic
Ring
Guest OS 2All Traffic
Ring
Guest OS 1HTTPSRing
Flow Classifier.. .
Guest OS 1Default
Ring .. . .Guest OS 1
HTTPRing.. . . .
Solaris Host OS
Host OS Virtual SQUEUE
All Traffic
Host OS VNIC
NICVirtualization
Engine
NICVirtualization
Engine
NICVirtualization
Engine
Stack Instances for Zones and VNICs
Specific To
Containers
Common To AllZones
Zone 1GlobalZone
SharedStack with
Global Zone
GlobalZone
Squeue
.. .VirtualNIC
VirtualNIC
NIC
Global ZoneRing
Zone 1Ring
Flow Classifier.. .
Zone nRing
VirtualNIC
SharedNetwork
Stack
Zone 1Squeue
Zone 2
Exclusive Network
Stack
Zone 2Squeue
NetworkStack
CrossBow Status• Available on OpenSolaris:
> Core VNIC functionality> Bandwidth Control for TCP
• Available in Nevada (SX*) and s10u?:> Stack Instances