copyright © 2008 vmware, inc. all rights reserved. computing on a distributed, virtualized...

Copyright © 2008 VMware, Inc. All rights reserved.

Computing on a Distributed, Virtualized Infrastructure

Beng-Hong Lim

Senior Director of R&D

VMware

CERCS Distinguished Lecture and KeynoteGeorgia Tech, October 17, 2008


Outline

Why virtualize?

How virtualization helps

Challenges and opportunities

Recap and speculate


Why virtualize?

Common reason: share limited resources

past: expensive hardware

Today: hardware is cheap, but …

space, power, cooling


BEFORE AFTER

Servers 1,000 80

Storage 270 TB DAS 140 TB SAN & NAS

Network 3,000 cables/ports 300 cables/ports

Facilities200 server racks

400 power whips

10 server racks

20 power whips

Customer Example: Leading North American Utility

Server consolidation


Virtual Machines, Real Savings

80% reduction in space, power and cooling

$8 million saved over 2 years

Server consolidation


Server consolidation is now mainstream

“Virtualization makes one computer act like many”- popular press tagline


Beyond server consolidation

Management complexity

hardware: distributed, heterogeneous

software: distributed, complex

Distributed infrastructure management:the next killer app for virtualization

“Why is your part taking so long?


BEFORE AFTER

Servers 1,000 80

Storage 270 TB DAS 140 TB SAN & NAS

Network 3,000 cables/ports 300 cables/ports

Facilities200 server racks

400 power whips

10 server racks

20 power whips

Customer Example: Leading North American Utility



Operational efficiency

server and app load from 40 hrs to 30 mins

10,000 person-hours saved per year



Workloads per Virtual Infrastructure Admin

50 workloads per admin for physical infrastructure admin

600

350

300

200

90

A Pharmaceutical Company

A Medical Center

An Investment Bank

A Large Consumer Bank

A Foreign Investment Bank


Distributed Infrastructure Management

To better manage modern hardware and software

Virtualization:

liberates software from hardware

encapsulates software within versatile virtual execution environments

automates management of hardware as a giant pool of resources


Liberating software from hardware

App-1

OS/1

HW-1

App-2

OS/2

HW-2


Liberating software from hardware

App-1

OS/1

App-2

OS/2

VMM

HW-1

VMM

HW-2

Virtual Hardware Abstraction


VMM VMM

Software as a highly-manageable object

Snapshot

Fork

Migrate

Record

Replay/Rewind

etc.

Log

Dis

kS

tora

ge

Ph

ysic

alM

ach

ines

Vir

tual

Mac

hin

es

Virtualization provides new ways to manipulate software


Distributed Virtualization Layer

Exchange Server

Hardware as a distributed pool of resources

Allocate resources and capabilities on-demand to applications

Utilize redundant hardware for reliability and availability

CPU 2 x 1GHz

Memory 4 GB

Disk 500 GB

Network 1 Gbps

Fault Tolerant Yes

Disaster Recovery Enabled

Security High

Virtualization makes many computers act like one


Distributed Infrastructure Management

To better manage modern hardware and software

How do we achieve this?


Distributed hardware and software trend

High-speed connectivity

Web-based applications


Server Management Virtual machines are hardware independent

VMotion (Live Migration)

Pre-copy memory

Virtual disks on networked storage

Preserve VM’s identity: name, MAC address, IP address


Automatic load balancing across machines

Distributed ResourceScheduling (DRS)

Dynamic BalancingContinuous Optimization


Adding and removing resources

Hot-plug machines

Add/remove capacity on demandImprove application availabilityDistributed power management

VMware VirtualCenter

X


Power Savings with DPM

50% Savings During VMmark™

0

200

400

600

800

1000

1200

1400

1600

1800

2000

Wat

ts

No DPM

With DPM

5 hour VMmark run

110 VMs, 4 server cluster


Storage Management

Optimize disk usage, availability and management

Storage resource pool

Storage VMotion: live storage migration

Thin provisioning


Storage VMotion

Migrate running VM to new storage

VM stays on same host

Virtual disks may be individually placed

Storage type independent

Migration does not disturb VM

No downtime

Transparent to guest OS and apps

Minimal performance impact


Storage VMotion Benefits

Retire or migrate between arrays

Arrays coming off maintenance/leasing cycles

Storage tiering

Migrate from FC to iSCSI, NAS or within or between enclosure(s)

Eliminate performance bottlenecks

Load balance through LUN reconfiguration

Seamlessly add and begin using new LUNs

Non-disruptive VM file system upgrade

Future proofing disk format


Thin Provisioning: Smart Allocation

VirtualDisks

PhysicalStorage

App

OS

App

OS

App

OS

VMware ESX

20GB 40GB 100GB

5GB20GB

40GB

5GB 40GB20 GB


Software Management


Dev

QA

Release Management

CCR

Operators

Ops

1 2 3Developers and QA

Traditional Software Lifecycle

develop test stage deploy manage integrate

(Change, Configuration and Release Management)

(Operations, Optimization and Support)

Different environments at each stage, hard to bridge


Virtualization eases software development

VMware Workstation

Indispensible software development tool

Developer-oriented features

VM teams to model distributed hardware

Multiple snapshots

Scripted control of VM operations

Record/Replay

VAssert

VProbes (DTrace for VMs)

A rich set of tools not available in physical environments

.5.5


Software Deployment and ManagementProcess Process Process

App Stack A App Stack B

DB2 9

App Stack C

Different application stacks have different configurations and requirements

http://images.google.com/imgres?imgurl=http://esw-webhosting.co.uk/images/win_server2003_logo.jpg&imgrefurl=http://esw-webhosting.co.uk/Windows-Hosting.asp&h=65&w=194&sz=17&hl=en&start=6&sig2=VK_NuP_6BDCgFKczp1z6sw&um=1&tbnid=sq2Ky0expYbbUM:&tbnh=35&tbnw=103&ei=GHTfRqSPNY7EiQH9i-2oAg&prev=/images%3Fq%3Dwindows%2Bserver%2B2003%2Blogo%26svnum%3D10%26um%3D1%26hl%3Den%26safe%3Doff%26rlz%3D1T4DKUS_enUS212US212

http://images.google.com/imgres?imgurl=http://www.h-net.com/english/pictures/products/oracle.gif&imgrefurl=http://www.h-net.com/english/navigation/services/databases.html&h=287&w=829&sz=10&hl=en&start=1&sig2=GrpoJJ9TPiL8ayYNLDSl6A&um=1&tbnid=qjBG97__udHr6M:&tbnh=50&tbnw=144&ei=JnvfRue6Dp3KiQHU4IGxAg&prev=/images%3Fq%3Doracle%26svnum%3D10%26um%3D1%26hl%3Den%26safe%3Doff%26rlz%3D1T4DKUS_enUS212US212%26sa%3DN


Checkpoint

Windows

Apache

Linux

Linux

Linux

FIREWALL

Tomcat

mySQLCRM

Virtual Appliance

Virtual Appliance

Virtual appliance as unit of deployment

Pre-built, pre-configured and ready-to-run software application packaged with the OS inside a Virtual Machine

Or packaged inside multiple Virtual Machines


Virtual appliance packagingOpen Virtual Machine Format (OVF)

a vendor and platform independent standard for packaging and distributing virtual machines

handles single VMs and multi-tier VM “teams”

OVF 1.0 spec released by DMTF

The OVF descriptor file includes critical infrastructure information

describes virtual hardware configuration and properties

tells management tools and hypervisors what to do with the virtual machines

resource management, security, and availability become properties of the virtual machine, independent of where or how it is deployed

X.Ovf

Security = HighPerformance = 500 msAvailability = 99.99%

Access controlIP port restrictions

Apache

Linux

Linux

LinuxTomcat

mySQL

SLADefinitions

Virtual Appliance


Dev, test, debug with virtual machines

Dev

QA

(Change, Configuration and Release Management)

CCR

(Operations, Optimization and Support)

Ops

develop test stage deploy manage integrate

1 2 3

SHARED VIRTUALINFRASTRUCTURE

Distributed, Multi-site developmentDev/Staging Process Automation

Deploy and Manage Complex ApplicationsAs “Enterprise Virtual Appliances”

Software Lifecycle on a Virtual Infrastructure


Draw the Logical Datacenter and…

VI automatically deploys to physical resources and…

Virtual Infrastructure

InterconnectPool

CPUPool

MemoryPool

StoragePool

dynamically adapts the mappings to optimize efficiency within SLAs and policies


Outline

Why virtualize?

How virtualization helps

Challenges and opportunities

Recap and speculate


Challenge: supporting mobility

Uniform hardware abstraction

CPU: x86 standard

I/O devices: emulated vs. passthrough

Network switch management

port state, configuration and control

long-distance VMotion


Passthrough I/O Devices

I/O MMU

VF VF VF

PF

PF = Physical Function, VF = Virtual Function

I/O Device

Guest OS

Device Driver

Guest OS

Device Driver

Guest OS

Device Driver

VirtualizationLayer

Basic passthrough support

I/O MMU for DMA isolation (Intel VT-d, AMD IOMMU)

Partitionable I/O devices for device sharing (PCI-SIG IOV)

However, we lose

uniform abstraction and VM mobility

Potential solutions:

paravirtualization

guest driver coordination

standardized hardware interfaces


Network Switch Management Network-centric view:

virtualization blurs the host/network boundary

Virtual network switches and topologies within a host

VMs can move anywhere

The virtual network switch is a new access layer.

Network administrators can no longer rely on physical port-based access control

distribution

core

access

Extend switching and management protocols to virtual environments.

hosts

VMs


Distributed Virtual Switches

vSwitch vSwitchvSwitch

Distributed Virtual Switch

Today’s vSwitch

Distributed vSwitch


Migrates port state with VM

ESX host 1 ESX host 3ESX host 2


Opportunity: making virtual better than real

Security

inspect OS activity from a safe platform, e.g., another VM.

Provide APIs for anti-virus software

Overshadow: defending applications from compromised OSes (Chen et al., ASPLOS 2008)

minimize and harden virtualization stack

Fault tolerance

software-based tolerance against hardware failures

run VMs in lockstep (Bressoud and Schneider, SOSP 1995)

Synthesize useful functions in virtual hardware


Virtual Machine Fault Tolerance

X

Lockstep Lockstep

Primary Secondary SecondaryPrimary

X


Virtual Machine Record & Replay

REPLAY

Application

Operating System

Virtualization Layer

Application

Operating System

Virtualization Layer

RECORD

Logging causes of non-determinism• Input (network, user), asynchronous I/O

(disk, devices), CPU timer interrupts

Deterministic delivery of events previously logged

• Result = repeatable VM execution


Determinism

Given the exact same inputs, a processor will deterministically execute the same instruction stream and end up in the exact same state

“Input” is anything outside the CPU/memory that is visible to software:

I/O and interrupts

non-deterministic processor behavior (e.g., CPU timestamp)

Hard to do on physical machines, but doable for virtual machines

Record/replay in VMware Workstation for over a year


Opportunity: client virtualization


Client virtualization

User experience

access to applications and data from any device

with good graphics and interactive performance

on one easy to manage “desktop” environment

Centralized management

OS and application deployment and management

data backup and recovery

recovering from corruption

Thick vs. Thin, Mobile vs. Fixed, Online vs. Offline



A solution: virtual desktop in a VM

all the benefits of software lifecycle management

maintain single master version

Distributed virtual hardware: migrate computation and data to appropriate locations

CPU and memory: client vs. server

Graphics processing: local vs. remote

Virtual disk storage: local vs. networked, replicated, deduplicated


Client virtualization scenario

Compute Graphics Storage

Online, standard graphics




Online, high-speed graphics




Offline



A solution: virtual desktop in a VM

all the benefits of software lifecycle management

Distributed virtual machine hardware: migrate computation and data to appropriate locations

CPU and memory: client vs. server

Graphics processing: local vs. remote

Virtual disk storage: local vs. networked, replicated, deduplicated

No lock-in to a particular configuration


Outline

Why virtualize?

Infrastructure Management

Challenges and Opportunities

Recap and Speculate


Recap

Why virtualize?

server consolidation

distributed infrastructure management



Recap and Speculate


Recap

Why virtualize?


data center management

software lifecycle management


Recap and Speculate


Recap

Why virtualize?



maintaining mobility

making virtual better than real

client virtualization

Recap and Speculate


Speculating about the future

A Virtual, Mobile and Cloudy Future?

in your hand: multi-function mobile devices

in the cloud: multiple data centers

virtual appliances everywhere

Everything more distributed, more mobile


Mobile clients

Proliferation of mobile client devices

notebooks, multi-function handhelds, smart phones

Can a mobile device serve all your computing needs?

limited size, compute power, network bandwidth

one (small) size does not fit all

We’ll be using various types of client devices

user environment and computation typically tied to device

User environment, data and computation should migrate automatically and seamlessly to appropriate hardware


In the data center

Convergence of data center hardware

Traditional roles of data center hardware

Servers host applications

Storage arrays serve and manage storage blocks

Network switches process and route network traffic

Storage arrays and network switches are appliances

In a virtualized world

move storage and network processing into virtual appliances

servers = switches = storage = VM hosts


In the cloud

Improved economics: shared, managed infrastructure

Lightweight entry and exit

Consumption-based pricing

Accessible using standard Internet protocols

Scalable and elastic

How Do We Define The Cloud?


Challenges of Cloud Computing

Need for New, Highly Efficient andFlexible Computing Infrastructure

Multi-tenancy

Application Compatibility

Lack of Standardization CreatesComplexity and Switching Costs


Cloud Services

Virtual Datacenter Off-Premise CloudsImage ManagementUser Accounts

ChargebackMobility

vCloud APIs


PrimaryDatacenter

Cloud Provider

Federation: Connecting the Clouds

Secondary or BranchDatacenter

vApp

vApp


Seamless Cloud Connectivity

vCloud Provider

1

Secondary or BranchDatacenter

3

3 SecurityvApp

PrimaryDatacenter

vApp


A Distributed, Virtualized Infrastructure

Storage Network

Firewall Mail Web

Virtual Infrastructure


In conclusion

Virtualization plays a central role in managing the globally-distributed infrastructure

run applications anywhere

manage software and hardware independently

break down traditional boundaries

Essential properties

uniform, isolated and mobile execution environment

global resource pools

compatibility

Do more with less


Thank You!

copyright © 2008 vmware, inc. all rights reserved. computing on a distributed, virtualized...

Documents

software slide

server cluster slide

distributed hardware

cooling slide

provisioning slide

management of hardware

hardware app

server racks