Everything goes virtual

Leveraging virtualization Techniques at every Level of an Enterprise Solution, from Storage and Network to Servers

and Software

Monday, 10-Nov-08 Heinz-Hermann Adam (adamh@uni-muenster.de)

Agenda

•  Who we are and what we do •  What means „virtual“? •  First virtual steps •  Goal •  Storage Virtualization •  Fileservice Virtualization •  Computer Virtualization •  Moving from Physical to Virtual

Who we are

•  WWU Münster is one of the three major universities in Germany –  ~ 40,000 students –  ~ 5,000 scientists and

staff –  Over 100 fields of

study

•  The Natural Sciences Department is ~¼ of the university –  Major user and

provider of compute resources

–  IT is a Volunteer driven operation

•  Not much dedicated staff

What we do

•  Provide and maintain resources for students, scientists and staff in Biology, Chemistry and Physics –  ~ 4,000 Computers –  ~ 12,000 Users

•  Compute resources –  Scientific Computing

•  SMP and Clusters •  Development

Environment

–  Desktop Applications •  Windows •  Linux •  Mac OS

–  File and Print Sharing

What means „virtual“?

•  If you can see and it‘s there –  It‘s real

•  If it‘s there and you can‘t see it –  It‘s transparent

•  If you can see it, but it‘s not there –  It‘s virtual

•  If you can‘t see and it‘s not there –  It‘s gone

First use of virtualization in production

•  License managment –  Products like FlexLM

•  Tied to an IP-Adress (if you are fortunate) •  Mostly tied to a MAC-address

–  Moving a license server (P2P) •  Upto two weeks of service disruption to acquire new license keys

•  VMware GSX-Server –  Virtual machine with a vNIC and a fixed MAC-address

00:50:56:11:11:11 –  Second GSX-Server for redundancy

•  Manual failover for maintenance only •  Parallel port dongles are a real challenge

Requesting more from virtualization

•  So far 1:2 anti-consolidation

•  Additional Services –  2 out of 3 Domain

Controllers (Active Directory)

–  Testbeds •  Software (re)packaging •  Desktop development

•  Snap-Shots for Backup –  Patching –  Configuration changes

•  Performance Concerns –  How powerful is the GSX

hardware •  CPU •  Memory

–  Snap-Shots •  15 Minutes to complete •  VM stops responding,

causing alert –  Failover time GSX-to-GSX

•  About 1 hour –  File copy on the LAN

Initial Situation 2005/2006

•  Replacement of IT Infrastructure in Operation since 1998: Overdue

•  Isolated Data-Silos of Direct Attached Storage –  OpenVMS –  Windows –  Tru64 UNIX –  Linux

•  Changed Focus Platforms

Goal

•  Consolidation –  Versatile Storage System

•  Storage Capacity •  Data Protection •  Reliability, Availabilty, Fault-Tolerance

–  Highly Available Fileservice •  Transparent to client operating systems

–  Unified Computersystem •  Scientific Computing ( HPC) •  Infrastructure Services (Active Directory etc.)

–  Manpower •  Data Pools

Storage system

•  „single“ SAN storage system •  Mirrored between two „independent“

locations •  Holds all server and user storage

–  Data –  No boot-from-SAN

•  Flexible allocation of storage space –  vRAID for vRAID –  vDisk for vDisk –  Gigabyte for Gigabyte

Enterprise Virtual Array

•  2 EVA 8000 –  2 Diskgroups (146 and

300 GB disks) –  Multi-Pathing SAN (2

complete fabrics) –  Continous Access

mirroring between sites –  All disks are on the

EVAs •  Superdome •  Windows Storage Server •  Blade Server

Fileservice

•  Multi-Protocol NAS-Filersystem –  SMB/CIFS + Dfs –  NFS –  AFP –  ...

•  Redundant at two sites –  Virtual servers (Cluster groups)

inside a Microsoft Cluster Service •  4 ProLiant Storage Server DL 380

–  Windows Storage Server 2003 R2 clustered

–  GroupLogic ExtremeZ-IP –  StorageWorks Cluster Extension

EVA

Unified computersystem

•  Proven Concepts –  Few, identical systems –  Serving both basic needs and scientific

computing •  Partitionable SMP-System

–  Running multiple operating systems at a time –  Windows, Linux, ...

•  Redundant at two sites •  Extensible •  Not feasible

–  Split up between •  Scientific Computing: Itanium2

•  Infrastructure Services: x86

Scientific Computing

•  Scientific Computing High Performance Computing –  SMP-System > 4 CPU –  Shared-memory (ccNUMA) –  64-bit architecture

•  Make a difference to PC clusters running Linux or Windows

•  Integrity Superdome –  4 Hardpartitions

•  1 Windows •  3 Linux

–  24 Itanium2 Montecito •  Dual-Core •  Hyperthreading

–  160 GB RAM –  Per Partition I/O

•  Ethernet •  SAN •  SCSI/RAID

Infrastructure Services

•  2 BladeSystem c7000 –  6 ProLiant BL480c

•  2 Xeon 5160 Woodcrest Dual-Core @ 3 GHz

•  8 GB RAM •  6 NICs •  2 FC HBAs

–  8 GbE2c LAN switches •  802.1Q VLAN tagging

–  4 Brocade 4/12 SAN switches for HP c-Class

•  Switched Fabric

•  VMware Virtual Infrastructure 3 Enterprise

12

Ways to become virtual

•  Virtual to virtual (V2V) –  Copy and import existing GSX Server VMs

•  Physical to virtual (P2V) –  Transfer running ProLiant installations into VMs

•  Setup from scratch –  Linux systems –  Systems comprising several services moving to

separate systems

Virtual to virtual migration

•  VMware GSX or Workstation as a basis –  Create directory on Datastore

•  [root@esx vmfs1]# mkdir ESX-VM –  Copy virtual disks to the ESX server

•  /vmimages/GSX-VM –  SMB –  SCP

–  Convert VMDK file to ESX format •  [root@esx vmfs1] vmkfstools –i /vmimages/GSX-VM/GSX-VM.vmdk /vmfs/

volumes/vmfs1/ESX-VM/ESX-VM.vmdk –  Create a new VM with same hardware configuration as on the GSX

•  Use converted VMDK a virtual hard disk –  Boot VM

•  Update VMware tools –  Delete VMDK at /vmimages

Physical to virtual migration

•  2006: VMware P2V Assistant (Windows only) –  Create a VM inside the ESX

•  same disk layout as the ProLiant –  Mount the created VMDKs to a P2Vhelper VM

•  Running Norton Ghost console and server –  Clone the contents of the ProLiant Disks to the VM over LAN

•  BartPE •  Norton Ghost Console

–  Run the P2V Asistant to change boot controller on the copied system disk •  2008: VMware Converter

–  Install on the ProLiant –  Create the VMDK while the server is running –  Transfer it to the ESX –  Boot up virtual Machine

•  Shutdown ProLiant first

•  HP ProLiant Essentials Server Migration Pack

Setup from scratch

•  Only necessary for Linux servers –  P2V Assistant does not support Linux –  Only experimental support in VMware Converter (2006-2008)

•  Or if you want to reassign tasks from one server to many •  VM templates save time

–  Windows Server 2003 Basis •  Installed •  Patched •  Virus protected

–  SuSE Linux Basis •  Installed •  Patched •  Bound to the Active Directory

Deploying Templates

•  Windows –  Automatically runs sysprep

•  Linux –  Just creates a copy

•  Adjust MAC address –  /vmfs/volumes/vmfs1/<vmname>/<vmname>.vmx

•  ethernet0.addressType = „vpx“ •  ethernet0.generatedAddress = „00:50:56:be:2a:37“

–  Change to •  ethernet0.addressType = „static“ •  ethernet0.address = „00:50:56:00:00:02“

–  Rather use VirtualMACTool than fiddle with the vmx files

VI 3 Enterprise Features we use

•  Virtual SMP –  Windows Terminal Server

•  Was a Dual-Xeon DL360 before –  Use with caution

•  Now using NLB Cluster with 1 CPU VMs

•  Virtual Center –  Single point of administration for all ESX servers –  Can be a VM itself –  Rather use a fully featured SQL server than MSDE/SQL Express

•  High Availability (HA) –  Making a single cluster spanning all ESX servers and sites –  Adjust node restrictions

•  otherwise no VMs can be powered on if one site fails

VI 3 Enterprise Features we use

•  Dynamic Resource Scheduling –  Only one rule: Keep same service on different VMs from running

on the same ESX host •  VMotion

–  30 seconds to move a running VM from one site to another –  Without interuption

•  Snap-Shots –  Multiple per VM

•  As long as you don‘t use VMBK to backup your VMs as a whole –  Takes less than 10 seconds

•  What we don‘t use –  VMware Consolidated Backup

•  Does not work in a multi-pathed SAN

What do we gain?

•  Consolidation –  Less hardware, better utilization

•  Availability –  Hardware „independence“ of services –  HA clustering

•  Provisioning –  Setup new systems is a matter of minutes

•  Management –  Less hardware to maintain

•  Heterogeneity –  Fewer hardware, but still different OS for different tasks

What do we face?

•  Anti-consolidation –  Virtual sprawl –  VM per task for application isolation

•  Steep learning curve –  More complex hardware –  Complex software

Status of Installation

Our Way to Data Pools

•  Versatile Storage System –  1 GB units

•  All Servers connected to the SAN

•  NAS-Cluster for Filesharing •  Partitionable SMP Shared

Memory System –  Itanium2

–  2-24 CPU •  Bladesystem

–  X86-64 –  VMware Virtual Infrastructure

Q&A – Questions? Please!