Virtualization Virtualization 101101

Virtualization Virtualization 101101

About Me• Eric SiebertEric Siebert• 25 year IT veteran• User Moderator on the VMTN

community forums and hold the elite Guru status as a forum member (esiebert7625)

• Run the website http://vmware-land.com

• Blogger and tip writer for http://searchservervirtualization.com and http://searchvmware.com

• Author of the forthcoming book from Pearson publishing: VI3 Implementation and Administration

Agenda• History of VMware• VMware product family• Virtualization architecture• Hosted product architecture• Bare-metal product architecture• VI3 Features• Virtual Machine anatomy - hardware• Virtual machine anatomy - files

History of VMware• 1998

– VMware founded by Diane Greene, Dr. Mendel Rosenblum and Ed Bugnion

• 1999– Workstation 1.0 released for Windows &

Linux• 2000

– Workstation 2.0 released• 2001

– ESX 1.0 (Elastic Sky X) & GSX 1.0 (Ground Storm X) released

– Workstation 3.0 released

History of VMware• 2004

– 1st VMworld conference in San Diego, CA (1,400 attendees)

– ESX 2.5 released– GSX Server 3.0 released– VMTN technical communities launched– EMC Acquires VMware

• 2005– 2nd VMworld conference (3,500 attendees)– Player 1.0 released– P2V Assistant 2.0 released– Workstation 5.0 & 5.5 released

History of VMware• 2006

– 3rd VMworld conference (6,700 attendees)– VI3 released with ESX 3.0 & VirtualCenter 2.0– Server 1.0 released (replaced GSX Server)

• 2007– 4th VMworld conference (10,800 attendees)– ESX 3i released– Fusion 1.0 for Mac released– Converter 3.0 released (formerly P2V Assistant) – ESX 3.5 & VirtualCenter 2.5 released– Workstation 6.0 released

History of VMware• 2008

– 5th VMworld conference (14,000 attendees)– Fusion 2.0 released– Server 2.0 released– Workstation 6.5 released– ESXi becomes available for free– Paul Maritz takes over as CEO from Diane

Greene• 2009

– Converter 4.0 released– VI4 to be released

VMware Product Family• VMware VI3

– ESX– ESXi

• Enterprise Desktop – View (formerly

VDI)– ThinApp– ACE

• Hosted products– Workstation– Server– Fusion– Player

VMware Product Family• Virtualization Management (vCenter

products)– vCenter Server (formerly VirtualCenter)– Lifecycle Manager– Stage Manager– Lab Manager– Site Recovery Manager– Capacity Planner– Converter– AppSpeed

HypervisorWhat is a hypervisor?• A hypervisor, also called a virtual machine

manager (VMM), is a program that allows multiple operating systems to share a single hardware host. Each operating system appears to have the host's processor, memory, and other resources all to itself. However, the hypervisor is actually controlling the host processor and resources, allocating what is needed to each operating system in turn and making sure that the guest operating systems (called virtual machines) cannot disrupt each other.

Architecture Differences

1. Traditional 1.2. Hosted virtualization 3. Bare-metal virtualization

2. 3.

It's all about Rings• x86 CPUs provide a range of protection levels

also known as rings in which code can execute. Ring 0 has the highest level privilege and is where the operating system kernel normally runs. Code executing in Ring 0 is said to be running in system space, kernel mode or supervisor mode. All other code such as applications running on the operating system operate in less privileged rings, typically Ring 3.

Rings in virtualizationTraditional systems

– Operating system runs in privileged mode in Ring 0 and owns the hardware

– Applications run in Ring 3 with less privileges

Virtualized systems– VMM runs in privileged mode in Ring 0 – Guest OS inside VMs are fooled into thinking

they are running in Ring 0, privileged instructions are trapped and emulated by the VMM

– Newer CPUs (AMD-V/Intel-VT) use a new privilege level called Ring -1 for the VMM to reside allowing for better performance as the VMM no longer needs to fool the Guest OS that it is running in Ring 0.

Hosted ProductsWorkstation/Server/Fusion/Player

– Requires a host operating system (Windows/Linux/Mac), installs like an application

– Virtual machines can use all the hardware resources that the host can see

– Maximum hardware compatibility as the operating system supplies all the hardware device drivers

Hosted Products• Overhead of a full general-purpose

operating system between the virtual machines and the physical hardware results in performance 70-90-% of native

Differences between Workstation and Server

Workstation– Optimized for desktop

OS (better graphics performance, i.e. 3D acceleration)

– Optimized for use for one person sitting in front of the PC running it

– Multiple snapshots and linked clones

– Record/Replay functionality

– $$$

Server– Optimized for I/O and

running server-like loads

– Designed to run headless with a network based admin interface

– Single snapshot capability

– Free

VMware Player• Stripped-down version of Workstation • Intended only for "playing" or running

virtual machines that someone else has made and provides no means for editing or creating them

• Great for running virtual appliances• Free

Bare-metal products• ESX & ESXi install right on the bare metal and

therefore offers higher performance but runs on a narrower range of hardware.

• Used for server consolidation for Data Centers• High performance and scalability• Many advanced features for resource

management, high availability and security• Centralized administration with vCenter Server• Supports more VMs per physical CPU then hosted

products do.• The hypervisor or VMM is referred to as the

VMKernel in ESX & ESXi

ESX & ESXi• Because there is no overhead from a

full host operating system performance is 83-98% of native. There is a small bit of overhead from the virtualization layer of the VMKernel.

Differences between ESX & ESXi

• ESX has a Service Console is based on Red Hat Enterprise Linux 3 (Update 6) that is heavily modified and stripped down and is used for management purposes. During the boot process the Service Console bootstraps the VMKernel using initrd and then turns over full control of all hardware resources to the VMkernel. When the VMkernel takes over the hardware resources of the host, the Service Console is warm booted and managed as a privileged virtual machine within the VMkernel.

• ESXi does not have a full Service Console but instead has a limited management console based on an implementation of the Posix variant of Unix within a Busybox framework and has many features that you will find in the full Service Console.

Differences between ESX & ESXi

ESX• Manage using VI Client, web

client, SSH, RCLI or VIMA• Supports scriptable

installations using utilities like Kickstart

• Supports boot from SAN• Patches are similar to OS

patches and may have dependencies

• Built-in firewall protects the service console and is more complex with over a dozen inbound and outbound connection types allowed by default.

ESXi• Manage using VI Client, RCLI

or VIMA• No support for scriptable

installations• No support for boot from SAN

Any given patch or update is all-inclusive of previous patches and updates

• Built-in firewall is much simpler because there is no service console to protect. Only two connection types are allowed by default.

• Free ESXi cannot be managed by vCenter Server

VI3 features by edition

VI3 FeaturesVMFS

• VMFS is VMware's unique clustering file system which allows for multiple hosts to read and write from the same storage location concurrently.

• It has adaptive block sizing and uses both large block sizes favored by virtual disk I/O and sub-block allocation for small files and directories.

• Uses on-disk disk file locking to ensure that the same virtual machine is not powered on by multiple servers at the same time.

VI3 FeaturesvSMP

• The virtual SMP (vSMP) feature allows you to assign more than one virtual CPU to a virtual machine. Up to 4 virtual CPUs can be assigned to any virtual machines.

• Just because you can assign a virtual machine more than one vCPU doesn’t mean you always should. The reason for this is the hypervisor’s CPU scheduler must find simultaneous cores available equal to the number assigned to the VM.

• ESX 3.x introduced a new “relaxed” co-scheduling scheme which provides more flexibility and increases the number of scheduling opportunities that are available to the scheduler and thereby increasing overall system throughput. Regardless of this you should still take care when creating multiple vCPU virtual machines.

VI3 FeaturesVMotion

• VMotion allows you to quickly move an entire running virtual machine from one host to another without any downtime or interruption to the virtual machine This is also known as a “hot” or “live” migration.

• The entire state of a virtual machine is encapsulated and the VMFS file system allows both the source and the target ESX host to access the virtual machine files concurrently. The active memory and precise execution state of a virtual machine can then be rapidly transmitted over a high speed network. The virtual machine retains its network identity and connections, ensuring a seamless migration process.

VI3 FeaturesVMotion - How It Works

1. Migration request is made to move the virtual machine from ESX1 to ESX2.

2. vCenter Server verifies that the virtual machine is in a stable state on ESX1 and checks the compatibility of ESX2 (CPU, networking, etc.) to ensure that it matches that of ESX1.

3. The virtual machine is registered on ESX2.

4. The virtual machine state information (including memory, registers and network connections) is copied to ESX2. Additional changes are copied to a memory bitmap on ESX1.

5. The virtual machine is quiesced on ESX1 and the memory bitmap is copied to ESX2.

6. The virtual machine is started on ESX2 and all requests for the virtual machine are now directed to ESX2.

7. A final copy of the virtual machines memory is done from ESX1 to ESX2.

8. The virtual machine is un-registered from ESX1.

9. The virtual machine resumes operation on ESX2.

VI3 FeaturesStorage VMotion

• Storage VMotion is a new feature introduced in ESX 3.5, it allows you to migrate a running virtual machine and its disk files from one datastore to another on the same ESX host

• The difference between VMotion and Storage VMotion is that VMotion simply moves a virtual machine from one ESX host to another but keeps the storage location of the VM the same, Storage VMotion on the other hand changes the storage location of the virtual machine while it is running and moves it to another datastore on the same ESX host. The virtual machine can be moved to any datastore on the ESX host which includes local and shared storage.

VI3 FeaturesStorage VMotion - How It Works

1. New virtual machine directory is created on the target datastore, virtual machine configuration files and all non-virtual disk files are copied to the target directory.

2. ESX host does a “self” VMotion to the target directory.

3. A snapshot (without memory) is taken of the virtual machines disks in the source directory.

4. Virtual machine disk files are copied to the target directory.

5. Snapshot that is located in the source directory is consolidated into the virtual machine disk files located in the target directory.

6. Source disk files and directory are deleted.

VI3 FeaturesHigh Availability (HA)

• Continuously monitors all hosts in a cluster and restarts virtual machines affected by a host failure on other hosts

• Can also monitor guest OS's for a failure via a heartbeat and restart them on the same host in case of a failure

• Continuously monitors and chooses the optimal physical servers within a resource pool on which to restart virtual machines (if used in conjunction with DRS)

VI3 FeaturesDistributed Resource Scheduler

(DRS)

• DRS enables your virtual environment to automatically balance itself across your host servers in an effort to eliminate resource contention. It utilizes the VMotion feature to provide automated resource optimization and automatic migration of virtual machines across hosts in a cluster.

• Distributed Power Management (DPM) (experimental) can consolidate workloads and power off hosts during periods of low activity. When activity increases DPM brings hosts back online so service levels can be met.

VI3 FeaturesUpdate Manager

• Update Manager is a new feature introduced in ESX 3.5 that provides automated patching of ESX & ESXi hosts and select Microsoft Windows and Linux virtual machine operating systems and applications.

• Update Manager can scan hosts and certain guest operating systems and compare them to a baseline and then apply updates and patches to them.

• Update Manager is integrated with DRS so hosts can be patched without effecting virtual machines (as long as the virtual machines are on shared storage).

VI3 FeaturesConsolidated Backup

• VMware Consolidated Backup (VCB) is a Windows based application that provides a centralized backup facility to backup virtual machines through a proxy server without affecting the virtual machine itself.

• VCB is an alternative to traditional agent based backup methods and is an enablement technology; it cannot backup virtual machines by itself but instead works with 3rd party backup products to help offload backup overhead from virtual machines and host servers.

Virtual MachinesSo what exactly is a virtual machine? • A virtual machine is defined as a

representation of a physical machine by software that has its own set of virtual hardware upon which an operating system and applications can be loaded. With virtualization each virtual machine is provided with consistent virtual hardware regardless of the underlying physical hardware that the host server is running. When you create a VM a default set of virtual hardware is given to it. You can further customize a VM by adding or removing additional virtual hardware as needed by editing its configuration.

Virtual MachinesVirtual machines provide:

– Hardware independence – VM sees the same hardware regardless of the host hardware

– Isolation – VM’s operating system is isolated from the host operating system

– Encapsulation – Entire VM encapsulated into a single file

Virtual Machine HardwareSo a VM has virtual hardware but what kind of

hardware is presented to it by the host server? • System Manufacturer: VMware• BIOS: Phoenix 6.0• Motherboard: Intel 440BX • CD-ROM: NEC VMware IDE CDR00• Processor: This will vary based on the processor in the

host server, AMD hosts will present an AMD processor(s) to a VM of the actual type in the host server and Intel hosts will present an Intel processor(s) to a VM of the actual type in the host server. A VM will only see the amount of processors that is assigned to it regardless of the amount the host has. Also all processors presented to VMs are single-core processors even if the host has multi-core processors.

Virtual Machine Hardware• Memory: 4 memory slots that can be populated with

memory of the speed and type (not size) of the memory in the host server. This is not configurable and happens automatically. For example a VM with 512MB of memory will typically see one slot configured with a 512MB DIMM and a VM with 4GB of memory will typically see two slots configured with 2048MB DIMMs.

• Network Controller (NIC): This will depend on the operating system that you choose when configuring the VM. The most commonly used NIC in most 32-bit VMs is based on the AMD PCnet physical NIC and is used with the flexible or vlance adapter types. Additional NICs include ones based on Intel’s e1000 (64-bit VMs and 32-bit Vista VMs) and also VMware’s own vmxnet (no physical counterpart, used in ESX 2.x).

Virtual Machine Hardware• IDE Controller: Intel 82371 AB/EB PCI Bus Master IDE

Controller• Video controller: VMware Standard VGA Graphics Adapter

with 4MB video memory• SCSI Controller: Can be either an LSI Logic PCI-X Ultra320

or a Buslogic BA80c30 PCI-SCSI MultiMaster depending on the operating system chosen when creating a VM. LSI Logic is the preferred choice and offers slightly better performance on some workloads. The Buslogic is typically used by older operating systems; you can manually select the SCSI controller type if you choose the Custom wizard type instead of Typical when creating a VM. You will notice that you cannot add a SCSI controller to a VM, controllers are automatically added or removed when you add hard disks and assign them a virtual device node ID.

Virtual Machine Hardware• Note that USB and audio devices (sound cards) are not

supported at all on ESX hosts. The total IDE devices in a VM can not exceed 4 which is the limit of CD/DVD drives that you can add to a VM. Also the total number of PCI devices in a VM can not exceed 6, included in this total are NICs, SCSI controllers and the Video adapter. Since you can’t remove the Video adapter from a VM that leaves 5 PCI devices to be divided up between NICs and SCSI controllers.

Virtual Machine Files• A virtual machine is comprised of

a number of files that are located in it's home directory.

• If you take a look at a VMs home directory on an ESX host using a file browser application like WinSCP or the Datastore Browser that is built into the VI Client you will see a list of files that are associated with the VM. Most of the files start with the actual name of the VM and have different file extensions based on the type of file that it is.

Virtual Machine Files• You may not see all of the possible file types until your VM is

in a certain state; for example the .vswp file is only present when the VM is powered on and the .vmss file is only present when a VM is suspended. Below is a typical VM directory listing using WinSCP. Note the VI Client's datastore browser will combine the listing of each of the 2 files that make up a virtual disk into one file.

Virtual Machine Files.nvram file - This small file contains the Phoenix BIOS

that is used as part of the boot process of the virtual machine. Similar to a physical server that has a BIOS chip that let’s you set hardware configuration options; a VM also has a virtual BIOS that is contained in the NVRAM file. The BIOS can be accessed by accessing when a VM first starts up by pressing the F2 key, whatever changes are made to the hardware configuration of the VM are then saved in the NVRAM file. This file is in binary format and if deleted it will be automatically re-created when a VM is powered on.

Virtual Machine Files.vmx file – This file contains all of the configuration

information and hardware settings of the virtual machine. Whenever you edit the settings of a virtual machine all of that information is stored in text format in this file. This file can contain a wide variety of information about the VM including things like its specific hardware configuration (i.e. RAM size, NIC info, hard drive info and serial/parallel port info) advanced power and resource settings, VMware tools options and power management options. While you can edit this file directly to make changes to a VM’s configuration it is not recommended that you do so unless you know what you are doing. If you do make changes directly to this file it’s a very good idea to make a backup copy of this file first.

Virtual Machine Files• vmdk files – All virtual disks are made up

of two files, a large data file equal to the size of the virtual disk and a small text disk descriptor file which describes the size and geometry of the virtual disk file. The descriptor file also contains a pointer to the large data file as well as information on the virtual disks drive sectors, heads, cylinders and disk adapter type. In most cases these files will have the same name as the data file that it is associated with (i.e. myvm1.vmdk and myvm1-flat.vmdk). You can match the descriptor file to the data file by checking the Extent Description field in this file to see which –flat, -rdm or –delta file is linked to it.

Virtual Machine FilesThere are three different types of

virtual disk data (vmdk) files that can be used with virtual machines:

• –flat.vmdk file - This is the default large virtual disk data file that is created when you add a virtual hard drive to your VM that is not a RDM. When using thick disks this file will be approximately the same size as what you specify when you create your virtual hard drive. One of these files is created for each virtual hard drive that a VM has configured as shown in the examples.

Virtual Machine Files

–delta.vmdk file - These virtual disk data files are only used when snapshots are created of a virtual machine. When a snapshot is created all writes to the original –flat.vmdk are halted and it becomes read-only; changes to the virtual disk are then written to these –delta files instead. The initial size of these files is 16MB and they are grown as needed in 16MB increments as changes are made to the VM’s virtual hard disk. Because these files are a bitmap of the changes made to a virtual disk a single –delta.vmdk file cannot exceed the size of the original –flat.vmdk file.

Virtual Machine Files A delta file will be created for each snapshot that you

create for a VM and their filenames will be incremented numerically (i.e. myvm-000001-delta.vmdk, myvm-000002-delta.vmdk). These files are automatically deleted when the snapshot is deleted after they are merged back into the original –flat.vmdk file.

Virtual Machine Files -rdm.vmdk file – This is the mapping file for the RDM

that manages mapping data for the RDM device. The mapping file is presented to the ESX host as an ordinary disk file, available for the usual file system operations. However to the virtual machine the storage virtualization layer presents the mapped device as a virtual SCSI device. The metadata in the mapping file includes the location of the mapped device (name resolution) and the locking state of the mapped device. If you do a directory listing you will see that these files will appear to be taking up the same amount of disk space on the VMFS volume as the actual size of the LUN that it is mapped to but in reality they just appear that way and their size is very small. One of these files is created for each RDM that is created on a VM.

Virtual Machine Files• When you create an RDM for a VM you can choose one

of two modes that it can operate in. The first mode is virtual compatibility mode which virtualizes the mapped device and is mostly transparent to the guest operating system. The other mode is physical compatibility mode which provides minimal SCSI virtualization of the mapped device and the VMkernel passes most SCSI commands directly to the device which allows for closer integration between the VM and the LUN.

Virtual Machine Files• .vswp file - When you power on a VM a memory swap file is

created that can be used in lieu of physical host memory if a ESX host exhausts all of its physical memory because it is overcommitted. These files are created equal in size to the amount of memory assigned to a VM minus any memory reservations (default is 0) that a VM may have set on it (i.e. a 4GB VM with a 1GB reservation will have a 3GB vswp file created). These files are always created for virtual machines but only used if a host exhausts all of its physical memory. As virtual machine memory that is read/written to disk is not as fast as physical host RAM your VM’s will have degraded performance if they do start using this file. These files can take up quite a large amount of disk space on your VMFS volumes so ensure that you have adequate space available for them as a VM will not power on if there is not enough room to create this file. These files are deleted when a VM is powered off or suspended.

Virtual Machine Files• .vmss file - This file is used when virtual machines are

suspended and is used to preserve the memory contents of the VM so it can start up again where it left off. This file will be approximately the same size as the amount of RAM that is assigned to a VM (even empty memory contents are written). When a VM is brought out of a suspend state the contents of this file are written back into the physical memory of a host server however the file is not automatically deleted until a VM is powered off (an OS reboot won’t work). If a previous suspend file exists when a VM is suspended again this file is re-used instead of deleted and re-created. If this file is deleted while the VM is suspended then the VM will start normally and not from a suspended state.

Virtual Machine Files• .vmsd file - This file is used with snapshots to store

metadata and other information about each snapshot that is active on a VM. This text file is initially 0 bytes in size until a snapshot is created and is updated with information every time snapshots are created or deleted. Only one of these files exists regardless of the number of snapshots running as they all update this single file. The snapshot information in this file consists of the name of the vmdk & vmsn file used by each snapshot, the display name and description and the UID of the snapshot. Once your snapshots are all deleted this file retains old snapshot information but increments the snapshot uid to be used with new snapshots. It also renames the first snapshot to “Consolidate Helper” presumably to be used with Consolidated Backups.

Virtual Machine Files• .vmsn file - This file is used with snapshots to store the

state of a virtual machine when a snapshot is taken. A separate .vmsn file is created for every snapshot that is created on a VM and is automatically deleted when the snapshot is deleted. The size of this file will vary based on whether or not you choose to include the VM’s memory state with your snapshot. If you do choose to store the memory state this file will be slightly larger then the amount of RAM that has been assigned to the VM as the entire memory contents including empty memory is copied to this file. If you do not choose to store the memory state of the snapshot then this file will be fairly small (under 32KB). This file is similar in nature to the .vmss that is used when VM’s are suspended.

Virtual Machine Files• .log file – These are the files that are created to log

information about the virtual machine and are often times used for troubleshooting purposes. There will be a number of these files present in a VM’s directory; the current log file is always named vmware.log and up to 6 older log files will also be retained with a number at the end of their name (i.e. vmware-2.log). A new log file is created either when a VM is powered off and back on or if the log file reaches the maximum defined size limit. The amount of log files that are retained and the maximum size limits are both defined as VM advanced configuration parameters (log.rotateSize & log.keepOld).

Virtual Machine Files• .vmxf file - This file is a supplemental configuration file

that is not used with ESX but is retained for compatibility purposes with Workstation. It is in text format and is used by Workstation for VM teaming where multiple VMs can be assigned to a team so they can be powered on/off or suspended and resumed as a single object.

That's all folks