Virtualizing I/OYiying Zhang

Announcements

• Quiz next Tue: cover everything until (including) today’s lecture

• Project proposal due on 1/29

• Decide project topics (and update Google sheet) before tomorrow

• Sign up for discussion lead

Outline

• Device virtualization techniques

• Storage virtualization

• Network virtualization

Acknowledgment: some slides from Scott Devine’s lectures at Columbia

Lec2: I/O Virtualization• Goal

• Multiplexing device across guest VMs

• Challenges

• Each guest OS has its own device driver

• How can one device be controlled by multiple drivers?

• What if one guest OS tries to format its disk?

Lec2: Possible Solutions of I/O Virtualization

• Direct access: VM exclusively owns a device

• Device emulation: VMM emulates device in software

• Para-virtualization: split driver into guest part and host part

• Hardware assisted: hardware devices offer isolated “virtual interfaces”

CPU

MMU

Memory

Controller

LocalBus

Interface

High-SpeedI/OBus

NIC Controller Bridge FrameBuffer

LAN

Low-SpeedI/OBus

USBCD-ROM

VMGuestOS

Sol-1: Direct Access Device Virtualization

IOMMU: Page Tables for Devices• Allow guest OS direct access to underlying device• Guest just reuses its own device driver

• Q: What if a VM asks device to access memory of other VMs? • It is possible because device accesses physical memory via DMA• Thus a device can access any memory

• Solution: Page tables for devices• Another MMU: IOMMU for devices• Q: what addresses will IOMMU translate?

IOMMU for Direct Access

• VT-d architecture defines a multi-level page-table structure for DMA address translation

• Similar to IA-32 processor page-tables, enabling software to manage memory at 4 KB or larger page granularity

Direct Access Device Virtualization

• Positives• Fast, since the VM uses device just as native machine• Simplify monitor: limited device drivers needed

• Negatives• Hardware interface visible to guest (bad for migration)• Interposition is hard by definition (no way to trap & emulate)• Now you need much more devices! (imagine 100 VMs)

Sol-2: Emulating Devices

• Emulate a device • Implement device logic in pure software• Can even emulate non-existing devices

• For example• VMware emulates a 8139 network card for VMs• Since 8139 is widely used, almost every OS has its driver• So no need to write a new driver

VMware Workstation

HostOS

Example: Emulating Serial Port

Monitor

SerialChipXYZ

LAN

DeviceEmulator Guest

SerialChipABC Driver

SerialChipABC

Emulation

GenericSerialLayer

Emulated Devices

• Positives• Platform stability (good for migration)• Allows interposition• No special hardware support is needed

• Negatives• Can be slow (it’s software emulated)

Sol-3: Para-Virtualized Devices• VMM offers new types of device• The guest OS will run a new driver (front-end driver)• The VMM will run a back-end driver for each front-end• The VMM will finally run device driver to drive the device

VirtIO Architecture by KVM

Sol-4: Hardware Support for I/O Virtualization

• A virtualization-enabled device can be configured to appear in the PCI configuration space as multiple virtual functions (VFs)

• The VMM assigns one or more VFs to a VM by mapping the actual configuration space of the VFs to the configuration space presented to the VM by the VMM

Direct Access with Device Virtualization Support

Application

TCP/IP Stack

Device Driver (VF) Device Driver (PF)

Guest Domain Domain-0

Xen

Hardware

User-level Config

VF VF VF VF PFPhysical Device

Application

TCP/IP Stack

Device Driver (VF)

Guest Domain

Outline

• Device virtualization techniques

• Storage virtualization

• Network virtualization

Virtual Disk

• A virtual disk is just a file in host file system

• Hypervisor maps disk blocks to file offsets

• Flat file (fix sized virtual disk)

• Indexed file (virtual disk can grow on demand)

Use Copy-On-Write Disks to Save Space

Use Copy-On-Write Disks to Save Space

Use Copy-On-Write Disks to Save Space

Outline

• Device virtualization techniques

• Storage virtualization

• Network virtualization

Network Virtualization with vnets

vnets in a Hosted Architecture

VMware’s Network Subsystem

Host-Only Network• Creates a network that is completely contained within the host computer• Provides a network connection between the virtual machine and the host system by using

a virtual network adapter in the host OS (hypervisior)

NAT Network• A VM does not have its own IP address on the external network

• A separate private network is set up on the host system (address assigned by virtual DHCP)

• External network communicate with VMs through port forwarding

Bridged Network• Connect a VM to the network to the real physical NIC (via a virtual bridge)

• Each VM gets its own IP address, in the same subnet

Virtual Machine Networking Performance (VMware Workstation)

Reducing Network Virtualization Overheads• Handling I/O ports in the VMM

• Send combining

• IRQ notification (use user-kernel shared memory for receiving notification instead of select syscall)

• Reducing CPU virtualization overhead

• Modifying guest OS (e.g., avoid page table switches when switching to idle task)

• Optimizing the guest driver protocol

• Modifying the host OS

• Bypassing the host OS

Virtualization Technologies SummaryVirtualization Software Solution Hardware Solution

CPU • Trap & Emulate• Instruction interpretation• Binary translation

• VT-x• Root / non-root mode• VMCS

• Para-virtualization

Memory • Shadow page table • EPT

• Para-virtualization

Device • Direct I/O• Device emulation

• IOMMU• SR-IOV

• Para-virtualization