Review and Intro to KVM

Remember:

Operating System Kernel 1

System Call Interface

Operating System Kernel K

System Call Interface

Hardware (CPU, MEM, IO)

… …

…

Type 1 Hypervisor Operating System Kernel

Process in Hypervisor Kernel Process in Hypervisor Kernel

Remember

• These need to be virtualized:– Memory

– CPU

– IO devices

Operating Systems Manage Hardware

• Which part of operating systems manage these:– Memory – CPU

– IO devices

Highly coupled with OS kernel proper

Somewhat independent from the main part of O.S. kernel, whereby the drivers are more like “plugins” that could be swapped in or out

Component-Wise Virtualization

• Therefore, it is clear that the “kernel proper” can be virtualized independently from the IO device drivers.

• Hereon, call these entities (kernel proper, or IO device drivers) “virtualizable components”

• So, how do we virtualize these components?

Full vs. Para Virtualization

• Full virtualization:– When the component is NOT specially made for

virtualization, but rather is software equal to what that component would look like if it were part of a regular host

• Para virtualization:– When the component IS specially made for

virtualization, that is, it knows about the VM layer

Example:

• Our main Linux host deployment uses a combination of full and paravirtualization:– The guest kernel proper is fully virtualization (via

QEMU/KVM)– The IO drivers are paravirtualized using special

“virtio” device drivers

Comparison• Full virtualization:– Advantages:

• Need no modification of component to run within VM infrastructure

– Disadvantages:• Was historically slower than para (and this is still the case for

IO components)

• Para virtualization– Advantages:

• Coupling between Virtualization layer and component can provide perf benefits (e.g. virtio for IO devices!)

– Disadvantage:• This coupling requires special code in component and

hypervisor

Ways of Fully Virtualizing Kernel Proper

• Binary Translation (historically done this way)– Example: QEMU hypervisor

• Hardware assisted full virtualization– Use special CPU instruction set and features to

achieve extremely high performance!• QEMU+KVM

• How do these really work? Another day.

Paravirtualized Kernel Proper

• The guest kernel is actually modified!• It makes “hypercalls” into the hypervisor– These are somewhat similar to “system calls”

between a process and a kernel (but at the guest kernel – host kernel interface)

• E.g., Xen

Paravirtualized (Two levels of calls)

Hypervisor

Guest Kernel 1 Guest Kernel 2

System calls

Process 1 Process 2 Process 1 Process 2

Hypercalls

Intro KVM

• Kernel Virtual Machine• Exists as 2 kernel loadable modules (i.e. “.ko” files) ,

where one is generic, and the other one is specific to amd vs. intel

• Is not a Hypervisor by itself, but instead “accelerates” QEMU by provide it assistance

• Requires:– Special CPU features– BIOS enabling virt flags

QEMU-KVM Stack

What is LIBVIRT?

• Often confused as part of KVM• But rather, it is a management layer and

simplification layer built on top of a concrete virtualization technology.

• There are drivers for different virtualization technologies, e.g. KVM, LXC, etc. etc.

• A daemon “libvirtd” runs in userspace alongside, for example, qemu processes (the VMs) to help manage our Libvirt-Qemu-KVM stack

Why do we use Libvirt with our QEMU-KVM stack?

• Starting a VM (as a qemu or qemu-kvm process) using the actual qemu is very hard

• Many many command line flags… !• Too many things to configure• E.g., https://linux.die.net/man/1/qemu-kvm• Live look at a box….• Libvirt, adds a configuration layer on top of qemu-

kvm that simplifies the knobs and switches we need to configure

Example of Config Simplification

• On a physical kernel/computer, the layout of:– Sockets– Cores– Logical Cores

• QEMU tries to emulate this an offers 3 settings when you boot a VM:– numSockets– numCoresPerSocket– numLogicalCoresPerSocket

Example cont…• Libvirt takes these 3 QEMU cpu configs and only

exposes a numVCPUs param• It then passes through reasonable best practice

values for the 3 QEMU configurations aforementioned:

• NOTE: It really does not matter what the layout of sockets/cores/lcores in the VM are, because these do not map cleanly to physical buses or caches

Zoomed in (inside guest OS):

lcore lcore

Our Full KVM Stack

• Tools like virt-manager, virt-install, virsh are used to access Libvirt API.

• XMLs define each VM (called “domains” in libvirt”)• QCOW images define the contents of hard disk

• In other words, in our stackDomain XML + QCOW == A virtual machine