CertiKOS Implementation Progress

Liang Gu

Yale University

2

Content

• Overview• Virtual Disk• Port to Intel platform• Reduce virtual device code– Virtual device at user mode– Pass through device with IOMMU

• Demo

3

Progress after Boston PI meeting

• Virtualization– Virtual devices: PCI, Virtual Disk ( virtio ,virtio block)– Support Multiple VM guests – Move virtual devices to execute in user mode– IOMMU based device pass-through to guest OS

• Port CertiKOS to work on bare metal • PCI and AHCI

• Port to Intel platform with VT-x Green- doneBlue- almost doneRed- ongoing

4

CertiKOS Architecture

CPU Cores Memory Hard Disk Network . . .Hardware

Hardware Abstraction Layer

SMP Management Memory

Management

Virtual Memory Interrupt

Handling SVM Primitives

Virtualization Abstraction

Process Management

Context IPC

Virtual Machine Management

Vconsole

Memory V-Interrupt V-Devices Hypercall

Master SlaveMaster Syscall Slave Syscall

Mgmt Shell

Mgmt OS(Linux)

Commodity OS

CertiKOS

Application

APP(uncertified)

APP(certified)

Virtual Devices

5

CertiKOS Architecture

CPU Cores Memory Hard Disk Network . . .Hardware

Hardware Abstraction Layer

SMP Management Memory

Management

Virtual Memory Interrupt

Handling SVM Primitives

Virtualization Abstraction

Process Management

Context IPC

Virtual Machine Management

Vconsole

Memory V-Interrupt V-Devices Hypercall

Master SlaveMaster Syscall Slave Syscall

Mgmt Shell

Mgmt OS(Linux)

Commodity OS

CertiKOS

ApplicationVirtual Devices

APP(uncertified)

APP(certified)

Virtual Devices

SVM /VMX Primitives

6

CertiKOS Architecture

CPU Cores Memory Hard Disk Network . . .Hardware

Hardware Abstraction Layer

SMP Management Memory

Management

Virtual Memory Interrupt

Handling SVM Primitives IOMMU

Virtualization Abstraction

Process Management

Context IPC

Virtual Machine Management

Vconsole

Memory V-Interrupt V-Devices Hypercall

Master SlaveMaster Syscall Slave Syscall

Mgmt Shell

Mgmt OS(Linux)

Commodity OS

CertiKOS

ApplicationVirtual Devices

APP(uncertified)

APP(certified)

SVM /VMX Primitives

7

Content

• Overview • Virtual Disk• Port to Intel platform• Reduce virtual device code– Virtual device at user mode– Pass through device with IOMMU

• Demo

8

Virtual Disk• Motivation

– Enable CertiKOS to boot guest OS on bare metal– Separate the storage of guest OS from CertiKOS physically

• Virtual PCI • Virtual disk based on virtio

c. CertiKOS-based APP

d.Mgmt tool in Linux

Linux

b. Legacy OS, e.g., Linux

disk0 disk1

a. Mgmt shell

…

Virtual DiskCertiKOS

9

Virtio• Virtio

– Rusty Russell, “virtio: Towards a De-Facto Standard For Virtual I/O Devices”– Available in both Linux and Windows– A simple and efficient framework to provide virtual devices to guest OS

• Virtio is an abstraction for a set of common virtual devices

b. Legacy OS, e.g., Linux

disk1

Front-end driver

Back-end driver

Disk driver

Virtqueue

…CertiKOS

10

Boot CertiKOS on Bare Metal

• Multiple settings for booting CertiKOS on bare metal– Boot CertiKOS and Guest on the same disk– Boot CertiKOS and Guest on different disks– Boot CertiKOS on USB and boot the guest on disk

11

Content

• Overview • Virtual Disk• Port to Intel platform• Reduce virtual device code– Virtual device at user mode– Pass through device with IOMMU

• Demo

12

Port to Intel platform• Motivation

– Another widely supported Hardware-based Virtualization solution– Widely available VT-d support

• Modularized implementation– Separate architecture dependent modules– Integrated by interfaces in the abstraction layer

• LOCs – Sys/virt/svm 1775– Sys/virt/vmx 2344

• VMX uses more sophisticated methods to control the virtualization– Access memory region for control data structures by special instructions, instead of direct

memory read and write– More sophisticated setup

SVM Primitive

Virtualization Abstraction

Virtual Machine Management

Virtual Devices

VMX Primitive

13

Content

• Overview • Virtual Disk• Port to Intel platform• Reduce virtual device code– Virtual device at user mode– Pass through device with IOMMU

• Demo

14

Virtual Device• LOCs in previous version at Boston PI meeting

– Sys/virt/ 4441*

– Sys/virt/dev/ 2384*

– With Virtual PIC, KBD, PIT, text mode VGA

• LOCs in current clean_code branch– Sys/virt/ 8237*#

– Sys/virt/dev/ 3643*

– Added virtual PCI, Virtio, Virtio-blk

• Considering more devices, such as USB, Network, …

• Moving virtual device to execute in user mode• Securely pass through device with IOMMU

( * counted by cloc 1.56)

( # with Intel vt-x )

15

Virtual Device at User Mode• For untrusted guest domains, their virtual devices don’t have to be trusted• Process model extension

– Multiple processes on a single core based on round-robin scheduling– Message passing via channels among processes

d. Legacy Linux

V-KBD

CPU0

a. Idle

…

CertiKOS

CPU1

V-PIC V-PIT

…

16

Virtual Device at User Mode

• Support multiple VM guests with VM session extension

d. Legacy Linux

CPU0

a. Idle

…

CertiKOS

CPU1

…

CPU2

Guest Linux 2 …

…

VM Session 1 VM Session 2

…

…

17

Pass Through Device

• Exclusively used devices can be directly exposed to guest VM, without introducing device virtualization code

• However, malicious DMA operations are capable of attacking memory spaces

• IOMMU / VT-d– allow a guest OS running under a VMM to have direct control

of a device– Provide fine-grain control of device access to system memory

18

IOMMU

from AMD IOMMU specification Revision 2

19

IOMMU

from http://en.wikipedia.org/wiki/IOMMU

Main memory

CPU

MMUIOMMU

Device

Device Address

Physical Address

Virtual Address

20

IOMMU

Based on image from http://en.wikipedia.org/wiki/IOMMU

CPU

MMUIOMMU

Device

Device Address Virtual Address………

Device Table

Page Table 2

Page Table 1

…

…

Interrupt Remapping Table

21

Pass through device with IOMMU

Legacy OS, e.g., Linux

device

CertiKOS

IOMMU

…

Device Table

Interrupt Remapping Table

NPT

22

Content

• Overview• Virtual Disk• Port to Intel platform• Reduce virtual device code– Virtual device at user mode– Pass through device with IOMMU

• Demo

23

c. CertiKOS-based APP

CertiKOS

b. Legacy OS, e.g., Linux

BSP AP AP

a. Mgmt shell

master slave slave

…

…

CertiKOS Demo SettingFor Previous Version

Qemu

BSP- Boot Strap ProcessorAP-Application Processor

AMD processor with SVM

Linux KVM

24

c. CertiKOS-based APP

CertiKOS

b. Legacy OS, e.g., Linuxa. Mgmt shell

master slave slave

…

CertiKOS Demo Setting

BSP- Boot Strap ProcessorAP-Application Processor

AMD processor with SVM/ Intel with VT-x

25

Thank you!