virtualization and securitycsg.csail.mit.edu/6.823/lectures/l23split.pdfvirtualization and security...

L23-1MIT 6.823 Spring 2020

Mengjia YanComputer Science & Artificial Intelligence Lab

M.I.T.

Based on slides from Daniel Sanchez

Virtualization and Security

May 7, 2020

MIT 6.823 Spring 2020

Evolution in Number of Users

May 7, 2020

IBM 16201959

Single User

Runtime loaded with

program

L23-2



May 7, 2020

IBM 16201959

Single User

Runtime loaded with

program

IBM 3601960s

Multiple Users

OS for sharing

resources

L23-3



May 7, 2020

IBM 16201959

Single User

Runtime loaded with

program

IBM 3601960s

Multiple Users

OS for sharing

resources

IBM PC1980s

Single User

OS for sharing

resources

L23-4



May 7, 2020

IBM 16201959

Single User

Runtime loaded with

program

IBM 3601960s

Multiple Users

OS for sharing

resources

IBM PC1980s

Single User

OS for sharing

resources

Cloud Servers1990s

Multiple Users

Multiple OSs

L23-5


Single-Program Machine

• Hardware executes a single program• This program has direct and complete access to all

hardware resources in the machine

May 7, 2020

Program

Hardware

Processor Memory Disk Network card Display

…

Keyboard

L23-6




hardware resources in the machine

May 7, 2020

Program

Hardware


…

Keyboard

ISA

L23-7




hardware resources in the machine• The instruction set architecture (ISA) is the

interface between software and hardware

May 7, 2020

Program

Hardware


…

Keyboard

ISA

L23-8


Operating Systems

• Operating System (OS) goals:– Protection and privacy: Processes cannot access each

other’s data

May 7, 2020

process1

OS Kernel

Hardware

processN…process2

ISA

Application BinaryInterface (ABI)

L23-9


Operating Systems


other’s data– Abstraction: OS hides details of underlying hardware

• e.g., processes open and access files instead of issuing raw commands to the disk

May 7, 2020

process1

OS Kernel

Hardware

processN…process2

ISA


L23-10


Operating Systems


other’s data– Abstraction: OS hides details of underlying hardware

• e.g., processes open and access files instead of issuing raw commands to the disk

– Resource management: OS controls how processes share hardware (CPU, memory, disk, etc.)

May 7, 2020

process1

OS Kernel

Hardware

processN…process2

ISA


L23-11


Operating System Mechanisms

• The OS kernel provides a privateaddress space to each process– Each process is allocated space in

physical memory by the OS– A process is not allowed to access

the memory of other processes

May 7, 2020 L23-12






May 7, 2020

free

Process 2memory

OS Kernelmemory

Process 1memory

Phys

ical

Mem

ory

free

…

L23-13






• The OS kernel schedules processesinto cores– Each process is given a fraction of CPU time– A process cannot use more CPU time than allowed

May 7, 2020

free

Process 2memory

OS Kernelmemory

Process 1memory

Phys

ical

Mem

ory

free

…

L23-14







May 7, 2020

free

Process 2memory

OS Kernelmemory

Process 1memory

Phys

ical

Mem

ory

free

…Runningprocess Process 1 Process 2

Time

Process 1

L23-15







• The OS kernel lets processes invoke system services (e.g., access files or network sockets) via system calls

May 7, 2020

free

Process 2memory

OS Kernelmemory

Process 1memory

Phys

ical

Mem

ory

free

…Runningprocess Process 1 Process 2

Time

Process 1

L23-16


Virtual Machines

• The OS gives a Virtual Machine (VM) to each process– Each process believes it runs on its own machine…– …but this machine does not exist in physical hardware

May 7, 2020 L23-17


Virtual Machines


May 7, 2020

Physical Hardware


…Keyboard

L23-18


Virtual Machines


May 7, 2020

OS Kernel (specially privileged process)

Physical Hardware


…Keyboard

L23-19


Virtual Machines


May 7, 2020


Physical Hardware


…Keyboard

ABIVirtualCPUs

VirtualMemory Events Files Sockets Syscalls

Process1

VM1

L23-20


Virtual Machines


May 7, 2020


Physical Hardware


…Keyboard

VirtualProcessor


Process2

VirtualProcessor


Process3

VirtualProcessor


Process4ABI

VirtualCPUs


Process1

VM1

L23-21


Virtual Machines

• A Virtual Machine (VM) is an emulation of a computer system– Very general concept, used beyond operating systems

May 7, 2020

ABIVirtualProcessor


Process1


Physical Hardware


…Keyboard

VirtualProcessor


Process2

VirtualProcessor


Process3

VirtualProcessor


Process4

VM1

L23-22


Virtual Machines Are Everywhere

• Example: Consider a Python program running on a Linux Virtual Machine

May 7, 2020 L23-23




May 7, 2020

Python program

L23-24




May 7, 2020

Python programPython Language

L23-25




May 7, 2020

Python program

Python interpreter (CPython)Python Language

L23-26




May 7, 2020

Python program

Python interpreter (CPython) Implements a Python VM Python Language

L23-27




May 7, 2020

Python program

Python interpreter (CPython) Implements a Python VM Python Language

Linux ABI

L23-28




May 7, 2020

Python program

Python interpreter (CPython) Implements a Python VM

Linux OS kernel

Python Language

Linux ABI

L23-29




May 7, 2020

Python program


Linux OS kernel Implements a Linux-x86 VM

Python Language

Linux ABI

L23-30




May 7, 2020

Python program



Python Language

Linux ABI

x86 ISA

L23-31




May 7, 2020

Python program



VirtualBox

Python Language

Linux ABI

x86 ISA

L23-32




May 7, 2020

Python program



VirtualBox

Python Language

Linux ABI

x86 ISAImplements an x86 system VM

L23-33




May 7, 2020

Python program


Win/Linux/MacOS/… ABI


VirtualBox

Python Language

Linux ABI


L23-34




May 7, 2020

Python program




VirtualBox

OS kernel (Win/Linux/MacOS/…)

Python Language

Linux ABI


L23-35




May 7, 2020

Python program




VirtualBox


Python Language

Linux ABI

x86 ISA

Implements an OS-x86 VM

Implements an x86 system VM

L23-36




May 7, 2020

Python program




VirtualBox


Python Language

Linux ABI

x86 ISA

Implements an OS-x86 VMx86 ISA


L23-37




May 7, 2020

Python program




VirtualBox


Hardware (e.g., your laptop)

Python Language

Linux ABI

x86 ISA



L23-38




May 7, 2020

Python program




VirtualBox

Implements an x86 physical machine


Hardware (e.g., your laptop)

Python Language

Linux ABI

x86 ISA



L23-39


Implementing Virtual Machines

• Virtual machines can be implemented entirely in software, but at a performance cost– e.g., Python programs are 10-100x slower than native Linux

programs due to Python interpreter overheads

May 7, 2020 L23-40


Implementing Virtual Machines

• Virtual machines can be implemented entirely in software, but at a performance cost– e.g., Python programs are 10-100x slower than native Linux

programs due to Python interpreter overheads

• We want to support virtual machines with minimal overheads à need hardware support!

May 7, 2020 L23-41


ISA Extensions to Support OS

May 7, 2020 L23-42



• Two modes of execution: user and supervisor

May 7, 2020 L23-43



• Two modes of execution: user and supervisor– OS kernel runs in supervisor mode– All other processes run in user mode

May 7, 2020 L23-44




• Privileged instructions and registers thatare only available in supervisor mode

May 7, 2020 L23-45





• Traps (exceptions) to safely transitionfrom user to supervisor mode

May 7, 2020 L23-46





• Traps (exceptions) to safely transitionfrom user to supervisor mode

• Virtual memory to provide private address spaces and abstract the storage resources of the machine

May 7, 2020 L23-47


Supporting Multiple OSs

May 7, 2020

process1

OS Kernel1

Hardware

processN…

…

process1

OS KernelK

processM…ABIABI

Virtual Machine Monitor (VMM)ISA ISA

ISA

L23-48



• A VMM (aka Hypervisor) provides a system virtual machine to each OS

May 7, 2020

process1

OS Kernel1

Hardware

processN…

…

process1

OS KernelK

processM…ABIABI


ISA

L23-49



• A VMM (aka Hypervisor) provides a system virtual machine to each OS

• VMM can run directly on hardware (as above) or on another OS– Precisely, VMM can be implemented against an ISA (as above) or

a process-level ABI. Who knows what lays below the interface…

May 7, 2020

process1

OS Kernel1

Hardware

processN…

…

process1

OS KernelK

processM…ABIABI


ISA

L23-50


Motivation for Multiple OSsSome motivations for using multiple operating systems on a single computer:

• Allows use of capabilities of multiple distinct operating systems.

May 7, 2020 L23-51




• Allows different users to share a system while using completely independent software stacks.

May 7, 2020 L23-52





• Allows for load balancing and migration across multiple machines.

May 7, 2020 L23-53





• Allows for load balancing and migration across multiple machines.

• Allows operating system development without making entire machine unstable or unusable.

May 7, 2020 L23-54


From (Machine we are attempting to execute)• Guest• Client• Foreign ISA

To (Machine that is doing the real execution)• Host• Target• Native ISA

May 7, 2020

Virtualization Nomenclature

L23-55

OS Kernel1

Hardware

… OS KernelK

Virtual Machine Monitor (VMM)

ForeignISA

NativeISA

Host OS/Hypervisor

Guest OS


Virtual Machine Requirements[Popek and Goldberg, 1974]• Equivalence/Fidelity: A program running on the

VMM should exhibit a behavior essentially identical to that demonstrated when running on an equivalent machine directly.

• Resource control/Safety: The VMM must be in complete control of the virtualized resources.

• Efficiency/Performance: A statistically dominant fraction of machine instructions must be executed without VMM intervention. – Every instruction is intervened by VMM: Virtual machines

implemented entirely in software using binary emulation– VMM only intervenes sensitive instructions: need hardware

support

May 7, 2020 L23-56


Virtual Machine Requirements[Popek and Goldberg, 1974]Classification of instructions into 3 groups:

• Privileged instructions: Instructions that trap if the processor is in user mode and do not trap if it is in a more privileged mode.

• Control-sensitive instructions: Instructions that attempt to change the configuration of resources in the system.

• Behavior-sensitive instructions: Those whose behavior depends on the configuration of resources, e.g., mode

Building an effective VMM for an architecture is possible if the set of sensitive instructions is a subset of the set of privileged instructions.

May 7, 2020 L23-57


Security and Side Channels

• ISA and ABI are timing-independent interfaces– Specify what should happen, not when

May 7, 2020 L23-58




• Hardware isolation mechanisms like virtual memory guarantee that architectural state will not be directly exposed to other processes…

May 7, 2020 L23-59




• Hardware isolation mechanisms like virtual memory guarantee that architectural state will not be directly exposed to other processes…

• …but timing and other implementation details (e.g., microarchitectural state, power, etc.) may be used as side channels to leak information!

May 7, 2020 L23-60

CoreL1

Shared LLC

CoreL1

CoreL1

CoreL1

VictimVM

AttackerVM

VM Isolation


Side Channels

• Side channels do not exploit software bugs or crypto algorithm weaknesses– E.g. Buffer overflow attack is not a side channel attack

L23-61May 7, 2020


Side Channels

• Side channels do not exploit software bugs or crypto algorithm weaknesses– E.g. Buffer overflow attack is not a side channel attack

• Side channels leak information based on the implementation of a computer system– E.g. acoustic side channel– Timing information, power consumption, electromagnetic leaks

etc.

L23-62May 7, 2020


Cache-Based Side Channels

• Attacker can infer shared cache behavior of victim– e.g., prime+probe attack, flush+reload attacks– Leaks address-dependent information, e.g., RSA [Percival

2005] and AES keys [Osvik et al. 2005]

May 7, 2020 L23-63

Core 0

Main Memory

Priv Caches

Core 1

Priv Caches

Core 2

Priv Caches

Core 3

Priv Caches

Shared Cache

Victim Attacker


Cache-based Side Channels• RSA example:

– Square-and-multiply based exponentiation

L23-64May 7, 2020

Input : base b, modulo m,exponent e = (en−1 ...e0 )2

Output: be mod mr = 1for i = n−1 down to 0 do

r = sqr(r)r = mod(r,m) if ei == 1 then

r = mul(r,b)r = mod(r,m)

end endreturn r




L23-65May 7, 2020





end endreturn r




L23-66May 7, 2020





end endreturn r

Secret-dependent memory accesses à transmitter


DRAM

Cache-based Side Channels

L23-67May 7, 2020

Victim Attacker

Shared LLC

Privatecaches

Privatecaches

• Prime+Probe


DRAM


L23-68May 7, 2020

Victim Attacker

Shared LLC

Privatecaches

Privatecaches

• Prime+Probe


DRAM


L23-69May 7, 2020

Victim Attacker

Shared LLC

Privatecaches

Privatecaches

• Prime+Probe


DRAM


L23-70May 7, 2020

conflict

Victim Attacker

Shared LLC

Privatecaches

Privatecaches

• Prime+Probe


DRAM


L23-71May 7, 2020

conflict

Victim Attacker

Shared LLC

Privatecaches

Privatecaches

• Prime+Probe


DRAM


L23-72May 7, 2020

conflict

Victim Attacker

Shared LLC

Privatecaches

Privatecaches

• Prime+Probe– Victim access a transmitter addressà Attacker has 1 cache miss during

Probeà Long access latency


DRAM


L23-73May 7, 2020

conflict

Victim Attacker

Shared LLC

Privatecaches

Privatecaches



- Victim does not access the transmitter address

à Attacker has all cache hits during Probe

à Short access latency


DRAM


L23-74May 7, 2020

conflict

Victim Attacker

Shared LLC

Privatecaches

Privatecaches





à Short access latency • Microarch side channels among

threads running on same SMT core?


DRAM


L23-75May 7, 2020

conflict

Victim Attacker

Shared LLC

Privatecaches

Privatecaches






threads running on same SMT core?L1/L2/L3 caches


DRAM


L23-76May 7, 2020

conflict

Victim Attacker

Shared LLC

Privatecaches

Privatecaches







Branch & other predictors


DRAM


L23-77May 7, 2020

conflict

Victim Attacker

Shared LLC

Privatecaches

Privatecaches







ROB/Issue/FU contentionBranch & other predictors


Microarchitecture Side Channels

L23-78May 7, 2020



L23-79May 7, 2020

Attack Platforms



L23-80May 7, 2020

Attack Platforms Target Applications


Exploiting Speculative Execution in Side-Channel Attacks• OoO cores run instructions speculatively and out of order• Problem: Speculative instructions can change

microarchitectural state à can leak data via side channel

May 7, 2020 L23-81




• Example: In x86, process page table can have kernel pages, but kernel pages only accessible in kernel mode

– Avoids switching page tables on context switches

May 7, 2020

Address Space User pages Kernel pages

0x0 0xFF...F

L23-82





– Avoids switching page tables on context switches– What does the following code do when run in user mode?

May 7, 2020


0x0 0xFF...F

val = *kernel_address;

L23-83






May 7, 2020


0x0 0xFF...F

val = *kernel_address;Causes a protection fault

L23-84






May 7, 2020


0x0 0xFF...F

val = *kernel_address;Causes a protection fault

In Intel processors, protection fault is handled lateà Kernel data speculatively loaded into val register!

L23-85


Meltdown[Lipp et al. 2018]

1. Setup: Attacker allocates 256-line probe_array, flushes all its cache lines

May 7, 2020 L23-86




2. Transmit: Attacker executes

May 7, 2020

Ld1: uint8_t byte = *kernel_address;Ld2: unit8_t dummy = probe_array[byte*64];

L23-87





May 7, 2020


L23-88

….Ld1Ld2….

ROB head





May 7, 2020


L23-89

….Ld1Ld2….

ROB head





May 7, 2020


L23-90

….Ld1Ld2….

ROB head





3. Receive: After handling protection fault, attacker times accesses to all cache lines of probe_array, finds which one hits à recovers byte

May 7, 2020


L23-91

….Ld1Ld2….

ROB head






May 7, 2020


L23-92

….Ld1Ld2….

ROB head

Ld2 is transmitter






• Result: Attacker can read arbitrary kernel data!– For higher performance, use transactional memory – Mitigation: Do not map kernel data in user page tables;

Register poisoningMay 7, 2020


L23-93

….Ld1Ld2….

ROB head

Ld2 is transmitter


General Attack Schema[Belay, Devadas, Emer]

May 7, 2020

Domain of victim

Secret

Transmitter

Attacker

Secret

ReceiverSide channelAcce

ss

L23-94


General Attack Schema[Belay, Devadas, Emer]

• Types of transmitter:1. Pre-existing (the victim itself leaks secret, e.g., RSA/AES keys)2. Programmed by attacker (e.g., Meltdown)3. Synthesized from existing victim code by attacker (e.g., Spectre)

May 7, 2020

Domain of victim

Secret

Transmitter

Attacker

Secret

ReceiverSide channelAcce

ss

L23-95


Spectre variant 1 — ExploitingConditional Branches [Kocher et al. 2018]

• Consider the following kernel code, e.g., in a system call

May 7, 2020 L23-96

Br: if (x < size_array1) {

Ld1: secret = array1[x]*4096

Ld2: y = array2[secret]

}




May 7, 2020 L23-97




}

Attacker to read arbitrary memory:1. Setup: Train branch predictor




May 7, 2020 L23-98




}

Attacker to read arbitrary memory:1. Setup: Train branch predictor2. Transmit: Trigger branch misprediction; &array1[x] maps to some desired kernel address




May 7, 2020 L23-99




}


….BrLd1Ld2…

ROBhead




May 7, 2020 L23-100




}


….BrLd1Ld2…

ROBhead




May 7, 2020 L23-101




}

Attacker to read arbitrary memory:1. Setup: Train branch predictor2. Transmit: Trigger branch misprediction; &array1[x] maps to some desired kernel address3. Receive: Attacker probes cache to infer which line of array2was fetched

….BrLd1Ld2…

ROBhead


Spectre variant 2—Branch Target Injection [Kocher et al. 2018]

• Assume the BTB stores partial tags but full target PCs. How can this be exploited?

May 7, 2020 L23-102

Br: if (…) {

… }

…






May 7, 2020 L23-103

Br: if (…) {

… }

…



1. Setup: Attacker chooses anyjump in kernel code, mistrainsBTB so that it predicts a target PC under the control of the attacker that leaks information

2. Transmit & receive: Like in Spectre v1




• Most BTBs store partial tags and targets…– Hard to get BTB to jump from a kernel address to a far-away

user address

May 7, 2020 L23-104

Br: if (…) {

… }

…








• Most BTBs store partial tags and targets…– Hard to get BTB to jump from a kernel address to a far-away

user address

• But most cores add an indirect branch predictor that stores full targets (e.g., to predict virtual function calls)– Spectre v2 exploits this predictor instead

May 7, 2020 L23-105

Br: if (…) {

… }

…






Spectre variants and mitigations

• Spectre relies on speculative execution, not late exception handling à Much harder to fix than Meltdown

May 7, 2020 L23-106




• Several other Spectre variants reported– Leveraging the speculative store buffer, return address stack,

leaking privileged registers, etc.

May 7, 2020 L23-107






• Can attack any type of VM, including OSs, VMMs, JavaScript engines in browsers, and the OS network stack (NetSpectre)

May 7, 2020 L23-108







• Short-term mitigations:– Microcode updates (disable sharing of speculative state when possible)– OS and compiler patches to selectively avoid speculation

May 7, 2020 L23-109







• Short-term mitigations:– Microcode updates (disable sharing of speculative state when possible)– OS and compiler patches to selectively avoid speculation

• Long-term mitigations:– Disabling speculation?– Closing side channels?

May 7, 2020 L23-110


The Age of Pervasive Hardware Security Attacks

On-chip Processor

Off-chip DRAM

core

L1/L2

L3

core

L1/L2

……

Flush+Reload, Prime+ProbeDirectory Attacks

RNG Unit Covert Chanel

CacheOut, RIDL, FalloutPort contention,cache banking, 4K Alias

RowHammer, DRAMARAMBleed

Spectre, MeltdownForeshadow,Arithmetic timing

L23-111May 7, 2020


The Age of Pervasive Hardware Security Attacks

On-chip Processor

Off-chip DRAM

core

L1/L2

L3

core

L1/L2

……

Flush+Reload, Prime+ProbeDirectory Attacks

RNG Unit Covert Chanel

CacheOut, RIDL, FalloutPort contention,cache banking, 4K Alias

RowHammer, DRAMARAMBleed

Spectre, MeltdownForeshadow,Arithmetic timing

L23-112May 7, 2020

New Special Topic course in Fall 20206.888 Secure Hardware Design

L23-113MIT 6.823 Spring 2020

Thank you!

May 7, 2020

virtualization and securitycsg.csail.mit.edu/6.823/lectures/l23split.pdfvirtualization and security...

Documents