certikos: a breakthrough toward hacker-resistant operating ... · certikos: a breakthrough toward...

CertiKOS: A Breakthrough toward Hacker-Resistant

Operating Systems

Zhong ShaoYale University

January 25, 2018

Acknowledgement: Ronghui Gu, Newman Wu, Hao Chen, Jieung Kim, Jeremie Koenig, Vilhelm Sjoberg, Mengqi Liu, Lionel Rieg, Quentin Carbonneaux, Unsung Lee, Jiyong Shin, David Costanzo, Tahina Ramananandro, Hernan Vanzetto, Shu-Chun Weng, Zefeng Zeng, Zhencao Zhang, Liang Gu, Jan Hoffmann, Joshua Lockerman, and Bryan Ford. This research is supported in part by DARPA CRASH and HACMS programs and NSF SaTC and Expeditions in Computing programs.

Computer System

Motivation

Transportation

Health

Aviation

Environment

Desktop

Mobile

Financial

cloud

Hardware

OS

Applications

Computer System

Motivation

Accident

Life

Loss

Environment

Crash

Mobile

Financial

cloud

Hardware

OS

Applications

Motivation

System Software Runs Everywhere

Software errors

Untrusted No!?Test

$312B cost

Motivation

“”— Edsger Dijkstra

Program testing can be used to show the presence of bugs, but never to show their absence.

“ ”— seL4 [SOSP’09]

Complete formal verification is the only known way to guarantee that a system is free of programming errors.

Motivation

“”— NSF SFM Report[2016]

Formal methods are the only reliable way to achieve security and privacy in computer systems.

“ ”— seL4 [SOSP’09]

Complete formal verification is the only known way to guarantee that a system is free of programming errors.

Motivation

“”— NSF SFM Report[2016]

Formal methods are the only reliable way to achieve security and privacy in computer systems.

mathematically prove

under all inputsunder all execution

program meets specification

rule out entire classes of attacks

Formal Verification

Motivation

System Software Runs Everywhere

Software errors

Untrusted No!?Test

$312B cost

Challenges?

Formal Verification

seL4 [SOSP’09] C 7.5k LOC

Proof 11 py

Asm500 LOCunverified

C1.3k LOCunverified

Challenges: huge proof efforts

Challenges: Compositionality

Asm

Abstraction Gap

C

A Complex System

Asm


C

A Complex System

Asm


C

Verify

Verify

Verify

VerifyVerify

Verify

Verify

Verify Verify

10 11

8

A Complex System

Asm

Compiler


C

Verify

Verify

Verify

VerifyVerify

Verify

Verify

5

42

7 6

9

1

3

Verify

Verify

10

8

5

114

2

7 6

9

1

3

A Complex System

Asm

Compiler


C

Complete Verification

multiprocessor

I/O concurrencymulti-thread

fine-grained lock fine-grained lock

Challenges: Concurrency

10 11

8

5

42

7 6

9

1

3


1 0 1 18

5

42

7 6

91

3


1 0 1 18

5

42

7 6

91

3

CPU i CPU j

fine-grained lock

Complete Verification

Challenges: New Domain

System Verification

Huge gap

Contribution

aim to solve all these challenges

Certified Abstraction LayersCertiKOS

Contribution

Certified Abstraction Layers

1 0 1 18

5

42

7 6

91

3

1 0 1 18

5

42

7 6

91

3

CPU i CPU j

fine-grained lock

untangle

Contribution


verify existing systems

build the next generation sssssssssystem software designed to be reliable

and secure

Contribution



build the next generation sssssssssystem software certified

Contribution



System Verification

Huge gapbuild the next generation sssssssssystem software certified

Contribution


System Verification



Contribution


Certified System Software



Contribution


M1

L

L1

R1

L

M2

L2

R1�

L0

M0

LR0

Contribution


R1

L0

M0

LR0

M1

L

L1

� M2

L2�

�

Contribution


R1

L0

M0

R0

M1

L1

� M2

L2�

�o

CompCertX

R1

L0

M0

R0

M1

L1

� M2

L2�

�o

C

Asm L0

M3

R0�

L3

Asm

R’1

L’0

M’0R’0

M’1

L’1

� M’2

L’2�

�o

CompCertX

R’1

L’0

M’0R’0

M’1

L’1

� M’2

L’2�

�o

C

Asm

Contribution

mC2 [OSDI’16]the first formally certified concurrent OS kernel with fine-grained locks6.5k C&Asm, 2 py

mCertiKOS [POPL’15]certified sequential OS kernels3k C&Asm, 1 py

Security [PLDI’16b] 0.5 py

Interrupt [PLDI’16a] 0.5 py Certified Abstraction

Layers[CCAL 2017]

Contribution

functional correctnessliveness

no stack/integer/buffer overflow

no race condition

Certified System

Software

ContributionmC2

CPU

Seria

l

VGA(Video)

Keyb

oard

IOAP

IC

Legend

Hardware

Driver

drive

Use

Data

Kern. Module

Core 0LAPIC 0

Core 1LAPIC 1

Core 8LAPIC 8...

Memory

Heap

BIO

S

DMA

Spin

Lo

cks

Ticket MCS Container

Alloc Tbl

PMM

IPC

SleepQPendQ

ELF Ldr

Trap & Syscall

Per Core

RdyQ

Scheduler

Thread

Cur TID PCPUPer T

hrea

d

k_stack

TCB

k_contextTSC

Hz

Timer

LAPIC

ProcessVM Monitor

Lib MemSync

. &

Mut

ual

Exclu

.

CVFIFOBBQ ...

Page Map VMM

Serial

VideoConsole Buffer

Kbd

Console

IOAP

IC

APIC

ContributionmC2

Coq

machine-checkable proof

C layers6.1k LOC

400 LOCCompCertX

Asm layers Asm layers�

ContributionmC2

Coq

machine-checkable proof

C layers6.1k LOC

400 LOCCompCertX

Asm layers Asm layers�

Proof AssistantACM Software System Award

Some of the significant results that were accomplished using Coq are proofs for the four color theorem, the development of CompCert (a fully verified compiler for C), the development at Harvard of a verified version of Google's software fault isolation, and most recent, the fully specified and verified hypervisor OS kernel CertiKOS.

“

”— ACM

Deployment

CertiKOS on Landshark, DARPA HACMS

Deployment

CertiKOS on Quadcopter

Build a Certified System

Spin-lock Module

Case Study

CPU 0

KeyboardDriver3

CPU 1

Thread Queue Module

Scheduling ModuleInter-Process Communication

Keyboard

User Application

SendCompiler

11certified objects

specification ofmodules to trust

1

Certified Sequential Layer [POPL’15]

11certified objects


1

abs-state


11certified objects


1

abs-state

primitives


111

memory

module

Certified Sequential Layer

M

L1

implementation

111

AT


L1

M

L2

111

AT

implementation


specification

L1

M

L2

111

AT

33

3

implementation

specification

L1


M

L2

implementation

Example: Thread Queue

typedef struct tcb { state s; tcb *prev, *next; } tcb;

tcb tcbp[1024];

typedef struct tdq { tcb *head, *tail; } tdq;

tdq* td_queue; C

tcbp[0] tcbp[1] tcbp[2]

3

M

implementation


s0 s1 s2


tcb tcbp[1024];


tdq* td_queue; C


3

M

implementation


s0 s1 s2

head tail



tcb tcbp[1024];


tdq* td_queue; C

3

M

3

implementation


tcb* dequeue(tdq* q) { tcb *head, *next; tcb *i = null; if (!q) return i; head = q -> head; if (!head) return i; i = head; next = i -> next;

if (!next) { q -> head = null; q -> tail = null; } else { next -> prev = null; q -> head = next; } return i; }

s0 s1 s2

head tail

C


M

Coq


Definition tcbp := ZMap.t state. Definition td_queue := List Z.

3

specification3

L2

Coq



3

specificationtcbp(0) tcbp(1) tcbp(2)

s0 s1 s2

3

L2

Coq



3

specification

s0 s1 s2

tcbp(0) tcbp(1) tcbp(2)

1

td_queue

:: 0 2:: :: nil

3

L2

Example: Thread Queue3


1

td_queue

:: 0 2:: :: nil

3

implementation

s0 s1 s2

head tail


R

s0 s1 s2

3

L2

M



1

td_queue

:: 0 2:: :: nil

3

Coq

Function dequeue (q) := match q with | head :: q’ => (q’, Some head) | nil => (nil, None) end.

s0 s1 s2

3

L2



1

td_queue

0 2:: :: nil

3

Coq

Function dequeue (q) := match q with | head :: q’ => (q’, Some head) | nil => (nil, None) end.

s0 s1 s2

executable

3

L2

Program Context33

specification3

implementation

Simulation Proof

R RM

L1

L2

R

Deep SpecificationL2

M

Deep Specification [POPL’15]

Deep spec captures all we need to know about over

No need to look at again

L2

M L1

M

Any property about can be proved using alone

M

L2

M

L1

L2

R

kernel

MM

TM

PM

Trap

code

seq machine

mCertiKOS

TM

PM

Trap

MM

seq machine

kernelmCertiKOS

mem MM

TM

PM

Trap

memory management

seq machine

kernelmCertiKOS

�

Trap

PM

MM

TM

TM

PM

Trap

mem

thread

proc

trap

seq machine

kernelmCertiKOS

Trap

PM

TM

mem

thread

proc

trap

certified sequential kernelMM

mCertiKOS

seq machine

VM

Trap

VM

mCertiKOS

mem

thread

proc

trap

seq machine virt�

virt�

PM

TM

MM

TrapmCertiKOS

mem

thread

proc

seq machine virt�

virt�VM

VM

PM

TM

MM

trap

vm

TrapmCertiKOS

mem

thread

proc

trap

seq machine virt

virt

vm

VM

PM

TM

MM

certified hypervisor

mCertiKOS [POPL’15]

3k LOC1 person year

Can boot Linux as a guest

Concurrent Framework [OSDI’16]

mem

thread

proc

trap

virt

seq machine

multicore machine CPU3CPU2CPU1CPU0

certified sequential kernel


mem

thread

proc

trap

virt

seq machine


machine liftingcontribution

certifiedconcurrent layer

CPU-local machine CPU0 CPU1 CPU2 CPU3


mem

thread

seq machine

spin-lock



proc

trap

virt

contributionmachine lifting

certifiedconcurrent layer


mem

thread

proc

trap

virt

seq machine

spin-lock

CPU-local machine CPU0 CPU1 CPU2

thread-local machine

CPU3


Certified Concurrent Layers

local certified objects

L1


atomic objects

logical loga sequence of events

L1

x


L1


atomic objectsx


x

L1


atomic objects


x

L1

to share


L1

L2

x

fine-grained locking

Concurrent Framework

machine lifting



x

step 0: raw x86 multicore model

CPU0

CPU1

atom

private

share

atom

0.a

1.a


atom

private

share

atom

0.a 1.a

CPU0

CPU1

logical log

non-determinism



atom

private

share

atom

0.a

1.a

0 1

1 0

CPU0

CPU1



non-determinism

atom

private

share

atom

0.a

1.a

0 1 1 0

CPU0

CPU1


oracle


non-determinism

atom

private

share

atom

0.a

1.a

0 1 1 0

CPU0

CPU1

step 1: logical hardware scheduler

Ehs


atom

private

share

atom

0.a 1.a0 1 1 0

CPU0

CPU1


Ehs

logical log



Ehs?


step 2: push/pull memory model

Ehs machine with hardware scheduler

share

shared mem

CPU0




shareCPU0

shared mem

pull

logical copy




shareCPU0 pull

shared mem

logical copy

shared mem

CPU1 pull


race condition



shareCPU0 pull

shared mem

logical copy

push




shareCPU0 pull push

shared mem

logical copy



machine with push/pull model

atom

private

private

atom

0.a

1.a

0 1 1 0

CPU0

CPU1

Ehs

step 3: environment context model




atom

private

private

atom

0.a

1.a

1 1 0

CPU0

CPU1

Ehs 0



0



atom

private

private

atom

0.a

CPU0

CPU1

Ehs

1.a

1 1 0





atom

private

private

atom

0.a

CPU0

CPU1

1.a1 1 00E

environment context





step 4: remove unnecessary interleaving

CPU i machine CPU j machine

share privateatom pull push







shuffle







merge





atom

CPU-local machine CPU-local machine

0.a 1.a0 1 1 0logical log

E

seq machine seq machine

Machine Lifting


Spin-lock Module

Case Study

KeyboardDriver3 Thread Queue Module


Keyboard

User Application

SendCompiler

Security

CPU 0 CPU 1


Case Study

KeyboardDriver3 Thread Queue Module


Keyboard

User Application

SendCompiler

Security

CPU 0 CPU 1Spin-lock Module


Acquire Lock Specification

logical copy

safely pull


logical copy

safely pull

pull will eventually return


logical copy

mutual exclusion liveness

C

Example: Ticket Lock

mutual exclusion liveness+

void acq_lock (uint i) {

uint64 t = FAI_ticket (i);

while ( get_now (i) != t) { }

pull (i); }

C






u ll (i); }

FAI_ticket

get_now

pull

FAIticket

C






u ll (i); }

FAI_ticket

pull

FAIticket

get_now getnow

C






u ll (i); }

FAI_ticket

pull

FAIticket

get_now getnow

getnow

C






u ll (i); }

FAI_ticket

pull

FAIticket

get_now

getnow

getnow

pull

C






u ll (i); }

FAI_ticket

pull

FAIticket

get_now

getnow

getnow pull

C


liveness+




u ll (i); }

FAI_ticket

pull

FAIticketget_now

getnow

getnow pull

unique t

#CPUs < 264

mutual exclusion

C






u ll (i); }

FAI_ticket

pull

FAIticket

get_now

getnow

getnow pull}bounded

#CPUs is boundeda fair hardware schedulerlock holders will release lock


FAIticket

getnow

getnow pull

acq_lock acqlock

acq_lock

C





u ll (i); }

FAI_ticket

pull

get_now < mutual exclusion will beviolated when there is an integer overflow for t

bug in the originalimplementation

3

Spin-lock Module

Case Study

KeyboardDriverThread Queue Module


Keyboard

User Application

SendCompiler

Security

CPU 0 CPU 1


3

Case Study



Keyboard

User Application

SendCompiler

Security

CPU 0 CPU 1


Spin-lock Module

Example: Shared Thread Queue

dequeue

local memory


dequeue

shared memory

logicalcopy

acq lock


dequeue

shared memory

logicalcopy

acq lock

rel lock


dequeue

shared memory

deq


dequeue

shared memory

Spin-lock Module

Case Study



Keyboard

User Application

SendCompiler

Security

CPU 0 CPU 1

3


Spin-lock Module

Case Study


Inter-Process Communication

Keyboard

User Application

SendCompiler

Security

CPU 0 CPU 1

3

Scheduling Module


void yield () {

uint t = tid(); … (t, rdq());

uint s = (rdq()); … (t, s)

}

enq

deq

context_switch

Thread-Local Machine

Thread-Local Machine

yieldsleep wakeup

Software Scheduler

CV

IPC

thread-local machine

[Operating SystemsPrinciples and Practice 2011]

Found hard bugs in the popularOS textbook

Spin-lock Module

Case Study



Keyboard

User Application

SendCompiler

Security

CPU 0 CPU 1

3


Spin-lock Module

Case Study

Thread Queue Module


Keyboard

User Application

SendCompiler

CPU 0 CPU 1

3

KeyboardDriver

Security


Device Driver [PLDI16’a]

Device

0 0External events

State

Log 0 0 0

read/write

Raw Device ObjDriver Layers

Logical CPU

CPU i

Interrupt iret0

Device Driver [PLDI16’a]

Device

0 0External events

Driver Layers

Logical CPU

CPU i

Interrupt iret0

State

Log 0 0 0

read/write

Raw Device Obj

DriverCode

Prims

Abs-State

Abstract Device Obj

Spin-lock Module

Case Study



Keyboard

User Application

SendCompiler

Security

CPU 0 CPU 1

3


End-to-End Security [PLDI16’b]

TM

PM

Trap

mem

thread

proc

trap

seq machine

OObservation functionspecify and prove general security policies with declassification

MM

non-interferencefound security-bugs: spawn, palloc,…

security-preservation simulation

O

O0

O1

O2

O3

secure

secure

secure

secure

secure

Spin-lock Module

Case Study

Thread Queue Module


Keyboard

User Application

SendCompiler

Security

CPU 0 CPU 1

3

KeyboardDriver


Summary: Certified OS

CertiKOS is the first fully certified OS kernel that is done economically (< 3 person years), proves more properties, runs on concurrent HW, and is truly extensible

Still very high barriers of entry: (1) OS kernel development is very difficult(2) Formal specifications and proofs are hard to build(3) Need intimate programming language expertise to succeed• These are three completely different communities• Most people can only do one out of the above three. • The Yale team has been working on all three for >15 years

Summary: OS Landscape (Nov 2017)

Desktop: Linux, macOS, Windows, ChromeOS, freeBSD, …Hypervisor/Cloud: Linux KVM & Docker, VMWare, Xen, … Mobile: Android (Linux), iOS, …Embedded: Embedded Linux, VxWorks, QNX, LynxOS, …

• All of them are bloated, old, and contain many bugs• Urgently need new OSes for emerging platforms & apps

(IoTs, Drones, Self-Driving Cars, Cloud, NetworkOS, Blockchains, …)

OS evolution has reached an inflection point: Need a certified OS that provides security, extensibility, performance, and can work across multiple platforms.

certikos: a breakthrough toward hacker-resistant operating ... · certikos: a breakthrough toward...

Documents