vm security - kiayias.comvm security steve maresca. introductions who i am work...
TRANSCRIPT
VM SecuritySteve Maresca
Introductions
Who I amWork interests/motivationsSecurity researcher
VM introspection, Honeynets, DarknetsDeveloper
Zentific, LibVMI, DARPA CFT, XenVM management softwarelots of miscellaneous glue
Security analyst for UConn
Shape of this discussion
● General concepts, history● Crash course in virtualization ● Virtualization challenges for the x86 CPU● x86 Hardware Assisted Virtualization● Detection of a virtual environment● Exploitation● Example vulnerabilities/exploits
Ask questions: a dialogue is most productive
General concepts
General concept
Virtualization is the art of abstraction ● IBM calls it "The process of hiding the
underlying physical hardware in a way that makes it transparently usable and shareable by multiple operating systems."○ http://www.ibm.com/developerworks/linux/library/l-hypervisor/
● Most hardware can be virtualized to the pointthat a guest doesn’t know/care● Underlying physical hardware may not be
dedicated to virtual machines
Virt and "Cloud" are not new
Timesharing (1960s, Big Iron)Adaptation to rapid pace of hardware/architecture improvements● ability to run program written for CPU x on
next model with very different CPU y● esoteric architectures (24 bit, 31 bit, 12 bit)● IBM, Honeywell, et al.● protecting customer investment in softwareContemporary examples● NES, C64, Atari, playstation emulators
Virt and "Cloud" are not new
Horizontal scaling of computational workload to N-many nodes● most effective for embarrassingly parallel
algorithms when they're possible○ Google's MapReduce
● brute force computations/clever data partitioning○ Rainbow tables○ multiplayer game sharding
MOSIX, Beowulf, Kerrighed clusters (and many others)
Nor are their security issues
Timesharing● Use of punchcards : laborious● Considerable competition for compute cycles● Denial of Service popular ● Passwords disclosure in CTSS (early MIT
timesharing system) http://osvdb.org/show/osvdb/23257
Modern HPC systems: very vulnerable● intersection of valuable research, powerful
hardware, and difficult-to-maintain software
Virtualization as we know it
● Each virtual OS is called a virtual machine (VM), guest, domU, instance, partition
● VMs are managed, controlled, scheduled by a hypervisor or virtual machine monitor (VMM). Hypervisor == VMM
● Ask 10 people about "cloud" - get 10 definitions
Virtualization as we know it
● Virtual machines ideally○ must be efficient (reasonably near the performance
of bare hardware)○ in a virtual context, must behave equivalently to real
hardware: determinism, correctness required● Utilized in pursuit of
○ cost/power/cooling/space savings○ workload isolation/consolidation○ continued support for legacy apps (or just
Windows/*nix on your Mac) ○ software validation and testing
Virtualization as we know it
Types of virtual machines● Fully emulated ● Hardware assisted
○ CPU has virt-specific instructions to help with expensive operations
● Paravirtual○ VM kernel is aware of its existence within a virtual
environment, avoids expensive operations● Hybrids
○ Hardware assisted with paravirtual "enlightenments"○ Hardware assisted with some emulation (etc)
Virt. platforms and their VM types
● Full virtualization (aka emulation)○ Bochs and QEMU
● Paravirtualization○ Xen, VMware, Lguest, z/vm
● Binary Translation○ VMware, VirtualPC, VirtualBox, QEMU
● Hardware Virtualization○ Xen, VMware, VirtualPC, VirtualBox, KVM, Bitvisor,
z/vm, ..
Virtualization as we know it
Types of hypervisors● Type 1: “bare metal” hypervisors run directly
on the host hardware○ guest OS runs at level above the hypervisor
● Type 2: “hosted” hypervisors run on top of an OS○ The hypervisor layer exists as distinct second
software level○ Guest operating systems run at the third level○ above the hardware
Virtualization as we know it
Types of hypervisors
https://en.wikipedia.org/wiki/File:Hyperviseur.png
Virt. platforms & hypervisor types
● Type 1 (bare metal)○ Xen, HyperV, VMware ESX/vSphere,
KVM, z/vm, Bitvisor● Type 2 (Hosted)
○ Bochs, QEMU, VirtualBox, VMware player, VMware server, VirtualPC
The boundary is sometimes blurry (e.g.) KVM is a kernel module that 'elevates' Linux to a type 1 hypervisor
Virtualization crash course
Virtualization crash course
At its core: hardware multiplexingGoal: delegate resources to users/software Problem: segregation guarantees required while● fairly sharing access to hardware (CPU, I/O,
mem)● encapsulation/isolation● sometimes cooperative sharing
○ VMs know they are being virtualized and yield as often as possible, avoid expensive device emulation (paravirtualization)
Virtualization crash course
Traditional non-virt solutions to this problem:● OS containers● Chroots● Sandboxing via policy frameworks (e.g.
SElinux) or some combination of the above● Physical separation
○ Relevant yet terrifying example - military requirements to use separate computers to access unclassified, restricted, secret and top-secret networks
These are all still valid approaches
Virtualization crash course
Modern solutions to this problem:● hardware assisted virtualization● containers, chroots, and sandboxes remain
in use○ reverse engineers make it a hobby to break out of
them● physical separation only IFF life depends
upon it ○ unplugged computers are (slightly) more secure
computers○ Sometimes virtual environments must be equally
separate (PCI-DSS)
Analogy: process multitaskingVirtualization is context switching at scale● Multitasking OS has a mechanism to switch
between tasks. ● described variously as the PCB (Process
Control Block) or TCB (Task Control Block) in generic literature○ Can be a memory segment, a register, or reserved
region of memory○ x86 calls it the TSS == task state segment○ save registers, state to stack for later restoration
Virtualization crash course
The kernel is "just" a process too, but uses privileged CPU instructions to reschedule itself, service interrupts from hardware, etc.● uses CPU privileged instructions/modes● protects itself with memory segmentation
and/or MMU paging permissions● each process in windows and linux has the
kernel mapped into upper half of address space
Virtualization crash course
Virtualization crash course
Virtual machines ≈ big processesAccordingly, VMMs have a mechanism to switch between VMs● Direct analogue to PCB/TCB/TSS is required
○ Intel calls this a VMCS : Virtual Machine Control Structure
○ AMD calls it a VMCB : Virtual Machine Control Block○ very similar between vendors
● Stores full description of VM, including data that would not be preserved during a process context switch - mode of processor
Virtualization crash course
Inefficiency● Process context switching is expensive.. ● Virtual machine context switching is more soVirtualization can be accomplished in software● emulators (at great cost, performance loss)Most efficient if hardware lends a helping hand● CPU virtualization instructions● AKA "hardware assists"● requirements for efficient virtualization of
CPU architectures have been formalized
Virtualization crash course
Popek and Goldberg requirements● Equivalence / Fidelity
○ A program running under the VMM should exhibit a behavior essentially identical to that demonstrated when running on an equivalent machine directly.
● Resource control / Safety○ The VMM must be in complete control of the
virtualized resources.● Efficiency / Performance
○ A statistically dominant fraction of machine instructions must be executed without VMM intervention.
Virtualization crash course● P&G classify types of CPU instructions
○ Privileged instructions (trap in user mode, not in system mode)
○ Control sensitive instructions (modify system state/mode)
○ Behavior sensitive instructions (those that operate differently according to system state/mode)
● An "effective" VMM achievable for CPUs having privileged instructions ⊂ sensitive instructions
● Recursive VMM possible if the above is met and VMM has no timing dependencies
Virtualization crash course
Popek & Goldberg● Being compliant can be expensive
○ while(1) { Trap && emulate sensitive/privileged instructions }
● E.g., System/370 experiments showed VM running at 21% native speed○ Many "hardware assists" added to avoid software
emulation and perform heavy lifting on the silicon○ doubled perf on System/370 when added
● Not compliant: traditional x86● Compliant: Motorola m68k, modern x86
Virtualization crash course
Popek and GoldbergWhen CPU is not fully compliant, hypervisor / emulator fills in the gaps
○ e.g. emulation is required for supporting some x86 guests (like those in real-mode) even with hardware virtualization
○ real mode assumes exclusive control of system memory/registers, full access
○ Interesting side note: Intel attempted to emulate real mode for its latest chips, and failed (graphical bootloaders hung). Most VMMs emulate in software as a result
Virtualization crash course
Popek and Goldberg● x86 is less-than-pure when it comes to
Pokek-Goldberg adherence● Virtualizing x86 requires handling (correctly,
every time) a very large body of exceptions, oddities, historical oddities, backwards-compatible addressing modes, etc.
● many bugs/vulnerabilities arise from this○ security issues galore
x86 virtualization
Today's x86 families come close to compliance with P&G via hardware assists Intel VT, AMD SVM) Challenges:● hardware that assumes it will only be
initialized once, maintains significant state● hardware memory layout assumptions● memory protection (a major issue)● DMA .. your {video card, network card, ...}
can read/write main memory
x86 Challenges (cont'd)● x86_32 has memory segmentation that
protect memory regions (and prevent code in region X from addressing Y), whereas
● x86_64 has had segmentation registers and semantics largely removed○ (except for fs/gs, which are used in a more general
purpose way by the OS)● MMU context switching can be costly itself
○ multiple levels of virtual memory are at work simultaneously (hypervisor, VM, process)
x86 virtualization
Virtualization challenges on x86
x86 Virtualization Difficulties
● Ring Aliasing● Ring Compression● Memory addressing● Non-faulting guest access to privileged state● 17 instructions do not meet Popek and
Goldberg criteria [Lawton and Robin] ● CPUID instruction
○ information exposure and must be trapped● Addressing these issues
2012
Ring Aliasing
● Ring 0 is most privileged ● OS kernels assume to be
running at ring 0● Our guest VM is no
different● VMM and guest cannot
share ring 0● If guest isn’t in ring 0,
could use PUSH CS to figure that out (CPL is in the last two bits of CS register)2012
Protection Rings for Intel VT
Ring Compression● Ring compression : the merging of privilege
levels. (Weaker separation..)● IA32 supplies two memory isolation
mechanisms: ○ Segmentation and Paging
● Segmentation registers largely removed from 64-bit○ Vestiges not used for memory protection○ Therefore not a protection mechanism for
64-bit virtual environments● 64-bit arch: paging is only choice for
isolating a guest2012
Ring Compression (2)● Problem: paging does not distinguish
between rings 0 - 2○ Intel arch manual Section 5.11.2 “If the
processor is currently operating at a CPL of 0, 1, or 2, it is in supervisor mode; if it is operating at a CPL of 3, it is in user mode.”
● And our host kernel is in ring 0 and guest software is in ring 3. No more rings means we’re compressed○ Therefore, our guest cannot be isolated from
user-space applications and cannot be reasonably protected from each other.
2012
Memory addressing
● OS kernel expects full (linear) virtual address space○ VMM may share this space or exist mostly
in separate address space● Why “mostly?”
○ There are some data structures used to transitions from guest to VMM (structures that need to be protected - VMCS/VMCB)
● Reminder: only protection in 64-bit mode is paging (there is no segmentation)○ This is a critical point
2012
VMWare-style Virtualization (pre x64)
2012
PanSec 2009, Tavis Ormandy, Julien Tinnes
Address-space compression
● Refers to the challenges of protecting virtual-address spaces & supporting access to guest-specific regions
● VMware’s older approach required segment limits; fails on x64 guests. ○ More http://www.pagetable.com/?p=25○ Intel manual: “The virtual machine monitor’s
trap handler must reside in the guest’s address space, because an exception cannot switch address spaces.”
2012
Faulting instructions
● Each faulting instruction causes an exception
● CLI (clear interrupt flag) and STI (set interrupt flag
● A ring 3 guest that calls CLI or STI raises CPU exception that must be handled
● VMMs address these via various mechanisms:○ options: turn these interrupts into virtual
interrupts, trap to VMM, binary translation2012
Non-faulting instructions (1)
● If you wanted to construct a VMM and use fault-then-emulate to virtualize the guest, x86 would fight you
● Some things just don’t fault, and silently fail instead (i.e., POPF, LAR)○ LAR == Load Access Rights
● The POPF instruction is an example of a sensitive instruction which is non-privileged ( one of the 17)
2012
Non-faulting instructions (2)
● Software can also execute the instructions that read (store) from GDT, IDT, LDT, TR using SGDT, SIDT, SLDT, and STR ○ at any privilege level
● If the VMM maintains these registers with unexpected values then clearly the guest can figure that out and violate one of our virtualization criteria
● Can be used explicitly to detect VMM presence○ More on this later (Red Pill)
2012
Instructions That Cause VM Exits Unconditionally● Meaning there should be a timing difference
caused by a VM exit (useful for detection)○ context switch from guest caller to
hypervisor for handling● E.g., CPUID, GETSEC, INVD, and XSETBV● Also true of instructions introduced with Intel
VMX, which include: ○ INVEPT, INVVPID, VMCALL,5
VMCLEAR, VMLAUNCH, VMPTRLD, VMPTRST, VMREAD, VMRESUME, VMWRITE, VMXOFF, and VMXON.
2012
Instructions That Cause VM Exits Conditionally● CLTS, HLT, IN, LGDT, OUT, SGDT and
others● The VMCS can be configured to have
specific instruction executions tocause a trap
● If you can figure out whether the software trying to escape detection will be forced to exit to support a particular pre-existing feature
● Extra reading - Intel manual Section 25.1.32012
Access to privileged state
● privileged instructions in the x86 instruction like LGDT, LIDT
● MOV to CR3, CR0, CR4● For setting paging modes, directing MMU to
particular page tables, etc. Dangerous if unprivileged!
● For example, contention for IDT between guest and host would result in a crash most
likely…
2012
CPUID instruction
● Returns processor identification and feature information
● Thought: When employing virtualization, are there certain undesirable features not to be exposed to the guest?● Some of these features could make the guest
believe it can do things it can’t● Might want to mask off some features from guest
(virtualization)
2012
Handling sensitive/privileged instructionsSoftware-based techniques● Binary translation
○ Emulation of one instruction set by another for same CPU
○ If host==guest instruction set, this is instruction set simulation
○ can be done “just in time” (JIT)○ can do some caching to be more efficient (i.e.,
hot spot detection)● Paravirtualization : guest kernel supports
being virtualized, avoids privileged instr.2012
Handling sensitive/privileged instructionsBinary Translation● Can make privileged instructions such as
POPF less dangerous● Instruction streams are modified on the fly
(think interpreter) to trap offending instruction sequences.
● Two kinds○ static and dynamic translation
2012
Handling sensitive/privileged instructionsStatic Binary Translation● May not be able to have full code coverage
○ Hidden code in data sections could be reached through an indirect jump or jump into the middle of an instruction
○ A problem if code is specifically trying to thwart the binary translation mechanism
2012
Handling sensitive/privileged instructionsStill relevant even with modern CPUsFurther reading● PAYER, M., AND GROSS, T. Requirements
for fast binary translation. In 2nd Workshop on Architectural and Microarchitectural Support for Binary Translation (2009).
● PAYER, M., AND GROSS, T. R. Generating low-overhead dynamic binary translators. In SYSTOR’10 (2010).
2012
Handling sensitive/privileged instructionsHardware Assisted Virtualization● Hardware performs heavy lifting to handle all
these issues (hardware assists, mentioned)● Host (OS, dom0, root partition, VMM, ..) -
whoever is there “first”○ in charge of creating VM “control
structures” (VMCSs)○ Before a guest runs, specify a number of
events/state that cause VM exits○ Think “bitmask” of interrupts to jump out to
VMM land2012
Other x86 virt challenges
Hardware ● Devices often assume a flat memory model● Devices often assume that they are mapped
in a very specific memory region● Legacy BIOS assumptions● New EFI requirements● Trusted computing● DMA and other MMU related fun
2012
x86 Hardware Assisted Virtualization
x86 Hardware Assisted Virtualization
AMD and Intel provide a solution for x86 for virtualizing instruction set● AMD SVM : Secure Virtual Machine● Intel VT-x (also called VT)● reported as svm/vmx in /proc/cpuinfoEach also provides some form of IOMMU ( input/output memory management unit )● Intel VT-d, AMD-Vi● Helps dodge the DMA threat, takes src/dst VM
into account relative to memory address● Extended page tables for translation speedup
Hardware assisted virtualization:new Intel x86 instructions
2012
Intel instruction purposeVMXON Enable VMXVMXOFF Disable VMXVMLAUNCH Start/enter VMVMRESUME Re-enter VMVMCLEAR Null out/reinitialize VMCSVMPTRLD Load the current VMCSVMPTRST Store the current VMCSVMREAD Read values from VMCSVMWRITE Write values to VMCSVMCALL Exit virtual machine to VMMVMFUNC Invoke a VM function in VMM without exiting guest
operation
Hardware Assisted Virtualization
● Guest runs until○ It does something that has been registered in
data structures (i.e., VMCS) to exit out to VMM,
○ It explicitly calls the VMCALL instruction● VMM can preempt guest regularly with on a
timer● VMM can virtualize access to guest’s
memory○ “guest-physical” addresses
● Instructions can cause trap to the VMM2012
Basic Idea
2012
AMD India Developer’s Conference Bangalore, 10-May-2006
VMCS (intel)
VMXON
High level VMX flow
VMM will take these actions● Initially enter VMX mode using VMXON● Clear guest’s VMCS using VMCLEAR● Load guest pointer using VMPTRLD● Write VMCS parameters using VMWRITE● Launch guest using VMLAUNCH● Guest exit (VMCALL or instruction, …)● Read guest-exit info using VMREAD● Maybe reenter guest using VMRESUME● Eventually leave VMX mode using VMXOFF
2012
VMCS Data Organization
Organized into 6 categories1. Guest state2. Host state3. Execution control fields4. Exit control fields5. Entry control6. VM exit info
2012
VMCS Guest State
● Describes the values of the registers the CPU will have after next VMEntry
● Divided into Register and Non-register state● Use of certain fields depends on the “1-
setting” of various VM controls● E.g., 1-setting of EPT control →PDPTE0 – PDPTE3
● VMCS link pointer is unused and reserved to FFFFFFFF_FFFFFFFFh
2012
VMCS Host State Area
● Tells the CPU how to return to the VMM after a VMExit
● Consists of register states and MSRs● Again some of the MSRs depend on “1-setting” of
VM controls
2012
VMCS: Execution Control Fields
● Pin-based (asynchronous) controls● Processor-based (synchronous) controls● Exception bitmap● I/O bitmap addresses● Timestamp Counter offset● CR0/CR4 guest/host masks● CR3 targets● MSR Bitmaps
2012
VM Exits (preface)
● VM exits have significant overhead1. Begin by recording information about the nature of
and reason for the VM exit in the VM-exit information fields (details on this later)
2. Each field in the guest-state area of the VMCS is written with the corresponding component of current processor state.
3. Save guest MSR values
4. Load host state
5. Load host MSRs
● A problem encountered during a VM exit leads to a VMX abort
2012
VMCS: Exit Control Fields
● VM Exits● Occur in response to certain instructions and
events in VMX non-root operation● i.e., what to load and discard in the case of a VM
Exit
● Control fields consist of 2 groups1. VM Exit controls 2. VM Exit controls for MSRs
2012
VM Exit Controls
● 32-bit vector● Save debug controls (bit 2)
● Whether DR7/IA32_DEBUGCTL are saved on VM exit
● Host addr-space size (bit 9)● Whether VM exits occur into 64-bit mode host
● Load IA32_PERF_GLOBAL_CTRL (bit 12)● Whether IA32_PERF_GLOBAL_CTRL is loaded on VM
exit
● Save VMX-preemption timer value (bit 22)● Whether pre-emption timer is saved on exit
2012
VM Exit (1)
● All VM exits load processor state from the host-state area of the VMCS that was the controlling VMCS before the VM exit.● This state is checked for consistency while being
loaded. Because the host-state is checked on VM entry, these checks will generally succeed.
● Failure is possible only if host software is incorrect or if VMCS data in the VMCS region in memory has been written by guest software (or by I/O DMA) since the last VM entry.
2012
VM Exit info
● The VM-exit information fields provide details on VM exits due to exceptions and interrupts. This information is provided through the exit-qualification, VM-exit-interruption-information, instruction-length and interruption-error-code fields
● Great place to hook : e.g., MSR write for system call handler --> VMEXIT
2012
VM Attack Surface
If it can break, it will. People too.
Virtualization Attack Surface
Virt attack surface is the sum of hardware, software, hypervisor and users● co-residence of VMs means that failure of
any portion could bleed through to others● to subvert hypervisor typically requires
deep access to real hardware○ usually requires root-level or kernel-level
control within a VM to accomplish
Breaking out of a guest
The penultimate hack in VM security● Allows access to other guests and/or the
host may not be a goal of breaking into a virtualization environment○ Relative "background noise" generated by
many VMs may obscure the presence of expensive computations, etc.
○ Bitcoin mining hiding in the average per-vm cpu load? Many ways to abuse this sort of environment
(Un?)fortunate VM tendencies
VMs are often less protected than physical systems (efficiency): ample attack opportunityFor example
● AV often absent - use within a VM is really computationally expensive when 50 VMs are sharing 16 physical CPU cores
● encrypted storage volumes are rare● Shared network fabric. ARP poisoning
VM general security posture
VMs are often less protected than physical systemsFor example● AV often absent - e.g., use within a VM is
really computationally expensive when 50 VMs are sharing 16 physical CPU cores
● encrypted storage volumes are rare
VM attack types
● Denial of Service● Information Disclosure● Privilege escalation● Detection of virtualization environment● Break-out-of-the-guest● Hypervisor insertion ● Vulns in guest/host communication● Bugs in virtual device emulation● Abuse of management layer
VM attack types
● cache monitoring to rebuild and extract private keys
● scheduler side channel● Information leak
○ Amazon AMI ssh keys etc (1000 copies of one VM with same SSH private key)
○ this is also a vulnerability of scale○ non-zeroed memory pages/disk
VM attack types
● Abusing transience of virtual hardware○ MITM via ARP poisoning attack ( because
VM MACs are so easily changed )● SMM attacks ("Ring -2")
○ Similar to bluepilling○ http://www.youtube.com/watch?
v=zMaRAGDUEb0○ http://blog.vodun.org/2010/03/slides-from-
shmoocon.html
Next steps
1. Compromise system2. Detect VMM running the VM3. Exploit VMM vuln4. ????5. Profit
Detecting use of virtualization
Detecting Virtualization
● Godsmith, Lawson proposed detection heuristics [1]○ Functional (behavior or state changes)○ Side-channel (timing variations)
● Point methods○ Processor errata○ VMCall functions/CPUID results○ Look for artifacts in processes, file system, and/or
registry, memory.
○ Look for specific virtual hardware○ Look for specific processor instructions and capabilities○ Skew between real hardware and virtual hardware○ Timing analysis
2012
[1] http://www.matasano.com/research/bh-usa-07-ptacek_goldsmith_and_lawson.pdf
Detecting Virtualization
● entropy● stability of TSC register● Timing analysis: measuring time it takes to run
sensitive instructions like CPUID (anything instruction that causes a VMEXIT), comparing with known hardware
● VM pci bridge and attached device○ vendor ID○ peculiarities of emulated device X vs real
device X● fuzzing for a shared page used by the Os to
communicate with hypervisor2012
Detecting Virtualization
Projects/tools that attempt to detect virt● RedPill● NoPill, and ScoopyNG● ScoopyNG = Scoopy Doo + Jerry● Pafish● trapkitOperating system facilities● ls /sys/hypervisor● cat /proc/cpuinfo
2012
RedPill
● Joanna Rutkowska, 2004 - “Red Pill... or how to detect VMM using (almost) one CPU instruction”● http://www.invisiblethings.org/papers/redpill.html
● Using SIDT (Store Interrupt Descriptor Table Register) instruction to profile the current value in the IDTR
● She had found that the most significant byte of the IDTR had a predictable value in VMWare 4 and VirtualPC, which was different from what it was in an non-virtualized system
2012
Measuring time (1)
Time in virtual machines can be a tricky thing● Idiosyncrasies useful for timing analysis and
profiling● Can run backward● Skips forward and even halt● Must be synchronized regularly or will drift● Unstable pace (ideally "monotonically
increasing"Check out "Timekeeping in VMware Virtual Machines" from vmware - great document2012
http://invisiblethingslab.com/resources/bh07/IsGameOver.pdf (Slide 54)
Measuring time (2)
2012
http://invisiblethingslab.com/resources/bh07/IsGameOver.pdf (Slide 54)
● CPU Tick Counter ● RDTSC instruction ● resolution: number of processor cycles (super high!) ● very accurate, but trivial to cheat!
● TSC offsetting/RDTSC VMExit
● High Precision Event Timer (HPET) and other local timers● Might have a high resolution ● But we can cheat them ● e.g. interrupt interception
Measuring time (3)
● Real Time Clock (RTC)● I/O with RTC device ● resolution: milliseconds (poor) ● relatively easy to cheat (I/O interceptions)
● External clock● e.g. NTP protocol ● resolution: 10 milliseconds (very poor) ● can not be cheated using generic approach – only
attacks against specific implementation
2012
http://invisiblethingslab.com/resources/bh07/IsGameOver.pdf (Slide 54)
TLB Profiling
● Based on the belief that a VMM puts the hardware TLB entries to 0 if it is intercepting an instruction.
● Technique● Detector can watch timing access of a page,
calling a possibly intercepted instruction, and then once again timing access to the same page
● Comparing both results should indicate a longer access time (from an external reference) if there has been an interception.
2012
Memory Virtualization: Virtual TLB
2012
Exploitation
General Hardware VM Based Rootkit
● Virtual Machine Based Rootkit (VMBR)● Start with CPL=0● Allocate some unpaged physical memory
● Ensure no linear mappings to VMM after guest entry
● Move running OS into VMCS
● Intercept access to hardware (IO ports, …)● Communicate to hardware VM rootkit via
sentinel instructions
2012
BluePill/HyperJacking Techniques
● In essence: insertion of a hypervisor under OS● Does not virtualize hardware
● BP’d systems see same hardware before/after
● Early PoCs, BlackHat 2006● BluePill (Rutkowska/Tereshkin @ COSEINC) ,
AMD-v, Windows● Vitriol (Dino Dai Zovi @ Matasano), Intel VT-x,
Mac OS X
● Early academic PoC● SubVirt, Samuel T. King et al
2012
Blue Pill Idea (Simplified)
2012
Vitriol
2012
BH USA 2007. Goldsmith and Lawson
NewBluePill
● Created for a Black Hat training session● Copyright terms are limiting (reproduced
below)● https://bluepillstudy.googlecode.
com/svn/trunk/nbp-0.32-public/
2012
; Copyright holder: Invisible Things Lab; ; This software is protected by domestic and International; copyright laws. Any use (including publishing and; distribution) of this software requires a valid license; from the copyright holder.;; This software is provided for the educational use only; during the Black Hat training. This software should not; be used on production systems.
Keylogging in VMBR
● Setup VMCS appropriately● Determine the keyboard’s IO ports● Intercept IO port access and handle/reinject the event
to the guest VM.● Keyboard i/o handler can send events out via network or
stored in a memory region within a malicious process
● See Hyperdbg● https://code.google.
com/p/hyperdbg/source/browse/trunk/hyperdbg/keyboard.c
2012
bpknock
2012
bpknock
2012
bpknock
2012
Anatomy of an attack
● Intel SYSRET silicon bug○ http://blog.xen.org/index.php/2012/06/13/the-
intel-sysret-privilege-escalation/○ Affects hypervisors AND operating systems
● Hardware bug: trap & emulate cannot be guaranteed to handle this class of bug
● System call IN GUEST can cause escalation of privilege
● Fast system call instruction on Intel processors SYSENTER/SYSRET
● OS/Hypervisor on SYSRET/vmenter returns to user-controlled addr, CPL=0
Other notable examples
● VMware shared folders traversal/exposure (CORE and IDefense)
● VMware NAT network, device emulation failure (Tim Shelton)
● 'Bitblt'-style bugs in video emulation (Ormandy, Rafal, Kostya)
● QEMU VGA device (Tavis Ormandy)● Xen paravirtual framebuffer (Rafal Wojtczuk)● VMware Cloudburst-related bugs (Kostya
Kortchinsky)
Other notable examples (cont'd)● VirtualPC privileged instr decoder failure,
facilitates priv escalation CVE-2009-1542● VMWare (various) Trap flag emulation failure,
priv escalation CVE-2008-4915● VMware PF exception handling in real mode,
priv escalation CVE-2009-2267● VMware DOS+crash VMSA-2009-0005● KVM hypercall abuse (write access to memory)
CVE-2009-3290● Xen debug register DOS CVE-2007-5906● Windows DOS VM access to '0 page' for *NULL
CVE-2007-1206
Other notable examples (cont'd)● CPU cache inspection: extract private keys
○ www.cs.unc.edu/~reiter/papers/2012/CCS.pdf
● communication via VMM side channel○ http://www.idontplaydarts.com/2012/08/data-
exfiltration-through-the-vmware-hypervisor/○ albeit at 4 bits per second
● Cloud mapping, deterministic co-residence○ "Hey, You, Get Off of My Cloud:Exploring
Information Leakage in Third-Party Compute Clouds" [Ristenpart]
○ "Detecting Co-Residency with Active Traffic Analysis Techniques" [Bates]
Improving VM security
Possibilities● much more flexible isolation of resources
is possible than in the physical world○ provided a break-out-of-the-guest scenario is
avoided)● Policy frameworks to allow role based
access control and privilege grant/restrictions exist○ KVM : selinux○ Xen : Xen Security Modules (XSM) -
extremely similar to selinux
Reflections on VM security
● VM security not really distinct from normal security○ sharing of resources (and resultant
aggregation of risk) causes the stakes to be much higher.
● Obsession with treating a VM as 1:1 with a physical system == wasted opportunities○ Instead, run a bare process within a VM
container. isolate risk.● Proof of concept attacks are valid (bluepill,
Xen compromise), but are terribly contrived. SYSRET == most personally terrifying
Questions?
Some portions copyright David Weinstein
License: Creative Commons: Attribution, Share-Alike (http://creativecommons.org/licenses/by-sa/3.0/)
http://opensecuritytraining.info/AdvancedX86-VTX.html