Virtual Machine TechniquesVirtual Machine Techniquesand their Emerging Applicationsand their Emerging Applications

Wei HsuWei Hsu8/23/20068/23/2006

OutlineOutline

Introduction and VM overviewIntroduction and VM overview Emulation: Interpretation and Binary Emulation: Interpretation and Binary

TranslationTranslation Process VMs and Dynamic TranslatorProcess VMs and Dynamic Translator HLL VMsHLL VMs Co-Designed VMsCo-Designed VMs Emerging Applications of VMsEmerging Applications of VMs

Introduction and VM overviewIntroduction and VM overview

Why are virtual machines interesting?– They allow transcending of standard interfaces

(which often seem to be an obstacle to innovation)– They enable innovation in flexible, adaptive

software & hardware, security, network computing (and others)

– They involve computer architecture in a pure sense Virtualization will be a key part of future

computer systems– A fourth major discipline? (with HW, System SW,

Application SW)

AbstractionAbstractionComputer systems are built on levels of abstraction

Higher level of abstraction hide details at lower levels

Example: files are anabstraction of a disk

Machines (defined by ISA)Machines (defined by ISA)For OS developers, amachine is defined by ISA (Instruction Set Architecture)

Major division between Hardware and software

Machines (defined by ABI)Machines (defined by ABI)For Compiler developers, a machine is defined by ABI (Application Binary Interface)

ABIUser ISA + OS Calls

Machines (defined by API)Machines (defined by API)For Application developers, a machine is defined by API (Application Binary Interface)

APIUser ISA + Library Calls

Virtual MachinesVirtual Machines

Add Virtualizing Software to a Host platform and support Guest process or system on a Virtual Machine (VM)

System Virtual MachinesSystem Virtual Machines

Provide a system environment

Constructed at ISA level

Persistent

Examples:

IBM VM/360, VMware, Transmeta Crusoe Virtual network communication

Process Virtual MachinesProcess Virtual Machines Execute applications with an ISA different from the

HW platform Couple at ABI level via runtime system Not persistent

Process Virtual Machines (cont)Process Virtual Machines (cont) Guest processes may intermingle with host Guest processes may intermingle with host

processesprocesses As a practical matter, guest OS and host OS As a practical matter, guest OS and host OS

are often the sameare often the same Same ISA dynamic optimizer is a special Same ISA dynamic optimizer is a special

casecase Example:Example:

IA-32 EL, FX!32, Aries, DynamoIA-32 EL, FX!32, Aries, Dynamo

HLL Virtual MachinesHLL Virtual Machines Java and CLI are current examples Binary class files are distributed “ISA” is part of binary class format OS interaction via API (part of VM platform)

Co-designed Virtual MachinesCo-designed Virtual Machines

Perform both translation and optimization VM provides interface between standard ISA software and implementation ISA software Primary goal is performance or power efficiency Use proprietary implementation ISA Example: Transmeta Crusoe and IBM Daisy

TaxonomyTaxonomy

EmulationEmulation

A method for enabling a (sub)system to A method for enabling a (sub)system to present the same interface and present the same interface and characteristics as anothercharacteristics as another

Ways of implement emulationWays of implement emulation– Interpretation: instruction at-a-timeInterpretation: instruction at-a-time– Binary translation: block-at-a-time optimized for Binary translation: block-at-a-time optimized for

repeated instruction executionsrepeated instruction executions

Decode-Dispatch InterpretationDecode-Dispatch Interpretationwhile (!halt) {

inst = code(PC);

opcode = extract(inst,31,6);

switch(opcode) {

case LdWord: LdWord(inst);

case ALU: ALU(inst);

case Branch: Branch(inst);

. . .}

Instruction Function: Load WordInstruction Function: Load WordLdWord(inst){

RT = extract (inst,25,5);RA = extract (inst,20,5);displacement =extract (inst,15,16);source = regs[RA];address = source + displacement ;regs[RT] = data[address];PC = PC + 4;

}

Binary TranslationBinary Translation

Generate custom code for every source Generate custom code for every source instruction. For example, a load instruction instruction. For example, a load instruction in source code could be translated into a in source code could be translated into a respective load instruction in native code.respective load instruction in native code.

Get rid of repeated parsing, decoding, and Get rid of repeated parsing, decoding, and jumping overhead.jumping overhead.

Register mapping is needed to reduce Register mapping is needed to reduce load/stores significantly.load/stores significantly.

Register MappingRegister Mapping

Easier if Easier if

No. of target registers > no. of source No. of target registers > no. of source registers. (e.g. translating x86 binary to registers. (e.g. translating x86 binary to RISC)RISC)

May be on a per-block, or per-trace, basisMay be on a per-block, or per-trace, basis

If no. of target registers not enoughIf no. of target registers not enough Infrequently used registers may not be Infrequently used registers may not be

mapped mapped

Binary Translation ExampleBinary Translation Examplex86 Source Binary

addl %edx,4(%eax)movl 4(%eax),%edxadd %eax,4

PowerPC Target

addi r16,r4,4 ; add 4 to %eaxlwzx r17,r2,r16 ; load operand from memoryadd r7,r17,r7 ; perform add of %edxstwx r7,r2,r16 ; store %edx value into memorymr r4,r16 ; move update value into %eaxaddi r3,r3,9 ; update PC (9 bytes)

r2 points to x86 imager3 points to x86 PCr4 holds %eaxr7 holds %edx

Code Discovery ProblemCode Discovery Problem In order to translate, emulator must be able In order to translate, emulator must be able

to discover code.to discover code.

Easier said than done, especially w/ x86Easier said than done, especially w/ x86

Dynamic TranslationDynamic Translation First InterpretFirst Interpret

– And perform code discovery as a byproductAnd perform code discovery as a byproduct

Translate CodeTranslate Code– Incrementally, as it is discoveredIncrementally, as it is discovered– Place translated blocks into Code CachePlace translated blocks into Code Cache– Save source to target PC mapping in lookup tableSave source to target PC mapping in lookup table

Emulation processEmulation process– Execute translated block to endExecute translated block to end– Lookup next source PC in tableLookup next source PC in table

If translated jump to target PCIf translated jump to target PC Else interpret and translateElse interpret and translate

Flow of ControlFlow of Control Control flows between translated block and Control flows between translated block and

emulation manageremulation manager

EmulationManager

Translation Block

Translation Block

Translation Block

Translation ChainingTranslation Chaining Jump from one translation directly to nextJump from one translation directly to next

– Avoid switching back to emulation managerAvoid switching back to emulation manager

EmulationManager

Translation Block

Translation Block

Translation Block Translation Block

Software Jump PredictionSoftware Jump Prediction

Form of “Inline Caching”Example Code:Say Rx holds source branch addressaddr_i are predicted addresses (in probability

order) determined via profilingtarget_i are corresponding target code blocks

If Rx == addr_1 goto target_1Else if Rx == addr_2 goto target_2Else if Rx == addr_3 goto target_3Else hash_lookup(Rx) ; do it the slow way

Source/Target ISA IssuesSource/Target ISA Issues Register architecturesRegister architectures

– Register mappings, reservation of special registersRegister mappings, reservation of special registers Condition codesCondition codes

– Lazy evaluation as neededLazy evaluation as needed Data formats and operationsData formats and operations

– Floating pointFloating point– DecimalDecimal– MMXMMX

Address resolutionAddress resolution– Byte vs word addressingByte vs word addressing

Address AlignmentAddress Alignment– Natural vs arbitraryNatural vs arbitrary

Byte orderByte order– Big/little endianBig/little endian

Process Virtual MachinesProcess Virtual Machines Perform guest/host mapping at ABI level Encapsulate guest process in process-level runtime Issues

• memory architecture•Exception architecture•OS call emulation•Overall VM architecture•High performance implementation•System environments

Runtime ComponentsRuntime Components InitializationInitialization

– Allocate memoryAllocate memory– Initialize runtime data structures and all signalsInitialize runtime data structures and all signals

Code Cache ManagerCode Cache Manager– Implement replacement algorithmImplement replacement algorithm– Flush when required (e.g. self-mod)Flush when required (e.g. self-mod)

OS Call EmulatorOS Call Emulator– Translate OS calls and OS responsesTranslate OS calls and OS responses

Exception EmulatorException Emulator– Handle signalsHandle signals

If registered by source code, pass to source handlerIf registered by source code, pass to source handler If not registered, emulate host responseIf not registered, emulate host response

– Form precise stateForm precise state

Memory Architecture EmulationMemory Architecture Emulation

Memory architecture emulation is challenging when the target machine has a very different memory architecture than the host machine

It may take many instructions to accurately emulate the memory protection mechanism in the target machine

This emulation could be sped up by using software TLB techniques.

Software Memory MappingSoftware Memory Mapping

Runtime Software to maintain mapping table

similar to hardware page table/TLB. Slow, but can always be made to work

Hardware MappingHardware Mapping VM software mapping is slowVM software mapping is slow Use underlying hardware Use underlying hardware

– If guest address space + runtime fit within host spaceIf guest address space + runtime fit within host space

Guest Memory ProtectionGuest Memory Protection

Runtime must be protected from guest processRuntime must be protected from guest process VM software mapping can be easily usedVM software mapping can be easily used

– Place (and check) protection in mapping tablePlace (and check) protection in mapping table

Better: use underlying hardwareBetter: use underlying hardware– Runtime must be able to set privilegesRuntime must be able to set privileges– Protection faults should be reported to runtimeProtection faults should be reported to runtime

(so it can respond as guest OS would)(so it can respond as guest OS would)– Requires some support from host OSRequires some support from host OS

E.g. a system call to set protection levelE.g. a system call to set protection level A signal A signal mechanism where protection faults trap

to handler in runtime

Self-Modifying CodeSelf-Modifying Code

Write-Protect original code to catch code Write-Protect original code to catch code self-modificationself-modification

Exception handler invalidates old translationException handler invalidates old translation Be sure to make Be sure to make forwardforward progress progress Pseudo self-modifying code may require Pseudo self-modifying code may require

optimizationsoptimizations– Data mixed with codeData mixed with code– Implement software-supported fine-grain checkImplement software-supported fine-grain check

Sometimes can rely on source binary to Sometimes can rely on source binary to indicate self-modification (e.g. SPARC flush)indicate self-modification (e.g. SPARC flush)

Self-Reference CodeSelf-Reference Code

Original copy is maintained by translatorOriginal copy is maintained by translator All reads are with respect to original copy – All reads are with respect to original copy –

correct data is returned.correct data is returned. Code Patching approach, such as Adore, Code Patching approach, such as Adore,

would not work for SRC.would not work for SRC.

Exceptions: InterruptsExceptions: Interrupts Application may register some interrupts

• Precise state easier than traps(because there is more flexibility wrt location)

Problem: Translated blocks may executed for an unbounded time period

Solution:• Interrupt signal goes to runtime• Runtime unchains translation block currently executing (eliminates loops)• Runtime returns control to current translation• Translation soon reaches end (and precise state is available)

If interrupts are common, runtime may inhibit all chaining

Exceptions: TrapsExceptions: Traps Can be detected directly via interpretation

• or explicit translated code checks

Can be detected indirectly via target ISA trap and signal• Runtime registers all trap conditions as signals

Semantic “matching”• If trap is architecturally similar in target and source then trap/signal may be used• Otherwise interpretive method must be used

Generally, more difficult than interrupts wrt precise state

Precise State: Program CounterPrecise State: Program Counter Interpretation: Easy – source PC is maintainedInterpretation: Easy – source PC is maintained Binary Translation: more difficult, source PC is only Binary Translation: more difficult, source PC is only

available at translation block boundariesavailable at translation block boundaries– Trap PC is in terms of target codeTrap PC is in terms of target code– Target PC must be mapped back to correct source PCTarget PC must be mapped back to correct source PC

SolutionSolution– Use side table and reverse translateUse side table and reverse translate– Can be combined with PC mapping tableCan be combined with PC mapping table– Requires search of table to find trapping blockRequires search of table to find trapping block– Reconstruct block translation to identify specific source Reconstruct block translation to identify specific source

PC of trapping instruction.PC of trapping instruction.

PC Side TablePC Side Table

Recovering statesRecovering states Recovering Register States

– Simple if target code update register state in the same order as source code

– Register state mapping can be used to generate register state values

– More difficult if optimizations reorder code Recovering Memory States

– Simple if target code updates memory in the same order as source code

– Restrict optimizations (more difficult to back-up than register states)

– Most process VM maintain the same store order

OS Call EmulationOS Call Emulation“Wrapper” or “Jacket” code converts source calls to target

calls

Code Cache ManagementCode Cache Management Code Cache is different from typical hardware

cache• Variable sized blocks• Dependences among blocks due to linking• No “backing store”; re-generating is expensive

These factors affect replacement algorithm • LRU replacement is typically not used(fragmentation problem) • Flush when full • Pre-emptive Flush

(flush when phase changed)

Example: FX!32Example: FX!32 x86/Windows ABIs on Alpha/Windows Runtime software

• Follows typical model• But, translations/optimizations are done between executions

First execution of binary: interpret and profileTranslate and optimize “off line”Later execution(s): use translated version, continue profiling

Persistence• Translations and profile data are saved on disk between runs

Very time consuming optimization with x86 sourceHybrid static/dynamic binary translation

Performance of FX!32Performance of FX!32 (comparing 200 MHz Pentium Pro and 500 MHz 21164) Goal: same as high-end x86 Byte benchmark integer ≈ 40% faster than Pentium Pro Floating point ≈ 30% slower than Pentium Pro Achieves 70% of native alpha performance Only certified applications are supported by FX!32

Example: Intel IA32 ELExample: Intel IA32 EL Software method for running IA-32 binaries

on IPF• Previous approach was in hardware (iVE)

Runs with both Windows and Linux• OS independent section (BTgeneric)• OS dependent section (BTlib)

Two stages• Fast binary translation (cold code)• Optimized binary translation (hot code)

Precise traps are an important consideration

HLL VMHLL VM Goal: complete platform independence for applicationsGoal: complete platform independence for applications Similar to Process VMsSimilar to Process VMs

– Major difference is specification level:Major difference is specification level:

VISA + LibrariesVISA + Libraries

instead of ISA + OS interface (ABI)instead of ISA + OS interface (ABI)

UCSD P-CodeUCSD P-Code

Popularized HLL VMsPopularized HLL VMs Provide highly portable version of PascalProvide highly portable version of Pascal Consists ofConsists of

– Primitive librariesPrimitive libraries– Machine-independent object file formatMachine-independent object file format– Stack based ISAStack based ISA– A set of byte-oriented “pseudo codes”A set of byte-oriented “pseudo codes”– Virtual machine definition of pseudo code Virtual machine definition of pseudo code

semanticssemantics

Modern HLL VMsModern HLL VMs Superficially similar to P-code schemeSuperficially similar to P-code scheme

– Stack based ISAStack based ISA– Standard librariesStandard libraries– But, Objective is But, Objective is Application PortabilityApplication Portability, not , not Compiler Compiler

PortabilityPortability Network computing environmentNetwork computing environment

– Untrusted softwareUntrusted software– Robustness (object programming)Robustness (object programming)– Bandwidth is a considerationBandwidth is a consideration– Good performance must be maintainedGood performance must be maintained

Java VM and MS CLIJava VM and MS CLI

Co-Designed VMsCo-Designed VMs Design hardware and VM Design hardware and VM

software concurrently and software concurrently and cooperativelycooperatively

Use proprietary target ISAUse proprietary target ISA– Or modified ISAOr modified ISA

No native OS or No native OS or applicationsapplications

Goal is performance or Goal is performance or power efficiencypower efficiency– Not compatibilityNot compatibility

Techniques also applicable Techniques also applicable to HW support for other to HW support for other VMsVMs

Concealed MemoryConcealed Memory

VM software resides in memory concealed from VM software resides in memory concealed from all conventional softwareall conventional software

Precise ExceptionsPrecise Exceptions Traps must be precise wrt original binaryTraps must be precise wrt original binary

– All conventional software is unaware of underlying VMAll conventional software is unaware of underlying VM– Code may undergo heavy duty reorganizationCode may undergo heavy duty reorganization

E.g. CISC E.g. CISC VLIW VLIW

Checkpoint and rollback• Have VMM periodically checkpoint state• Consistent with a point in original binary• On fault, rollback and interpret original binary

In-order state update• Keep out-of-order results in scratch registers, updatearchitected registers in-orderI.e. software renaming

Checkpoint and RollbackCheckpoint and Rollback

Precise interrupt via ckpt/rollbackPrecise interrupt via ckpt/rollback As in Transmeta Crusoe Shadow copies of

registers Gated store buffer Code divided into

translation groups• Precise state between groups

Commit when trans. group is exited• Release gated store buffer• Copy current registers into shadow

Precise interrupt via ckpt/rollbackPrecise interrupt via ckpt/rollback When a trap occurs

• Flush store buffer• Backup with shadow registers• Interpret forward until trap occurs

Advantage:• Larger precise interrupt units=> coarser grain optimizations, dead code elimination, etc.

Disadvantage:• Store buffer size limitstranslation unit size

VM Technology in Future ComputingVM Technology in Future Computing Virtualization enables the introduction of Virtualization enables the introduction of

new system capabilities without adding new system capabilities without adding complexity to already existing and already-complexity to already existing and already-complex HW and SW.complex HW and SW.

Virtual machine technology is poised for Virtual machine technology is poised for pervasive deployment in future computing pervasive deployment in future computing systemssystems

Emerging ApplicationsEmerging Applications Security : intrusion detection systems that isolate users Security : intrusion detection systems that isolate users

from malicious attacksfrom malicious attacks Migration of computing environments: VM state saved as a Migration of computing environments: VM state saved as a

network accessible file.network accessible file. Enhanced reliability through redundant execution of a Enhanced reliability through redundant execution of a

program on multiple VMsprogram on multiple VMs Emulate one ISA on the other, supports the concept of Emulate one ISA on the other, supports the concept of

new computing centernew computing center Allow isolation of data on a system, thereby allowing the Allow isolation of data on a system, thereby allowing the

same physical system to work on both confidential and same physical system to work on both confidential and public datapublic data

Grids: virtual organizations – allowing a systematic sharing Grids: virtual organizations – allowing a systematic sharing of resources between geographically distributed systems.of resources between geographically distributed systems.