virtual hierarchies to support server consolidation mike marty mark hill university of...

Virtual Hierarchies to Support Server Consolidation

Mike MartyMark Hill

University of Wisconsin-Madison

ISCA 2007

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation

64-core CMP

Motivation: Server Consolidation

www server

database server #1

database server #2

middleware server #1

L2 Cache

64-core CMP

www server

database server #1

database server #2

64-core CMP

www server

database server #1

database server #2

Optimize Performance

64-core CMP

www server

database server #1

database server #2

Isolate Performance

64-core CMP

www server

database server #1

database server #2

Dynamic Partitioning

64-core CMP

www server

database server #1

database server #2

Inter-VM Sharing

VMWare’s Content-based Page Sharing Up to 60% reduced memory

Executive Summary

Motivation: Server Consolidation• Many-core CMPs increase opportunities

Goals of Memory System:• Performance• Performance Isolation between VMs• Dynamic Partitioning (VM Reassignment)• Support Inter-VM Sharing• Hypervisor/OS Simplicity

Proposed Solution: Virtual Hierarchy• Overlay 2-level hierarchy on physically-flat CMP• Harmonize with VM Assignment

Outline

Motivation• Server consolidation• Memory system goals• Non-hierarchical approaches

Virtual Hierarchies

Evaluation

Conclusion

TAG-DIRECTORY

duplicate tagdirectory

A Read A

TAG-DIRECTORY

getM A

1fwd data3

Read AA

STATIC-BANK-DIRECTORY

getM A

A Read A

getM A

A Read A

with hypervisor-managed cache

Isolate Performance

Allow Dynamic Partitioning

Support Inter-VM Sharing

Hypervisor/OS Simplicity

TAG-DIRECTORY

STATIC-BANK-DIRECTORY w/ hypervisor-managed cache

Outline

Motivation

Virtual Hierarchies

Evaluation

Conclusion

Virtual Hierarchies

Key Idea: Overlay 2-level Coherence Hierarchy on CMP

- First level harmonizes with VM/Workload

- Second level allows inter-VM sharing, migration, reconfig

VH: First-Level Protocol

Intra-VM Directory Protocol w/ interleaved directories

Questions: • How to name directories?• How to name sharers?

Dynamic home tile selected by VM Config Table• Hardware VM Config Table at each tile• Set by hypervisor during scheduling

Full bit-vector to track any possible sharer• Intra-VM broadcast also possible

INVgetM

VH: First-Level Protocol

Example:

Hypervisor/OS can freely change VM Config Table • No cache flushes• No atomic updates• No explicit movement of directory state

Address……000101

Home Tile: p14offset

VM Config Table

p12p13p14

63 p12

p12p13

L2 Cache

per-Tile

Dynamic

Virtual Hierarchies

Two Solutions for Global Coherence: VHA and VHB

memory controller(s)

Protocol VHA

Directory as Second-level Protocol• Any tile can act as first-level directory• How to track and name first-level directories?

Full bit-vector of sharers to name any tile• State stored in DRAM• Possibly cache on-chip

+ Maximum flexibility

- DRAM State

- Complexity

VHA Example

getM A

Protocol VHB

Broadcast as Second-level Protocol

Attach token count for each block [token coherence]• T tokens for each block. One token to read, all to write• Allows 1-bit at memory per block• Eliminates system-wide ACK

+ Minimal DRAM State

+ Enables easier optimizations

- Global coherence requires more activity

VHB Example

ergetM A

3global getM A

getM A

Data+tokens

Memory System Goals

Isolate Performance

Allow Dynamic Partitioning

Support Inter-VM Sharing

Hypervisor/OS Simplicity

VHA and VHB

Outline

Motivation

Virtual Hierarchies

Evaluation

Conclusion

Evaluation: Methods

Wisconsin GEMS• Full-system, execution-driven simulation• Based on Virtutech Simics• http://www.cs.wisc.edu/gems

64-core tiled CMP• In-order SPARC cores• 512 KB, 16-way L2 cache per tile• 2D mesh interconnect, 16-byte links, 5-cycle link latency• Four on-chip memory controllers, 275-cycle DRAM latency

Evaluation: Methods

Workloads: OLTP, SpecJBB, Apache, Zeus• Separate instance of Solaris for each VM

Approximating Virtualization• Multiple Simics checkpoints interleaved onto CMP• Assume workloads map to adjacent cores• Bottomline: No hypervisor simulated

Evaluation: Protocols

TAG-DIRECTORY:• 3-cycle central tag directory (1024 ways!)

• Home tiles interleaved by frame address

• All data allocates in L2 bank of dynamic home tile

• Unshared data always allocates in local L2

All Protocols: one L2 copy of block on CMP

Result: Runtime

OLTP Apache

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

Zeus SpecJBB

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

Eight VMs x Eight Cores Each = 64 Cores(e.g. eight instances of Apache)

Result: Memory Stall Cycles

Off-chip Local L2 Remote L1 Remote L2

OLTP Apache

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

Zeus SpecJBB

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

Eight VMs x Eight Cores Each = 64 cores(e.g. eight instances of Apache)

Executive Summary

Server Consolidation an Emerging Workload

Goals of Memory System:• Performance• Performance Isolation between VMs• Dynamic Partitioning (VM Reassignment)• Support Inter-VM Sharing• Hypervisor/OS Simplicity

Proposed Solution: Virtual Hierarchy• Overlay 2-level hierarchy on physically-flat CMP• Harmonize with Workload Assignment

Backup Slides

Omitting 2nd-Level Coherence: Protocol VH0

Impacts:• Dynamic Partitioning• Inter-VM Sharing (VMWare’s Content-based Page Sharing)• Hypervisor/OS complexity

Example: Steps for VM Migration from Tiles {M} to {N}

1. Stop all threads on {M}

2. Flush {M} caches

3. Update {N} VM Config Tables

4. Start threads on {N}

Omitting 2nd-Level Coherence: Protocol VH0

Example: Inter-VM Content-based Page Sharing• Up to 60% reduced memory demand

Is read-only sharing possible with VH0?

VMWare’s Implementation:• Global hash table to store hashes of pages• Guest pages scanned by VMM, hashes computed• Full comparison of pages on hash match

Potential VH0 Implementation:• How does hypervisor scan guest pages? Are they modified

in cache? • Even read-only pages must initially be written at some point

Physical Hierarchy / Clusters

Shared L2 Shared L2

Physical Hierarchy / Clusters

Shared L2 Shared L2

www server

database server #1

virtual hierarchies to support server consolidation mike marty mark hill university of...

middleware server

server consolidationmotivation

virtual hierarchyoverlay

vm assignmentmike marty

dynamic partitioningmike

isolate performancemike

reduced memorymike marty

flat cmpharmonize

Documents

isca amendments

isbn - isca

big hierarchies, little hierarchies, and flat directions ......

isca junho - 2011

ca final isca

isca 11 fire

varna swara - isca

isca presentation.key

(c) 2005 multifacet project isca tutorial june 5 th, 2005...

isca - setembro 2011

type hierarchies

l'oasi di isca

information systems capabilities assessment (isca) mental...

isca general assembly

isca notes

osp isca 2013

isca roralpresentation20140930

obiee hierarchies

isca 2019 rulebook

isca-ca final