virtual hierarchies to support server consolidation mike marty mark hill university of...

Virtual Hierarchies to Support Server Consolidation

Mike MartyMark Hill

University of Wisconsin-Madison

ISCA 2007

Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation

64-core CMP

Motivation: Server Consolidation

www server

database server #1

database server #2

middleware server #1


Core

L2 Cache

L1


64-core CMP


www server

database server #1

database server #2




64-core CMP


www server

database server #1

database server #2



data

data

Optimize Performance


64-core CMP


www server

database server #1

database server #2



Isolate Performance


64-core CMP


www server

database server #1

database server #2



Dynamic Partitioning


64-core CMP


www server

database server #1

database server #2



data

Inter-VM Sharing

VMWare’s Content-based Page Sharing Up to 60% reduced memory


Executive Summary

Motivation: Server Consolidation• Many-core CMPs increase opportunities

Goals of Memory System:• Performance• Performance Isolation between VMs• Dynamic Partitioning (VM Reassignment)• Support Inter-VM Sharing• Hypervisor/OS Simplicity

Proposed Solution: Virtual Hierarchy• Overlay 2-level hierarchy on physically-flat CMP• Harmonize with VM Assignment


Outline

Motivation• Server consolidation• Memory system goals• Non-hierarchical approaches

Virtual Hierarchies

Evaluation

Conclusion


TAG-DIRECTORY

duplicate tagdirectory

A Read A



TAG-DIRECTORY

A

getM A

1fwd data3


2

Read AA


STATIC-BANK-DIRECTORY

getM A

1

2

fwd

data3

A

A Read A



getM A

1

2fwd

data3

A

A Read A

with hypervisor-managed cache


Goals


Isolate Performance

Allow Dynamic Partitioning

Support Inter-VM Sharing

Hypervisor/OS Simplicity

Yes

Yes

?

Yes

No

No

No

Yes

Yes

Yes


TAG-DIRECTORY

STATIC-BANK-DIRECTORY w/ hypervisor-managed cache

No

No

Yes

Yes

Yes


Outline

Motivation

Virtual Hierarchies

Evaluation

Conclusion


Virtual Hierarchies

Key Idea: Overlay 2-level Coherence Hierarchy on CMP

- First level harmonizes with VM/Workload

- Second level allows inter-VM sharing, migration, reconfig


VH: First-Level Protocol

Intra-VM Directory Protocol w/ interleaved directories

Questions: • How to name directories?• How to name sharers?

Dynamic home tile selected by VM Config Table• Hardware VM Config Table at each tile• Set by hypervisor during scheduling

Full bit-vector to track any possible sharer• Intra-VM broadcast also possible

INVgetM


VH: First-Level Protocol

Example:

Hypervisor/OS can freely change VM Config Table • No cache flushes• No atomic updates• No explicit movement of directory state

Address……000101

Home Tile: p14offset

6

VM Config Table

p12p13p14

012

63 p12

p12p13

p14

345

p13

p12

p14

Core

L2 Cache

L1

per-Tile

Dynamic


Virtual Hierarchies

Two Solutions for Global Coherence: VHA and VHB

memory controller(s)


Protocol VHA

Directory as Second-level Protocol• Any tile can act as first-level directory• How to track and name first-level directories?

Full bit-vector of sharers to name any tile• State stored in DRAM• Possibly cache on-chip

+ Maximum flexibility

- DRAM State

- Complexity


VHA Example

dir

ecto

ry/m

emo

ry c

on

tro

ller

getM A

1

2

data6

A

AFwd

data

4

3

getM A

5

Fwd

data

A


Protocol VHB

Broadcast as Second-level Protocol

Attach token count for each block [token coherence]• T tokens for each block. One token to read, all to write• Allows 1-bit at memory per block• Eliminates system-wide ACK

+ Minimal DRAM State

+ Enables easier optimizations

- Global coherence requires more activity


VHB Example

mem

ory

co

ntr

oll

ergetM A

1

2

3global getM A

getM A

Data+tokens

4

A

A


Memory System Goals


Isolate Performance

Allow Dynamic Partitioning

Support Inter-VM Sharing

Hypervisor/OS Simplicity

VHA and VHB

Yes

Yes

Yes

Yes

Yes


Outline

Motivation

Virtual Hierarchies

Evaluation

Conclusion


Evaluation: Methods

Wisconsin GEMS• Full-system, execution-driven simulation• Based on Virtutech Simics• http://www.cs.wisc.edu/gems

64-core tiled CMP• In-order SPARC cores• 512 KB, 16-way L2 cache per tile• 2D mesh interconnect, 16-byte links, 5-cycle link latency• Four on-chip memory controllers, 275-cycle DRAM latency


Evaluation: Methods

Workloads: OLTP, SpecJBB, Apache, Zeus• Separate instance of Solaris for each VM

Approximating Virtualization• Multiple Simics checkpoints interleaved onto CMP• Assume workloads map to adjacent cores• Bottomline: No hypervisor simulated


Evaluation: Protocols

TAG-DIRECTORY:• 3-cycle central tag directory (1024 ways!)


• Home tiles interleaved by frame address

VHA

• All data allocates in L2 bank of dynamic home tile

VHB

• Unshared data always allocates in local L2

All Protocols: one L2 copy of block on CMP


0

0.2

0.4

0.6

0.8

1

1.2

1.4

No

rma

lize

d R

un

tim

e

Result: Runtime

OLTP Apache

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

Zeus SpecJBB

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

Eight VMs x Eight Cores Each = 64 Cores(e.g. eight instances of Apache)


Result: Memory Stall Cycles

0

0.2

0.4

0.6

0.8

1

1.2

1.4

No

rma

lize

d M

em

ory

Sta

ll C

yc

les

Off-chip Local L2 Remote L1 Remote L2

OLTP Apache

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

Zeus SpecJBB

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

TAG-DIR

STATIC-B

ANK-DIR

VH AVH B

Eight VMs x Eight Cores Each = 64 cores(e.g. eight instances of Apache)


Executive Summary

Server Consolidation an Emerging Workload

Goals of Memory System:• Performance• Performance Isolation between VMs• Dynamic Partitioning (VM Reassignment)• Support Inter-VM Sharing• Hypervisor/OS Simplicity

Proposed Solution: Virtual Hierarchy• Overlay 2-level hierarchy on physically-flat CMP• Harmonize with Workload Assignment


Backup Slides


Omitting 2nd-Level Coherence: Protocol VH0

Impacts:• Dynamic Partitioning• Inter-VM Sharing (VMWare’s Content-based Page Sharing)• Hypervisor/OS complexity

Example: Steps for VM Migration from Tiles {M} to {N}

1. Stop all threads on {M}

2. Flush {M} caches

3. Update {N} VM Config Tables

4. Start threads on {N}


Omitting 2nd-Level Coherence: Protocol VH0

Example: Inter-VM Content-based Page Sharing• Up to 60% reduced memory demand

Is read-only sharing possible with VH0?

VMWare’s Implementation:• Global hash table to store hashes of pages• Guest pages scanned by VMM, hashes computed• Full comparison of pages on hash match

Potential VH0 Implementation:• How does hypervisor scan guest pages? Are they modified

in cache? • Even read-only pages must initially be written at some point


Physical Hierarchy / Clusters

PL1 $

PL1 $

PL1 $

PL1 $

PL1 $

PL1 $

PL1 $

PL1 $

Shared L2 Shared L2

PL1 $

Shared L2 Shared L2

PL1 $

PL1 $

PL1 $

PL1 $

PL1 $

PL1 $

PL1 $


Physical Hierarchy / Clusters

PL1 $

PL1 $

PL1 $

PL1 $

PL1 $

PL1 $

PL1 $

PL1 $

Shared L2 Shared L2

PL1 $

Shared L2 Shared L2

PL1 $

PL1 $

PL1 $

PL1 $

PL1 $

PL1 $

PL1 $

www server

database server #1


virtual hierarchies to support server consolidation mike marty mark hill university of...

Documents

middleware server

server consolidationmotivation

virtual hierarchyoverlay

vm assignmentmike marty

dynamic partitioningmike

isolate performancemike

reduced memorymike marty

flat cmpharmonize