Kaleidoscope: Cloud Micro-Elasticity via VM State ColoringRoy Bryant*, Alexey Tumanov†, Olga, Irzak*, Adin Scannell*, Kaustubh Joshi§, Matti Hiltunen§, H. Andrés Lagar-Cavilla§, Eyal de Lara**University of Toronto, †Carnegie Mellon University, §AT&T Labs Research

• Long instantiation times• Coarse granularity of footprint• Network bandwidth & latency overhead

• Common practice: dedicated migration bandwidth!• Wide-range variability in instantiation times

• Fresh workers → cold state• Number of servers with runtime > month – doubled

• Strong need for elasticity• 15.3% of peak capacity –

average usage• Wide-range short-term

variability in workload• Elastic workers – short lived

• 85% of workers – live < 1hr• 10 min → mean worker lifetime

• Allocate resources proportionally to the working set• Stateful replication of VMs (inherit expensive warm state)• Color-guided state propagation

• proactive prefetch• reactive (on-demand)

• Continuum between full copy and minimal/on demand copy

1. VM memory state coloring• Semantically-aware state propagation• Architectural & introspective runtime memory state

information2. Time & space efficient implementation of coloring3. Micro-elastic cloud server

• Color-aware replication & sharing• Near-ideal post-clone QoS• Resource consumption proportional to demand

4. Real world datacenter savings• Fundamental tradeoff:

• Instantiation time vs. warmup period• Continuum between lazy and

eager replication• Continuum between minimal

and maximal footprint• Architecture-based coloring

• Performed on translated page table pages undergoing mandatory canonicalization at clone time

• Extracting the NX bit (executable)• Identifying the kernel/user-space split

• Introspective coloring• Performed on a "frozen" memory image of the master• File cache radix trees• Frame page structure: used/unused physical pages

• State prefetch of consecutive pages in color space• Significant reduction in bandwidth consumption

• Per-color differentiation in the width of the prefetch window• Insight: reduced window size for executable state• User/kernel data windows : 3-4 times wider than executable

• Color-directed memory deduplication• Per-color probabilities of inter-VM page similarities• One-time, color guided calculation of hashes for the master

memory state • Definition: ability to consume physical resources at sub-VM granularity, tracking effective resource demand• Physical host memory• Network bandwidth

• Description: Kaleidoscope workers allocate memory as necessary, an advantage when spikes are short lived.

• Blocking time reduction• Elastic footprint / resource preservation• Scalability• Runtime• QoS

Freq

uenc

y

200018001600140012001000800600400200

0 0 10 20 30 40 50 60

100.00%

80.00%

60.00%

40.00%

20.00%

0.00%

Clone Lifetime (min)

FrequencyCumulative %

footprintmax

SnowFlock

lazy

min

livemigration

replication

eagerKaleidoscope

0

0.2

0.4

0.6

0.8

1.0

0 10 20 30 40 50 60 70 80Frac

tion

of A

lloca

tion

Use

d

Time (seconds)

SupportBanking

EcommerceHttperfOLAP

0

0.2

0.4

0.6

0.8

1

0 20 40 60 80 100 120

Frac

tion

Spen

t Blo

cked

Time (seconds)

0

0.5

1

1.5

2

2.5

3

3.5

4

0 1 2 3 4 5 6 7 8

OLA

P T

hrou

ghpu

t(q

uerie

s pe

r sec

ond)

Simultaneous New Workers

KaleidoscopeCold Standby

Current Elasticity Models: Issues

0 5

10 15 20 25 30 35 40 45

Support Ecommerce Banking

SP

EC

web

"Deg

rade

d" D

urat

ion

(sec

onds

)

Cold StandbyMinimal Clone

Conservative CloneAggressive Clone

KaleidoscopeWarm Static

Kaleidoscope Design Space

Uses of Color

Evaluation Categories

Motivation

Kaleidoscope Contributions

VM Memory State Coloring

Fractional Footprint

Evaluation Results

Kaleidoscope Vision: Cloud u-elasticity

0

20

40

60

80

100

120

140

OLAP Httperf

Run

Tim

e (s

econ

ds)

Cold StandbyMinimal Clone

Conservative CloneAggressive Clone

KaleidoscopeBlocked

Warm Static

0

50

100

150

200

250

300

Httperf Support Ecommerce Banking OLAP

Num

ber o

f Unf

etch

ed

Pag

es (,

000s

)

FreeFile System Cache

User DataKernel DataUser Code

Kernel Code

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 25 30 35 40

Frac

tion

Spe

nt B

lock

ed

Time (seconds)

Kaleidoscope Worker Blocked for File System CacheUser Data

Kernel DataUser Code

Kernel CodeFree

Reduction of page faults and blocking Near-optimal QoSPrefetching reduces the length of time that prototypes fail to meet SPECweb's minimum acceptable QoS.

Resource preservationKaleidoscope workers avoid unnecessary pages. Unfetched pages save precious bandwidth and are mostly file system cache and a free list.

RuntimeRuntime approaches near-optimal (warm static) as VCPU blocking and page faults are anticipated by colordirected prefetch and eliminated from the critical path.

ScalabilityThe database's OLAP throughput scales well with the number of simultaneous clones

Before

After

Traces in this figure include transferred state plus new allocations.