spoton: a batch computing service for the spot …spoton: a batch computing service for the spot...

SpotOn: A Batch Computing Service

for the Spot Market

Supreeth Subramanya, Tian Guo, Prateek Sharma, David Irwin, Prashant Shenoy

University of Massachusetts Amherst

infrastructure cloud

On-demand


On-demand

Cost vs. Availability Tradeoff in IaaS CloudsC

heap

Expe

nsiv

e

Guaranteed, Non-revocable

Not guaranteed, Non-revocable

Not guaranteed, Revocable

Cos

t (pe

r hou

r)

Availability

Reserved

Spot


Spot Instances

Preemptible VM

On-demand

Cost vs. Availability Tradeoff in IaaS CloudsC

heap

Expe

nsiv

e

Guaranteed, Non-revocable

Not guaranteed, Non-revocable

Not guaranteed, Revocable

Cos

t (pe

r hou

r)

Availability

Reserved

Spot

spot markets

Amazon EC2

Bid in a 2nd price auction Acquire when bid > spot price Terminate when spot price > bid

Time of the day

Pric

e

Spot User bid

spot markets

Amazon EC2


How do we mitigate the impact of revocation?

1. Raise the bid 2. Employ fault-tolerance mechanisms

Time of the day

Pric

e

Spot User bid

spot markets

Amazon EC2


How do we mitigate the impact of revocation?

1. Raise the bid 2. Employ fault-tolerance mechanisms

Time of the day

Pric

e

Spot User bid

2. Employ fault-tolerance mechanisms

challenges — spot market complexity

Amazon EC2 operates ~4000 spot markets


Scatterplot of ranks for EC2 spot markets



Scatterplot of ranks for EC2 spot markets


Selecting an instance that yields

lowest cost per unit of computation

while also considering the

probability of revocation is complex

challenges — application complexity

CPU : IO 1:1

Working Set 8GB

Running Time 1 hour

Resource Vector

Disk Type Remote


Cost 20% On-demand

Revocation Rate 2.4 per day

Spot VM

Fault-tolerance Mechanism Checkpoint (every 900s)

CPU : IO 1:1

Working Set 8GB

Running Time 1 hour

Resource Vector

Disk Type Remote

Com

plet

ion

time

(s)

0

2000

4000

6000

8000

Cost

(¢)

0

25

50

75

100

On-demand Checkpoint


Cost 20% On-demand


Spot VM


CPU : IO 1:1

Working Set 8GB

Running Time 1 hour

Resource Vector

Disk Type Remote

CPU : IO 1:1

Working Set 8GB

Running Time 1 hour

Resource Vector

Cost 20% On-demand


Spot VM


Disk Type Remote

Replicate (deg=2)


Local

CPU : IO 1:1

Working Set 8GB

Running Time 1 hour

Resource Vector

Cost 20% On-demand


Spot VM


Disk Type Remote

Com

plet

ion

time

(s)

0

2000

4000

6000

8000

Cost

(¢)

0

25

50

75

100

On-demand Checkpoint Replicate

Replicate (deg=2)


Local

CPU : IO 1:1

Working Set 8GB

Running Time 1 hour

Resource Vector

Cost 20% On-demand


Spot VM


Disk Type Local


>24 per day

CPU : IO 1:1

Working Set 8GB

Running Time 1 hour

Resource Vector

Cost 20% On-demand


Spot VM


Disk Type Local

Com

plet

ion

time

(s)

0

2000

4000

6000

8000

Cost

(¢)

0

25

50

75

100

On-demand

Checkpoint

Replicate

Replicate (Revoked)


>24 per day

Spoton: A batch computing service

SpotOn


SpotOn

Service that accepts batch jobs from users and runs them on spot instances

Manages application and spot market complexity transparently


SpotOn

Service that accepts batch jobs from users and runs them on spot instances

Manages application and spot market complexity transparently

Run batch jobs at on-demand performance but paying spot market price

greedy selection algorithm

Selecting the best spot market and fault-tolerance mechanism

Jb

Batch Job



Jb

Spot Markets

∀Si

Batch Job



Jb

Spot Markets

∀Si

Batch Job

∀Ft

Fault Tolerance

Migrate Dup Chkp



Jb $

Minimum cost of running Jb using Ft on Si

Spot Markets

∀Si

Batch Job

∀Ft

Fault Tolerance

Migrate Dup Chkp



Jb $


Acquire Si

Spot Markets

∀Si

Batch Job

∀Ft

Fault Tolerance

Migrate Dup Chkp



Jb $


Acquire Si

Repeat on spot revocation

(until Jb finishes)

Spot Markets

∀Si

Batch Job

∀Ft

Fault Tolerance

Migrate Dup Chkp

Reactive Migration

fault tolerance (1/3)

Migrated

Reactive Migration


Migrated

size of memory + local disk remote disk bandwidth

TM ∝

Reactive Migration


Migrated

Zk → Random variable measuring time to revocation

Pk → Probability job gets revoked before completion


TM ∝

Reactive Migration


Migrated

Zk → Random variable measuring time to revocation

Pk → Probability job gets revoked before completion

E[Pricek] [(1 — Pk) * T + Pk * (E(Zk) + TM)] * spot-price

E[Timek] (1 — Pk) * T + Pk * E(Zk)

=Cost =


TM ∝

Proactive Checkpoint


Restored



Restored

𝜏: checkpoint frequency


TL ∝ (𝜏/2)

TC ∝



Total Overhead = (T /𝜏) * Tc + TL

Restored



TL ∝ (𝜏/2)

TC ∝



Total Overhead = (T /𝜏) * Tc + TL

Restored



TL ∝ (𝜏/2)

TC ∝

Cost overhead is primarily a function of job’s resource usage

Spot Replication


TL ∝ market volatility

Spot Replication


TL ∝ market volatility

Total Overhead ∝ (Replication factor, TL)

Cost overhead is primarily a function of market characteristics

evaluation (1/3)

SpotOn Prototype

App Emulator to create synthetic jobs with varying resource usage

Built on Linux Containers for efficient checkpointing / migration

App emulator

SpotOn job manager

daemondaemon

Effect of Application and Spot Market Characteristics

evaluation (2/3)

Effect of Application and Spot Market Characteristics

evaluation (2/3)

Best choice of fault-tolerance mechanism is a function of the spot market and job characteristics

Effect of Cost-aware Selection

evaluation (3/3)


evaluation (3/3)

2x better than just checkpointing

On Google cluster trace, Cost-aware selection achieved 91.9% savings with little impact on performance


evaluation (3/3)

2x better than just checkpointing

Spot markets offer arbitrage opportunities

SpotOn manages application and market complexities

We model fault tolerance and propose a selection algorithm

Prototype on Amazon EC2

Achieves ~90% cost savings

conclusion

Spot markets offer arbitrage opportunities

SpotOn manages application and market complexities

We model fault tolerance and propose a selection algorithm

Prototype on Amazon EC2

Achieves ~90% cost savings

conclusion

Thank you!

backup slides

bidding

Spot price distribution has long tail Spot price changes are peaky

bidding

Bidding does affect volatility but not drastically

SpotOn always bids at the on-demand price If spot price goes above on-demand, SpotOn would choose on-demand

Spot price distribution has long tail Spot price changes are peaky

spoton: a batch computing service for the spot …spoton: a batch computing service for the spot...

Documents