varun gupta carnegie mellon university 1 partly based on joint work with: anshul gandhi mor...
TRANSCRIPT
Energy-efficient Dynamic Capacity Provisioning in Server Farms
VARUN GUPTACarnegie Mellon University
1
Partly based on joint work with:
Anshul Gandhi Mor Harchol-Balter Mike Kozuch (CMU) (CMU) (Intel Research)
2
The “provisioning for peak” problem
Demand(t)
Time (t)
?
Data CenterCapacity
PROBLEM: Want to turn servers OFF/ON to match (t)…
… and also minimize setup penalties!
3
First Attempt: ON/OFF policy
0 2 4 6 8 10 120
50
100
150
200
250r(t)#Busy servers
Time (hrs)
(t)# Busy servers# Jobs
Mean Job Size = 1sSetup delay = 200sBusy power = 240W
4
First Attempt: ON/OFF policy
0 2 4 6 8 10 120
50
100
150
200
250r(t)#Busy servers
Time (hrs)
(t)# Busy servers# Jobs
Mean Job Size = 1sSetup delay = 200sBusy power = 240W
5
First Attempt: ON/OFF policy
0 2 4 6 8 10 120
50
100
150
200
250r(t)#Busy servers
Time (hrs)
(t)# Busy servers# Jobs
Mean Job Size = 1sSetup delay = 200sBusy power = 240W
6
First Attempt: ON/OFF policy
0 2 4 6 8 10 120
50
100
150
200
250r(t)#Busy servers
Time (hrs)
(t)# Busy servers# Jobs
Mean Job Size = 1sSetup delay = 200sBusy power = 240W
WITH SETUP: E[Power] = 51.9 kW E[Response time] = 13s
NO SETUP: E[Power] = 14.4 kW E[Response time] = 1s
7
First Attempt: ON/OFF policy
0 2 4 6 8 10 120
50
100
150
200
250r(t)#Busy servers
Time (hrs)
(t)# Busy servers# Jobs
Mean Job Size = 1sSetup delay = 200sBusy power = 240WCan we do better than ON/OFF? .
Add inertia while turning servers OFF
8
Our Prescription: DELAYEDOFF
Turn OFF a server after idle for twait
twait independent of (t)!
new arrival routed to themost recently busy (MRB) server
arriving job turns a server ONif all servers busy
9
twait timers
10
twait timers
11
twait timers
12
Our Prescription: DELAYEDOFF
THEOREM: For Poisson arrivals, as the load , the number of ON servers is concentrated around
--- Proof Intuition ---
Step 1: An equivalent system view
k jobs run on the “first” k servers
Step 2: Analysis of idle periods of M/G/∞
Time from a k(k-1) to the next (k-1)k transition
13
Intuition for idle periods
1. # jobs Normal with mean and variance
2.
3. Events happen at rate
4. Mean idle period of server
1
2
3
MRB servers for any constant twait !
In practice, we choose twait to amortize setup cost
14
0 2 4 6 8 10 120
50
100
150
200
250
r(t)#Busy servers
Time (hrs)
0 2 4 6 8 10 120
50
100
150
200
250
Time (hrs)
Mean Job Size = 1sSetup delay = 200sBusy power = 240W
(t)# Busy+idle servers# Jobs
ON/OFF
DELAYEDOFF
15
0 2 4 6 8 10 120
50
100
150
200
250
r(t)#Busy servers
Time (hrs)
0 2 4 6 8 10 120
50
100
150
200
250
Time (hrs)
Mean Job Size = 1sSetup delay = 200sBusy power = 240W
(t)# Busy+idle servers# Jobs
ON/OFF
DELAYEDOFF
16
0 2 4 6 8 10 120
50
100
150
200
250
r(t)#Busy servers
Time (hrs)
0 2 4 6 8 10 120
50
100
150
200
250
Time (hrs)
Mean Job Size = 1sSetup delay = 200sBusy power = 240W
(t)# Busy+idle servers# Jobs
ON/OFF
DELAYEDOFF
DELAYEDOFF: E[Power] = 18.9 kW E[Response time] = 1.002s
NO SETUP: E[Power] = 14.4 kW E[Response time] = 1s .
17
0 2 4 6 8 10 120
50
100
150
200
r(t)#Busy servers
Time (hrs)
0 2 4 6 8 10 120
50
100
150
200
Time (hrs)
Mean Job Size = 1sSetup delay = 200sBusy power = 240W
(t)# Busy+idle servers# Jobs
DELAYEDOFF
SQ. ROOT W/LOOKAHEAD
(“optimal offline”)
18
0 2 4 6 8 10 120
50
100
150
200
r(t)#Busy servers
Time (hrs)
0 2 4 6 8 10 120
50
100
150
200
Time (hrs)
Mean Job Size = 1sSetup delay = 200sBusy power = 240W
(t)# Busy+idle servers# Jobs
DELAYEDOFF
SQ. ROOT W/LOOKAHEAD
(“optimal offline”)
19
0 2 4 6 8 10 120
50
100
150
200
r(t)#Busy servers
Time (hrs)
0 2 4 6 8 10 120
50
100
150
200
Time (hrs)
Mean Job Size = 1sSetup delay = 200sBusy power = 240W
(t)# Busy+idle servers# Jobs
DELAYEDOFF
DELAYEDOFF: E[Power] = 18.9 kW E[Response time] = 1.002s w/LOOKAHEAD: E[Power] = 15.8 kW E[Response time] = 1.036s
SQ. ROOT W/LOOKAHEAD
(“optimal offline”)
DELAYEDOFF mods
1. Speed scaling algorithms2. A simple proxy for MRB routing3. Heterogeneous servers4. Managing Virtual Machines in the Cloud5. Wear-leveling/Performance tradeoffs
20
21
Effect of DELAYEDOFF on speed-scaling
PowerP(s)
Speed (s)
Q: Optimal speed to balance energy-performance?
A: your favorite metric =
22
A simple proxy for MRB routing
MRB requires a lot of state updates
Proxy policy• Assign static ranks to servers• Route a new arrival to highest ranked idle server
Almost the same performance as MRB+ easier to implement than MRB+ easy to extend
(MRB)
Rank
1
2
3
4
5
Rank-based
23
DELAYEDOFF for heterogeneous servers
more efficient
less efficient
Assign ranks to servers based on efficiency (1 = most efficient)
Route a new arrival to highest ranked idle server
1
2
3
4
5
24
Managing Virtual Machines in the Cloud
2GB RAM
2GB RAM
2GB RAM
1GB
1GB
1GB
25
Managing Virtual Machines in the Cloud
1
2
3
4
5
6Split physical servers into “virtual” servers
Assign static ranks to virtual servers
Route a new VM request to highest ranked idle virtual server
Turn the physical server OFF after each virtual server has idled for twait
Conclusions Problem: unpredictable
demand and non-trivial setup costs
DELAYEDOFF : A new traffic-oblivious capacity scaling scheme
Extensions to real-world scenarios
26
Time (hrs)
Demand
?Time
MaxCapacity
27
The Importance of Being MRB
THEOREM [MRB]: As the load , the number of ON servers is concentrated around
THEOREM [Round-Robin]: As the load , for constant
job sizes, the number of ON servers is
N serversArrival rate = λ