Power-efficient server provisioning in server farms

Anshul Gandhi (Carnegie Mellon University)

Varun Gupta (CMU), Mor Harchol-Balter (CMU)Michael Kozuch (Intel, Pittsburgh)

Motivation

Server farms are important for today’s IT infrastructure (Amazon, Google, IBM, HP, …)

However, server farms cost a lot of money to power ($4 billion in 2006)

Server FarmRequests

High-level problem statement

How many servers, given request rate ? Don’t want to waste power

Requests

Server Farm

Outline

1. Server farm model

2. Provisioning for fixed arrival rate

3. Provisioning for unpredictable, time-varying arrival rate

4. Future work

5

Server farms

IDLE servers consume a lot of power

~ 60 % of BUSY

BUSY

BUSY

BUSY

IDLE

IDLE

OFF

OFF

6

Server farms

Turn IDLE servers OFF to save power

BUSY

BUSY

BUSY

OFF

OFF

OFF

OFF

HOWEVER

7

Setup cost

To turn on an OFF server ..

BUSYOFF SETUP

Time delay (setup time)• 1 min – 5 mins

and

Power penalty • peak power during setup time

8

Setup cost

To turn on an OFF server ..

BUSYOFF SETUP

Should we ever turn servers OFF ?

9

Server model

Server states:BUSY PBUSY 240 W

IDLE PIDLE 150 W

OFF POFF 0 W

SETUP PSETUP 240 W

Setup times:TOFF→ON 200 s

TON→OFF 0 s

Intel Xeon E5320• 2 X 1.86 GHz quad-core• 4GB memory

ON

10

Server farm model

Poisson arrival process: λ(t) requests/sec Exponentially distributed job sizes: E[S] secs Load: ρ(t) = λ(t) E[S]∙

Minimum # servers to handle incoming load

RequestsFCFS Server Farm

11

Metric

Interested in response time and power conumption

Perf/W = 1/(Mean RT X Mean Power)

Maximize Perf/W

Outline

1. Server farm model

2. Provisioning for fixed arrival rate

3. Provisioning for unpredictable, time-varying arrival rate

4. Future work

13

Provisioning for fixed arrival rate

Existing solutions: prediction based, reactive controllers.

Is there a simple, yet, near-optimal solution ?

Poisson arrivals

Server Farm

Max. Perf/W

14

NEVEROFF

Keep n servers always ON (M/M/n) Servers are BUSY or IDLE

*n

15

Perf/W for NEVEROFF

16

INSTANTOFF

Turn servers OFF when IDLE Servers are BUSY, OFF or in SETUP

*n

Auto-scales if n is high

17

Perf/W for INSTANTOFF

18

NEVEROFF vs. INSTANTOFF

TON→OFF < γ E[S]/√ρ

19

Near-optimality

Best of {NEVEROFF, INSTANTOFF} is optimal for single-server

Multi-server ?

For ρ > 10, we are within 20% of OPT

Outline

1. Server farm model

2. Provisioning for fixed arrival rate

3. Provisioning for unpredictable, time-varying arrival rate

4. Future work

21

Unpredictable, time-varying demand

Data center demand has daily variations

INSTANTOFF can auto-scale

22

Unpredictable, time-varying demand

NEVEROFF requires continual updates based on predicted load

Predictions are not always accurate

Can we find a simple traffic-oblivious policy? Auto-scaling in nature

23

DELAYEDOFF

Like INSTANTOFF, except we wait for twait seconds before turning IDLE servers OFF

Routing ?

MRB routing is crucial !

24

twait

Rule of thumb: twait P∙ IDLE = TOFF→ON P∙ ON

25

Near-optimality

Worse at higher frequencies

26

Auto-scaling capabilities

1998 World Cup Soccer trace (ITA)

Outline

1. Server farm model

2. Provisioning for fixed arrival rate

3. Provisioning for unpredictable, time-varying arrival rate

4. Future work

28

Future work

Experimental evaluation of proposed schemes Preliminary experiments on 15-server testbed using

CPU-bound workload and sinusoidal arrival pattern Experimental results agree with analysis Web workloads:▪ What does the experimental setup look like ?

Try out various arrival traces and workloads

29

Thank You!

Anshul Gandhi, Varun Gupta, Mor Harchol-Balter, Michael KozuchOptimality analysis of energy-performance trade-off for server farm management, PERFORMANCE 2010

Anshul Gandhi, Mor Harchol-Balter, Ivo AdanServer farms with setup costs, PERFORMANCE 2010

Anshul Gandhi, Varun Gupta, Mor Harchol-Balter, Michael KozuchEnergy-efficient dynamic capacity provisioning in server farms, CMU technical report CMU-CS-10-108