Power Comparison of Cloud Data Center Architectures

Pietro Ruiu

Andrea Bianco, Paolo Giaccone

Claudio Fiandrino, Dzmitry Kliazovich

13th Italian Networking Workshop: San Candido, Italy January 13 - 15, 2016

2

The quest for green data centers• Data centers are the new polluters of 21st

century– in 2012, accounted for 15% of the global ICT

energy consumption– expected to increase in the next years

• Data center consumption– 75% ICT equipment

• powering and cooling• mostly due to servers

– 25% power distribution and facility operations• Strong interest in designing and operating data

centers with higher energy efficiency– not only to reduce OPEX costs

3

Key question

• Given• a data center topology• a power consumption profile for each ICT device

• Define the min-power job allocation policy• Evaluate power consumption as function of the data center

load

Classical questions

• Given• a power consumption profile for each ICT device• a generic job allocation policy

• Compare the power consumption behavior in function of the data center topology

Our question

4

Data center model

Data center network (DCN)

Pow

er

Load

Pow

er

Load

Pow

er

Load

Servers

ICT device Power profile

5

Local vs global energy proportionality

• consume proportional to the load• consume (and pay) only if really needed

Ideal energy proportionality

• Constant power (CONST)• Full Energy Proportional (FEP)• Linear (LIN)

Local power consumption for single device

• maybe very different from local power consumption

Global power consumption for overall system

CONSTPo

wer

Load

FEP

Pow

er

Load

LIN

Pow

er

Load

Pow

er

Load

6

Global power consumption

• N resources/devices to be allocated for a set of requests/VMs• power consumption profile for each resource/device

• allocation policy• consolidate: activate the minimum number of resources• load-balance: distribute load across the resources

Resource allocation

• depends on granularity of the resources (i.e. the value of N)• depends on allocation policy

Overall power consumption

1 2 N

7

Global Power ConsumptionLocal power

consumption Consolidate policy Load-balance policy

Normalized power = Power / Load

FEP

Pow

er

Load

CONST

Pow

er

Load Load

Pow

er

• Local energy proportionality implies global energy proportionality• If N is enough large, consolidate policy reaches global energy

proportionality for CONST local power

Local and Global Energy Proportionality

8

Our contributions

• Network-aware min-power VM online allocation policy

• Flow-level C++ simulator of data centers • Power comparison of different network

topologies

• Energy profiles for each ICT device (switch, link, server)

• DCN topology• VM arrival process

Flow-level simulator

• Global power consumption

• Load on each ICT device

• VM blocking probabilityVM allocation

policy

9

DCN topologies

• traditional Clos-based switch topologies– classical 2, 3 tiers– Jupiter• Google’s disclosed DCN architecture • “Jupiter Rising: A Decade of Clos Topologies and Centralized Control in

Google’s Datacenter Network”, ACM SIGCOMM CCR, Oct. 2015

10

• Switches: 10 core sw – 18 TOR sw

• Servers 180

• Total ICT devices: 208 nodes

CORE

TOR

10Gb

ps40

Gbps

2-tiers DCN

11

• 3 core sw – 6 aggregation sw – 18 TOR sw

• 180 servers – 27 switches – 207 nodes

CORE

AGGREGATION

TOR

10Gb

ps40

Gbps

40Gb

ps

3-tiers DCN

12

• 24 spine sw – 16 aggregation sw – 16 TOR sw

• 192 servers – 44 switches – 236 nodes

= 4p@40Gbps (or 16p@10Gbps)

SPINE

AGGREGATION

TOR

10Gb

ps10

Gbps

40Gb

ps40

Gbps

40Gb

ps

MB MB MB MB

Jupiter-like DCN

13

Online VM allocation policy

• for each VM, select a server at random• connect through the minimum incremental DCN

power• load-balance on the servers

RSS (Random Server Selection)

• for each VM, select the server with minimum incremental power (server + DCN)

• consolidate VMs in the same server, in the same rack, in closeby racks, etc

• variant of min-cost Dijkstra algorithm

MNP (Minimum Network Power)

14

VM generation

• time is slotted• at each timelot, a new VM arrives and must

communicate B bps to a previously randomly allocated VM – B is randomly chosen– destination VM is chosen with Bernoulli trials

• simulation can run until saturating the data center

VM1 VM2 VM3 VM4 VM5

15

RSS (Random Server Selection)• small datacenter (180-192 servers)• Jupiter appears to be the most energy proportional

– due to the larger number of switches (44 vs 27-28)

16

MNP (Minimum Network Power)• small datacenter (180-192 servers)• MNP allows to achieve global energy-proportionality• under FEP, power jumps due to abrupt activation of new layers

17

Conclusions

• Global energy proportionality of an overall data center depends on• local power profile of each device• topology (number of devices)• VM allocation policy

Take-home message

• consider large topologies with 10,000 servers• compare data center networks given the same bisection

bandwidth• consider the allocation of clusters of VMs

Future works