energy efficiency in cloud computing and optical networking

Energy Efficiency in Cloud Computing and Optical Networking

Rod Tucker, Kerry Hinton, Rob Ayre

Centre for Energy-Efficient TelecommunicationsUniversity of Melbourne

ECOC 2012 Tutorial 2

Outline

• Introduction and overview of energy consumption and

efficiency in communications networks

• Estimating energy consumption in ICT equipment

– Telecommunications

– The “cloud”

• Improving network energy efficiency

– Technologies

– Architectures

– Protocols

– The cloud


World’s technology capacity

Source: M. Hilbert, P. Lopez, Science, 2011

1E+18

1E+19

1E+20

1E+21

1E+22

1E+23

1E+24

1985 1990 1995 2000 2005 2010 2015

Storage (bits)

Telecommunications

(bits/year)

1024

1023

1022

1021

1020

1019

1018

Ca

pa

city (b

its, b

its/y

ea

r)


Internet traffic growth trends

Co

mp

ou

nd

Ave

rage

Gro

wth

Rat

e (%

)

Source: Kilper et al., JSTQE 2011


Source : Cisco Cloud Index 2011

0

1000

2000

3000

4000

5000

2010 2011 2012 2013 2014 2015

Exab

yte

s/an

nu

m

Year

Data Centre Traffic 2010-2015

Datacentre to user

Between Datacentres

Within Datacentres

Exab

ytes

/an

nu

m

Data Center to User

Between Data

Centers

Within Data

Centers

Projections of data centre traffic


Po

we

r C

on

su

mp

tio

n (

W)

Year

40% p.a. Data growth

10% p.a. Growth in user numbers

Power consumption of the global Internet

2010 2015 2020

109

1011

1010

1012

Global electricity supply

1013

Power Consumption of Internet

(Including servers)

1.5 billion users


Why does energy matter?

• If nothing is done to address the growing “efficiency gap”:

– ICT will consume ever larger proportion of global energy

– Energy consumption could become a barrier to network growth

• Economic and engineering imperatives:

– Energy is a growing component of OPEX

– Increased energy consumption increased footprint


Estimating ICT power consumption

• Inventory based estimates

– Look at what is out there

• Use sales and deployment data from vendors and surveys

• Accessing accurate data is problematic

• Network design and dimensioning based estimates

– Design a network that will satisfy current and projected demands

• Use typical network design rules

• Difficult to include network inefficiencies, overlays & legacies

• Transaction based estimates

– Look at services required and design a network to provide them

• Similar to network design approach


Summary of estimates

Author Year

% national

electricity

use

Country

PC’s,

office

equip. &

servers

Wireless

accessNotes

Huber 1999 13% USA Yes No Severe over estimate

Koomey 1999 2% USA Yes No Users & equipment estimate

Kawatomo 2001 3% USA Yes No Users & equipment estimate

Turk 2001 0.5 - 1.7% Germany Yes No Users & equipment estimate

Barthel 2001 0.9 – 1.5% Germany Yes Yes Users & equipment estimate

Roth 2002 < 2.3% USA Yes Yes Users & equipment estimate

Cremer 2003 7.1% Germany Yes Yes Users & equipment estimate

Baliga 2007 0.5% OECD No No Network design & dimensioning

Vereecken 2010 Not given Not given No No Network design & dimensioning

Lange 2010 Not given Not given No Yes Network design & dimensioning

Kilper 2011 Not given USA No Yes Transaction

Pickavet 2007

2012Global Yes Yes Users and equipment estimate


Design and dimensioning approach

1. Split network into

– Access

– Metro/Edge

– Core

– Data centres, content storage

2. Model with representative architecture and equipment

3. Dimension network to accommodate expected traffic

4. Calculate power consumption per customer for network

Baliga et al., 2007


Key model parameters

• Peak vs average access speed– Contention & aggregation

• Network dimensioning

– Traffic growth

• Deployed capacity > demanded capacity

– Equipment redundancy

• Multi-homing , back-up storage

– Service protection

• 1 + 1, 1:1, 1:N protection

• Router hops between source and destination

• Data centre Power Usage Effectiveness (PUE)

Time

Tra

ffic

Peak

Average


Ethernet

Switch

OLTSplitter

Metro/Edge NetworkCore Network

Edge Routers

FTTP

FiberCore Router

Content Distribution Network

Storage

Server

Server

Storage

Fiber

Access Network

DSL

DSLAMCu

OLT

ONU

Cabinet

FTTN

DSLAM

Cu

Broadband Network Gateways

Hot

spots

PON

Data Center

Network segmentation


Access network energy consumption30

Peak Access Rate (Mb/s)

Po

wer

Per

User

(W)

1 1000

0

FTTP (PON)

100

20

10

10

FTTN

Wireless

HFC

PON is “greenest”

Baliga et al., OFC 2009


Data centers and content servers

Racksof Servers

Aggregation Switches

Load-Balancing Switches

Border Routers

RacktopSwitches

80% of traffic stays in data center

5% of traffic to other data centers

15% of traffic to users


Power consumption of equipment

• Which metric(s) are appropriate?

– Power consumption per “throughput”

– Power consumption per “good put”

– Energy per bit, power per bit rate

– Energy per customer bit

– etc.

• Different metrics provide different optimal solutions to

energy efficiency

• Total power and energy/bit are widely used


Power consumption in routers

Sources: METI, 2006, Nordman, 2007

Router Throughput

Pow

er

consum

ption

(W)

10

100

1,000

10,000

100,000

1,000,000

P = C2/3

where P is in Watts

where C is in Mb/s

5 nJ/bit

100 nJ/bit

1

1 Pb/s

P ~ 10

1 Tb/s1 Gb/s1 Mb/s

?


Energy efficiency of equipmentE

nerg

y p

er

bit (

nJ)

0.01

0.1

1

10

100

MEMS

OXC

Optical

Amp

PIC

Tx/Rx

WDM

Tx/Rx

Ethernet

Switch

Core

Router

FEC

Chip

0.001

Sub-wavelength Wavelength

2010 Data

Tucker et al., 2010


Equipment Energy Consumption Trends

1

10

100

1000

10000

1985 1990 1995 2000 2005 2010 2015

nan

o-J

ou

les

pe

r b

it

Year

Router Energy Efficiency

Cisco AGS

Wellfleet BCN

Cisco GSR 12000

Cisco GSR 12000b

Avici TSR

Cisco CRS 1

Cisco CRS-3

ALU7750

Actual improvement

may be declining

Linear fit gives ~25% improvement pa

Nielsen, ECOC 2011


Trends in transport energy consumption

First Trans-Atlantic

Marconi

Trans-Atlantic

Fessenden

Trans-Atlantic

NY - Paris

Key West - Havana

TAT-1

TAT-3

TAT-5TAT-8

TAT-9

TAT-10

TAT-11

TAT-12/13

Newhaven - Azores

10-6

1840 1860 1880 1900 1920 1940 1960 1980 2000 2020

Year

En

erg

y/B

it/1

000 k

m (

mJ)

Wireless

Telegraphy

Coax

Optical + Regen

Optical + EDFA

10-4

104

10-2

102

106

108

1

~15% improvement p.a.Current

Tucker, JSTQE 2011


Putting it all together

• Design and dimensioning approach

• 40% p.a. growth in network traffic

• 10% p.a. growth in user numbers

• 15% p.a. improvement in all technologies

• Projections of data centre traffic


Power consumption of the global Internet

Year

15% p.a. technology improvement

Po

we

r C

on

su

mp

tio

n (

W)

109

1011

1010

108

1012

2010

2015

2020

Access (PON)

Global electricity supply (3% p.a.)

TotalTotal (2010 Technology)


Network energy per user bit

Year

En

erg

y p

er

Use

r b

it (m

J)

1.0

100

0.1

10

0.01

2010 2015 2020

Total

PON

Core and Metro

Switches/Routers

Optical Transport

0.1

1

10

100

Avera

ge A

ccess R

ate

(M

b/s

)


Gap between theory and practice

Tucker, JSTQE 2011

10-6

10-9

10-10

10-8

10-11

10-7

10-5

10-12

Current

Trends

Lower Bounds

x 104

Year2010 2015 2020 2025

Netw

ork

Energ

y p

er

Bit (

J)

Transport

Switches

Transport

Access

10-4

Global Network

X 103

X 102


Gap between theory and practice

Management and control overheads, interconnects,

power supplies, etc.

Protocols, device efficiency, interconnects, system penalties,

system margins, etc

Subsystem

PTotal

Function

Overhead

Poverhead

Pfunction

Inefficiency Traffic (b/s)

Pow

er

consum

ption

Traffic (b/s)

Energ

y p

er

bit

Ideal

Ideal


Improving network energy efficiency

• Many ideas for improving energy efficiency

– Insufficient time to cover all of them

• Key approaches

– Technologies

– Architectures

– Protocols

– The cloud


A. Technologies

• Fundamental physical technologies for telecommunications:

– Electronics: primarily CMOS for signal and data processing and

storage - Improvements by Moore’s Law

– Optics/photonics, primarily used to transport data

– More than 99% of network energy is consumed by electronics

• Advances are needed in

– Optical and electronic switch technologies

– Optical and electronic interconnects at all levels


B. Architectures

• Architectures that reduce the number of network hops

• Optical bypass

• Layer 2 rather than Layer 3 where possible

• Dedicated content-delivery networks


Bypass optionsWithout bypass:

• All traffic goes to IP layer for processing

• 10 nJ per bit

• Allows aggregation of incoming traffic flow

• Statistical multiplexing increases utilisation of paths

WDM

LinksWDM

Links

IP

Patch

panel

IPTime

Pa

cke

ts

Time

Pa

cke

ts

Time

Pa

cke

ts

Time

Pa

cke

ts

Peak rate

(channel

capacity)


With bypass:

• TDM Layer

– Some traffic streams processed at TDM level

– ~ 1 nJ per bit

• WDM layer

– Some traffic switched at WDM layer

– < 0.1 nJ per bit

WDM

OXC

IP

TDMTDM layer

bypass

WDM layer

bypass

TDM

WDM

OXC

Bypass options (cont’d)


C. Protocols

• Service transactions and protocols

• Efficiency of multi-layer protocol suite

• XGPON framing

• Sleep and standby states

• Energy-efficient Ethernet

• Dynamic rate adaption


Service transactions & protocolsService based energy model reflects how services are transported through the network

NIC

IP

TCPUDP...

Ap

p 1

Ap

p 2

Ap

p 3

NIC

IP

TCPUDP...

Ap

p 1

Ap

p 2

Ap

p 3

NIC

IP

TCPUDP...

Ap

p 1

Ap

p 2

Ap

p 3

NIC

IP

TCPUDP...

Ap

p 1

Ap

p 2

Ap

p 3

NIC

IP

TCPUDP...

Ap

p 1

Ap

p 2

Ap

p 3

NIC

IP

TCPUDP...

Ap

p 1

Ap

p 2

Ap

p 3

End users

Eth Eth

Aggregation Edge

Eth Eth

IP

IP

L2/L1

L2/L1

CoreNetwork

node

node

node

node

Source: Zhong et al., OFC 2012


Efficiency of multi-layer protocol suite

Combining boxes for multi-layer protocol suites can improve efficiency

Encapsulation of

small packets

Packet

fragmentation

Multiple

O/E/O

IP/Ethernet

IP/COE/WDM

(integrated box)

IP/COE/WDM

(separate boxes)

IP

Eth

WDMWDM

LinkWDM

Link

PPP

IP

SDH

WDMWDM

LinkWDM

Link

IP

COE

WDM

Link

WDM

Link

IP

WDM

OXC

COE

Efficiency =Customer payload IP layer energy

Total multi-protocol suite packet energy

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

20 200 2000 20000

Customer payload (Bytes)

Effic

ien

cy IP/PPP/SDH/WDM

Source: Zhong et al., OFC 2012

XGPON DS Framing Structure

Port-ID

Chow et al.,

ECOC 2012

XG-PON framing protocols

ONU 1

TxOLT-MAC

(FPGA) Rx

ONT-FPGA

UN

I

Line card10Gbit/s

ONU 2Bit-interleaved data

BIPONDS

BI

DS

ONU 1

TxOLT-MAC

(FPGA)Rx

ONT-FPGA

UN

I

Line card10Gbit/s

ONU 2Packet data

XGPONDS

XGPON

DS

Deser

Deser

Delta

Solution: BiPONBit-Interleaved PON

Conventional PON

~ 5 W /user

< 200 mW /user

Chow et al., ECOC 2012


Energy-efficient protocols• Sleep & standby states

– Network devices enter low power state when not in use

– Can apply to systems and sub-systems

– Need to ensure network presence is retained

• Use Network Connection Proxy with sleep protocol

– Need to account for state transition energy and time

– May have multiple lower energy states

• IEEE Energy Efficient Ethernet (802.3az)

– Low power idle mode when no packets are being sent

– Approved Sept. 2010

– Currently applies to copper interface only; not optical


PON burst-mode laser driverKoizumi et al., ECOC 2012

• Driver turned off between bursts

• Power reduced by 93%


Energy-efficient protocols• Dynamic rate adaptation

– Modify capacity of network devices in response to traffic demands

– Change clock frequency, processor voltage

– Slower speed to reduce power consumption

– Need to allow transition time between rates

2Power C Voltage Frequency

Power

Packetstime

time

time

No protocol

Sleep state

Adaptation

Both

Bolla et al., 2011

time

• Dynamic rate adaptation and standby states can be combined


• Cloud services widely promoted as greener than on-site facilities:

– Cloud Computing – The IT Solution for the 21st Century

• Carbon Disclosure Project Study 2011

– Salesforce.com & the Environment

• WSP Environment & Energy 2011

• Strong case for enterprise private cloud

• What about the public cloud?– Apple iCloud

– Google drive

– Microsoft sky drive

D. The cloud


0.01

0.1

1

10

100

0.01 0.1 1 10 100

Po

we

r C

on

su

mp

tio

n (W

)

Downloads per Hour

Transport

energy dominates

1 GB Cloud Storage

5 GB Cloud Storage

Local HDD

Example: Public storage as a service (SaaS)Storage of application & data “in the cloud” compared with storing on a local disk.

50 MBytes per download. Modern laptop-style HDD 20% read/write and 80% idle.

Baliga et al., Proc IEEE, 2011


Rethinking the “Green Cloud”

• Need to improve access energy efficiency

– Small wireless cells

– PON

• Keep some processing power in user’s device

• Reduce the number of router hops

– Avoid public Internet

– Use optical layer by-pass of routers

• Improve protocol efficiency

– Less overhead bytes

– Smart scheduling


Ethernet

Switch

OLTSplitter

Cabinet

Metro/Edge NetworkCore Network

Edge Routers

PON

Fiber

EDFA

Core Router

Distant

IPTV Server

Fiber

Broadband Network Gateways

“Local”

IPTV Server

Server

Storage

• Location of movies is a trade off

between energy for storage and

energy for delivery

• Each movie is replicated on R

servers throughout the network

Direct optical link to edge routerServer

Storage

IPTV over the public Internet


P2P

En

erg

y p

er

do

wn

load

(W

h)

(lo

g s

cale

)

10

100

104

Downloads per hour

10310210110010-110-2

R = 2000R = 200R = 20

R = 2


Baliga et al., OFC, 2009


Total

Storage

Transport

R = 20

Po

wer

per

mo

vie

(W

)

100

103

10-1 101 10-2

Downloads per hour10-2

101

102

Video

Servers

100


Baliga et al., OFC, 2009


Conclusions

• Energy consumption of the network is growing

• Access network energy dominates– Servers in data centres are likely to become dominant in ~2015

– Core and metro networking to overtake access in ~2020

– Optical transport is relatively “green”

– Beware “the cloud”

• Many opportunities for improving network energy efficiency– Technologies

– Architectures

– Protocols

• If you are inspired, join one of the networks or consortia:

www.greentouch.org, www.fp7-trend.eu

http://www.greentouch.org/

http://www.fp7-trend.eu/



energy efficiency in cloud computing and optical networking

Documents