data center efficiency metrics - the green grid

Data Center Efficiency Metrics:

mPUE™, Partial PUE, ERE, DCcE

Dan Azevedo, Symantec

Jud Cooley, Oracle

Michael Patterson, Intel

Mark Blackburn, 1E

Join The Green Grid Technical Forum conversation on Twitter! As you are tweeting, use #TGG11.

Copyright © 2011, The Green Grid

Presenters

Dan Azevedo, Symantec

Director of Data Center Architecture, Strategy & Innovation

Board Member and Metrics & Measurements Chairman, The Green Grid

Jud Cooley, Oracle

Senior Director, Design for the Data Center

Alternate Board Member, The Green Grid

Michael K. Patterson, Intel

Senior Power & Thermal Architect

Technology & Strategy Chairman, The Green Grid

Mark Blackburn, 1E

Co-Founder & Chief Technologist

Metrics & Measurements Vice Chairman, The Green Grid

The United States of America, European

Union, and Japan have successfully

completed harmonization efforts around

one data center energy efficiency metric

Power Usage

Effectiveness (PUE)


U.S. Data Center Industry Session13 Jan 2010

• Attendees:

• Session Objectives: Data Center Efficiency Metrics: Develop common definitions for key

metrics and seek consensus on guiding principles for metrics

Measurement Protocols: Define current status and gaps and coordinate

plans for further development

Tools: Review status and coordinate development


U.S. Data Center Industry SessionTaskforce Results

• Released two memos:

Data Center Industry Leaders Reach Agreement on Guiding Principles for Energy Efficiency Metrics (Feb 2010)

Recommendations for Measuring and Reporting Overall Data Center Efficiency – Version 1 Measuring PUE at Dedicated Data Centers (Jul 2010)

• Work in Progress

Recommendations for Measuring and Reporting Overall Data Center Efficiency – Version 2 Measuring PUE in a Mixed Use Facility

US Consensus Achieved with Public Announcement for PUE with Guidance and Refinement

http://www.energystar.gov/ia/partners/prod_development/downloads/DataCenters_AgreementGuidingPrinciples.pdf

http://www.energystar.gov/ia/partners/prod_development/downloads/DataCenters_AgreementGuidingPrinciples.pdf

http://www.energystar.gov/ia/partners/prod_development/downloads/Data_Center_Metrics_Task_Force_Recommendations.pdf






Global Metric Harmonization Meetings26 Mar 2009, 02 Feb 2010, 26 Oct 2010, 15 Feb 2011

• Attendees: Ministry of Economy, Trade and Industry

Green IT Promotional Council

The U.S. Department of Energy‟s Save Energy Now and Federal Energy

Management Programs

The U.S. Environmental Protection Agency‟s ENERGY STAR Program

The European Commission Joint Research Center Data Centers Code of

Conduct

The Green Grid

• Session Objectives: Share global lessons and practices with an objective of arriving at a set of

metrics, indices, and measurement protocols which can be formally endorsed or

adopted by each participant organization to improve data center energy

efficiency globally.


Global Metric Harmonization MeetingsTaskforce Results

• Released two memos:

The United States of America, European Union and Japan

Reach Agreement on Guiding Principles for Data Center Energy

Efficiency Metrics (Feb 2010)

Global Taskforce Reaches Agreement on Measurement

Protocols for PUE – Continues Discussion of Additional Energy

Efficiency Metrics (Feb 2011)

• Work in Progress:

Prioritized further areas of work to drive Data Center Energy

Efficiency (see appendix)

8

The United States of America, European Union and Japan Successfully Complete Harmonizing PUE

xxxx

xxxx

xxxx

xxxx

xxxx

xxxx

xxxx

xxxx

xxxx


Influences Specific to PUE

Global Harmonization

of Metrics

U.S. Industry Data Center Taskforce

The Green Grid

Partial PUE (pPUE)


Rationale for Partial PUE

• PUE can be used to compare data center designs

• Often PUE's are presented that do not take into account all of the infrastructure components in order to highlight one particular portion of a data center

– “Container only” PUE

– “Cooling system” PUE

– “Power delivery” PUE

• These comparisons have value, but are not truly PUE

• “Partial PUE” provides a formal language for this

• Partial PUE is abbreviated “pPUE”


pPUE Definition

• PUE = Total Facility Energy divided by the IT Equipment Energy

• This takes into account energy use within a facility

• Partial PUE is for energy use within a boundary

pPUE = Total Energy within a boundary divided by the IT Equipment Energy within that boundary


Example 1

• PUE = 600/475 = 1.26

• pPUE of Container = 500/475 = 1.05


Example 2

PUE = 2700/2000 = 1.35

PPUE1

= 1700/1500 = 1.13

PPUE2

= 700/500 = 1.40

• Zones 1 and 2 have the same chiller power, but different IT loads

• So their Partial PUE's are different


Notations• N is for “non-IT” power

• I is for “IT” power

• PUE =

(N0 + N1 + N2 + I1 + I2)

(I1 + I2)

(N1 + I1)

(I1)

(N2 + I2)

(I2)pPUE2 =

pPUE1 =

Zone 0 has no IT, so pPUE is undefined


pPUE Relative to PUE

(I1)

(I1

+ I2)

• Zones 1 and 2 each hold a portion of the IT load:

r1

= r2

=(I

2)

(I1

+ I2)

• PUE = (N

0)

(I1

+ I2) + (r

1* pPUE

1) + (r

2* pPUE

2)

The PUE is the overhead seen by all zones plus the Partial PUE of each zone multiplied by the percentage of that zone's contribution to the overall IT load

(The proof is in the White Paper, note that r1 + r2 = 100%)


What It Means

• PUE = (N

0)

(I1

+ I2)

+ (r1

* pPUE1) + (r

2* pPUE

2)

• If one of the pPUE's is bad, work on that

• But if one of the r's is very large, the payback for working on that is great


Example 3

This is the same as example 2 except that Zone 1 is 10 times as large

• PUE = 20,000/15,500 = 1.29

• pPUE1

= 17,000/15,000 = 1.13

• pPUE2

= 700/500 = 1.40

• r1

= 15,000/15,500 = 0.97

• r2

= 500/15,500 = 0.03


Summary

• Partial PUE can be used to describe the PUE contribution of a subset of facility components

• Partial PUE refers to components within a boundary

• The boundary can be physical or logical, for example:

– All cooling components

– All power components

– All Tier II components

– All baseline load components

– All departmental components

• PUE is a valuable metric for monitoring and managing data center power

• If only a portion of the facility is being analyzed, use partial PUE

Energy Reuse

Effectiveness (ERE)


“I am re-using waste heat from my data center on another part of my site and my PUE is 0.8!”


“I am reusing waste heat from my data center on another part of my site and my PUE is 0.8!”

• While re-using excess energy from the data center can be a good thing to do, it should not be rolled into PUE• The definition of PUE does not allow this• There is a metric to do this! ERE


ERE Definition

IT

Reused-IT+Lighting+Power+Cooling =ERE

IT

IT+Lighting+Power+Cooling =PUE

Energy IT

Energy Total =PUE

Energy IT

Energy Reused -Energy Total =ERE


ERE Alternate Development

PUE ERF)1(IT

Reused-IT+Light+Pwr+Cool =ERE

Energy Total

Energy Reuse =ERF

Define energy reuse factor (ERF) as: Then:

PUE ERF)-(1 =ERE

And finally:

ERF and PUE are mathematically related, but differ and

need to be defined and reported clearly.


PUE and ERF Ranges

ERE0

PUE1

1ERF0

IT

IT+ Lighting+Power + Cooling =PUE

IT

Reused-IT+Light +Pwr + Cool =ERE

Energy Total

Energy Reuse =ERF


Data Center Boundary

Consideration and ERF

Utility

UPS PDU

IT

Cooling E

F

B

A

D

Waste

Data Center

Boundary

D

B+A =PUE

C


Use of ERE

Data Center

Boundary

D

B+A =PUE

D

F-B+A =ERE

Utility

UPS PDU

ICT

Cool

E

F

B

A

DC

Absorp

Chiller

H

G

I

waste

Both PUE & ERE valid metrics

Non-DC

space


Improper Use of ERE

Data Center

BoundaryD

B+A =PUE

D

F-B+A =ERE

Utility

UPS PDU

ICT

Cool

E

F

B

A

DC

AbsorpC

hiller

H

G

I

waste

PUE is still the right metric; note that “A” (and PUE) will be less than in the design w/o added recovery and absorption chiller


Comparison with PUE

• One view of PUE is that is the “tax” or burden in energy costs you

must pay above the IT load to run the data center; ERE allows the

same vision

• PUE = 1.0 means 100% of the energy you bring in to the data center

goes to the IT

• ERE = 1.0 means you only need to bring into the site an amount

equal to 100% of the IT energy to support the data center

• We need both!

• Case 1 Case 2

• PUE = 2.0 PUE = 1.2

• ERF= 0.55 ERF=0.25

• ERE = 0.9 ERE=0.9

• Case 1 focuses on PUE, Case 2 focuses on ERF

Data Center Compute

Efficiency (DCcE)


Unused Servers

• ~15% of servers powered on but not being used

• CPU utilization doesn‟t tell the whole story

• Virtualization doesn‟t cure the problem – it can

make it worse

• Removing unused servers will increase data

center efficiency

Reduce power for physical servers

Reduce resource usage for virtual servers


How to Find Unused Servers

• Servers are procured to provide a primary service

• If they are no longer providing this service then they are

no longer needed and should be decommissioned or

repurposed.

• Useful work <> CPU utilization

• Secondary & tertiary services can cause utilization

Virus scanning

Disk indexing / defragmentation

Backup

Etc..


Determining Primary Service Usage

• Tracking primary services resource utilization is

impractical – too many different primary service apps

• Tracking secondary & tertiary services is easier –

typically a well known small set of applications

• Primary service work = All work – secondary & tertiary

work

True for CPU & I/O

• Some applications do not lend themselves to

measurement like this – e.g. terminal services

Track incoming network sessions for „useful‟ processes

Track interactive logons – always assumed to be useful


Calculating Primary Service Usage

• Over a time period

If

– All CPU minus secondary & tertiary CPU > noise threshold

Or

– All I/O minus secondary & tertiary I/O > noise threshold

Or

– There have been incoming network sessions for primary services

Or

– There has been an interactive logon

Then

– Server was being useful

Else

– Server was not being useful


Server Compute Efficiency (ScE)

• The proportion of samples that the server is

providing primary services over time (as a

percentage)

• Any server with an ScE of 0% over a prolonged

period is not being used and can be

decommissioned or repurposed

• Any servers with low ScE are worth investigation

– may be candidates for virtualization


Data Center compute Efficiency (DCcE)

• Data Center compute Efficiency (DCcE) aggregates ScE

across all servers in the data center

• Provides a benchmark against which to improve (like

PUE)

• DCcE is NOT a productivity metric – it does not measure

how MUCH work is done, just the proportion of work that

is useful

• DCcE CANNOT and SHOULD NOT be compared

between data centers due to the subjective

determination of secondary and tertiary processes

Thank you for attending

The Green Grid Technical Forum 2011Join The Green Grid Technical Forum

conversation on Twitter! As you are tweeting, use #TGG11.

data center efficiency metrics - the green grid

Documents