interpack '09 asme/pacific rim technical conference and exhibition

© 2009 Hewlett-Packard Development Company, L.P.The information contained herein is subject to change without notice

Unified Thermal and Power Management in Server Enclosures

Niraj Tolia, Zhikui Wang, Manish Marwah, Cullen Bash, Parthasarathy Ranganathan, Xiaoyun Zhu*

*Now at VMware

2005: $26.5 Billion

“The cost of power and cooling is likely to exceed that of hardware…”

- Luiz Barosso, Google

“In the data center, power and cooling costs more than the IT equipment it supports.”

- Christian L. Belady, Microsoft (former HP)

Zephyr• Unified Power and Cooling Management

Contributions• Demonstrate how to do per-blade cooling• Save both cooling (21--30%) + system power (23--29%)

• Without impacting performance

Compact DesignDense Compute and Storage

Blade Servers:

Workloads are Virtualized

Compact DesignDense Compute and Storage

Blade Servers:

Workloads are Virtualized

Web ServerOS

Hardware

Virtualization:

VM MonitorHardware

Virtual Machine

WebServerOS

Virtualization:

Virtual Machine

WebServerOS

OSVM MonitorHardware

VM MonitorHardware

Virtual Machine

App ServerOS

Virtualization:

Virtual Machine

WebServerOS

OSVM MonitorHardware

VM MonitorHardware

Virtual Machine

App ServerOS

Virtualization:

Virtual Machine

WebServerOS

OSVM MonitorHardware

VM MonitorHardware

Virtual Machine

App ServerOS

Virtualization:

CPU P-states BladeEnclosure

PowerUtil

Temp.Zephyr

Min.Power

Tref

Utilref

Models

Fan Speed

VM Migr.Blade On/Off

Power ModelsTemperature Models

0

50

100

150

200

250

0 20 40 60 80 100

Pow

er (

W)

Utilization

Power - P0Power - P4Power Model - P0Power Model - P4

Power Model: Single Blade

Power Model: Single Fan

0 6000 12000 180000

50

100

150

200

Speed (RPM)

Pow

er (W

)

Measured Fan PowerFan Power Model

Unfortunately, temperature models are significantly more complex.

Models don’t existsMany things affect temperatureInsufficient sensors in servers

Zonal Variations

Fan No.

Blad

e N

o.

1 2 3 4 5 6 7 8 9 10

123456789

10111213141516

Zonal Variation: Fan Influence

Temperature Models• Steady State Model−Assumes workloads, ambient environment is constant

• Transient Model−Finer-grained control in real-world environments

Temperature Models• Steady State Model−Assumes workloads, ambient environment is constant

• Transient Model−Finer-grained control in real-world environments

ZephyrBlade

Enclosure

PowerUtil

Temp.

Min.Power

Tref

Utilref

Models

Fan SpeedCPU P-statesVM Migr.

Blade On/Off

ZephyrBlade

Enclosure

PowerUtil

Temp.

Min.Power

Tref

Utilref

Models

Fan SpeedCPU P-statesVM Migr.

Blade On/Off

FC

ECGC

Group Controller (GC)• Goal: Minimize Power(Enclosure)

• Changes state every 10 minutes• Monitors Utilization, Inlet Temperature• Uses two-step optimization−Simulated Annealing for space search−Convex Optimization for Fan Cost + Blade Power Model

Experimental Setup

Storage Array

Controller

Temp and Util. DataVM Migration

Fan Speed Control

16 Blades (4 cores)5 P-States

10 Fans, 64 Xen VMs

0

50

100

150

200

250

300

IT

Pow

er (W

)Static Integral Predictive Zephyr Zephyr + On/Off

Cooling Power Savings

626

System Power Savings

0

500

1000

1500

2000

2500

3000

3500

4000

IT

Pow

er (W

)Static Integral Predictive Zephyr Zephyr + On/Off

Simulated Servers

239

34

68

102

136

170

204

120

154

190

222

256

290

0

50

100

150

200

250

300

0 20 40 60 80 100

Power Usage (W

)

Utilization

Real

EnergyProportional

PracticalNext-Gen

Simulation: Tamb+15C &13 servers types

Conclusions• Zephyr combines:−Concepts from Heat Transfer Theory with Systems

• Unified power and cooling management can save:−30% Cooling Power−29% Enclosure Power

Full Length Papers:www.hpl.hp.com/personal/Niraj_Tolia/