effect of rack server population on temperatures in data centers

Effect of Rack Server Population on Temperatures in Data Centers

CEETHERM Data Center LaboratoryG.W. Woodruff School of Mechanical Engineering

Georgia Institute of TechnologyAtlanta, GA 30332-0405

[email protected]

Rajat Ghosh, Vikneshan Sundaralingam, Yogendra Joshi

Collaborators: Pramod kumar, Vaibhav Arghode, Steven Isaacs.

Ghosh, Joshi: ITherm 2012 2

Outline

• Problem Statement.• Methodology

– Experiments.– Computational fluid dynamics (CFD) analysis.

• Results.• Conclusions.

May 30-June 1


Problem Statements• Characterize rack-level air temperatures for

a full-capacity server rack.

• Estimate effect of rack server population on air temperatures in a data center.

• Estimate effect of server location on its CPU temperature and average fan speed.

May 30-June 1


Experimental Setup

May 30-June 1

• Measurement Uncertainty:– Temperature:

– Flowrate:

– Length:

• A data center with 10 server racks arranged in a 5x2 architecture.

• Raised floor plenum supply and overhead plenum return.

• Alternating cold-aisle/ hot-aisle.• CRAC-1 is only active CRAC

–

00.2 .C

1 .mm

320 0.00944 / .CFM m s ( )

9300 .CFM


Test Rack

May 30-June 1

• Server rack (consists of 42 1-U server and headnode) with heterogeneous heat load ranging from 240 W to 0 W.


Containment System

May 30-June 1

• Isolate airflow into the test rack.

Cooling Air from

Plenum

Test RackHot Aisle Containment

Cold Aisle Containment

Perforated Tile

Hot Exhaust Air

Exhaust Plane


Thermocouple Grid

May 30-June 1

600x

z

x

y

Steel Frame

Thermocouple

Tube

• Grid: 21 T-type copper-constantan thermocouples made from 28 gauge (0.9 mm diameter) wire.

• Response time: 28 ms• x-axis: Parallel to rack width.• y-axis: Parallel to tiles.• z-axis: parallel to rack height.


Experimental Procedure1. Deploy the rack-level containment.2. Vary the server population in the test rack

(N=42, 32, 22,12) and measure air temperatures in cold and hot aisles.

3. Vary the location of a server stack and measure temperatures of CPUs and speeds of fans inside servers.

May 30-June 1


Server Population as Parameter

May 30-June 1


Varying Position of a Server Stack

May 30-June 1


CFD Simulation

May 30-June 1

• x-dimension: 2.46 m• y-dimension: 0.60 m• Z-dimension: 1.95 m• 1-U server: 1m x 0.6 m x0.4 m• Server fan:

• Tile: Velocity inlet with 0.8 m/s to match 639 CFM (0.3 m3/s) supply

• Exhaust: Pressure outlet• Server inlet and outlet: Porous jump• Grid number: 1.4 millions for grid-

independent solution

3

2

( ) 0.187

6.76 38.48315.4

p v v

v v


Transient CRAC Supply Air Temperature

May 30-June 1

• CRAC-1 has return air temperature control.

• Variable supply air temperature– Mean=12.2 0C. Std. Dev.=0.9 0C.


Cold Aisle Temperature Variation

May 30-June 1

• With height in the cold aisle, average temperature varies irregularly.

• N=42


Hot Aisle Temperature Variation

May 30-June 1

• With height in the hot aisle, average temperature varies irregularly.

• N=42


CFD-predicted Airflow

May 30-June 1

• Recirculation in the airflow explains irregular pattern of air temperatures in the cold and hot aisles.

• N=42


Temperature Difference Variation

May 30-June 1

• Temperature difference increases with height.

• N=42


Average Temperature in Exhaust Plane

May 30-June 1

• Average temperature in the exhaust plane increases with server population.


Effect of Server Population on Temperature Difference

May 30-June 1

• For all heights, temperature difference increases with server number.


Effect of Containment

May 30-June 1

13 13.5 14 14.5 15 15.5 16 16.5 170

304.8

609.6

914

1219.2

1524

1828.8

2000

Average Temperature in Cold Aisle (0C)

Hei

ght a

long

the

Rac

k (m

m)

Without ContainmentWith Containment

• The containment system reduces average temperature in the cold aisle

- Blocks hot air recirculating from other parts of the room.


Effect of Server Location

May 30-June 1

• Keeping server stack near the highest possible location is more energy-efficient practice in this case- Lowest CPU temperature.- Lowest average server fan speed.

7800 8000 8200 8400 8600 8800 9000 92000

44

89

133

178

222

267

311

356

400

445

489

533

600

Average Server Fan Speed (rpm)Lo

cal H

eigh

t alo

ng th

e Se

rver

Sta

ck (m

m)

TopMiddleBottom

18 20 22 24 26 28 30 320

44

89

133

178

222

267

311

356

400

445

489

533

600

Average CPU Temperature (0C)

Loca

l Hei

ght a

long

the

Serv

er S

tack

(mm

)

TopMiddleBottom


Conclusion• Rack-level temperature field (average temperatures in cold and

hot aisles; and average temperature difference between cold and hot aisles) is characterized for a full capacity server rack (N=42). – Air recirculation thorough the void affects convective temperature field.

• Rack server population has a significant impact on air temperatures– Temperature difference across the rack increases with the server

population in the test rack.

• Recommended best practice for filling out a server stack in an empty stack– Sever stack should be placed at the highest possible location.

May 30-June 1


Acknowledgement

The authors acknowledge support for this work from IBM Corporation, with Dr. Hendrik Hamann as the Technical Monitor. Acknowledgements are also due to the United States Department of Energy as the source of primary funds. Additional support from the National Science Foundation award CRI 0958514 enabled the acquisition of some of the test equipment utilized.

May 30-June 1


Questions?

May 30-June 1

effect of rack server population on temperatures in data centers

Documents