cfd containment modeling

8/17/2019 CFD Containment Modeling

1/32

Computational Fluid Dynamic (C.F.D.)

Modeling of Containment Solutions

Design Model Deploy


2/32

What is C.F.D

CFD or computational fluid dynamic modeling is aprogram designed to simulate how fluids, like airact and react in an environment, from determiningflow patterns to heat absorption. Recently CFD

tools have been designed specifically for the datacenter or IT environment. Great Lakes has begunto offer CFD modeling to its customers in order tohelp address issues, plan capacity increases incurrent operating data centers, or to test a datacenter design or room design before ever startingthe build.


3/32

CFD Model of Cooling Solutions

• Physical Space – Dimensions: 40′ Wide by 60′ Long

– Raised floor height: 24″

– Ceiling height from raised floor: 14′ – 2403 sq. ft traditional raised floor data center

• IT Specifications – 7.2 kW per enclosure

– Total IT Load = 459kW

– 64 Racks at 44 RMU (1408 servers @ 2U) 2240 Ports (servers * 2 ports)

• Room Conditions – CRAC unit exhaust 75 degrees Fahrenheit

– False Ceiling used with chimneys

– Aisle contains 16, 40% perforated floor grilles

An evaluation of containment solutions for datacenters using CFD modeling. A base

line datacenter was used to provide a comparison of different containment

solutions. The following slides outline the datacenter facility and IT specifications.


4/32

3 Phase 480V 1000A service

500 kVA Generator x 1

500kVA UPS system x 2

In Room PDU for 480 to 220V step downPower run under floor (Rack PDU’s hardwired to

BusWay)

30AMP 3Phase Rack PDU with 208V output x 2

per enclosure

Power


5/32

Cooling

• 150 Ton Centrifugal Chiller Unit

• 40 Ton CRAC units with plenum x 4

Traditional Hot Aisle/Cold Aisle configuration500 floor tiles (388 solid, 112 perf when needed)


6/32

Enclosure Configurations

P S U F A N S U P F A B I O M

Cisco Nexus 7000 Series

1 2 3 4 5 6 7 8 9 1

S T A T U S

I D

1

N 7 K - M 1 4 8 G S - 1 1 L

2

3

4

5

6

7

8

1 2

1 1 1 2

1 3

1 4

2 3

2 4

2 5 2 6

3 5 3 6

3 7 3 8

4 7

4 8

9

1 0

1 1

1 2

1 3

1 4

1 5

1 6

1 7

1 8

1 9

2 0

2 1

2 2

2 3

2 4

2 5

2 6

2 7

2 8

2 9

3 0

3 1

3 2

3 3

3 4

3 5

3 6

3 7

3 8

3 9

4 0

4 1

4 2

4 3

4 4

4 5

4 6

4 7

4 8

S T A T U S

I D

1

N 7 K - M 1 4 8 G S - 1 1 L

2

3

4

5

6

7

8

1 2

1 1 1 2

1 3

1 4

2 3

2 4

2 5 2 6

3 5 3 6

3 7 3 8

4 7

4 8

9

1 0

1 1

1 2

1 3

1 4

1 5

1 6

1 7

1 8

1 9

2 0

2 1

2 2

2 3

2 4

2 5

2 6

2 7

2 8

2 9

3 0

3 1

3 2

3 3

3 4

3 5

3 6

3 7

3 8

3 9

4 0

4 1

4 2

4 3

4 4

4 5

4 6

4 7

4 8

S T A T U S

I D

1

N 7 K - M 1 4 8 G S - 1 1 L

2

3

4

5

6

7

8

1 2

1 1 1 2

1 3

1 4

2 3

2 4

2 5 2 6

3 5 3 6

3 7 3 8

4 7

4 8

9

1 0

1 1

1 2

1 3

1 4

1 5

1 6

1 7

1 8

1 9

2 0

2 1

2 2

2 3

2 4

2 5

2 6

2 7

2 8

2 9

3 0

3 1

3 2

3 3

3 4

3 5

3 6

3 7

3 8

3 9

4 0

4 1

4 2

4 3

4 4

4 5

4 6

4 7

4 8

S T A T U S

I D

1

N 7 K - M 1 4 8 G S - 1 1 L

2

3

4

5

6

7

8

12

1112

13 1

4

23 2

4

2526

3536

3738

47 4

8

9

1 0

1 1

1 2

1 3

1 4

1 5

1 6

1 7

1 8

1 9

2 0

2 1

2 2

2 3

2 4

2 5

2 6

2 7

2 8

2 9

3 0

3 1

3 2

3 3

3 4

3 5

3 6

3 7

3 8

3 9

4 0

4 1

4 2

4 3

4 4

4 5

4 6

4 7

4 8

S T A T U S

I D

1

N 7 K - M 1 4 8 G S - 1 1 L

2

3

4

5

6

7

8

1 2

1 1 1 2

1 3

1 4

2 3

2 4

2 5 2 6

3 5 3 6

3 7 3 8

4 7

4 8

9

1 0

1 1

1 2

1 3

1 4

1 5

1 6

1 7

1 8

1 9

2 0

2 1

2 2

2 3

2 4

2 5

2 6

2 7

2 8

2 9

3 0

3 1

3 2

3 3

3 4

3 5

3 6

3 7

3 8

3 9

4 0

4 1

4 2

4 3

4 4

4 5

4 6

4 7

4 8

S T A T U S

I D

1

N 7 K - M 1 4 8 G S - 1 1 L

2

3

4

5

6

7

8

1 2

1 1 1 2

1 3

1 4

2 3

2 4

2 5 2 6

3 5 3 6

3 7 3 8

4 7

4 8

9

1 0

1 1

1 2

1 3

1 4

1 5

1 6

1 7

1 8

1 9

2 0

2 1

2 2

2 3

2 4

2 5

2 6

2 7

2 8

2 9

3 0

3 1

3 2

3 3

3 4

3 5

3 6

3 7

3 8

3 9

4 0

4 1

4 2

4 3

4 4

4 5

4 6

4 7

4 8

S T A T U S

I D

1

N 7 K - M 1 4 8 G S - 1 1 L

2

3

4

5

6

7

8

1 2

1 1 1 2

1 3

1 4

2 3

2 4

2 5 2 6

3 5 3 6

3 7 3 8

4 7

4 8

9

1 0

1 1

1 2

1 3

1 4

1 5

1 6

1 7

1 8

1 9

2 0

2 1

2 2

2 3

2 4

2 5

2 6

2 7

2 8

2 9

3 0

3 1

3 2

3 3

3 4

3 5

3 6

3 7

3 8

3 9

4 0

4 1

4 2

4 3

4 4

4 5

4 6

4 7

4 8

S T A T U S

I D

1

N 7 K - M 1 4 8 G S - 1 1 L

2

3

4

5

6

7

8

1 2

1 1 1 2

1 3

1 4

2 3

2 4

2 5 2 6

3 5 3 6

3 7 3 8

4 7

4 8

9

1 0

1 1

1 2

1 3

1 4

1 5

1 6

1 7

1 8

1 9

2 0

2 1

2 2

2 3

2 4

2 5

2 6

2 7

2 8

2 9

3 0

3 1

3 2

3 3

3 4

3 5

3 6

3 7

3 8

3 9

4 0

4 1

4 2

4 3

4 4

4 5

4 6

4 7

4 8

S T A T U S

I D

1

N 7 K - M 1 4 8 G S - 1 1 L

2

3

4

5

6

7

8

1 2

1 1 1 2

1 3

1 4

2 3

2 4

2 5 2 6

3 5 3 6

3 7 3 8

4 7

4 8

9

1 0

1 1

1 2

1 3

1 4

1 5

1 6

1 7

1 8

1 9

2 0

2 1

2 2

2 3

2 4

2 5

2 6

2 7

2 8

2 9

3 0

3 1

3 2

3 3

3 4

3 5

3 6

3 7

3 8

3 9

4 0

4 1

4 2

4 3

4 4

4 5

4 6

4 7

4 8

S T A T U S

I D

1

N 7 K - M1 4 8 GT - 1 1 L

2

3

4

5

6

7

8

1 2

1 1 1 2

1 3

1 4

2 3

2 4

2 5 2 6

3 5 3 6

3 7 3 8

4 7

4 8

9

1 0

1 1

1 2

1 3

1 4

1 5

1 6

1 7

1 8

1 9

2 0

2 1

2 2

2 3

2 4

2 5

2 6

2 7

2 8

2 9

3 0

3 1

3 2

3 3

3 4

3 5

3 6

3 7

3 8

3 9

4 0

4 1

4 2

4 3

4 4

4 5

4 6

4 7

4 8

Server Enclosure (63 enclosures)GL840ES-304220 x Dell R710 2 RMU servers @ 400W

Network Enclosure (1 enclosure)GL840ES-30481 x Cisco Nexus 7010

Remaining RMU on each rack to be populated with FillerPanels. Sides of enclosures to have Brush Grommet kitsinstalled.


7/32

Configurations

Enclosure Level Containment

With Air Managers and Chimneys

Standard Cold Aisle Containment

Cold Aisle Containment With

Horizontal Polargy Panels

Standard Hot Aisle Cold Aisle


8/32


With Air Manager and Chimneys(5 floor tiles per aisle)

Server Enclosure ConfigurationGL840ES-3042LC

Frame: 8401ES-3042 – 84”H x 30”W x 42”D

Top Panel: Solid Top - TPES-S

Front door: Mesh Contour – 8402E-MC29

Rear Door: Solid - 8403E-29

Air Manager: GL-AMES

Chimney: GL-EC-42-3246

BGRK30 – Side rail grommet kit

Network Enclosure Configuration

GL840ES-3048LC

Frame: 8401ES-3048 – 84”H x 30”W x 48”D



Rear Door: Solid - 8403E-29

Air Manager: GL-AMES

Chimney: GL-EC-48-3246BGRK30 – Side rail grommet kit

Containment Components:

1.75-FPTL19 – Tool-less filler panels for

unused RMU

Additional Notes:

5 floor tiles per cold aisle


9/32



PerformanceAir Handler Supply: 75

Air Handler Return: 98.2

Room Temp. Change: 6.7


10/32



Temperature, cabinet bottom height

Cold aisle

temperatures are

very consistent with

no air recirculation.


11/32



Temperature, halfway cabinet height

Use of chimneys

eliminates the hot

aisle maintaining

a consistenttemp. throughout

the data center.


12/32



Temperature, cabinet top height

Temperatures in

the cold aisle at

the top of the

enclosures arewithin range of

the air handler

set-points due to

the elimination of

the hot aisle. The

overall temp inthe data center is

consistent.


13/32



Temperature, 12″ above cabinet

Exhaust air from

the chimneys is

directed into the

drop ceilingwhere the air

handlers with

baffles installed

into the ceiling

draw the hot air

in to be

conditioned.


14/32



cross section

Equipment

intake

temperatures

are consistentfrom top to

bottom. Hot air

is ducted into

the plenum

where hot air is

returned to the

CRAC unit.

Overall the system is highly efficient in segregating hot air

from the room providing a high return temperature to the

CRAC unit. However a plenum ceiling or plenum ducting is

needed to return to the CRAC unit.


15/32

Standard Cold Aisle ContainmentWith Horizontal Polargy Panels

Server Enclosure ConfigurationGL840ES-3042MS

Frame: 8401ES-3042 – 84”H x 30”W x 42”D



Rear Door: Mesh Contour – 8402E-MC29



GL840ES-3048MSFrame: 8401ES-2448 – 84”H x 30”W x 48”D






ACD840ES – Aisle Containment Doors

PPLEX-HPNL24 – Horizontal PolargyContainment Panels

1.75-FPTL19 – Tool-less fil ler panels for

unused RMU


16/32


With Horizontal Polargy Panels

Performance Air Handler Supply: 75°F

Air Handler Return: 98.2°F

Room Temp. Change: 19.2°F


17/32


With Horizontal Polargy PanelsTemperature, cabinet bottom height

End of row

containment

creates a

consistent cold

aisle temperature


18/32


With Horizontal Polargy PanelsTemperature, halfway cabinet height


19/32


With Horizontal Polargy PanelsTemperature, cabinet top height

Overhead

containment panels

reduce

recirculation in the

top of enclosures

creating very

consistent temps in

the aisles of the data

center.


20/32


With Horizontal Polargy Panels(cross section view)

Equipment

intake

temperatures

are consistentfrom top to

bottom

Overall the system is efficient in segregating hot air from the

room providing a high return temperature to the CRAC unit.

However end of row doors and overhead panels add

additional infrastructure and safety considerations.


21/32



Frame: 8401ES-3042 – 84”H x 30”W x 42”D












ACD840ES – Aisle Containment Doors

1.75-FPTL19 – Tool-less filler panels forunused RMU


22/32






23/32

Standard Cold Aisle ContainmentTemperature, cabinet bottom height

Cold Aisle

containment reduces

recirculation creating

more consistenttemperatures in the

hot and cold aisles


24/32

Standard Cold Aisle ContainmentTemperature, halfway cabinet height


25/32

Standard Cold Aisle ContainmentTemperature, cabinet top height

Recirculation still

occurs at the tops of

the enclosures creating

higher exhaust tempsand less even

temperatures in the

data center.


26/32

Standard Cold Aisle Containment(cross section view)

Equipment

intake

temperatures

fluctuate at theends of rows

and tops of

enclosures due

to re-circulation.

Overall the system is efficient in segregating hot air from the room

providing a high return temperature to the CRAC unit. However end ofrow doors add additional infrastructure and safety considerations and

cold air can by-pass the cold aisle resulting in losses in efficiency as well

as hot air re-circulation.


27/32

Standard (Hot Aisle/Cold Aisle)


Frame: 8401ES-3042 – 84”H x 30”W x 42”D











Additional Accessories

1.75-FPTL19 – Tool-less fil ler panels for

unused RMU


28/32

Standard (Hot Aisle/Cold Aisle)





29/32

Standard (Hot Aisle/Cold Aisle)Temperature, cabinet bottom height

Recirculation in

the cold aisle

raises equipment

inlet temperature


30/32

Standard (Hot Aisle/Cold Aisle)Temperature, halfway cabinet height


31/32

Standard (Hot Aisle/Cold Aisle)Temperature, cabinet top height

Recirculation in the

cold aisle raises

equipment inlet

temperature which

creates a higher

exhaust temperature


32/32

Standard (Hot Aisle/Cold Aisle)(cross section view)

Recirculation from

around the aisle and

over the top of the

enclosures creates

high equipment

intakes in the cold

aisle.

Overall the system segregates hot air from the cold aisle.

However, hot air recirculation can occur in the top third RMU

of the equipment causing early equipment mortality and

thermal shutdowns.

cfd containment modeling

Documents