pros and cons of various dc distribution architectures · dc - dc ac - dc distribution pdu ups...

Pros and Cons of Various DC Distribution Architectures

Randy Malik

Power Technology And Qualification

IBM RTP Raleigh NC

Rick Fishbune

Power Technology And Qualification

IBM Rochester, MN

Source: Solar Powered Datacenter by Google

High Voltage Distribution Bus

© 2004 IBM Corporation2 9/26/2008

Large Energy ConsumersFor 100 units of Coal (BTU) only 33 units available for use !!!

Data Centers Consume 1.5% of total Electricity in USA

http://images.google.com/imgres?imgurl=http://cdn.channel.aol.com/aolr/global-issues-warming-400a042007.jpg&imgrefurl=http://teens.aol.com/originals/red-cares/global-issues&h=400&w=400&sz=22&hl=en&start=65&tbnid=7p2Z990V4g1LcM:&tbnh=124&tbnw=124&prev=/images%3Fq%3Dglobal%2Bwarming%26start%3D60%26gbv%3D2%26ndsp%3D20%26svnum%3D10%26hl%3Den%26sa%3DN



Sources: US EIA, US Census Bureau, (Patzek, 2007).

Energy Crisis for the Planet Earth



Critical Power &

Cooling

Hardware & Software

Desired Business Objectives

Investment is 2 – 4% of the total

Capital Expenditure in the Datacenter

Power and Cooling - the foundation to achieve business Objectives

Annual Energy Cost in a datacenter is approaching Capital Expenditure in a Datacenter …… Source IDC



A Typical Tier 4 Datacenter Infrastructure



Raised Floor

ROW 1

ROW 2

ROW 3

ROW 4

STS

PDU

PDU

Air

Han

dler

Air

Han

dler

Battery Plant

3 Phase

AC UPS 2

3 Phase

AC UPS 1

ATS for Chillers

Row of IT Racks

480V AC 3 Phase

480V AC 3 Phase

Free Air

Free Air

Simplified Diagram of a Tier 4 Datacenter

480 V

208 V

208 V



Power Distribution in a Tier 4 Datacenter

13.2KV/480

Transformer

13.2KV/480

Transformer

AC Generators

AC Generators

Double Conversion

AC UPS

Double Conversion

AC UPS

Battery Bank

3 Phase

480Vac

3 Phase

480Vac

Electronic

Transfer

Switch

PDU

480/208 V AC

PDU

480/208 V AC

AC

AC 2

Raised Floor Data Processing Room

A row of Data Processing Racks

208 V AC

3 Phase

208 V AC

3 Phase

96.3%

96.3%

99% 98.9%

98.9%

90%Best efficiency for 12V AC – DC Rectifier

Efficiency From 480 Vac – 12V DC : ( .99x .989 x .90 ) = 88%

13.5 KV

13.5kV



48V or 400V DC Distribution Concept for Tier 4 Datacenter

13.2KV/488 Vac

Transformer

13.2KV/488 Vac

Transformer

AC Generators

AC Generators

AC – DC Rectifiers in

parallel

Battery Bank

Raised Floor


parallel

DC Bus 1 ( 400V DC or 48V DC)

480V AC 3 Phase

480V AC 3 Phase


98%

98%

97%

13.5KV

13.5KV

Neutral

Neutral

400V - 12V Unregulated

Best Possible Efficiency (Non-isolated) = .98 x .97 = 95%

Best Possible Efficiency (Isolated) = .96 x .97) = 93%

400V DC Distribution Issues:

1. Safety Concern

2. Customer Perception

3. Certified connectors are not available

Rack



13.2KV

13.2KV

13.2KV/488 Vac

Transformer

13.2KV/488 Vac

Transformer

AC Generators

AC Generators


parallel

Battery Bank

Raised Floor


parallel


480V AC 3 Phase

480V AC 3 Phase


98%

98%

97%

Neutral

Neutral

400V - 12V Unregulated Rack

Solar Powered Datacenter with DC Distribution

DC – DC

Converter

X

X

Estimated Savings in Power = 5 MW

Estimated Savings in Electric Bill = 5 M Dollars/Year !

http://www.biblelife.org/energy-photovoltaic.jpg



DC Distribution in a Datacenter

480V AC

UPS

PDU

PDU

208 /120V

208 /120V

IT Racks

UPS

6.5%

2.5%

Rectifiers

400V DC or 48V DC

Battery Plant

http://images.google.com/imgres?imgurl=http://www.crucialpower.com/img/products/PDU1.jpg&imgrefurl=http://www.crucialpower.com/PDU2.html&h=387&w=360&sz=38&hl=en&start=13&um=1&tbnid=HjXm_4d-KQ5VuM:&tbnh=123&tbnw=114&prev=/images%3Fq%3Dpdu%26svnum%3D10%26um%3D1%26hl%3Den%26sa%3DG

http://images.google.com/imgres?imgurl=http://www.crucialpower.com/img/products/PDU1.jpg&imgrefurl=http://www.crucialpower.com/PDU2.html&h=387&w=360&sz=38&hl=en&start=13&um=1&tbnid=HjXm_4d-KQ5VuM:&tbnh=123&tbnw=114&prev=/images%3Fq%3Dpdu%26svnum%3D10%26um%3D1%26hl%3Den%26sa%3DG



70% 75% 80% 85% 90% 95%

0

500

1000

1500

2000

2500

3000

3500

Max

imum

Out

put P

ower

(W)

X Series Power Supply Efficiency Spread @ 100% Load

Multiple Output Power Supplies



480Vac Plug to Processor Efficiency for a typical DatacenterOverall

Efficiency and Normalized Power

Raised Floor Transformer

Distribution AC – DC Power Supply

DC – DC Regulator

Processors, Memory, ICs)

AC Distribution

63.236 %

1.58

77.16 %

1.295

85.10%

1.175

85.25%

1.14

89.605%

1.116

480V AC– 400 VDC – 480V AC

400V AC – 208V AC : 3 Phase

Data Center Distribution

208Vac – 12V DC 12 VDC – 1V DC 1V to Heat

Typical Efficiency 91.6 97.5 98 85 85

Normalized Power 1.58 1.448 1.412 1.384 1.176 1

Best in Class Efficiency 96.3 98.9 99 93 88


Best Efficiency Technology limit

96.3 98.9 99 95 95


400V DC Or 48V NA NA Distribution Efficiency

AC – DC Rectifiers

12 VDC – 1VDC

Or 5Vdc – 1V1V to Heat

Efficiency :Isolated 99 93 95

Normalized Power 1.14 1.129 1.05 1

Efficiency No Isolated Rectifiers

99 95 95

Normalized Power 1.116 1.105 1.05 1

UPS System



Compute Capacity Increase due to DC Distribution and Power System Efficiency Increase

51%

10%

38%

1%

IT Power System Cooling Lighting

Increase in Compute Capacity in a Datacenter

Power Consumption in a Typical Datacenter

38%

23%

38%

1%

IT Power System Cooling Lighting



0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Typical Best in Class Best Possible Isolated Nonisolated

Pow

er D

issi

pate

d (W

att)

DC - DC AC - DC Distribution PDU UPS

Power Saved by using DC Distribution and Efficient Power Supplies

DC DistributionAC Distribution

- 0.464W

Power Saved = 0.464W to power 1 Watt LoadTotal Electrical Utility Bill savings = 464000 Dollars/year for 1 MW IT load



N+1 vs. N+N Downtime (seconds) in a year

0

10

20

30

40

50

60

1+1 2+1 3+1 4+1 5+1 6+1 7+1 8+1 9+1Number of rectifiers

Dow

ntim

e in

Sec

onds

)

Downtime

( MTTR = 48 Hours, MTBF = 250K Hours

Availability (A) = 1 – MTTR/MTBF for a single Rectifier

Availability with N +1 Rectifiers in parallel = An+1 + (n+1) x An x (1-A)



N+1 vs. N+N Service Requirement Frequency

0.00%

0.05%

0.10%

0.15%

0.20%

0.25%

1+1 2+1 3+1 4+1 5+1 6+1 7+1 8+1 9+1Number of rectifiers

Ser

vice

Fre

quen

cy

Service Frequency

Service Frequency = 1 – An+1 MTTR = 48 Hours, MTBF = 250K Hours



Comparison of Different Power Architectures

Typical AC AC – Best in Class

AC – Best Possible

Isolated 48V Isolated / Non-isolated 400V DC

DC Redundancy At Server or Rack Level

At Server or Rack Level

At Server or Rack level

Central Rectifier Central Rectifier

Industry Standard Components

Yes Yes Yes Yes No

Customer Acceptance

High High High Acceptable Very Low

Battery Issue NA NA NA Low Failure High Failure due to cells in series

AC UPS required Yes Yes Yes No No

Neutral availability on Raised Floor

NA NA NA No Concern Not Required /Required

Repair Frequency High High High Low Low

Overall Cost Low at Power Supply level but high at Datacenter level

Low at Power Supply level but high at Datacenter level



High at Power Supply level but low at Datacenter level

Safety Concern No Concern No Concern No Concern No Concern Great Concern



Minimum Power Point Tracking Tool

Air Handler

UPS

Rectifiers

CRAC

Humidifier

Server Racks

MPP

Tracking Tool



Additional Energy Saving Techniques

% Energy Savings

16%

18%

23%11%

29%

3%

Low power Processors

Power Management

Server Virtualization

VFD Cooling

Supplemental Cooling

Cooling units work as ateam

Source: Emerson Network Power White Paper



Conclusion

Lower Overall Cost

No Step Down Transformers

No AC UPS

Less Feeds from Wall

Higher Reliability, Availability, and Scalability

Higher overall Efficiency

DC Distribution best suited for Solar Array Power

Less Repair frequency

pros and cons of various dc distribution architectures · dc - dc ac - dc distribution pdu ups...

Documents