DC Power Distribution Alternatives for Datacom Applications

BJ Sonnenberg Manager Business Development

Data Center Trends

Increasing rack power density Increasing drive for efficiency Increasing component integration on site

`
AC Power DC Power
Future Higher Voltage Topologies

>40kW

New High Efficiency Topologies

New 48V DC Topologies

< 2 kW
Traditional Telecom 48V DC AC UPS Powered Configurations Modularity & Scalability

Forces re-evaluation of all available topologies

2 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Data Center Power Protection and Distribution Basic Architectures

Traditional AC UPS
480V MV AC AC/DC DC/AC Maintenance Bypass
B

Primary Dist.
B

PDU CDU PSU DC/DC 12V VR

UPS

Bypass
B B B B B

AC/DC

Server

Rack 48V/380DC

300-400VDC
CDU PSU
B

DC UPS
480V AC MV

Modular System, Row-based

AC/DC
B

DC/DC

12V VR

Rectifier

PDU Rack

Server

Attributes of a DC UPS: Simple input and output distribution, easy to parallel ,no load balancing (derating) Minimal conversion stages grid to chip high end-to-end efficiency and reliability High power quality - isolated from mains, no harmonics Safe voltages at point of use (48VDC systems)

Advantages of integrated, modular UPS systems: High power density in a compact footprint Reduced field wiring and copper content Targeted for row-based deployments; no need for dedicated power room

System optimization challenge best overall system topology for a specific application , not necessarily optimization of individual existing components.
Source: Intel

Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Improved Efficiency with DC Solutions

Elimination of conversion stages and simplified distribution

4 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Megatrends in Data Center Distribution Evolution

Renewable resources Microgrids

Facility distribution optimization

Rack/server distribution optimization

Decrease utility power consumption peak shaving and over time. Address all loads , not only compute loads (lighting , cooling , etc) Server virtualization Maintain high availability Best system utilization rightsizing , no stranded power Scalability TCO

Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Impact of Renewable Energy Sources - Example of Telecom Facility Transformation Vision

Present
Facility AC loads

Utility
AC UPS AC compute loads
AC DC

Ultimate
Facility AC loads

Gen

Batt DC UPS Batt

Telco and dc compute loads

48VDC

Many transition paths

Utility

Facility DC loads

Interim

DC Energy Station

Batt

DC compute And telco loads

Facility AC loads

Green Generator

Facility DC loads

400VDC

AC DC

AC compute Legacy loads

Utility
DC UPS

Gen

Batt

DC compute And telco loads

DC DC

DC facilitates and optimizes use of renewable resources

400VDC

48VDCTelco Legacy loads

6 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Elements of an Optimized Power Distribution Grid to Chip

BEST COMPROMISE OF THE FOLLOWING
Efficient facility distribution to equipment racks Ability to easily connect renewable energy resources Efficient equipment rack distribution Efficient server PCB distribution optimize conversion stages

Eliminate stranded power Operate at peak efficiency at any load Scalability Low TCO Low initial cost Reliability
7 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Why 380VDC - Voltage Range Selection Based on Several Criteria

1500 IEC 1000 800 900 BS (U.K.) 750 Ordinance (JPN) 600 600 NEC(U.S.) 450 428 (192cell) Validus 374 (168cell) 380 321 (144cell) 300
US DC Demo 575 [to 48]

High Long

DC Voltage

Distribution efficiency

Cable Distance

Server PS

405

380

373(U.K.)

420 380

350 320
Operating bulk - voltage

354(EU) 324V (US 311(JPN)

300
NEDO PJ. (Sendai)

260
Low Short
ETSI Std. (draft)

200

Law, Regulation, Code, and Stds.

Distr. Gen.

Telecom (Number of Cells)

Demos (Reference)

Rating voltage of parts and elements

AC ICT input voltages ( Peak )

Benefit of HVDC system

Edited from source: NTT FACILITIES, INC.

Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Which DC Voltage?
Depends on facility size/application : 1. Scalable facility level distribution
Sites/PODs requiring battery back-up - -48VDC in row solutions Facility/application with renewable energy availability 380VDC simpler interface

2. Server rack level distribution

-48VDC or 380VDC for racks with power densities > 10kW

9 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Power Room Constraint

As rack densities increase, power room size increases proportionally; i.e. a 10x increase in density results in a 10x increase in power room size.
Data Center @ 2kW per Rack High-Density Data Center @ 20kW per Rack

Power and Switchgear Room

Switchgear

Room-based UPS

Data Center Raised Floor

UPS / Battery Room

Data Center Raised Floor

Row-based UPS

No UPS Room, Increased computing space

10

Row-based UPS architecture offers ~30% more computing capacity

Switchgear

Additional Data Center Raised Floor Space

Data Center Raised Floor

Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

380V DC Application Alternatives

Facility level distribution Server rack distribution

11 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

380VDC System Components

CONVERSION
Communications

FACILITY DISTRIBUTION

RACK DISTRIBUTION

System cabinet Rectifier Controller Storage , battery flywheel ,etc

Bus duct, ancillary switchgear

Power strips connectors to servers

Power supply

12 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

DC Rectifier and System Design Philosophy

Modular approach selected for scalability and reliability Maintain same reliability and maintainability as 48VDC systems Provide input/output isolation to avoid protection coordination issues Minimize footprint space savings High scalability and ease of expansion ( from 12.5kW up ) Efficiency gains through conversion improvement and facility design Ready to accept renewable energy sources Offer multiple ratings by use of standard building blocks and conversion cabinet configurations and a set of input/output interface accessories

13 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Advantage of High Modularity Improved Efficiency with Energy Optimization Mode

Active power management increases overall system efficiency
Unneeded power conversion units (PCUs) set to standby in low-load conditions Optimizes individual PCU load for greater system efficiency Useful for sites with variances in load Periodically cycles energized PCUs for even run time

33% 33% 33% 33% 33% 33%

14

33% 33% 33% 33% 33% 33%

56% 56% Standby

56% 56% Standby 56% Standby 56%

N+1 redundancy maintained

56% Standby Standby

Without Energy Optimization

With Energy Optimization

Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

1U high 10KW Rectifier

Feature Input voltage windows Output power Power factor THD Output voltage Max output current Efficiency Safety 260 to 530VAC 10KW 0.99 <5% 297 to 400V 30A 95% UL/CE

15 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Emersons 380V V0 Solution (Present)

Feature Input Output 480VAC with 3 phase 4 wire 297 to 400V

Provides up to 100KW in each rack Load insulation detection function Max 7 output load connection with load measurement Over load and on/off detection function Standard server rack 600(w)*1000(D)*2000(H) :mm

16 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Future Offer -System Sizing and Configuration Considerations

Rectifiers space in cabinet (cooling,safety) DC output configurations bus duct vs hard wire/Tier level AC input configurations Tier level Maintainability level facility level and DC UPS system Components rating limitation (today) Footprint optimization Ease of installation/expansion
Tier 1/2/3 N+M configuration with multiple inputs/outputs Tier 4 configuration 2N

17 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

400VDC System Building Block Implementation Example

480VAC Battery or
Include provision for internal bussing Current / Voltage Monitor
Draw-out or Plug-in Breakers Preferred Fused Disconnects acceptable
1000A CB

Loads A

Loads B

flywheel

1000A CB

Distr. A
40 MCCBs

Distr. B
40 MCCBs 8 brks up to 125A Individually monitored option

C 20 -22 Rectifiers 250kW N+1

400A CB 400A CB

8 brks up to 125A Individually monitored option

1. Any part of the system can be maintained without system shutdown 2. Distribution breakers can be added and wired without system shutdown 3. Top and bottom entry provided 4. All external wiring run in conduits sufficient space to be provided for conduit terminations 5. Failure of either bus A or B should not affect the other bus 6. For single corded loads only one distribution is required 7. Rectifiers can be added or replaced without system shutdown 8. System can operate without battery 9. Individual Rectifier failure does not affect the output bus 10. All cabinet connections must be with conduit 11. UL / CSA / CE approved

18

Battery or Loads A Loads B flywheel Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

480VAC

Controller

Output Connectivity Options Bus Duct Advantage

Bus duct Wire in conduit

Distribution options:
A
Battery Flywheel

1.Fuses shorter clearing time 2.Breakers easier to operate 3.Bus duct plug-ins space,scalability

Server Rack

400VDC UPS

DistrA

DistrB

Bus duct

800A CB limit 230kW

Bus duct

A
Battery Flywheel

B
Battery Flywheel

A
400VDC UPS

400VDC UPS

Server Rack

Server Rack

19 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

AC Input Configurations
Example of 250kW System Scalable to 750kW
Utility A Utility Utility Utility

1200A 250kW 250kW 250kW

OR

400A 250kW

400A 250kW

400A 250kW

TIER 2

Utility Utility A B

Utility Utility Utility Utility Utility Utility A B B A B A

OR 1200A 250kW 250kW 250kW 400A 250kW 400A 250kW 400A 250kW

TIER 3

20 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Partial 2N Concept
Server Room or Electrical Room Server Room or Electrical Room Server Room

Battery Flywheel Optional

400VDC UPS N+1

B A

Battery Flywheel Future

Battery Flywheel

400VDC UPS n=+1

400VDC UPS N+1

400VDC UPS Red. N+1

400VDC UPS Future N+1

Server Rack Critical

Server Rack Non critical

Server Rack Future

Facility Loads

21 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Server Rack Distribution

Feed A Feed B Feed A Power strip 50A input ~14.5kW 12 positions Single strip can power up to 12 2U servers at 1200W A or A+B or 24 1U servers at 600W
12- 40 servers

Feed B

Power strip 50A input ~14.5kW 12 positions Single strip can power up to 2 blade chassis at 14.5kW A or A+B For A only configuration up to 4 blade chassis (two strips) total rack power available 29kW

For A only configuration up to 20 2U servers,40 1U servers (two strips) total rack power available 29kW

Blade chassis example HPc7000 today 3x2700W power supplies ~8kW max Typical load- 4-6kW

Connector UL listed at 5A or 20A End of discharge voltage ~ 290VDC

22 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

380V DC Application Alternatives

Facility level distribution Server rack level distribution

23 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

380VDC Sub-Rack Sun ContainerUCSD POC Site

2 X 10kW 400VDC Controller Output Breakers 400V-48V Converters

24 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

48VDC or 380VDC Sub-Rack Concept

Horizontal shelf mounting shown . Shelves can also be mounted vertically

Building blocks: 12.5-15kW 380VDC 6000W 48VDC

25 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Rack Distribution Copper Content

Assumptions: 1. 92 servers/rack at 275W-300W 2. Two power systems/rack 13.5kW each, n+1 3. Rack power ~ 27kW 4. Currents calculated at end of battery discharge 1.75VDC

26 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Rack Layout -Optimized

11x300W 11x300W

servers 3375W

servers 3375W

Cross bar 20 12V 625A @1.75V 48V - 156A @1.75 V 380V 23A @1.75V 30A,480V, AC,3phase input plug

13.5kW
11x300W 11x300W

servers 3375W

servers 3375W

11x300W

11x300W

servers 3375W

servers 3375W

Distribution bar 47.25 12V 312 A 48V 78A 380V 12A

13.5kW
11x300W 11x300W

servers 3375W

servers 3375W

Server plug 12V - 25A 48V 6.25A 380V - ? (5A plug standard)

27 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Rack Layout Full Bus Sizes

11x300W 11x300W

servers 3375W

servers 3375W

Cross bar 20 12V 1250A @1.75V 48V - 312A @1.75 V 380V 46A @1.75V 30A,480V, AC,3phase input plug

13.5kW
11x300W 11x300W

servers 3375W

servers 3375W

11x300W

11x300W

servers 3375W

servers 3375W

Distribution bar 47.25 12V 614A 48V 156A 380V 24A

13.5kW
11x300W 11x300W

servers 3375W

servers 3375W

Server plug 12V - 25A 48V 6.25A 380V - ? (5A plug standard)

28 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Back-up

29 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Example of Hybrid System Evolution in Enterprise Building Existing System

100kW
350-800V 480Vac

Meter
480Vac

Solar Array

PV Inverter Eff 93%

480Vac

AC Distr.

Utility

Load larger than PV output .Solar power fed back to utility.

Lighting

480/208V PDU
208/120VA servers

Eff 98%?

PS eff 92%?

Solar to server eff = .93x.98x.92= .838 Utility to server eff = .98x.92=.90 Lighting and distribution losses not included
30 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Hybrid System Migration -Scenario 1

100kW
350-800V 480Vac

Meter
480Vac

Solar Array

PV Inverter
480Vac

AC Distr.

Utility

NetSure ITM 48V DC UPS

Already existing on site

48VDC Loads

380VDC Converter
380VDC Loads

Existing Emerson products

PS eff 95%?

Solar to server eff = .93x.97x.95=.856 Utility to server eff = .97x.95=.92 Lighting and distribution losses not included
31

Converter to 24VDC Lighting 24VDC

Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

Hybrid System Migration -Scenario 2

100kW
300-400V

Solar Array

DC/DC Regulator 380VDC output Eff- 97%

380VDC Converter No battery

Eff- 97%

480Vac

AC Distr.

Utility

Replaces Existing PV Inverter

380VDC distr. backbone

(already considered for buildings/residential applications)

NetSure ITM With Battery

Converter to 24VDC Lighting 24VDC

380VDC Loads PS eff 95%?

380-48VDC Converter

48VDC Loads
Vehicle charge station Appliances

32

Estimated efficiencies detailed study in progress Solar to server eff = .97x.95 = .92 Utility to server eff = .97x.95 = .92 Lighting and distribution losses not included

Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.

400VDC System Voltages

Cell type Number of cells 144 150 156 162 168 174 180 186 Number of blocks 24 25 26 27 28 29 30 31 Nominal VDC 288 300 312 324 336 348 360 372 Operating VDC @2.25VDC/cell 324.00 337.50 351.00 364.50 378.00 391.50 405.00 418.50 Operating VDC @2.17VDC/cell Min VDC @ 1.75VDC/cell end of discharge 252.00 262.50 273.00 283.50 294.00 304.50 315.00 325.50 48VDC multiples 6.00 6.25 6.50 6.75 7.00 7.25 7.50 7.75

VRLA -6 V block

Wet cells Single cell

equalize 162 324 364.50 163 326 366.75 164 328 369.00 165 330 371.25 166 332 373.50 167 334 375.75 168 336 378.00 169 338 380.25 170 340 382.50 171 342 384.75 172 344 387.00 173 346 389.25 174 348 391.50 175 350 393.75 176 352 396.00 177 354 398.25 178 356 400.50 179 358 402.75 180 360 405.00 To optimize battery applications the voltage range should be:

351.54 353.71 355.88 358.05 360.22 362.39 364.56 366.73 368.90 371.07 373.24 375.41 377.58 379.75 381.92 384.09 386.26 388.43 390.60

283.50 285.25 287.00 288.75 290.50 292.25 294.00 295.75 297.50 299.25 301.00 302.75 304.50 306.25 308.00 309.75 311.50 313.25 315.00

Outside ETSI spec

Nominal Min Max Typical operating voltage

336VDC-352VDC 294VDC 400VDC 378VDC VRLA 386VDC Flooded cells

33

Not counting cable voltage drop.

Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information.