You are on page 1of 41

Understanding Load Banks

and Best Practices for Multi-Level


Data Center Testing
FM 3.2 Commissioning
Mark Siira, Director, Business Development
ComRent International

Session Description
Data center owners and commissioning agents
make three common mistakes around load bank
testing for large data center systems:
First, they typically push load bank testing
lower on the priority list and do not consider it
a critical part of the commissioning process.
Second, they use load bank testing at only one
level of the data center.

Third, some data center owners have not set


goals for testing.
All of these mistakes have repercussions and
typically stem from a lack of understanding of
how load banks work and which different types
and configurations of load banks available today.

Agenda
Overview of Load Banks
Load Bank Testing and Reliability
Benefits and Economics of Load
Testing
Strategies For Commissioning And
Testing
Best Practices in Load Testing
Lessons Learned
4/26/2013
3

OVERVIEW OF LOAD BANK


TESTING
4/26/2013
4

Overview of Load Bank


Testing
Load testing is done at multiple
locations within a data center
Load
Testing

Load
Testing
Load
Testing

Load
Testing

Overview of Load Bank Testing


The load testing goal in critical
facilities is to balance the
electrical and thermal loads
-We have to check the electrical systems
functionality AND the HVAC systems
performance in keeping the center cool.
-Every component and all connections are
tested in advance

Overview of Load Bank Testing


Commissioning tests ensure
equipment is rated, specified Performs as system designer intended
and installed properly

Regular load testing ensures


proper operation of
equipment

Allows failure prediction to occur during the test, and not


during an unintended outage

Over time, as systems


become more complex

System test calibration ensures operation at all levels


Future standards may recommend actual load testing to
ensure performance and safety

Corrective Action Validation

Load Testing Ensures validation or verification of corrective


actions to failures in the facility
Electric power system

Improve Availability and Reliability

LOAD BANK TESTING AND


RELIABILITY
Load Banks:

Load Bank Testing and Reliability


Businesses can no longer tolerate
equipment failures as the signal
that maintenance of equipment is
necessary.
- Just-in-time (JIT) manufacturing and Six
Sigma programs dictate an effective
preventive maintenance program that
ensures continuity of operations and
prevents equipment breakdowns.
- Breakdowns affect quality and production
schedules but they also disrupt customers'
supply chains. JIT falls apart when suppliers
cannot perform

Load Bank Testing and Reliability


Reduce Risk, Improved Reliability
- Probability and frequency of failures (MTBF)

Improved Availability
- Availability is defined as the percentage of time that a
system is available to perform its function(s)

Load Bank Testing and Reliability


Survey Conducted by the IEEE
Industrial and Commercial Power
Systems Committee

Number of Failures
Maintenance
quality

All causes

Inadequate
maintenance

Failures due to
inadequate
maintenance
(%)

Excellent

311

36

11.6

Fair
Poor
Total

853
67
1469

154
22
240

18.1
32.8
16.3

Source: IEEE 493 - Recommended Practice for the Design of Reliable


Industrial and
Commercial Power Systems

Load Bank Testing and Reliability


Survey Conducted by the IEEE
Industrial and Commercial Power
Systems Committee
- Showed correlation between maintenance and
testing and failures
Percentage of failure caused by inadequate
maintenance
All electrical
equipment
Classes combined

Circuit
breakers

Motors

Open wire

Transformers

16.4%

20.8%

15.8%

30.6%

11.1%

Source: IEEE 493 - Recommended Practice for the Design


of Reliable Industrial and Commercial Power Systems

Note: 1469 failures reported

Load Bank Testing and Reliability


The larger the facility, the lower
the reliability

System
Complexity

- More parts to fail


System

MTBF

Availability

Probability of
Failure (< 5 years)

2 Gen, 2UPS, 12
Loads

189 hrs.

0.9999913

16.6%

2 Gen, 2UPS, 24
Loads

100 hrs

0.9999825

31.1%

4 Gen, 4 UPS, 12
Loads Each

96 hrs

0.9999827

30.6%

Higher
Prob.
Of
Failure

Load Bank Testing and Reliability


Key Component Failure Rates
- Drives Focus on inspection and functional Testing
Component

Failure Rate
(Failures per year)

Downtime per
Failure (Hours)

Circuit Breaker
(Metal Clad)

0.0030

7.6

Disconnect
Switches

.006100

2.8

Bus Duct

0.01890

4.0

Inverter

1.254000

185.0

Cables

0.00387

10.0

Terminations

0.004192

11.5

Transformers

.0059

297.4

BENEFITS AND
ECONOMICS OF LOAD
TESTING
4/26/2013
1

Benefits of Load Bank Testing


Data Center power system is an
investment in total system reliability
aimed at reducing business risk
Risk Types

Risk
Assessment

Risk
Treatment

Capital
Project Risk

Identification

Monitoring
and Review

Operational
Risk

Analysis

Random
Failure Risk

Evaluation

Continuous
Improvement

Need to have data to


support these steps

Source: ISO/FDIS 31000:2009(E),


pages VII and 9.

4/26/201

Benefits of Load Bank Testing


Reduce Risk, Improved Reliability
- Probability and frequency of failures (MTBF)

Improved Availability
- Availability is defined as the percentage of time that a
system is available to perform its function(s)

Benefits of Load Bank Testing


Validation of System Performance to
Specification
- Systems need to perform at rated power with underlying assumptions
- Power Factor
- Altitude and climate

Lower Maintenance Costs


- Running an engine with minimal load causes residual fuel buildup.
- Decreases the efficiency of the engine and reduces the useful life of critical parts.

- As a system evolves with updated software, firmware and replacement


hardware
- Systems need to be periodically calibrated

Reduced Business Risk


- Damage to Reputation
- Brand Impact

Benefits of Load Bank Testing


Reduced Business Risk
60% of Data Center failures could be
impacted/prevented/deferred through
effective load testing.

Benefits of Load Bank Testing


Uncover Random Failure Problems not diagnosed by sub
system

Generat
or Start

ATS

Generator
calibration
for altitude
and
temperature

Low cranking
amps

Circuit
Breaker

UPS

Commun
ications

Return to
Normal Time
Delay

Faulty Trip
due to inrush

UPS Switches
to battery
power when
Gens. Run

Time delay
and control
options need
tweaking

Faulty time
delay relay

Ground Fault
Relay sensor
imbalance

Out of
tolerance
battery
string

Upgraded
control
firmware
incompatibility

Benefits of Load Bank Testing


A well run full load test can Reduce Project Costs
- Faster setup, testing and takedown
- Reduced overall time for commissioning test (days)
- Reduced load bank setup time (hours)
- Less checking and reworking of connections by Installer
Faster set up of downstream tests with buss track adapters
and rack mounted load banks
- Less changeover when switching tests to multiple service
providers

Reduced labor from stakeholders involved in


commissioning
-

Generator, UPS Mfr.


General Contractor, Electrical Contractor
Consulting Engineer
Facilities Owner/Manager

STRATEGIES FOR
COMMISSIONING AND
TESTING
Load Banks:
Generating

22

Strategies For Commissioning


And Testing
Commissioning has become part of
the project life cycle
- Recognized as a fundamental element in most
projects along with the design and construction
elements

Commissioning should be a program


level activity
- Repeatable processes and results

Commissioning Agent should be a


Prime or Tier 1 Partner
Data Center

23

Strategies For Commissioning


And Testing
How Much Commissioning is Needed?
Uptime goals
Balance schedule, Cost and Risk/Benefit
Benchmarking of similar facilities
Reliability Modeling and MTBF Analysis
Sampling strategies vs. testing 100% of components
Business Drivers:
- Energy Savings
- M&V Plans

- LEED - Sustainable Design


- Regulatory Issues (HIPPA, JCAHO, Sarbanes Oxley)

Data Center

24

Strategies For Commissioning


And Testing
Use proven tools that have been
applied in other industries

Commissioning Agent provide many services:


- Transition and on-going support of the operations
group
- Maintenance planning
- Operations staff training

Comprehend Seasonal Challenges


- Winter Low outside humidity, Economizer
systems functional, Low Chiller loading
- Spring/Fall Conditions changing from AM to PM
- Summer Design Day, High Humidity

Strategies For Commissioning And


Testing
Use Published Reference Material as a
guide
- IEEE Gold Book - Design of Reliable Industrial and Commercial
Power Systems
- Survey data supports that effective maintenance practices
improves reliability and availability
- HSB Recommended Practices for Thorough and Complete
Maintenance
- Utility practice of system coordination studies (IEEE 1547.7)
- NFPA 70B, Recommended Practice for Electrical Equipment
Maintenance. Provides specific information on how to set up a
maintenance program.
- Scheduled maintenance tends to be more systematic and
orderly, while breakdown maintenance is often performed under
stressful conditions that could tempt workers to take dangerous
safety shortcuts.
- ANSI/NETA Standard For Maintenance Testing Specifications for
Electrical Power Equipment and Systems

LOAD TESTING BEST


PRACTICES
4/26/2013
2

Load Testing Best Practices


Load testing at commissioning
and maintenance intervals
reduce the incidence of
catastrophic failures.

Load Testing Best Practices

Circuit Breaker

Arc Flash Explosion

Facility Damage

Bus Bar

Load Testing Best Practices


Data Center Integration Testing
-THE PROBLEM:
- Data center testing occurs months before servers
are installed
- Owners need to ensure the complex systems
operate as intended.

4/26/2013

Load Testing Best Practices


Power System
Generator Individual
Generators in Parallel
Automatic Transfer Switch
UPS Modules
Power Distribution Units
(PDUs)
Remote Power Panel
(RPPs)

Overhead or under floor


distribution or Bus System
In Rack Power Strips
IR inspection at all points

Load Testing Best Practices

HVAC
Heating Ventilation Air
Conditioning
Chillers
Cooling Towers
CRAC Units Computer Room
Air Conditioners

Humidity controllers
Economizers (outside air
blending)

Load Testing Best Practices


IST

Integrated Systems Test

- Combines all HVAC and


Power Systems and
applies load to the data
center as a whole.
- As the load banks
produce electrical load,
it is discharged as heat.
The cooling system is
tested, Tuned and
Balanced.
- System failure modes
are explored and
documented.
3

Load Testing Best Practices


Critical Facility Power
Quality
-THE PROBLEM
- Power Quality Measurement requires
costly, equipment that takes a high
level of training to interpret results.

- Equipment suppliers pointing fingers


when problem occurs

34

Load Testing Best Practices


Power Quality Measurement
- Technology can provide comprehensive
results with little test time

Power Quality Test Report

Recovery Time

35

Harmonics

Load Testing Best Practices


Medium Voltage Feeds and
Emergency Generators
-THE PROBLEM
- Many data centers use Medium Voltage Utility
Feeds and employ Medium Voltage Emergency
Generators
- Load Testing with low voltage load banks results
in complicated connections, space constraints and
sloppy job site appearance.

4/26/2013

Load Testing Best Practices


CR922A 5MW MV Load Bank:
Transportable medium voltage
load bank
No Transformer
CAPACITY:
5MW @13,800 VAC
VAC

4MW @ 4,160

Easy set up
Linkable for larger loads
Reduces time & labor cost
4/26/2013

Load Testing Best Practices


Critical Facility Load Testing and
Commissioning
-THE PROBLEM

- Load testing can take several days at a


site and requires skilled labor to be
available to control load and load steps
- Often the sites are considered unsafe
while load is being energized

Load Testing Best Practices


Wi-Fi / Enhanced Communications:
IP addressable load banks
Manage all load banks via a
single mobile device
- In the rack
- In the aisle
- In the room

Customized MS Windows GUI

Reduces Test Time


Reduced labor cost
Android Smartphone Interface
Application

Lessons Learned
Load testing is a critical and beneficial
aspect of building commissioning and
maintenance.

Innovations can make testing safer


and faster
Bottom line is effective power system
commissioning and testing saves you
project costs and reduces operating
expense.

Thank You
Mark Siira

Director, Business Development


ComRent International
msiira@comrent.com
(410) 257-3000

You might also like