Uptime Institute

Tier Standard for Data Centers

Facts vs. Myths and Misconceptions
Presented by

Kenneth (Ken) Brill, Founder

Uptime Institute
kgb@uptimeinstitute.com

Agenda
Introduction to Tiers
Benefits and misconceptions
Impact of IT availability trends on required
facility Tier level
Certifications
Training
More Tier information

2011 Uptime Institute

Uptime Institute Is the ONLY Authority

for Tier Standard Certification

2011 Uptime Institute

Uptimes International Tier Standard Has

Two Parts
Topology

Tier I
Tier II
Tier III
Tier IV

Operational Sustainability (new 2010)

Gold
Silver
Bronze

Chronology
Topology introduced in 1996 with update in 2008
Operational Sustainability introduced July, 2010
4

2011 Uptime Institute

This Table Was Eliminated in 2008 to

Reduce Confusion & Misconceptions
Building Type
Staffing
Usable for critical load
Initial build-out UPS output
watts/ft (typ)
Ultimate UPS output watts/ft
(typical)
Class A uninterruptible
cooling
Support space to raised floor
ratio
Raised floor height (typical)
Floor loading lbs/ft (typical)
Utility voltage (typical)
Single points-of-failure
Annual site caused IT
downtime (actual field data)
Representative site
availability
Months to implement
Year first deployed
5

TIER I
Tenant
None

TIER II
Tenant
1 Shift

TIER III
Standalone
1+ Shifts

100%N
20-30

100%N
40-50

90%N
40-60

TIER IV
Standalone
24 by
Forever
90%N
50-80

20-30

40-50

100-150

150+

No

No

Maybe

Yes

20%

30%

80-90+%

100+%

12
85
208,480
Many +
human error
28.8 hours

18
100
208,480
Many +
human error
22.0 hours

30-36
150
12-15 kV
Some +
human error
1.6 hours

30-36
150+
12-15 kV
Fire+ EPO +
human error
0.8 hours

99.67%

99.75%

99.98%

99.99%

3
1965

3 to 6
1970

15 to 20
1985

15 to 20
1995

2011 Uptime Institute

All Drawings Have Been Eliminated

2011 Uptime Institute

Common Myths & Misconceptions

Two utility sources are required for Tier IV

Tier III has no single points-of-failure
Fault Tolerance of Tier III is higher than Tier II
Site location can preclude a topology rating
MTBF calculation determines Tier
Tier is determined by equipment count
Tier III+, Tier III.6, etc., have meaning
Tier IV eliminates human error

2011 Uptime Institute

Common Myths & Misconceptions

Two utility sources are required for Tier IV

Tier III has no single points-of-failure
Fault Tolerance of Tier III is higher than Tier II
Site location can preclude a Tier rating
MTBF calculation determines Tier
Tier is determined by equipment count
Tier III+, Tier 111.6, etc. have meaning
Tier IV eliminates human error

2011 Uptime Institute

Data Center Site Infrastructure Tier

Standard: Topology

2011 Uptime Institute

Tier Topology Certifications

Tier IV Fault Tolerance
Constructed facility

5 Complete
0 In-progress
Design documents
14 Complete
3 In-progress

Tier III Concurrent Maintenance

Constructed facility

10 Complete
8 In-progress
Design documents
33 Complete
23 In-progress
As of March 1, 2011

10

2011 Uptime Institute

Data Center Site Infrastructure Tier

Standard: Operational Sustainability

11

2011 Uptime Institute

Site Certification:
Topology + Operational Sustainability
Tier III Gold Certifications awarded
UnitedHealth Care, Next Technology Center
Target Corporation, Target Technology Center
Brooklyn Park, MN
Target Corporation, Target Technology Center
Elk River, MN
Fujitsu Services, London North Data Centre
London, UK

Others are in progress

12

2011 Uptime Institute

Uptimes Tier Standard System

Provides an unbiased review of a single site or
a comparison of two sites or a portfolio of sites
Assists owners in maximizing the life-cycle value of
their investment
Assists purchasers of third-party services in
quantifying what they are getting

Encourages operations to maximize uptime,

minimize risks, and achieve efficiency
Incents and measures processes optimizing energy
efficiency
Quantifies site infrastructure management
effectiveness
13

2011 Uptime Institute

Tier Standard Does NOT Compete With

or Replace Other Standards & Codes
Tier Standard exclusions:
Construction and building codes
Environmental regulations
Safety policies and practices
Security (physical and data)
Personnel policies
Energy efficiency
International Standards Organization (ISO )

14

2011 Uptime Institute

Tier Standard Was Developed By and For

Owners with Skin in the Game

15

BMO Financial Group

Boeing
Delta Airlines
Fidelity Investments
Fujitsu Services
GE
HSBC
Johnson and Johnson
JP Morgan Chase
Kaiser Permanente
Lowes

Morgan Stanley
Nationwide Mutual
Insurance
Progressive Insurance
Saudi ARAMCO
Social Security
Administration
State Farm Insurance
Target
United Health
US Bank
VISA

2011 Uptime Institute

Tier Standard Topology Definitions

Tier I
Tier II
Tier III

Basic Capacity
Redundant Components
Concurrent Maintenance - applies to
Each and Every component and path on
a planned basis*
Tier IV Fault Tolerance - anticipates a single,
mechanical or electrical system or
equipment event and subsequent
consequential impact(s)*
* Excludes EPO (North America only), fire, single-corded
IT equipment, incorrectly installed dual-corded IT
equipment, sabotage, rules violations, and/or mistakes.
See Standard for additional details and exclusions.
16

2011 Uptime Institute

Tier Standard Topology Ratings Are

Vendor and Solution Agnostic
Encourages innovative engineering
Allows for client equipment preferences
Not a proscriptive checklist, design menu, or a
recipe

17

2011 Uptime Institute

Availability Trends - Today and Future

Do you expect prime time and/or availability
requirements to go up or down in the future?
Is your compensation tied to an Availability
Service Level Agreement?
Is planned and unplanned Facilities downtime
included or excluded from your SLA?
Do you have a financially enforceable SLA with your
Facility provider (internal or external) that reflects
ITs commitments to users?

18

2011 Uptime Institute

Tier I & II Sites Require IT Downtime

Tier I and II sites require total shutdown of all IT
hardware during facility maintenance, i.e., all IT
hardware dependent upon
UPS power
Critical power distribution
Cooling backbones

Some capacity units (Tier II) MAY be

maintained without impacting IT (but often with
considerable risk of unplanned downtime)

19

2011 Uptime Institute

Single Power Path

Utility
Generator

Battery
THREE POWER PATHS
ONE POWER PATH

COMPUTER
HARDWARE

20

SINGLE POINTSOFFAILURE
UPS system level failure
Major circuit breakers (2-20)
Minor circuit breakers (20-500)
Plugs and receptacles (21-505)
Electrical connections (258-6180)
Human error
False EPO

2011 Uptime Institute

Dual Power Path

Utility
Generator
Battery

Utility
Generator
Battery

THREE POWER PATHS

TWO POWER PATHS

COMPUTER
HARDWARE
1
2
3
4
5

21

SINGLE POINTSOFFAILURE
Go to UPS failure
Human error
False EPO
A/B circuiting errors

2011 Uptime Institute

Planned or Unplanned Site Downtime

Can Trigger a Corporate Heart Attack!
10,000s

Customers/Users

100s

Network Nodes

100s

Applications

10s to 100s

Operating System

10s to 100s

Hardware

22

Site

2011 Uptime Institute

Fault Tolerance Is Justified By

Eliminating Unplanned User Downtime

23

Site Restoration

to

1hr

Hardware Restore

to

1hr

Software Restore

to

2hr

Telecom Restore

to

hr

TOTAL OUTAGE

to

4hr

2011 Uptime Institute

Concurrent Maintainability Is Justified by

Eliminating Planned User Downtime
Tier I sites require 2 total shut downs per year
2 x 14 hours = 28 hours of IT downtime per year
Plus the impact of any unplanned failures

Tier II sites require 3 total shut downs every two

years
3 x 14 hours/2 = 21 hours of IT downtime per year
Plus the impact of any unplanned failures

Tier III and Tier IV sites do NOT require

planned shut downs*

*Be aware that many self proclaimed Tier III

Concurrently Maintainable) and Tier IV (Fault Tolerant)
sites are not!

24

2011 Uptime Institute

Tier Standard: Topology

Requirements Are Deceptively Simple

Active capacity components

Distribution paths
Compartmentalization
Concurrent Maintainability
Fault Tolerance

25

2011 Uptime Institute

Topology Requirements by Tier Level

Active Capacity
Components
to Support IT Load
Distribution
Paths
Compartmentalization
Concurrent Maintainability
Fault Tolerance (1 event)
Continuous Cooling,
Continuous Duty Engines
Ambient Temperature
26

Tier I Tier II
N
N+1
1

No
No
No

No
No
No

2011 Uptime Institute

Tier III
N+1
1 active,
1 inactive
No
Yes
No
Load
Yes

Tier IV
N after
any
failure
2 active
Yes
Yes
Yes
Yes
Yes
Worst

Accredited Tier Designer (ATD)

Intense, three-day curriculum
with concluding test
Exclusively for professional or
chartered engineers responsible
for infrastructure center design
Focuses on practical application
of Tier topology
Examples drawn from real-life
designs submitted for Design
Certification
Addresses, in technical detail,
the misconceptions that lead to
needless cost and complexity
27

2011 Uptime Institute

A Worldwide Growing Community of

ATD Professionals
Australia
Belgium
Brazil
Canada
China
Costa Rica
Czech Republic
Egypt
England
France
Greece
India
Hong Kong
Kenya
Luxembourg
Mexico
Malaysia

2
1
5
3
2
7
1
1
6
1
2
1
1
1
1
3
1

Netherlands
Russia
Saudi Arabia
Slovakia
Spain
South Africa
Taiwan
Thailand
Turkey
United Emirates
United States
Venezuela

As of September 31, 2010

28

2011 Uptime Institute

1
15
2
1
3
1
6
1
1
3
30
1

Site Certification Process:

Tier Certified Design Documents
Uptime, in-office, Principallevel review of 30%, 60%,
85%, 100% design document
package
Topology deficiencies and
potential design
enhancements
Conference call with owner
and design team to discuss
deficiencies report
Compliance review of revised
drawings
Award letter and foil
29

2011 Uptime Institute

Site Certification Process:

Tier Certified Constructed Facility
On-site inspection by Uptime
team during commissioning
Identify discrepancies between
certified design drawings and
as-built construction
Observe delivered functionality
Topology deficiencies and
other enhancements
Conference call with owner
team
Award letter, foil, and plaque
30

2011 Uptime Institute

Tier Standard:
Operational Sustainability (New 2010)
Operational Sustainability is defined as the
behaviors and risks beyond design Topology
that impact the ability of a data center to meet
business objectives over the long term.

31

2011 Uptime Institute

Abnormal Incident Reports (AIRs) By:

Root Cause
2009
Failures

(Previous
Years)

2009
(Previous
Incidents
Years)

Manufacturer

19%

(25%)

211

52%

(43%)

Design

4%

(3%)

20

5%

(5%)

External Environment

0%

(14%)

10

2%

(4%)

Installation

15%

(7%)

25

6%

(7%)

Operations

16

62%

(51%)

144

35%

(41%)

100% (100%)

410

Total Root Cause

32

26

2011 Uptime Institute

100% (100%)

Gold Operational Sustainability

Behaviors Prevent 70% of AIRsFailures
Failures are never the result of one factor
Between 5 and 10 things (typically 7)
contribute to a failure
Management is the leading cause (4 out of 7)

A failure will consistently occur every time the

7 things line up
Swiss cheese or dominos analogy

33

2011 Uptime Institute

Operational Sustainability: Categories

Management & operations (immediate ability to improve)

Staffing and organization

Maintenance
Training
Planning, coordination, and management

Building characteristics (limited ability to improve)

Features
Infrastructure
Operating conditions
Pre-operational

Site location (limited ability to eliminate or even mitigate)

Natural disasters
Man-made disasters
Utility robustness and redundancy
34

2011 Uptime Institute

Accredited Tier Specialist (ATS)

(New 2010)
Intense three-day curriculum with concluding
test
For data center professionals (i.e., for nonengineers)
Includes both Topology and Sustainability
concepts
ATS accredited individuals now in 12
countries
As of March 1, 2011

35

2011 Uptime Institute

Conclusions
Tier Standards for Data Centers
Topology + Operational Sustainability
For owner/operators and buyers of hosting services
Together, assure life-cycle business value
Downtime frequency, duration, and recoverability
CapEx and OpEx effectiveness
Life-cycle (resale) value

Operational Sustainability practices often have

greater impact on actual IT availability than
Topology

36

2011 Uptime Institute

More Information
Tier Standard: Topology at
http://uptimeinstitute.org/content/view/302/281/#TierClassification
Tier Standard: Operational Sustainability at
http://uptimeinstitute.org/content/view/302/281/#Operation
al-Sustainability
Current world-wide listing of ATD engineers at
http://atd.uptimeinstitute.com/award_list.htm
Certified sites or certifications in process at
http://professionalservices.uptimeinstitute.com/tiercert.htm
Upcoming ATD and ATS seminars at
http://atd.uptimeinstitute.com/schedule.htm
Tier Myths and Misconceptions at
http://professionalservices.uptimeinstitute.com/myths.htm
37

2011 Uptime Institute

Questions

38

2011 Uptime Institute