Professional Documents
Culture Documents
White Paper
JAN KREMER
CONSULTING SERVICES
TABLE OF CONTENTS
1. INTRODUCTION .............................................................................................................................. 4
1.1. DOCUMENT OUTLINE ........................................................................................................................ 4
2. GENERAL DESIGN PRINCIPLES ......................................................................................................... 5
2.1. INTRODUCTION ................................................................................................................................ 5
2.2. GREEN DATACENTERS ....................................................................................................................... 6
2.3. VIRTUALIZED DATA CENTERS .............................................................................................................. 7
2.4. MANAGED SERVICES ......................................................................................................................... 9
2.5. SECURITY ...................................................................................................................................... 11
2.6. ITIL BASED MANAGEMENT AND SERVICES ........................................................................................... 13
2.7. SERVICE ORIENTED ARCHITECTURE (SOA)........................................................................................... 15
2.8. BUSINESS CONTINUITY AND DISASTER RECOVERY.................................................................................. 16
3. DATA CENTER DESIGN EXAMPLE ................................................................................................... 19
3.1. INTRODUCTION .............................................................................................................................. 19
3.2. CCTV AND ACCESS CONTROL ........................................................................................................... 24
3.2.1 Introduction .......................................................................................................................... 24
3.2.2 Physical Access Restrictions details ...................................................................................... 24
3.2.3 Door Control Systems............................................................................................................ 25
3.2.4 Server Area Protection .......................................................................................................... 26
3.2.5 Closed-Circuit Television Coverage ....................................................................................... 26
3.2.6 Access Policies and Procedures ............................................................................................. 27
3.2.7 ISO 27001 .............................................................................................................................. 27
3.2.8 CCTV ...................................................................................................................................... 29
3.2.9 Access Control ....................................................................................................................... 31
3.3. CABLING ....................................................................................................................................... 32
3.3.1 Introduction .......................................................................................................................... 32
3.3.2 How to Label: ........................................................................................................................ 36
3.3.3 Verification ........................................................................................................................... 39
3.3.4 Network Cabling Infrastructure ............................................................................................ 39
3.3.5 Implementation of Pods........................................................................................................ 41
3.3.6 Top of Rack (ToR) Model ....................................................................................................... 42
3.3.7 End of Row (EoR) Model ....................................................................................................... 44
3.3.8 Point of Distribution (POD) ................................................................................................... 45
3.4. FIRE DETECTION AND SUPPRESSION .................................................................................................... 46
3.4.1 Introduction .......................................................................................................................... 46
3.4.2 Detailed Information ............................................................................................................ 48
3.5. HVAC .......................................................................................................................................... 51
3.5.1 Introduction .......................................................................................................................... 51
3.5.2 Details ................................................................................................................................... 53
3.6. LIGHTING ...................................................................................................................................... 58
3.6.1 Introduction .......................................................................................................................... 58
3.6.2 Occupancy Sensor Application .............................................................................................. 58
3.6.3 Lighting Capacity .................................................................................................................. 60
3.7. MONITORING AND MANAGEMENT .................................................................................................... 61
3.7.1 Introduction .......................................................................................................................... 61
3.7.2 Details ................................................................................................................................... 61
3.8. POWER......................................................................................................................................... 65
1. INTRODUCTION
This white paper provides an overview of Data Center Design principles and sample
Data Center Design
2.1. Introduction
Knowing what the client needs are the essentials of good data center design, and the
general infrastructure that a data center includes are the basic starting principles.
Now we need to concentrate on its exact scope. How many layers of infrastructure
should the data center include, will it be only server environment for one or many
managed services capabilities, how does the main data center purpose relate to the
disaster recovery data center
capabilities as to scope, capabilities
and service levels and what kind of
tier level is required etc.
The servers hot air returns blow heat exhaust out the back of cabinets into hot aisles.
The hot air is then sucked into the CRAC unit to be cooled and redistributed through
cold aisles.
As computing demands skyrocket, servers in data centers proliferate. And now, the
equation is rapidly spinning out of control as environmental concerns and cost-
efficiency are overwhelmed by server sprawl. excessive energy consumption from
servers running hot leads to high cooling costs, overuse of fossil fuels, pollution,
depletion of natural resources and release of harmful co2 as waste. For every kilowatt
of energy consumed by a server, roughly another kilowatt must be expended to cool
that machine. By the end of 2008, the power costs of a server have exceeded the cost
of the server itself. Reduction of the number of servers can be achieved by
implementing a Virtualized Data Center. Using less equipment to do more goes to
the heart of being LEAN & GREEN. Consolidating and virtualizing storage and
using efficient computing practices and power-saving tactics are the route to
achieving environmental efficiency and reduction of cost.
Microsoft offers server virtualization technology within their new MS Server 2008
Operating System platform. Windows Server 2008 Hyper-V is a built-in operating
system technology that hosts virtual machines on the Windows Server 2008 platform,
using server hardware virtualization. It provides a scalable and secure platform for
supporting enterprise server virtualization infrastructures. Windows Server 2008
Hyper-V uses Type 1 hypervisor-based virtualization, which runs directly on
hardware, thereby enabling direct access to difficult-to-virtualize processor calls.
Companies find it advantageous to outsource services that provide key functions such
as security, business continuity, disaster recovery, data integrity, and high
availability, so they can instead focus internal IT resources on core activities and
processes.
2.5. Security
The increasing multiplicity of data centre locations and often the geographical
dispersion of IT administrators increases the importance of a sound security strategy.
To work effectively, the strategy should establish guidelines and responsibilities to
protect the information assets of a company.
Physical security
Public: areas that all employees can access
Controlled: areas that can and must be locked when unattended
Very controlled: areas where access is restricted to registered or authorized users
The question for many IT managers is how to supplement physical security strategy.
The answer is to give secure, remote access and control of data centre servers and
devices to authorized personnel no matter where they or the devices are located.
Data Center physical security includes components such as:
CCTV System with central control room monitors and video recording
units\
Data Center Access Control System with role based access control for the
different zones and rooms within the Data Center including biometrics
fingerprint scanners (employees only)
Visitor temporary card issuance system for Data Center access for
visitors
Employee Access Card Issuance system with Digital Camera (capturing
digital photo for card surface) and Biometrics Fingerprint Scanner
(Fingerprint minutiae on card contactless chip for 1-1 verification at
access points). Additional Biometrics systems such as Iris and facial
recognition are also supported
Outside CCTV cameras for Data Center perimeter security management
The security systems can utilize the existing IP network for functionality for both
access control requests and CCTV. This reduces the cost and complexity of adding
separate physical lines. Additionally, it will allow for remote monitoring and
management from any Facility.
Logical security
Logical security strategy requires the IT manager to identify and authenticate users.
User IDs need to be established to identify the person connecting to the system.
Logical security includes defining and protecting resources. What resources can users
access when they have been authenticated?
ITIL has become the de facto best practice for running IT. The wide
spread adoption of ITIL within an industry will provide guides to what
works and what doesnt.
ITIL brings with it a common dictionary, an item that has been lacking in
the present IT world.
Improved financial management of IT and a better matching of the
services of IT to the needs of the overall organization.
Improved relationship between IT and the organization for which it
provide services.
Improved utilization of the IT infrastructure.
Improved utilization of IT personnel.
Improved reputation of IT within the organization that IT services
The recovery time objective (RTO) is the maximum allowable downtime after an
outage for recovering systems, applications, and functions (see Figure below). RTO
provides the basis for developing cost-effective recovery strategies and for
determining when and how to implement these recovery strategies during a disaster
situation
means of achieving these results. The impact of reaching these high-availability goals
will likely require greater levels of expertise, automation, and, ultimately, capital
investment.
3.1. Introduction
This section provides Data Center Design examples for the following components
This document represents the second deliverable for this project which is a Low
Level design for the main components of the Data Center such as:
General Design
o Floor Plan
o Final layout for the Communications Room and Power Distribution Room
o Labeling and Mapping
o Shielding
Power System Design
o Final Design for the Generator Sets
o Final Floor Plans for the Generator Sets room
o Final Design for the UPS systems
o Overhead power cabling since water piping is under raised floor
Cooling/AC high level design
o Basic design for using water chillers
o Models of chillers recommended
o Water piping under raised floor
Detailed Cabling Design based on TIA 942 and TIA 568-A and B
Detailed design for a Data Center Monitoring System
Detailed design for Fire Protection and Detection based on FM200
Detailed design for Water Leakage detection and monitoring whole room
Overall Design Summary
Knowing what the client needs are the essentials of good data center design, and the
general infrastructure that a datacenter includes are the basic starting principles now
we need to concentrate on its exact scope.
The design must where possible meet Tier 4 requirements based on the Tier 4
standards defined by the Uptime institute. Where physical existing building
restrictions do not allow for certain components being Tier 4 they must be Tier 3. See
a quick overview summary of Tier 3 and Tier 4 below.
All elements of the Data Center physical security deliverables must be installed and
tested including:
CCTV System within Data Center (Computer Room) with central control
room monitors and video recording units.
Datacenter Access Control System with role based access control for the
different zones and rooms within the Datacenter including biometrics
fingerprint scanners (employees only).
Visitor temporary card issuance system for Data Center access for
visitors. (Optional)
Employee Access Card issuance system with Digital Camera (capturing
digital photo for card surface) and Biometrics Fingerprint Scanner
(Fingerprint minutiae on card contactless chip for 1-1 verification at
access points).
Outside CCTV (around the inside building entrance door(s) to the
Computer Room) and cameras for Datacenter perimeter (outside
Generator Set/UPS building for security management.
The security systems will utilize the existing IP network for functionality for both
access control requests and CCTV. This reduces the cost and complexity of adding
separate physical lines. Additionally, it will allow for remote monitoring and
management.
Access control should be established using contactless smartcards which store on the
chip (suggest 16-32Kb) the information of the cardholder for access control to the
different Datacenter security levels:
Name, Phone, Position, and Company organization group
Security Access level
Biometrics including digital photo and two fingerprint minutiae
Access control doors must have a contactless smartcard reader with fingerprint
scanner. Each card reader for each location will perform the required check. When
the person holding the card requires to access the Datacenter, and any higher level
security rooms he holds his card close to the reader, the system logs:
Date and Time accessing (and leaving)
Name etc
Then validates the Fingerprint scanned from the reader against the minutiae in the
card, when OK validates the security level allowed and opens the door or rejects
access.
All secure areas including leaving the data center will force also the employee (or
visitor when given temporary pass) to use the card on a reader in the exit area in
order to open the door for leaving. This system now can also be utilized for:
Security audits
Time Management for employees for maintaining a log when employees were
present (automated time sheets)
Cages can then have their own access control with the related security level related to
the server group and functions
There are several reasons why an organization should implement ISO 27001 standard
and the primary one is the business demand. The ISO 27001 certification confirms
that certain levels of protection are in place so as to protect the information / data
handled.
Our methodology for assessing and managing information risks, as well as for the
development of information security policy and procedures will be based on
ISO27001:2005 international standard and best practices.
3.2.8 CCTV
The CCTV implementation should be based on IP CCTV solutions making use of
existing or new network cabling using the CCTV cameras as standard IP configures
network devices.
In addition Power over Ethernet could also be used to power the cameras
For the computer room the CCTV cameras should be installed as a minimum at:
Each corner of the main computer room
Monitoring the entrance of the Communications Room
Monitoring the door between the UPS room and the computer room
Monitoring the entrance door to the computer room
Monitoring the hallway to the computer room
In the middle of the computer room on each side
3.3. Cabling
3.3.1 Introduction
Basic principles of a network cabling infrastructure include:
Creating a network cabling infrastructure
Points of Distribution
Avoiding Spaghetti
Labeling and Color Coding
Verification
These individual identifiers can be combined to create an overall and accurate picture
of a cabling plant. Test reports will use a combination of these pieces to completely
identify any piece of the cabling plant, where it is connected and the pathway that it
follows. This requires that every piece of equipment should be labeled.
Fiber cable should be labeled on the outside jacket of the cable. Fiber panels should
be labeled on the outside of the box. Individual modules or ports inside a fiber panel
should be clearly labeled. Documentation should be located inside the fiber panel
that clearly identifies what fiber strands are connected to which bulkhead. Under no
circumstances should a technician need to open the installer's side of an LIU in order
to determine the identifier for a bulkhead or what fiber is attached to that bulkhead.
Order of the termination points in the label is decided alphanumerically, not based on
physical location itself.
Numeric identifiers for cables and cable strands are used solely to differentiate
themselves from other cables sharing their same characteristics. A cable should only
be identified with a 0047-1A/0193-1A, FMM2 if there is already a 0047-1A/0193-
1A, FMM1 in existence.
3.3.1.6 Examples
Fiber examples:
0047-1A/0193-1A, FMM1
Cable terminates in Building 047, Telecommunications Room 1A
Cable terminates in Building 193, Telecommunications Room 1A
This is the first multimode cable connecting these rooms in these buildings
0047-1A/0193-1A, FSM1.1
Cable terminates in Building 047, Telecommunications Room 1A
Cable terminates in Building 193, Telecommunications Room 1A
This is the first strand in the first single mode cable connecting these rooms in these
buildings
0047-1A/0193-1A, FCM1
Cable terminates in Building 047, Telecommunications Room 1A
0047-1A/0193-1A, FCM1.SM1
Cable terminates in Building 047, Telecommunications Room 1A
Cable terminates in Building 193, Telecommunications Room 1A
This is the first strand of single mode fiber in the first composite cable connecting
these rooms in these buildings
Hardware examples:
0047-1A-1FPL1
Fiber panel is located in Building 047, Telecommunications Room 1A
Fiber panel is mounted in rack number 1.
This is the first fiber panel, in the first rack, in Telco Room 1A
0047-1A-WFPL1.1/1
Fiber panel is located in Building 047, Telecommunications Room 1A
Fiber panel is mounted on the wall.
This is the first bulkhead position in the first module of this fiber panel
PCB001-WFPL1.2/4
Fiber panel is located in Pathway Cabinet #1
Fiber panel is mounted on the wall.
This is the fourth bulkhead position in the second module in this fiber panel.
2) When deciding which end of the fiber to denote first in the label, use the lower
alpha numeric characters first. For example, 0147-1A/0347-1A, FSM1 would be
proper and 0347-1A/0147-1A, FSM1 would not.
3) Individual fiber strands should be inserted into any fiber panel following the
standard color code for fiber with Blue being first and so on. This color code should
be followed so it can be read from left to right and from up to down for each
module as viewed from the front of the fiber panel. In the documentation, strand
numbers will begin at 1 and ascend in keeping with the color code. i.e. blue=1,
orange=2, green=3, and so on.
Blue-Orange-Green-Brown-Slate-White-Red-Black-Yellow-Violet-Rose-Aqua
2) A fiber panel should have a list of all fiber cables that are held in the box itself.
Often times, this will just be one fiber cable but could be much more. This list
should be preceded with an introduction of 'This FPL holds:' or the like to prevent
confusion between the fiber name and the recorded name of the fiber panel. This list
should be in the upper left hand corner of the fiber panel.
3) In the event that both ends of a particular fiber cable terminate in the same room, the
name of that cable on the front of the fiber panel should be followed by an additional
label that specifies the rack and fiber panel numbers on both ends of that cable. For
example, 0019-2A/0019-2A, FMM1 followed by WFPL6/1FPL1 would
communicate that one end of the cable terminates in a wall mounted fiber panel
labeled fpl6 and a rack mounted fiber panel labeled fpl1 in rack 1. This additional
label does not add to the cable name for record purposes but exists solely to assist
technicians in the field
Inside
1) Fibers should be installed in each module of a fiber panel from left to right and
up to down in accordance as you look at the face of the bulkheads with the standard
color code for fiber installation.
2) Each fiber termination should be labeled on the boot by a number that
corresponds to its placement in the color-code of the cable
Numbers should begin at 1 and ascend from there with duplicate numbers used for
different types of fiber strands in one cable. For example, a composite fiber cable will
have multiple strands designated with a 1 to correspond to the first MM fiber cable
and the first SM fiber cable. Numbers will not refresh for different binder groups,
only for different classifications of fiber.
3) Each bulkhead will have an independent identifier. In a fiber panel that has been
subdivided in to modules, label the modules with numbers beginning with 1 and
ascending. The individual bulkheads need not be labeled and they will be identified
with numbers that begin with 1 and will be read from left to right or up to down in
accordance with the orientation of the module. In fiber panels that have not been
subdivided, the individual bulkheads will need to be identified with a number. If the
fiber panel does not come preprinted, the installer will be responsible for labeling the
bulkheads.
4) A documentation page will be supplied inside the panel and should be marked
with which fiber strand matches up to which bulkhead. The installer may create a
simple spreadsheet similar to that pictured below. In this case, labeling should make
clear the identity of each bulkhead and the fiber strand that is connected to it. At this
time, copies of this spreadsheet should be sent to Network Services.
This is the first fiber panel mounted on the wall in Telco Room 1A in Building
#0047. Bulkhead #1 holds the first strand of the first fiber cable between Telco
Room 1A of Building #0047 and Telco Room 3A of Building #0149.
3.3.2.6 Conduit
An installed conduit should be labeled with the point of origin, point of termination
and a unique identifier to differentiate it from other conduit sharing the same
pathway. This label follows the same guidelines as discussed above. 0147-1A/0347-
1A, PCO1 would designate the first conduit running between building 147
telecommunications room 1A and building 347 telecommunications room 1A.
Labels should be affixed to both ends of the conduit. Labels are to be applied within
6 inches of the termination of each end of the conduit.
3.3.3 Verification
During implementation each and every patch panel port MUST be verified and
certified by the installer as part of that contract. Obviously cable testing equipment
and additional tools must be utilized to ensure proper cabling installations.
When deploying large volumes of servers inside the data center it is extremely
important that the design footprint is scalable.
However, access models vary between each network, and can often be extremely
complex to design. The integrated network topologies discussed in this guide take a
modular, platform-based approach in order to scale up or down as required within a
cabinet or room.
It is assumed that all compute resources incorporate resilient network, power, and
storage resources. This assumption translates to multiple LAN, SAN, and power
connections within the physical layer infrastructure. One way to simplify the design
and simultaneously incorporate a scalable layout is to divide the raised floor space
into modular, easily duplicated sub-areas.
The logical architecture is divided into three discrete layers, and the physical
infrastructure is designed and divided into manageable sub-areas called Pods. This
divides a typical data center with multiple zones and Pods distributed throughout the
room; core and aggregation layer switches are located in each zone for redundancy,
and access layer switches are located in each Pod to support the computer resources
within the Pod.
Once the logical topology has been defined, the next step is to map a physical layer
solution directly to that topology. With a ToR model it is important to understand the
number of network connections needed for each server resource. The basic rule
governing the number of ToR connections is that any server deployment requiring
more than 48 links requires an additional access layer switch in each cabinet to
support the higher link volume. For example, if thirty (30) 1 RU servers that each
require three copper and two fiber connections are deployed within a 45 RU cabinet,
an additional access layer switch is needed for each cabinet. Figure below shows the
typical rear view ToR design including cabinet connectivity requirements at
aggregation and access layers.
The typical EoR Pod contains two Cisco Nexus or Cisco Catalyst switches for
redundancy. The length of each row within the Pod is determined by the density of
the network switching equipment as well as the distance from the server to the
switch.
For example, if each server cabinet in the row utilizes 48 connections and the switch
has a capacity for 336 connections, the row would have the capacity to support up to
seven server cabinets with complete network redundancy, as long as the seven
cabinets are within the maximum cable length to the switching equipment.
One way to simplify the design and simultaneously incorporate a scalable layout is to
divide the raised floor space into modular, easily duplicated sub-areas. Figure below
illustrates the modular building blocks used in order to design scalability into the
network architecture at both OSI Layers 1 and 2. The logical architecture is divided
into three discrete layers, and the physical infrastructure is designed and divided into
manageable sub-areas called Pods.
This example shows a typical data center with two zones and 20 Pods distributed
throughout the room; core and aggregation layer switches are located in each zone
for redundancy, and access layer switches are located in each Pod to support the
computer resources within the Pod.
3.4.1 Introduction
The first line of fire defense and containment is the actual building structure. The
rooms and storage rooms of the data center must be isolated by fire resistant walls.
The floor and ceiling must be constructed of noncombustible or limited combustible
material. Also the HVAC system must be dedicated to the data center only.
The Datacenter will consist of a gaseous fire suppression system using FM200.
FM200 works by physically cooling the fire at a molecular level, and is safe for use
around operating electronic devices and in human occupied areas. Fire detection in
the Data Center will use cross zoned photo-electric and ionization spot detectors.
Additionally, High Sensitivity Smoke Detection (HSSD) will be used for the earlier
possible detection of combustion. The Fire detection system will be integrated into
the IP network. This will allow the use of existing infrastructure instead of running
dedicated lines, and allow for remote monitoring and control. The remainder of the
Datacenter will be protected to local code standards utilizing hand held fire
extinguishers as applicable.
The Chemetron Fire Systems Gamma Series Systems are automatic suppression
systems using the FM-200 chemical agent and consisting of four basic components
and their associated accessories.
FM-200 Components
Control Panels
Detection and Alarm Devices
Completer Kits
3.4.2.1 Features
The FM-200 components consist of agent containers, container supports (racks),
and discharge nozzles.
The control panel is the brains of the system and is used to monitor the detection
and accessories.
The detection, alarm devices, and accessories are the external devices that act as
the eyes and voice of the system as they give audible or visual signals.
The completer kits consist of warning signs, hoses, connection fittings, pressure
gauges or solenoid valves, and the actuator required to operate the cylinder valve.
The system and its components are agency tested for total flooding applications and
should be used in accordance with the guidelines contained in National Fire
Protection Association 2001. A total flooding application can be defined as injecting
FM-200 into an enclosure or volume having the structural integrity to retain the agent
during and after discharge.
The design of such a system requires that the FM-200 chemical agent be discharged
from its container within 10 seconds and be thoroughly mixed throughout the
protected volume, reaching a minimum concentration level of 6.25%, but not
exceeding 9% in normally occupied spaces.
After Hold time, when the fire is extinguished, the agent is exhausted from the space
by exhaust fans before any inspection is performed. For the design of the system,
NFPA Code 2001, "Standard on Clean Agent Fire Extinguishing Systems" is
followed.
FM 200 design includes determination of the agent quantity, piping layout, pressure
drop through the piping and accessories, as well as fixing the location and quantities
of discharge nozzles for uniform distribution of the agent throughout the space. This
also includes determining the filling density in the agent cylinders to take care of the
pressure drop through the system, for determining the number of cylinders.
From above, the agent quantity required for total flooding of the space is determined
independently based on the design concentration of the agent necessary for the type
of fire to be extinguished, Hold Time for extinguishing the fire, additional quantity
required to take care of the leakage, etc.
Tentative pipe sizing and pipe routing with nozzle location are done by the owner or
the engineer in harmony with the other facilities in the space. This is, however,
finalized by the agent supplier's authorized system designer based on the pressure
drop software program for two-phase flow of the agent.
To take care of the system pressure drop and to establish the required pressure at the
nozzles, the authorized agent determines the agent fill density in the cylinder. They
also finalize the number of cylinders based on the fill density and their standard
cylinder size.
The areas to be protected are identified from the fire risk analysis of the plant and the
various codes (like NFPA, etc). The requirements are guided by the functional
criticality of the system protected, amount of loss involved, fire insurance premium,
etc
A typical case of protecting a power station using the FM 200 total suppression
system is the basis for the following design information. Design Code: NFPA 2001,
"Clean Agent Fire Extinguishing System," is the governing code for designing the
system, and NFPA 72, "National Fire Alarm Code," is followed for fixing the fire
alarm system, an important part of the clean agent total suppression system.
Agent Concentration: Since FM 200 is the most expensive item of the total system, a
careful analysis is required before fixing the required concentration and the total
quantity of the agent.
Regarding design concentration of the agent, there are various guidelines available,
such as:
120% of cup burner value verified by listing/approval tests, minimum design
concentration (%V/V) of FM 200 is 7%, (refer to Table 4-7.5 Weight and Storage
Volume Equivalent data for New Technology Halocarbon Gaseous alternatives'
SFPE Handbook on Fire Protection Engineering).
The same agent concentration of 7% is accepted by Factory Mutual (FM) as the
design agent concentration.
Underwriters' Laboratories (UL), however, recommends the agent design
concentration as 7.44%.
To satisfy both FM and UL, it seems prudent to consider the design concentration as
7.44% by volume. The FM 200 supplier's authorized agent normally recommends 7%
as the design concentration, based on their experience with the type of fire
anticipated in the areas protected. Increase of the agent concentration from 7% to
7.44% has the repercussion on the cost of the agent. If possible, the recommendation
of the AHJ (Authority of Jurisdiction) should be solicited before fixing the agent
design concentration.
The maximum limit of the FM 200 concentration is restricted by NFPA 2001 due to
the safety considerations of the toxicological and physical effects on human life.
The recommended FM-200 installation will include 2 large Gas containers placed on
the right wall next to one of the main pillars and include app. 300 nozzles distributed
over the Computer Room floor space as well as the Communications Room
3.5. HVAC
3.5.1 Introduction
HVAC and other environmental controls are essential for a data center. Computer
Hardware requires a balanced and appropriate environment for continuous system
operation.
Temperatures and relative humidity levels outside of the specified operating ranges
or extreme swings in conditions can lead to unreliable components or system failures.
3.5.2 Details
We recommend a Tier 4 Chiller (Chilled Water Supply for all Rack cooling units as
well as the chilled water CRAC units (HP Superdomes, a TANDEM Base24 system
and HP SAN as well as overall room AC).
Weight requirements etc are included in the design document for the Generator Set
and UPS construction requirements.
3.5.2.5 Chillers
3.6. Lighting
3.6.1 Introduction
Use Occupancy Sensors
o Occupancy sensors can be a good option for datacenters that are
infrequently occupied. Thorough area coverage with occupancy sensors or
an override should be used to insure the lights stay on during installation
procedures when a worker may be 'hidden' behind a rack for an extended
period.
Provide Bi-Level Lighting
o Provide two levels of clearly marked, easily actuated switching so the
lighting level can be easily changed between normal, circulation space
lighting and a higher power detail work lighting level. The higher power
lighting can be normally left off but still be available for installation and
other detail tasks.
Provide Task Lighting
o Provide dedicated task lighting specifically for installation detail work to
allow for the use of lower, circulation space and halls level lighting
through the datacenter area.
We also recommend that the lighting rows are above the cold aisles so that any
heat generation is controlled by the Hot/Cold Aisle cooling implementation. Using
motion sensors with occupancy sensors will switch off lighting when not needed and
considerably reduce power usage as well as heat dissipation into the computer room.
Sample diagram
21.3m.
Main Room
530 sq m.
5700 sq ft.
25.2m.
3.7.1 Introduction
A comprehensive monitoring system is important for the design and maintenance of a
data center facility. It also provides an invaluable tool for diagnosing and correcting
problems, collecting historical data for systems evaluations and for day to day
verifications and corrections. The following data is critical for monitoring systems
designs:
Room conditioning feedback should not be based on one sensor in one part of
the Data Center room since it could provide confusing and incorrect data.
Multiple sensors are required with multiple data gathering capabilities
Historical trend capabilities must be provided
Critical alarms must be provided
Must be integrated with centralized data center tracking system, including
Building Management and Security
Must include HVAC and AC monitoring
Must include standard IP and SNMP protocols
Monitoring System Status, Health and load is a useful tool for understanding how
each system is working, by itself and in relationship with other connected systems. It
is important to understand that systems monitoring should conform to industry
standards such as SNMP. All systems including HVAC, UPS etc., should be
connected through the network complying with the SNMP basic standard. The most
critical monitoring components in the Data Center:
3.7.2 Details
The Monitoring System recommended is a combination of an integrated Data Center
Infrastructure monitoring system provided by APC/Schneider. Main Power
Monitoring is based on the PMS system provided by Schneider (See Below)
3.8. Power
3.8.1 Introduction
A well designed electrical system for the data center ensures adequate and consistent
power to the computer hardware and reduces the risk of failures at every point in the
system. The system should include dedicated electrical distribution panels and
enough redundancy to guarantee consistent uptime.
Two (2) Power Backup Generator Sets (N+1) will be designed each providing the
required backup power supply needed. Accommodations will be made to allow the
installation of a third generator when the second power input is delivered.
Backup Power Generators must be able to carry the load of the data center load.
Although it is sometimes necessary, sharing breakers is not recommended.
Maintenance Bypass
The Power Design System must provide the means for bypassing and isolating any
point of the system to allow for maintenance, repair, or modifications without
disrupting operations.
A final reason for proper grounding is noise control which is an important aspect of
power quality. Bonding is the means by which two or more grounding rods are
connected. Proper bonding techniques are critical to proper grounding. A solid and
well bonded grounding system will allow circuit breakers and power sequencers
connected to grounded outlets and have a safe path to ground if an over current
situation occurs.
Voltage Spikes
Voltage spikes are rises in the voltage caused within the power distribution units.
This is why a proper UPS system is required.
Lightning Protection
The potential damaging effects of lightning on computer systems can be direct or
indirect.
It might be on the utility power feed, directly on the equipments, or through high-
frequency electromagnetic interference or sure currents. The design includes paths
for surge entry and surge arrestors.
3.8.3 Details
Transformers
We recommend transformers of 1.5MW capacity. These transformers are normally
provided by SCECO although they, as well as the Cummins Generator Sets can be
provided by APC/Schneider. Also the UPS systems can be provided by
APC/Schneider. In summary the whole Power Solution can be provided by a single
organization (Schneider) making warranty, maintenance and support much more cost
effective.
Standby Mode
Standby Mode is the output available with varying load for the duration of the
interruption of the normal source power. Average power output is 70% of the standby
power rating. Typical operation is 200 hours per year, with maximum expected usage
of 500 hours per year. Fuel stop power in accordance with ISO3046
3.9. Racks
3.9.1 Introduction
Historically, Data Center managers didn't invest much thought in their deployment of
server racks beyond basic functionality, air flow, and the initial cost of the rack itself.
Today, the widespread deployment of high-density configurations is causing major
hot spot concerns and capacity issues. These factors, along with the high cost of
power, require a sound understanding of how your server rack deployment plan
relates to your overall efficiency strategy.
We recommend the NetShelter SX rack enclosure from APC for the datacenter. The
APC NetShelter SX is the next generation rack enclosure solution and addresses
current IT market trends for high-density server and networking applications. With a
strong focus on cooling, power distribution, cable management and environmental
monitoring, the NetShelter SX provides a reliable rack-mounting environment for
mission-critical equipment.
3.9.2 Details
3.9.2.1 Rack Cooling Unit
InfraStruXure architecture features modular cooling solutions as well as scalable
solutions for chilled water distribution. Coupling these in-row cooling units with the
IT heat load improves operational efficiency, agility, and availability for small and
large data centers including high density applications.
InfraStruXure In-Row RP Air Conditioner
Up to 70 kW Capacity in chilled water
Up to 37 kW Capacity in air-cooled DX
Proactive Controls
Variable Speed Fans
In Row Architecture
Horizontal Air Distribution
Rack Inlet Control
3.10.2 Summary
The whole computer room space will have a raised floor (except the storage and main
power distribution rooms). The raised floor will be based on the standard size tiles
which are 2 feet by 2 feet. The raised floor height recommended is 45 cm.
In summary, the Tate recommended solution for Computer Rooms and Data Centers
is the ConCore model starting at 1250 (Lbs Designed Load). We recommend the
model 1500 for the communications room and rack area and the model 2500 for the
Superdome area.
3.11. RF Shielding
3.11.1 Introduction
Shielding is a way of preventing electronic emissions that are generated from a
computer or network from being used by unauthorized users for gathering
information. It minimizes the chances of eavesdropping within a network. Shielding
can be provided by surrounding a computer room with a Faraday cage.
3.11.2 Details
We suggest a product called SM-10 Flexible Metallic Fabrics. SM-10 Metalized
Fabrics provide:
EMI/RF shielding up to 60 db
Exceptional resilience and comfortably attaches to a variety of surfaces
Superior RF leak prevention
We have marked the walls that are outside walls of the building. They are clearly the
vulnerable areas for RF and EMI. We suggest applying SM-10 to these walls to
avoid exposure of wireless RF signals to the outside world as well as blocking any
external interference sources.
3.12.1 Introduction
Like fire, flooding can be caused by either equipment failure or by natural causes.
While the design should attempt to prohibit water pipes from passing through the
data center, sometimes this cannot be avoided.
Moisture below the floor can damage wiring or equipment and cause costly
downtime. An under floor water detection system can give you an immediate
warning. One with a LCD display will show you the exact location of the leak
reducing the chance of costly damage. The location of the leak is displayed on the
control panel so you can use your time to resolve the leak rather than looking for it.
The design requires a proper water detection system design which is described in
more detail in the next section.
3.12.2 Details
The Water Leakage High level Design is based on a Distance Detection cabling
system under the raised floor. The two components required are the control unit and
the leakage detection cable which must support the 660 square meter floor space of
the Computer Room.
Following is a summary of the two main components of the water leakage detection
system. The option presented here from RLE Technologies a US based technology
supplier with distribution in the Middle East.
The LD2000 is the markets first web-accessible distance read leak detection panel.
When integrated with SeaHawk Water Leak Detection Cable (SC) and/or zone spot
detectors (SD-Z), the LD2000 detects the presence of any conductive fluid and
reports the distance to the leak. Within seconds, the distance to the leak is shown on
the LED display. The physical location of the leak can then be determined by cross
referencing the distance displayed on the LED display with a cable reference map
(FM1114) or by linking to a saved image through the HTML (webpage) interface.
The LD2000 can easily integrate into existing Building Management Systems (BMS)
and Network Management Systems (NMS) or be configured for direct alarm
notification via email. The LD2000 can accommodate a continuous run of up to 2000
feet (609m) of SC and is ideal for leak detection in areas where the SC may not be
visible. Common applications of this system include data centers (under raised
floors), clean rooms, telecommunication centers and other critical areas. The LD2000
offers a reliable leak detection solution that mitigates potential water damage, costly
business outages, and downtime.
RLE patented SeaHawk Water Leak Detection Cable (SC) is used to reliably sense
the presence of water or any conductive liquid. SC is durable, easy to clean, fast
drying, and able to resist damage from most contaminants. The cables abrasion-
resistant polymer core increases its strength and durability. The cable is constructed
from non-conductive polymers which help eliminate false alarms commonly
associated with leak detection cable. When connected to a SeaHawk single or multi-
zone control panel, SC senses the presence of water in each zone and the panel
indicates which zone is in alarm.
When connected to a SeaHawk distance read panel, SC not only determines the
presence of a fluid, but also pinpoints the exact location of the fluid along the cable
route. Each SC connection to a SeaHawk panel requires a Leader Cable Kit (LC-
KIT). SC is available in standard and custom lengths. The cables ends terminate
with mating connectors which make installation and expansion of existing leak
detection systems quick and easy. SC offers a reliable leak detection solution that
mitigates potential water damage, costly business outages, and downtime.
We anticipate a total of 300 meter detection cable for the main Data Center Room
and 60 meters for the Communications Room with the LD2000 rack mountable
detection monitoring system.
This unit only shows the possible location units with the detection protected room,
however the additional offered FM1114 hardware and software provides a detailed
map of the computer room and will identify the exact room spot location in case of a
water leakage. See next page for more details
An 11" x 14" reference map of your facility is used in conjunction with the SeaHawk
Distance Read panels - LD5100 and LD2000. Once the panel displays the distance to
a leak, that distance is cross-referenced with the map to determine the location of the
leak, conductive fluid, or problem within your facility.
Our locating system also displays the distance to the leak. Our microprocessor-based
alarm and locating module continues to monitor after a leak and alarms if any major
change occurs. It keeps a log of events and has built-in system wide diagnostic
functions. In addition to alarm relays, its interfaces include a 4 20 mA current
transmitter and an RS-232/RS-485 communications port.
3.13. Labeling
3.13.1 Introduction
We propose the standard TIA 606-A for labeling all data center elements such as:
Cabling
Patch Panels
PDU units
Racks and Contents
Cooling Units
Monitoring Panels
Etc.
3.13.2 Features
Web-enabled Solution
Log-in Security
Full ANSI/TIA/EIA 606A Compliance
Documentation Wizards
Seamless Link With LabelMark Software
Spreadsheet import tool
Multi-view and multi-task capability
Import data from testers
User and Date Stamping on all Notes
Customizable Fields
Attachment Capabilities
Track Horizontal and Backbone Cabling, Termination Hardware, Assets,
Contacts, Fire Stopping, Pathways, Cable Splices, and much more!
3.13.3 Benefits
Proper documentation allows you to quickly locate, review and correct network
issues.
Quickly export all your ID's for anything being tracked to Brady's Label Mark
label design software with little effort and time.
Minimal resources are needed to implement NetDoc. Brady experts are available
to help with expert training, support, and consulting work as needed.
All of your documentation, test results, drawings, and any other attachments you
need are at your fingertips in one secure location.
Staff members will be better able to manage all areas of your network by knowing
where equipment is located and what it is connected to, which will save time and
expenses.
3.13.4 Provides
Cable Management
Network Documentation
Asset Management
ANSI/EIA/TIA-606-A Complaint
Sample screen
TIA 606-A
Each end of a horizontal cable shall be labeled with the horizontal link identifier
within 300mm (12 in) of the end of the cable jacket and be visible on the exposed
part of the cable Jacket
Each individual telecommunications outlet & connector shall be labeled with the
horizontal link identifier. Labeling to appear on the faceplate, connector, or Mutoa
(multi-user telecommunications outlet assembly)