Reliable Data Centres: Guide

Reliable Data Centres
Guide Version 2
Impressum
Publishedby: BITKOM BundesverbandInformationswirtschaft, TelekommunikationundneueMediene.V. Albrechtstrae10A 10117Berlin-Mitte Tel.:+4930.27576-0 Fax:+49.30.27576-400 bitkom@bitkom.org www.bitkom.org Contact: HolgerSkurk Tel.:+49.30.27576-250 h.skurk@bitkom.org Responsible BITKOM-committee: Editorialteam: Assistanteditor: Design/Layout: Copyright: Titelbild: HolgerSkurk BilianaSchnberg DesignBureaukokliko/AnnaMller-Rosenberger(BITKOM) BITKOM2010 AlejandroMendoza,istockphoto.com AKDataCentre&IT-Infrastructure
Thecontentsofthisguidehavebeencarefullyresearched,andreflectBITKOMsunderstandingatthetime ofpublication.Wedonotmakeanyclaimsregardingthecompletenessofthispublication,however.Despite takingthegreatestpossiblecare,wedonotassumeanyliabilityforthecontents. Thelatestversionoftheguidecanbeobtainedfreeofchargefromwww.bitkom.org/publikationen.BITKOM reservesallrights,includingthosegoverningthereproductionofpartsofthisdocument.

Guide Version 2
Content
1 Introduction 2 Theavailabilityofadatacentre 3 Theinfluenceofsafetystandardsonthedesignofdatacentres 3.1 ISO27001/ISO27002:2008 3.2 ITIL 3.3 Sarbanes-OxleyActandSAS70 3.4 Assessmentofstandards 4 ThebasisofITinfrastructure:therack 4.1 Secureservercabinets 4.2 Networktechnology 4.3 Reliabledatacentre 5 Energy,airconditioningandcooling 5.1 Powersupplycompanies(PSCs)electricityfeed-inanddistributioninthecompany 5.2 Powerdistributioninthecompany 5.3 Uninterruptiblepowersupply(UPS) 5.4 Backuppower 5.5 Service/maintenance 5.6 Airconditioning 6 Firesafety 6.1 Technicalfireprotection 6.2 Structuralfiresafetymeasures 7 Designofpremisesandsafetyzonesfordatacentres 8 Wiring 8.1 Currentsituation 8.2 Basicstandards 8.4 Structure 8.5 Redundancyandsecurity 8.6 Installation 8.7 Documentationandlabelling 9 Certificationofareliabledatacentre 9.1 Themanagementsystem 9.2 Certificationofamanagementsystem 10 Annex 11 Glossary 12 Acknowledgements 3 4 7 7 8 8 8 9 9 10 11 12 12 14 16 22 27 28 34 34 38 41 43 43 43 4 4 45 4 6 4 6 47 47 47 50 53 54
1 Introduction
Planning,developingandoperatingITinfrastructuresfor companiesvitalapplicationsinreliabledatacentresisa realchallenge.NotonlydoyouhavetoselecttherightIT equipment,thedesignofthedatacentreandtheresultingrequirementsforthesizeandtypeofconstruction, theelectricalpower,heatdissipation,wiring,safetyand availabilitynottomentionpurchaseandoperatingcosts allthesearedecisivefactors. Thisguideoffersassistancewithplanningandimplementingadatacentre.Itsupplementsexistingstandardsand specifications,whichcanalsobereferredtoforsupport. Theseareoftenextremelygeneral,however,whereasthis guideismorein-depthandprovidesconcreteinstructions onhowtodesignadatacentre.Italsosupplementsthe PlanningGuideforaReliableDataCentrematrixwhich, liketheguide,isavailabletodownloadfreeofchargeon theBITKOMwebsite.Thecontentsofthematrixarecontainedinextractsinthesectionsoftheguide.Theguide andtheplanningaidarenosubstituteforexpertadvice andsupportfromexperiencedadvisorsandspecialist planners,however.1
ThisguidewillbecontinuallyrevisedbytheBITKOMReliableDataCentre&ITInfrastructureworkinggroup.Aspartofthisprocess,itwillbeadaptedin linewiththelatesttechnicaldevelopmentsandotherboundaryconditions,andextendedtoincludetopicssuchastherequirementsoftheinsurance industry,riskmanagement,processesandservices,forexample.
2 The availability of a data centre

Therapidpaceofdevelopmentandtheintegrationof informationtechnologyinallareasofbusinessmeans thattoday,nocompanycanaffordthistechnologytofail. Justafewyearsago,manycompaniescouldsurvivethe failureoftheirITinfrastructure,evenforseveralhours. Today,thenumberofthoseforwhomcontinuousavailabilityoftheirITisindispensableisgrowingdramatically. AccordingtoastudybytheMetaGroup,a10-dayfailure ofkeyITsystemscanpermanentlydamageacompanyto suchanextentthatthereisa50%probabilitythatitwill disappearfromthemarketwithinthenextthreetofive years. AvailabilitytiersUSUptimeInstitute AsthedemandforavailabilityoftheITinfrastructure grows,therequirementsbecomeincreasinglystringent notjustfortheITsystemsthemselvesbut,aboveall, forcontinuallyensuringtheambientconditionsandthe powersupply.Asaresult,redundancyintheairconditioningandelectricitysupply,dualfeed-ins,andinterruption-freemaintenanceofsystemshaveestablishedthemselvesasthenormforhigh-availabilityITinfrastructures. Butbeforeyoustartworkontheplanningandlayoutof technicalcomponentsforachievingtheavailabilityyou desire,additionalreflectionbasedonriskanalysisand thechoiceofsiteisimperative.Thisshouldcoverpossible risksassociatedwiththearea,whichcouldinfluencethe probabilityofpotentialfailuregeographically(airtraffic, flooding,etc.),politically(war,conflict,terrorism,etc.)orin theformofneighbouringsites(premisesatparticularrisk offiresuchasservicestations,chemicalstoragefacilities, etc.).Furthermore,potentialwilfuldamagebyonesown employeesandfromoutsidethecompanymustalsobe takenintooverallconsideration. However,thedemandforhighavailabilitydoesnotonly entailgettingtogripswithpossibletechnicalsolutions, butrequirestheownertodesignandsetupacomprehensiveorganisationalstructure.Thisincludestheprovisionoftrainedservicepersonnel,forexample,ofspare partsoraserviceagreement.Preciseinstructionsonthe procedureintheeventoffailureoremergencymustalso bedefined.Inaddition,thisstructuremustenablerapid, preciseandtargetedcommunication,withtraceable documentationofevents. WhatmaximumITdowntimescanthecompany tolerate? Thesedays,whencreatingandexpandinganITconcept, orexamininganexistingone,itisvitaltofindouthow importanttheavailabilityofacompanysITinfrastructure is.Consequently,thequestiontobeaskedisthis:
Tier IntroExplanation classification duction

TierI 1960s Singlepowersupply channel,single air-conditioning supply,noredundant components99.671% availability Singlepowersupply, singleair-conditioning supply,redundant components99.741% availability Severalpowersupply channelsavailablebut onlyoneactive,redundantcomponents, maintenancepossible withoutinterruption 99.982%availability Severalactiveelectricityandchilledwater supplychannels, redundantcomponents,error-tolerant 99.995%availability
TierII
1970s
TierIII
Endof 1980s
TierIV
1994
Source:USUptimeInstitute:IndustryStandardsTierClassification
Thetermavailabilityreferstotheprobabilitythatasystemcanactuallybeusedasplannedatagivenmoment. Availabilityisthereforeameasurethatcanberecorded anddeterminedquantitatively.Ontheotherhand,there existqualitativeavailabilityclasses,asshowninthetable below,AvailabilityclassesaccordingtotheHV-Kompendium(HighAvailabilityCompendium)oftheBSI.Here, theavailabilityclassofaserviceisameasureofitsquality intermsofavailabilitybasedonhoursperyear. Asystemisregardedasavailableifitiscapableof fulfillingthetasksforwhichitisintended.Availabilityis measuredastheratiooffailure-induceddowntimetothe overalltimeofasystem.
Availability = (in percent)
1-
Downtime Uptime+ Downtime
100
Ifwecalculateavailabilityovertheperiodofayear,using theaboveformula,anavailabilityof99.99%,forexample, wouldcorrespondtoadowntimeof52.6minutes. 99%*87.66hoursperyear 99.9%*8.76hoursperyear 99.99%*52.6minutesperyear 99.999%*5.26minutesperyear 99.9999%*0.5265minutesperyear TheGermanFederalOfficeforInformationSecurity(BSI) hasdefinedthefollowingavailabilityclasses:
Availability class
AC0~95%
Norequirements approx.23weeks foravailability Normal availability Highavailability Lessthan90hrs. Lessthan9hrs.
Effects Description Cumulated, probable downtime per year
AC199,0% AC299.9%
AC399.99%
Veryhigh availability
Lessthan1hr.
AC499.999%
Maximum availability
Approx.5min.
AC5100%
Disaster-tolerant
Nomeasuresarenecessaryasregardsavailability. ImplementingtheITGrundschutz(formerlyknownas theITBaselineProtectionManual)fortheotherbasic valueswillhaveabeneficialeffectonavailability. Availabilityrequirementsaresatisfiedbythesimple applicationoftheITGrundschutz(BSI100-1andBSI 100-2) ThesimpleapplicationoftheITGrundschutzhasto besupplementedbytheimplementationofmodules recommendedforhighavailabilityrequirements,e.g. modulesB1.3ContingencyPlanningConceptandB1.8 HandlingSecurityIncidents,andariskanalysisonthe basisoftheITGrundschutz(BSI100-3). Implementationofthemeasuresrecommendedfor selectedobjectsinaccordancewiththeITGrundschutz,withparticularemphasisonbasicavailability, e.g.measureM1.28UPSintheserverroomorM1.56 SecondaryPowerSupplyinthedatacentre,supplementedbyHA(highavailability)measuresfromthe HACompendium ITGrundschutzwithadditionalmodellingonthebasis oftheHACompendium. TheITGrundschutzasthebasisisincreasinglysupplementedandreplacedbyHAmeasures. ModellingaccordingtotheHACompendium. TheITGrundschutzcontinuestoserveasabasisfor theaboveareasandothersafety/securityvaluessuch asintegrityandconfidentiality.
Table1:BSIavailabilityclasses
TheFederalOfficeforInformationSecurity(BSI)hasdevelopedaratingsystemfordatacentres:VAIR(VerfgbarkeitsanalysederInfrastrukturinRechenzentrenAvailabilityAnalysisofDataCentreInfrastructure).Atwww. vair-check.de,datacentreoperatorscanenterthedetails oftheirdatacentreinfrastructureandcheckthereliability oftheirdatacentreanonymouslyandfreeofcharge.
3 The influence of safety standards on the design of data centres

Theplanninganddesignofdatacentresisgovernedbya largenumberofsafetystandards.Theyprovideplanners anddesignerswithassistanceontheonehand,butalso setoutvariousrequirements. Here,wewillpresentthemostimportantstandardsin thefieldofISMS(InformationSecurityManagement Systems)andITIL(ITInfrastructureLibrary),aswellasthe Sarbanes-OxleyAct. ThePLANphaseentailstheplanningoftheISMS.Above all,theareaofapplicationandboundariesoftheISMS aredefinedhere,andthenapprovedbythemanagement. Ariskanalysisisalsocarriedout.Thisascertainswhich systemsandapplicationsareimportantformaintaining acompanysbusinessoperations,andhowgreatdependencyonthemis.Basedontheresultsofthisanalysis, TheISO/IEC27001standardsthatcameintoforcein October2005covertheprotectionofconfidentialbusinessinformation.Itisbecomingincreasinglyimportant, asitprovidesafoundationonwhichcompaniesarein apositiontosatisfytherequirementsofthirdparties. Thesecanbelegalrequirements(suchasKonTraG,HGB andGoB,GoBS,GDPdU,BDSG,TMG,TKGandStGB), contractualrequirements(e.g.fromcustomers)orother requirements.ThisstandardreplacesthepreviousBritish standardBS7799-2,whichwasannulledwithoutreplacementinFebruary2006. Thespecialistbusinesstermusediscompliance,and denotesbothconformitytolawsandguidelinesand voluntarycodesofpracticebybusiness. TheISO/IEC27001assiststhecreationofaprocessfor buildingandoperatinganInformationSecurityManagementSystem.Thisprocessofcontinuousimprovement takesplaceinfourknownsteps:Plan,Do,Check,Act, withwhichyouwillbefamiliarfromISO9001(Quality Management). Inthefourthphase(ACT),measuresdefinedasimproveMajorassistanceisalsoprovidedbytheGrundschutz manuals(catalogues),whichhavebeendevelopedbythe mentsareputintopractice. TheDOphaseincorporatesconcretemeasurestominimiseanddetectriskswiththeaidofariskhandlingplan. TheISO27002:2008(formerly17799)isaCodeofPractice forInformationSecurityManagementand,assuch, providesvaluabletipsforcomplyingwiththeControls/ MeasurescontainedinISO27001.Itismoreorlessa guideforputtingtheISO27001intopractice.Section9 PhysicalandEnvironmentalProtectionalsosetsout measuresandsuggestionsforroomsandinfrastructures. CertificationisonlygrantedonthebasisofISO27001or BSIISO27001,basedontheITGrundschutz. DuringtheCHECKphase,regularmonitoringandperiodic auditsensurethatimplementedmeasuresarecheckedon aregularbasis,inorderthatpotentialimprovementscan beflaggedup(e.g.firesafetymonitoringmechanisms, firesafetytests). conclusionsaredrawnabouttherequiredlevelofprotection,andthedesiredavailabilityofsystemsandapplicationsisdetermined. BSI(FederalOfficeforInformationSecurity)overmany years,andconformstoISO27001,basedonITGrundschutz.Themodulesinthecataloguesareextremely valuableintheimplementationofanInformationSecurityManagementSystem.
3.1 ISO27001/ISO27002:2008
3.2 ITIL
ITServiceManagementisanimportantconsideration duringtheplanningandoperationofareliabledata centre.BestpracticerecommendationsforITservice managementhavebeeninexistencesincetheendofthe 1980s,whentheBritishgovernmentsCentralComputer andTelecommunicationsAgency(CCTA,nowpartofthe OfficeofGovernmentCommerceOGC),publishedthe firstelementsoftheITInfrastructureLibrary(ITIL).These guidelines,whichhavebeensetoutinwriting,rangefrom detailedadvicetoindividualprocesseswithintheITIL, fromrulesofproceduretothenewlypublishedstandard ISO20000(formerlyBS15000). Forexistingdatacentres,customersalsouseaservice managementsysteminaccordancewithITILasorientation.Computerservicecentresareoftenconfrontedby invitationstotenderthatrequiretheparticipatingcompaniestohaveITIL.Twokeyareasarealwaysincluded: Service-Support Service-Delivery ThissetofrulesappliestoallITorganisationsinallcompaniesnomatterwhattheirsize.
incorrectfinancialstatements.Companiesaremonitored byapprovedauditorsinaccordancewiththeSAS70 audit.ThisinturnislargelybasedontheCobit4.1rules fromtheISACA(USA). IfacompanythatmustcomplywithSOXhasoutsourced individualsystemsoritsentireIT,forexample,theSAS70 auditmustalsobeappliedtotheproviderinquestion, althoughtheresponsibilityalwaysremainswiththe contractor.Inthiscase,eitherthecustomersauditorscan carryoutcheckstoSAS70inthecomputerservicecentre, orthecentreitselfcanarrangetohavecontrolscarried out.Theauditorsreportmustnotexceedthetimeof thecustomersannualfinancialstatementbymorethan sixmonths.Forthisreason,SOXcontrolsmustbasically beconductedtwiceayear,whichentailsconsiderable expenditure. Atinternationallevel,possibleconflictsbetweenthe Sarbanes-OxleyActandnationalregulationshavebeen discussed.Anysolutiontotheseconflictsisstillfarfrom clear.AEuroSOXisinprogress,however.Inaddition,the IDW(InstitutderWirtschaftsprferGermanInstitute ofPublicAuditors)iscurrentlybasingitsinstructionsfor compliancewithtestrequirementsonCobit4.1.
3.3 Sarbanes-OxleyActandSAS70
TheSarbanes-OxleyAct(SOX)isaUSlawintendedto improvecompanyreporting,andwasissuedinresponse tothebalancesheetscandalsinvolvingcompaniessuch asEnronandWorldcom.Thislaw,whichhasbeeninforce since30July2002,doesnotonlyapplytofinancialdata,it alsodemandssecurityinIT. Firstly,thelawapplestoallcompanieslistedonthe Americanstockexchange.Thenitalsoappliestonon-US companiesthathaveaparentcompanyorsubsidiary listedontheAmericanstockexchange. TheSarbanes-OxleyActstipulatesthatcompanyprocessesmustbedescribedanddefinedandmonitoringproceduresestablished,withtheaimofminimisingtheriskof
3.4 Assessmentofstandards
Thestandardsdescribedabovearefrequentlychecked bycustomers,certificationsocieties,auditorsandother institutions.WhethertheSarbanes-OxleyActandSAS 70makeadatacentremorereliablemaybedebatable however,thegeneralrequirementscontainedinISO/IEC 27002:2008andISO/IEC27001:2005governingmeasures toimprovesecurityarethoroughlyjustifiedandsensible. ITILandISO20000demonstrablysafeguardandimprove processesinadatacentre.Publicsectorcontractorsoften requirecertificationtoBSIhere,however,theexpenditurefordocumentingandoperatingtheISMSisvery high.ThebestcombinationisISO27001withreferenceto theITGrundschutz(whereapplicable),notcertificationby theBSIinBonn.
4 The basis of IT infrastructure: the rack

Whetheryouhaveaseparatedatacentreoranindividual servercabinet,thebasisforsecureaccommodationof ITsystemsisalwaystherack.Sinceinmostcompanies ITsystemsconsistofstandardised19-inchcomponents, scalable,flexibleracksystemsinthisdesignofferthebest choicewhenputtingtogetherastableandresilientIT infrastructure.Itensuresperfectinteractionbetweensystemandsupportcomponents,suchasthepowersupply andairconditioning.Acompanysdecisionastowhether tohouseitsITsystemsinadedicateddatacentreorto haveastandalonesolutioninindividualservercabinets dependsontheITandthestructuralrequirements.In bothcases,however,thesamefiresafetyandothersecuritystandardsapply,fortheyareintendedtoprotectICT systemsanddatafromtheinside. Anotherimportantaspectistheroutingofcables.As increasinglyfastnetworksareusedincombinationwith copperwiring,itisessentialthatpoweranddatacables areroutedseparately,topreventthemfromcausingone anotherinterference.Whenselectingarack,itisalso Asecureservercabinetshould,ifpossible,bemodular indesign.Itprovidesthecompanywithreasonable securityatamanageablecost.Amodularcabinetcanbe dismantled,convertedormovedtoanotherlocationif necessary.Thisflexibilityalsooffersadvantagesinterms oftransportandre-erectionifacompanyrelocates. Modularityisalsoadvantageouswhenwishingtoexpand housingsolutionsorair-conditioningconcepts. Whenplanningasecureservercabinetaswithareliable datacentrethefollowingcharacteristicsarenecessary forensuringthatsystemshavecomprehensivesecurity andavailability: Ensuringconstanttemperatureandairhumidity throughprecisionairconditioning Safeguardingtheelectricitysupplybymeansofan uninterruptiblepowersupply(UPS)andanadditional backuppowersupplyifnecessary Protectionagainstaccessbythirdparties(locking systems,network-monitoredrackaccess,biometric dataacquisition) Asservercapacityandthepackingdensityintherack increase,theimportanceofventilationconceptssuchas perforateddoorsandpartitionsbetweenhotandcold areasoftherackhasgrownenormously.Otherenergyoptimised,performance-enhancingsolutionscanbe achievedbyhotorcoldhousingconceptsthatformpart oftherack.Inthecaseofextremepowerlossesinthe rack,liquid-cooledsolutionsintheformofair/waterheat exchangersareindispensable. vitaltoensurethatthepowerdistributionsystemcanbe easilyintegrated,forintheendareliablepowersupply isaprerequisiteforavailableIT.Fuse-protectedlowvoltagesub-distributionmustalsobeavailable,aswell asaflexiblepowerdistributionsystemintherack,which canbeconnectedbothtothemainspowersupplyand anuninterruptiblepowersupply(UPS).Modernsolutions haveupto88kWinarack.Thisismadepossiblebyfour independent,three-phaseincomingfeeders,whichguaranteeareliablepowersupplyeveninthefaceofmore exactingrequirements. Thesubjectofstabilityisavitalaspectinallracksolutions.Duetothehighpackingdensityofmodernserver systemsandstoragesolutions,aswellasthepower supplyunitstheycontain,serverrackswithaload-bearing capacityofupto1,000kgmayberequired,depending ontheapplication.Consequently,thebasesofunitsand slidingrailsmustalsobedesignedforhighloads.Bases andrailscancurrentlywithstandaweightofupto150kg each. Firepreventionandreaction Modulescanbeintegratedinacentralmonitoring andmanagementsystem
4.1 Secureservercabinets
Bothelements,safeguardingthepowersupplyandtheair conditioning,canbemonitoredbysensorsintegratedin theinfrastructure.Whetherwiredorwireless,thesesensorsrecordtheservershumidityandtemperature,and alsotheirpowerinput.Amodern,sensor-basedmonitoringsystemmayalsotakeonthetaskofaccesscontrol.
4.2 Networktechnology
Acompleteexaminationofdatacentres,withspecial considerationofsecurityaspects,cannotfailtoinclude thesubjectofnetworktechnology.Manycompanieshave alreadyswitchedtheirtelephonesystemstoVoiceoverIP (VoIP).Virtualisedclientsarethenextstep.Thisenables anincreasingnumberofbasicservicescriticaltobusiness totakeplaceviadatalineswhich,withPoweroverEthernet(PoE),alsosupplypowertotheterminaldevices.As networktechnologybecomesincreasinglyimportantfor problem-freebusinessoperations,securityrequirements aretighteningupinthisarea,too. Aswiththeservers,fornetworktechnologytherack providesthebasichousing.Sincetheseactivecomponentsarealsostandardisedto19inches,networkcabinets aregenerallybasedonthesameplatform.Requirements arealsocomparableintermsofstability,firesafetyand accesscontrol.However,sincethenetworkinfrastructure installedinthebuildingtendstobedesignedtolastmore than10years,long-termplanningisrecommendedwhen purchasingnetworkcabinets,andflexibilityofaccessoriesisdesirable.Thisway,futuredevelopmentswillalso betakenintoaccount.Itmustbeborneinmindthatthe interiorstructureofracksdiffersgreatlyfromonetothe next. Thefrequentswitchingbetweenthevariousconnections ofthenetworkcomponentsmeansthatcablesinnetwork cabinetshavetobereroutedconsiderablymoreoften thanisthecaseinservercabinets.Thesemovements,also knownasMACs(Move,Add,Change),aswellasincreasingportdensity,lendspecialimportancetothetask ofcablemanagement.Thisstartswiththeroofpanels andbases.Simplecableentryatthesepointsfacilitates retrofittingandkeepscablerunsshort.Routingductsand guidepanelsensureorderlyfinedistributionintherack. Atthesametime,cablemanagement,inparticular,must placegreatimportanceonthestabilityofcomponents. Formoderncurrent-carryingnetworkcablesaremuch moreexpensiveandrigidthantheirCat-5predecessors.
4.1.1 Makinganinventoryoftheserver cabinet

Indatacentresespeciallythoseofacertainsizeitis hardtomaintainanoverviewofallinstalledhardware components.Althoughitispossibletocommunicate withanyintelligentITdevicetoday,theequipments physicalconnectiontotherackandtherelevantheight unitisanothermatter.Thestructureofequipmentin theindividualcabinets,withservers,fans,UPS,etc.is oftennottransparent.Asaresult,makinginventories andcontinuallyupdatingdataaboutthearrangementof componentsinthedatacentreisexpensiveand,inmost cases,time-consuming.Inmanycases,existingmanuallywrittendocumentationisnotcheckedforaccuracy. However,thisinformationisvitalformakingdecisionsin theeventoffailure. Anotherproblemisthehalf-lifeoftheacquiredinformation,fortheserecordsorupdatesstillonlyeverrepresent asnapshotofthedatacentreinventory.Yetefficientrack assignmentandtransparentcomponentmanagement requirecontinual,up-to-dateandthereforereliabledata. Toensureconstantaccesstoup-to-dateinventorydata, moderninventorysystemsexistdirectlyintherack,which registertheequipmentofthe19levelcompletelyand withoutcontact. Ontheonehand,therackconfigurationsarevisuallydisplayedonawebsitebelongingtothemonitoringsystem inquestion;ontheotherhand,theusercanopenalistor import/exportdatausingtheXMLfileformat.Thesedata recordscanthenbefurtherprocessedinexternaldatabases,forexample,andconsiderablyfacilitateeveryday workduringdatacentreoptimisation.
10
Onesubjectthatiscurrentlygaininginimportancewhen itcomestonetworkcabinetsisairconditioning.Switchesandroutersarebecomingevermorepowerful,and producemoreandmorewasteheat.Forthisreason,here, too,possibilitiesforexpansionhavetobeconsidered.The spectrumrangesfrompassiveairconditioningviaroof panels,ventilationattachmentsanddual-walledhousings tofans,rightuptoroofcoolingunits.
Amodular(andhenceextendible/modifiable)fireproof securityroom,certifiedasfaraspossible,mayformpart oftheequipmentofthedatacentre.Theuseofastable, multilayeredfiresafetydoorwiththesameprotection valuesasthesecurityroomisalsocompulsory.Other constructionssuchasahermeticallysealedceiling,wall andfloorsystemtoprotectagainsttheingressofsmoke orwater,forexample,oramultistageveryearlyfiredetectionsystemwithmultiplesamplingpoints,including intheraisefloor,arenowthestateoftheart.Theseare joinedbyanappropriatelydimensionedautonomousfire extinguishingsystemwithpressurereliefandventilation ductdampers,personalaccesscontrolusingcardreaders orbiometrictechnology,anddatacentreperipherymonitoringviaLANvideotechnology. Toensuretheflexibleexpansionofdatacentres,itisa goodideatoworkwithplannersandsupplierswhocan ensurethelong-termavailabilityoftherequiredproducts.
4.3 Reliabledatacentre
Inadditiontothebasicrequirementsforareliabledata centrementionedabove,numerousprojectdetailshave tobeclarifiedinrelationtoconstruction. Firstofall,apreciseanalysisofrisksandweakpointsin thecompanymustbecompiled,flagginguppossiblerisks forITsystems.Responsibilityforplanningandbuilding adatacentre,personswithaccessauthorisationand regularsafetychecksbyindependentauditorsmustbe covered. Manypeopleareresponsiblefortheplanning,constructionandoperationofadatacentre.InadditiontoITspecialists,theseincludebuildingexpertssuchasarchitects andconstructionengineers,specialistplannersforair conditioning,energyandactivedefence,thedepartment inchargeoforganisationand,lastbutnotleast,the managementboard. Thephysicalrequirementsfacingadatacentredonot consistonlyofpureIT-relatedmatters,suchasthenumberandtypeofserversandnetworkandstoragedevices, butalsoactiveandpassivedefence.
11
5 Energy, air conditioning and cooling

5.1 Powersupplycompanies(PSCs) electricityfeed-inanddistributioninthe company 5.1.1 Currentsituation
Theelectricitysupplyisofmajorimportancefortheoperationofservercabinetsorentiredatacentres. Whenbuildingadatacentre,ready-madepowersupply Thepowersupplychainbeginsatthepowersupply companies,whichgenerateandsupplyprimaryenergy. Theprimaryenergygeneratorstransporttheelectricity throughcablesviahigh-voltagemaststomedium-voltage substations.Fromthemedium-voltagesubstations,the electricityisconveyedviaundergroundcablestotransformerstations.Transformerstationsaremostlysituated inlargebuildingsorattheedgeoftheroadonspecially designedpremises. Largedatacentreswithanareaofseveralthousand squaremetresoftenfeatureadditionalfeed-insviaa secondmedium-voltagesubstation,ensuringfullredundancyi.e.severalchannelstoincreaseavailabilityeven asfarasthepowerplantsthemselves. Possiblecausesforaninterruptiontothepowersupply couldbe: Technicalfaultsinequipment(e.g.serversandtheir components) Technicalfaultsinthepowerdistribution(e.g.cables, sub-distributionboards) Faultsinalternativepowersupplysolutions(e.g. standbygeneratingsystems,alsoknownasemergencydieselgenerators,battery-backeduninterruptiblepowersupply(UPS)systems) Faultsduetotheprocess(e.g.designfaultsinthepower supply,logisticalerrors) TheoutputfromtheUPSsystemsisroutedviathe UPSsub-distributors,andfromtheretotheindividual servercabinets.Junctionboxesordistributionboxesare installedintheraisedfloorforthispurpose.Fromthese branchesorjunctions,morecablesfeedtheelectricity tothepowersupplyunitsoftheserversinthecabinet. IfonlyoneUPSsystemisinstalled,powersupplyunitsA andBhaveajointsupply;iftherearetwoUPSsystems, theirsupplyisseparate.A2xNsupplythereforeincreases availability. solutionscannotsimplybepulledoutofahat.However,basicprincipleshavebeensetout,whichmustbe adaptedtoindividualcircumstances.Thechallengefacing theplannerliesinputtingtheseprinciplesintopractice inaccordancewiththecustomer,hisrequirementsand wishesand,lastbutnotleast,hisbudget. Pastexamplesshowhowdramaticallysituationscan escalateifthepowersupplyfailsforlongerperiodsand nobackupsolutionisavailable.Thegeneralpowersupply cansuccumbtofailureforseveraldaysinlargeareas. Fromreportsofthedamagecausedinsuchcases,itis easytoseehowessentialanautonomouspowersupply is,particularlyinareasofcriticalimportancetocompanies,suchasIT.
5.1.2 Howtheinfrastructurefunctions
Thepowersupplyfunctionsbymeansofringmains,the electricityfromwhichistransformedto400Vinthe transformerstationsandwhichisconveyedtothedata centrethroughcablesorbusbarsviathelow-voltage maindistributionandutilitypowersupplynetwork.The normalmainssub-distributionalsosuppliestheuninterruptiblepowersupply(UPS)systemswithelectricity.
12
5.1.3 Recommendedequipmentfordifferentdowntimes Data centre category Permitted data centre downtime* Electricity feed-in from PSC Servers cabinet up to 5 kW Servers cabinet from 5 kW to 30 kW
Standard Standard Redundantfeed-ins Redundantfeed-ins Redundantfeed-insfromdifferenttransformerstations
Data centre / server room 500 to 2500 W/m2
A B C D E
72h 24h 1h 10min 0min
Table2fromtheBITKOMmatrixPlanningGuideforaReliableData CentreElectricityfeed-infromPSC
GreatersecurityisofferedbyCategoryC.Here,thepower supplyisredundantevenfromthelow-voltagemain distribution.Ifasupplychannelupstreamofthelowvoltagemaindistributionfails,electricityisautomatically suppliedviathesecond,redundantchannel.Iftheprimary energysuppliersuffersfailure,thepowersupplyisstill assuredbythemobilestandbygenerator. UnlikeCategoriesAandB,CategoryCfeaturesasecond UPSwithtwodedicatedUPSsub-distributors.This ensuresaredundantsupplyfromtheUPSsystemstothe serverpowersupplyunits. CategoryDdiffersfromCategoryCinthatthesupplyis ensuredbytwodifferenttransformerstations,eachwith theirowndownstream,redundantinfrastructure.Moreover,intheeventofapowercutattheenergysuppliers end,electricityissuppliedbythestationarystandbygenerator.Thiscategorypromisesaveryhighlevelofsecurity. CategoryErepresentsthegoldstandard.Notonlydoes itofferadditionalredundancyviaasecondstandbygenerator,italsoprovidesanadditionalfeed-infromanother independentmedium-voltagesubstation.However,in almostallcasesthesecondcablefeedfromanother medium-voltagestationfirsthastobeproducedbythe responsibleprimaryenergysupplier.Thiscanmeanthat severalkilometresofnewcablemustberoutedtothe
CategoriesAandBarecurrentlyinplaceinmanysmallto medium-sizedbusinesses,oftenevenwithoutanypossibilityoffeed-inforamobilestandbygenerator.When examinedmoreclosely,however,thisversiondoesnot provideanyrealsecurity,andplacesitstrustsolelyinthe energysuppliers.WearealwayshearingtheoldsayingIt wonthappentome.Anditmaynothavehappenedyet. Butifasinglelinkinthesupplychainfails,thisinterrupts theentiresupplyfromthePSC,andelectricityhastobe suppliedbytheUPSsystem.Moreover,thebackuptime ofaUPSsystemisgenerallyseverelylimited.Itdepends onthenumberofbatteriesinstalled,andthepowerthat hastobeprovided.AUPSsystemisgenerallyunableto bridgeapowercutlastingmorethan30minutes.Inthis case,afunctioningcomputershutdownroutineshould automaticallybeinitiated,messagestransmitted,data saved,applicationsclosedand,finally,computersproperly shutdown. Plannersmustthereforetakeparticularcaretoensure thatthebackuptimeoftheUPSsystemislongerthanthe timerequiredtotransportandactivateamobilestandby generator.Asarule,batteriesareusedfortheabove constellations.
13
datacentrepremises,whichisextremelycost-intensive andshouldbetakenintoaccountduringcosting. Regularinspectionandmaintenanceoftheentireinfrastructurebyqualifiedpersonnelandcompliancewith regulationsandguidelinesonoperationofthesystems areallindispensableformaintainingavailability.
cut-outsmaybeused.Acabinettypicallycontainstwo separatepowerstrips,whichpermitredundantoperation ofITsystems. Overviewofpoweroutputclasses:
Phases
One One Three Three
Maximum amperage
16A 32A 16A 32A
Maximum power output

3.6kW 7.2kW 11kW 22kW
5.2 Powerdistributioninthecompany 5.2.1 Currentsituation

Powerdistributorsconveythepoweroftheutilitypower supplynetwork,thegeneratorandtheUPStotheequipment,systemsandlighting.Twodistributionsystemscan beemployedtoensurehigheravailability.
(Furthercombinationswithtwophasesarealsopossible butareuncommoninGermany) Modernpowerdistributionunits(PDUs)alsofeature measuringorswitchingfunctions,aswellasanetwork connectionforextendedenergymanagement.Inaddition,variousmodelsofferenvironmentalmonitoringwith arangeofsensorsformeasuringthetemperatureandair humidity,forexample. SincemostITunitsindatacentresareinstalledin19cabinets,thequestionarisesastowherethepowerdistributorsshouldbesituatedandhowthepowercablesshould beroutedtothe19cabinets.PDUsareeitherinstalledin thewallorsurface-mounted,inseparatecabinetsorintegratedina19version.Thecablesareoftenroutedunder theraisedfloor,whichisalsousedasaconduitforcoldair. Thepassageofaircanbeimpaired,whileaccesstocables isdifficult.Thisproblemcanberemediedbyrouting cablesintothe19cabinetfrombelowviaaraisedfloor plate.Alternatively,cablescanberoutedalongtheceiling orwallsincablerunsorviapowerdistributionsystems,so thattheyhavetoenterthe19cabinetfromabove.IntegratedPDUshaveanadvantageinthattheyarelocatedin thevicinityofthepointofuse,andarethereforewithin easyreachofthe19cabinets.Cablescanberoutedon theroofofthe19cabinets,providedthatpoweranddata cablesareroutedseparately.
5.2.2 Howtheinfrastructurefunctions
Inthepowerdistributionsystem,thenormalmainssuppliesthebuildinginfrastructure,includinglifts,lighting butnotemergencylightingsystemstoVDE0108compressorsinDXairconditioners(DX=directexpansion), chillersandotherinstallations.Intheeventofpower failure,thispowersupplyisinterrupteduntilanavailable generatorstartsupandanautomaticswitchcausesitto restoretheelectricitysupply. Allpowerdistributionsystemsmustbeequippedwithan inputfuse.Thesizeandtypeofpowerdistributionsystem dependsuponthepowertobedistributed,thedesired numberofcircuitsandthepowerpercircuit.Pleaserefer tothetablebelow. Ideally,fuseprotectioniseffectedselectivelywithinthe powerstrip,i.e.theoutputsaremonitorednotbyan overallfusebutbyseveralfuses,eitherseparatelyorasa group.Asaresult,intheeventoffailureonlytheaffected outputorgroupisdisconnectedfromthemains,instead oftheentirepowerstrip.Eitherfusesorautomatic
14
SpecialattentionmustbepaidtothePDUstripsinthe cabinets.Thankstotheunitsmodern,compactdesign, thesedaysmanysystemscanbeinstalledinonecabinet. Inextremecases,acabinetmayhave42heightunitswith 42serversperheightunitandtwopowersupplyunitsper server,forexample.Atotalof84socketswouldhavetobe availableforthiscabinet.
toassignspecialistpersonneltoconverttheirsystems. Todaysmultipleoutletstripsallowmodulestobereplacedduringongoingoperation.High-endsystemssuch asthisgenerallyalsofeatureHTTPorSNMPmonitoring andmanagementoptionsanduseradministration,which guaranteesthatonlyauthorisedpersonnelcanconfigure theoutletstrip.Thesemodularsystemspermitbasic equipmentoftheracksbymeansofaverticalmounting railwiththree-phasefeed-in.Thevariousplug-inmodules cansimplybeinsertedinthisrail,considerablyreducing wiringandassemblytimeandexpenditure.
5.2.3.Intelligentmultipleoutletstrips
Formanagementatracklevel,transparency,orderand easeofhandlingarevital.Ideally,themultipleoutlet stripsusedinadatacentrewillhavedifferentplug-in modulesforcountry-specificsystems,forexample,which canbeusedbyuntrainedstaff.Inthiscase,international organisationshavetheoptionofusingthesametypeof multipleoutletstripinalltheirbranches,withouthaving
Finally,hostingcompanies,forexample,whichmust demonstrateahighdegreeofaccuracyinthedistribution ofenergycostsperserver(inonerack),cannowbenefit fromofficiallycalibratedsocketmodules.Calibrating measuringinstrumentsofthiskindarealsoavailablefor thePDU.
5.2.4 Recommendedequipmentfordifferentdowntimes Data centre category Permitted data centre downtime* Distribution Servers cabinet up to 5 kW Servers cabinet from 5 kW to 30 kW Data centre / server room 500 to 2500 W/m2
A B C D E
Standard, connectionofserversviaUPSandutilitypowersupplynetworkisrecommended Standard, connectionofserversviaUPSandutilitypowersupplynetworkisrecommended Redundantsetup(AandB) Redundantsetup(AandB) Redundantsetup
Table3fromtheBITKOMmatrixPlanningGuideforaReliableDataCentreDistribution
15
Theformredundancytakesdependsonthenumberof powersupplyunitsintheITdevices.Agoodbasisforhigh availabilitywouldbetwopowersupplyunitsperdevice, whichareredundant.Then,intheeventoffailureofa powersupplyunit,theremainingpowersupplyunitis abletocontinuesupplyingtheITdevicenormally.These twopowersupplyunitsperdeviceshouldbesupplied bythePDUviatwoseparatePDUstripstotwoseparate powercircuits.Availabilitycanbefurtherincreasedby usingtwoseparatePDUs,whichreceivepowerfromtwo separateUPSsystemsviatwoseparatetransformersand twoseparategenerators.
besafelyisolatedwithoutanyilleffectsonothercabinets andITunits. Wherepersonalprotectionisconcerned,newrequirementsexistforadditionalprotectionforfinalcircuitswith sockets.Since01.06.2007,DINVDE0100-410:2007-06, ProtectionAgainstElectricShockappliestonewlyconstructedsystems.Modificationsandextensionstoexisting systemsmustalsobecarriedoutinaccordancewiththis standard. Theabovestandardstipulatesadditionalprotectionby meansofresidualcurrentdevices(RCDs)forallsockets ina.c.voltagesystems,ifthesysteminquestionisto beusedbylaypeopleorisforgeneraluse.Caremust betakentoensurethatfaults/damagesareeliminated immediatelybyanelectricalengineer,includinginconnectedelectricaldevices/consumables/equipment.This requiresapermanentmonitoringsystem,plusorganisationalmeasurestoensurerapidtroubleshooting. Acontinuousresidualcurrentmonitorsatisfiesthe currentstandardgoverningprotectivemeasuresandalso offersincreasedfireprotection,evenwithoutswitch-off byanRCD.
5.2.5 Protectivemeasuresandhigh availability

Datacentresaresubjecttomaximumrequirements regardingavailability.Theenergysupplymustbeassured permanentlytoreflectthis.Itthereforegoeswithout sayingthatthepowersupplytothedatacentreitselfand allareasinthesamebuildingtowhichdatacablesare routedmusttaketheformofaTN-Ssystem.Permanent monitoringofacleanTN-Ssystem(e.g.withRCMs) andforwardingmessagestoaconstantlymannedpoint, e.g.thecontrolcentre,isabsolutelyessentialforreliable operation.Messagesinformtheelectricalengineerabout whatactionneedstobetaken,sothathecantakeappropriateservicemeasurestopreventdamage. AllPDUsmustbeequippedwithaninputfusetoprotect thelines.Thesizeandtypeofpowerdistributionsystem dependsuponthepowertobedistributed,thedesired numberofcircuitsandthepowerpercircuit.Power circuitshaveeitheroneorthreephases.Typicalpower supplysizesare16A,single-phase(approx.3.5kW),32A, single-phase(approx.7kW)or32A,three-phase(22kW) forhigh-performancecabinets.Aparticularlydifficult subjectisthatofselectivefuseco-ordination,which enablesashortcircuitorshorttogroundinacabinetto
5.3 Uninterruptiblepowersupply(UPS) 5.3.1 Currentsituation

Thecausesofpowerfailureareoftenquitebanal:even simplevoltagefluctuationsortransientfailuresinthe electricitygridcanbeenoughtodamagehardwareor softwareortointerferewithittosuchanextentthat severeerrorsoccurinITprocesses.Irregularitiesinthe gridmayberare,buttheyaremorefrequentthanmany peopleassume.
16
UPSsystemsareemployedtopreventthepossible negativeconsequencesofbriefpowerfailures.Theyfilter outdisturbances,suchassurgesanddipsinvoltage,and bridgeinterruptionsinthemainssupply.Thisreduces transmissionerrors,computercrashes,bugsandlossof data.
5.3.2 UPSsystemtechnology
UPSsystemsdifferintheirdifferenttechnology.Themost frequentlyusedtechnologyisthestaticUPSsystem. Rechargeable(secondary)cells(accumulators)areused tostoreenergy.Whentwoormoreconnectedcellsare linkedinacircuit,thisisknownasasecondarybattery,or justarechargeablebattery.Intheeventofapowerfailure, theenergyfromtheaccumulatoriskeptreadyforcritical loadsbyastatictransformer(inverter)attheoutputof theUPSsystem.Thebackuptimeisdeterminedbythe loadandthecapacityoftheaccumulatorbatteries,butis typicallywithintherangeof10tomax.30minutes. ThesecondtypeoftechnologyusedisthedynamicUPS systemeitherwithorwithoutpistonengine.Dependingonthedesign,theenergyisstoredbyakineticbulk storagedeviceor,asabove,arechargeablebatterysystem. ThedynamicUPSsystemmakesthebatterysenergy availabletocriticalloadsbymeansofarotatingtransformer(generator)attheoutputoftheUPSsystem.With kineticstorage,thebackuptimedependsontheloadof theITunitsandthekineticenergyofthestoragedevice
Number of power failures and brief power losses p.a 60 50 40 30 20 10 0 0-10 ms 10-20 ms 20 ms-1s Most dangerous lurks here: in the range up to 1 second 1 s-1h > 1h
Duration of power failures
Figure1:Frequencyofmainsfaultsinrelationtotheiraverageduration
(massandspeed),andismeasuredinseconds. ThedynamicUPSsystemwithcombustionengine
Sincethe1980s,investmentinGermanpowersupply systemshasfallenbyapproximately40%.Inrecentyears, averagedowntimehasrisenfrom15minutesto23minutesperyear.Andthegrowinguseofregenerativeenergy meansthatelectricitygridsareincreasinglyreachingthe limitsoftheircapacity.Theriskofbottlenecksandblackoutsisrising.ThesearetheconclusionsofthenewVDE studyonpowersupplyquality.
combinesaUPSsystemwithastandbygenerator,andis thereforecapableofbridgingpowerfailuresoveralonger period.
5.3.3 Methodofoperation
StaticUPSversionsaredividedinto3categories.The classificationandassociatedmethodsfordetermining staticUPSsystemsaredefinedanddescribedinEuropean standardEN62040-3.
17
Mains failures
1. Powerfailure 2. Voltage fluctuations 3. Voltagepeaks 4. Undervoltage 5. Overvoltage 6. Surge 7. Effectsof lightning 8. Voltagedistortion (burst) 9. Voltageharmonics 10.Frequency fluctuations
Time
>10ms <16ms
EN 62040-3
VFD Voltage+Frequency Dependent
UPS-solution
Classification3 Passivestand-by-operation(offline)
Arresto solution
............................. .............................
4...16ms ............................. continuous continuous ............................. <4ms sporadic VFI Voltage+Frequency Independent Classification3 Doubleconversion operation(online) Deltaconverter ............................. Lightningandover voltageprotection IEC60364-5-534 ............................. continuous ............................. sporadic ............................. VI*) VoltageIndependent Classification2 Lineinteractive operation .............................
periodic
*)Alternativetechnologiesarecapableofovercomingmainsdisturbances1to9.
Table4:TypesofmainsfaultsandsuitableUPSsolutionsaccordingtoEN62040-3(ref.:UninterruptiblePowerSupplies,EuropeanGuide;Hsgr.ZVEI2004)
DynamicUPSsystemswithandwithoutcombustion enginearesubjecttoDIN6280-12. Indatacentres,staticUPSsystemswiththeclassification VFItoEN64040-3ordieselUPSsystemstoDIN6280-12 shouldalwaysbeused. StaticUPSsystemswiththisclassificationareavailable withanoutputrangeof10kVAto1600kVA,andcanbe
connectedinparalleluptoanoutputof4800kVA,dependingonthemake. DieselUPSsystemsareavailablewithanoutputof200to 1750kVA.Theyareabletocoverthelowandmedium-voltagerange.Numerousunitscanbeconnectedinparallel.
18
5.3.4 UPSredundancy
RedundancyistakesthefollowingformwhenUPSsystemsareemployed.
presentintheinputandoutputterminalsoftheUPSsystemeveninbypassmode.Thesystemcannotbereplaced withoutcuttingofftheloads.However,whenanexternal manualbypassisused,theUPSsystemcanbereplaced withouttheloadshavingtobeswitchedoff.Ifthemanual bypasstakestheformofmodularblocksconnectedin
100%
2N
100% 100%
parallel,orparallelblocks,itmustbedesignedformaximumconsumerload.
N+1
100% 100% 50% 50% 50% 2(N+1) 50% 50% 50% 50%
5.3.6 Energystorageunits
Kineticenergystorageunitsaredesignedordimensioned almostexclusivelybythemanufacturersofUPSsystems. Theachievablebackuptimesarerestrictedtoseconds,so thattheiruseislimitedtodieselUPSsystemsorcombinationswithfast-startstandbygenerators. Theelectrochemicalstorageunitsusedinconnection withUPSsystemsincludeleadandnickelcadmium batteries.Useoflithiumionbatterieshasnotbecome widespread.Nickelcadmiumrechargeablebatteriesare relativelyinsensitivetoincreasedambienttemperatures, buttheiruseiscontentiousbecauseofenvironmental pollution. TheenergystorageunitmostcommonlyusedinUPSsystemsistheleadbattery.Leadbatteriesareverysensitive totemperature.Lowtemperaturesreducethebattery capacityandthereforethebackuptimeorpower,while hightemperaturesreducethebatterysusefullife.The optimumambienttemperatureis20C. Theusefullifeofbatterysystemsvariesdependingonthe technology,thematerialsusedandotherfactors.AccordingtoEurobat,usefullifeisbasedonanambienttemperatureof20Candlaboratoryconditions.Thefollowing usefullifefiguresarespecified: 35yearsstandardcommercial 69yearsgeneralpurpose 1012yearshighperformance 12yearsormorelonglife
N+1
50% 50%
Figure2:RedundancyinUPSsolutions
5.3.5 Electronic/manualbypass
Theelectronicbypasshasthetaskofswitchingtheloads withoutinterruptionfromthemainstotheinverterof theUPSsystem(safebusbar)andback.Intheeventof faultsduringinverteroperationorwithlargeamounts ofoverload,theelectronicbypassswitchestheloadback tothemainswithoutabreak.Dependingondesign,the electronicbypasscaneitherbeintegratedintheUPSsystem(singleblockandmodularblock)ortaketheformof anexternalcomponent(parallelblockwithexternalelectronicbypass).Anotherelectronicbypasscanbeconnected inparalleltocreateredundancy(N+1). EveryUPSsystemshouldbeequippedwithamanual bypass.ThisswitchesthevoltageoftheUPSsystemoff forthepurposeofserviceandmaintenancework.Ifthe manualbypassisintegratedinthesystem,voltageis
19
Toensureareliablepowersupply,thebatterysystem mustbeinspectedregularlyandreplacedbeforetheservicelifehaselapsed.Inaddition,itmustbeborneinmind thatthebatterylosescapacityoveritslife.Settingthe systemupforveryshortbackuptimescarriestheriskthat thealreadyagingsystemmaynolongerbeabletoprovidetherequiredpowerandmaycutofftheUPSsystem. Inareasofimportancetosafety,overdimensioning(factor 1.25)isrequired,toensurethatsufficientlyhighcapacityis stillavailableattheendofthebatterysservicelife. Iftheoperatordecidestodispensewithredundancyin hisUPSsystem,thebatterysystemshouldatleastbe dividedintotwosections.Theachievablebackuptimeof asectionisjustonepartoftheplannedbackuptime.This ensuresthatpoweroutages,atleast,arebackedupfor afewseconds.Thisisnotasuitableapproachforhighly availabledatacentres,however.
Furthermore,redundancyarrangementsandthepossibilitiesofinputandoutputsupplyareotherimportant considerations. Awholerangeofdifferentconceptsareavailablefor selectionforconstructingtheUPSsystem.Smaller,individualUPSdevicesarepopularforsafeguardingthesupply ofafewserversandITstoragesystems.Differentversions aretheUPScabinetortowerunitswithintegratedbattery,theexternalbatterypack,andtherackversionfor installationina19cabinet.LargerUPSsystemsassingle blockorparallelsystems,mostlywithexternalbattery cabinets,batteryracksorflywheelaccumulatorsystems, aremostlysetupandoperatedindedicatedoperating rooms.Here,amodern,liquid-cooledUPSsystemoffers low-cost,efficientanddirectUPSairconditioningwithout anyspecialroomairconditioning.Furtheradvantagesof operatingroomsdedicatedtoUPSarethatthickpower cablesincomputerroomsareavoided,asarebatteries, whichalsorepresentafirerisk.ModularUPSsystems combineservicefriendlinesswithrapidadaptabilityto frequentlychangingmaximumpowerrequirements. However,thenumberofmodulesusedhastobeconsidered,asavailabilitydecreasesthemorecomplexthe systembecomes.WhenUPSsystemsareinstalledin servercabinetsortaketheformofadedicatedUPSrack inroomssharedwithITequipment,theadditionalfire riskposedbytherechargeablebatteriesmustbetaken intoconsiderationinthedesignofalarmandfiresafety devices.
5.3.7 Recommendedequipmentfor differentdowntimes

ThemostimportantfactorstobearinmindwhendesigningaUPSsystemistherequiredelectricalpowerof connected,criticalloadsandtheinstallationconditions.To providebackuppowerduringoutages,anenergystorage unit,suchasabatterysystem(cabinetorrackwithdisconnectorsandfuses),oraflywheelstoragedevicemust beplannedthatsuitsthepowersupplyenvironment.
20
Data centre category
Permitted data centre downtime*
UPS Servers cabinet up to 5 kW Servers cabinet from 5 kW to 30 kW Data centre/ server room 500 to 2500 W/m2
Standard, min.backuptime10minutes,maximumdurationdependentoncontrolledshutdowntime ofservers Standard, min.backuptime(incl.ventilation)10minutes, maximumdurationdependentoncontrolled shutdowntimeofservers
Standard, min.backuptime1h,maximumduration dependentoncontrolledshutdowntimeof servers Standard, min.backuptime1h,maximumduration dependentoncontrolledshutdowntimeof servers
72h
24h
C D E
Redundant(N+1), backuptime1020min Redundant(N+1), backuptime1020min Redundant, backuptime1030min
1h 10min 0min
Table5:FromtheBITKOMmatrixPlanningGuideforaReliableDataCentreUPS
Ventilationunits,chilledwaterpumpsandcoolingunits/ compressorsmayneedtobesuppliedviaaUPSsystem, dependingontheenergydensityandselectedbackup time.Theamountofenergyrequiredforcoolingcanalso bemadeavailableviaastorageunit,insteadofcooling units/compressors.Nocoolingathighpowerdensities resultsinoverheating,causingITdevicestoshutdown withoutusebeingmadeoftheestablishedbackuptime foranyplannedshutdown.
Efficiencyandpowerlossforthevariousloadratios duringtypicaloperatingcycles(e.g.day/night,weekday/weekend),examinationofenergyefficiency DetailsonUPSfuseprotectionforthevariousoperatingmodes Effectsonthemainsinputandinputpowerfactor. However,theeffectsoftheconnectedloadwhen theUPSisinbypassmodemustalsobetakeninto consideration Availablebackuptimeofabatterysystemorflywheel accumulatoratactualload Maximumavailablebackuptimeofabatterysystem orflywheelaccumulatoratactualload Informationontheenergystorageunitandcharge/ dischargebehaviour Permittedambientparameters,suchasoperating temperatureandhumidity;implementeddegree ofprotection,firesafetyandair-conditioning requirements Noise Electromagneticcompatibility(EMC) Dimensionsandweights
5.3.8 Specialfeatures
ImportantplanningfeaturesfordimensioningandinstallingaUPSsystemare: Outputpowerratingattherequiredloadpowerfactor (thesedaysmin.0.95) Connectionvaluessuchasinput/outputvoltageand frequency Current,wirecrosssectionsandpossibilitiesforconnectingUPSinputsandoutputs
21
Thisguidecannotofferapreciseanalysisoftheseindividualfeatures,becausethecircumstancesofthedata centrepowersupplywillalwaysrequiredetailedplanning. Someinterdependenciestobeconsideredarementioned hereasexamples: Theimportanceoftheconnectedbattery/flywheel accumulatorforthebackuptimeonpowerfailure,if anemergencygeneratorisavailable. Considerationoftheinputpowerfactorwhendimensioninganemergencygenerator.OperationviaUPS powerelectronicsandoperationviathebypassmust alsobetakenintoaccounthere. TheinfluenceoftheUPSoutputpowerfactoronthe possibilityofsupplyingmodernswitchingpower supplyunitswithelectricity,evenatfullload Thelimitationofpowerduringoperationathigh altitudes. Theimportanceofconsideringefficiencyoveratypical operatingcycle(fluctuationsincapacityutilisation),in ordertoobtainrealisticestimatesofrunningcosts ThepriceofaUPSdependsonequipmentdetailssuch asfilters,transformers,fans,electronicbypass,integratedorexternalmanualbypassanddifferentswitching concepts.Calculatingthepriceofbest-practicesolutions isextremelycomplexforUPSsystems,andrequirestimeconsuminganalysisofthecircumstances,boundaryconditionsanddependenciesandconsiderationofnumerous individualparameters.
inordertomaintaintheoperationofadatacentreandits technicalsystems,suchasairconditioning,electricityand safety. Theamountofpermitteddowntimehasmaximumprioritywhenplanningemergencypowersystems.Standby generatorsaredividedintovariousgroups,accordingly: Generatorswithoutarequiredtimefortakingonthe load.Systemsareputintooperationmanually.These systemsareunsuitableforautomaticoperationin datacentres. Generatorswitharequiredtimefortakingonthe load.Here,aninterruptionmustbelessthan15 secondslongbeforethegeneratorassumesthetask ofsupplyingpowerafterautomaticallybeingbrought intooperation.ADINstandardsetsouttherequirementsforgeneratingsetswithcombustionengines forsafetypowersuppliesinhospitalsandinbuildings andpremisesintendedforgatheringsofpeople.This standardshouldalsoberegardedasaminimum requirementforgeneratingsetsinthedatacentre sector. Generatorswithauto-reclosingintheformofstandby powersupplyunits.Here,theinterruptiontimemust belessthanonesecond.Thesesystemsarenolonger usedindatacentres,asaninterruptiontimeofless thanonesecondisnotnecessary. Generatorsforuninterruptiblepowersupplyinthe formofdieselUPSsystems.Here,onpowerfailurethe loadistakenonwithoutinterruption.
5.4 Backuppower 5.3.1 Generatingsetsforsupplying backuppower(emergencypower)inthe eventofpowerfailure

Electricitysupplierscannotguaranteeanundisturbed powersupplyatalltimesandallplaces,andpowersupply companies(PSCs)alwayswaiveanyliabilityintheirstandardcontracts.Therefore,briefinterruptionsorsustained powerfailuresmustbebridgedbybackuppowersystems,
5.4.2 Emergencypowersupplies
Inthetwolastcases,specialversionsofgeneratingsets arerequired,whicharestandbyunitswithanenergy storageunit.Thelattermustbecontinuouslyfed.The resultingoperatingcostsmeanthattheconsumerpays forhismorereliablesupply. Variousversionsofstandbyunitsexist,ascombinations madeupofdieselengine,flywheel,electricmachineand thenecessarycouplings.
22
Standbypowersupplyunitsarerequiredwheneveran interruptiontime,suchasthatcausedbytheuseof simpleemergencygenerators,wouldnotbeafeasible optionforreliablycontinuingongoingoperationatthe consumersbusiness. Themostcommonlyusedsystemsinthedatacentreare thosementionedsecond.Theinformationbelowrefersto thesesystems
Switch-on behaviour
Thestartingandswitch-onbehaviourofelectricmotors, transformersandlargelightingsystemswithbulbshas aninfluenceongeneratoroutputpower.Whereasynchronousmotorsareused,theapparentpowercanreach 6times,therealpower23timesnominalpower.The requiredgeneratoroutputpowercanbeconsiderably reducedbyphasedswitch-on.Allavailablemeasuresto limitstartingpowershouldbeusedtothefull.
5.4.3 Designingtheemergencypower system

Thefactorsbelowaredecisivewhendesigningtheoutput powerofthegenerator: Sumtotalofconnectedloads Demandfactor Startingcurrentsandstartingcosphiofloads Circuitfeedbackofloads(rectifiertechnologyofUPS systemsorfrequencyconverters) Permitteddynamicbehaviour Reserveforexpansions Supplementfordifferentambientconditions
Dynamic behaviour
Thedynamicbehaviourofthegeneratorwhenthefull loadisswitchedonandduringexpectedloadchanges duringoperationmustbeadaptedtothepermitted valuesoftheconsumers. Themotor,generatororbothmayhavetobeoverdimensionedinordertosatisfytherequiredvalues.
Ambient conditions
AccordingtoDIN6271,themotorreferencetemperature is27C.Ifoperatingtemperaturesexceedthis,themotor mustbeoflargerdimensions.Themotorsreduction factorsmustbeestablished.
Consumer power
Whencalculatingtheconsumerpower,bearinmindthat theapparentpowerandrealpowerhavetobestated.
Demand factor
Indatacentres,thegeneratoroutputpowermustbeset upwithademandfactorof1,asoperationofthedata centremustbemaintainedinbothsummerandwinter.
23
5.4.4Recommendedemergencypowersupplyasafunctionofthepermitteddowntimes: Data centre category Permitted data centre downtime* Emergency power Servers cabinet up to 5 kW Servers cabinet from 5 kW to 30 kW
Optional Optional Redundant, availabilityin15s,fuelreserve:24hours Redundant, availabilityin15s,fuelreserve:72hours Redundant, availabilityin15s,fuelreserve:72hours
Data centre/ server room 500 to 2500 W/m2
A B C D E
Table6:FromtheBITKOMmatrixPlanningGuideforaReliableDataCentreEmergencyPower
Itispossibletohiregeneratingsetsfromyourpower supplycompany,whichguaranteetheemergencypower supplyduringmaintenanceandrepairsviaanexternal connection.Buthiredgeneratorsarenoanswertounforeseenpoweroutages,asyoucannotbecertainwhetheror nothiredgeneratorsareactuallyavailableatthetimeof occurrence.
Basic room requirements

Theroominwhichanemergencygeneratoriserectedis anelectrotechnicaloperatingroom.Itmustbeprotected toF90quality,andisafirezoneinitsownright.Ventilationaperturesmustbeprovidedtosupplythecooling andcombustionair,andforremovingheatedcoolingair. Theseaperturesmustleaddirectlytotheoutside.Rooms withoutexteriorwallsareunsuitablebecauseofthecross sectionsrequiredforventilation.Ifnecessary,ventilation ductsinF90qualitymustbebuilt,whichleaddirectlyto theoutside.Inordertoavoidshortcircuitsofair,supply andextractionaperturesmustnotbesituatedimmediatelynexttooneanother.Thegeneratorroommusttake theformofacatchbasin,withacircumferentialbarrier of10cmwith3coatsofoil-resistantpaint,forprotection againstfloodingandforenvironmentalprotection.This basinmustbemonitoredforleaks.Thesizeoftheroom mustallowanescaperoute1mwide.Thedoorsofthe roommustbeofatleastT30qualitywithapaniclock.
Room planning/detailed planning for emergency generators

Thefollowingdetailsmustbeconsideredduringroom planning/detailedplanning: Compliancewiththefollowingregulations(DINVDE, VDS,WHG,TANoise,TAAir,VAws,TRbF,VDN) Basicconstruction/versionofthegenerator(stationaryinstallation,containerorhood-typegenerator) Designofthetanksystem(daytankandstoragetank) Designoftheexhaustsystem Enginecooling(front-mountedradiator,tablecooler andtheuseofheatexchangers) Backuppowercontrol/switchingsystems Immissioncontrol
24
Applicable regulations
Theregulationsandlawslistedhereensurethecorrectfunctionofthesystem,operationalreliabilityand protectionoftheenvironment.Approvalauthoritiesmay imposefurtherrestrictionsandrequirements.Inallcases, dialoguewiththeauthoritiesshouldbeinitiatedatan earlystageoftheplanningphase. Noiseprotectionisanespeciallyimportantconsideration. Belowisalistofcontinuousemissionreferencevaluesfor emissionssitesoutsidebuildings. Industrialestate Businesspark Coreregions, villagesandmixed regions Residentialareasand smallhousingestates Purelyresidentialareas Spa/rehabilitationareas forhospitals/carehomes 70dB(A) daytime 65dB(A) daytime 60dB(A) daytime 55dB(A) daytime 50dB(A) daytime 45dB(A) night-time 50dB(A) night-time 45dB(A) night-time 40dB(A) night-time 35dB(A) night-time 35dB(A)
availableinthebuilding,orinstallationinthebuildingis unsuitableforotherreasons.Aswithastationarybuilt-in generator,powercanrangefromafewkVAtomanyMVA. Thethirdtypeisthehood-typegenerator.Thisismostly employedwhenpowerofafewkVAtoseveralhundred kVAisrequired.Itsadvantageliesinitsspace-saving design.Adisadvantageisthedifficultyofaccesstoall systemcomponentsintheeventofserviceorrepair.The diagramsbelowshowstandbygeneratorsinabuilding andinacontainer.
Exhaust gas system
Switchgear
Discharge air section
Machine unit
Batteries
Supply air section
Theresidualnoiselevelisassessedatanappropriate distance,notatthesiteofemission.
Figure3:Standbygeneratorinabuilding
Basic generator construction/version

Therearethreepossiblegeneratorconstructions/versions.Withabuilt-ingenerator,theentiresystemis installedinthebuilding.Interfacestotheoutsidearethe airsupplyandextractionapertures,theexhaustsystem and,ifnecessary,anexternaltablecooler.Inthisversion, powercanrangefromafewkVAtomanyMVA.Container generatorsarefrequentlyusedwheninsufficientspaceis
25
entiretankcontents.Iffuellinesareinplacebetween thedaytankandthestoragetank,theentirelengthof whichcannotbeseen,theymusthavedoublewalls.These double-walledpipes,thecatchbasinsandthejacketof double-walledtanksmustbemonitoredforleaks.The componentsofthetanksystemaredescribedbelow:
Design of the exhaust system

Thenominalwidthoftheexhaustsystemisbasedonthe nominalpoweroftheemergencygenerator,theplanned pipelength,thenumberandtypeofchangesindirection andtherequiredsoundabsorption.Exhaustsystemsof emergencygeneratorsarepressuresystems,andreach temperaturesofupto500C.Theymustberestrictedin suchawayastoexcludeanydangertopeopleandvaluables.Theexhaustsystemisdesignedasdescribedbelow:
Design of the engine cooling

Enginecoolingusingafront-mountedradiatorispossible uptoanoutputrangeofapprox.1150kVA.Thismeansthat allthecoolingairhastobeconveyedthroughthegeneratorroom.Atanoutputofapprox.800kVAandabove, partoftheheatfromtheenginecanbedissipatedviaa tablecooler.Thisreducestheamountofcoolingairthat hastobeconveyedthroughthegeneratorroom.Ifthereis adifferenceinheightbetweenthedieselengineandthe
Figure4:Standbygeneratorinacontainer
tablecoolerinexcessof10m,aheatexchangermustbe usedtoreducethepressureontheenginescoolingcircuit. Thedesignoftheenginecoolingandexternalcoolingsys-
Design of the tank system

Thekeyfactorthatdeterminestanksizeisthenecessary operatingtimeandthesystemoutputpower.Aquantityoffueloflessthan5000litrescanbestoredinthe generatorroom.Ifmorethan5000litresarerequired,a separatestorageroominF90quality,atankforabovegroundstorageoutsidethebuildingoranunderground tankmustbeprovided.Thedaytankisasingle-walled tankwithcatchbasin.Itmustbefittedinsuchaway thatstaticpressureisappliedtotheenginesinjection system.Thestoragetankmusthavedoublewalls,orthe storageroommustbedesignedasacatchbasinforthe
temsbasedonthedifferenceinheightbetweenthediesel engineandthecoolingcircuitisdescribedbelow:
Design of backup power control/switching systems

Eachgeneratingunitmustatleastbeequippedwitha backuppowercontroller,whichassumesthefollowing tasks: Monitoringthemainspowerforcompliancewiththe permittedtolerances Communicationwiththeenginemanagement system Startingandstoppingthedieselengine
26
Monitoringthegeneratornetworkforcompliance withthepermittedtolerances Monitoringtheengineparametersandcontrolling thenecessaryparameters Managingandcontrollingtherequiredauxiliary drives(motor-drivenventilationflaps,supplyand extractorfans,fuelpumps,solenoidvalves,leakage sensors,pipeheaters,coolantpreheating,starterbatterycharging,controlbatterycharging,etc. Managingthenecessarymainsandgeneratortie switchesforautomaticmode Chargingandmonitoringthebattery
5.5 Service/maintenance 5.5.1 Service/maintenanceofUPSsystems

Annualmaintenanceinaccordancewiththe manufacturersinstructionsbyspecialistpersonnel authorisedbythemanufacturerisaprerequisitefor maintainingproperfunction.Wearingpartsmustbe replacedbeforetheirservicelifeexpires,inaccordance withthemanufacturersinstructions. Lessemphasisisplacedonmaintenanceofcommonly
Thefollowingpossibilitiesexistforthepowerunit: Themainsandgeneratorswitchesarelocatedinthe backuppowercontroller Themainsswitchislocatedinthelow-voltagemain distributor,thegeneratorswitchinthebackuppower controller. Themainsandgeneratorswitchesarelocatedinthe low-voltagemaindistributor,thegeneratorpower ismonitoredbymeansofexternalvoltagetaps,and thegeneratorisprotectedbyastar-pointcurrent transformer. Hereisanexamplepowersupplydiagram:
usedsealedleadbatteries,asthesearemaintenance-free. Thetermmaintenance-freereferstotheinteriorofthe battery,however,andmeansthatdistilledwaterdoesnot havetobetoppedup.Ontheotherhand,allconnections andbatteryconnectionscrewshavetobecheckedto ensuretheyhavethecorrecttorque.Thevoltagesofthe individualbatteriesmustberecordedandloggedinthe chargeholdinganddischargingphases.Thecondition ofthebatterycanonlybeevaluatedonthebasisofthis data.Justasimportantisregularcleaningofthebattery system,topreventleakagecurrentorshortcircuits. Theavailabilityofappropriatespecialistpersonnelto remedyproblemsisanothersafetyaspectthatshouldnot beignored.
Switchgear room
Generator room G
U<
5.5.2 Service/maintenance/testrunsof theemergencygenerator

Annualmaintenanceinaccordancewiththe manufacturersinstructionsbyspecialistpersonnel
E Generatorswitch
Mains switch
authorisedbythemanufacturer,andmonthlytestruns, areprerequisitesformaintainingproperfunction.To ensurecorrectfunction,thesemonthlytestrunsmustlast atleastonehourandoperateat50%ofratedload.They maybeperformedbytheplantoperatorhimselfifhe
60%
Load
40%
hasreceivedadequateinstructiononthesystem.During atestrun,thesystemmustreachitsoperatingtemperature.Afixed,installedresistor,theconsumerstobe suppliedbybackuppower,ortheexistingpowersystem,
Figure5:Powersystemmonitoring/switchover
27
viamainsparalleloperation,maybeemployedasaload. Thelatterrequiresagreementandacceptancetestingon thepartofthepowersupplycompany,however.
ofheatwhich,justafewyearsago,wouldhavebeen unimaginableinsucharestrictedspace. Arangeofair-conditioningsystemsisavailableonthe
AswiththeUPSsystem,theavailabilityofappropriate specialistpersonneltoremedyproblemsmustalsobe considered.
market,tosuittheoutputandpowerlossi.e.wasteheat oftheICTcomponents.Withmorethan130W/cmper CPUequivalenttotwostandardlightbulbspersquare metrethehugetaskofdatacentreairconditioningis becomingclear:today,thispowerdensityresultsinheat loadsfarinexcessof1kWpersquaremetre. Theairconditioningofdatacentresgivesrisetofurther
5.5.3 Maintenance/testingofthe electricalinstallation

Electricalsystemsmustbeinspectedandmaintainedat regularintervalsinaccordancewiththeapplicableregulations(VDE0105)andstipulationsoftheprofessional association.Tothisaim,thesystemsmustbeoff-load,and appropriaterepeatmeasurementsandtestsconducted.If necessary,anA/Bsupplymustalreadybeincorporatedin planningoftheinfrastructure.Thisenablesthenecessary isolationandtestingtotakeplace.
challenges.Measurementsandexperiencefromthefield showthatpowerlossesofupto8kWinarackorhousing canstillbetackledwithtraditionalairconditioningusing coolingairintheraisedfloor,stillcommonlyusedinmany datacentres.However,airconductionthroughtheraised floorintheclassicmainframedatacentrecannotalways keeppacewiththesometimeshighdemandsthatprevail today. Afterdecadesduringwhichacoolingcapacityof1to3 kWper19cabinetwasfullyadequate,todayithastobe possibletoconsiderablyincreasethecoolingcapacityper rack.ModernITdevicesina19cabinetwith42height unitscanconsumeover30kWofelectricityandemitover 30kWofheat.Afurtherincreasecanbeanticipateddue todemandsforeverincreasingperformanceinasmaller installationspace.
5.6 Airconditioning 5.6.1 Currentsituation

Eachkilowatt(kW)ofelectricalpowerthatisconsumed byICTdevicesisreleasedasheat.Thisheatmustbe conductedoutthedevice,thecabinetand theroom,inordertokeeptheoperating temperatureconstant.Air-conditioning systemswithdifferentmethodsofoperation andperformanceratingsareemployedto dissipatetheheat.
5.6.2 Thechallengeofair conditioning

TheairconditioningofICTsystemsisacrucialfactorintheiravailabilityandsecurity. Theincreasingintegrationandpackingdensityofprocessorsandcomputer/serversystemsproducesquantities
Figure6:Basicsketchofacoldaisle/hotaislesolution
28
Inordertoimprovetheperformanceofexistingairconditioningsolutionswitharaisedfloor,todayICT componentsareequippedwithapartitionandarranged accordingtotheso-calledcoldaisle/hotaisleprinciple. Sectionsofthecoldorhotaislesareenclosed,toenable moreheatperracktobeemitted. Someofthekeyfactorsinfluencingthechoiceofan air-conditioningsolutionaretheexpectedmaximum powerloss,therunningcosts,purchasecost,installation conditions,costsofexpansion,futureproofness,costof downtimesandphysicalsafety.
Optimumconditionsintermsoftemperatureandrelative humiditycanonlybeachievedwithclosed-circuitair-conditioningunitsso-calledprecisionair-conditioningunits. Inthesesystemstheenergyusedisputtobetteruse,i.e. loweringthetemperatureofthereturnairisofprimary importance.Incontrasttothese,comfortair-conditioning unitsforhomesandofficesexist,suchassplitair-conditioningunits,forexample,whichpermanentlyusealarge proportionoftheenergytodehumidifythecirculating air.Thisgivesrisetocriticalroomconditions,butalsoto considerablyhigherrunningcosts,whichiswhytheuseof thesesystemsindatacentresisuneconomical.
5.6.3 Methodofoperationofclosedcircuitairconditioning
Inthepast,theroomconditionsfordatacentreairconditioningwerearoomtemperatureofapprox.22Cand roughly50%relativehumidity(r.h.).Today,duetothe prevalenceofthecoldaisle/hotaisleprinciple,wetendto referinsteadtosupplyairandexhaustairconditions,as theroomconditionsassuchnolongerapplytoallareas oftheroominquestion. Dependingontheapplication,thesupplyairconditionsin thecoldaisleshouldliebetween20andupto27Cthe relativehumidityofthesupplyairbetween40and60% r.h.Lowerhumiditywouldresultinelectrostaticcharging, highhumiditytocorrosionofelectricalandelectronic components.Itisabasicprerequisitethatoperating conditionswithverycoldtemperaturesbelow18Canda highhumiditythatresultsintheformationofcondensationonITdevicesmustbeavoided. Air-conditioningsystemsinICTroomsoperatewithavery highproportionofcirculatingair.Intheclosed-circuitairconditioningsystem,supplyaircooledbytheair-conditioningsystemcirculatestotheICTcomponentsandabsorbs theheat.Thisheatedairreturnstotheair-conditioning systemforrenewedcooling.Onlyasmallproportionof ambientairisbroughtintotheroom,andthisisusedfor airexchange.
5.6.4Designofclosed-circuitairconditioningsystems
Closed-circuitair-conditioningsystemsdifferconsiderably intheirdesign.Veryroughly,theycanbeclassedasDX systems(DirectExpansionsystems)andCWsystems (ChilledWatersystems).Bothsystemscanadditionallybe equippedwithIndirectFreeCooling.
DX systems
Coldairisgeneratedintheclosed-circuitair-conditioning cabinet,andassuchissituatedinthevicinityoftheICT room.Thecirculatingairiscooleddirectlybyexpanding refrigerant,andtheabsorbedheatisconveyedtoanoutdoorunit,theair-cooledcondenser. WhenIndirectFreeCoolingisused,anadditionalheat exchangerforFreeCoolingisinstalledintheclosed-circuit air-conditioningcabinet.Atlowambienttemperatures,onlyawater/glycolmixturecirculatesbetweenthe closed-circuitair-conditioningunitandtheoutdoorunit, ordrycooler.Thismixtureisheatedbythecirculatingair andcooledagainoutsidebythedrycooler.Thissystemis knownasIndirectFreeCoolingbecausethewater/glycol mixtureisinterconnectedbetweenthecirculatingaircircuitintheICTroomandtheambientair.Whenambient temperaturesarehigh,thesystemswitchestoDXmode, andtheheatofthecirculatingairisfirsttransferredtoa liquid-cooledcondenserintheair-conditioningcabinet,
29
beforetheheatisemittedintotheambientairbythedry cooler. DXsystemswithorwithoutIndirectFreeCoolingare employedforsmalltomedium-sizedinstallationswitha heatloaduptoapprox.500kW.
theflowofairismoreorlessforcedtoabsorbheatfrom theICTcomponentsonitswayfromtheraisedfloorback totheair-conditioningunit. Performanceisincreasedstillfurtherbyenclosingthe coldaisleorthehotaisle.Theheightoftheraisedfloor, viawhichthedatacentrecanbesuppliedwithcoldsupplyair,isalsoanextremelyimportantfactor.Thesupply airisguidedintothecoldaislethroughlouveredpanelsor grilles.AfterithasbeenheatedbytheICTequipment,the returnairisconveyedbacktotheair-conditioningsystem forrenewedcooling.
CW systems
Here,coldisgeneratedinchillers,whicharegenerally installedoutdoors.Awater/glycolmixturecirculatesin thebuilding.Theheatfromthereturnairistransferred tothecoldwater/glycolmixtureintheclosed-circuitairconditioningcabinet,whichiscooledbychilledwater.The heatedwater/glycolmixtureiscooledoncemoreinthe chiller,andreturnstotheclosed-circuitair-conditioning cabinet. Inthissystem,too,IndirectFreeCoolingispossible.To thisaim,anadditionalfreecoolingcoilmustbeinstalled outsideonthechiller.Atlowambienttemperatures, thewater/glycolmixturecirculatesbetweenthechilled water-cooledair-conditioningunitsandthefreecooling coil.Duringthisprocess,theheatofthecirculatingairis absorbedintheair-conditioningunitandemittedoutside bythefreecoolingcoil.Athighambienttemperatures, thewater/glycolmixtureiscooledbythechiller. CWsystemswithorwithoutIndirectFreeCoolingare employedforlargerinstallationswithaheatloadof approx.500kWormore.
5.6.6Directcoolingofracks
Whenheatloadsexceed1015kWperrack,directcooling oftheracksisnecessary.Directrackcoolingisachievedby heatexchangersintheimmediatevicinityoftheservers. Asarule,theseareliquid-cooled,andaresituatedeither belowornexttothe19racks.Upto40kWperrackcanbe dissipatedwiththismethod.
5.6.5 Rackarrangementandcirculatingair conduction

Therackarrangementandairconductionarefurther factorsinfluencingtheperformanceofclosed-circuitairconditioningsystems. Thesedays,19racks,inparticular,arearrangedaccording totheso-calledhotaisle/coldaisleprinciple,inorderto permitaswellaspossibletherequiredhorizontalflowof airaroundtheICTcomponents.Insuchanarrangement,
Figure7:Basicsketchofaliquid-cooledserverrack
30
Achilledwaterinfrastructuremustbeestablishedaround theracksforthispurpose.Liquid-cooledracksensurethe rightclimaticconditionsforeachservercabinet,andare thereforeautonomousintermsofroomairconditioning.
Inexistingbuildingswithalowceilingheight,liquidcooledserverracksrepresentagoodmethodofreliably dissipatinghighheatloadswithouttheuseofaraised floor.
5.6.7 Recommendedequipmentfordifferentdowntimes Data centre category Permitted data centre downtime* Air conditioning Servers cabinet up to 5 kW
Precisioncooling
Servers cabinet from 5 kW to 30 kW

High-performancecooling orliquidcooling Completecold/hotseparation High-performanceorliquidcooling,on-callservice,redundancy requiredforcabinetswithhigh powerdensity. UPSbackupforventilation, Completecold/hotseparation High-performanceorliquidcooling,Redundantarrangement, UPSbackupforventilation, Completecold/hotseparation High-performanceorliquidcooling,Redundantarrangement, UPSbackupforventilation, Completecold/hotseparation
Data centre/ server room 500 to 2500 W/m2

Precisioncooling
72h
B Precisioncooling
24h Precisioncooling, on-callservice
Precisioncoolinginredundant arrangement
Precisioncoolinginredundant arrangement Precisioncoolinginredundant arrangement, UPSbackupforventilation Precisioncoolinginredundant arrangement, UPSbackupforventilation, Emergencycoolingfunctionsviaanadditionalairconditioningsystem(e.g.well water,mainswater,ventilation system)
1h
Precisioncoolinginredundant arrangement,UPSbackup
10min
E Precisioncoolinginredundant arrangement,UPSbackup High-performanceorliquidcooling,Redundantarrangement, UPSbackupforventilation, Completecold/hotseparation
0min
Table7:FromtheBITKOMmatrixPlanningGuideforaReliableDataCentreAirConditioning
31
Thehigh-performancecoolingrequiredathigherpower outputsalsoincorporatestheneedforcoldorhotaisle enclosures.
Asthecoolingcapacityincreases,theair-conditioning systemsmustbeadaptedaccordingly.Thediagrambelow showsthepowerdensitiesthatcancurrentlybehandled bythedifferentair-conditioningsystems.
Airconditioningviatheraisedfloor,racksnotarranged toensurebestpossibleventilation
Airconditioningliquid-cooled withenclosedwarmaisles
Airconditioningviatheraisedfloor,racksarrangedin cold/hotaisles
Airconditioningliquid-cooled withenclosedcoldaisles
Airconditioningviatheraisedfloor, withenclosedcoldaisles
Airconditioningwithliquid-cooledracks (closedsystem)
Figure8:Air-conditioningsolutionsfordifferentcoolingloads
32
5.6.8Furtherrecommendations
Energy efficiency
Asenergycostscontinuetorise,thequestionofenergy efficiencyisofparticularimportanceduringtheplanning phaseofanair-conditioningsystem.Here,theoverall costsi.e.capitalinvestmentinthenewsystemandanticipatedrunningcoststhroughouttherunningtime,plus maintenancecostsmustbecalculatedandevaluated.A systemwithfreecoolinggenerallyentailshighercapital investment,buttheadditionalexpenseisoffsetovera shorttomediumperiodoftimeduetotheconsiderably reducedrunningandmaintenancecosts.
5.6.9Serviceconcept
Inanair-conditioningsystem,wearingparts,e.g.filter matsandsteamcylinders,areused,butsoarenumerous mechanicallymovedcomponents.Therefore,preventive maintenanceatregularintervalsisamust.Thetaskstobe undertakenaredescribedinDIN31051andVDMA24186, amongothers. Suitabletypesofservicecontractsareavailable,dependingonindividualrequirementsforair-conditioning availability.Thesecontractsdifferintermsoftheservices offered: Repaircontract Comesintoforceafterafailure orfault.Systemoperabilityis restoredbydownstreamcorrectiveservicemeasures Regularworkthatensures systemavailabilitythroughpreventiveservicemeasures. Combinationofrepairandservice,whichcombinespreventive andcorrectiveservicework Completemaintenance,with constantcoststhroughoutthe termofthecontractoffering budgetsecurity
Scalability
Inmanydatacentres,ICTsystemsreachtheirmaximum finalstageonlyafterafewyears.Air-conditioningsystemsmustthereforebescalabletoreflectthis.Consequently,achilledwatersystemoperatingatpartialload mayrunwithpoorefficiencyformanyyears,untilthe plannedefficiencyisatleastreachedinthefinalstage. Here,solutionsthatgrowtomatchrequirements,i.e. modularsystems,arealwaysadvantageous. Maintenance contract Fullmaintenance contract Servicecontract
Redundancy
Thesecontractscansometimesalsobecombinedwith Afailureprobabilitymustalwaysbetakenintoaccount, duetothenumerousmechanicalcomponentsinthe air-conditioningsystem.Redundantair-conditioning unitsassumethetaskofgeneratingtherequiredcooling powerintheeventoffailure,andofferalmost100% operationalreliability.Duringfailure,redundancyinthe air-conditioningsystemisgenerallynolongeravailable, andcorrectiveactionmustbeinitiatedwithoutdelay. otherwise,thereisariskthatitwillnolongerbepossible toairconditiontheroomintheeventoffurtherfailure. a24/7emergencyservice,andoffercontractuallyfixed onsitetime.Thisensuresthatcorrectivemeasuresare initiatedimmediately,andtheavailabilityofthesystemis restoredasquicklyaspossible.
33
6 Fire safety
Experienceshowsthatwemustexpecttheoutbreak offireatpracticallyanytime.Thefactthatnofireshave occurredinmanybuildingsfordecadesdoesnotprove thatnoriskexists,butisratherapieceofluckforthose concerned,whichcanbeexpectedtocometoanendat anytime. Eventoday,nothingneedstobeaddedtothispronouncementbyaHigherAdministrativeCourtfrom1987.This iswhyareliable,fastfiredetectionandextinguishing systemorfirepreventionsystemisanindispensableprerequisiteforthereliableoperationofadatacentre. Waterasanextinguishingagentisinappropriateinthe datacentre.Today,specialistfirmsintheindustryoffer suitableextinguishingsystemsforeveryrequirement. Whenconstructinganewdatacentreorretrofittingfire safetymeasures,however,preciseplanninganddesignof thesesystemsisvital. Thechoiceofthemostsuitablefiredetectordepends ontherequirementsoftheintendedsiteofuse.Inareas withoutparticulardetectionrequirements,fixedpoint detectorsaregenerallyused.Indatacentres,ontheother hand,acushionofheatedairupto1metrethickcanform undertheceiling.Thisisproducedashotairfromair-conFire,smoke,aggressivefumesandextinguishingwater representalatentdangerfordatacentres.Toensure safety,atechnicallyhigh-qualityveryearlyfiredetectionsystemcombinedwithveryhigh-qualityextinguishingtechnologyisnecessary.Onealternativeisfire preventionbyloweringtheamountofoxygenintheair. Here,nitrogenisintroduced,whichreducestheoxygen concentrationpreciselytoapresetvalue.Despitethis, roomsprotectedinthiswayremainaccessibletopersons withoutjeopardisingtheirhealth. Extinguishingfoam,ontheotherhand,cannotbeused asitwoulddamageITsystemsandtheirpowersupply units.Powderextinguishingsystemsarealsounsuitable fordatacentreswithverysensitiveequipment,asthe extinguishingprocessmaycausemoredamagethanthe fireitself.Today,therefore,gasextinguishingagentisused almostexclusively. FortheprotectionofventilatedITequipment,theVdS (VerbandderSachversichererGermanAssociationof PropertyInsurers)pointsoutthatearlyfiredetection usingfixedpointsmokedetectorsisdifficult,tosaythe least,orevenimpossible.Here,too,theuseofahighly sensitivesmokeaspiratingsystemisrecommended,as thisoffersthepossibilityofanearly,localisedreaction. ditioningsystems,powerpackfansandrackventilation systemsisdrivenupwards,andmaypreventsmokefrom reachingfixedpointdetectorsinsufficientquantitiesfor earlyfiredetection.Inareassuchasthis,inroomswitha lotofairconditioningandinbuildingswithhighceilings, fixedpointdetectorsquicklycomeupagainsttheirlimits. Wethereforerecommendinsteadtheuseofhighlysensitiveaspiratingsystems. Smokealarmsthatworkontheprincipleofscattered lightareprimarilyusedforfiredetectioninITrooms.Here, thescatterofarayoflightonanysmokeparticlespresent providesameasureofthesmokedensity.Thistechnique isemployedbothinconventional,pointsmokedetectors (opticalsmokedetectors,fixedpointdetectors)andin highlysensitivesmokeextractionsystems(aspirating detectors,activedetectors).Incontrast,theradioactive ionisationdetectorusedyearsagohasnowalmostcompletelydisappearedfromtheEuropeanmarket.
6.1.1 Methodofoperationofthe infrastructure

Fire alarms
6.1 Technicalfireprotection
34
Modernserverracks,whichhavetheirowncoolingsystem andareencapsulatedfromtheenvironment,representa newchallengeforfiresafetyinITrooms.Smokeproduced insidetheracksonlyescapesafteralongdelay,which rendersnormalroommonitoringwithsmokedetectors totallyineffectual.Furthermore,extinguishingagent releasedintheeventofafirecanonlypenetratethe servercabinetafteradelay.Forrackssuchasthis,manufacturersofferfiredetectionandextinguishingsystems forequipmentprotectionintheformof19plug-inunits. Oneofthesedevicescanprotectupto5racks.Forthis application,too,inwhichmajordilutionofthesmokecan beexpectedbecauseofthehighlevelofairconditioning, highlysensitive19aspiratingsystemsarenowusedto anevergreaterextent.Thelatterarealsoavailablewith integratedextinguishingsystem,externalextinguisher activationandintegrateddualdetectorinterdependency. Wheretheactivationofautomaticextinguishingsystems isconcerned,falsealarmsmustberuledoutasfaras possible.Tothisaim,firedetectorsforroommonitoring mustbeinstalledaccordingtotheprincipleofdualdetectorinterdependency;inotherwords,twodetectorsmust soundthealarmbeforeitisescalated.Singledetectors triggeronlytechnical,internalalarms.
AswithveryearlyfiredetectionforITrooms,here,too, veryearlyfiredetectionsystemswithaspiratingdetectorsofferclearadvantages,becauseinthiscasetheair samplescanbetakendirectlyinthecabinetandtheinlets ofthecirculatingairorairconditioning.Thesedays,the extensivemodularityofmodernsystemsmeanstheyare bestequippedforenhancingallstagesoffiredetection andfirefighting.Theyareideallysuitedfornew-generationEDP,serverandcontrolcabinets,whicharesealed withintegratedcoolingsystems.Theytaketheformof 19plug-inunitsforhighlysensitivefiredetection,emit controlsignalsforsoftshutdownandcanindependently extinguishafireonanobject.Anextinguishingunitcan eitherbeintegratedintheunitoractivatedexternally. PreferredextinguishingagentsareNovec1230andnitrogen.Nitrogenhasanadvantageinthatitdistributesitself extremelywellandhasalongerlife,whileNovec1230is verycompactandspace-savingtokeepinstock.Devices in19designareavailableonthemarketthathavebeen approvedbytheVdSfortheirfiredetectionability,and alreadysatisfythelatestAandBsensitivityclassesof DINstandardEN54-20.
Fire extinguishing systems

Theeffectivenessandreliabilityofafiresafetysystem
Forsomeyearsnow,manufacturersoffiredetectionand extinguishingsystemshavebeenproducingdevicesthat arecapableofindependentlymonitoringindividualracks orentirecabinetunitsforfireaerosols,andextinguishing them.Anoutbreakoffireinlivecabinetscanbetriggered byscorching,smoulderingorglowing,oftencausedby overloadedcomponentsandfaultycontacts.Afireona printedcircuitboard,forexample,coverslargeareasof hardwarewithsootandcauseswidespreadcorrosion.A furtherdifficultyarisesthroughthefactthatmodern,airconditionedracksworkwithextremelyhighairchange rates,whichimmediatelydiluteanysmoke,renderingit virtuallyundetectablebyconventionalsmokedetectors whilethefireisinitsinitialstages.
dependsuponplanningthattakesaccountofalltherisks, dimensioningandevaluationoftherequiredvolume. Gasextinguishingsystemsaredividedintoinertgases andchemicalgases.
Inert gases:
Extinguishingfiresbyoxygenremoval Thegasextinguishingmethodissuitablefordatacentres andtheirequipment.Itusesthetechniqueofoxygen removal.Here,theextinguishingagentreducestheproportionofoxygenintheroomairtosuchanextentthat thecombustionprocessissuppressed.Theextinguishing gasusedisaninertorchemicalgas.
35
Carbondioxide(CO2) Carbondioxideispresentintheatmosphere,butisno longerusedinnewsystemsbecauseoftheriskitposesto people. Argon(Ar) Argonisaninertgasandcanbeextractedfromtheambientair.Argonitselfisnottoxic,butcanleadtooxygen deprivationorariskoffireeffluentattheconcentration requiredtoextinguishafire. Nitrogen(N2) Nitrogenisalsocontainedintheatmosphere.Itiscolourless,odourlessandtasteless.Nitrogenisnottoxic,butcan poseathreatduetofireeffluentandoxygendeprivation.
Whatisofoverridingimportanceisthatthegasextinguishingtechnologyoffersthetypeofobjectandroom protectionneeded.Thereisnowaterdamage,andpowder andfoamresiduesarealsoavoided.Floodingtakesplace within120secondsinthecaseofinertgases,andwithin 10secondsinthecaseofchemicalgases.Fireextinguishinggasesarenotelectricallyconductive,andshort circuitsduringorafterextinctionareavoided. Withgasextinguishingsystems,itisimportanttobearin mindthatsecureandreliablepressurereliefviapressure reliefdampersisrequiredtodealwiththeoverpressure thatresultswhenthegasesaretriggered.Theapplicable regulationsstipulatethatthecalculatedmaintained concentrationmustbeupheldfor10minutes.Thepower supplymustbeswitchedoffthroughoutthedatacentre inordertopreventre-ignition.
Fire prevention system

ExtinguishinggasFM-200(HFC227ea),Novec1230 Thisversiondoesnotjustreactintheeventofafire,but Theseextinguishinggasestakeeffectbyabsorbing theheatintheflame.Aphysicalcomponentand,toa lesserextent,achemicalcomponent,areinvolvedinthis process.Thesegasesareapprovedforstorageinextinguishingagentcontainersinadedicatedfireextinguishingzone,undercertainpreconditions,whichmaybea decisivefactorwhenretrofittinganexistingdatacentre withoutafireextinguishingsystem. preventsit.Theoxygenreductioniskeptatanappropriate levelbyveryprecisecontrols.Atthesametime,nitrogen isintroducedintotherooms.Insuchanatmosphere,no openflamescandevelop.Theprotectedareasremain accessibletopeople.
36
6.1.2 Recommendedequipmentfordifferentdowntimes Data centre category Technical fire protection Servers cabinet up to 5 kW Servers cabinet from 5 kW to 30 kW Data centre/ server room 500 to 2500 W/ m2
A B C D E
Monitoringunitwithfiredetection Monitoringunitwithearlyfiredetectionandfire extinguishingtechnology
Monitoringunitwithearlyfiredetection Firealarmsystem,monitoringunit withveryearlyfiredetectionandautonomous fireextinguishingtechnology
Firealarmsystem,monitoringunitwithveryearlyfiredetection andautonomousfireextinguishingtechnology Firealarmsystem,monitoringunit withveryearlyfiredetectionandautonomous fireextinguishingtechnology Firealarmsystem,monitoringunitwithvery earlyfiredetectionandautonomousfireextinguishingtechnology/oxygenreducingsystem
Firealarmsystem,monitoringunitwithveryearlyfiredetectionandautonomousfireextinguishing technology/oxygenreducingsystem,eachinredundantarrangement
Table8fromtheBITKOMmatrixPlanningGuideforaReliableDataCentreTechnicalfireprotection
Data centres
Ifthetolerateddowntimesare24hoursmaximum,detectionandmonitoringunitswithoutdownstreamextinguishingsystemsaresufficient.Ifmaximumdowntimesof lessthanonehourarerequired,adownstreamfireextinguishingsystemwithgasastheextinguishingagent isindispensable.Ifevenshorterornodowntimesdueto firecanbetolerated,firepreventionusingthetechnique ofoxygenreduction,combinedwithactiveveryearlyfire detectionandagasextinguishingsystemwithNovec1230 isapossiblealternative. Thedecisioninfavourofgasextinguishingsystemsorfire preventionsystemscanonlybetakenfollowingthorough analysisbyexperts.Aspectstobetakenintoconsideration whendimensioningandinstallingafireextinguishing systemwithcontrolare: Requirementsforthesystem(s),tolerabledowntimes, etc. Typeofdetectors(fixedpointdetectors,aspirating detectors)
Typeoffireextinguishingsystemandextinguishing agent Possibleplanningofafirepreventionsystem
Server cabinets
Amonitoringunitwithfiredetection(veryearlyfire detection)detectsextremelytinyquantitiesofsmoke aerosolsthatarereleasedintheearliestphaseofemergenceoffire.Thisgainsvaluabletime,whichcanbe utilisedfororganisationalmeasures(automaticphone callers,paging,etc.)andtheinitiationofcountermeasuressuchasasoftshutdown,externaldatabackup, selectiveswitch-offorpossiblytargetedfireextinctionof particularareasorobjects. Intheeventofafire,unitswitch-offisthesafestalternativetopreventfurtherspreadingofthefireorofcorrosive fumes.Modern,softshutdowninnowayimmediately shutsoffthepowersupply,however.Ifsoftshutdown occurs,theveryearlyfiredetectionsystemactivatesan intelligentservermanager,whichquicklydivertsvaluable
37
datatoneighbouringdataorservercabinets.However, thiscanonlybeachievediftheappropriatesoftware/ hardwareenvironmentiscombinedwithahigh-performanceveryearlyfiredetectionsystem.Thedefinitive cut-offofthepowersupplythentakesplacewhenthe datatransferiscomplete.
planningphaseyoumustclarifywhethertheregulations, guidelinesandfiresafetyobjectivesthemselvescan beputintopractice.Theuseofexperiencedplanners isrecommended,becausestructuralandtechnicalfire safetymeasuresmustbebroughtintobalancewith interruption-freedatacentreoperation.Subsequent installationsandconversionsdevourhugesumsofmoney orleadtoaspectacularriseinpremiumsforinsurance relatingtofireandelectronicsystems.
6.2 Structuralfiresafetymeasures
Theaimofstructuralfiresafetymeasuresistosave humanlives.Thisdemandsmaximumqualityofmaterials andworkmanship,andstrictadherencetoregulations andprocedures.
6.2.2 Methodofoperationandroom requirements

Structuralelementsarecategorisedinfireresistanceclassesaccordingtotheirfirebehaviour.Thefireresistance stagesaremostlysetat30,60,90and120minutes.So,F 30means,forexample,thatduringafiretest,atleast30 minutesmustpassbeforethewallgivesway.Thebuilding authoritiesdenotetheseclassesasF60forfire-retardantandF90forfire-resistant. Walls,floorsandceilingsmustbeconstructedwithat leastfireresistanceclassF90.DoorsmustconformtoT90 qualityasaminimum,i.e.doorsmustberesistanttofire for90minutes.Protectionagainstfumesandspraywater isalsoindispensable. Cableandinstallationductsfromandtothedatacentre mustbeeffectivelyprotected.Flame-retardantcable ductscanhaveprotectionwithE30orevenE90quality.InstallationductsmustbeinI30orI90quality,and independentventilationductsinL90.Ifelectriccablesare routedthroughfire-resistantceilingsandwalls,thecable bushingsmustalsobefireandfumeresistantinother words,sealedoff.Thisisolationmayinsomecasesbe achievedbymeansofintumescentfirepillows. Cablerunsconstituteaveryhighriskintheeventofa
Thebasicsofstructuralfiresafetyaresetoutinnational buildingregulations,andprovisionsgoverningtechnicalfiresafetyequipment,firesafetyplans,firewalls andemergencyescaperoutes.Thefirecharacteristics ofbuildingmaterialsandcomponentsisregulatedby DIN4102,whichdoesnot,however,coverthenecessary objectivesoffiresafetymeasures,suchasarevitalforIT datacentres. Furtheraspectstobeconsideredcarefullyarethefire resistanceratingsofload-bearingstructuresandfire protectioninelectricalinstallationsandpowersupply systems.Whenplanningadatacentre,thefireresistance ratingandescaperoutesmustalsobeconsideredwitha viewtofirefighteraccessandsafety.Firefightingliftsand safetystairwellsmustbeincluded.Moreover,datacentres aregovernedbyfiresafetyregulationsaimedspecifically atbusinesses,suchastheregulationsbytheVdS. Firefighting,extinguishingagentsandsmokeextraction mustalsobeincorporatedinplanning.Thiswouldinclude mobilefireextinguishers,thepossibilityofextinguishing agentretentionfeatures,etc.
6.2.1 Firesafetyobjectives
Whenplanningadatacentre,itisoftheutmostimportancetodefinethefiresafetyobjectives.Duringthe
fire,andshouldbecoatedordesignedtobewaterand moistureresistant.Thiswillmakethemintumescent, capableofreliablypreventingfirefromspreadingalong thecables.Thecablesthemselvesshouldconsistof
38
flame-retardantmaterial,whichalsodoesnotproduce aggressivefumes. Firespreadsrapidlyanduncontrollablythrough(flammable)pipes,whicharelaidonandinceilingsandwalls.Pipe shieldingprovidesafire-resistantandfume-proofbarrier. However,simpletestingofstructuralelementsisinno wayadequateforcomplex,high-availabilitydatacentres. Whenhighavailabilityisrequired,theroomsormodular
safetycellstobeconstructedmustundergoastandardisedsystemtesttoEN1047-2,asmustthestructural elementsoftheceiling-wallandwall-floorconnections, thecableentries,overpressurereliefandthedoorsand surroundingareas.TheEuropeanstandardforthedata centreinfrastructurespecifiesboththeintensityandthe durationofpreciselydefinedloads.Theuseristhussafein theknowledgethathisentiresystemnotjustonewall orthedoorisfire-resistant.
6.2.3 Recommendedequipmentfordifferentdowntimes Data centre category Structural fire safety Servers cabinet up to 5 kW Servers cabinet from 5 kW to 30 kW Data centre/ server room 500 to 2500 W/m2
Walls,floors,ceiling:Fireresistanceclassmin.F90, protectionagainstfumesandwater,doorsmin. T90,protectionofequalqualityforcablewalls Walls,floors,ceiling:Fireresistanceclassmin.F90, protectionagainstfumesandwater,doorsmin. T90,protectionofequalqualityforcablewalls Systemtestofstructuralfiresafety Walls,floors,ceiling,doors:asperEuropean StandardEN1047-2,protectionagainstfumesand waterfor60min,protectionofequalqualityfor cablewalls Systemtestofstructuralfiresafety Walls,floors,ceiling,doors:asperEuropean StandardEN1047-2,protectionagainstfumesand waterfor60min,protectionofequalqualityfor cablewalls
Walls,floors,ceiling:Fireresistanceclassmin. F90,protectionagainstfumesandspraywater, doorsmin.T90,protectionofequalqualityfor cablewalls Walls,floors,ceiling:Fireresistanceclassmin. F90,protectionagainstfumesandspraywater, doorsmin.T90,protectionofequalqualityfor cablewalls Systemtestofstructuralfiresafety Walls,floors,ceiling,doors:asperEuropean StandardEN1047-2,protectionagainstfumes andspraywaterfor60min,protectionofequal qualityforcablewalls Systemtestofstructuralfiresafety Walls,floors,ceiling,doors:asperEuropean StandardEN1047-2,protectionagainstfumes andspraywaterfor60min,protectionofequal qualityforcablewalls
72h
24h
1h
10min
Systemtestofstructuralfiresafety Walls,floors,ceiling,doors:asperEuropeanStandardEN1047-2,protectionagainstfumes andspraywaterfor60min,protectionofequalqualityforcablewalls
0min
Table9:FromtheBITKOMmatrixPlanningGuideforaReliableDataCentreStructuralfiresafetymeasures
39
6.2.4Specialfeatures
Aspectstobetakenintoconsiderationduringproject planningare: Determinationoffiresafetyobjectives,under considerationofthespecialrequirementsoftheIT infrastructure Determinationofstructuralcharacteristics Planningtheconstructionbyprofessionalplanners, ifpossible Compilationofspecificationsfortheindividualelementstobeofferedfortender Collectionofincomingquotations,comparison, evaluation Drawingupacontract-awardingproposalforthe decision-makers
40
7 Design of premises and safety zones for data centres

Securityofinformationtechnologyisasweepingterm thatcoverslogisticaldatasecurity,thephysicalsafetyof systemsandtheorganisationalreliabilityoftheprocesses.Theobjectiveofacomprehensivesecurityconcept istotakeallareasintoconsideration,torecogniseand assessrisksearlyonandtoinitiatemeasuresinsucha waythatacompanyscompetitivenessonthemarketis notjeopardised. ThroughanoverallexaminationoftheITinfrastructure andvariousfunctionalareasoftheIT,awellthought-out plancanreduceoreveneliminatemajorriskstophysical safety.Adecisiveroleisplayedbythepremisesinwhich theITislocatedontheonehand,andthespatialarrangementofthevariousfunctionsinrelationtooneanother ontheotherhand. Whendesigningandplanningadatacentre,thevariousfunctionalareasarearrangedinaccordancewith therequirementsfortheirsafetyandsecurity,andtheir importanceformaintainingthefunctionoftheinformationtechnology. Ifallriskfactorsandtheconstraintsspecifictothe companyaretakenintoconsideration,potentialdanger andtheresultingtimeandexpensecanbeeradicatedin advanceduringthedesignoftheITinfrastructure. Aprotectedlocationtoensureprotectionagainst sabotage Assessmentofpotentialthreatsonthegroundsofthe companyssocialpolicy
Structure of a data centre
Location of IT rooms
Thevariousfunctionalareascanbesetoutasfollows: ThedesignofanITinfrastructureandthustheselection ofasiteforadatacentremustbebasedonthecompanys datasecurityprinciples,whichreflectavailabilityrequirementsandthecompanysstrategicdirection. Thefollowingcriteriashouldbetakenintoconsideration whenexaminingthephysicalsafetyofasite: Lowriskpotentialfromneighbouringusage,adjacent partsofthebuildingorfunctions Avoidanceofrisksfrommediaandpowersupplylines, tremorsandchemicals,whichwouldcompromisethe physicalsafetyoftheITsystems Avoidanceofpossibledangersposedbyelementary risks(water,gales,lightningstrikes,earthquakes),risk assessmentofaspectspeculiartotheregion Thedatacentreasaseparate,independentfunctional zone
Table10:Functionalareasofadatacentre
Safety zones
1 2 3 4 5
Function
Plot Semi-publicarea, neighbouringoffices
Identification (example)
White Green
Operatingareas,rooms Yellow adjoiningIT Technicalsystemsfor operatingIT ITandnetwork infrastructure Blue Red
41
Layout of safety zones

Aschematicrepresentationofthevarioussafetyzones producessomethingliketheexampleillustratedhere:the ITarea(red)issituatedintheinteriorandisprotectedby theadjacentzones3and4(yellow/blue).Safetyzones1 and2(white/green)formtheouterlayers.Theindividual safetyzonesaredividedbysafetylines. Thesafetylinesrepresentthemonitored,safetransition fromonezonetoanother,andaresetupinconformity withthecompanyssafetyrequirements. Thelocationsofthedifferentfunctionalareasandthe Afittingsolutionforavoidingpossiblesabotageisto separatethefunctionalareasbyensuringlimitedaccess tosensitiveareas.Forexample,amaintenancetechnician fortheair-conditioningsystemsorUPShasaccessonlyto thetechnicalzones(blue),nottothecompanysITzone (red). divisionofthesafetyzonesorsafetylinesareimportantforguaranteeingthesafetyoftheITinfrastructure. However,continuousITavailabilitycanonlybeachievedthroughacomprehensivesafetyplan,whichtakes accountofallaspectsofITsecurityandsafety.
42
8 Wiring
8.1 Currentsituation
Theprimary,originaltaskofdatacentresistorunITapplicationsonmainframesandservers,tomaintaindataand saveitonstoragesystems. FromanITpointofview,theprincipalrequirementis availability,i.e.theabilityofITapplicationsthataregenerallycriticalforcompaniestoremainoperationalpossibly withoutanyinterruption.ThesetypicallyincludeERP systems,productionapplicationsinindustrialcompanies, databases,officeapplicationsandtheiroperatingsystems,andalsoaccesstoprovidernetworks(MAN,WAN) andtotheinternet. TheISO-OSI7LayerReferenceModelappliestotheIT, andthisdefinestheapplicationastheuppermostlayer, andthephysicalinfrastructurerequiredfortransporting thedata,theITwiringanddatatransportdevicesasthe lowest,orfirst,layer,e.g.layer1switches. Requirementsformaximumavailabilityandeverfaster Therefore,theITwiringisoffundamentalimportancefor theavailabilityofITapplicationsinadatacentre:without functioningITwiring,ITdevicessuchasservers,switches andstoragedevicescannotcommunicatewitheachother orexchange,process,holdorsavedata. Itisfrequentlythecase,however,thatITwiringhassimply grownovertime,andonlysatisfieswithdifficultycurrent requirementssuchas highductdensities hightransmissionspeeds interruption-freehardwaremodifications servicesupport ventilationissues Whetherfibre-opticorcoppercablesareused,theIT ThestructuringofITwiringandmeticulous,anticipatory planningarethereforekeytasksforadatacentreoperator.LegalprinciplessuchasBaselIIorSOXalsodemand rigorousandcomprehensivetransparency. Preassembledsystemshavethebestpossible,reproduciblequality,andthereforepromiseverygoodtransmission wiringcomponentsmaytaketheformoffactory-assembled,turnkeysystemsforplug&playinstallations. datatransmissionratesmeanthatthequalityrequirementsfacingITwiringcomponentsfordatacentresare manytimesmoreexactingthanthoseforproductsused inLANs.Whenitcomestoselectingsystems,quality principlesshouldalreadybeincorporatedintheveryearly planningphase,withaviewtosatisfyingperformance requirementssuchas Cabledesignforcopperandfibre-optics Bandwidthsforcoppersystemsandfibre-opticcables Insertionandreturnlossbudgetsforfibre-optics EMCimmunityforcoppersystems Updatecapabilitytohigherspeedclasses 19cabinetdesign Theplanning,installationandacceptancetestingofIT wiringindatacentresisdescribedintheDINEN50174 seriesofstandards.Importantcontentsincludethequalityplan,potentialequalisation,safetydistancesbetween copperITwiringandotherelectricalsources,andthe documentationandacceptancetestingoftheentiredata centre.
8.2 Basicstandards
WhatallstandardstoISO/IEC,ENandTIAhaveincommonisthattheydemandorstipulatestructured,application-neutralITwiring.Inaddition,theyexpressclear recommendationsforsettingupredundantITwiring,for ensuringhigh-levelreliabilityofthedatacentre.
8.3 Quality/selectionofcomponents/ systems
43
characteristicsandhighreliability.Onlyscreenedsystems maybeusedincoppersystems,duetothedemanding availabilityrequirements.Globalstandardsrequirecopper wiringwithatleastclassEA.
ITwiringstructurethatisclearlyarranged,transparent andisolatedfromthesurroundingdevicefleetishighly recommended. Itshouldconnecttherespectivedevicelocationstoa
Sufficientprioritymustalsobelenttotheselectionof suppliersofITwiring.Besidesthequalityofthewiring components,thechiefrequirementsforareliablesupplier arespecialistdatacentreexpertise,experiencewithdata centreITwiringandlastingabilitytosupply.Ideally,the suppliershouldalsooffercomprehensiveplanning,installationandmaintenanceservices.
uniform,consistentITwiringstructure. StandardsDINEN50173-5andISO/IEC24764dividethis equipmentwiringintothesegmentsareaanddevice connectionwiring,withthedeviceconnectioninterfacein between.Theactivedevicesareconnectedtothedeviceneutralareawiringviathedeviceconnectioninterface bymeansofdevice-specificconnectingcablesthatareas shortaspossible.Consequently,whenadeviceisreplaced whichoftenrequiresreplacingtheconnectionfaceon thedeviceonlythecablespecifictothisconnectionhas tobereplacedwithnoneedtomodifyorstripthearea wiring. Areaswithahighpackingdensityareworthyofparticular attentioninthiscontext.
8.4 Structure
Datacentresarethecompanyscentralnervoussystem. Theyarethereforesubjecttocontinuouschange,driven bytheshortlifecyclesoftheactivecomponents.Inorder toavoidfundamentalorfar-reachingchangestotheIT wiringwitheverynewpieceofequipment,aphysical
Office
Interface to external netwrok
Telecommunication room
Main distributor
Area distributor Local distributor Device connection
Area distributor
Area distributor
Device connection
Device connection Data Center...
backbone wiring
horizontal wiring
Figure9:SchematicdiagramtoEN50173-5/TIA942,wiringlayout
44
Theabovetechniqueensuresthatanyrewiringinvolved duringequipmentreplacementsarereducedtoaminimumbothfinanciallyandintermsoftimewhilethe definedstructureisfullypreserved. Wherenecessary,thehorizontalwiringmustbemadeof copperandfibre-opticcables,inorderthatvariousdevices canbeconnected.Thebackbonewiringshouldbeinfibreopticandcoppercables,andofferredundancy.
Suitableplug-insystemsmustbeselectedforthedevice connectioninterfaces,inaccordancewiththerequired packingdensityoftheconnecteddevices.Suitableplug-in systemsarenamedinstandardsDINEN50173-5andISO/ IEC24764.
8.5 Redundancyandsecurity
Therequirementforhighavailabilitynecessitatesredundancyofconnectionsandcomponents:Thus,itmustbe possibletoreplacehardwareduringongoingoperation, andanalternativecomponentmusttakeontherunning oftheapplicationwithoutinterruptionintheeventof cablefailure. Itisthereforeobviousthatanappropriate,all-inclusive ITwiringplatformmustbeprovidedthatensuresthe correctbendingradii,safeguardsperformanceandcanbe fittedquicklyandreliablyduringoperation.
Figure10:Representationofhorizontalwiring(Cuandfibre-optic)with areadistributorandserver/storagecabinetswithdeviceconnection
Theavailabilityofapplicationscanbeincreasedthrough theuseofpreassembledITwiringsystems.Thisreduces thetimethatinstallationpersonnelhavetospendinthe safetyzoneofthedatacentretoaminimum,bothduring initialinstallationandeventualhardwaremodifications, andalsopromisesadditionaloperationalreliability. Inaddition,caremustbetakentoensurethatallproductsareinspectedanddocumentedaspartofquality management. Forthemutualconnectionofdatacentres,e.g.redundant datacentres,backupdatacentres,orsimplyforbacking upandsavingdataatadifferentsite,theincorporation andsecurityofMANandWANprovidernetworks(data transportservicesorso-calleddarkfibres)ordedicated fibre-opticcablerunsisofenormousimportancewhere
Figure11:Representationofbackbonewiring(fibre-optic)withmain distributorandconnectiontohorizontalwiring(Cuandfibre-optic)with areadistributorandserver/storagecabinetswithdeviceconnection
reliabilityandavailabilityareconcernedand,liketheIT wiringinsidethedatacentre,mustberedundant.
45
8.6 Installation
Engineersandtechniciansmustbetrainedinthespecificationsofthesystemsinordertoensurethesafe,reliable operationoffibre-opticITwiringinthedatacentre, particularlyinthematterofinstallationandworkwith patches.Duetotherequiredwiring,whenselectingthe 19serverorITwiringcabinetandwithreferencetoitem 4.1Secureservercabinets,theuseofcabinetsystems atleast800mmwideisrecommended.Theseenablethe installationofanintegralcablemanagementsystemin theverticalandhorizontaldirection.Asarule,thedepth ofthecabinetisdeterminedbythepassiveandactive componentstobeinstalledinside.Cabinetsystemswitha minimumdepthof800mmhavealsoproventheirworth forpassivedistributors.Adepthof1000to1200mmis recommendedforcabinetsystemsdestinedtoaccommodateactivecomponents. Thepossibleadvantageoffactory-preassembledITwiring systemsalreadymentionedinthesectiononsafety principlescomestotheforeduringinstallationinthe formoftimesavings.Whenthesesystemsareemployed, itisworthmentioningthatwhendatacentrecapacity isexpandedduetoanincreaseinITequipment,these devicesandhencetheITapplicationsthemselvescanbe wiredtooneanotherandbroughtintooperationatthe greatestpossiblespeedandthesameappliestohardwaremodifications.
8.7 Documentationandlabelling
Painstaking,continuallyupdateddocumentationisan importantmeansofensuringthesimplemanagement ofITwiringandreliableplanningofconversionsand upgrades.Alargerangeofpossibilitiesexistinthis respect,fromindividualExcelworksheetstoproven, software-baseddocumentationtools.Whatisimportant isthatthedocumentationisalwayskeptup-to-date,and reflectstheactual,installedITwiring.Thechoiceoftoolis attheusersdiscretion. Closelyconnectedtodocumentationistheclearand eveninlimitedlighteasilylegiblelabellingofcables. Here,too,numerousidentificationmethodsareavailable, e.g.fromcabletagswithreplaceablelabelstobarcode labels.Thetypeselectedwilldependonindividual requirements.Itisimportanttoensurethatidentical nomenclatureisusedthroughoutthecompany.Central managementofthedataisrecommendedtoensureclear, unambiguouslabellingofcables.
46
9 Certification of a reliable data centre

9.1 Themanagementsystem 9.1.1 Thedesignofamanagementsystem
Forallcompanies,customerorientation,costefficiency andcompetitivenessareintheforeground.Decisivefactorsareresourceplanning,faultpreventionandaprocessorientedapproachtotheperformanceofservices. InternationalstandardssuchasISO/IEC27001:2005,etc. provideconcreteassistanceandastructureforimplementingamanagementsystem,andcanbecertifiedby neutralassociations. Amanagementsystemformsthebasisfor processorientation continuousimprovementofbusinessprocesses(CIP) greatercustomerandsupplierorientation interpretationofinteractionsbetweenbusiness processes faultprevention(preventiveandcorrectivemeasures), and compliancewithlegalrequirements Examplesofsynergiesarethemergingofthefollowing elements Corporatepolicyandgoals Managementreview Existingdocumentation(assignmentofnames,revision,approval,etc.) Internalaudits Evaluationoferrorfrequencyandsources Evaluationofcustomerandemployeesatisfaction Evaluationofsuppliers Procurementprocess Asuccessfulmanagementsystembringsthecompany decisiveadvantages: Businessprocessesbecometraceable,productsand servicesareimproved Weakpointsinthesystemareflaggedup Risksarerecognised,analysed,minimisedoraverted Thecompanyispreparedforemergencies Continuousmonitoringlowerstheerrorrateand thereforeimprovestheservicesprovided.Thisresults inreducedcostsandhigherquality Oncethequalitymanagementsystemhasbeendocumentedandeffectivelyintroducedinthecompany,itcan becertifiedbyanindependent,neutralcompanythatis authorisedtoperformcertification.Firstofall,thecertificationcompanyinspectsthedocumentationandthen thesystemonsite.Theauditorpossessesthenecessary TheuniformstructureofISOmanagementsystemsbasicallypermitsseveralstandardstobecombined.Ifsucha combinationispossible,itwillbeeasytoincorporatenew standardsandexploitsynergisticeffectsindifferentareas ofthecompany. Employeesaremotivatedbythecleardelineationof responsibilities Thedocumentationofprocessescreatestransparency Customerstrustinthecompanysabilitytodeliver qualityisboosted Thecompanysimageisenhanced Competitivenessisstrengthened Liabilityisminimised
9.1.3 Combinationofdifferentstandards
9.1.2 Advantagesofamanagement system
9.2 Certificationofamanagement system
47
qualificationsandprofessionalexperience.Apositive resultleadstocertification,whichisgenerallyvalidfor3 years. Afterthecertificationaudit,themanagementsystemis checkedforitseffectivenessinannualmonitoringaudits.
Performing a preliminary audit

Theaimofthepreliminaryauditistoascertainwhether thebasicprerequisitesforcertificationofthequality managementsystemaremet.Itdetermineswhetherthe certificationauditcanbecarriedoutontheplanneddate withsuccessasthelikelyoutcome.
9.2.1 Theadvantagesofcertification
Certificationisneutralproofoftheeffectivenessof themanagementsystem,andcanofferthefollowing advantages: Acquisitionofnewcustomers,openingthedoorto newmarkets Strengtheningcompetitiveness Fostersthecustomerstrustinthecompany Improvesthecompanysrankingandcreditworthiness Reducesthetimeandcostofdemonstratingquality Internationalrecognitionandacceptance
Alternatively,thecheckperformedduringthepreliminary auditmayincorporatetheinspectionoftheQMmanual andassociateddocuments.Thepreliminaryauditis basicallyarandomsampletest,anddoesnotclaimtobe complete.
The certification process

Duringthecertificationaudit,theauditorscheckwhether thedocumentedproceduresandroutinessatisfythe requirementsoftherelevantrules,andwhethertheprocessesandagreementsdefinedbythecompanymatch theQMdocumentation.
9.2.1 Atypicalcertificationprocedure
Aninformativediscussionshouldbeheldbeforeacertificationcompanyisselected.
Thecertificationauditconsistsofacomprehensiveassessment.Indetail,itcoversinspectionofthefollowing: QMsystemdocumentation Structuralorganisation Implementationofthedocumentedprocessesand theireffectsinthecompany First-timecertificationtakestheformofa2-stageprocess. Instage1,importantkeyelementsarecheckedandan assessmentascertainstowhatextentthecompanyis preparedforcertification.ThecompanysQMdocumentationisalsocheckedforconformitytostandardsatthis point. Instage2,theimplementationandeffectivenessof themanagementsystemthroughoutthecompanyis checked.Ifdeviationsaredisclosedduringtheassessment ofthequalitymanagementsystem,apostauditmay
The informative discussion

Thesubjectsunderdiscussionbasicallyincludequestions aboutcertificationandauditing,organisationalprocedure (e.g.scopeandschedule)andcosts.
The certification order

Byplacinganorderforcertification,thecontracting companyplacesitselfunderobligationtoprovidethe certificationcompanywiththenecessaryQMdocumentation.Alternatively,theQMdocumentationmayalsobe inspectedonsite.Apreliminaryauditmaybeconducted, ifthecompanysodesires.
48
needtobeperformed.Thecertificateisthenonlygranted afterallsuchdeviationshavebeeneliminatedandwhen thenegativefindingsfallbelowthemaximumnumber recommendedbytheauditor. Theresultisdocumentedinanauditreport.Thisisthe basisuponwhichthecertificationcommitteeofthe certificationcompanydecideswhetherornottogranta certificate.
anotherthreeyears.Therecertificationauditmustbe carriedoutbeforethecertificatesvalidityexpires.The requirementsoftherecertificationauditarevirtually identicaltothoseofthecertificationaudit.Here,however, theauditmaybeperformedinasinglestage.
9.2.3 Selectingtherightcertification partner

Theselectionoftherightcertificationpartnerisdecisive forthesuccessoftheprocess.Aswithanyservice,aprice marginapplies.Itisthereforeadvisabletoobtainseveral quotations,ortoextendtheexistingsystemwithatrustedcertificationpartnerwhoisalreadyfamiliarwiththe system. Incertaincases,thecertificationcompanysinternational organisationmayconstituteamajorcostfactorif,for example,companysitesinothercountriesaretobeincludedinthecertificationprocess. Auditorswithdifferentcertificationbodiesarequalified toasimilarlyhighlevel,astheapprovalofauditsisregulatedcentrallybyaccreditationbodiesandISOstandards.
The monitoring audit

Annualmonitoringauditstakeplaceduringthethreeyearperiodofvalidityofthecertificate. The1stand2ndmonitoringauditsconstitutearandom samplecheckastowhether: Negativefinding(s)fromthepreviousaudithas/have beenremedied Organisationalchangeshavetakenplaceinthe company TheQMsystemhaschanged Thecertificateandcertificationlogoarebeingused correctly Recentchangestorelevantstandards,lawsandregulationshavebeentakenintoconsideration Themanagementsystemcontinuestosatisfythe requirements,and Thequalitymanagementsystemhasbeenadheredto andcontinuestobeeffective. Iftheresultofthemonitoringauditsissuccessful,a completeinspectionoftheQMsystemwilltakeplace inanewprocessinthreeyearstime,beginningwitha recertificationaudit.
The recertification audit

Arecertificationauditisnecessaryforensuringaseamlessextensionoftheperiodofvalidityofacertificateby
49
10 Annex
Selectionofimportantrulesandregulations DIN6280,Part1-15 Part1 Part2 Part3 Part4 Part5 Generatingsetswithreciprocatinginternalcombustionengines Generaldefinitions Powerratingsandpowerratingplates Limitingvaluesfortheoperatingbehaviouroftheengine,thegeneratorandthegeneratingset Speedgoverningandspeedbehaviourofreciprocatinginternalcombustionengines;definitions Operationalbehaviourrelatingtosynchronousalternators forgeneratingsets Part6 Operationalbehaviourrelatingtoasynchronousalternators forgeneratingsets Part7 Part8 Part9 Part10 Part11 Controllingandgoverningequipmentforgeneratoroperation Operationalbehaviourrelatingtogeneratingsets;definitions Acceptancetest Smallpowergeneratingsets;requirementsandtests Measurementandassessmentofmechanicalvibrationsingeneratingsetswithreciprocatinginternal combustionengines Part12 GeneratingsetsUninterruptiblepowersupplyDynamicUPSsystemswithandwithoutreciprocatinginternalcombustionengine Part13 GeneratingsetsGeneratingsetswithreciprocatinginternalcombustionengineforemergency powersupplyinhospitalsandpublicbuildings Part14 Combinedheatandpowersystem(CHPS)withreciprocatinginternalcombustionengineBasics, requirements,componentsandapplication Part15 Combinedheatandpowersystem(CHPS)withreciprocatinginternalcombustionengineTests
50
ISO8528
Reciprocatinginternalcombustionenginedrivenalternatingcurrentgeneratingsets.
GermanFederalControlofPollutionAct 4. Regulationontheimplementation oftheFederalControlofPollutionAct 9. Regulationontheimplementation oftheFederalControlofPollutionActTA TA DIN/VDE0107 Supplement1 Supplement2 DIN/VDE0108 Supplement1 Part2 Part3 Part4 Part5 Part6 Part7 Part8 DIN/VDE0100 Part728 AirGermanTechnicalGuidelinesonAirQualityControl NoiseGermanTechnicalGuidelinesonNoisePrevention Electricalinstallationsinhospitalsandlocationsformedicaluseoutsidehospitals Extractsfrombuildingandoccupationalhealthandsafetyregulations Interpretation,explanation Powerinstallationsandsafetypowersupplyincommunalfacilities Buildingregulations Placesofassembly Stores,shopsandexhibitionrooms Multi-storeybuildings Restaurants Closedcarparks Workingpremises Temporarystructures,etc. Standbypowerinstallations Basicprinciplesoftheapprovalprocedure Regulationonplantsandsystemsrequiringapproval.
51
PSC VDEW VDEW EltBauVO VDS WHG
ConnectionconditionsofPSCs GuidelinesforemergencygeneratingsetsbytheGermanelectricityassociation(VDEW) Paralleloperationwiththelow-voltagenetworkbytheGermanelectricityassociation(VDEW) Ordinancegoverningtheconstructionofoperatingroomsforelectricalinstallations RegulationsfromtheGermanAssociationofPropertyInsurers GermanWaterResourcesAct
FuelOilTaxationAct (operationofstationarysystemswithfueloil) DIN31051 Maintenance
52
11
Glossary
19cabinet Rackwithapproximately40heightunits,overallheightapproximately2metres,installation width483mm,installationheightismeasuredinheightunits(HU),1HU=44.45mm CW Datacentre DX Emission EMC Immission IT Modular Paralleloperation PDU ChilledWater;air-conditioningsystemswithchilledwater Serverroomand/ordatacentre DirecteXpansion;air-conditioningsystemswithrefrigerant Influencesemittedbyadevicethataffecttheenvironment ElectroMagneticCompatibility Influencesoriginatingfromtheenvironmentthataffectaparticularlocation InformationTechnology(formerlyEDPElectronicDataProcessing) Describesasystemthatisconstructedofseveralmodules(assemblies) Twoormoreinstallationsjointlysupplyconnectedconsumers PowerDistributionUnitorlow-voltagedistributionsystem
Precisionair-conditioning Air-conditioningsystemthatisabletokeepboththetemperatureandtheairhumidityconssystem tant.TheparametersoftheairattheinletopeningsoftheITunitsshouldbebetween22and 27Cand40and60%r.h. PSC Redundant Scalable SG UPS PowerSupplyCompany Duplicatearrangementtoincreaseavailability(errortolerance) Step-by-stepadaptabilitytorequirements Standbygenerator(mostlyanemergencydieselgenerator) UninterruptiblePowerSupply
53
12 Acknowledgements
ThisReliableDataCentreguidewascompiledinconsultationwiththeBITKOMDataCentre&ITInfrastructure workinggroup. Wewishtoexpressoursincerethankstoallmembers oftheworkinggroupforthevaluablediscussionswe hadwiththem,andourparticularthanksgotothe followingfortheirinvolvement: SilviaBader(DEKRAcertificationGmbH) HaraldBecker(Rosenberger-OSIGmbH&Co.OHG) Dr.GeraldBerg(Rosenberger-OSIGmbH&Co.OHG) KlausClasen(NotstromtechnikClasenGmbH) PeterClauss(WagnerGroupGmbH) AykutGven(DEKRAcertificationGmbH) FrankHauser(ServerTechnologyInternational) DieterHenze(RittalGmbH&Co.KG) Dr.SiegbertHopf(MasterguardGmbH) KnutKrabbes(QMKIT-Security+Quality) StephanLang(WeissKlimatechnikGmbH) HelmutMuhm(Dipl.-Ing.W.BenderGmbH&Co.KG) Hans-JrgenNiethammer(TycoElectronicsAMP GmbH) TorstenPed(NotstromtechnikClasenGmbH) AchimPfleiderer(StulzGmbH) ThorstenPunke(TycoElectronicsAMPGmbH) ZeynepSakalli(euromicronsolutionsGmbH) MichaelSchumacher(APCDeutschlandGmbH) KarlheinzVolkert(OrangeBusinessGermanyGmbH) PeterWsch,SCHFERAusstattungs-SystemeGmbH RalphWlpert(RittalGmbH&Co.KG) ThefollowingpeopleassistedwithVersion1fromNovember2006: DieterHenze(RittalGmbH&Co.KG) SiegbertHopf(MasterguardGmbH) PeterKoch(KnrrAG),HelmutGhl(O2GmbH) KnutKrabbes(QMKIT-Security+Quality) MatthiasLohmann(TVSecure) IngoLojewski(EmersonNetworkPowerGmbH) AchimPfleiderer(StulzGmbH) JrgRichter(I.T.E.N.O.SGmbH) HarrySchnabel(HarrySchnabelConsult) MichaelSchumacher(APCDeutschlandGmbH) JrgenStrate(IBMDeutschlandGmbH) JudithWagener(BullGmbH) RalphWlpert(LampertzGmbH&Co.KG) EckhardWolf(AEGPowerSolutionsGmbH) SandraSchulz(Giesecke&DevrientGmbH)
54
TheFederalAssociationforInformationTechnology,TelecommunicationsandNewMedia(BITKOM)represents morethan1,300companiesinGermany.Its950directmembersgenerateasalesvolumeof135billioneuros annuallyandemploy700,000people.Theyincludeprovidersofsoftware,ITandtelecommunicationservices, manufacturersofhardwareandconsumerelectronicsaswellasdigitalmediabusinesses.BITKOMisworking, inparticular,toimprovetheregulatoryframeworkinGermany,formodernizationoftheeducationsystemand foraneconomicpolicywhichencouragesinnovation.
Thistranslationwaskindlysponsoredbythefollowingcompanies:
BundesverbandInformationswirtschaft, TelekommunikationundneueMediene.V. Albrechtstrae10A 10117Berlin-Mitte Tel.:03o.27576-0 Fax:030.27576-400 bitkom@bitkom.org www.bitkom.org

Reliable Data Centres: Guide

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Reliable Data Centres: Guide

Uploaded by

Copyright:

Available Formats

Reliable Data Centres