Deploying and Managing Private Clouds

The Essentials Series

Tips and Best Practices for Managing a Private Cloud

sponsored by

Dan Sullivan

TheEssentialsSeries:DeployingandManagingPrivateClouds

DanSullivan

TipsandBestPracticesforManagingaPrivateCloud.............................................................................1 EstablishingPoliciesandProcedures.........................................................................................................1 CostAllocationandReporting..................................................................................................................2 ImageManagement.......................................................................................................................................2 . SecurityandPatchManagement..............................................................................................................3 BackupandDisasterRecovery.................................................................................................................4 StandardizingHardwareandApplicationStacks..................................................................................4 FormalizeDiscoveryandMonitoringProcedures................................................................................5 Summary.................................................................................................................................................................6

TheEssentialsSeries:DeployingandManagingPrivateClouds

DanSullivan

Copyright Statement
2011 Realtime Publishers. All rights reserved. This site contains materials that have been created, developed, or commissioned by, and published with the permission of, Realtime Publishers (the Materials) and this site and any such Materials are protected by international copyright and trademark laws. THE MATERIALS ARE PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. The Materials are subject to change without notice and do not represent a commitment on the part of Realtime Publishers its web site sponsors. In no event shall Realtime Publishers or its web site sponsors be held liable for technical or editorial errors or omissions contained in the Materials, including without limitation, for any direct, indirect, incidental, special, exemplary or consequential damages whatsoever resulting from the use of any information contained in the Materials. The Materials (including but not limited to the text, images, audio, and/or video) may not be copied, reproduced, republished, uploaded, posted, transmitted, or distributed in any way, in whole or in part, except that one copy may be downloaded for your personal, noncommercial use on a single computer. In connection with such use, you may not modify or obscure any copyright or other proprietary notice. The Materials may contain trademarks, services marks and logos that are the property of third parties. You are not permitted to use these trademarks, services marks or logos without prior written consent of such third parties. Realtime Publishers and the Realtime Publishers logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. If you have any questions about these terms, or if you would like information about licensing materials from Realtime Publishers, please contact us via e-mail at info@realtimepublishers.com.

ii

TheEssentialsSeries:DeployingandManagingPrivateClouds

DanSullivan

TipsandBestPracticesforManaginga PrivateCloud
Privatecloudsarearelativelynewmodelfordeliveringcomputingandstorageservices, butthismodelbuildsonalonghistoryofITinfrastructuremanagement.Privatecloudsare adeliverymodelthatbuildsonwellestablishedITpractices,suchasvirtualization, networkmanagement,systemsadministration,andoperationsmanagement.These practiceshavedevelopedoveryearsofrepeateduseandrefinementinawidevarietyof applicationareas.Wedrawfromthesepracticeshereandhighlightthreeareasthatare especiallyapplicabletoprivatecloudmanagement: Establishingpoliciesandprocedures Standardizinghardwareandapplicationstacks Formalizingdiscoveryandmonitoringprocedures

Together,thesehelptoestablishasustainablemanagementframeworkthatpromotesthe efficientuseofcloudresourceswithoutcreatingunnecessarymanagementburdensforIT staff.

EstablishingPoliciesandProcedures
Thefirstsetoftipsandbestpracticesisnotaboutsomearcanetechnologythatenables cloudcomputingbutisinsteadaboutmanagementpractices.Inmanyways,thebest hardwareintheoptimalconfigurationwillonlycontinuetoperformwellforsolongbefore changesindemands,hardwarefailures,andsoftwarerevisionsstarttoadverselyimpact operations.Privatecloudsrequireaminimalsetofoperatingpoliciesandproceduresthat areimplementedbyautomatedsystemsandsupportstafftoensuretheprivatecloud continuestodelivercomputing,storage,andnetworkingservices. Someofthemostimportantpoliciesandproceduresentail: Costallocationandreporting Imagemanagement Securityandpatchmanagement Monitoring Backupanddisasterrecovery

TheEssentialsSeries:DeployingandManagingPrivateClouds

DanSullivan

CostAllocationandReporting
Cloudcomputingallowsyoutoefficientlyallocatecomputingandstorageresourceson demandonanasneededbasis.Whenthefinancedepartmenthasalargenumberofendof quarterreportstogenerate,theycanallocatemultiplevirtualserversintheprivatecloud foraslongasneededtocompletethereports.Adatawarehouseprojectwithalarge amountoflegacydatacanusethecloudfortheinitialdataextraction,transformation,and loadprocesstorapidlyaddlegacydatatoanewdatawarehouse.Whenadvertising campaignsaremoresuccessfulthananticipatedandathereisasurgeinorders,theonline businesscanscaleupbyaddingapplicationserversandWebserverstoaccommodatethe demand.Unlessalloftheseservicesareprovidedwithoutchargetoendusers,youmust haveamechanisminplacetotrackusage. Acostrecoverysystemcanusedatafromtheselfservicemanagementsystemtotrack whichusersareallocatingvirtualservers,howlongtheyrun,andwhichapplicationsare runonthevirtualservers.Thelatterisimportanttorecoverthecostofsoftwarelicenses. Similardataisrequiredontheamountofdatastorageusedovertimeaswellastheamount ofnetworkbandwidthusedwhilerunningapplicationsintheprivatecloud. PoliciesareneededsothatITproviderscanplantorecovertheircostsandpossiblybuild capitaltofinanceadditionalinfrastructurepurchase.Usersneedthesepoliciessothatthey canplanhowtoefficientlyusethecloud.Anadvantageofcostrecoverypoliciesisthatthey canbeusedtodistributejobsacrosstime.Forexample,ifthecostofanhourofCPUtimeis thesameatalltimesoftheday,usershavenoincentivetoruntheirjobsatanyparticular timeoftheday.If,however,thecostofaCPUhourwas50%lessduringnonbusiness hours,userswithbatchreportingjobsmightmovetheirjobstooffhoursleavingmore resourcestotimecriticalapplications.

ImageManagement
Partofaprivatecloudsserviceofferingsisaservicecatalog.Thissetofvirtualmachine imagesisavailableforuseinthecloud.Policiesandproceduresshouldbeinplacethat definewhattypesofimageswillbeavailableintheservicecatalogaswellasrules governingtheuseofprivatelycreatedandmanagedimagesinthecloud. Policiesshoulddefineaprocessforaddingnewimagestotheservicecatalogand reviewing,andpossiblyremoving,images.Thegoalistomaintainthesetofimagesthatare neededbyuserswhilestayingincompliancewithsoftwarelicensesandreducingsecurity riskstovulnerabilitiesthatmayexistintheoperatingsystems(OSs)andapplications withintheseimages.Thisbeginstogetintotherealmofsecuritywithintheprivatecloud.

TheEssentialsSeries:DeployingandManagingPrivateClouds

DanSullivan

SecurityandPatchManagement
Theneedforpreservingtheconfidentialityofinformation,theintegrityofdata,andthe availabilityofresourcesarethekeydriversbehindITsecurity.Aprivatecloudshouldbuild onexistingsecuritypolicies,especiallywithregardto: Userauthenticationandauthorizations SoftwareallowedtorunonITmanagedhardware Vulnerabilityscanning Operationsmonitoring Patchmanagement

Aprivatecloudbydefinitionisrestrictedtoaspecificsetofpotentialusers.Policiesand proceduresshouldbeinplacetoensurethatonlyqualifiedusersareallowedtoaccess cloudresources,thatauthorizationstousesoftwareandhardwarearealignedwithausers rolesandresponsibilities,andthatthoseauthorizationsandprivilegescanbeeasily modifiedasneeded. PoliciescanalsobeusedtobalancetheneedofITadministratorstocontrolwhich applicationsruninthecloudwiththespecializedneedsofsomeusers.Forexample: Ifadepartmenthiresateamofconsultantstodesignacustomdatabase application,whatkindofreviewprocessisrequiredtoaddittoruninthecloud? Canusersrunanyapplicationthatusesastandarddatabasemanagement system,suchasMicrosoftSQLServer? WhatifitusesadatabasemanagementsystemnotsupportedbyIT?

Planningforhowtomakedecisionssuchasthisarebestdonewhileplanningforthe privatecloud;thishelpstoreducetheneedforadhocdecisionmakingwithregardsto policiesandprocedures. Complexsoftwarecanharborvulnerabilitiesthatcanbeexploitedformaliciouspurposes. VulnerabilityscanningisanestablishedpracticeofcheckingdeployedapplicationsandOSs forknownrisks.Thistypeofpracticeshouldcontinuewithprivateclouds.Bothpublic imagesintheservicecatalogandprivatelymanagedimagesshouldbecheckedaccording toapolicydefinedscheduleusingvulnerabilityscanningtoolsthatmeetfunctional requirementsdefinedinthosepolicies. Policiesshouldalsodefinethetypeofoperationaldatatocollectandthefrequencywith whichitshouldbecollected.ThegoalofthispolicyistoensureITadministratorshavethe informationtheyneedtooptimallymanagetheprivatecloudonadaytodaybasis.This policyalsoprovidesbaselinedataandtrendinformationthatmanagerscanusefor planningforthelongtermgrowthoftheprivatecloud.

TheEssentialsSeries:DeployingandManagingPrivateClouds

DanSullivan

Anotherpolicyshouldgovernthepatchmanagementprocessandtherelatedprocessof rebuildingimages.Afteranimageisbuiltandstoredintheservicecatalog,theremaybeOS upgradesandpatchestoapplicationsthatshouldbeapplied.Apolicyshoulddescribe conditionsunderwhichapatchisconsideredcriticalandshouldbeappliedimmediately;it shouldalsodefineroutinepatchschedulesfornoncriticalupdates.Aswithotherpolicies, itisimportanttohavethispolicyinplacewhendeployingaprivatecloudtoreducethe needforadhocpolicymaking.

BackupandDisasterRecovery
Aprivatecloudmaybeusedforproductionoperations,soitisimportanttohaveabackup anddisasterrecoverypolicyinplace.Thebackuppolicyshoulddefinewhatdataisbacked up,howlongbackupsarekept,aswellascostsassociatedwiththoseservices.Similarly,in theeventofacatastrophicfailureofaprivatecloud,afailoverplanshouldbeinplace.This planmayincludeusingmultipledatacenterstohostaprivatecloudorrunningjobsina moreconventionallyorganizedclusterenvironmentwithmanualmanagementofjobs.The detailsofhowtoimplementbackupanddisasterrecoverywillvarybyyourneedsand resources,butitisessentialforbusinesscontinuityplanningtohavesomepolicyinplace.

StandardizingHardwareandApplicationStacks
Anothersetofbestpracticesfocusesonstandardizinghardwareandapplicationstacks.It isnotthatavarietyofhardwareorsoftwareisnecessarilyabadthing,butitoftenrequires additionaltimetomanage.Considerasimplescenario:Supposeyoubuildacloudwith serversfromthreevendorswithdifferentnetworkandstoragecontrollers.Inorderto minimizedowntime,youmaintainspareparts;however,youhavemultipleconfigurations, soyoumustmaintainalargersetofsparesthanifyouhadasinglestandardconfiguration. Theadditionaloverheaddoesnotstopwithhardware.Itisnothardtoimaginethatone configurationofLinuxmightworkoptimallygivenonehardwareconfigurationbutsub optimallyinanotherconfigurationand,asaresult,youstarttomaintaintwoormore configurations. Themanagementobjectivewithregardstostandardizingistohavetheminimalnumberof distincthardwareandsoftwareconfigurationsthatmeetalluserrequirements. Fortunately,itisfairlyeasytostandardizehardware,especiallyifyouarepurchasingnew serversandstoragearrays.Evenifyouareworkingwithlegacyhardware,youcan incrementallymovetostandardconfigurationsasolderhardwareisretiredorrepurposed.

TheEssentialsSeries:DeployingandManagingPrivateClouds

DanSullivan

Businesseswithawiderangeofapplicationneedswillfindthattheymustmaintainafairly broadservicecatalogofimages.Thisisnotnecessarilyaproblemifyoucanatleast standardizeonsomeofthekeycomponentsintheapplicationstack: OS Applicationservers Networkservices Transactionprocessingservers

Forexample,abusinessmayhaveoneortwoversionsofWindowsServerandtwoorthree versionsofLinuxOSsfordifferenttasks.Buildingonthese,theITdepartmentcanoffer.Net FrameworkapplicationsontheWindowsserverswhileprovidingJavaapplicationsonthe Linuxservers.Applicationsthatrequiredirectoryservicesmaybeabletorunastandard LDAPserver.Similarly,theprivatecloudmayofferapreconfiguredtransactionprocessing serverthatisgeneralizedenoughtomeetmostuserrequirements. Standardizationdoesnotrequirethatyoufiteveryonesneedsintoasinglesetof applicationimages.Therewillbeexceptionsandthoseshouldbeaccommodated.The purposeofstandardizationistoreducemanagementoverhead,notconstrainbusiness requirements.

FormalizeDiscoveryandMonitoringProcedures
Knowingwhatyouaremanagingandunderstandinghowitisusedisessentialtoefficiently deliveringcloudservices.Businessesthatdeployprivatecloudswilllikelyhavesome resourcesdedicatedtothecloudandothersusedoutsidethecloud.Anongoingobjective willbetoallocateservers,storage,andnetworkservicesoptimallybetweentheprivate cloudandotheruses.Ifserversareunderutilizedoutsidethecloudwhileatthesametime jobqueuesaregrowinginthecloudbecausethereisnotsufficientCPUcapacity,thenyou shouldconsiderreallocatingresources.Tocollectthiskindofdata,youneedtohave discoveryandmonitoringproceduresinplace.

TheEssentialsSeries:DeployingandManagingPrivateClouds

DanSullivan

Discoveryandmonitoringsoftwarecanmeetatleastthreemanagementneeds.Automated discoveryhelpstomaintainanaccurateinventoryofresources.Thisisespeciallyimportant whenhardwareisfrequentlymovedandrepurposed;manualrecordkeepingcaneasilyfall behind.Asecondobjectiveistousethenetworkandservermonitoringtocollectdataon utilizationandavailability.Cloudadministratorscanusethisdatatoidentifybottlenecks, potentialhardwarefailures,andotherareasthatneedtheirattention.Bothdiscoveryand monitoringdataisusefulforestablishingoperationalbaselinesandplanningforgrowth. Thisdatacanhelpjustifytheneedfornewhardwareaswellaschangestopolicies;for example,ifjobqueuesarefilledduringthedayandrelativelyemptyatnight,achangein pricingpolicycouldbeusedtospreaddemandmoreevenlythroughouttheday.

Summary
Asprivatecloudsevolve,sotoowilltheirmanagement.Fortunately,youcanleveragemany ITbestpractices,particularlywithrespecttoestablishingpoliciesandprocedures, standardizinghardwareandapplications,andformalizingassetdiscoveryandmonitoring procedures.