data center design & enterprise networking€¦ · data center design and networking, students...
TRANSCRIPT
Rochester Institute of TechnologyRIT Scholar Works
Theses Thesis/Dissertation Collections
2009
Data center design & enterprise networkingChristian Mahood
Follow this and additional works at: http://scholarworks.rit.edu/theses
This Thesis is brought to you for free and open access by the Thesis/Dissertation Collections at RIT Scholar Works. It has been accepted for inclusionin Theses by an authorized administrator of RIT Scholar Works. For more information, please contact [email protected].
Recommended CitationMahood, Christian, "Data center design & enterprise networking" (2009). Thesis. Rochester Institute of Technology. Accessed from
Data Center Design & Enterprise Networking
By
Christian Mahood
Thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in
Networking & Systems Administration
Rochester Institute of Technology
B. Thomas Golisano College of
Computing and Information Sciences
February 25th, 2009
Rochester Institute of Technology
B. Thomas Golisano College of
Computing and Information Sciences
Master of Science in Networking and Systems Administration
Thesis Approval Form
Student Name: Christian Mahood
Thesis Title: Data Center Design & Enterprise Networking
Thesis Committee
Name Signature Date
Luther Troell Chair Charlie Border Committee Member Sylvia Perez-Hardy Committee Member
Thesis Reproduction Permission Form
Rochester Institute of Technology
B. Thomas Golisano College of Computing and Information Sciences
Master of Science in
Networking & Systems Administration
Data Center Design & Enterprise Networking I, Christian Mahood, hereby grant permission to the Wallace Library of the Rochester Institute of Technology to reproduce my thesis in whole or in part. Any reproduction must not be for commercial use or profit. Date: ___________ Signature of Author: __________________________
Abstract:
Today’s enterprise networks and data centers have become very complex and have
completely integrated themselves into every facet of their represented organization.
Organizations require Internet facing services and applications to be available at any
part of the day or night. These organizations have realized that with centralized
computing and highly available components, their technological presence with
customers can be greatly enhanced. The creation of an infrastructure supporting
such high availability takes numerous components and resources to function
optimally. When an organization makes the decision to design a data center, they
utilize resources to provide insight into what components to deploy. Much of this
information is based off of recommendations made by third party vendors or limited
past experiences. This research provides a course offering as a solution to help
provide students with the information to design and comprehend the major
components within a modern data center. The information included in the course
offering has been compared with industry accepted standards and various other
resources to provide reliable and accurate information. Course topics have been
architected around eight major topics. These topics covered are network design,
electrical systems, HVAC (Heating, Ventilation, and Air Conditioning) systems,
security, management, redundancy, disaster recovery and site planning. The
information contained within the lectures has been compiled from multiple sources to
provide a single location for information. Furthermore, the course offering will utilize
class discussions, case study analysis and activities to re-enforce key points.
Providing such a course for students to learn about data center components will
provide organizations with the ability to rely less on outside information and design
scalable data centers that can provide years of growth.
TableofContents
1Introduction .................................................................................................................... 11.1Problem ............................................................................................................................................................................11.2Importance......................................................................................................................................................................3
2Reviewofcurrentresearch.............................................................................................. 3
3Background ..................................................................................................................... 8
4ResearchDesignandMethodology................................................................................ 104.1AssumptionsandLimitations ..............................................................................................................................104.2ResearchMethodology ...........................................................................................................................................124.3TestSystemDesign...................................................................................................................................................145Results .......................................................................................................................... 155.1CourseTextbook........................................................................................................................................................155.2TopicDesign ................................................................................................................................................................175.3TopicExpansion.........................................................................................................................................................195.3.1NetworkDesign.......................................................................................................................................................195.3.2Electrical ....................................................................................................................................................................225.3.3HVACSystems ..........................................................................................................................................................245.3.4Security .......................................................................................................................................................................275.3.5DisasterRecovery...................................................................................................................................................285.3.6Tools.............................................................................................................................................................................30
6Conclusion..................................................................................................................... 31
7FutureWork .................................................................................................................. 328.1CourseSyllabus ..........................................................................................................................................................338.2CourseICCForm........................................................................................................................................................398.3CourseLectureSlides ..............................................................................................................................................438.3.1Week2Slides............................................................................................................................................................438.3.2Week3Slides............................................................................................................................................................528.3.3Week4Slides............................................................................................................................................................608.3.4Week5Slides............................................................................................................................................................718.3.5Week7Slides............................................................................................................................................................80
8.4MidtermExamExampleQuestions ...................................................................................................................898.5MidtermExamExampleQuestions ...................................................................................................................919References: ................................................................................................................... 93
2
Table of Figures
Figure1 ............................................................................................................................ 12
Figure2 ............................................................................................................................ 17
Figure3 ............................................................................................................................ 18
Figure4 ............................................................................................................................ 30
1
1 Introduction
Organizationalnetworkshavebecomevastandcomplex.Manycompaniesthroughoutthe
worldhavedatacentersthathaveevolvedfromsingleroomswithinlargercompany
operationcentersintoenormousstandalonebuildingsthathousethousandsupon
thousandsofservers.Thereisnolongeranotionthatadatacenterbelongsonlytothelarge
organizationsoftheworldassmallerandsmallerbusinessesarebeginningtorelyonthe
centralizationandbenefitsofrunningadatacenterthatcanassistwiththeiroverall
efficiency.Datacentersrangeinvaryingsizeandcapabilitiestodayandalsoarecomprised
withvaryingcomponents.Thereareanumberofdifferentpieceswithinadatacenterthat
mustmeshtogetherseamlesslyforperformancetobeoptimal.Inorderforanorganization
toimplementandbuildadatacenter,heavyplanningisinvolvedtounderstandtheneedsof
theorganizationandtodeterminethefullscaleoftheendresult.Someorganizationsmay
payparticularattentiontofinancialsideofthedatacenter,whileotherorganizationsmay
payattentiontooverallspacethatcanbeachieved.Thisparticularneedwillvaryfrom
organizationtoorganization,howevertherearemultiplekeyareasthatmustbeaddressed
todesignafunctionaldatacenter.
Thisresearchaimstoprovideasolutionforacademicinstitutionstoprovidestudentswitha
locationtolearnaboutdatacenterdesignandnetworking.Byprovidinginformationabout
datacenterdesignandnetworking,studentswillbeabletoutilizeexperiencesand
informationfromtheclassinfuturecareersfororganizationsthatwishtoreconfigure,plan
orretrofitenvironmentsforusageasadatacenter.
1.1 Problem
Datacentersrequirealargeamountofplanning,interactionwithdifferentresourcesanda
largeunderstandingofdifferentareasofexpertisetoimplementfromstarttofinish.Thereis
2
alargeamountofcostandtimededicationtounderstandalltypesoftechnologyand
componentsthatmustbeimplementedwithinadatacenter.Thereisinformationavailable
acrossmanydifferentplatforms(theInternet,books,articles,etc,)howevertherelacksa
resourcethatcanhelpstudentsunderstandthecomponentsandtheimportanceofdesign
intheplanningofadatacenter.Thedevelopmentofacourseofferingthatcoversthese
topicswillallowstudentstoshareexperiencesthattheyhaveencounteredandalsowill
allowstudentstocollaborativelylearndifferentfundamentalstogetherthroughdiscussion.
Thereisampledataavailableinlibraries,theInternet,variousbooks,andmagazinearticles
thatcanassistintheplanningandimplementationofadatacenterforanorganization.
Manyoftheseresourcesaredifficulttoobtain,containinformationonasinglepieceofthe
infrastructure,lackcomprehensiveinformationinonesourceanddonotprovideadequate
recommendationsbasedonexperiences.Theinformationcanbedifficulttoobtaindueto
financialinvestmentandalsocantakelongamountsoftimetocorrelate.Thelargeamounts
ofdatamakeitdifficultforausertodecipherexactlywhatcomponentsarecriticalandvital
tothesuccessoftheproject.Itisimportanttodesignthedatacenterforcurrent
organizationalneeds,yetplanforthegrowthofthedatacentertomeettheneedsofthe
organizationsforthenexttentotwentyyears.
Acourseofferingthatcompoundstheinformationandallowsindividualstudentexperiences
tobesharedisoneapproachtodecreasetheneedforvariousdifferentexternalresources.
Thisresearchhasnotfocusedonthebenefitstocentralizationofinformationinregardsofa
datacenter,butpresentsasolutiontoassiststudentscomprehendandunderstandthe
necessarycomponentswithinadatacenterandtakesalargestepofgatheringthe
informationoutofthepicture.Manysourcesdiscusshowtocreateadatacenter
surroundingcurrentorganizationalneeds.Adatacenteristhendesignedsurroundingthese
“needs,”howeverwithintwotothreeyearstheorganizationcouldrealizethatithasrunout
ofspaceandneedstodeterminetheircourseofactionyetagain.Howcouldtheresponsible
personthengobacktotheorganizationandaskagainforhundredsofthousandsofdollars
3
foryetanotherdatacenter?Thoughtfulplanningandestimationthroughoutthedesign
processcouldhavemitigatedthisbyunderstandingscalabilityconcepts.
1.2 Importance
Planningthroughouttheentireprojectofadatacenterisimportantasitallowsforthe
correctestimationandscopeofthevariouscomponentssupportingthedatacenter.The
infrastructureofthedatacentercanallowflexibilitiesinthesystemcomponentsinstalledin
thedatacenterifplannedcorrectly.Thevariouscomponentsfromelectrical,andheating,
ventilationandairconditioning(HVAC)systemsinstalledinadatacenterareusually
modifiedtoprovidehighercapacitiesandmanyorganizationscouldutilizetheskillslearned
inancourseofferingtoprovideconsciousdecisionsduringprojectssurroundingtheirdata
centers.Thisresearchwillbenefitanyorganizationthatislookingtobuild,retrofit,ormodify
adatacenter.Itwillalsobenefitanyinstitutionlookingtoimplementsuchacoursetoassist
studentsinunderstandingdatacenterdesignandmajorcomponents.
2 Review of current research
Researchregardingthecomponentsofadatacenterandtheirimportancehasbeencovered
inindustryandacademialedresearch.Thereisanimportanceunderstoodthatthedata
centeritselfprovidesmanyopportunitiesanduniquefeaturesthatothertypesofcomputing
donotfeature.Highlypoweredcondensedserverclustersandhighavailabilityhardware
makethedatacenteracriticalcomponenttotoday’sorganizationalsuccess.Howcanthese
typesofenvironmentsbeplannedcorrectlyforfullscalability?Furthermore,howdoallof
thesecomponentsworktogetherinafashionthatcanbemanagedsuccessfully?Inthis
section,priorresearchispresentedaboutdatacentercriticality,poorsiteplanning,network
components,managementandlastlyscalabilityimportance.
4
(Alger,2005)focusedontheneedforadministratorstorealizeandfullyunderstandthe
criticaltieofaDataCentertothebusinessitsupports.Thepurposeofhispublicationwasto
informadministratorsandexecutivesoftheplanninginvolvedwithaDataCenter.Theother
focusofthepublicationwastoaddressthemagnitudeofcomponentsthatgointoaData
centerandhowtochoosethebestsolutionfortheneedsofthebusiness,orcustomer.His
researchwasqualitativeregardingthecomponentsandincludedquantitativemethods
involvingdesigninganalysis.Thebookwaswrittenpullingtheexperiencefromtheauthor
andthosefromtheCiscoInfrastructureTeamtogiveadministrators,professionalsand
executivesabetterunderstandingofaDataCenter.Thebookstructuresitselfintothekey
areasofthedatacenter;thesite,thespace,theelectricalsystem,thenetwork,theairand
temperatureofthesite,andthemanagementoftheDataCenter.Theresultsofthe
publicationaremultipledifferentviewsandopinionsaboutthedesignandimplementation
oftheDataCenterandtheavailableoptionstothecustomerallinasinglepublication.This
publicationisuniqueinthatitcombinesalmostallaspectsofthedatacentertogetherinan
analysistothedepththatallkeypointsarecovered.Thetechnicalflawofthispublication
wasthatitdoesnotcoverthedepthnecessaryinDataCentersfornetworkingsegregation
andflowlayout.
Anothersimilarpublication(Snevely,2002)discussesthoroughlyaboutthelackof
constructionplanningwhenapproachingaDataCenter.Themainfocusofhisresearchwas
toprovidethoseinterestedinplanningtheconstructionorre‐constructionofaDataCenter
andevenasminuteasreconfiguringorretrofittinganexistinginfrastructurewiringcloset.
HisattemptwasnottocovertheallaspectsofthedesignofaDataCenter,buttoprovide
resourcestohelpassistthedesignofaDataCenterbasedoffoftheneedsandrequirements
oftheDataCenterforconstructionalpurposes.Hisresearchincludedqualitativemethodsof
constructionanddesignfrompreviouscasestudies,experienceanddocuments.The
materialcoveredinsidethebookrangesfromexteriorspacedesignsdowntothewiring
insideaDataCenterrackandevenincludestrashcontrolmethods.Robextensivelycovers
andstressestheimportanceofplanninginaDataCenter.Eachsectionwithinthebook
5
coversatleastonetiporkeypointtomakenote.Thebookalsocoversindepththeneedto
followlocalandfederalbuildingcodeinregardstotheconstructionandimplementationof
componentsinDataCenters.Thisisauniquefeaturethatshouldbetakennoteduring
planningstages.TheconclusionthatSnevelymadeduringhisbookwasthatwhilethe
planningofDataCenteriscomplicated,ifdonecorrectlytheDataCentercanprovidea
scalableandmodularenvironmentforanysizeandcapabilityneededbythecustomer.The
otherresultofthebookisapublicationthatcanprovidequickreferencefordesign
requirementsandtips.Thedifferenceinthispublicationincomparisontootherworksisthat
whileitdidnotcomprehensivelycoverservercomponents,ornetworkcomponentsitdid
coverthemajorityoftheenvironmentalcomponentsnecessarytodatacenterdesignand
shouldlookedatasavalidresourceforstudents.
(Berktowitz,1999)presentedanin‐depthanalysisofenterprisenetworkingandcomponents
ofenterprisenetworks.Inparticular,thispublicationfocusedfullyonenterprisenetworking
ratherthannetworkingingeneral.Theauthorrecognizedaproblemwithnetworking
“cookbooks”thatclaimtoteacheverything,butlackthefollow‐throughofexplainingthe
“whatandwhy”ofnetworking.Inthispublication,particularlychapterssevenandeight,the
authordistinguishedthedifferentlayersofthenetwork.Itisauniquepublicationthattakes
directlyintoconsiderationtheflowofthenetwork,theperformance,andmostnotable,the
implementationofnetworksegmentation.Theresultofthissectionwasacomparison
betweenthedifferenttechnologiesavailableandtherecognitionofcostintheequation
duringplanninganddesignstages.Theauthoralsorecognizedthevastarrayofdifferent
technologiesavailabletothecustomeranddiscussestheneedtounderstandthebusiness’
customer’sneedspriortodesigningthenetworkportionoftheDataCenter.Thispublication
directlyaddressedtheneedforenterprisenetworkdesignandwentintogreatdetailabout
thistopic.ThepublicationdoesnotaddressanyintegrationorcasestudiesofDataCenter
integrationandalsodoesnotmentionotherportionsofnetworkingintheDataCenterother
thanLayers2andLayer3oftheOSImodelcommonlyreferredtoinnetworking.Whilethe
6
studyisusefulforasingleimportantcomponentwithinthedatacenter,itdoesnotpresent
anyinsightintotheintegrationofanyothercomponentsrelevanttoadatacenter.
AgroupatCarnegieMellonUniversity(Sun,2007)focusedonmanagementofDataCenters.
TheyfoundthatmonitoringisacrucialcomponenttoaDataCenter’sfunctionalityand
maintenance,yettherewerenotcomprehensivesolutionstoaddressmanagement.The
teamatCarnegieMellonthendevelopedanalgorithmthatwillassistinthemanagementof
theDataCenterwithlessofanimpactontheadministrator.Thepurposeofthisresearch
wastointroducetothearenaofmonitoringanothertoolthatprovidedadaptiveandreliable
resultsfortheDataCenter.Theirresearchusedmixedmethodstotakeexistingknowledge
abouttheSNMPprotocolandmanagementsoftware,whilenewmethodswereproposed
basedupontheseexistingmethods.ThealgorithmwastestedonalargeclusteratCarnegie
Mellontotestfunctionalityandscalability.TheirimplementationutilizedSNMP,whichis
oneofthemostubiquitousprotocolsavailableformonitoringpurposes.Thefocusoftheir
algorithmwastoassistinandfurtherenhancetheconfigurationportionofmonitoring,the
reasoningalgorithmusedtodetermineifaneventisanomalous,andthestorageofthe
historicaldata.Theresultsoftheirstudyincludedresultsthatwerenotonlypositive,but
exceededwhattheyfeltweretheirgoalsintheproject.Theirresearchwasrelevanttothe
topicofthisthesisinthatmanagementandmonitoringarecrucialaspectstoaDataCenter.
Al‐Fares,Loukissas,andVahdat(Al‐Fares,2008)discussedthoroughlytheneedforScalable
interconnectivityinclusters,economicavailability,andbackwardscompatibilitywithlegacy
protocolsandsystems.Theirprojectaddressedaconcernforconnectivityinhierarchical
networkswherebandwidthiscrucialandexpensive.Thepurposeoftheprojectwasforthe
teamtointroducetheproblemtothenetworkcommunityandproposeaneconomic
solutionthatutilizedexistingequipmenttosatisfybandwidthneeds.Theirsolutionwasa
technologycalledfat‐tree,whichsegregatesthenetworkinto“Pods”andtwo‐levelrouting
thatallowslookupsformulti‐pathrouting.Theprojectimplementednewcodeontoexisting
routersandswitchesthathelpedandallowedthemtoperformthetwo‐levelroutingand
7
lookups.Theimplementationandtestingofthistechnologywassuccessful,howevernot
ideal.Thecore‐switchinglayercanbecomecongestedduetocollisionandroutingdecisions
madebytheroutersdownstream.Theotherresultthatwasnotoptimalwasthenumberof
cablesrequiredtocreateameshtopologyforthenetwork.Thisprojectvisitsaunique
problemthatoccursinDataCenters.Theneedforscalablebandwidthforinterconnectivity
isstillnotascapableastheclustersandtechnologythatisavailableforusage.
(Sharma,2008)addressedtheimportanceofcooling,power,andcomputinginaDataCenter
withtherecentchangestoserverequipmentanddesign.Sharma,Shih,BashandPatel
describethechangesinserverhardwareandthatithasbecomemoreandmoredenseand
powerfuloverthelastdecade.Thishascreatedahigherlevelofnecessaryresources
availablefortheseserverstooperatesuccessfullyandefficiently.Toaddressthisissue,the
groupfromHewlett‐PackardproposedacontrolenginetocreateaDataCenterthatcan
throttleandchangeitsrequirementsforcooling,powerandcomputing.Theauthorsalso
discusshowthissolutionvariesfromtraditionalDataCentermethodologyinthateachgiven
areainaDataCenterissegmentedfromothersviacontrolengines.Thethreeareas(Power,
Cooling,andComputing)canbethrottledupordownbasedoffalgorithmsthattheauthors
havespecified.Ifahigherdemandisnecessaryforonesection,otherscanthrottledownto
allowresourcestobedistributedtotheotherworkloadareasintheDataCenter.Theyalso
discussedthatthethreeneededtobemanagedasanensembleandnotseparateentitiesfor
benefitstobeobtained.Theoutcomeofthisprojectwastherealizationthatenergy
consumptionandflowswillcontinuetobecriticalinDataCentersandthatthemanagement
ofthesecommoditieswillbecomemoreandmoreofafocusduringthedesignoftheData
Center.
8
3 Background
Inpastimplementations,manydifferentgroupsfromdifferentexpertiseandbackgrounds
havedesignedthevariouscomponentsindatacenters.Eachdifferentgrouphadanichein
theirrespectivefieldandcompletedtheirinstallationbasedoffofinformationthatwas
providedtothembytheprojectmanagerorleadoftheproject.Serverrequirementsinthe
pastweremuchdifferentthanthatofserversandequipmenttoday.Theequipmentwas
morebulky,andmostsystemsdidnotresideinrackcabinetsoftoday’sstandards.Most
systemscalledmainframeshadentirecomputerroomsbuiltaroundthemrequiringhigh
amountsofpowerandHVACsystemsforoperation.Theirfunctionwastoprovidehighly
processorintensivecalculationsforanorganization.Theyalsoallowedforhighdatatransfer
andmulti‐clientconnectionasthedevelopmentofthesesystemsprogressedthroughthe
years.SomeofthemostwellknownofthesesystemswerebuiltandinstalledbyIBM
(Mainframe.)Thecomponentswerenotasdemandingincomparisontotoday’sdatacenter
ascomputingpowerwasneitherhighlycondensednorscalable,butthepowerandHVAC
requirementsofthesemainframeswereuniqueandrequiredspecialdesignsfor
implementation.Computershaveadvancedconsiderablysincethepeakofthemainframein
the1980’sandmanysystemshavebeenreplacedforserversystemswithinthesamerooms
ordatacentersthatoncehousedtheselargemainframes(Mainframe.)
Asthedemandfortheclient‐serverbasedarchitecture,theInternetandindividual
computershasgrownexponentiallyinthepasttwodecades,theneedtodeploysystemshas
alsoincreased.Whilethesenewersystemsdonothavethefullcomputingpowerthat
comparablemainframeshave,theirfootprint,technologicalflexibilityandlowerindividual
costcreatesalargedemandthusmoreandmoreserverswereimplementedinshort
amountsoftime(DiMinico.)Withfurthertechnologicaladvancetheseserversnowcanbe
placedinracksthatcanhouseasmanyas50serversperrack.Theirfootprintcondensesthe
heatoutputandelectricalintake.Withthismanysystemsinasinglerack,onecanclearlysee
thechangesandstressthatinfrastructuresnowplaceonthesameenvironmentalsupportof
9
thesedatacentersthatwereoriginallydesignedformainframeusage.Becauseofthese
changes,manyorganizationschoosetoredesign,rebuiltorstartfreshwiththeirdata
centers.
Asenvironmentsarere‐built,ithasbecomeanorganization’sabilitytoplanforthescalable
futureofitsinfrastructure.Fromtheexperiencecompanieshavegainedfromconversionof
mainframetoserverclustersandimplementations,itcanbeexpectedthatthetrendof
higherpowerinasmallerlocationwillcontinue.Thiscreatesuniquerequirementsfor
electricityandHVACsystem,whicharethetwomostcriticalenvironmentalfactorsinadata
center(Alger.)Planningforthesetwosystemsmustprovideadequatescalabilityforatento
fifteenyearperiodinregardstoanorganizationalgrowthoutlook.
Inthepastdatacenterimplementationsindividualsresearchcomponentsduringtheproject.
Componentsareresearchedonaneedtoknowbasisandsolutionsareprovidedbythese
individualsorbyspecializedthirdpartycontractors.Individualswiththeunderstandingofall
facetsofthedatacenterarehardtofindandcanchargehighratestouseforconsultation.
Informationforindividualsororganizationstocompleteaprojectofthisnaturehastypically
beendispersedinmultiplelocationswithinbooks,Internetarticlesandpersonal
experiences.Havingthisinformationavailableisalargetoolthatcangreatlyassistinthe
planningofthedatacenter.Havingtheinformationavailable,inonelocationand
understoodbyanemployeeorgroupofemployeeswithinanorganizationcanonlyassist
withtheplanningandallowforabetterprojectoutcome.Thisisanimportantpieceinadata
centerprojectandisafocusofthisresearch.
10
4 Research Design and Methodology
Inthissection,anapproachtoacourseofferingfeaturingdatacenterdesignandenterprise
networkingispresented.Theapproachthattheresearchfollowedwasbasedonthe
compilationofvariousinformationsourcestoprovidecoverageofcrucialdatacenter
componentsincoursepresentations.Alsopresentedistheconfigurationofatestsystemto
showstudentsvarioustoolstoutilizeformanagement.Section4.1providesassumptions
andlimitationsforthediscussedportionsofresearch.Theremainingsections4.2and4.3
discusstheapproach,andexplanationoftheresearchandtestsystem.Insection5,the
deliverableresultsarepresented.
4.1 Assumptions and Limitations
Thisresearchdoesnotintendtospecifythatthetopicscoveredbythespecifiedcourseare
theonlysolutionavailabletoorganizationsorcustomers.Thereforeitisassumedthatthe
researchwillbeappliedgenerallytoorganizationsasawaytoensurethatallcrucial
componentsarecoveredandtoprovideassistancewhereweaknessesinaspecifictopicmay
exist.
Itisalsoassumedthatwhilebrandnamesmaybeutilizedwithinthematerialofthecourse,
therearenorecommendationsimpliedbythisresearch.Thestudentororganizationshould
decideequipmenttypesbasedoffofinternalpartnershipsandvendorsandalsowhich
equipmentfitstheirprojectorenvironmentoptimally.
Itcanalsobeassumedthatbyhavinginformationinasinglelocationbeingthecourse,less
timewouldbespentresearchingandpollinginformationfromdifferentresources.Itisalso
assumedthatbyutilizinglesstimeperformingresearchtounderstandtopicfundamentals,
11
theindividualcanthenspendmoretimelearningintricatedetailsandsolutionstoachievea
bettersolutionforthatdatacenter.
Anotherassumptionforthisresearchisthatthefundamentalsofbasicnetworking
technologieswillbepriorcovered.CoveragesuchastheTCP/IPprotocols,basiccomputer
networking,interconnectivityandbasicadministrationresponsibilitieswillnotbecoveredby
thisresearch,butwillbenecessarytounderstandsometopicsoutlined.
Therearevastdifferencesinorganizationsandcustomerneedsthatwillnotbemetbyall
aspectsofthisresearch.Manysolutionscomprisedwithcomponentswillfitafew
customers’needs;howeveroverallnotallsolutionsprovidetheanswertoalldatacenter
projects.Crucialcomponentsinrelationtothedatacenterwillbecovered,andbasedoffof
thesecomponentsafunctionaldatacentercanbeachieved.
Theidealcourseofferingregardingdatacenterdesignandnetworkingcoverseachfacetof
thedatacenterfromtheoutsidebrickandmortartotheindividualpatchcableconnectinga
serveruptoapatchpanelwithinarack.Therearemanyreasonsthatacourseshouldcover
allofthesedifferenttopics.Thefirstandforemostreasonisthatwithinadatacenter,all
componentsmeshtogethertocreateafullsolutionthatisaimedatprovidingandprotecting
digitalinformation.Eachdifferentpartofthedatacenterplaysakeyroleinaccomplishing
thattask.Ifonecomponentisarchitectedorimplementedinawaythatdoesnotsuitthe
datacentercorrectly,itcouldcompromisetheintegrityandreliabilityofthatdigital
information.Anotherreasonthatcomprehensivecoverageisimportantisthatwithoutthe
fullunderstandingofhowcomponentswork,anorganizationmaybeunabletocreatea
scalabledesignthatwillfitthegrowthandfinancialneedsiftheorganizationoveraperiod
ofyears.Whilethisresearchwillcoveralargeamountofinformation,itwillnotencompass
thefulldatacenterprojectfromstarttofinishasoutlinedabove.Theprojectwillfollowa
setofdeliverablesthatisshowninfigure1below.Thesedeliverableswillbediscussedinthe
resultssection,butothertopicsinrelationtothecoursewillnotbediscussedforresearch
12
timeconsiderations.ThesedeliverablesarespecifictotheRITenvironmentanddonot
outlinegeneraldeliverablesforallcoursedevelopment.
Deliverables
Comprehensivelistofcoursetopics
ICCform
Textbookchoice
ListofCasestudies
LectureSlidesfor½ofcourse(5weeks)
Figure1
4.2 Research Methodology
Tocompileinformationthatwillserveagoodbasisforacourseofferingfordatacenter
designandnetworking,coverageofkeycomponentsneedstobecoveredtooffer
informationthatstudentscanutilizeasabaselinefortheirunderstanding.Thissectionwill
explaintheprocedurestodeterminethesekeyareasfortheresearch.
Thecoursewillbedesignedaroundbuildinguponthecoreservicethatitwillprovidewhich
isdata.Thenetworkisthefirstcomponentthatshouldbethoughtofduringthedesignof
thedatacenter.Whilethephysicalspecificationsofthedatacenterareindeedimportant,
withoutanetworkdesign,theremainderofthelayoutmustbemadepriortothe
understandingofthedatarequirements.Thephysicalconsiderations,whichincludethesite,
theelectricalsystems,HVACsystems,physicalsecurity,firesuppressionsystemsand
flooring,shouldbedesignedandimplementedutilizingspecificspecialiststhatunderstand
minutedetailsandspecificationsofthosecomponents.Duringthecourse,keypointswillbe
re‐enforcedtoprovidealevelofunderstandingsurroundingthesedifferentareasof
complexinfrastructuredeterminedduringthecoursedevelopment.
13
Itcanbeseenintheparagraphabovethatthereisanimmenseamountofdatatobe
compiledtofullycompleteaprojectofthisscale.Section1makesthestatementthatthere
isalotofinformationavailableregardingthesetopics,andlikewisefulfillingthesolutionof
centralizingthoseresourcesforstudentswillbeastrongfocusoftheresearch.
Astherearemanydifferentcomponentsthatareimplementedintoadatacenter,alltopic
coveragewillbecompiledusingthesamedefinedmethods.Consistencyfortopiccoverage
depthwillalsobeutilizedtoensurethatinformationisportrayedallowingenoughtimein
thecourseforadequatecoverage.Topicdecisionswillbebasedoffofthecoursetextbook
offerings,andindustrystandardcomparisons.TheTelecommunicationsIndustry
Association’s(TIA)documentTIA‐942isanIndustrystandardforDatacenterdesign.Its
topicswillbereviewedasasourceagainstthetextbookforcoursetopics.Eachtopicwillbe
researchedforitsbackground,differenttypesoftechnology,andimplementationintoadata
center,componenthardeningforreliabilityandredundancyandexamplesofthegiven
component.Thisinformationwillbecompiledfromvarioussourcesprovidingacceptable
informationbasedoncomparisonswithothersources,thecoursetextandindustry
standards.Providingadequatecoverageofeachtopiciscrucial,buttoextendthatcoverage
toshowhowthecomponenttiesintothefullschemeofthedatacenterprovidesamore
realisticapproach.Thementionedstepswillbeutilizedforeachcomponentandwillthus
provideequalunderstandingwhetherthecomponentisthemostcriticaloraddseaseof
managementtothedatacenter.
Toprovideabetterunderstandingforthecourseandforstudentstoreferenceinformation
outsideoftheclassroom,acoursetextbookisnecessary.Searchesonlineandthroughlibrary
databaseswillbeutilizedtolookforasolutionthatprovidesthebestcoverageandinsight
intoalloftheresourcesmentionedabove.Thetextbookwillbecomparedtoothersources
fromresearchandbasedonthebestcomparisonthebookwillbechosen.Topicstobe
concernedwitharedepthofknowledgeportrayedinthesource,numberoftopicscovered,
relevancetodatacenters,casestudypresenceorpersonalinsight,anddateofcoverage.
14
4.3 Test System Design
Todiscussandprovideinsightofdifferenttoolsutilizedincertainlecturepresentations,aset
ofsystemswillbecreatedwithtoolsinstalledfordemo.Thetoolsinstalledontothedemo
systemsaretoolsthatarecommonlyutilizedbydatacenteradministratorstoprovideease
ofdutiesonadailybasis.Toolsassistadministratorsdailywithoperations,monitoring,and
recordkeepingofassets,events,andlogicaldataensuringthatreproductionofsystemsand
designscanbecompletedintheeventoflossordisaster.Thesetoolsarealsocommonly
utilizedtoproviderecordkeepingandchangemanagementofthenumeroussystemsthat
arehousedwithinthedatacenter.Toprovideexamplesofthesetoolsduringlectures,a
systemcomprisedofWindowsXPandanotherpartitionwithadistributionofLinuxcalled
CentOS5willbeutilized.ThesesystemswillbebuiltonaVMwareworkstationplatformto
allowfortransparentusageduringlectures.Avirtualplatformallowsforsoftwaretoolsto
remaininstalledwhilechangesaremadetothesystemthatmayjeopardizetheir
functionality.Quickreversioniscapablewhichprovideseasyfixestomodelsoftwareduring
classes.
15
5 Results
Forthisresearch,informationwascompiledandcentralizedtoprovideresourcesfor
studentstounderstanddatacentercomponents,designstepsandnetworking.The
informationwillalsoallowstudentstounderstandcommonserviceswithinthedatacenter
environmentsuchasreplacinganddesigningrackstobeutilizedco‐existentwithother
systems.Studentswillbeabletodesignnetworkingcomponentsfromthelowestrack
cabinetwithinthenetworkhierarchyuptotheserviceproviderlevel.Toprovideadequate
understandingofmaterialscovered,asetofdeliverableswassetforthintheproposalofthis
projectandisprovidedinfigure1.Thesedeliverableswillbediscussedwithinthisresults
section.Thecourseproposedtextbookwasthefirstdeliverablediscussedtoprovidea
locationforstudentstoreferencematerial.Thisdeliverableisdiscussedinsection5.1and
providesthetextbookchosenfromtworeputablesources.Acomprehensivetopiclistwas
alsocreatedtoprovideahighleveloverviewofwhatwouldbecoveredinthiscourse.This
topiclistwillbecoveredinsection5.2andalsocanbefoundinAppendix8.1underthe
coursetopicssection.Aselectionoffivelecturepresentationsispresentedtoprovide
contentcoverageandmaterialexpansioninrelationtothecourseobjectivesandtopics.
Thesepresentationsarecoveredinsection5.3andalsoareshownintheirentiretyin
appendix8.3.Theremaindersofthedeliverablesoutlinedincludehalfofthemidtermand
finalexam,whichisincludedinappendix8.4and8.5.AlsoincludedisthedepartmentalICC
form,whichhasbeencompletedandprovidedasappendix8.2.
5.1 Course Textbook
ThecoursetextbookwaschosentobeBuildtheBestDataCenterFacilityforyourBusiness
byDouglasAlger.ThebookbyDouglasAlgerwascomparedtoEnterpriseDataCenterDesign
andMethodologywrittenbyRobSnevely.BothofthesebookscoverthemajorityofData
Centerdesignandnetworking,butAlger’sbookcoversbetterin‐depthmosttopicsandalso
16
injectspersonalexperienceintoeverychapterofthebook.Snevely’sbookpertainsmoreto
theup‐frontinformationanddoesprovideabetterinsightintonationalbuilding,fireand
electricalcodes.InformationfromRobSnevely’sbookwillbeutilizedintothelecturesand
willbeprovidedasanoptionalreferenceasthebookhasbeenreleasedasafreepublication
fromSunCorporation(Snevely.)
Anotherdifferencebetweenthetwobooksistheusageofdiagrams,mathematical
calculationformulasandcomparisonsbetweentechnologies.Theseareallimportantfactors
whenbuildinganddesigningadatacenter.Theorganizationorcustomerneedsto
determinethecorrectmaterialsandsizesthatwilloptimallyfittheirdatacenter.Differences
betweenproductsmeancostsavingsorincreases,whichultimatelyaffectsthebudgetofthe
datacenter.Instillingtheabilitytoperformthecalculationsanddesignspermitsfaster
designtimesandalsoallowsforverificationofbidsestimatedbythird‐partycontractors.
Thisisanimportantcost‐savingsapproachtoensurethatthesolutionisgearedexactly
towardstheorganizationalrequirementsandgoalsoftheproject.Itmaybefoundthata
solutionrecommendedorputinplansisnotadequatefortheenvironmentorinopposite
cases,ismuchtoolargefortheenvironmentfortheorganization’sgrowthscale.
Diagramsutilizedinthecoursetextbookallowsforstudentstovisualizeconceptsthatare
beingdescribedwithinthechapters.Agivenexamplewouldbeexistentinthedescriptionof
airhandlerplacement.Whilereadingabouttheplacementofanairhandlerperpendicularto
rowsofserverracksprovidesmaybeaninsightastudentmaynothavepreviouslyknown,it
doesmaynotdepictaclearunderstanding.Placingadiagramthenrepresentsanexampleof
theairhandlerandallowsdifferenttypesoflearnerstounderstandthesameconcepts
visually.Alger’sbookutilizesdiagramsmoreeffectivelyforconceptsandmaterialsthat
shouldbeunderstood.Thisprovidesabetterplatformforstudentstounderstandthebasic
conceptsandcomponentsfordatacenterdesignandnetworking.
17
5.2 Topic Design
Thecoursetopicsweredesignedbasedoffofthecoursetextbookandcomparisonswiththe
TIA‐942standard.Whiletherearemultipledifferenttopicsandcomponentsthatmakeupa
datacenter,majorsectionsofthedatacenterwillbecoveredtoprovidethefeasibilityto
supportabasicdatacenter.Thetopicsthatarediscussedindepthinboththeindustry
standardandthebookareshowninfigure2below.Thesetopicswerethemostprevalent
topicsinthetwosourcesandhavebeenreviewedbymanyinindustry.Togetherthese
topicsprovideover80%oftheinfrastructurepresentinadatacenter.Thetopics
themselvescontainsub‐topicsthatallowsforabetterbreakdownofexplanation.Thesesub‐
topicscanbefoundinappendix8.1.
Withthetopicsdetermined,overallcourseobjectivescouldbegintobesetforth.While
manyofthesearegeneralized,theypresentthegoalsthatstudentsshouldbelooking
towardsattheverybeginningofthecourse.Attheendofthecourse,studentsshouldbe
abletofulfillthecourseobjectivesoutlinedinFigure3below.Thesecourseobjectiveswill
aimtoassiststudentsinunderstandingandfocusingonkeycoursematerialofthecourse.
Theyalsoservedtoprovideguidanceduringtheresearchperiodsofthisprojectto
determinewhatpiecesofmaterialwererelativetothecoursedevelopment.
Identifyandunderstanddifferententerprisenetworkfundamentals.Understandthefundamentalprocessesandproceduresofplanning,anddesigningacomplexdatacenterand/orenterprisenetwork.Defineresourceneedsofadatacenterandnetworkneeds.
Cabling/Networking HVAC Electrical Redundancy
Management SiteDesign Security DisasterRecovery
Figure2
18
Describethedifferentcomponentsinanenterprisedatacenterandproviderecommendationstoothersaboutthesecomponents.
Design,plan,manageandimplemententerprisedatacentercomponents.Compareexistingsolutionstodeterminewhatwillfulfillthedefinedneedsofthecustomerororganization.Understandhierarchicalnetworksandtheirimportanceinscalableandsuccessfuldatacentersandenterprisescenarios.Writecleardocumentationthatallowsbettermanagementofimplementedsolutions.
Figure3
19
5.3 Topic Expansion
Thetopicsthatwerechosenwerethendevelopedintocourselectures.Theselecturesare
presentedinawaythatrampsthestudent’sunderstandingofcomponentsfromthe
individualcomponentuptotheirfullintegrationinadatacenter.Asoutlinedinsection4,
thebackgroundofthecomponentiscoveredofeachtopicandthenthedifferentoptionsof
thetechnologyarepresented.Afterthecomponentisdiscussedasitself,thecomponentis
thendiscussedasitsintegrationtothedatacenterandwhyitisimportanttothedata
center.Waysforthecomponenttobemeshedintoahighavailabilityorredundancystate
arediscussedandthensecurityimplicationsinregardstothecomponent.Highavailabilityor
redundancyistheabilityforagivencomponenttoremainonline,stable,andavailablewhen
it’sotherhalforanotherportionofthecomponenthasfailedorisdownformaintenance.
Thistermiscommonlyusedwithindatacentersbecauseofthedemandforservicestobe
available24hoursaday.Manycomponentsarethusdesignedtoprovidesomeformof
redundancyorhighavailability.Thisisafeaturethatisdiscussedwithineachofthetopics
fromFigure1above.Tofinalizeeachcomponent,examplesordiagramsarereferencedto
showthecomponentinanactualsetting.Somecomponentswillbediscussedwithrelation
tocasestudiesandstoriestoassistwithkeypoints.
5.3.1 Network Design
Coveringthedifferenttypesofnetworkingthatoccurswithinthedatacenterandhowto
designanetworktobeutilizedinadatacenterwasthefocusofthislecture.Byfarthedata
networkisthemostcriticalcomponentforserviceoffering,transfersbetweendifferent
systemsandforstorage.Whilethereasonswillbediscussedlaterinthissection,thedata
networkprovidestheabilityforserverstoperformworkandalsotoprovideresults,data,
andservicestoendcustomersand/orothersystems.Datacentersusenetworking
componentssimilartoothertypesofnetworkingenvironments;howeverthedifferencelies
inthehardwaredevicesthatinterconnectsystemsandthewaythatthenetworkhierarchyis
designed.Therearealsotypesofnetworksthatarepredominantlylocatedinthedata
20
center,whichprovideuniqueabilitiestoothersystems.Thesetypesofnetworksare
referredtoasstorageareanetworks.Theremainderofthissectionwilldiscussthislecture
anditscomponents.
Oneofthemostimportanttopicstobediscussedwithinthislectureisthenetworkhierarchy
thatisusedwithinthedatacenter.Thethreetiersthatarefocusedonarethe‘edgelayer,’
the‘distributionlayer,’andthe‘hostoraccesslayer’(Oppenheimer.)Theselayersprovide
meanstosegregatetrafficflowandtoallowresourcestobeprovidedinahighavailability
mode.Discussedwillbethedifferentprotocolsutilizedatthedifferenthierarchiesofthe
design.
TheedgelayerofthenetworkprovidesthefirstlayeredaccessfromtheInternetservice
providertotheinternalnetworkofthedatacenter.Thislayerisutilizedwithprotocolssuch
asbordergatewayprotocol(BGP)andEnhancedInternalGatewayProtocol(EIGRP)to
providefailovertoleranceandalsotoprovidethebasicserviceofroutingtrafficonthe
Internetamongdifferentautonomoussystems(Snevely.)Thelayeralsoprovidesthe
locationwherethemajorityofthedatacenter’sroutersarelocated.Theyusuallyprovide
aggregationfrommultiplevendorsandcouldprovideahigherlinkspeedtoproviders.
Thedistributionlayerallowsnetworktraffictobetransferredtodifferentsub‐networks,
customers,andin‐houserequirementsandtoprovidedifferenttypesofnetworkresource
allocations(Kerr.)Thedistributionlayerisalsotheprimarylayerwheretrafficfiltering
occurs.Thisactsasawaytokeepcertaintrafficfrompollutingthenetworkbandwidthand
alsohelpstolimitbroadcastandmulticastdomains.Thedistributionlayerutilizesamixture
ofroutersandswitchestosegregatedifferenttypesofnetworktraffic,whileallowinglower
branchesofthehierarchydifferentaccesslevelstotheInternetoredgelayerservices.The
distributionlayerisalsoconsideredthegatewaylayerasdifferentrouterscommunicate
withinthislayerutilizingprotocolssuchasOSPF(OpenShortestPathFirst)orRIP(Routing
InformationProtocol)tocommunicatenetworkinformation.Hostswithintheaccesslayer
21
utilizegatewaysinthedistributionlayertoaccessresourcesbasedontheirclassification.
Thelecturecoversthislayeranddenotesdifferenthardwaretypesanddiagramstoshow
howconnectionscouldexist.
Thehostandaccesslayeristhethirdandlowestlayerwithinthenetworkhierarchy.This
layerprovidesdifferentnetworkingcomponentssuchasswitching,hubs,firewalls,and
occasionalrouters.Theprimaryfocusofthislayeristoprovidetheconnectionfromtheend
device,intothedatacenternetwork.Attheaccesslayer,separatenetworksmaybefound
thatdonotconnectsystemstothedistributionlayerandedgelayer.Thesenetworksmay
providestoragefeatures,andtransportfrombackendsystemstosystemsthatconnectto
theaccesslayer.Thesenetworktypesserveastheremainderofthelectureforthatclass
period.
StorageAreaNetworks(SAN)andprivatenetworksmakeupthemajorityofothernetworks
withintheaccesslayerofthenetworkhierarchy.SANnetworksprovidestoragesolutions
overtraditionalnetworkingmediums.Thesestoragesolutionstypicallyaresharedstorage
thatcanallowmultiplesystemstoaccessthesamestorageatthesametime.Systemscan
accessthesestoragecollectionsovertraditionalhardware(NetworkInterfaceCards,Fiber
Cards,HostBusAdapters),whichallowsforlowercostimplementationandredundancy.This
topicwillbediscussedinaseparatelectureinfurtherdetail,butthenetworkportionofthe
technologyshouldbearchitectedinawaythatprovidesmultiple(2ormore)connectionsto
thesystemforredundancyandhighavailability.Privatenetworksprovidedatatransfers
acrossbackendsystemstofront‐endsystemswithoututilizinglimitedbandwidth.These
networksusuallyareimplementedtoprovidebettersegmentationandcanalsobeutilized
formanagementpurposes(Alger.)
Thefinalportionofthiscomponentistodiscusstheneedfornetworkredundancy.
Redundancywithinthedatanetworkcanoccurinmanydifferentways.Itcanoccuronthe
hostlevelwithmultipleportconnectionstoswitchesorhubs,oritcouldbeastransparent
22
asrunningmultiplerouterswithintheedgelayerutilizingtheHSRPprotocol.Althoughthe
HSRP(HotStandbyRouterProtocol)isaCiscoproprietaryprotocol,itcanbeutilizedwith
differenthardwarevendorsandsystemsandcanprovidetransparentrouterredundancy
withinthenetwork.Redundancyincreasestheoverallavailabilityofthenetworkto
customersandserversthemselves.Theoverallgoalistocontinueserviceintheeventone
serviceprovider,onecable,ornetworkportfails.Therearedifferentrequirementsfor
availabilityofthenetworkindifferentdatacentertiers.Thesetierswillbediscussedwithin
depthduringthelectureandwillprovidestudentswiththeunderstandingthatwhatever
datacentertheirorganizationfunctionstheyshouldbeawareofnetworkavailability.
5.3.2 Electrical
Electricalinfrastructureandconfigurationisalsoacrucialcomponentofadatacenterasit
providesthenetworkhardware,servers,HVACsystems,andlightswiththenecessaryenergy
tofunction.Thislecturefocusesonthediscerningofdifferentelectricalspecificationsaswell
astheinfrastructurethatisnecessarytoprovideaconsistent,reliableandscalableelectrical
infrastructureforthedatacenter.Itwaschosenasakeytopicbecauseofitsimportancein
theoveralldesignofthedatacenterandbecauseitisusuallynotfocuseduponby
individualswithnetworkingbackground.Thegoalofthelectureandthistopicistoprovide
studentswiththeabilitytoanalyzeinformationpresentedtothemaboutelectricaldesignin
thedatacenterandtobeabletocalculateneedsforpowerrequirements.Overallthemost
criticaltopicregardinganelectricalinfrastructureistheredundancyandstandbypower
offering.Thisisnottheonlysectionofelectricalinfrastructurethatiscovered;howeveritis
themostcrucialtoprovidinghighavailabilitywithinthedatacenter.Similartoredundancy
inthedatanetwork,electricalredundancyfocusesonprovidingmultiplecircuitriesthatcan
servesystemsanddevicesintheeventofafailedcircuit,electricalproviderorstandby
system.Thisbecomesanimportanttopicwhendiscussingdifferenttypesofelectrical
power.Thereisalargedebatesurroundingalternatinganddirectcurrentelectricalpower
andbecomesforefrontduringthediscussionofstandbyandredundantpower.Thereisa
23
lossofelectricalpowerbetweenACandDCpowerduringconversionwithinuninterruptable
powersupplies(UPS).
Voltageselectionbecomesadebateinmanydatacenterenvironments.Manyexperts
believethatdirectcurrentisabettersolutionthanthattoAC(Rasumussen.)Thislecturewill
coverthedifferencesinbothtypesofelectricalpowerandwilloutlinethebenefitsand
drawbackstoboth.Theclasswillultimatelybeabletodecidewhichtechnologyisbetterfor
thedatacenter.Aslidewithinthelectureshowsthecomparisonofthesetwotechnologies
andthedifferentfinancialimplicationsforeithertechnology.Althoughpreferenceisafactor
inthedecisionprocess,costtotheorganizationorindividualusuallyisthewinningfactor.
Thissectionwillprovideuniquediscussionandwillreflectuponpersonalimplementations
fromstudents,andthebook.
Anothertopiccoveredinthislectureistheabilitytoforecastandprovideadequate
electricityforthedatacenter.Therearemanytoolstoutilizeforthisandmanytoolsranging
fromgovernmentprovidedtoolstoindividualproprietortoolswouldbediscussed.Manyof
thesetoolsutilizekilovoltamperageasthemetrictodetermineneed,asitallowsforan
industrystandardtodevelopsolutions.Theabilityforanorganizationtounderstandtheir
requirementsandforecastneedscanattributetoascalableinfrastructurethatwillprovide
longevityinreturnsfrompurchasedequipment.Scalableinfrastructurealsolowerscost
overallbycreatingmeansforanorganizationtoaddcapacitieswithlittleoverheadcosts.An
exampleofscalabilitycoveredinthelecturesistheadditionofcircuitpanelsthatleaveroom
forextracircuitstobeallocatedforrackspace,networkdevices,andHVACsystems.Whips
aredefinedascablingthatisrunfromcircuitpanelstotheendsystemforconnectiontothe
electricalsystems.Thesewhipsusuallyarecomprisedofasingleordualcircuitthatdoesnot
shareitselectricalloadwithothersystems.Thisusuallyconsumeselectricalcircuitsatafast
ratebecauseofdedicationtoindividualsystems.Theabilitytoforecastandimplementthe
capabilitytocontinuesupportingsinglecircuitwhipswhilemaintaininghealthyelectrical
24
loadsconstitutesgoodforecastingfromadesignstandpoint.Examplessuchastheseare
coveredwithinthelecturestoprovidebetterinsightfortheimportanceofscalability.
Thelastsectionthatisimportantforelectricalsystemswithinadatacenteristheabilityto
sustainpowerintheeventofpowersourcefailure.Therearemanydifferenttypesof
solutionsthatcanprovideadatacenterwithmeansofelectricityintheeventofan
extendedoutage.Someofthesesolutionsproposeutilizingbatterysystemstoensurecritical
systemsremainonlineduringtheoutage.Thesesystemsprovidealimitedextensionof
electricity,asbatterycapacitiesaregenerallynotasextensiveasothersolutions.Ensuring
thatthereisatleastathirty‐minutewindowforadministratorstoprovidetriagetosystems
isagoodmeasurementtoscopebatteryimplementation.Anothersolutionforbackup
electricityisforagenerator.Thesegeneratorsusuallycanprovideelectricalmeansforadata
centertoallowsystemtoremainonlineforhoursandevendays.Manyofthesystemsutilize
typicalfossilfuelsasdiesel,naturalgasorgasolinetogenerateelectricity.Thesesystems
shouldhaveagreementswithfuelprovidersintheareatoguaranteeprovisionsintimesof
outages(TIA‐942.)
Manydatacentersuseacombinationoftheabovesolutionstoprovideafullyredundant
systemthatcanallowthedatacentertoremainindependentofthirdpartelectrical
providers.Whetherthesystemisbatteryorgenerator,itshouldbescaledlargeenoughto
powertheentiredatacenterintheeventofanoutage.Thisassiststoensurethedata
center’sSLAtocustomersandincometotheorganizationisnotlostduetodowntime.
Thesetopicswillbefullycoveredwithinthislecture,andtheabilityforstudentstoreflecton
experienceswilladdtothediscussionofelectricalimportance.
5.3.3 HVAC Systems
Thislecturewilldiscusstheimportanceofheating,ventilationandairconditioning(HVAC)as
itrelatestothedatacenterenvironment.Ensuringthattheserversandotherequipmentof
thedatacenterfunctioninaconstanttemperaturewithcorrectamountsofhumidityisa
25
keycomponent.Fluctuationswithintheenvironmentcancauseequipmentfailuresandin
certainscenarioscausedamagetodata.Becausethisisatopicthatisalsonotcoveredin
depthinmanyoftheothertypesofclassestakenclassicallybyindividualsofnetworking
background,thistopicwaschosenasafocusforalecture.Thegoalistoprovidestudents
withthecapabilitytomakeconsciousdecisionswithintheirdesignplansandtoalsohave
theabilitytocriticallyanalyzedesignsproposedbythirdpartyproviders.Thebiggest
componentwithintheHVACsystemofadatacenteristhecoolingcomponentsasbynature
computersgenerateheat.HandinhandwiththeHVACsystemisthewaythatthedata
centerhandlesairflow.Thisishandledwiththeinstallationofoverheadorunderfloorair
plenums,whichprovidesupplyandreturnairtotheHVACsystems.Theseplenumsarethe
areawithinthedatacenterwhereairflowandpressureoccurstoprovidetheairflow
necessarytoregulatetheenvironmenttemperature.
Inpastgenerationsofhardware,temperaturecontrolwasveryimportantintheoperationof
thedevices.Ifthedevicebecametoowarm,stabilityofthedevicewascompromised
(Sharma.)Thesedeviceshavesincebeenrevisedandimprovedtooperateintemperatures
thatarewarmerandcanallowthesystemstoremainstable.Ithasbeenestablishedin
industrythattemperatureswithintherangesof64degreesto81degreesFahrenheitare
acceptabletemperatures(TIA‐942.)Thisismainlytoprovidecomfortabletemperaturesfor
employeestoperformworkwithinthedatacenterandisnotlimitedbecauseofdevices.
Asmentionedearlier,themostimportantfacetoftheHVACsystemisthecoolingsystem
(Alger.)Therearemanytechnologiesthathavebeencreatedtoensurethatsystemscan
remaincool.Thesesystemsdiscussedthoroughlyinthelectureslidesinappendix8.3.3will
assiststudentsrealizethatnotonlyaretheredifferenttechnologiesavailable,butthat
certaintechnologieswillworkbetterinvariousgeographicallocations.Different
technologieswillalsoofferdifferentlevelsofredundancytoprovidecontinuouscooling
capabilitiestothedatacenter.Somesystemsaswillbediscussed,providecoolingtothe
datacenterthatisfreeandprovidesmonthsworthofcostfreeenvironmenttemperature
26
regulation.Thispresentslargecostsavingstotheorganizationforutilitybillsandpresents
uniquedecisionstobedeterminedbythedesigner.
RedundancywithintheHVACsystemprovidestheabilitytocontinuallyprovideenvironment
regulationintheeventofsystemmaintenanceorsystemfailures.Manysystemscannow
utilizethesamecoolingtowersorcoolingsystemswhilemaintainingseveralbackend
systems.Thisallowsforfailoverwhilestillprovidingthesamelevelofservicetothe
environment.
Asdiscussedearlier,theredifferentdesignstodelivercoolingtothesystemswithinthedata
center.Oneofthesedesignswhichisunderfloorplenumwithreturnairaboveutilizesthe
fanpowerofmanyserverstomoveairfromthegrounduptothereturnductsintheceiling.
Thesedifferentapproachestodelivertheairarediscussedindepthwithinthelectureslides
andprovideexamplesofwhatcanhappenifairisnotdistributedcorrectlyorevenly
throughoutthedatacenter.TomitigatemanyoftheproblemswithHVACsystems,the
lecturealsocoversdifferenttoolsutilizedbydatacenterstolocatepocketsofmismanaged
airandthenmitigatetheproblembychangingairflowdesignswithintheplenum.Thermal
designprovidesthebestwayforsystemstoreceivecooledair;howevertheenddesign
dependsonthefacilitythatisprovided.Therearebenefitsanddrawbackstooverheador
underfloordesigningeneral,butgiventherightscenario,bothsystemdesignscanbe
beneficial.
27
5.3.4 Security
Securityprovidesprotectiontoallsystemsthatsupportandexistwithinthedatacenter.
Therearemanytypesofsecuritywithinthedatacenterrangingfromphysicalsecurityofthe
premisestopersonnelsafetyandaccess.Itwasdeterminedthatbecausethereissuchavast
amountofdatasurroundingthistopic,thatitwouldbeagoodfocustocoverinalecture
aboutitsintegrationintothedatacenter.Thetopicwasalsochosenbecauseitinherently
tiesbackintotheinfrastructuresystemofthedata,whichassistsinitsmanagementofthe
datacenter.Thegoalofthelectureisprovideinsightintothecrucialsystemssurrounding
security.
Physicalsecurityisthemostprevalentfeatureofadatacenter.Inmanyways,datacenters
presentthemselvesassecurelocationsbyensuringhighqualitylockingmechanismsarein
place,hightechnologyaccessvalidationsystemsareputinplaceandsurveillancesystems
areprovidedtodetermaliciousactivityaroundandinternallytothedatacenter.Different
technologieswillbepresentedassolutionstothestudentsandbenefitsofthosesolutions
willbeoutlined.
Heatandwaterpreventionisanotherkeytopicdefinedbysecurityofthedatacenter.
Withoutthesetwopreventionmechanisms,thedatacentercouldbevulnerabletofiresand
flooding.Thesetwohazardspresentuniquechallengestoovercomeinadatacentersetting.
Typicallyfirepreventionsystemsareimplementedbyusingwatersprinklersystemstodouse
fireswhentheyaredetected.Withthelargeamountofelectricalcomponentswithinthe
datacenter,itisdifficulttousewater,asalossofdataisalargerisk.Alternatively,watercan
presentchallengesbecauseitcouldshortcircuitelectricalcomponentsunderraisedflooring
withoutnoise,orwarning.Therearemanysolutionsthatareavailabletopreventbothof
thesehazards.Therearemanysolutionstopreventfiresthatarepresentedinthelecture
slidesinappendix8.3.4.Inverselysystemslikeawaterdetectionsystemareavailableto
preventwaterdamagetoflooringsystemsandelectricalcomponents(Alger207.)Toensure
28
thatthesesystemsareworkingcorrectly,continuoussamplingmustbecompletedtoprotect
thedatacenter.
Thelasttopiccoveredwithinthesecuritylecturewillbetodiscusspersonnelsafetyitems.
Therearemanydifferentitemswithinthissectionthatpertaindirectlytothedatacenter.
Thebiggestofthesesectionsistoprovidepolicymanagementandenforcement.Thesetwo
managementitemsprovideastructuralsensetoenforceruleandprovideinformationto
othersofhowthedatacenterfunctions.Thisisimportantforthirdpartyprovidersto
understandwhentheyenterthefacilityandperformservicesforthedatacenter.Ifpolicies
arenotimplementedandenforced,controlandregulationofresourcesisnotfeasible.These
differentpolicyitemsareincludedwithinthislectureandwillprovidethestudentswiththe
abilitytosafeguardthedatacenterandthosewhoworkdailywithinthefacility.
Anothersectiontobecoveredistheadherencetolocal,stateandfederalbuilding,fireand
electricalcodes.Thesecodesareimplementedtoprotectindividualswhomworkinthese
facilitiesandtoensurethatdevicesandinfrastructureareinstalledwithmaterialswhose
integritywillmatchstressputonthem.Thisinformationwascompiledfromthenationalfire
protectionassociation(NFPA),NFPA70(Snevely.)Thiscodeisalargereferenceforthose
workingwithdatacentercomponentsandelectricalcomponents.Thegoalofthissectionis
toprovidestudentswithresourcestolocatecurrentcodesandregulationsasmanyfacetsof
thedatacenterarestillundercodedevelopment.
5.3.5 Disaster Recovery
Thelastlecturewithinthedeliverablesisalecturesurroundingdisasterrecoveryofthedata
center.Whilethistopicisonethathopefullyorganizationsneverhavetoutilize,itis
imperativethatpreventionmechanismsbeutilizedandputinplacetodefendagainstdata
loss.Becausethistopicisauniquetopicinrelationtoadatacenter,itwasincludedasahigh
focustopictobringawarenesstostudentsofitsimportanceindatacenteroperation.The
goalofthislectureistoprovidestudentswiththeabilitytodesign,implementandmanage
29
disasterrecoveryinfrastructuresthatcanbeutilizedintheeventofaccidentaldatalossor
disaster.
Theprimaryfocusofthelectureistocoverthedifferentsolutionspertainingtobackup
systemsinthedatacenter.Solutionssuchasonsitetapebackupsandoff‐sitestorage
solutionswillbediscussedprovidingthebenefitsanddrawbackstobothsituations.
Solutionscomprisedofremovablemediaotherthanthatoftapesystemswillalsobe
discussedandhowtheyfitintothedatacentertoday.Off‐sitestoragesolutionswillbe
discussedinrelationtotherotationalperiodsinwhichmediashouldbestoredoff‐site.
Discussionstodeterminewhatsolutionsstudentshavetestedwillbeengagedtopromote
informationsharing.
Anotherimportantsituationthatshouldbenotedwithindisasterrecoverysystemsisthe
wayfordatacenterstotestanddevisetestprocedurestoensurethatthesystems
implementedworkinthewaysthatarespecified.Manysystemscanbeimplementedwithin
thedatacentertopreventdisasterfromoccurring,howeverifthesesystemsdonotfunction
thewaythattheyweredevised,theyprovidethedatacenterwithnobenefit.Toassistwith
thesetestprocedures,documentationshouldbestoredinsecurelocationsoncontacts,
emergencypersonnelcontactsandguidelinesonhowtorestorethefunctionalityofthedata
centerintheeventoftotalcatastrophe.Thelecturecoversdifferentfacetsofthedata
centershouldprovideproceduresanddocumentationfortoensurethatstudentscanthen
documentandprovidemeaningfulpreventiontoanydisasterthatcouldoccurwithinthe
datacenter.
30
5.3.6 Tools
Throughoutthecoursemiscellaneoustoolswillbepresentedtothestudentstoshowthe
vastcapabilitiesthattheycanprovidefordailyadministrativefunctionsandthedesigningof
adatacenter.Thesetoolsprovidetherobustabilitytoforecastentiredatacenterpower
requirementstothesimplicityofmanagingIPaddressreservationsandsubnetinformation.
AnexampleofsomeofthetoolsutilizedwithinthecourseisshowninFigure4.Thetoolswill
bedemoedonavirtualsystemdiscussedinsection4.3.Thetoolsthatwillbeshownalsoare
opensourcetoolsorfreewareandcanbedistributedtoprovidefuturereference.The
impactofthetoolsfortheadministratoristhattheywillshortenandsimplifymanagement
tasks.Anexampleofthesetoolsisthepowercalculatorspreadsheetthatisdistributedby
thegovernmenttoassistwithcalculatingenergyneeds.Ratherthantakeinventoryofall
systemsmanually,thischecklisthelpstoestimateandforecastfutureneeds.Thisprovides
aneasierwaytoobtainquotesfromvendorsandwillcuttimetodevelopsolutionswithin
theplanningstagesofthedatacenter.
RackTables ClockingIt.com
PassSafe SharePoint
Nagios IPPlan
DellRackCalculator LinkandSpeed
TestingTools
Figure4
31
6 Conclusion
Datacentersprovidehugeamountsofrevenuefororganizationsandhostvastamountsof
datathatiscriticalfororganizationstofunction.Theirdesignandimplementationisan
importantfeaturefororganizationsthusthisresearchisimportant.Theproblemof
providingacourseofferingforstudentssurroundingdatacenterdesignandnetworkingcan
besolvedbyusingsolutionspresentedbythisresearch.Studentsandorganizationsthat
employstudentswithnetworkingbackgroundcanbenefitfromthiscourseofferingbecause
itpresentsinformationthatallowsthemtodesignandmakedecisionsbasedoffoftheirown
requirements.Informationisplacedwithineasyaccessandprovidespersonalinsightthat
canbeofassistancewhenconfrontedwithtechnologyorrequirementsthatmaynotbe
clear.Further,theresearchcoverstheneedtounderstandallfacetsofthedatacenter.
Whileunderstandingsinglesystemsisanimportantskillsettohave,understandingthe
integrationofmultiplecomponentstogetherallowstheindividualtohavebetterinsightof
howthedatacentermeshestogethertoprovideservices.Allfacetsofthedatacentermust
meshtogethertoprovidetheworkingdatacenter.Ifonesystemfails,theintegrityofthe
datacenterisnolongeroptimal.Thisiswhyredundancyisastressedimplementationwithin
thedatacenter.Withouttheredundancy,serviceswouldnotbeashighlyavailableasmany
hostedserviceshavebecometoday.Focusingontheredundancyandsolidimplementation
ofdatanetworks,electricalinfrastructure,HVACsystems,andsecuritywillprovidethe
meansforanorganizationorstudenttodesignandprovidesolutionsforacustomizeddata
center.
32
7 Future Work
Ithasbecomeatrendinrecentyearstoutilizevirtualcomponentstoprovideabetter
scalableinfrastructurewithbetterserviceprovisioning.Infutureresearch,attentionshould
befocusedontheintegrationofthesevirtualenvironmentsintothedatacenter.Ifthenew
environmentcouldsupportandpresentvirtualredundancyaswellasphysicalredundancy,it
thenwouldpresentanN+1redundancythatcouldgreatlypreventfailuresandbolsterhigh
availability.Virtualtechnologymayalsobeabletotakeadvantageofthemultiplecore
processingtechnologiesthusallowingforhighercondensationofresourcesintoasinglerack
location.Thiswillpresentnewscenariosfordatacenteradministratorstoovercome,asthe
concentrationofresourceswillrequirenewelectrical,HVAC,andnetworkresources.One
couldassumethatvirtualtechnologywouldlowerthestrainondatacenterresources,but
theconversionofagingsystemstonewerconcentratedsystemsmayactuallyincreasethe
requirementsondata,electricalandHVAC.Furthermore,itcouldbeusefultodetermine
howprimarilynetworkresourceswillbeimpactedastheserequirementswillbeeasily
condensedintosystemstoprovidededicatedlinkstovirtualmachinesandtheclusteringof
networkresourcesonsingleresources.
Anewtypeofmodulardatacentersisaimingtoprovidemobilesolutionstofulfillcustomer
needs.Thesetypesofdatacentersincorporateallnecessarycomponentsofthedatacenter
intoonelocationtoprovidevendorswiththeabilityofbuildingadatacenterinacontainer
andthenshippingthatcontainertoalocationthatprovidesmodularconnectivity.Thistopic
couldbeutilizedinfutureexperimentstodeterminethetypesofinfrastructurerequiredto
implementsuchadesignandtoalsotheimplicationitwouldhaveonthetopicsspecifiedby
thisresearch(Hamilton.)
33
8 Appendices
8.1 Course Syllabus
CourseSyllabus4055883
EnterpriseNetworking200CourseSyllabus
REMINDER: The information presented in this syllabus is subject to expansion, change, or modification during the quarter Instructor: Name Office:Bldg.70‐emailaddress:
Office Hours: M:Orbyappointment.
http://mycourses.rit.edu CourseTextandMaterialsRequiredTextbook:DouglasAlger,BuildtheBestDataCenterFacilityforyourBusiness.CiscoPress,2005.–ISBN:1‐58705‐182‐6OptionalMaterials:RobSnevely,EnterpriseDataCenterDesignandMethodology.SunPress,2002–ISBN:0‐13‐047393‐6ImportantRITDeadlines Last day of add/drop is March 16th. Last day to withdraw with a grade of “W” is May 1st – The deadline for withdrawing from a course with a W grade is the end of the 8th week of the quarter. Forms may be obtained from your department office and need your instructor’s signature. NOTE: The department policy states that a student has one quarter to challenge any grade. After that, grades cannot be challenged. CourseDescription
Thiscoursewillprovidestudentswiththeknowledgeandunderstandingtoapplydesign
34
techniquesandlogicalsegmentationtolarge‐scaleenterprisenetworks.Theoreticalconceptsoflarge‐scalenetworkswillbediscussedandstudentswillcreatedesignsbasedonthistheory.Casestudieswillbeutilizedtohelpunderstanddifferentcomponentsandtheirimportanceintheoverallscheme.Thiscoursewillprovidestudentswiththeknowledgeneededtoapplyavailabletoolsformodelingnetworkfunctionalitytodeterminetheimpactofnetworkinfrastructuremodification,devicereconfiguration,andtheimpactofnewapplicationrollouts.Design/casestudyprojectrequired.CourseObjectives
General
Attheendofthiscourse,studentswillbeabletoeffectivelyuseandimplementtoolsneededforthemanagement,designandcreationofenterprisescalenetworksanddatacenters.Studentswilldevelopprocedurestoensurethattheintegrityofthenetworkisusableattherequiredlevels.
Specifically: 1.Identifyandunderstanddifferententerprisenetworkfundamentals2.Understandthefundamentalprocessesandproceduresofplanning,anddesigningacomplexdatacenterand/orenterprisenetwork.3.Defineresourceneedsofadatacenterandnetworkneeds.4.Describethedifferentcomponentsinanenterprisedatacenterandproviderecommendationstoothersaboutthesecomponents.5.Design,plan,manageandimplemententerprisedatacentercomponents.6.Compareexistingsolutionstodeterminewhatwillfulfillthedefinedneedsofthecustomerororganization.7.Understandhierarchicalnetworksandtheirimportanceinscalableandsuccessfuldatacentersandenterprisescenarios.8.Writecleardocumentationthatallowsbettermanagementofimplementedsolutions.CourseOrganizationMyCourses
The course is organized by using RIT's myCourses platform. You are required to have a DCE account to access myCourses at mycourses.rit.edu. The myCourses is not only used by faculty to organize, create and manage the course activities and course materials, but also by students to communicate with peers and instructors, access the course content, assignments, course grades, group discussion and feedback. myCourses drop boxes are used for the submissions of homework, projects and exams. The instructor will NOT make use of the FirstClass mail and conferencing system. Please check mycourses’ announcement area at least twice a week. CourseCommunication
35
Course communication relies heavily on the content area and the discussion area of myCourses. Course syllabus, outline, lectures, assignments and exams will be posted at the content area. The weekly discussion topics will be posted in the discussion area. Students are highly encouraged to participate in the discussions by posting your comments and/or questions related to that week’s material to that area.
TheinclasslecturesarescheduledonMondaybetween6:00PM‐7:50PMforstudentswhoenrollin4055‐883‐39.TheseinclasslectureswillberecordedduringclassandpostedinMyCoursescontent‐area.However,Istronglyencouragestudentsfromsection90toattendtheselectureseitheronsiteorremotelyviaconnect.rit.eduduringthelecturetimeforliveinteraction.TheURLforattendingtheConnectconferenceswillbepostedeveryweekinMyCoursespriortothelectures.ThecoursemayalsoutilizeateleconferenceusingAdobeConnect’spremiereglobalservicesespeciallyfortheonlinestudents.Theexacttimeofthephoneconferencesalongwiththeinstructionsfordialinginwillbedeterminedandannouncedwhentheclassprogresses.WrittenExamsTheonlinewrittenexamswillincludemultiple‐choicequestions,shortanswers,casestudyanalysisandothercontentformatasappropriate.Theexamsaredesignedtobeopenbook,opennotesandopenreferences.Studentsarerequiredtofinishtheirexamsindependentlyandinthetimeframegiven.IndividualHomeworkandAssignmentsTheinstructorthroughoutthequarterwillassignindividualhomeworkandlabs.Theseassignmentsaretobecompletedindependentlyunlessnoteddifferentlybytheassignment.GroupProjects/PresentationsStudentsmaybeassignedtogroupstoworkonthegroupprojectassignmentsthroughoutthequarter.Thegroupmaymeetanddiscussproblems/issuesusingthediscussionareaand/orchatfunctionofMyCourses.Studentsarerequiredtopresenttheirgroupreportslivethroughtheclassroomorthroughconnect.rit.edu.Eachmemberofthegroupisrequiredtoparticipateinthegroupeventsactively.Peerevaluationswillbeadministeredthroughouttheprojects.Ifthereisaproblemthatappearsonthegroupevaluationswhereamemberisnotcooperatingordoingtheirfairshare,thenthatperson’sgradeonthatportionoftheprojectwillbesubjecttoreductionatthediscretionofyourinstructor.Somakesurethatyouareinvolvedandincontactwithyourothergroupmembers.
Course Topics 1. Principles of Enterprise Networks and Data centers 2. Networking Design 2.1 Cabling design and types
2.2 Network Hierarchy 2.2.1 Edge
2.2.2 Distribution
36
2.2.3 Host/Access 2.3 Network redundancy and fault tolerance
2.4 Storage Area Networks 2.5 High Availability
2.5.1 Time systems 2.5.2 EIGRP
2.5.3 IGP 2.5.4 BGP 3. Physical Design Considerations
3.1 Power 3.1.1 220v/120v
3.1.2 Electrical Segmentation 3.2 Cooling
3.2.1 HVAC systems 3.3 Flooring
3.3.1 Raised 3.3.2 Ground
3.3.3 Weight Factors 3.4 Air/Thermal Design
3.4.2 Hot/Cold Design 3.4.3 Hot/Hot Design
3.4.4 Air Sampling 4. Disaster Planning / Recovery
4.1 Backup systems 4.1.1 Onsite storage
4.1.1.1 Tape and Disk Backup systems 4.1.1.2 Removable media
4.1.2 Offsite storage 4.1.2.1 Long-term storage solutions 4.1.3 Test procedures
4.1.3.1 Policy development 4.1.4 Redundancy
4.1.4.1 Electrical 4.1.4.2 Data
4.1.4.2.1 Multiple ISP 4.1.4.2.2 Protocols (BGP, eBGP) 4.1.4.3 Storage
4.1.4.3.1 Storage Area Networks 5. Security 5.1 Physical Security
5.1.1 Surveillance 5.1.2 Safeguard systems
5.1.3 Access control systems 5.2 Heat/Water Prevention
5.2.1 Fire Suppression 5.2.2 HVAC/Air Quality control
5.3.3 Water/Moisture prevention 5.3 Personnel Safety
5.3.1 Policy management & enforcement 5.3.2 Fire codes
5.3.3 Building codes 5.3.4 Access control management 6. Management
6.1 Remote Management
37
Component Weight
Discussions 10FinalExam 25Midtermexam 25FinalPresentation 25Assignments 15
Range Grade
>=90.0% A>=80.0%&<90%
B
>=70.0%&<80.0%
C
>=60.0%&<70.0%
D
<60.0% F
6.1.1 Nagios or Network Monitoring 6.1.2 Microsoft Operations Manager/System Center Operations Manager
6.1.3 SMS and E-mail Alerting 6.2 Onsite Management
6.2.1 Organizational Tools 6.2.1.1 Rack Tables
6.2.1.2 IP Plan / Spreadsheets 6.2.2 Server and environment monitoring
6.2.2.1 Service health monitors 6.3 Scalability
6.3.1 Growth capacity planning 6.3.2 Cost Recovery modeling
7. Future Technologies Grading
Thegradingscaleusedalongwiththegradingcriteriaisasfollows:
TentativeCourseScheduleWeek Topics/Exams AssignedReading Activities1‐2 Introduction to Data Centers and
Enterprise Networks
- Network Design - Segmentation - Capacity Planning
Textbook:Chapter1,2&7Lecturenotesandonlinematerial/articles
MyCourses:IntroductionAssignment1(Week2)
3 Physical Design Considerations - Site planning
Text book: Chapter 3,4 Notes, Tools, Slides
MyCoursesDiscussion#1
4 Physical Design Considerations - Components
Text book: Chapters 5 &6
MyCourses Discussion #2
5 Security - Physical & Environmental
Text book: Chapter 13,14 Notes, Tools, Slides
Assignment 1 Due Midterm Exam
38
6 Security/Disaster Recovery - Personnel - Onsite Systems
Notes, Tools, Slides Assignment 2 MyCourses: Discussion #3
7 Disaster Recovery - Proactive vs. Reactive
Text book: Chapter 8, 9 Slidesandonline
MyCourses:Discussion#4
8 Management‐RemoteandOnsiteManagement
Textbook:Chapters10,11,15
MyCourses:Discussion#5
9 Management/FutureTechnologies
OnlineArticles Assignment2Due(endofweek9)
10 StudentTopics‐Presentations
Text book: Chapter 12
ReviewforfinalExam
11 FinalExam
Cheating Policy: Please review the departmental policy on cheating as described at http://www.it.rit.edu/policies/dishonesty.html Student Responsibilities: Please review the general student responsibilities as outlined at http://www.nssa.rit.edu/~netsyslab/Responsibilities.htm
39
8.2 Course ICC Form
B. Thomas Golisano College of Computing and Information Sciences
Department of Networking, Security, and System Administration
REVISED COURSE: 4055-883 1.0 Title: Enterprise Networking Date: 2/25/2009 Credit Hours: 4 Prerequisite(s): N/A Co-requisite(s): N/A Course proposed by: Chris Mahood 2.0 Course information:
Contact hours Maximum
students/section Classroom 2 12 Lab Active Learning/Active Learning Extended Other (specify DL) Online delivery 12
Quarter(s) offered Fall X Winter X Spring X Summer Students required to offer this course: None Students who might elect to take the course: Matriculated students in the M.S. in Networking, Security & Systems Administration.
3.0 Goals of the course: Today’s large organizations heavily rely on data services, high availability, reliability and scalability to deliver products and services to their customers. As the online audience continues to grow, more of these organizations are realizing that they must convert existing offerings into digital and online services to enhance their profits and exposure. The challenge many of these organizations face is that they either do not have an existing infrastructure that can host the services necessary, or their existing infrastructures can not accommodate the growth in technology. In the past, many organizations retrofitted rooms within their existing infrastructures to accommodate servers and network equipment. These rooms were usually
40
small and held a few computers or servers. These challenges have been met by enterprise data centers that are wholly owned, built and designed by organizations, or met by enterprise data centers that are owned by third party companies whom rent space to other organizations. Data centers present a unique difference in the way that resources or networks are traditionally allocated. This course will explore the numerous facets of a data center along with the important design steps necessary to provide a smooth and reliable operation. At the end of this course, students will be able to describe multiple different technologies used within a data center and will also be capable of designing hierarchical networks that can interact with multiple systems. This course will consist of a combination of lectures, discussions, case study analysis, and a field trip to a dynamic collocation datacenter. 4.0 Course description: This course will provide students with the knowledge and understanding to apply design techniques and logical segmentation to large-scale enterprise networks and data centers. Theoretical concepts of large-scale networks will be discussed and students will create designs based on this theory. Case studies will be utilized to help understand different components and their importance in the overall scheme of a data center and enterprise network. This course will provide students with the knowledge needed to apply available tools for designing an effective enterprise network that may be implemented in a data center. The course will also present different technologies available currently for usage in a datacenter as well as some emerging technologies aimed towards the enterprise. Design/case study project required.
Class 4, Credit 4
5.0 Possible resources (texts, references, computer packages, etc.) 5.1 Douglas, Alger. Build the Best Data Center Facility for your Business. Cisco
Press, 2005. ISBN: 1-58705-182-6. 5.2 Snevely, Rob. Enterprise Data Center Design and Methodology. Sun
Microsystems Press, 2002. ISBN: 0-13-047393-6. 6.0 Topics (outline) 6.1 Principles of Enterprise Networks and Data centers 6.2 Networking Design 6.2.1 Cabling design and types
6.2.2 Network Hierarchy 6.2.3 Network redundancy and fault tolerance
6.2.4 Storage Area Networks 6.2.5 High Availability 6.3 Physical Design Considerations
6.3.1 Power 6.3.2 Cooling 6.3.3 Flooring
41
6.3.4 Air/Thermal Design 6.4 Disaster Planning / Recovery
6.4.1 Backup systems 6.5 Security 6.5.1 Physical Security
6.5.2 Heat/Water Prevention 6.5.3 Personnel Safety 6.6 Management
6.6.1 Remote Management 6.6.2 Onsite Management 6.6.3 Scalability
6.7 Future & Emerging Technologies 7.0 Intended learning outcomes and associated assessment methods of those
outcomes 7.1 Identify and understand different enterprise network fundamentals 7.2 Understand the fundamental processes and procedures of planning, and designing a complex data center and/or enterprise network. 7.3 Define resource needs of a data center and network needs. 7.4 Describe the different components in an enterprise data center and provide recommendations to others about these components.
7.5 Compare existing solutions to determine what will fulfill the defined needs of the customer or organization. 7.6 Fully understand hierarchical networks and their importance in scalable and successful data centers or enterprise scenarios.
8.0 Program outcomes supported by this course
8.1 Program Objective 1: Design, deploy, and manage data center environments to meet the goals of an organization or customer.
8.2 Program Objective 2: Communicate and document clear information that can be used throughout an enterprise organization.
9.0 Other relevant information
None 10.0 Supplemental information None
42
Approvals: Date
Date
43
8.3 Course Lecture Slides 8.3.1 Week 2 Slides
44
45
46
47
48
49
50
51
52
8.3.2 Week 3 Slides
53
54
55
56
57
58
59
60
8.3.3 Week 4 Slides
61
62
63
64
65
66
67
68
69
70
71
8.3.4 Week 5 Slides
72
73
74
75
76
77
78
79
80
8.3.5 Week 7 Slides
81
82
83
84
85
86
87
88
89
8.4 Midterm Exam Example Questions
4055-883 Data Center Design and Networking Midterm Exam Questions
1. Please describe the hierarchy commonly referred to in data center network design.
Explain activities that occur at those layers, their function and types of devices that exist at these layers.
2. This type of cabling design provides a hierarchy of cabling:
a. Cat 5e b. Distributed c. Aggregation d. Direct e. None of the above
3. _____________ provides the ability for systems to remain online while another
portion of the network may be failed.
4. This type of fiber optic cable provides faster data transmission speeds over longer distances.
a. Multimode b. Single mode c. Mega mode d. Dual pair
5. Scenario: The organization you work for has purchased new switches for the data
center that run only on DC power. Your existing switches utilize AC power and you do not have any DC power available within the data center. Provide a solution to utilize these switches within the data center and provide justification for your answer.
6. Name and explain two benefits to utilizing DC power within the data center.
7. Name and explain two benefits to utilizing AC power within the data center.
90
8. Scenario: The organization that you are designing a data center for is comparing raised floor versus flat floor installation. The maximum height of the raised floor that they could install would be 17 inches due to building code. Based off of this information please provide a recommendation for raised flooring or flat flooring. Discuss reasons for your choice and reasons why the other choice was not made.
9. Ensure that UPS devices or PDUs are not utilized above _____ percent
10. Which of the following cooling technologies would benefit a data center in Rochester
the best? a. Air Cooled b. Water Cooled c. Glycol d. Fan Based e. None of the above
11. Normal humidity operating range is from 20% to ____% humidity
12. An air __________ is an enclosed space utilized in data centers to provide airflow.
91
8.5 Midterm Exam Example Questions
4055-883 Data Center Design and Networking
Final Exam Questions
1. Scenario: You are consulting for an organization where the finalization of security designs is occurring for a new data center and the organization is looking to you to provide recommendations on physical security. The organization is a web hosting organization that processes credit card payments through a series of e-commerce sites. The organization would like to implement a maximum of three (3) different physical security technology solutions to provide access control over not only the data center in general, but to the server room as well. Please provide three different recommendations of physical security to protect the payment card systems and justify the reasons to recommending these systems.
2. What is usually the most susceptible location to physical break-ins of a data center:
a. Back door b. Front door c. Loading dock d. Roof access e. None of the above
3. _____________ prevent someone from propping a door open and allowing free access
to a server room or maintenance room.
4. This tool can be utilized to check air flow patterns and where hot pockets are forming within the data center.
a. Radar gun b. Manometer c. Multimeter d. Bilometer
5. Scenario: In the data center you work within, your building monitoring system
continually warns of smoke particles in the environment, however all systems have been tested to be clean. This occurs frequently when other employees are located within the server room or maintenance is being performed. Discuss what steps should be taken to remedy this situation. Justify your answers.
92
6. Name two techniques to prevent water damage to the data center.
7. Which of the following organizations provides a standard for the protection of computer equipment?
a. National Electrical Contractors Association b. Institute of Electrical and Electronics Engineers c. National Fire Protection Association d. Telecommunications Industry Association e. Electronics Industries Alliance f. None of the above g. All of the above
8. This protocol can provide redundancy and failover capabilities at the edge layer
a. OSPF b. RIP c. BGP d. IGP e. None of the above
9. This type of tape library system has historically provided the most interior space with
the least footprint in the data center a. Helical b. Linear c. Tower d. Hexagon e. None of the above
10. Scenario: A data center is being implemented in New York City, NY. The
organization also owns an existing data center located in Seattle, WA. The organization wants to create a site replication solution to provide critical files and data base backups to the existing data center in Seattle. Explain two different solutions that the organization may utilize to connect and transfer files of these types to the remote location and how they can easily replicate their data center at this location. You can assume that their topology at both locations will be identical and that no configuration of systems will be necessary.
93
9 References: "80PlusProgram."The80PLUSProgram.13Jan.2009<http://80plus.org/80what.htm>."AC,DCandElectricSignals."ElectronicsClub.Ed.JohnHewes.4June2007.13Jan.2009
<http://www.kpsec.freeuk.com>.Al‐Fares,Mohammad,AlexanderLoukissas,andAminVahdat."AScalable,
CommodityDataCenterNetworkArchitecture."Thesis.AScalable,CommodityDataCenterNetworkArchitectureACM(2008):63‐74.
Alger,Douglas.BuildtheBestDataCenterFacilityforYourBusiness.NewYork: CiscoPress,2005.Arregoces,Mauricio,andMaurizioPortolani.DataCenterFundamentals.NewYork: CiscoP,2003.Berkowitz,HowardC."DefininganArchitecture."DesigningRoutingandSwitching ArchitecturesforEnterpriseNetworks.ByHowardC.Berkowitz.GrandRapids: NewRiders,1998.Blazek,Michele,HuiminChong,WoonsienLoh,andJonathanG.Koomey.DataCenters
Revistited:AssessmentoftheEnergyImpactofRetrofitsandTechnologyTrendsinaHigh‐DensityComputingFacility.Thesis.ASCE,2004.JournalofInfrastructureSystems,2004.
Caldwell,Chris."PowerSupplies:MakingtheLeaptoHigherEfficiency."Presentationto APEC.Texas,Austin.4Feb.2009.Creswell,JohnW.ResearchDesign.ThousandOaks:SAGEPublications,2003.DataCenterEnergyBenchmarkingCaseStudy.Oakland:LawrenceBerkelyNational Laboratory,2007. "DataCenterSecurity."ePaymentsNetwork.20Jan.2009.20Jan.2009
<http://www.epaymentsnetwork.com/DataCenter/tabid/124/Default.aspx>.DiMinico,Chris."TelecommunicationsInfrastructureStandardforDataCenters."TelecommunicationsInfrastructureStandardforDataCenters.25Jan.2009.IEEE.Dunlop,Jodi."DataCenterCaseStudiesFocusofASHRAEBook."ASHRAE.7May2008.
ASHRAE.20Jan.2009<http://www.ashrae.org/pressroom/detail/16715>.
94
Hamilton,JamesR.AnArchitectureforModularDataCenters.Thesis.Microsoft,2008.
Redmond:MicrosoftCorporation,2008.IEEE.13Jan.2009<http://www.ieee802.org/3/hssg/public/nov06/diminico_01_1106.pdf>.
Kerr,Devon.NetworkHierarchy.Kovacs,Brian.DataCenterOutsourcingCaseStudy.Rep.Carmel,IN:NFrame,2004."MainframeHistory."MainframeHistory.IBM.4Feb.2009<http://www‐
03.ibm.com/ibm/history/exhibits/mainframe/mainframe_intro.html>.Mitchell,RobertL."CaseStudy:WellsFargo'sFreeDataCenterCoolingSystem."Computer
World.5Nov.2007.4Feb.2009<http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=299301&pageNumber=2>.
Patrizio,Andy."CleanYourDatacenter,CutYourElectricBill."InternetNews.29Jan.2009.4
Feb.2009<www.internetnews.com/hardware/article.php/3799451>.Rasmussen,Neil.ACvs.DCPowerDistributionforDataCeters.Publication.American
PowerConversion,2007.Oppenheimer,Priscilla.Top‐downNetworkDesign.NewYork:CiscoP,2004.GoogleBooks.Google.12Jan.2009<http://books.google.com/books?id=YKWfuGGSmXMC&pg=PA141&lpg=PA141&dq=access+layer+hierarchical&source=web&ots=HklxUBvIep&sig=M8h96f9fK_9rBLmt‐nb4sWZvVCE&hl=en&ei=Q5qPSfW4NJ6DtweQxtGDCw&sa=X&oi=book_result&resnum=2&ct=result#PPA143,M1>.Sharma,RatneshK.,RockyShih,CullenBash,andChandrakantPatel."OnBuilding NextGenerationDataCenters."ACM(2008).Snevely,Rob.EnterpriseDataCenterDesignandMethodology.UpperSaddleRiver: PrenticeHallPTR,2002.Sun,Jimeng,EvanHoke,JohnD.Strunk,GregoryR.Ganger,andChristos
Faloutsos."IntelligentSystemMonitoringonLargeClusters."Thesis.IntelligentSystemMonitoringonLargeClustersACM(2007):47‐52.
TIA‐942DataCenterStandardsOverview.Publication.Minneapolis:ADC,2006.
"WhatisNetworkCabling."WhatisNetworkCabling.14May2008.UniversityofSouthernFlorida.12Jan.2009<http://fcit.usf.edu/network/chap4/chap4.htm>.