CapacityPlanningintheCloudanini2alpeekintoanewworld

CMG08PanelSessionAdrianCockcro=‐Ne@lix

PaulStrong–eBay

WhatisCloudCompu2ng?hGp://www.slideshare.net/StuC/cloud‐compu2ng‐for‐architects‐qcon‐2008‐tutorial‐presenta2on

WhatisCapacityPlanning

•  WecareaboutCPU,Memory,NetworkandDiskresources,andApplica2onresponse2mes

•  Weneedtoknowhowmuchofeachresourceweareusingnow,andwilluseinthefuture

•  Weneedtoknowhowmuchheadroomwehavetohandlehigherloads

•  Wewanttounderstandhowheadroomvaries,andhowitrelatestoapplica2onresponse2mesandthroughput

CapacityPlanningNorms

•  Capacityisexpensive•  Capacitytakes2metobuyandprovision

•  Capacityonlyincreases,can’tbeshrunkeasily•  Capacitycomesinbigchunks,paidupfront

•  Planningerrorscancausebigproblems

•  Systemsareclearlydefinedassets

•  Systemscanbeinstrumentedindetail

CapacityPlanninginClouds

•  Capacityisexpensive•  Capacitytakes2metobuyandprovision

•  Capacityonlyincreases,can’tbeshrunkeasily•  Capacitycomesinbigchunks,paidupfront

•  Planningerrorscancausebigproblems

•  Systemsareclearlydefinedassets

•  Systemscanbeinstrumentedindetail

CapacityisexpensivehGp://aws.amazon.com/s3/&hGp://aws.amazon.com/ec2/

•  Storage(AmazonS3)–  $0.150perGB–first50TB/monthofstorageused–  $0.120perGB–storageused/monthover500TB

•  DataTransfer(AmazonS3)–  $0.100perGB–alldatatransferin–  $0.170perGB–first10TB/monthdatatransferout–  $0.100perGB–datatransferout/monthover150TB

•  Requests(AmazonS3StorageaccessisviahGp)–  $0.01per1,000PUT,COPY,POST,orLISTrequests–  $0.01per10,000GETandallotherrequests–  $0perDELETE

•  CPU(AmazonEC2)–  Small(Default)$0.10perhourtoExtraLarge$0.80perhour

•  Network(AmazonEC2)–  Inbound/Outboundaround$0.10perGB

Capacitycomesinbigchunks,paidupfront

•  Capacitytakes2metobuyandprovision– Nominimumprice,monthlybilling–  “AmazonEC2enablesyoutoincreaseordecreasecapacitywithinminutes,nothoursordays.Youcancommissionone,hundredsoreventhousandsofserverinstancessimultaneously”

•  Capacityonlyincreases,can’tbeshrunkeasily–  Payforwhatisactuallyused

•  Planningerrorscancausebigproblems–  Sizeonlyforwhatyouneednow

Systemsareclearlydefinedassets

•  Youarerunningina“stateless”mul2‐tenantedvirtualimagethatcandieorbetakenawayandreplacedatany2me

•  Youdon’tknowexactlywhereitis•  Youcanchoosetolocate“USA”or“Europe”•  Youcanspecifyzonesthatwillnotsharecomponentstoavoidcommonmodefailures

Systemscanbeinstrumentedindetail

•  Needtousestatelessmonitoringtools•  e.g.Ganglia–automa2cconfigura2on– Mul2castreplicatedmonitoringstate

– Noneedtopre‐definemetricsandnodes

December11,2008 AdrianCockcro=andMarioJauvin

Ganglia–www.ganglia.info

•  WebbasedRRDtoolGUI•  Goodmanagementofclustersofsystemsanddevices,useful

forhundredstothousandsofnodesinahierarchyofclusters•  Providesmanysummarysta2s2cplotsatclusterleveland

collectsdetailedconfigura2ondata•  XMLbaseddatarepresenta2on•  Useslowoverheadnetworkprotocol(mul2castorunicast)•  IncommonuseathundredsoflargeHPCGridsites,lessvisibly

inuseatsomelargecommercialsites•  hGp://wiki.apache.org/hadoop/AmazonEC2includesganglia

asastandardfeatureofHadooponEC2.

December11,2008 AdrianCockcro=andMarioJauvin

December11,2008 AdrianCockcro=andMarioJauvin

December11,2008 AdrianCockcro=andMarioJauvin