leveraging elastic web scale computing with aws

LeveragingElasticWeb-ScaleComputingwithAWS

Shiva Narayanaswamy

EC2BasicsInstanceLifecycleEC2InstanceTypesUsingAmazonMachineImagesBootstrappingEC2InstancesMonitoring EC2withCloudWatchAutoscaling

EC2 Basics

Virtual Servers in the Cloud• One instance to thousands of instances• In any public AWS region• Create, start, stop, configure, monitor as desired• Install any software: web, business, client/server,

batch processing• Pay only for capacity you use• Variety of cost models AmazonEC2

EC2 Basics: costmodels

On-Demand Reserved Spot Dedicated

Payupfrontinexchangeforhourlypricesthatare50-75%lowerthan

On-Demand

Payforcomputecapacitybythehour.Nolong-term

commitments

BidforunusedAmazonEC2capacity

LaunchinstancesinVPCondedicatedcustomerhardware

Customerscancombinemultiplepurchasetypestooptimizepricingbasedoncurrentandforecastcapacityneeds.

Spikyworkloads Committedutilization Time-insensitiveworkloads Highlysensitiveworkloads

Provisioning and Lifecycle

• Create -> Start -> Stop -> Terminate• Manually in console• Automate via API (or other tools)• Automatically based on demand

(demand curve)

Instance TypesGPU

EnabledGeneralPurpose

StorageandIOOptimized

ComputeOptimized

MemoryOptimized

M3 C3 I2

CG1M1 C1 CR1CC2 HI1 HS1

G2

M3 C3 I2 HS1

M2

R3G2

AddedInstance

Types

Amazon Machine Images

Yourmachineimages

AMIsyouhavecreatedfromEC2instancesCanbekeptprivateorsharedwithother

accounts

Amazonmaintained

SetofLinuxandWindowsimagesKeptuptodatebyAmazonineach

region

Communitymaintained

ImagespublishedbyotherAWSusersManagedandmaintainedbyMarketplace

partners

Amazon Machine Images

Bootstrapping:metadataanduserdata

• EveryEC2Instancehasaccesstolocalinstancemetadataanduserdataservice

Instance request

Userdata

Instance

Meta-dataservice

Bootstrapping:metadata anduserdata

• Metadata:immutableinformationabouttheinstance• AccessiblefromwithintheinstanceviaHTTPathttp://169.254.169.254/latest/meta-data/

• Script(s)oninstancemayretrieveusefulinformationabouttheinstance,suchas:

• Hostname• AMIID• InstanceID• Public/PrivateDNS• AvailabilityZone


• UserData:passupto16KBoftexttoaninstanceonlaunch• AccessiblefromwithintheinstanceviaHTTPathttp://169.254.169.254/latest/user-data/

• Textcanbeparsedbyscriptoninstanceandusedtoconfigurethemachine

Custom script on AMI

(script_runner.py) fetches userdata,

parses it, and configures EC2 Instance

on boot


• CloudInit executesUserDataonfirstbootifUserDatabeginswith:• #!(Linux)• <script>(Windows;technically,EC2Config,notCloudInit,doesthis)

• CloudInit isinstalledonAmazonLinux,Ubuntu,andRHELAMIs• EC2ConfigisinstalledonWindowsServerAMIs• Bothmaybeinstalledonotherdistributionsviaapackagerepoorsource

Bootstrapping:UserData andCloudInit

• UserDatatoinstallApacheandMySQLonboot,andattachanEIP:

#!/bin/bash

# Install Apache, PHP, and MySQLyum install –y httpd mysql-server

# Attach an Elastic IP to this instanceec2-associate-address \

23.34.45.56 \-i $(curl http://169.254.169.254/latest/meta-data/instance-id)

Bootstrapping:UserData andCloudInit

Bootstrapping

BakeanAMI

Startaninstance

Configuretheinstance

CreateanAMIfromyourinstance

StartnewonesfromtheAMI

Configuredynamically

Launchaninstance

Usemetadataserviceandcloud-init toperformactionson

instancewhenitlaunches

Useconfig managementtoolslikePuppet/Chef/Opsworks

vs

Bootstrapping

BakeanAMI Configuredynamically

Buildyourbaseimagesandsetupcustominitialisation

scripts

Maintainyour‘golden’base

Usebootstrappingtopasscustominformationinandperformpostlaunchtasks.

+

Sweetspot

Bootstrapping: AMIs

Linux

JEE

YourCode

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

JavaAppStack

Example full stack required to run your application.

Let’s use the 3 bootstrapping

techniques

Bootstrapping: AMIbake

Fully-functional AMI is pre-build and ready to launch from the AMI inventory

InventoryofAMIs

Linux

JEE

YourCode

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

AmazonEC2

Linux

JEE

YourCode

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Linux

JEE

YourCode

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Linux

JEE

YourCode

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Linux

JEE

YourCode

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

JavaAMI

Bootstrapping: Configuredynamically

Base OS AMI

An AMI with minimal components (OS, J2EE, and Chef/Puppet) is launched.

All configuration occurs via Chef/Puppet after instance launch

InventoryofAMIs

AmazonEC2

OSAMI

Fetchonboot

Linux

JEE

YourCode

S3

Hibernate

Tomcat

Log4J

Spring

Struts

Apache

Linux

JEE

Linux

JEEChef/Puppet

Chef/Puppet

scripts

Bootstrapping: Sweetspot

Partially-configured AMI

A “Golden Image” is launched, with scripts fetching/installing app code

and other supporting components on boot

InventoryofAMIs

AmazonEC2

JavaAMI

YourCode

S3

Log4J

Spring

Struts

Linux

JEE

Hibernate

Tomcat

Apache

Fetchonboot

Fetchonboot

Linux

JEE

Hibernate

Tomcat

Apache

Linux

JEE

Hibernate

Tomcat

Apache

Linux

JEE

Hibernate

Tomcat

Apache

Linux

JEE

Hibernate

Tomcat

Apache

Whydothis?

AutomationLessfingers,lessmistakes

AvailabilityDrivehigher

availabilitywithself-healing

SecurityInstanceslockeddownbydefault

FlexibleShell,

Powershell,CloudFormation,Chef,Puppet,OpsWorks

ScaleManagelargescale

deploymentsanddriveautoscaling

EfficiencyAuditandmanageyourestatewithlesstime&effort

Do Don’t

Somedosanddon’ts

UseIAMroles

Gokeylessifyoucan

StrikeabalancebetweenAMIanddynamicbootstrapping

PutyourAPIaccesskeysintocode(andthenpublishtoGIT)orbake

intoAMIs(andshare)

L

Monitoring EC2 with CloudWatch

TypesofScaling

• VerticalScaling• Changinginstancesize• IncreasingEBSCapacity

• HorizontalScaling• Adding/removinginstances• ELB• Autoscaling

r3.8xlarge

c3.2xlarge

m3.medium

m3.medium m3.medium m3.medium

m3.medium m3.medium m3.medium

VerticalScaling

• DifferentEC2instancetype• Highmemoryinstances• HighCPUinstances• HighI/Oinstances• Highstorageinstances

• Easytochangeinstancesizes• Willhitanendpointeventually• Requiresinstancetobestopped

r3.8xlarge

c3.2xlarge

m3.medium

TraditionalITUsagePatterns

OnandOff FastGrowth

Variablepeaks Predictablepeaks

TraditionalITUsagePatterns

OnandOff FastGrowth


PoorService

WASTE

CloudITUsagePatterns(AutoScaling)

OnandOff FastGrowth


AutoScaling• Automaticresizingofcomputeclustersbasedondemand• Defineminimumandmaximumnumberofinstances• Definewhenscalingoutandinoccurs• UsemetricscollectedinAmazonCloudWatchtodrivescaling• RunAutoScalingforOn-DemandandSpotinstancetypes• ItsFree! Amazon

CloudWatch

UsageMetrics

ScalingInstructions

AutoScalingGroup

QueueMetrics

AutoScaling

DescribeswhatAutoScalingwillcreatewhenadding

Instances- Similartoec2-run-instancesAPIcommand

AMIInstanceTypeSecurityGroupInstanceKeyPair

Onlyoneactivelaunchconfigurationatatime

AutoScalingwillterminateinstanceswitholdlaunch

configurationfirstrollingupdate

AutoScalingmanagedgroupingofEC2instances

Automatichealthchecktomaintainpoolsize

Automaticallyscalethenumberofinstancesbypolicy– Min,Max,

Desired

AutomaticIntegrationwithELB

Automaticdistribution&balancingacrossAZs

ParametersforperforminganAutoScalingaction

ScaleUp/Downandbyhowmuch

ChangeInCapacity (+/- #)ExactCapacity (#)

ChangeInPercent (+/- %)

CoolDown(seconds)

PolicycanbetriggeredbyCloudWatch events

Launch Configuration Auto-Scaling Group Auto-Scaling Policy

Scalingplan

• Scalebasedondemand• Manualscaling• Scalebasedonschedule• Maintaincurrentinstancelevelsatalltime

AutoScaling

AutoScalingLifecycles

Autoscaling

Availability Zone A Availability Zone B

Autoscaling: AutoScalingGroup

Latency

CloudWatchAutoScaling

ELB

Auto scaling Group

Autoscaling: ELB+CloudWatch

• ToolsUsed:• CloudFormation script–

• Createamulti-AZ,loadbalancedandAutoScaledsamplewebsiterunningonanApacheWebServer(m1.small).TheapplicationisconfiguredtospanallAvailabilityZonesintheregionandisAuto-ScaledbasedontheCPUutilizationofthewebservers.

• BeeswithMachineGuns– Performancetestingtool• Acloudformation scriptthatspinsupadistibuted performancetestingtoolbasedonapacheeb tool.ThistoolwillhittheELBwith1000’sofconcurrentrequestsforatotalof100’softhousandsofrequest,thusloadingthewebserverbehindtheELB.

• Expectedresult• TheApachewebserverwillscaletoservetrafficwithoutanycustomerimpact.

Autoscaling: DEMO

• CloudFormation script(Autoscalingapachewebserver)• Auto-scalinggroupconfiguration:

• Min:1• Max:3• Cooldown:300

• ScalingPolicies:• ScalingUp:

• CPUUtilization>20%for1consecutiveperiodof60seconds• Action:Add1instance• Thenwait:60secondsbeforenextoperation

• ScalingDown:• CPUUtilization<10%for2consecutiveperiodsof60seconds• Action:Remove1instance• Thenwait:60secondsbeforenextoperation

• BeeswithMachineguns(NASTY)

Demo Information

Autoscaling isn’tonesizefitsall

• Choosetherightmetrics• CPUUsage• QueueDepth• Numberofconcurrentusers

• Scaletooaggressively• Overprovisioning:increasescosts• Bounciness:Addmorethanyouneedandhavetopartiallyscalebackshortlyafterscalingup,increasingcosts.

• Scaletootimidly• Poorperformance• Outagesduetolackofcapacity

• Scaleoutearly/Scaleinslowly

Stopdoingthese:Provisioningandfixingservers

TreatingcomputeasphysicalthingsThinkingofcomputeasafinitecommitment

andstartdoingthese

SecurityBuildsystemssecureby

default

ElasticityStatelessautoscaling

applications

ReplacenotfixBuildfromscratch,don’t

fixsomethingUnconstrainedSaygoodbyeto

traditionalcapacityplanning

BecostawareTagresources,playwith

instancetypes

AutomationCreateinstanceswhenyouneedthem,drop

themwhennot

What’smore?

• Attach/DetachInstancesfromAutoScalingGroups• PlaceinstancesintoStandbyStatetoTroubleshoot• HoldinstancesinPendingstateforinstallingsoftware/retrievelogs• CreateanAutoScalingGroup/LaunchConfigurationbasedonarunninginstance

• AutoscalingLifecyclehooks

Questions?

leveraging elastic web scale computing with aws

Internet