leveraging elastic web scale computing with aws

Leveraging Elastic Web-Scale Computing with AWSShiva Narayanaswamy

EC2 BasicsInstance LifecycleEC2 Instance TypesUsing Amazon Machine ImagesBootstrapping EC2 InstancesMonitoring EC2 with CloudWatchAutoscaling

EC2 Basics

Virtual Servers in the Cloud• One instance to thousands of instances• In any public AWS region• Create, start, stop, configure, monitor as desired• Install any software: web, business, client/server, batch processing• Pay only for capacity you use• Variety of cost models Amazon EC2

EC2 Basics: cost models

On-Demand Reserved Spot Dedicated

Pay upfront in exchange for hourly prices that are 50-75% lower than

On-Demand

Pay for compute capacity by the hour. No long-term

commitments

Bid for unused Amazon EC2 capacity

Launch instances in VPC on dedicated customer hardware

Customers can combine multiple purchase types to optimize pricing based on current and forecast capacity needs.

Spiky workloads Committed utilization Time-insensitive workloads Highly sensitive workloads

Provisioning and Lifecycle

• Create -> Start -> Stop -> Terminate• Manually in console• Automate via API (or other tools)• Automatically based on demand

(demand curve)

Instance TypesGPU

EnabledGeneral Purpose

Storage and IOOptimized

ComputeOptimized

Memory Optimized

M3 C3 I2

CG1M1 C1 CR1CC2 HI1 HS1

G2

M3 C3 I2 HS1

M2

R3G2

Added Instance

Types

Amazon Machine Images

Your machine images

AMIs you have created from EC2 instancesCan be kept private or shared with other

accounts

Amazon maintained

Set of Linux and Windows imagesKept up to date by Amazon in each

region

Community maintained

Images published by other AWS usersManaged and maintained by Marketplace

partners

Amazon Machine Images

Bootstrapping: metadata and userdata

• Every EC2 Instance has access to local instance metadata and userdata service

Instance request

User data

Instance

Meta-data service


• Metadata: immutable information about the instance• Accessible from within the instance via HTTP at

http://169.254.169.254/latest/meta-data/• Script(s) on instance may retrieve useful information about the

instance, such as:• Host name• AMI ID• Instance ID• Public/Private DNS• Availability Zone

http://169.254.169.254/latest/meta-data/



• User Data: pass up to 16KB of text to an instance on launch• Accessible from within the instance via HTTP at

http://169.254.169.254/latest/user-data/• Text can be parsed by script on instance and used to configure the

machine





Custom script on AMI

(script_runner.py) fetches userdata,

parses it, and configures EC2 Instance

on boot


• CloudInit executes UserData on first boot if UserData begins with:• #! (Linux)• <script> (Windows; technically, EC2Config, not CloudInit, does this)

• CloudInit is installed on Amazon Linux, Ubuntu, and RHEL AMIs• EC2Config is installed on Windows Server AMIs• Both may be installed on other distributions via a package repo or

source

Bootstrapping: UserData and CloudInit

• UserData to install Apache and MySQL on boot, and attach an EIP:

#!/bin/bash

# Install Apache, PHP, and MySQL yum install –y httpd mysql-server # Attach an Elastic IP to this instance ec2-associate-address \ 23.34.45.56 \

-i $(curl http://169.254.169.254/latest/meta-data/instance-id)

Bootstrapping: UserData and CloudInit

Bootstrapping

Bake an AMI

Start an instance

Configure the instance

Create an AMI from your instance

Start new ones from the AMI

Configure dynamically

Launch an instance

Use metadata service and cloud-init to perform actions on

instance when it launches

Use config management tools like Puppet/Chef/Opsworks

vs

Bootstrapping

Bake an AMI Configure dynamically

Build your base images and setup custom initialisation

scripts

Maintain your ‘golden’ base

Use bootstrapping to pass custom information in and perform post launch tasks.

+

Sweet spot

Bootstrapping: AMIs

Linux

JEE

Your Code

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Java App Stack

Example full stack required to run your

application.

Let’s use the 3 bootstrapping

techniques

Bootstrapping: AMI bake

Fully-functional AMI is pre-build and

ready to launch from the AMI inventory

Inventory of AMIs

Linux

JEE

Your Code

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Amazon EC2

Linux

JEE

Your Code

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Linux

JEE

Your Code

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Linux

JEE

Your Code

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Linux

JEE

Your Code

Log4J

Spring

Hibernate

Struts

Tomcat

Apache

Java AMI

Bootstrapping: Configure dynamically

Base OS AMI

An AMI with minimal components (OS,

J2EE, and Chef/Puppet) is launched.

All configuration occurs via

Chef/Puppet after instance launch

Inventory of AMIs

Amazon EC2

OS AMI

Fetch on boot

Linux

JEE

Your Code

S3

Hibernate

Tomcat

Log4J

Spring

Struts

Apache

Linux

JEE

Linux

JEE

Chef/Puppet

Chef/Puppet

scripts

Bootstrapping: Sweet spot

Partially-configured AMI

A “Golden Image” is launched, with

scripts fetching/installing app code

and other supporting components on

boot

Inventory of AMIs

Amazon EC2

Java AMI

Your Code

S3

Log4J

Spring

Struts

Linux

JEE

Hibernate

Tomcat

Apache

Fetch on boot

Fetch on boot

Linux

JEE

Hibernat

e

Tomcat

Apache

Linux

JEE

Hibernat

e

Tomcat

Apache

Linux

JEE

Hibernat

e

Tomcat

Apache

Linux

JEE

Hibernat

e

Tomcat

Apache

Why do this?

AutomationLess fingers, less mistakes

AvailabilityDrive higher

availability with self-healing

SecurityInstances locked down by default

FlexibleShell,

Powershell, CloudFormation,

Chef, Puppet, OpsWorks

ScaleManage large scale

deployments and drive autoscaling

EfficiencyAudit and manage your estate with less time & effort

Do Don’t

Some dos and don’ts

Use IAM roles

Go keyless if you can

Strike a balance between AMI and dynamic bootstrapping

Put your API access keys into code (and then publish to GIT) or bake

into AMIs (and share)

Monitoring EC2 with CloudWatch

Types of Scaling• Vertical Scaling

• Changing instance size• Increasing EBS Capacity

• Horizontal Scaling• Adding / removing instances• ELB• Autoscaling

r3.8xlarge

c3.2xlarge

m3.medium

m3.medium m3.medium m3.medium

m3.medium m3.medium m3.medium

Vertical Scaling• Different EC2 instance type

• High memory instances• High CPU instances• High I/O instances• High storage instances

• Easy to change instance sizes• Will hit an endpoint eventually• Requires instance to be stopped

r3.8xlarge

c3.2xlarge

m3.medium

Traditional IT Usage Patterns

On and Off Fast Growth

Variable peaks Predictable peaks

Traditional IT Usage Patterns



PoorService

WASTE

Cloud IT Usage Patterns (Auto Scaling)



Auto Scaling• Automatic resizing of compute clusters based on demand • Define minimum and maximum number of instances• Define when scaling out and in occurs• Use metrics collected in Amazon CloudWatch to drive scaling• Run Auto Scaling for On-Demand and Spot instance types• Its Free! Amazon

CloudWatch

UsageMetrics

ScalingInstructions

Auto Scaling Group

QueueMetrics

Auto Scaling

Describes what Auto Scaling will create when adding

Instances - Similar to ec2-run-instances API command

AMIInstance Type

Security GroupInstance Key Pair

Only one active launch configuration at a time

Auto Scaling will terminate instances with old launch

configuration firstrolling update

Auto Scaling managed grouping of EC2 instances

Automatic health check to maintain pool size

Automatically scale the number of instances by policy – Min, Max,

Desired

Automatic Integration with ELB

Automatic distribution & balancing across AZs

Parameters for performing an Auto Scaling action

Scale Up/Down and by how much

ChangeInCapacity (+/- #)ExactCapacity (#)

ChangeInPercent (+/- %)

Cool Down (seconds)

Policy can be triggered by CloudWatch events

Launch Configuration Auto-Scaling Group Auto-Scaling Policy

Scaling plan

• Scale based on demand• Manual scaling• Scale based on schedule• Maintain current instance levels at all time

Auto Scaling

Auto Scaling Lifecycles

Autoscaling

Availability Zone A Availability Zone B

Autoscaling: Auto Scaling Group

Latency

CloudWatchAuto Scaling

ELB

Auto scaling Group

Autoscaling: ELB + CloudWatch

• Tools Used:• CloudFormation script –

• Create a multi-AZ, load balanced and Auto Scaled sample web site running on an Apache Web Server (m1.small). The application is configured to span all Availability Zones in the region and is Auto-Scaled based on the CPU utilization of the web servers.

• Bees with Machine Guns – Performance testing tool• A cloudformation script that spins up a distibuted performance testing tool based on

apache eb tool. This tool will hit the ELB with 1000’s of concurrent requests for a total of 100’s of thousands of request, thus loading the web server behind the ELB.

• Expected result• The Apache web server will scale to serve traffic without any customer impact.

Autoscaling: DEMO

• CloudFormation script (Auto scaling apache web server)• Auto-scaling group configuration:

• Min: 1• Max: 3• Cooldown: 300

• Scaling Policies:• Scaling Up:

• CPU Utilization > 20% for 1 consecutive period of 60 seconds• Action: Add 1 instance• Then wait: 60 seconds before next operation

• Scaling Down:• CPU Utilization < 10% for 2 consecutive periods of 60 seconds• Action: Remove 1 instance• Then wait: 60 seconds before next operation

• Bees with Machine guns(NASTY)

Demo Information

Autoscaling isn’t one size fits all• Choose the right metrics

• CPU Usage• Queue Depth• Number of concurrent users

• Scale too aggressively• Overprovisioning: increases costs• Bounciness: Add more than you need and have to partially scale back shortly after scaling

up, increasing costs.

• Scale too timidly• Poor performance• Outages due to lack of capacity

• Scale out early / Scale in slowly

Stop doing these:Provisioning and fixing servers

Treating compute as physical thingsThinking of compute as a finite commitment

and start doing these

SecurityBuild systems secure by

default

ElasticityStateless autoscaling

applications

Replace not fixBuild from scratch, don’t

fix somethingUnconstrained

Say goodbye to traditional capacity

planning

Be cost awareTag resources, play with

instance types

AutomationCreate instances when you need them, drop

them when not

What’s more?• Attach / Detach Instances from Auto Scaling Groups• Place instances into Standby State to Troubleshoot• Hold instances in Pending state for installing software / retrieve logs• Create an Auto Scaling Group / Launch Configuration based on a

running instance• Auto scaling Lifecycle hooks

Questions?

leveraging elastic web scale computing with aws

Internet