t1 – architecting highly available applications on aws

80
AWS Summit 2014 Architecting Highly Available Applications on AWS Alex Sinner Solutions Architect @alexsinner

Upload: amazon-web-services

Post on 15-Jan-2015

1.638 views

Category:

Technology


2 download

DESCRIPTION

 

TRANSCRIPT

Page 1: T1 – Architecting highly available applications on aws

AWS Summit 2014

Architecting Highly Available Applications on AWS

Alex Sinner Solutions Architect @alexsinner

Page 2: T1 – Architecting highly available applications on aws

Architecting Highly Available Applications on AWS

•  ME: Alex Sinner – AWS Solutions Architect •  YOU: Here to learn more about running highly

available, scalable Applications on AWS •  TODAY: about best practices and things to think

about when building for large scale

Page 3: T1 – Architecting highly available applications on aws

Going from 1 User to >10 Millions

Page 4: T1 – Architecting highly available applications on aws

So how do we scale?

Page 5: T1 – Architecting highly available applications on aws

Hi, I have NO IDEA what I am doing!!

Page 6: T1 – Architecting highly available applications on aws
Page 7: T1 – Architecting highly available applications on aws

a lot of things to read

Page 8: T1 – Architecting highly available applications on aws

not where we want to start

a lot of things to read

Page 9: T1 – Architecting highly available applications on aws

Auto Scaling is a tool. It’s not the single thing that

fixes everything.

Page 10: T1 – Architecting highly available applications on aws

What do we need first?

Page 11: T1 – Architecting highly available applications on aws

Some basics…

Page 12: T1 – Architecting highly available applications on aws

Regions US-WEST (Oregon)

EU-WEST (Ireland) ASIA PAC (Tokyo)

US-WEST (N. California)

SOUTH AMERICA (Sao Paulo)

US-EAST (Virginia)

AWS GovCloud (US)

ASIA PAC (Sydney)

ASIA PAC (Singapore)

CHINA (Beijing)

Page 13: T1 – Architecting highly available applications on aws

Availability Zones US-WEST (Oregon)

EU-WEST (Ireland) ASIA PAC (Tokyo)

US-WEST (N. California)

SOUTH AMERICA (Sao Paulo)

US-EAST (Virginia)

AWS GovCloud (US)

ASIA PAC (Sydney)

ASIA PAC (Singapore)

CHINA (Beijing)

Page 14: T1 – Architecting highly available applications on aws

Compute  Storage  &  Content  Delivery  

AWS  Global  Infrastructure  

Database  

App  Services  

Deployment  &  Administra=on  

Networking  

Service Reference Model

Page 15: T1 – Architecting highly available applications on aws

Compute  Storage  &  Content  Delivery  

AWS  Global  Infrastructure  

Database  

App  Services  

Deployment  &  Administra=on  

Networking  

Amazon CloudSearch

Amazon SQS

Amazon SNS

Amazon Elastic

Transcoder

Amazon SWF Amazon SES

Amazon DynamoDB

Amazon RDS

Amazon ElastiCache

Amazon RedShift

AWS Storage Gateway

Amazon S3

Amazon Glacier

Amazon CloudFront

Amazon CloudWatch AWS IAM AWS

CloudFormation Amazon Elastic

Beanstalk AWS Data

Pipeline

AWS OpsWorks

AWS CloudTrail

Amazon EC2

Amazon EMR

Amazon VPC

Amazon Route 53

AWS Direct

Connect

Amazon Kinesis

Page 16: T1 – Architecting highly available applications on aws

So let’s start from day one, user one ( you )

Page 17: T1 – Architecting highly available applications on aws

Day One, User One

•  A single EC2 Instance –  With full stack on this host

•  Web app •  Database •  Management •  Etc.

•  A single Elastic IP •  Route53 for DNS

EC2 Instance

Elastic IP

Amazon Route 53

User

Page 18: T1 – Architecting highly available applications on aws

“We’re gonna need a bigger box”

•  Simplest approach •  Can now leverage PIOPs •  High I/O instances •  High memory instances •  High CPU instances •  High storage instances •  Easy to change instance sizes •  Will hit an endpoint eventually

i2.4xlarge

m3.xlarge

m1.small

Page 19: T1 – Architecting highly available applications on aws

“We’re gonna need a bigger box”

•  Simplest approach •  Can now leverage PIOPs •  High I/O instances •  High memory instances •  High CPU instances •  High storage instances •  Easy to change instance sizes •  Will hit an endpoint eventually

i2.4xlarge

m3.xlarge

m1.small

Page 20: T1 – Architecting highly available applications on aws

Day One, User One: •  We could potentially get

to a few hundred to a few thousand depending on application complexity and traffic

•  No failover •  No redundancy •  Too many eggs in one

basket

EC2 instance

Elastic IP address

Amazon Route 53

User

Page 21: T1 – Architecting highly available applications on aws

Day One, User One: •  We could potentially get

to a few hundred to a few thousand depending on application complexity and traffic

•  No failover •  No redundancy •  Too many eggs in one

basket

EC2 instance

Elastic IP address

Amazon Route 53

User

Page 22: T1 – Architecting highly available applications on aws

Day Two, User >1: First, let’s separate out our single host into more than one: •  Web •  Database

–  Make use of a database service?

Web instance

Database instance

Elastic IP address

Amazon Route 53

User

Page 23: T1 – Architecting highly available applications on aws

Self-Managed Fully-Managed

Database server on Amazon EC2

Your choice of

database running on Amazon EC2

Bring Your Own License (BYOL)

Amazon DynamoDB

Managed NoSQL database service

using SSD storage

Seamless scalability Zero administration

Amazon RDS

Microsoft SQL, Oracle, MySQL or PostgreSQL as a managed service

Flexible licensing BYOL or License

Included

Amazon Redshift

Massively parallel,

petabyte-scale, data warehouse service

Fast, powerful and

easy to scale

Database Options

Page 24: T1 – Architecting highly available applications on aws

But how do I choose what DB technology I need? SQL? NoSQL?

Page 25: T1 – Architecting highly available applications on aws

Some people won’t like this. But…

Page 26: T1 – Architecting highly available applications on aws

Start with SQL databases

Page 27: T1 – Architecting highly available applications on aws

Why start with SQL? •  Established and well-worn technology •  Lots of existing code, communities, books, background,

tools, etc. •  You aren’t going to break SQL DBs in your first 10 million

users. No really, you won’t*. •  Clear patterns to scalability * Unless you are manipulating data at MASSIVE scale; even then, SQL will have a place in your stack

Page 28: T1 – Architecting highly available applications on aws

AH HA! You said “massive amounts”, I will have massive amounts!

Page 29: T1 – Architecting highly available applications on aws

If your usage is such that you will be generating several TB of data in the

first year OR have an incredibly data-intensive workload… you might

need NoSQL

Page 30: T1 – Architecting highly available applications on aws

Regardless, why NoSQL? •  Super low latency applications •  Metadata driven datasets •  Highly non-relational data •  Need schema-less data constructs* •  Massive amounts of data (again, in the TB range) •  Rapid ingest of data ( thousands of records/sec ) •  Already have skilled staff *Need != “it is easier to do dev without schemas”

Page 31: T1 – Architecting highly available applications on aws

But back to the main path… Let’s see how far SQL at the core

can grow

Page 32: T1 – Architecting highly available applications on aws

User >100 First let’s separate out our single host into more than one •  Web •  Database

–  Use RDS to make your life easier

Web Instance

Elastic IP

RDS DB Instance

Amazon Route 53

User

Page 33: T1 – Architecting highly available applications on aws

User > 1000 Next let’s address our lack of failover and redundancy issues •  Elastic Load Balancing •  Another web instance

–  In another Availability Zone

•  Enable Amazon RDS multi-AZ

Web Instance

RDS DB Instance Active (Multi-AZ) Availability Zone Availability Zone

Web Instance

RDS DB Instance Standby (Multi-AZ)

Elastic Load Balancing

Amazon Route 53

User

Page 34: T1 – Architecting highly available applications on aws

Scaling this horizontally and vertically

will get us pretty far ( 10s-100s of thousands )

Page 35: T1 – Architecting highly available applications on aws

User >10 ks–100 ks

RDS DB Instance Active (Multi-AZ)

Availability Zone Availability Zone

RDS DB Instance Standby (Multi-AZ)

Elastic Load Balancing

RDS DB Instance Read Replica

RDS DB Instance Read Replica

RDS DB Instance Read Replica

RDS DB Instance Read Replica

Web Instance

Web Instance

Web Instance

Web Instance

Web Instance

Web Instance

Web Instance

Web Instance

Amazon Route 53

User

Page 36: T1 – Architecting highly available applications on aws

Shift some load around: Let’s lighten the load on our web and database instances: •  Move static content from the

web instance to Amazon S3 and CloudFront

•  Move dynamic content from the load balancer to CloudFront

•  Move session/state and DB caching to ElastiCache or Amazon DynamoDB

Web instance

RDS DB Instance Active (Multi-AZ) Availability Zone

Elastic Load Balancer

Amazon S3

Amazon CloudFront

Amazon Route 53

User

ElastiCache

Amazon DynamoDB

Page 37: T1 – Architecting highly available applications on aws

Now let’s revisit the beginning of our talk…

Page 38: T1 – Architecting highly available applications on aws

Auto Scaling!

Page 39: T1 – Architecting highly available applications on aws

Automatic resizing of compute clusters based on demand

Trigger auto-scaling policy

Feature   Details  

Control   Define  minimum  and  maximum  instance  pool  sizes  and  when  scaling  and  cool  down  occurs  

Integrated  to  Amazon  CloudWatch  

Use  metrics  gathered  by  CloudWatch  to  drive  scaling  

Instance  types   Run  Auto  Scaling  for  On-­‐Demand  and  Spot  Instances;  compa=ble  with  VPC  

aws  autoscaling  create-­‐auto-­‐scaling-­‐group  -­‐-­‐auto-­‐scaling-­‐group-­‐name  MyGroup  -­‐-­‐launch-­‐configuration-­‐name  MyConfig  -­‐-­‐min-­‐size  4  -­‐-­‐max-­‐size  200  -­‐-­‐availability-­‐zones  us-­‐west-­‐2c  

Auto Scaling Amazon

CloudWatch

Page 40: T1 – Architecting highly available applications on aws

Auto Scaling can scale from one instance to thousands

and back down

Page 41: T1 – Architecting highly available applications on aws

User >500k+:

Availability Zone

Amazon Route 53

User

Amazon S3

Amazon CloudFront

Availability Zone

Elastic Load Balancing

Amazon DynamoDB RDS DB Instance

Read Replica

Web instance

Web instance

Web instance

ElastiCache RDS DB Instance Read Replica

Web instance

Web instance

Web instance

ElastiCache RDS DB Instance Standby (Multi-AZ)

RDS DB Instance Active (Multi-AZ)

Page 42: T1 – Architecting highly available applications on aws

ARCHITECTING DATA-DRIVEN MASS PRODUCED VIDEO

AWS SUMMIT 2014 | JUNE 10, 2014

Page 43: T1 – Architecting highly available applications on aws

JASPER JAGER SENIOR DEVELOPER AND AWS ARCHITECT REDNUN, AMSTERDAM

Page 44: T1 – Architecting highly available applications on aws

THIS IS WHAT WE DO

‣ Automatically mass produce data-driven, personalised or profiled video

‣ ING, KLM, Essent België, T-Mobile

‣ Run everything in AWS

Page 45: T1 – Architecting highly available applications on aws
Page 46: T1 – Architecting highly available applications on aws

‣ Small campaign, 25.000 personalised videos

‣ Self hosted 3x 8 core Xserves with 96GB RAM

‣ 300 videos an hour

HOW WE STARTED

Page 47: T1 – Architecting highly available applications on aws

PROBLEMS WITH THE OLD IN-HOUSE SETUP‣ 250.000 videos would take us 35 days

‣ Or we would have to buy more hardware

‣ Systems which would idle most of the time

‣ Storing and serving all videos - HELP

Page 48: T1 – Architecting highly available applications on aws

REBUILD REDNUN IN THE CLOUD‣ Ability to start 100’s of machines, based on preconfigured AMI

‣ High availability for our campaign sites, behind load balancers

‣ Big campaign, big Dutch lottery, 1.200.000

‣ Batched, pre-rendered videos, stored on S3

‣ Took us just a couple of days

Page 49: T1 – Architecting highly available applications on aws

SECOND INFRASTRUCTURE

Page 50: T1 – Architecting highly available applications on aws

AUTOSCALE EVERYTHING

‣ Automated daily flows, welcome video, birthday video etc.

‣ API, videos can be produced on the fly

‣ Autoscaling based on Cloudwatch metrics for web and app servers

‣ Custom autoscaling scripts for video rendering

‣ Use spot instances when available

Page 51: T1 – Architecting highly available applications on aws

LOOSE COUPLING

‣ Decoupled components

‣ Use SQS as a buffer

‣ Continuous monitoring and adjusting

Page 52: T1 – Architecting highly available applications on aws

AUTOMATE EVERYTHING

‣ Cloudformation and Opsworks

‣ Flexibility to start environment in different region

‣ Dev and QA environments

Page 53: T1 – Architecting highly available applications on aws

CURRENT INFRASTRUCTURE

Page 54: T1 – Architecting highly available applications on aws

THE THINGS WE’VE LEARNED‣ AWS service limits

‣ Autoscale on Cloudwatch or custom metrics

‣ Automate your infrastructure

‣ AWS can help you scale with ease

Page 55: T1 – Architecting highly available applications on aws

[email protected] WWW.REDNUN.NL

Page 56: T1 – Architecting highly available applications on aws

On Tools: Managing your infrastructure will become an ever increasing important part of your time. Use tools to automate repetitive tasks. •  Tools to manage AWS resources – AWS CloudFormation •  Tools to manage software and configuration on your

instances – AWS OpsWorks •  Automated data analysis of logs and user actions

Page 57: T1 – Architecting highly available applications on aws

User >500k+: You’ll potentially start to run into issues with speed and performance of your applications: •  Have monitoring/metrics/logging in place

–  If you can’t build it internally, outsource it! (3rd party SaaS) •  Pay attention to what customers are saying works well •  Squeeze as much performance as you can out of each

service/component

Page 58: T1 – Architecting highly available applications on aws

HOST LEVEL

METRICS

AGGREGATE LEVEL

METRICS

LOG ANALYSIS

EXTERNAL SITE

PERFORMANCE

Page 59: T1 – Architecting highly available applications on aws

Not having proper monitoring/metrics is like flying a plane

with an eye mask on in a thunderstorm.

Oh, and your wing is on fire.

Page 60: T1 – Architecting highly available applications on aws
Page 61: T1 – Architecting highly available applications on aws

AWS Marketplace & Partners Can Help •  Customer can find, research,

and buy software

•  Simple pricing, aligns with Amazon EC2 usage model

•  Launch in minutes

•  AWS Marketplace billing integrated into your AWS account

•  1300+ products across 20+ categories

Learn more at: aws.amazon.com/marketplace

Page 62: T1 – Architecting highly available applications on aws

There are further improvements to be

made in breaking apart our web/app layer

Page 63: T1 – Architecting highly available applications on aws
Page 64: T1 – Architecting highly available applications on aws

SOA = Service Oriented Architecture

Page 65: T1 – Architecting highly available applications on aws

SOA’ing Move services into their own tiers/modules. Treat each of these as 100% separate pieces of your infrastructure and scale them independently. Amazon.com and AWS do this extensively! It offers flexibility and greater understanding of each component.

Page 66: T1 – Architecting highly available applications on aws

Loose coupling sets you free! •  The looser they're coupled, the bigger they scale

–  Independent components –  Design everything as a black box –  Decouple interactions –  Favor services with built-in redundancy and scalability rather than

building your own

Controller  A   Controller  B  

Controller  A   Controller  B  

Q   Q  

Tight  coupling  

Use  Amazon  SQS  for  buffers  

Loose  coupling  

Page 67: T1 – Architecting highly available applications on aws

Loose coupling + SOA = winning

Examples: •  Email •  Queuing •  Transcoding •  Search •  Databases •  Monitoring •  Metrics •  Logging

Amazon CloudSearch

Amazon SQS Amazon SNS

Amazon Elastic Transcoder

Amazon SWF Amazon SES

In the early days, if someone has a service for it already, opt to use that instead of building it yourself. DON’T RE-INVENT THE WHEEL

Page 68: T1 – Architecting highly available applications on aws

On re-inventing the wheel… If you find yourself writing

your own: queue, DNS server, database, storage system,

monitoring tool

Page 69: T1 – Architecting highly available applications on aws

Take a deep breath and stop it. Now.

Page 70: T1 – Architecting highly available applications on aws

Back to SOA

Page 71: T1 – Architecting highly available applications on aws

Users > 1 Million

RDS DB Instance Active (Multi-AZ)

Availability Zone

Elastic Load Balancer

RDS DB Instance Read Replica

RDS DB Instance Read Replica

Web Instance

Web Instance

Web Instance

Web Instance

Amazon Route 53

User

Amazon S3

Amazon Cloudfront

Amazon DynamoDB

Amazon SQS

ElastiCache

Worker Instance

Worker Instance

Amazon CloudWatch

Internal App Instance

Internal App Instance

Amazon SES

Page 72: T1 – Architecting highly available applications on aws

The next big steps

Page 73: T1 – Architecting highly available applications on aws

From 5 to 10 Million Users You may start to run into issues with your database around contention on the write master. How can you solve it?

•  Federation - splitting into multiple DBs based on function

•  Sharding - splitting one data set up across multiple hosts

•  Moving some functionality to other types of DBs (NoSQL)

Page 74: T1 – Architecting highly available applications on aws

…and there you have it. 10 Million

Page 75: T1 – Architecting highly available applications on aws

A Quick Review

Page 76: T1 – Architecting highly available applications on aws

Review •  Multi-AZ your infrastructure •  Make use of self-scaling services

–  Elastic Load Balancing, Amazon S3, Amazon SNS, Amazon SQS, Amazon SWF, Amazon SES, etc.

•  Build in redundancy at every level •  Most likely start with SQL •  Cache data both inside and outside your

infrastructure •  Use automation tools in your infrastructure

Page 77: T1 – Architecting highly available applications on aws

Review (cont) •  Make sure you have good metrics/monitoring/

logging tools in place •  Split tiers into individual services (SOA) •  Use Auto Scaling when you’re ready for it •  Don’t reinvent the wheel •  Move to NoSQL when it really makes sense but

do your best not to administer it

Page 78: T1 – Architecting highly available applications on aws

Putting all this together means we should now

easily be able to handle 10+ million users!

Page 79: T1 – Architecting highly available applications on aws

To infinity…..

Page 80: T1 – Architecting highly available applications on aws

Thank You!

AWS EXPERT? GET CERTIFIED! aws.amazon.com/certification

Alex Sinner Solutions Architect @alexsinner