architecting for aws

133
ARCHITECTING FOR AWS Carlos Conde Solutions Architect

Upload: amazon-web-services

Post on 28-Nov-2014

5.030 views

Category:

Business


5 download

DESCRIPTION

 

TRANSCRIPT

Page 1: Architecting for AWS

ARCHITECTING

FOR AWS

Carlos Conde │Solutions Architect

Page 2: Architecting for AWS

1. ELASTICITY

2. DESIGN FOR FAILURE

3. LOOSE COUPLING

4. SECURITY

5. PERFORMANCE

Page 3: Architecting for AWS

#1 ELASTICITY

●○○○○

Page 4: Architecting for AWS

AMAZON EC2 ELASTIC COMPUTE CLOUD

Page 5: Architecting for AWS
Page 6: Architecting for AWS

AMAZON

CLOUDWATCH MONITORING FOR AWS RESOURCES

Page 7: Architecting for AWS
Page 8: Architecting for AWS

AUTO SCALING SCALE UP/DOWN EC2 CAPACITY

Page 9: Architecting for AWS
Page 10: Architecting for AWS

ELASTIC LOAD

BALANCING NETWORK TRAFFIC DISTRIBUTION

Page 11: Architecting for AWS
Page 12: Architecting for AWS

6 am

Page 13: Architecting for AWS

10 am

Page 14: Architecting for AWS

10 am

Page 15: Architecting for AWS

10 am

Page 16: Architecting for AWS

7 pm

Page 17: Architecting for AWS

7 pm

Page 18: Architecting for AWS

7 pm

Page 19: Architecting for AWS

without elasticity you can't accelerate

Page 20: Architecting for AWS

what you really need is…

…adjustable capacity

Page 21: Architecting for AWS

#2 DESIGN FOR FAILURE

●●○○○

Page 22: Architecting for AWS

« Everything fails

all the time »

Werner Vogels

CTO of Amazon

Page 23: Architecting for AWS

YOUR GOAL

Applications should continue to function even if

the underlying physical hardware fails or is

removed or replaced

Page 24: Architecting for AWS

Avoid single points of failure.

Assume everything fails, and design

backwards.

Page 25: Architecting for AWS

Avoid single points of failure.

Assume everything fails, and design

backwards.

Page 26: Architecting for AWS
Page 27: Architecting for AWS

AMAZON RDS RELATIONAL DATABASE SERVICE

Page 28: Architecting for AWS
Page 29: Architecting for AWS
Page 30: Architecting for AWS

AMAZON

ROUTE 53 DOMAIN NAME SERVICE

Page 31: Architecting for AWS
Page 32: Architecting for AWS
Page 33: Architecting for AWS
Page 34: Architecting for AWS
Page 35: Architecting for AWS

AMAZON RDS

MULTI-AZ DEPLOYMENT

Page 36: Architecting for AWS
Page 37: Architecting for AWS
Page 38: Architecting for AWS
Page 39: Architecting for AWS
Page 40: Architecting for AWS

AMAZON CLOUDWATCH

ALARMS

Page 41: Architecting for AWS
Page 42: Architecting for AWS
Page 43: Architecting for AWS
Page 44: Architecting for AWS
Page 45: Architecting for AWS
Page 46: Architecting for AWS
Page 47: Architecting for AWS
Page 48: Architecting for AWS
Page 49: Architecting for AWS

AWS BUILDING BLOCKS

Inherently Fault-Tolerant Services Fault-Tolerant with

the right architecture

Amazon S3

Amazon SimpleDB

Amazon DynamoDB

Amazon CloudFront

Amazon SWF

Amazon SQS

Amazon SNS

Amazon SES

Amazon Route53

Elastic Load

Balancing

AWS IAM

AWS Elastic

Beanstalk

Amazon

ElastiCache

Amazon EMR

Amazon

CloudSearch

Amazon EC2

Amazon EBS

Amazon RDS

Amazon VPC

Page 50: Architecting for AWS
Page 51: Architecting for AWS

NETFLIX

CHAOS MONKEY

Page 52: Architecting for AWS
Page 53: Architecting for AWS

#3 LOOSE

COUPLING ●●●○○

Page 54: Architecting for AWS

BUILD LOOSELY

COUPLED SYSTEMS

The looser the are coupled,

the bigger they scale

Page 55: Architecting for AWS

Create independent components

Page 56: Architecting for AWS

Create independent components

Design everything as a Black Box

Page 57: Architecting for AWS

Create independent components

Design everything as a Black Box

Think in terms of services

Page 58: Architecting for AWS

RECEIVE TRANSCODE

& PUBLISH

Page 59: Architecting for AWS

RECEIVE TRANSCODE

& PUBLISH

QUEUE

Page 60: Architecting for AWS
Page 61: Architecting for AWS
Page 62: Architecting for AWS

AMAZON SQS SIMPLE QUEUE SERVICE

Page 63: Architecting for AWS
Page 64: Architecting for AWS
Page 65: Architecting for AWS

SPAM?

CHECK

VIDEO

TOO

LONG?

SHORTEN

VIDEO

NO

YES NO

YES

TRANSCODE

SPAM

CHECK

START

PUBLISH

& NOTIFY

STOP REJECT

Page 66: Architecting for AWS
Page 67: Architecting for AWS
Page 68: Architecting for AWS

MAINTENANCE

AUDITABILITY

FLEXIBILITY

LOW-LEVEL

Page 69: Architecting for AWS

WHAT ARE WE

TRYING TO DO?

Page 70: Architecting for AWS

SPAM?

CHECK

VIDEO

TOO

LONG?

SHORTEN

VIDEO

NO

YES NO

YES

TRANSCODE

SPAM

CHECK

START

PUBLISH

& NOTIFY

STOP REJECT

Page 71: Architecting for AWS

SPAM?

CHECK

VIDEO

TOO

LONG?

SHORTEN

VIDEO

NO

YES NO

YES

TRANSCODE

SPAM

CHECK

START

PUBLISH

& NOTIFY

STOP REJECT

Page 72: Architecting for AWS

SPAM?

CHECK

VIDEO

TOO

LONG?

SHORTEN

VIDEO

NO

YES NO

YES

TRANSCODE

SPAM

CHECK

START

PUBLISH

& NOTIFY

STOP REJECT

Page 73: Architecting for AWS

TASKS

DECISIONS

HISTORY

STATELESS !

Page 74: Architecting for AWS

STATELESS SCALES

HORIZONTALLY

Page 75: Architecting for AWS

AMAZON SWF WORKFLOW SERVICE FOR SCALABLE,

RESILIENT APPLICATIONS

Page 76: Architecting for AWS

AMAZON SWF

Keeps track of :

State

Executed tasks

Timeouts

Errors

Page 77: Architecting for AWS

WORKFLOW

ACTORS

Page 78: Architecting for AWS

DECIDERS COORDINATION LOGIC

1. Poll for work on a decision list Long polling: 60 seconds

2. Evaluate workflow execution history SWF sends full history in JSON format

3. Return decision to Amazon SWF Usually scheduling another task

Page 79: Architecting for AWS

WORKERS COORDINATION LOGIC

1. Poll for work on a specific task list Long polling: 60 seconds

2. Execute works, send heartbeats SWF sends input data from deciders

3. Return success / failure Detailed data can be provided to deciders

Page 80: Architecting for AWS
Page 81: Architecting for AWS
Page 82: Architecting for AWS

ALL HORIZONTAL SCALING

PATTERNS APPLY

Page 83: Architecting for AWS
Page 84: Architecting for AWS
Page 85: Architecting for AWS
Page 86: Architecting for AWS
Page 87: Architecting for AWS
Page 88: Architecting for AWS
Page 89: Architecting for AWS

NO NEW LANGUAGE

TO LEARN

YOUR CODE IS YOUR WORKFLOW LANGUAGE

SWF MAINTAINS STATE

Page 90: Architecting for AWS

AWS FLOW

FRAMEWORK

Java Library • Entire workflow can be

expressed in sequential code •

Integrated with Java Utils API

Page 91: Architecting for AWS

CHAINED TASKS

WITHOUT DECISIONS?

use AMAZON SQS

NOTIFY RECEIVE TRANSCODE

Page 92: Architecting for AWS

TASK GRAPH

WITH DECISIONS?

use AMAZON SWF

SPAM

CHECK

RECEIVE

VIDEO

CHECK

LENGTH

REJECT SHORTEN

VIDEO

PUBLISH

& NOTIFY

GOOD

LONG

OK

SPAM

TRANSCODE

Page 93: Architecting for AWS

#4 SECURITY

●●●●○

Page 94: Architecting for AWS

AWS IAM IDENTITY AND ACCESS MANAGEMENT

Page 95: Architecting for AWS
Page 96: Architecting for AWS

MULTI- MULTI-FACTOR AUTHENTICATION

Page 97: Architecting for AWS

SECURITY GROUPS LAYERED SECURITY

Page 98: Architecting for AWS
Page 99: Architecting for AWS
Page 100: Architecting for AWS
Page 101: Architecting for AWS
Page 102: Architecting for AWS

CERTIFICATIONS

& COMPLIANCE

SAS 70 Type II audit

ISO 27001 Certification

Sarbanes-Oxley SOX

HIPAA healthcare

FISMA US Federal Government

DIACAP MAC III Sensitive IATO

Page 103: Architecting for AWS

PCI DSS Compliant LEVEL 1 SERVICE PROVIDER

Page 104: Architecting for AWS

#5 PERFORMANCE

●●●●●

Page 105: Architecting for AWS

VERTICAL

SCALING

Page 106: Architecting for AWS
Page 107: Architecting for AWS

CLUSTER COMPUTE

EIGHT EXTRA LARGE

Intel Xeon E5-2670, eight-core

Sandy Bridge architecture

60.5 GB of memory – 3.3 TB of storage

2x

Page 108: Architecting for AWS

CLUSTER GPU

QUADRUPLE EXTRA LARGE

Intel Xeon X5570, quad-core

Nehalem architecture

NVIDIA Tesla Fermi

M2050 GPUs

22 GB of memory – 1.7 TB of storage

2x

2x

Page 109: Architecting for AWS

AMAZON

ELASTICACHE MEMCACHED COMPATIBLE

IN-MEMORY CACHE

Page 110: Architecting for AWS
Page 111: Architecting for AWS

SEARCH

ENGINES

Page 112: Architecting for AWS
Page 113: Architecting for AWS

Relevance &

Ranking

Page 114: Architecting for AWS

Faceting

Page 115: Architecting for AWS

Range

Searching

Page 116: Architecting for AWS

AMAZON

CLOUDSEARCH FULLY-MANAGED SEARCH SERVICE

Page 117: Architecting for AWS

+

Page 118: Architecting for AWS

DEMOCRATIZE SEARCH

Amazon CloudSearch delivers a fully-managed search service in the cloud

that can be set up and running in less than 1 hour, with automatic scaling for

data & traffic, at a price starting at less than $100 per month.

Page 119: Architecting for AWS
Page 120: Architecting for AWS
Page 121: Architecting for AWS
Page 122: Architecting for AWS

HTTP

POST author=carlos;

date=2012-06-07;

article=text;

Page 123: Architecting for AWS
Page 124: Architecting for AWS
Page 125: Architecting for AWS

MEDIA SHARING REFERENCE ARCHITECTURE

Page 126: Architecting for AWS
Page 127: Architecting for AWS

HTTP

POST author=carlos;

date=2012-06-07;

article=text;

+

Page 128: Architecting for AWS
Page 129: Architecting for AWS

1. ELASTICITY

2. DESIGN FOR FAILURE

3. LOOSE COUPLING

4. SECURITY

5. PERFORMANCE

Page 130: Architecting for AWS
Page 131: Architecting for AWS

« If you're not embarrassed when you ship

your first version you waited too long »

Matt Mullenweg

CEO & Founder of WordPress

Page 132: Architecting for AWS
Page 133: Architecting for AWS