level up your web apps with amazon web...

67
Level Up Your Web Apps with Amazon Web Services Brian Klaas [email protected] @brian_klaas

Upload: others

Post on 02-Oct-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Level Up Your Web Apps with Amazon Web ServicesBrian Klaas

[email protected]

@brian_klaas

Page 2: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Hello

Page 3: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Hello

Page 4: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Hello

Page 5: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Hello

Oh no, my job security!

Page 6: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Hello

Page 7: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Hello

Good

Fast Cheap

Page 8: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Hello

Page 9: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Hello

Page 10: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Hello

Page 11: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Hello

Page 12: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Hello

Page 13: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

HelloEC2 Lambda

Step Functions

OpsWorks

Cloud Watch

Data Pipeline

VPC IAMDirect

Connect

Cloud Search Elastic Transcoder

SES SNS SQS SWF

Dynamo DB

Elasti Cache

RDS Redshift

Cloud Front Glacier S3 Storage

Gateway

Kinesis

Cloud Trail

Elastic Map Reduce

Elastic Beanstalk

FPS (Payment)

App Stream

WorkMail and Docs

Work Spaces

CognitoMobile

Analytics

Coud HSM

Aurora Athena

Snowball

Xray

Code Commit

Code Deploy

Route 53

Cloud Formation

Elastic Container

ServiceBatch Lightsail Code Build Code

Pipeline

Elasticsearch

AWS Config

API Gateway

Page 14: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

+ =Hello

Page 15: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Hello

1. Go play in the console 2. Learn about IAM roles and permissions

Things you have to do on your own:

Page 16: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

S3

Page 17: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

S3

Store all the things.

Page 18: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

S3

99.999999999% durability

Page 19: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

S3

$0.023 per GB stored $0.004 per 10,000 GET $0.005 per 1,000 PUT

$0.10 per GB out after 1GB

Page 20: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

S3

myfiles.s3.amazonaws.com

Bucket

Page 21: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

S3

http://mybucket.s3.amazonaws.com/path/to/file.png

Page 22: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

S3

C:/ s3://

Page 23: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

cffile( variable="fileData", file="s3://somebucket/somefile.txt", action="read" );

cfdirectory( directory="s3://somebucket/someDirectory", action="list" );

S3

Page 24: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

S3

if ( !directoryExists("s3://somebucket.s3.amazonaws.com") ) { perms = [ {group="all", permission="read"}, {id="canonicalIDofYourAWSAccount", permission=“full_control"} ]; cfdirectory( directory="s3://somebucket.s3.amazonaws.com", storeacl=perms, action="create" ); }

fileWrite("s3://somebucket.s3.amazonaws.com/myFile.txt", "#someOutput#");

files = directoryList("s3://somebucket.s3.amazonaws.com");

Page 25: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

S3

IAM Credentials Required

Page 26: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

S3

Inline:

cffile( variable="fileData", file="s3://accessKey:secretKey@somebucket/somefile.txt", action="read" );

Or, in application.cfc:

this.s3.accessKeyId="accessKey"; this.s3.awsSecretKey="secretKey";

Page 27: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

S3

๏ Expire URLs

๏ Change properties on a per-request basis

๏ Upload to S3 from browser

Requires request signing.

Page 28: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

S3

S3 Request Signing Utils

github.com/brianklaas/ctlS3utils

Page 29: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

S3

S3 is storage, not a file system.

Can get basic file info withcfhttp( url="http://bucket.s3.amazonaws.com/filename",

method="head" );

Page 30: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

S3

Store all the things.

Page 31: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

CloudFront

Page 32: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Cloud Front

Page 33: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Cloud Front

API HTML/Markup

JS/CSS/Images/Videos User Content

Page 34: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Cloud Front

๏ Geographic distribution of video

๏ Faster downloads

๏ Less traffic on our network

๏ ~3TB transfer/month

April 2017:

$148.96

Page 35: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Cloud Front

1. Set up source

S3 Bucket

EC2 Instance

2. Create a distribution 3. Point distribution to source

4. Use generated DNS entry, or your own

5. Point to CloudFront URLs in your code

Page 36: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Congratulations!Cloud Front

Page 37: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Lambda

Page 38: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Lambda

Page 39: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Lambda

Page 40: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Lambda

No server is easier to manage than no server.

Page 41: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Lambda

= Event–driven computing

Page 42: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Lambda

<1.5 GB RAM <5 minutes

Page 43: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Lambda

2. Upload ZIP1. Write a handler function

3. Invoke an event 4. Handler runs

Page 44: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Lambda

= Microservices infrastructure without having to worry about running containers

or scaling your infrastructure!

Page 45: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Lambda

Logging and Monitoring Third–Party Services

Page 46: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Lambda

Logging and Monitoring Third–Party Services

?

Page 47: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Lambda

Logging and Monitoring Third–Party Services

Customer uploads Emits event with file info Processes event

File details

Alert

Page 48: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Lambda

Async Image Resizing

Emits event with file info Processes eventHandle customer upload, posts to S3

cffile ( action=“write” file=“s3://somebucket/images.zip” ) Calls “done” http endpoint

Multiple sizes written to S3

Page 49: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Video Production Workflow

Lambda

Upload to ingest bucket Processes event

Posts to S3

Launches Elastic Transcoder job Puts encoded files

Processes event

Page 50: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Video Production Workflow (v2)

Lambda

Upload to ingest bucket Processes eventPosts to S3

Launches Elastic Transcoder job Puts encoded files

Processes event

Calls http endpoint Grabs source video Calls transcription API Calls http endpoint Posts to S3

Page 51: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Demo

Invoking Lambda functions from CF

Lambda

github.com/brianklaas

Page 52: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Lambda

= Focus on building apps, not infrastructure

Page 53: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

DynamoDB

Page 54: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Dynamo DB

= Hugely scalable, high-write throughput document data store

Page 55: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Dynamo DB

4. Write3. Set secondary indexes

2. Set primary and sort keys1. Set read/write capacity

Page 56: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Logging and Monitoring Third–Party Services

Customer uploads Emits event with file info Processes event

File details

Alert

Dynamo DB

Page 57: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Exam Activity Analytics

Capture learner action Writes batch data Stores batch data

Dynamo DB

?QuickSight

Athena

ElasticSearch

Page 58: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Demo

List, Put, Scan, Filter from CF

Dynamo DB

github.com/brianklaas

Page 59: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Preparing for Outages

Page 60: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

The Great S3 Outage of 2017

Outages

Page 61: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Plan for outages.

Outages

(Blame Amazon)

Page 62: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Batch upload from your servers

Outages

Page 63: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Use multiple regions

Outages

Page 64: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Shut off services

Outages

Page 65: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Have a plan.

Outages

Page 66: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

Go Do!

Page 67: Level Up Your Web Apps with Amazon Web Servicesbrianklaas.net/assets/pdf/intoTheBox2017-LevelUpAppsWithAWS.pdf · EC2 Lambda Step Functions OpsWorks Cloud Watch Data Pipeline VPC

[email protected]

@brian_klaas

github.com/brianklaas