(sov203) understanding aws storage options | aws re:invent 2014

November 12, 2014 | Las Vegas, NV

Guy Farber, AWS Storage Business Development

AWS storage options

•

•

•

•

• Customer presentation: Ooyala

We are constantly producing more data

From all types of industries

AMAZON S3Simple Storage Service

Of Unique Customer Objects

Q4 2006

Q1 2007

Q2 2007

Q3 2007

Q4 2007

Q1 2008

Q2 2008

Q3 2008

Q4 2008

Q1 2009

Q2 2009

Q3 2009

Q4 2009

Q1 2010

Q2 2010

Q3 2010

Q4 2010

Q1 2011

Q2 2011

Q3 2011

Q4 2011

Q1 2012

Q2 2012

Q3 2012

Q4 2012

Q1 2013

Q2 2013

Q3 2013

peak transactions per second

durability

99.99% saves ~20%

Amazon S3

Amazon S3

1 PB raw storage

800 TB usable storage

600 TB allocated storage

400 TB application data

S3 pay only for what you use!

Amazon S3

Amazon S3

Amazon S3

Amazon S3

Amazon S3

Amazon

CloudFront

Low-cost archiving service

per GB / month

per TB / year

durability

data retrieval

vs.

Amazon Glacier

Policy-based archiving service

Amazon S3 →

Amazon Glacier

• World’s leading social sound

platform

• Audio files must be transcoded and

stored in multiple formats

S3

Amazon Glacier

S3

•

•

• Amazon Glacier

Elastic Block Storage

10GB1TB

Price Performance

Latency Business impact

EBS

Magnetic General Purpose Provisioned IOPS

Use Cases Infrequent data access

Boot volumes

Small to Med DBs

Dev and Test

I/O intensive

Relational DBs

NoSQL DBs

Storage Media Magnetic disk-backed SSD-backed SSD-backed

Performance 100 IOPS on average.

Ability to burst to

hundreds of IOPS

Burst to 3,000 IOPS

maximum, with a base

performance of 3 IOPS/GiB

Consistently performs at

provisioned level, 4000

IOPS maximum

Price$.05/GB - Month

$.05/million I/O$.10/GB – Month

$.125/GB - Month

$.065/provisioned IOPS

Amazon EBS

Amazon EC2

AWS Storage Gateway

Your on-ramp to AWS cloud storage:• Backup into Amazon S3

• DR on Amazon EC2

• Archive into Amazon Glacier

For a deeper dive, join: BAC202 - Introducing AWS

Solutions for Backup and Archiving

Performance-orientedCost-oriented

CLOUD STORAGE

AT OOYALA

Ilan Rabinovitch

Manager, Infrastructure and Site Reliability Engineering

HISTORY OF AWS AT OOYALA

• Ooyala born in the cloud in 2007

• Winner of the first AWS Startup Challenge in 2007

• Our mission is to help our customers bring

personalized video to every screen everywhere

• Today we use a number of AWS services

including:

• Amazon S3

• Amazon EC2 + Amazon VPC

• Amazon CloudFront

• AWS Direct Connect

Storage use cases

Log storage with Amazon S3 and Amazon Glacier

• As a data-focused company, we log and analyze

EVERYTHING

• Needed a scalable and cost-effective way to transport,

store, and access billions of log files

• Enter Amazon S3 and Amazon Glacier

• Store application logs and telemetry data in S3

• Download and process in Hadoop and Spark

• Archive older data to Glacier to reduce storage

costs

Log storage with Amazon S3 and Amazon Glacier

Infinitely scalable storage with S3

Automatically archive older data

to Glacier via life cycle polices

Low latency connectivity to on-

premises facilities via Direct

Connect

Benefits Tips

Know your access patterns:

Large batch restorations can be

time intensive and costly

Plan ahead! S3 Import / Export

requires you to thaw from Glacier

first

Secure your data with custom

IAM keys and instance roles

WALLE: Safe deletion in S3

• Ooyala stores and transcodes customer videos in S3 as origin for CDNs

• Overtime needed to delete over 200 million stale files safely

• Enter WALLE:

• Internal API for asset deletion

• Spawns Spot Instances based on workload to issue deletion requests

• S3 copy to temporary “Trash” bucket with lifecycle policies

WALLEWALLE: Safe deletion in S3

Tips

• Spring Cleaning Can Generate Real Savings

Automate your data retention and deletion with lifecycle policies

• Safety First

Use S3 versioning or temporary storage to protect against mistakenly

overwriting or deleting data

• Scale Cost-Effectively

Use Spot Instances and Auto Scaling where possible to keep costs in

check

Client-side web applications

• Ooyala develops web applications, which our clients use to

manage their content libraries

• Many applications have migrated to client-side JavaScript to

allow for a richer and more performant user experience

• Needed a simple way to deploy and scale these client-side

applications

• Enter S3 websites and CloudFront

• Upload static content such as HTML + JS to S3

• Accelerate and cache content via Amazon CloudFront

• Use AJAX to interact with server side APIs and data

Client-side web applications

Easy and repeatable deployments

Scales easily and securely; no web

servers to manage

Low-cost, highly performant delivery via

CloudFront

Benefits Tips

Reuse S3 buckets: AWS allows up to 100

S3 buckets per account; use CloudFront

behaviors and IAM to host multiple domains

on a single S3 bucket

Cache-control headers Don’t forget to set

CacheControl headers on your objects to

ensure caching benefits

Secure delivery: Use CloudFront to provide

SSL delivery with custom certificates on

your domain

Learn more about AWS at Ooyala: engineering.ooyala.com

Join our team and build on the cloud: ooyala.com/careers

THANK YOU

http://bit.ly/awsevals