2016 utah cloud summit: aws s3
TRANSCRIPT
![Page 1: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/1.jpg)
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Raul Frias, Solutions Architect
January 2016
Amazon S3
![Page 2: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/2.jpg)
New for 2015…
Cross-region replication
Amazon S3 Standard-IA
AWS CloudTrail support for Amazon
S3
Amazon CloudWatch metrics for Amazon
S3
VPC endpoint for Amazon S3
Amazon S3 bucket limit increase
Event notifications
Read-after-write consistency in all regions
![Page 3: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/3.jpg)
Video sharing serviceVidShare
• Allow friends and families to record, edit and share video clips
• Great user experience with low latency
• Create thumbnails to playback videos
• Millions of customers
![Page 4: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/4.jpg)
Video sharing serviceVidShare Roadmap:
Launch
• Scale: Millions of users• Focus on US market• Create thumbnails for users to
playback videos
V2 Optimize on cost
V3 Expand globally
V4 Enterprise enablement
![Page 5: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/5.jpg)
Video sharing service
VidShare Design
• Use event notifications to trigger an AWS Lambda function
• Add thumbnails and metadata to Amazon DynamoDB
• Playback videos
VidShare
![Page 6: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/6.jpg)
- Thumbnail- Update Index- Update WebApp
Video sharing service
Event
MetadataThumbnaillogs
VidShare
VidShare Design
• Use event notifications to trigger an AWS Lambda function
• Add thumbnails and metadata to Amazon DynamoDB
• Playback videos
![Page 7: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/7.jpg)
Amazon S3 event notifications
Events
SNS topic
SQS queue
Lambda function
• Notification when objects are created via PUT, POST, Copy, or Multipart Upload, DELETE
• Filtering on prefixes and suffixes for all types of notifications
Fast IntegratedSimple
![Page 8: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/8.jpg)
Setup delete notifications – config and execute
![Page 9: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/9.jpg)
Launch
• Scale: Millions of users• Focus on US market• Create thumbnails for users to
playback videos
V2 Optimize on cost
V3 Expand globally
V4 Enterprise enablement
Optimizing VidShare
• 1 PB of storage and growing• No impact on user experience• Minimal code change
VidShare Roadmap:
TTTTTTTTTTTTT+3 daysT+3 daysT+3 daysT+3 daysT+3 daysT+3 daysT+3 daysT+3 daysT+3 daysT+3 daysT+3 daysT+3 daysT+5 daysT+5 daysT+5 daysT+5 daysT+5 daysT+5 daysT+5 daysT+5 daysT+5 daysT+5 daysT+5 daysT+5 daysT+ 15 daysT+ 15 daysT+ 15 daysT+ 15 daysT+ 15 daysT+ 15 daysT+ 15 daysT+ 15 daysT+ 15 daysT+ 15 daysT+ 15 daysT+ 15 daysT + 25 daysT + 25 daysT + 25 daysT + 25 daysT + 25 daysT + 25 daysT + 25 daysT + 25 daysT + 25 daysT + 25 daysT + 25 daysT + 25 daysT + 30 daysT + 30 daysT + 30 daysT + 30 daysT + 30 daysT + 30 daysT + 30 daysT + 30 daysT + 30 daysT + 30 daysT + 30 daysT + 30 daysT + 60 daysT + 60 daysT + 60 daysT + 60 daysT + 60 daysT + 60 daysT + 60 daysT + 60 daysT + 60 daysT + 60 daysT + 60 daysT + 60 daysT + 60 daysT + 90 daysT + 90 daysT + 90 daysT + 90 daysT + 90 daysT + 90 daysT + 90 daysT + 90 daysT + 90 daysT + 90 daysT + 90 daysT + 90 daysT + 90 daysT + 150 daysT + 150 daysT + 150 daysT + 150 daysT + 150 daysT + 150 daysT + 150 daysT + 150 daysT + 150 daysT + 150 daysT + 150 daysT + 150 daysT + 150 daysT + 250 daysT + 250 daysT + 250 daysT + 250 daysT + 250 daysT + 250 daysT + 250 daysT + 250 daysT + 250 daysT + 250 daysT + 250 daysT + 250 daysT + 250 daysT + 365 daysT + 365 daysT + 365 daysT + 365 daysT + 365 daysT + 365 daysT + 365 daysT + 365 daysT + 365 daysT + 365 daysT + 365 daysT + 365 daysT + 365 days
Acc
ess
Freq
uenc
y
![Page 10: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/10.jpg)
Choice of storage classes on Amazon S3
Standard
Active data Archive dataInfrequently accessed data
Standard - Infrequent Access Amazon Glacier
![Page 11: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/11.jpg)
11 9’s of Durability
Standard-Infrequent Access storageInfrequently accessed data
Designed for 99.9% availability
Durable AvailableSame throughput as
Amazon S3 Standard storage
High performance
• Server-side encryption• Use your encryption keys• KMS managed encryption keys
Secure• Lifecycle management• Versioning • Event notifications• Metrics
Integrated• No impact on user
experience• Simple REST API• Single bucket
Easy to use
![Page 12: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/12.jpg)
- Transition Standard to Standard-IA- Transition Standard-IA to Amazon Glacier
storage- Expiration lifecycle policy- Versioning support
Standard-Infrequent Access storage
Integrated with lifecycle
Integrated: Lifecycle management
![Page 13: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/13.jpg)
Save money on VidShare
Lifecycle policy
Standard Storage -> Standard-IA
<LifecycleConfiguration> <Rule> <ID>sample-rule</ID> <Prefix>documents/</Prefix> <Status>Enabled</Status> <Transition> <Days>30</Days>
<StorageClass>STANDARD-IA</StorageClass> </Transition> <Transition> <Days>365</Days>
<StorageClass>GLACIER</StorageClass> </Transition> </Rule> </LifecycleConfiguration>
![Page 14: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/14.jpg)
Save money on VidShare
Lifecycle Policy
Standard Storage -> Standard-IA
<LifecycleConfiguration> <Rule> <ID>sample-rule</ID> <Prefix>documents/</Prefix> <Status>Enabled</Status> <Transition> <Days>30</Days>
<StorageClass>STANDARD-IA</StorageClass> </Transition> <Transition> <Days>365</Days>
<StorageClass>GLACIER</StorageClass> </Transition> </Rule> </LifecycleConfiguration>
Standard-IA Storage -> Amazon Glacier
![Page 15: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/15.jpg)
Setup lifecycle policy
![Page 16: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/16.jpg)
Transition older videos to Standard-IA
![Page 17: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/17.jpg)
Save money on VidShare
• 1 PB of storage and growing• 1 PB for S3 Standard = $28,816/- per month for storage cost
• Videos older than 30 days are watched less than once a month on average
1 PB for S3 Standard-IA = $17,600/- per month for storage costSaving 39% on storage
* Assumes the highest public pricing tier
![Page 18: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/18.jpg)
VidShare is global!• Expand to global users• Share videos with families
across the globe• New user base in Japan• Need low latency for global
users
VidShare
Launch
• Scale: Millions of users• Focus on US market• Create thumbnails for users to
playback videos
V2 Optimize on cost
V3 Expand globally
V4 Enterprise enablement
Roadmap:
![Page 19: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/19.jpg)
Remote replicas managed by separate AWS accounts
SecureDistribute data to regional customers
Lower Latency
Store hundreds of miles apart
Compliance
Amazon S3 cross-region replicationAutomated, fast, and reliable asynchronous replication of data across AWS regions
![Page 20: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/20.jpg)
VidShare v3.0 – Global expansion
![Page 21: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/21.jpg)
VidShare for enterprises!• Enterprise customers want to record
meetings and share videos• Logically isolated section of the cloud• All users within the Amazon Virtual
Private Cloud (Amazon VPC)• Restricted Internet access
VidShare
Launch
• Scale: Millions of users• Focus on US market• Create thumbnails for users to
playback videos
V2 Optimize on cost
V3 Expand globally
V4 Enterprise enablement
Roadmap:
![Page 22: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/22.jpg)
Using Amazon S3 with VPC endpoints – Previously…
mybucket
InternetPUT S3
PU
T S3NAT\
Internet Gateway
• Public IP on EC2 instances and IGW
• Private IP on EC2 instances and NAT
Amazon S3 VPC endpointsAccess Amazon S3 from your Amazon VPC using VPC endpoints
![Page 23: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/23.jpg)
VP
C
End
poin
tPUT S3
VPC Policy
mybucket
Internet
Amazon S3 VPC endpointsAccess Amazon S3 from your Amazon VPC using VPC endpoints
Using Amazon S3 VPC endpoints
![Page 24: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/24.jpg)
Improved throughput from VPC resources to Amazon S3
High availability
High performanceHigh availability
Reduce cost by avoiding expensive NAT, Internet gateways
Lower cost
Simple to setup, no need to manage NATs and Internet gateways
Simple
Improved security, no need to route traffic through the internet
Secure
Amazon S3 VPC endpointsAccess Amazon S3 from your Amazon VPC using VPC endpoints
![Page 25: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/25.jpg)
Amazon S3 VPC endpoints
Using Amazon S3 VPC endpoints
• Control access to buckets from specific Amazon VPC endpoints, or specific VPCs
• Control which VPCs or VPC endpoints have access to your S3 buckets by using S3 bucket policies
![Page 26: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/26.jpg)
Audit logs Amazon S3Demonstrate compliance, improve security
Log Amazon S3 API using AWS CloudTrail
Track bucket-level operations• Creation and deletion of buckets• Changes to access control, lifecycle policy,
cross region replications policy etc.
Integrated with Amazon CloudWatch• Alarm if a specific API called
Configure once per AWS Account• Track multiple services with AWS CloudTrail
![Page 27: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/27.jpg)
Amazon S3 storage metricsUnderstand your Amazon S3 buckets
Amazon CloudWatch metrics for Amazon S3
Bucket-level metrics include:• Total bytes for Standard storage
• Total bytes for Standard-IA storage
• Total bytes for Reduced-Redundancy storage
• Total number of objects for a given S3 bucket
Alarm on S3 metrics• Set thresholds for alarms
Daily metrics• Metrics emitted daily, after midnight GMT
![Page 28: 2016 Utah Cloud Summit: AWS S3](https://reader035.vdocuments.net/reader035/viewer/2022062823/5870647e1a28ab48378b4a5d/html5/thumbnails/28.jpg)
Thank you!