amazon cloudfront best practices and anti-patterns

BEST PRACTICES & ANTI-PATTERNS

CLOUDFRONT

06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS

ABOUT ME

• Solutions Architect

• Early AWS adopter (2007)

• Built Cotton On Group’s AWS Infrastructure (2012)

2


AGENDA

• CloudFront in a nutshell

• CloudFront Best Practices/Anti-Patterns

3


01Content Delivery Network Serves Static and Dynamic Content

4

CLOUDFRONT


01Global, Redundant, Scalable Low Latency, High Bandwidth Applications

5

CLOUDFRONT


01Cost Effective Transparent, PAYG, Price Classes

6

CLOUDFRONT


01Wish list Edge Side Includes and Reporting

7

CLOUDFRONT


MINIMUM TTL

REQUESTS

PROTOCOL

VIEWER

COOKIES & QUERY STRING FORWARDING

PATH PATTERNS

CNAME ALIASES

CACHE BEHAVIORS

STREAMING

DISTRIBUTIONDELIVERY METHOD

ORIGIN SERVER

DOMAIN NAME

OBJECTS OBJECT PATH/

CACHE KEY

ORIGIN PROTOCOL

DOWNLOAD

CLOUDFRONT (CF) CONCEPTS

8

[1-10, ORDERED]

[M:M]

[1:1]

[1-10]

06-11-2013 @ABHISHEKTIWARI | CLOUDFRONT BEST PRACTICES & ANTI-PATTERNS 9

CloudFront Distribution

Origin Servers Cache Behavior

S3 Bucket with images

S3 Bucket with photos

EC2 Instance

default (*)

photos(photos/*)

thumbnail(thumbnail/*)

Cached objects with cache key

Edge Locations Viewer

Client Browser

Cache key = Object path relative to origin + forwarded query string/cookies

From nearest edge location GET http://my.cloudfront.net/photos/profile.png

Returns object with matching cache key photos/profile.png

Cached Etag, Date, LastModified for each objects

http://<CloudFront domain name>/<object name in origin>

01CLOUDFRONT (CF) CONCEPTS

http://my

CloudFront Distribution

Origin Servers Cache Behavior

S3 Bucket with images

S3 Bucket with photos

EC2 Instance

default (*)

photos(photos/*)

thumbnail(thumbnail/*)

Cached objects with cache key

Edge Locations Viewer

Client Browser

Cache key = Object path relative to origin + forwarded query string/cookies

From nearest edge location GET http://my.cloudfront.net/photos/profile.png

Returns object with matching cache key photos/profile.png

Cached Etag, Date, LastModified for each objects

http://<CloudFront domain name>/<object name in origin>

Custom Origin Dynamic Content

S3 Origin Static Content

01CLOUDFRONT (CF) CONCEPTS

http://my


HOW IT WORKS

10

1. Client request file

2. If file in cache CF return fileNearest Edge Location

a. If file not in cache, CloudFront request to origin

b. Origin returns latest version of file, status 200, cached in CloudFront

1. Client request file

2. If file in cache CF return fileNearest Edge Location

a. If file in cache but expired, CloudFront request to origin

b. Cached file is latest, status 304, CloudFront keeps file in cache,

No change in Date, Expire Refreshed Age

Condition GET

Routing based on low latency/price class 01


1. 10 Origin Servers per CF Distribution

2. 10 Cache Behavior per CF Distribution

3. 10 CNAME per CF Distribution

4. 10 Cookies forwarded per Cache Behavior

11

01MAKE A NOTE


02Cache invalidation

ANTI-PATTERNS

12


02BEST PRACTICES


1. Versioning

13

02BEST PRACTICES


1. Versioning

2. Compression

13

02BEST PRACTICES


1. Versioning

2. Compression

3. Domain Sharding

13

02BEST PRACTICES


1. Versioning

2. Compression

3. Domain Sharding

4. Expiration

13

02BEST PRACTICES


1. Versioning

2. Compression

3. Domain Sharding

4. Expiration

5. Bucket Organization

13

02BEST PRACTICES


1. Versioning

2. Compression

3. Domain Sharding

4. Expiration


6. Logging

13

02BEST PRACTICES


1. Versioning

2. Compression

3. Domain Sharding

4. Expiration


6. Logging

7. Performance Testing

13

02BEST PRACTICES


02Expensive Unmanageable if you have object dependencies

CACHE INVALIDATION

14


02Programmatic 3 invalidation requests at any given time with each include maximum of 1000 files

CACHE INVALIDATION

15


02Eventual Consistency Invalidation takes time to propagate across all edge locations

CACHE INVALIDATION

16


02Deal breaker: Browser cache Versioning is best way to avoid the invalidation related issues

CACHE INVALIDATION

17


02Versioning using query strings File name plus query string with version: /static/profile.png?versionID=123

VERSIONING

18


02Requires Versioning enabled Origin (Custom/S3)

VERSIONING

19


UPLOAD TO S3 STATIC/IMAGES/PROFILE.PNG

PROFILE.PNG RETURN X-AMZ-VERSION-ID = 644C69E1

CF KEY STATIC/IMAGES/PROFILE.PNG?

VERSIONID=644C69E1

VERSIONED ENABLED S3 CONTENT

20

02


MINIFY+COMBINE MAIN.CSS MAIN.JS

CSS AND JS PHOTO.CSS, APP.CSS, PROFILE.CSS

PHOTO.JS, APP.JS, PROFILE.JS

VERSIONIFY BUILD NUMBER OR HASH OF

GIT/HG HEAD

CF KEY MAIN.CSS?V=468DF6B MAIN.JS?V=468DF6B

VERSIONIFY APPLICATION CONTENT

21

02


02Versioning using content based hash key File name as unique key based on file content: /static/712vds57tr18929812312enb.png

VERSIONING

22


MD5 HASH USING FILE CONTENT

B723EH0F0DF

PROFILE.PNG UPLOAD TO S3 USE HASH AS FILE NAME

STATIC/IMAGES/B723EH0F0DF.PNG

CF KEY STATIC/IMAGES/B723EH0F0DF.PNG

FOR S3 CONTENT

23

Avoid content duplication


MINIFY+COMBINE STATIC/MAIN.CSS STATIC/MAIN.JS

CSS AND JS PHOTO.CSS, APP.CSS, PROFILE.CSS

PHOTO.JS, APP.JS, PROFILE.JS

MD5 HASH OF FILE CONTENT AS FILE NAMES

STATIC/8972BW7DYF2H.CSS STATIC/67BFWU9HHUW.JS

CF KEY STATIC/8972BW7DYF2H.CSS STATIC/67BFWU9HHUW.JS

VERSIONIFY APPLICATION CONTENT

24

02


02Versioning using prefix/suffix File name with version prefix/suffix: /static/profile_123.png

File name with version prefix/suffix: /static/profile_ 712vds57tr18929812312enb.png

VERSIONING

25


02Compressed content is served faster and uses less bandwidth

COMPRESSION

26


02Can serve both compressed and uncompressed version of files

COMPRESSION

27


02For compression CF relies on the origin servers (Custom vs S3)

COMPRESSION

28


55Viewer request To receive compressed content browser request must include Accept-Encoding: gzip

COMPRESSION

29


02Upload both gzip and non-gzip versions of the file in the same S3 bucket

COMPRESSION

30


02When uploading gzip file to S3 set Content-Encoding to gzip

COMPRESSION

31


02HTML, CSS, JS etc On-the-fly compression by custom origin server (Nginx, Apache)

COMPRESSION

32


02Use one/both of these •On Origin server set Cache-Control max-age header •In Cache Behavior set Minimum TTL

EXPIRATION

33


02Default expiration 24hrs Set Cache-Control max-age or Minimum TTL very far future max {Cache-Control max-age directive, Cache Behavior Minimum TTL}

EXPIRATION

34


02Depending on request frequency, CloudFront might remove the object before its expiration

EXPIRATION

35

02Avoid Expire header After expiry browser cachability issues (stale response) Can not cache object if {Age <=0 OR Expire Date - Current Date <=0}

EXPIRATION

36


02Browser limit number of simultaneous synchronous connections to one server (n)

DOMAIN SHARDING

37

Client Browser Server

time ->

GET

200 OK

GET

200 OK

X n


02Browser treat CNAM aliases as different servers

Bypass parallel download limit using 2 or more CNAM aliases Route 53, Wild Card CNAME, Custom SSL

DOMAIN SHARDING

38


02Sharding across 2 CNAME aliases will double the parallel download

DOMAIN SHARDING

39

Client Browser CNAME2 ->Server

time ->

GET

200 OK

GET

200 OK

X n

Client Browser CNAME1 ->Server

time ->

GET

200 OK

GET

200 OK

X n


02On application side Enable CF sharding logic (template compilation etc)

DOMAIN SHARDING

40

02

41

 <a href="http://mysite.com/about"><img src="http://cf.mycloudfront.net/about.jpg" alt=""/></a>

 <a href="http://mysite.com/blogs"><img src="http://cf.mycloudfront.net/blogs.jpg" alt=""/></a>

 <a href="http://mysite.com/contact"><img src="http://cf.mycloudfront.net/cont.jpg" alt=""/></a>

 <a href="http://mysite.com/about"><img src="http://cf1.mycloudfront.net/about.jpg" alt=""/></a>

 <a href="http://mysite.com/blogs"><img src="http://cf2.mycloudfront.net/blogs.jpg" alt=""/></a>

 <a href="http://mysite.com/contact"><img src="http://cf1.mycloudfront.net/cont.jpg" alt=""/></a>

 <a href="http://mysite.com/about"><img src="{{ CDN_DOMAINS }}/about.jpg" alt=""/></a>

 <a href="http://mysite.com/blogs"><img src="{{ CDN_DOMAINS }}/blogs.jpg" alt=""/></a>

 <a href="http://mysite.com/contact"><img src="{{ CDN_DOMAINS }}/cont.jpg" alt=""/></a>

 <a href="http://mysite.com/about"><img src="http://mysite.com/about.jpg" alt=""/></a>

 <a href="http://mysite.com/blogs"><img src="http://mysite.com/blogs.jpg" alt=""/></a>

 <a href="http://mysite.com/contact"><img src="http://mysite.com/cont.jpg" alt=""/></a>

CDN with domain shardingCDN without domain sharding

HTML Template

Without CDN

Compiled OR DOM Manipulation

Compiled OR DOM Manipulation

http://mysite.com/about

http://cf.mycloudfront.net/about.jpg

http://mysite.com/blogs

http://cf.mycloudfront.net/blogs.jpg

http://mysite.com/contact

http://cf.mycloudfront.net/cont.jpg


http://cf1.mycloudfront.net/about.jpg


http://cf2.mycloudfront.net/blogs.jpg


http://cf1.mycloudfront.net/cont.jpg





http://mysite.com/about.jpg


http://mysite.com/blogs.jpg


http://mysite.com/cont.jpg


02Performance Considerations •When to shard?

•DNS lookup, Connection time, Evenly distribute

•Mobile browsers, Network congestion, Battery Life •SPDY (HTTP MULTIPLEXING)

DOMAIN SHARDING

42


02Limited Cache Behavior Plan your bucket organisation (object keys)

BUCKET ORGANISATION

43


02Watch out for the object paths /video/uploads/, /audio/uploads/, /images/uploads/ BAD

/uploads/video/, /uploads/audio/, /uploads/image/ GOOD

Use wild card patterns (/uploads/*)

BUCKET ORGANISATION

44


02Reporting non-existence •Object/Content Popularity Report

•Cache Hit Ratio

•Edge Location Report

•Error Report •Mapping Geo-IP and Geo-location

LOGGING

45


02Enable logging Store access log file to S3. One bucket per distribution.

LOGGING

46


02Tons of log files As traffic levels climb, the number of log files will increase Delayed (24H), missing logs, not in order

LOGGING

47


02Combine log files and analyze Using EMR (Custom scripts, CloudFront LogAnalyzer, EmrEtlRunner)

LOGGING

48


02Measuring Latency Delay in DNS resolution and content delivery

PERFORMANCE TESTING

49


02Backbone Testing (Ideal Word) Measuring latency from

CDN Server → Backbone

PERFORMANCE TESTING

50


02Last Mile Testing (Real World) Measuring latency from

CDN Server → Backbone → Device 3rd party service (Gomez), Application instrument

PERFORMANCE TESTING

51


02Quantifying Scalability Tradition Load Testing not useful One Client → Same DNS Request → Same Set of IPs

PERFORMANCE TESTING

52


02Load Testing (Edge Location, EIP) 1.Multiple Client from different geolocations 2.Independent DNS request, different set of IPs 3.Distribute load across set of IPs

PERFORMANCE TESTING

53

Q & A

THANK YOU

http://bit.ly/abhishektiwarihttp://abhishek-tiwari.com

http://bit.ly/abhishektiwari

http://bit.ly/abhishektiwari

amazon cloudfront best practices and anti-patterns

Technology