extending your applications to the edge with cdns

Extending Applica/ons To The Edge With CDNs

Hooman Behesh/, VP Technology

Webperf meetup NYC – May 12 2015

What is a CDN

•  Global network of HTTP caches – Get content closer to users

•  Performance benefits •  offload

In the past…

Classic CDNs

•  Black box service •  Making changes sucked! •  LiRle to no real-‐/me interfaces – APIs or configura/on management

•  Lots of professional services •  Decent caching, terrible uncaching

Cacheable

Uncacheable Sta3c Dynamic

How classic CDNs look at your content

Cache-Control: max-age=31536000

Classic CDNs: dynamic content

•  Focused on delivery: transport op/miza/on

But some3mes…

I know you’re calling it dynamic, but it’s kind of sta3c some3mes

I know you’re calling it dynamic, but it’s kind of sta3c some3mes

That’s cool, how long is it sta3c for?

That’s cool, how long is it sta3c for? Some3mes a liFle bit,

some3mes a lot

Some3mes a liFle bit, some3mes a lot Neat!

Some3mes a liFle bit, some3mes a lot You look great today!

You look great today! Don’t change the subject! How do I

cache it?

Don’t change the subject! How do I

cache it? Cache it for half an

hour?

Cache it for half an hour? Some3mes it changes

much faster

Some3mes it changes much faster

Cache it for 10 minutes?

Cache it for 10 minutes? Are you just guessing

right now?

Are you just guessing right now?

Just trying to figure out your comfort

level with mistakes!

Expiry guessing!

Terrible Caching

Terrible Caching

Terrible Anxiety

or

(ex: news story)

Things are different today

Today…

•  Influenced by clouds •  Devops requires programmability and control •  CDNs should be part of the applica/on •  Real-‐/me integra/on –  Changes and APIs

•  Unpredictable content expiry •  Your content à you need control

What does this all mean?

Control

Visibility

Cacheability

Cacheability

•  Granularity in caching rules (i.e. cache the way you want) –  Customizable cache keys –  Ex: geo-‐ip caching –  Ex: normalizing user agents

•  Cache everything –  Expect high cache-‐hit ra/os – No cache-‐control anxiety!

Cache hit ra3os

Cache Hit Ra3o: Tradi3onal calcula3on

1 -‐ Requests to Origin Total Requests

Origin

Origin

Cache

TCP

Origin

Cache

HTTP

Origin

Cache

Origin

Cache

HTTP

Origin

Cache

HOT COLD

Origin

Cache

cache “hit”

Cache Hit Ra3o: Tradi3onal calcula3on

1 -‐ Requests to Origin Total Requests

Isn’t this beFer?

Hits Total Requests

@edge

Isn’t this beFer?

Hits Hits + Misses

@edge

Cache Hit Ra3o

vs. 1 -‐ Requests to Origin Total Requests

Hits

Hits + Misses @edge

Cache Hit Ra3o


Hits

Hits + Misses @edge

Offload

Cache Hit Ra3o


Hits

Hits + Misses @edge

Offload Performance

Effect on long tail content

Long Tail: Cacheable but seldom fetched

Popular Medium Tail (1hr) Long tail (6hr)


Connect (median)

Popular 14msec

1hr Tail 15msec

6hr Tail 16msec


Connect (median) Wait (median)

Popular 14msec 19msec

1hr Tail 15msec 26msec



Connect (median) Wait (median)

Popular 14msec 19msec


6hr Tail 16msec 32msec 6,400+ measurements

77,000+ measurements

38,000+ measurements


Isn’t this beFer?

“Dynamic” Content

We’ve been told:

Sta3c = Cacheable

Dynamic = Uncacheable

Cacheable

Uncacheable Sta3c

Dynamic

How our sites used to look

Cacheable

Uncacheable Sta3c Dynamic

How we’re told our sites look

Cacheable

Uncacheable Sta3c

Dynamic

How our sites actually look

Event Driven

Event-‐Driven content:

Sta3c for unpredictable periods of 3me

Three types of content •  Sta/c

–  Infrequent changes, can cache for a long /me –  Cache-‐control headers are good enough –  Examples: Images, javascript , css, etc

•  Dynamic –  Can’t cache at all -‐ must go to origin –  DSA and transport is key for performance –  Examples: Logins, credit card transac/ons, Ajax, etc

•  Event-‐driven –  Sta/c, but unpredictably –  Cache-‐control headers are not good enough –  Examples: That news story, wiki pages, user comments, sports scores, etc

Caching event-‐driven content

•  Not easy – Classic CDNs con/nue to consider it the same as dynamic (Bad caching or panic!)

– Need the right tools to cache •  You may not know expiry ahead of /me, but your app knows when things change

Cache normally and then

“uncache” when it changes

Uncaching!

•  Purging – Accessible through API – Granular •  Dependencies through labels, tags, or surrogate keys

– Must be instantaneous •  Ex: news story, part 2

Instant Purge

•  Enables caching of event-‐driven content –  Can deal with unpredictable invalida/on –  As pages update/publish, purge command is sent to CDN

•  Instant? –  Instant ≠ 12 minutes! –  Instant ≠ quick acknowledgement! –  <1sec –  Determinis/c and predictable

Example: CMS + Purge

WordPress: Before

CDN Node

WordPress: Before

CDN Node

Cache

WordPress: Ajer

CDN Node

WordPress: Ajer

CDN Node

HTTP/1.1 200 OKContent-Type: text/htmlContent-Length: 55,666Cache-Control: Totally Long Time!

WordPress: Ajer

CDN Node

WordPress: Ajer

CDN Node PURGE

WordPress: Ajer

CDN Node

(Has to be instantaneous!)

PURGE

WordPress: Ajer

CDN Node

HTTP/1.1 200 OKContent-Type: text/htmlContent-Length: 55,666Cache-Control: Totally Long Time!

WordPress: Ajer

CDN Node

Example: customer1.js

Before

CDN Node

Origin http://www.3rdparty.com/customer1.js

(Referer: www.customer1.com)

Before

CDN Node


Before

CDN Node

Origin

HTTP/1.1 200 OKCache-Control: max-age=60Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835

http://www.3rdparty.com/customer1.js

Before

CDN Node


(Ajer 1 min)

Before

CDN Node

Origin http://www.3rdparty.com/customer1.js VALIDATION

If-Modified-Since: Tue, 12 May 2015 19:51:30 GMT

(Ajer 1 min)

Before

CDN Node


304 Not Modified

(Ajer 1 min)

Ajer

CDN Node


Ajer

CDN Node

Origin

HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835


Ajer

CDN Node


(Ajer 1 min)

This happens many many 3mes! (many many happy visitors!)

Ajer

CDN Node

Origin

Customer1 changes config

(Ajer 1 min)

Ajer

CDN Node

Origin PURGE customer1.js

Customer1 changes config

(Ajer 1 min)

Ajer

CDN Node


(Ajer 1 min)

Ajer

CDN Node


HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Wed, 24 Sep 2014 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835

(Ajer 1 min)

Ajer

CDN Node

Origin

HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Wed, 24 Sep 2014 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835


(Ajer 1 min)

Control

Visibility

Cacheability

Programmability

•  Granular Invalida/on API •  Configura/on API •  Logic at the Edge •  All must be instantaneous and real /me

Control at the Edge

•  Moving app logic to the edge •  Ex: VCL (Varnish Configura/on Language) – Script-‐like configura/on for func/onality at the edge

•  Any mechanism offered by CDN to let you do stuff at the edge is good

Logic at the Edge: Examples •  Generate content at the edge •  HTTP header manipula/on •  Origin selec/on •  Caching rules •  Geo-‐IP rules •  Forcing SSL •  Serving stale content

–  stale-‐while-‐revalidate, stale-‐while-‐error •  Etc…

Example: Edge-‐generated Content

JSON Data Center ID

CDN Node

Origin http://www.site.com/which_datacenter.js

JSON Data Center ID

CDN Node

Origin

{ ‘datacenter’ : ‘LAX’ }

http://www.site.com/which_datacenter.js

VCL Snippet

JSONP Geo IP

CDN Node

Origin

myFunction({ “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237”});

http://www.site.com/geo_ip.js?jsonp=myFunction

JSONP Geo IP

CDN Node

Origin

somethingElse({ “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237”});

http://www.site.com/geo_ip.js?jsonp=somethingElse

Control

Visibility

Cacheability

Real-‐3me Analy3cs

•  Stats API •  Network stats •  HTTP stats (status codes, etc) •  Caching stats (hits, misses, errors, etc) •  All have to be real-‐/me – Historic data is also a must for long term stats analysis

Logging

•  Real /me! •  Log streaming – To any logging des/na/on endpoint – Ex: S3, FTP, syslog, etc

Example: Beacon Termina3on at the Edge

Before

CDN Node

Origin

Log Analysis

http://collector.site.com/beacon.img?a=1&b=2&c=3

Before

CDN Node

Origin

Log Analysis

HTTP/1.1 200 OKPragma: no-cacheExpires: Wed, 19 Apr 2000 11:43:00 GMTCache-Control: no-cache, no-storeLast-Modified: Wed, 21 Jan 2004 19:51:30 GMTContent-Type: image/gifDate: Fri, 20 Jun 2014 12:22:20 GMTServer: ApacheContent-Length: 35


Ajer

CDN Node

Origin http://collector.site.com/beacon.img?a=1&b=2&c=3

Ajer

CDN Node

Origin

HTTP/1.1 200 OKPragma: no-cacheExpires: Wed, 19 Apr 2000 11:43:00 GMTCache-Control: no-cache, no-storeLast-Modified: Wed, 21 Jan 2004 19:51:30 GMTContent-Type: image/gifDate: Fri, 20 Jun 2014 12:22:20 GMTServer: ApacheContent-Length: 35


Ajer

CDN Node

Origin

HTTP/1.1 204 No ContentDate: Sat, 21 Jun 2014 23:21:12 GMTServer: Awesome ServerContent-Length: 0


Ajer

CDN Node

Origin

Syslog / S3 / FTP/etc


Summary

•  CDNs should be an extension of your applica/ons: –  Flexible caching, predictable edge caching, and instant purging to un-‐cache!

–  Control at the edge –  Real-‐/me visibility and analy/cs

•  Full real-‐/me programma/c interface is a must

Thank you!

[email protected]

extending your applications to the edge with cdns

Internet

hits totalrequests

http origin cache

origin cache http

tcp origin cache

origin cache cachehit

origin cache hot cold

geoipcaching ex

edge ooad performance