experiences with coralcdn a five-year operational view
DESCRIPTION
Experiences with CoralCDN A Five-Year Operational View. Michael J. Freedman Princeton University www.coralcdn.org. A Cooperative, Self-Organizing CDN. Client. Resolver. 2. 5. 1. CoralCDN HTTP Proxy. CoralCDN DNS Server. CoralCDN HTTP Proxy. CoralCDN DNS Server. Coral index node. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/1.jpg)
Experiences with CoralCDN
A Five-Year Operational View
Michael J. FreedmanPrinceton University
www.coralcdn.org
![Page 2: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/2.jpg)
CoralCDN
Goal: To make desired content widely availableregardless of publisher’s own resources, by
organizing and utilizing any cooperative resources
CoralCDNDNS Server
CoralCDNHTTP Proxy
Coral index nodeCoral index node
Coral index nodeCoral index node
CoralCDNHTTP Proxy
CoralCDNDNS Server
Coral index nodeCoral index node
CoralCDNHTTP Proxy
ClientResolver
12
34
5
6
CoralCDNDNS Server
Coral index nodeCoral index node
CoralCDNHTTP Proxy
A Cooperative, Self-Organizing CDN
![Page 3: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/3.jpg)
http://example.com/path
http://example.com.nyud.net/path
![Page 4: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/4.jpg)
Adopted by:
Clients
Servers
Third-parties
![Page 5: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/5.jpg)
Many of you have used CoralCDN
![Page 6: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/6.jpg)
Many of you have used CoralCDN
![Page 7: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/7.jpg)
Many of you have used CoralCDN
![Page 8: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/8.jpg)
Many of you have used CoralCDN
![Page 9: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/9.jpg)
Many of you have used CoralCDN
![Page 10: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/10.jpg)
Daily Request Volume2M clients – 2 TB content – 20K origin domains
From 300-400 PlanetLab servers
![Page 11: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/11.jpg)
CoralCDNDNS Server
CoralCDNHTTP Proxy
Coral index nodeCoral index node
CoralCDNDNS Server
Coral index nodeCoral index node
CoralCDNHTTP Proxy
CoralCDNDNS Server
CoralCDNHTTP Proxy
CoralCDN
CoralCDNHTTP Proxy
Based on peer-to-peer DHT
1.Distributed sloppy hash table, or DSHT
2.Democratizing content publication with Coral
Coral index nodeCoral index node
Coral index nodeCoral index node
![Page 12: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/12.jpg)
CoralCDNDNS Server
CoralCDNHTTP Proxy
Coral index nodeCoral index node
CoralCDNDNS Server
Coral index nodeCoral index node
CoralCDNHTTP Proxy
CoralCDNDNS Server
CoralCDNHTTP Proxy
CoralCDN
CoralCDNHTTP Proxy
Based on peer-to-peer DHT
1. Weakened consistency + algorithms that prevent tree saturation during lookup
2. Decentralized clustering for locality and hierarchical lookup
3. Cooperative HTTP / DNS that leverages locality
Coral index nodeCoral index node
Coral index nodeCoral index node
![Page 13: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/13.jpg)
CoralCDNDNS Server
CoralCDNHTTP Proxy
Coral index nodeCoral index node
CoralCDNDNS Server
Coral index nodeCoral index node
CoralCDNHTTP Proxy
CoralCDNDNS ServerCoralCDN
CoralCDNHTTP Proxy
Coral index nodeCoral index nodeCoralCDN
HTTP Proxy
Coral index nodeCoral index node
Virtualization Layer
Clients Origin Domains
Interactions with the External Environment
![Page 14: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/14.jpg)
1. Experiences
– Naming
– Fault Tolerance
– Resource management
2. Revisit CoralCDN’s design
![Page 15: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/15.jpg)
Naming
x
Flexible, open API
Mismatch with domain-based access control policies
![Page 16: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/16.jpg)
Rewrite rules in origin webservers
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx
RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$
RewriteRule ^(.*)$ http://%{HTTP_HOST}.nyud.net%{REQUEST_URI} [R,L]
CoralCDN’s Platform-as-a-Service API
![Page 17: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/17.jpg)
Sites integrate with load/bandwidth monitoring
Elastic Provisioning
CoralCDN’s Platform-as-a-Service API
Rewrite rules in origin webservers
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx
RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$
RewriteCond %{HTTP_REFERER} slashdot\.org [NC]
RewriteCond %{HTTP_REFERER} digg\.com [NC,OR]
RewriteCond %{HTTP_REFERER} blogspot\.com [NC,OR]
RewriteRule ^(.*)$ http://%{HTTP_HOST}.nyud.net%{REQUEST_URI} [R,L]
![Page 18: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/18.jpg)
Naming Conflation
1. Location to retrieve content
2. Human-readable name for administrative entity
3. Security policies to govern objects’ interactions
xx
http://domain /path.service2.service1
![Page 19: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/19.jpg)
Domain-based Security Policies
evil.com
target.comCookies
Web Page
Document Object Model
![Page 20: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/20.jpg)
Domain-based Security Policies
evil.com
target.comCookies
Web Page
Document Object Model
Defaults violate least privilegeDefaults violate least privilege
.nyud.net
.nyud.net
![Page 21: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/21.jpg)
Fault Tolerance: Failure Decoupling
x
Internal failures:•DHT nodes•DNS servers, HTTP proxies•Management service
External failures:•Interactions with origin sites
![Page 22: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/22.jpg)
happens!
Origin Status
1. Unresponsive
2. Returns error code
3. Reply truncated
CoralCDN Reaction
• Cache negative results
• Serve stale content
• Use whole-file overwrites
![Page 23: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/23.jpg)
happens!
Origin Status
1. Unresponsive
2. Returns error code
3. Reply truncated
CoralCDN Reaction
• Cache negative results
• Serve stale content
• Use whole-file overwrites
Maintain status quo unless improvements are possibleMaintain status quo unless improvements are possible
![Page 24: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/24.jpg)
What is “failure”?
Return values should have fail-safe defaultsReturn values should have fail-safe defaults
![Page 25: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/25.jpg)
Resource Management
x
Control over bandwidth consumption
Control and visibility into environment’s resources
![Page 26: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/26.jpg)
Some timeline…
Mar 2004CoralCDNreleased on PlanetLab
![Page 27: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/27.jpg)
Some timeline…
Mar 2004CoralCDNreleased on PlanetLab
Aug 2004Slashdotted
![Page 28: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/28.jpg)
Some timeline…
Mar 2004CoralCDNreleased on PlanetLab
Aug 2004Slashdotted
Dec 2004AsianTsunami
1. PlanetLab traffic jumps2. Site threatens to yank PL3. PL admin kills slice4. Slice restored next day5. Initiates discussion of
resource limits for slices
![Page 29: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/29.jpg)
?
Avg
MB
per h
our (
d i)Demand >> Supply:
Enter Fair-Sharing Algorithms
Domains with heaviest consumption
Σi di ≤ S
![Page 30: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/30.jpg)
Avg
MB
per h
our (
d i)Demand >> Supply:
Enter Fair-Sharing Algorithms
λ
find max λ, s.t.
Σi min (λ, di) ≤ S
Domains with heaviest consumption
![Page 31: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/31.jpg)
Demand >> Supply:Enter Fair-Sharing Algorithms
find max λ, s.t.
Σi min (λ, di) ≤ Sλ
Domains with heaviest consumption
![Page 32: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/32.jpg)
Admission Control under Fair-Sharing~10 kB imgs
3.3% rejected~10 kB imgs
3.3% rejected~5 MB videos89% rejected~5 MB videos89% rejected
Demand > 10 TB Supply ≤ 2 TB
![Page 33: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/33.jpg)
Some timeline…
Mar 2004CoralCDNreleased on PlanetLab
Aug 2004Slashdotted
Dec 2004AsianTsunami
1. PlanetLab traffic jumps2. Site threatens to yank PL3. PL admin kills slice4. Slice restored next day5. Initiates discussion of
resource limits for slices
Mar 2006PL deploysbandwidththrottling
![Page 34: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/34.jpg)
Resource Management: Us vs. Them
Application Hammer
• Track HTTP traffic
• If site > fair share rate, reject via HTTP 403
• If total > peak rate, close server socket
Platform Hammer
• Track all network traffic
• If total > 80% daily rate, BW shaping in kernel
![Page 35: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/35.jpg)
Resource Management: Us vs. Them
• Track HTTP traffic
• If site > fair share rate, reject via HTTP 403
• If total > peak rate, close server socket
• Track all network traffic
• If total > 80% daily rate, BW shaping in kernel
Application Hammer Platform Hammer
Lower layers should expose greater visibility and control over resourcesLower layers should expose greater visibility and control over resources
Result: HTTP traffic is 1/2 - 2/3 of all traffic
![Page 36: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/36.jpg)
1. Experiences
– Naming
– Fault Tolerance
– Resource management
2. Revisit CoralCDN’s design
![Page 37: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/37.jpg)
Usage Scenarios
1. Resurrecting old content2. Accessing unpopular content
![Page 38: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/38.jpg)
Usage Scenarios
1. Resurrecting old content2. Accessing unpopular content3. Serving long-term popular content
![Page 39: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/39.jpg)
Usage Scenarios
1. Resurrecting old content2. Accessing unpopular content3. Serving long-term popular content
![Page 40: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/40.jpg)
Usage Scenarios
1. Resurrecting old content2. Accessing unpopular content3. Serving long-term popular content
Top URLs % Reqs Agg Size (MB) 0.01 % 49.1 % 14 0.10 % 71.8 % 157 1.00 % 84.8 % 3744 10.00 % 92.2 % 28734
Result FrequencyLocal Cache 70.4 %Origin Site 9.9 %
CoralCDN Proxy 7.1 %4xx/5xx Error 12.6 %
![Page 41: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/41.jpg)
Usage Scenarios
1. Resurrecting old content2. Accessing unpopular content3. Serving long-term popular content4. Surviving flash crowds to content
![Page 42: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/42.jpg)
Usage Scenarios
1. Resurrecting old content2. Accessing unpopular content3. Serving long-term popular content4. Surviving flash crowds to content
![Page 43: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/43.jpg)
Usage Scenarios
1. Resurrecting old content2. Accessing unpopular content3. Serving long-term popular content4. Surviving flash crowds to content
24% epochs ≥ 1 domain with10x incr
0.006% epochs ≥ 1 domain with 100x incr
0 % epochs ≥ 1 domain with 1000x incr
5 second epochs 10 minute epochs
99.93% epochs ≥ 1 domain with10x incr
28% epochs ≥ 1 domain with100x incr
0.21% epochs ≥ 1 domain with 1000x incr
![Page 44: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/44.jpg)
Conclusions?
• Most requested content is long-term popular and already cached locally
• “Flash” crowds occur, but on order of minutes
![Page 45: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/45.jpg)
Conclusions?
• Most requested content is long-term popular and already cached locally
• “Flash” crowds occur, but on order of minutes
• Focus on long-term popular
• Little / no HTTP cooperation
• Global discovery (e.g., DNS)
• Focus on long-term popular
• Little / no HTTP cooperation
• Global discovery (e.g., DNS)
• Focus on flash crowds
• Regional coop. as default
• Global coop. as failover
• Focus on flash crowds
• Regional coop. as default
• Global coop. as failover
![Page 46: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/46.jpg)
Reconfiguring CoralCDN’s design• Leverage Coral hierarchy for lookup
Latency 90% Origin Load 5% Failover to global 0.5%
![Page 47: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/47.jpg)
Reconfiguring CoralCDN’s design• Leverage Coral hierarchy for lookup
• During admission control, bias against long-term use
Σi min (λ, di) < S
heavily weight history in ewmaλ
![Page 48: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/48.jpg)
1. Experiences– Naming
– Fault Tolerance
– Resource management
2. Revisit CoralCDN’s design– Current design unnecessary for deployment / most use
– Easy changes to promote flash-crowd mitigation
Conclusions
![Page 49: Experiences with CoralCDN A Five-Year Operational View](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815037550346895dbe3357/html5/thumbnails/49.jpg)
Can we reach Internet scale?
www.firecoral.net
Initial beta-release
of browser-based P2P web cache