Bellwether: Surrogate Services for Popular Content

Duane Wessels & Ted Hardie

NANOG 19

June 12, 2000

The Slashdot Effect is a DDOS

• “CNN Events” can:– melt your network– overwhelm servers dedicated to specific content– prevent maintenance designed to fix the

problem.

• This creates a denial of service for other content hosted on that network.

A moving target is harder to hit.

• A demand-driven surrogate located at the network border:– Moves the content away from low capacity

networks.– Can handle the traffic for sites which

experience sudden popularity.– Can help keep internal links uncongested

What is a surrogate?

surrogate: An intermediary program which acts as a server or tunnel for the purpose of responding to requests on behalf of one or more origin servers. Requests are serviced internally from a cache or by tunnelling them on to origin servers. Surrogates are also known as "reverse proxies" and "(origin) server accelerators".

» Draft-ietf-wrec-taxonomy-03.txt

No, really, what is a surrogate?

• Proxies act on behalf of users; surrogates act on behalf of content providers.

• A surrogate is any network element that acts on behalf of an origin server to respond to queries:– A mirror is a pre-populated surrogate.– A content delivery network (Akamai, Adero, Mirror

Image) may provide surrogate services.– A demand-driven surrogate is a system activated only

when popularity overloads an origin server or its network.

Bellwether

• Demand-driven surrogate based on – Squid – Zebra, – FreeBSD

• IP firewall

• GRE

– And ideas stolen from CenterTrack.

A picture is worth 1K words:

Step 1: Administrative Setup

• Configure a GRE tunnel from the surrogate to an internal router.

• Configure the surrogate as a BGP peer of the border router.

• Add origin hostnames to Squid access control list.

Step 2: Activation

• The surrogate injects a route to the popular origin server into border router’s BGP table.

• The surrogate configures firewall rules to divert new HTTP connections to Squid.

• Existing TCP connections and other traffic flow through GRE tunnel to the origin.

Step 3: Operation

• Squid creates a cache of popular content by forwarding requests to the origin server via the GRE tunnel and storing responses.

• Cache hits are served from Squid, reducing the load on origin server and network alike.

Simulation Workload

• An origin server with a network bottleneck publishes suddenly popular content.

• Client requests increase from 5 to 100 per second over 15 minutes.

• Content remains popular for 2 hours, then trails off over 4 hours.

• Target hit ratio is 90%.• Surrogate is PII/333 with 512 RAM and 2 SCSI

disks.

What if you need more?

• For this result set, the surrogate is a dual PIII/550 Xeon with 2GB RAM and 10 SCSI disks.

• Peak throughput is 475 HTTP requests per second.– Mean response size is 13KB.– About 45 Mbps of data flow.

Next Steps

• Improve error handling.– Handle overload by passing overflow traffic back to

origin server.

– Withdraw route in the event of Squid failure.

• Use NECP to signal surrogate to start/stop service.– NECP daemon process and API

– Prototype integration in Apache

• Integration with higher layer switches.

Final Questions

• When you see a popularity spike, what melts?

• What kinds of processes and devices need to activate a surrogate?

Handy URLs:

• To pick up a copy of bellwether:– ftp://ftp.equinix.com/bellwether

• To discuss surrogate deployments:– surrogates-request@equinix.com

– (Majordomo syntax)

• Contact Ted or Duane:– hardie@equinix.com– wessels@packet-pushers.com