choosing a proxy server oscon 2014

51
Choosing A Proxy Server OSCON 2014 Bryan Call Yahoo

Upload: diane

Post on 22-Feb-2016

53 views

Category:

Documents


0 download

DESCRIPTION

Choosing A Proxy Server OSCON 2014. Bryan Call Yahoo. About Me. Apache Committer and PMC for Apache Traffic Server (ATS) Apache Member IETF HTTP/2 TLS TCP Enhancement Yahoo - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Choosing A Proxy Server OSCON 2014

Choosing A Proxy ServerOSCON 2014

Bryan CallYahoo

Page 2: Choosing A Proxy Server OSCON 2014

About Me• Apache

– Committer and PMC for Apache Traffic Server (ATS) – Apache Member

• IETF– HTTP/2– TLS– TCP Enhancement

• Yahoo– WebRing, GeoCities, People Search, Personals, Tiger Team, Platform Architect,

Edge Group, Network Research, Apache/IETF

Page 3: Choosing A Proxy Server OSCON 2014

Overview• Types of Proxies• Features• Architecture• Cache Architecture• Performance• Pros and Cons

Page 4: Choosing A Proxy Server OSCON 2014

How are you going to use a proxy server?

Page 5: Choosing A Proxy Server OSCON 2014

Reverse Proxy

Page 6: Choosing A Proxy Server OSCON 2014

Reverse Proxy• Proxy in front of your own web servers• Caching?• Geographic location?• Connection handling?• SSL termination?• SPDY support?• Adding business logic?

Page 7: Choosing A Proxy Server OSCON 2014

Forward Proxy

Page 8: Choosing A Proxy Server OSCON 2014

Intercepting Proxy

Page 9: Choosing A Proxy Server OSCON 2014

Forward / Intercepting Proxy• Proxy in front of the Internet• Configure clients to use proxy?• Caching?• SSL - CONNECT?• SSL - termination?

Page 10: Choosing A Proxy Server OSCON 2014

Choices

Page 11: Choosing A Proxy Server OSCON 2014

Plenty of Proxy Servers

PerlBal

Page 12: Choosing A Proxy Server OSCON 2014

Plenty of Proxy Servers

Page 13: Choosing A Proxy Server OSCON 2014

Features And Options

Page 14: Choosing A Proxy Server OSCON 2014

FeaturesATS NGiNX Squid Varnish Apache httpd

mod_proxy

Reverse Proxy Y Y Y Y YForward Proxy Y N Y N YTransp. Proxy Y N Y N YPlugin APIs Y Y partial Y YCache Y Y Y Y YESI Y N Y partial NICP Y N Y N NSSL Y Y Y N YSPDY Y Y N N partial

Page 15: Choosing A Proxy Server OSCON 2014

SSL Features

Source: https://istlsfastyet.com/ - Ilya Grigorik

Page 16: Choosing A Proxy Server OSCON 2014

What type of proxy do you need?• Of our candidates, only three fully supports all

proxy modes

Page 17: Choosing A Proxy Server OSCON 2014

HTTP/1.1 Compliance

Page 18: Choosing A Proxy Server OSCON 2014

HTTP/1.1 Compliance• Accept-Encoding - gzip• Vary• Age• If-None-Match

Page 19: Choosing A Proxy Server OSCON 2014

How things can go wrong: Vary$ curl -D - -o /dev/null -s --compress http://10.118.73.168/HTTP/1.1 200 OKServer: nginx/1.3.9Date: Wed, 12 Dec 2012 18:00:48 GMTContent-Type: text/html; charset=utf-8Content-Length: 8051Connection: keep-aliveCache-Control: public, max-age=900Last-Modified: Wed, 12 Dec 2012 17:52:42 +0000Expires: Sun, 19 Nov 1978 05:00:00 GMTVary: Cookie,Accept-EncodingContent-Encoding: gzip

Page 20: Choosing A Proxy Server OSCON 2014

How things can go wrong: Vary$ curl -D - -o /dev/null -s http://10.118.73.168/HTTP/1.1 200 OKServer: nginx/1.3.9Date: Wed, 12 Dec 2012 18:00:57 GMTContent-Type: text/html; charset=utf-8Content-Length: 8051Connection: keep-aliveCache-Control: public, max-age=900Last-Modified: Wed, 12 Dec 2012 17:52:42 +0000Expires: Sun, 19 Nov 1978 05:00:00 GMTVary: Cookie,Accept-EncodingContent-Encoding: gzip

EPIC FAIL!

Note: no gzip request

Page 21: Choosing A Proxy Server OSCON 2014

CoAdvisor HTTP protocol quality tests for reverse proxies

ATS 3.3.1

Nginx 1.3.9

Squid 3.2.5

Varnish 3.0.3

0 100 200 300 400 500 600

Failures Violations Success

49%

81%

51%

68%

Page 22: Choosing A Proxy Server OSCON 2014

CoAdvisor HTTP protocol quality tests for reverse proxies

ATS 3.3.1

Nginx 1.3.9

Squid 3.2.5

Varnish 3.0.3

0 100 200 300 400 500 600

Failures Violations Success

25%

6%

27%

15%

Page 23: Choosing A Proxy Server OSCON 2014

Architecture

Page 24: Choosing A Proxy Server OSCON 2014

Architecture And Process Models• Multithreading• Events• Process• Fibers

– Co-operative multitasking, getcontext/setcontext

Page 25: Choosing A Proxy Server OSCON 2014

Threads

Page 26: Choosing A Proxy Server OSCON 2014

Threads• Pros

– Easy to share memory– Lightweight context switching

• Cons– Easy to (accidently) share memory

• Overwriting another threads memory– Locking

• Deadlocks, race conditions, starvation

Page 27: Choosing A Proxy Server OSCON 2014

Event Processing

Page 28: Choosing A Proxy Server OSCON 2014

Problems with Event Processing• Doesn’t work well with

blocking APIs– open(), locking

• It doesn’t scale on SMP by itself

Page 29: Choosing A Proxy Server OSCON 2014

Process Model And ArchitectureATS NGiNX Squid Varnish Apache httpd

mod_proxy

Threads X X XEvents X X X partial XProcesses X X X

Page 30: Choosing A Proxy Server OSCON 2014

Caching Architecture

Page 31: Choosing A Proxy Server OSCON 2014

Cache• Mainly two types

– File system– Database like

• In memory index– Bytes per object

• Minimize disk seeks and system calls

Page 32: Choosing A Proxy Server OSCON 2014

CacheATS NGiNX Squid Varnish Apache httpd

mod_cache

File system X X Xmmap XRaw disk/direct IO X XRam cache X XMemory index X X X*Persistent cache X X X X

Page 33: Choosing A Proxy Server OSCON 2014

Performance Testing

Page 34: Choosing A Proxy Server OSCON 2014

ATS Configurationetc/trafficserver/remap.config:

map / http://origin.example.cometc/trafficserver/records.config:

CONFIG proxy.config.http.server_ports STRING 80CONFIG proxy.config.accept_threads INT 3

Page 35: Choosing A Proxy Server OSCON 2014

NGiNX Configurationworker_processes 24;access_log logs/access.log main;

proxy_cache_path /mnt/nginx_cache levels=1:2 keys_zone=my-cache:8m max_size=16384m inactive=600m;proxy_temp_path /mnt/nginx_temp;

server { set $ae ""; if ($http_accept_encoding ~* gzip) { set $ae "gzip"; }

location / { proxy_pass http://origin.example.com; proxy_cache my-cache; proxy_set_header If-None-Match ""; proxy_set_header If-Modified-Since ""; proxy_set_header Accept-Encoding $ae; proxy_cache_key $uri$is_args$args$ae; }

location ~ /purge_it(/.*) { proxy_cache_purge example.com $1$is_args$args$myae }

Page 36: Choosing A Proxy Server OSCON 2014

Squid Configurationhttp_access allow allhttp_port 80 accelworkers 24

cache_mem 4096 MBmemory_cache_shared oncache_dir rock /usr/local/squid/cache 1000 max-size=32768cache_peer origin.example.com parent 80 0 no-query originserver

Page 37: Choosing A Proxy Server OSCON 2014

Varnish Configurationbackend default { .host = ”origin.example.com”; .port = "80";}

Page 38: Choosing A Proxy Server OSCON 2014

Varnish Configuration (Cont)

sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=4000sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=200sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=1000 -p session_linger=0sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=1000 -p session_linger=10

Page 39: Choosing A Proxy Server OSCON 2014

Apache httpd ConfigurationLoadModule cache_module modules/mod_cache.soLoadModule cache_disk_module modules/mod_cache_disk.soLoadModule proxy_module modules/mod_proxy.soLoadModule proxy_http_module modules/mod_proxy_http.soInclude conf/extra/httpd-mpm.confProxyPass / http://origin.example.com/

<IfModule mod_cache_disk.c> CacheRoot /usr/local/apache2/cache CacheEnable disk / CacheDirLevels 5 CacheDirLength 3</IfModule>

MaxKeepAliveRequests 10000

Page 40: Choosing A Proxy Server OSCON 2014

Benchmark 1• 1,000 clients• 8KB response• 100% cache hit• Keep-alive on• 100K rps rate limited

Page 41: Choosing A Proxy Server OSCON 2014

• Squid used the most CPU and the worst median latency

• 95th percentile latency with NiGNX, Squid and httpd

ATS NGiNX Squid Varnish httpd0

500

1000

1500

2000

2500

RPS / CPU Usage

ATS NGiNX Squid Varnish httpd0

20000

40000

60000

80000

100000

120000

Requests Per Second

ATS NGiNX Squid Varnish httpd02468

1012141618

Latency

Median

95th

Page 42: Choosing A Proxy Server OSCON 2014

Benchmark 2• 1,000 clients• 8KB response• 100% cache hit• Keep-alive off

Page 43: Choosing A Proxy Server OSCON 2014

• Squid used the most CPU again

• NGiNX had latency issues• ATS most throughput

ATS NGiNX Squid Varnish httpd0

500

1000

1500

2000

2500

RPS / CPU Usage

ATS NGiNX Squid Varnish httpd0

5000

10000

15000

20000

25000

30000

Requests Per Second

ATS NGiNX Squid Varnish httpd0

5

10

15

20

25

30

35

40

Latency

Median

95th

Page 44: Choosing A Proxy Server OSCON 2014

ATS• Pros

– Scales well automatically, little config needed– Best cache implementation

• Cons– Too many config files– Too many options in the default config files (5.x fixed

this)

Page 45: Choosing A Proxy Server OSCON 2014

NGiNX• Pros

– Lots of plugins– FastCGI support

• Cons– HTTP/1.1 compliance– Latency issues around accepting new connections– Rebuild server for new plugins

Page 46: Choosing A Proxy Server OSCON 2014

Squid• Pros

– Best HTTP/1.1 compliance• Cons

– Memory index for cache using 10x that of ATS– Least efficient with CPU– Worst median latency for keep-alive benchmarks

Page 47: Choosing A Proxy Server OSCON 2014

Varnish• Pros

– VCL (Varnish Configuration Language)• Can do a lot without writing plugins

• Cons– Thread per connection– mmap for cache

• Persistence is experimental– No SSL or SPDY support

Page 48: Choosing A Proxy Server OSCON 2014

Apache httpd• Pros

– Lots of plugins– Most used http server– Best 95th percentile latency for non-keep-alive

• Cons– SPDY Support

Page 49: Choosing A Proxy Server OSCON 2014

Why ATS?• Scales well

– CPU Usage, auto config• Cache scales well

– Efficient memory index, minimizes seeks• Apache Community• Plugin support

– Dynamic loading of plugins and easy to port existing plugins over

Page 50: Choosing A Proxy Server OSCON 2014

References• ATS - http://trafficserver.apache.org/• NGiNX - http://nginx.org/• Squid - http://www.squid-cache.org/• Varnish - https://www.varnish-cache.org/• Apache httpd - http://httpd.apache.org/

Page 51: Choosing A Proxy Server OSCON 2014