apache performance

Apache Web PerformanceLeveraging Apache to make your site fly

1Friday, March 16, 12

Who am I?&

Why would you listen to me?


Devon HillardWeb Application Architect

Built 10MinuteMail

Run Spark::red, an enterprise eCommerce Hosting company

Complex Web Apps with extremely high traffic and critical performance needs

[email protected]


Been doing this for 14 years.10MinuteMail handled a Slashdotting, front page listing on Yahoo Tech, and front page Digg at the same time.Worked on sites such as AT&T.com, People’s Choice Awards, Payless Shoes, Vermont Country Store.

What is Performance?


Server-Side Performance• Traffic Capacity

• Request handling speed

• Request handling throughput

• Lower CPU/memory/bandwidth usage

• Scalability


Client-Side Performance• Page Load Time

• First draw

• Inter-actable

• Complete

• Page Interaction Responsiveness

• Time to Complete Use Case


When a user sees the page start loading, to when they can do stuff, to when the browser’s loading animation is done.Use Ajax to pre-populate fields, type-ahead, real-time field validation, reduce the number of full page submits/requests.

Solving for client side performance can have a dramatic impact on the server side performance. Cached assets draw pages faster AND reduce requests the server has to handle.

These two are TIED

• Browser caching of static assets reduces page load time AND reduces the number of requests the server has to handle for the same number of page views

• AJAX requests can typically be handled with far fewer resources than full page requests

• Reduced asset sizes means less bandwidth used and shorter request response sending times

• Solving for the Client brings Server performance gains!


In engine building they say “Build for torque and the horsepower will follow”

Why Should I Care?


Everyone Wins!

• Client-Side Performance means

• happier users

• increased conversions

• increased SEO ranking

• Server-Side Performance means

• more capacity on same hardware

• saves money

• scaling is easier


Increased Conversions

• +100 ms of page load time = 1% drop in sales

• +500 ms of page load time = 20% drop in searches

• +400 ms of page load time = 5-9% increase in clicking “back” button before the page finished loading


We’ve all seen these stats, but they’re worth repeating. This isn’t ivory tower stuff, this is simple business.

Why Apache?


Why I Use Apache• Popular web server - it’s everywhere

• Easy to install, troubleshoot, and find information on

• Mature and stable

• Lots of extensions

• Enterprise support requirements

• It Is fast enough to max out your hardware!


Why not nginx (engine-X), Lighttpd, etc...?Extensions like mod_pagespeed, mod_cluster, and more.

CPU Util @ 930 Mbit/sechttp://www.webperformance.com/load-testing/blog/2011/11/what-is-the-fastest-webserver/


http://www.webperformance.com/load-testing/blog/2011/11/what-is-the-fastest-webserver/

http://www.webperformance.com/load-testing/blog/2011/11/what-is-the-fastest-webserver/

PCA Awards

• Saw 4x the planned for traffic during spikes

• 140,000+ pages/minute - 2,333 pps (2012)

• 3,000+ Mbit/sec (2011)

• 1,200+ Mbit/sec PLUS Akamai CDN offloaded traffic (2012)

• The site stayed up and was quick to load and responsive to interact with the whole time

iftop output - 613 Mbit/sec


Puppies15Friday, March 16, 12

Basic Apache Configs


The problem with Apache configs are that the defaults are typically extremely out of date and based on much older hardware, network bandwidth, browser abilities, and application complexity

Every Appis

Different


Which MPM?• Worker MPM scales for high traffic without running out

of memory better

• Less time spent tuning worker configs

• Unless you’re still using non-threadsafe Apache code (some PHP, etc..)

• New in Apache 2.4 is the Event MPM


Limited testing I did with Event MPM shows it’s ~5% faster than Worker, without real tuning.

Default<IfModule worker.c> StartServers 2 MaxClients 150 MinSpareThreads 25 MaxSpareThreads 75 ThreadsPerChild 25 MaxRequestsPerChild 0</IfModule>

Mine<IfModule worker.c> ThreadLimit 100 StartServers 5 MaxClients 5500 ServerLimit 200 MinSpareThreads 100 MaxSpareThreads 1000 ThreadsPerChild 100 MaxRequestsPerChild 0</IfModule>

MPM Worker Configs


Just set it and forget it.If you have memory leak issues, set MaxRequestsPerChild to something like 5,000.

ab with small fileVersion Apache 2.2.3 MPM Worker

Document Size 119 bytes

Concurrency Level 1,000

Total Requests 100,000

Requests Per Second 20,790 r/s

Throughput 8,077 kb/s

hex core X5675


ab with large fileVersion Apache 2.2.3 MPM Worker

Document Size 95,002 bytes

Concurrency Level 1,000

Total Requests 100,000

Requests Per Second 1,258 r/s

Throughput 119,966 kb/s

hex core X5675


Linux Kernel Configsnet.ipv4.tcp_max_syn_backlog=30000

net.core.netdev_max_backlog=1200

net.ipv4.tcp_timestamps=0

net.ipv4.tcp_sack=0

net.ipv4.tcp_window_scaling=0

net.ipv4.tcp_max_tw_buckets=2000000

net.ipv4.tcp_mem=100000000 100000000 100000000

net.ipv4.tcp_wmem=100000000 100000000 100000000

net.ipv4.tcp_rmem=30000000 30000000 30000000

net.ipv4.ip_conntrack_max = 231072


This last setting (or larger) will keep your kernel from dropping incoming connections under high load

Keepalive - On or Off?• reduces overhead of establishing new connections for

each request from the browser

• can waste memory and other resources if left open too long

• Common practice is to disable them

• I turn keepalive on, set to 6 seconds or 500 requests

• For CDNs like Akamai, you’ll want to turn up the time to more like 120 seconds


Defaults are On, 15 seconds (too long), 100 requests (too small)

GZip - mod_deflate• gzip compressing text resources - html, js, xml, css

dramatically decreases the size of the response for those assets: often up to 90% reduction in size

• reduces transfer time, especially for clients with slower connections (or big files)

• This decreases page load time (Client) and reduces the time the server thread is sending the response (Server)# Removing Hosts varyHeader unset Vary

<IfModule mod_deflate.c> AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css application/x-javascript BrowserMatch ^Mozilla/4 gzip-only-text/html BrowserMatch ^Mozilla/4\.0[678] no-gzip BrowserMatch \bMSIE !no-gzip !gzip-only-text/html</IfModule>


Browser Caching• Tell the browser to cache all static assets, for as long as

you can bear it

• Set the ETag

• Set cache control and expiration response headers

<LocationMatch '^/pca/images/.*'> FileETag MTime Size Header set Cache-Control "max-age=2764800, public" <IfModule mod_expires.c> ExpiresActive On ExpiresByType image/jpg A2764800 ExpiresByType image/jpeg A2764800 ExpiresByType image/png A2764800 </IfModule></LocationMatch>


This is HUGE! You can reduce the number of HTTP requests you have to handle by an order of magnitude or two by setting good cache headers.

Disk Caching• Why not use mod_mem_cache?

• Transparent web server caching of assets reduces load without deployment and development complexities

• Some warnings!

• Doesn’t auto clean itself

• Can grow to be huge if you don’t check the headers being cached

• Can grow to be huge if you have dynamic URI content - blah.jpg;jsessionid=foo

• Another place to purge cache for changed files


Mod_mem_cache has some bad bugs. mod_disk_cache lets the kernel’s in memory file caching algorithm do all the heavy lifting without tuning or exhausting available memory.For 3 tier environments, it’s common to have static assets that get deployed to the web servers separately from the application itself. This makes deployments more complex and error prone and makes local dev environments much harder to keep up to date.Seam and richfaces image tags insert jsessionid to URI, etc...

Disk Caching - Config<IfModule mod_disk_cache.c> CacheRoot /var/cache/mod_disk_cache CacheDirLevels 2 CacheDirLength 1

CacheEnable disk /pca/img CacheEnable disk /pca/flash CacheEnable disk /pca/css CacheEnable disk /pca/js CacheEnable disk /pca/images CacheMaxFileSize 200715200 CacheDefaultExpire 3600 CacheIgnoreHeaders Set-Cookie</IfModule>


Mod_PageSpeed• Open Source Apache Module• Lots and Lots of filters

• image compression• combine js/css• sprite images• domain sharding• etc...


I know a previous presentation was done on mod_pagespeed, so I won’t dive too deeply into all of it’s features and configs. Trying to optimize for performance can make dev more complicated. packtag. pagespeed.

Three Tier • SSL Termination• Load Balancing• Mod_proxy/mod_cluster• Mod_Security


Apache is key for 3-tier architectures. It provides significant performance, security, and PCI compliance wins. Big improvements in Mod_proxy_balancer with 2.4 (dynamic cluster, session bleedoff, etc)

Building Scalable Clusters w/Apache• VIP• heartbeat• haproxy• DNS LB

• simple round robin• smart


Questions?


Thank you!

[email protected]


mailto:[email protected]

mailto:[email protected]

apache performance

Technology