improving the performance of your web app
DESCRIPTION
These are the slides from my FOWA workshop on how to scale your web apps.TRANSCRIPT
![Page 1: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/1.jpg)
Improving the Performance of your
Web ApplicationJoe Stump, Lead Architect, Digg.com
![Page 2: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/2.jpg)
Introductions
![Page 3: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/3.jpg)
Users want access to all of their crap at all times. I, personally, don’t find your dog funny or cute, but I’ll be damned if I’m the one who’ll stand in the way of you posting it and others consuming it.
“Web 2.0 sucks (for scaling).”Joe Stump, Lead Architect, Digg.com
![Page 4: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/4.jpg)
Backend Considerations Language considerations Scaling out Caching strategies Content storage and delivery Parallel data requests Near time data processing Partitioning data
![Page 5: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/5.jpg)
Frontend Considerations
Reduce HTTP requests Avoid inline JavaScript and CSS Compression and Minification Learn to love HTTP/1.1
![Page 6: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/6.jpg)
“PHP doesn’t scale.”Cal Henderson, Director of Development, Flickr.com
Languages don’t scale Bytecode caching (PHP, Python, etc) Robust library & driver support Active developer communities
![Page 7: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/7.jpg)
What language do you use? Why? Does it help you or hurt to use it?
Discussion!
![Page 8: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/8.jpg)
Your mom lied; don’t share.
Decentralize data, storage, processing, etc. Increased redundancy Scaling becomes simple; add more boxes
![Page 9: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/9.jpg)
Scaling Up
![Page 10: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/10.jpg)
Scaling Up
![Page 11: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/11.jpg)
Scaling Up
![Page 12: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/12.jpg)
Scaling Out
![Page 13: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/13.jpg)
Scaling Out
![Page 14: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/14.jpg)
Scaling Out
![Page 15: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/15.jpg)
Scaling Out
![Page 16: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/16.jpg)
Scaling Out
![Page 17: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/17.jpg)
Scaling Out
![Page 18: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/18.jpg)
Scaling Out
![Page 19: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/19.jpg)
Scaling Out
![Page 20: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/20.jpg)
Scaling Out
![Page 21: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/21.jpg)
Scaling Out
![Page 22: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/22.jpg)
How do I scale easily?
1.Caching
2.Caching
3.Caching!
![Page 23: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/23.jpg)
What are my options?
Disk based caching (e.g. Cache_Lite) In memory caching (e.g. APC, Memcached) Cloud caching (e.g. MogileFS, S3)
![Page 24: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/24.jpg)
Disk based caching Stupid simple Cheap Fairly easy to scale out Dynamic images Slower than others Use fast disks! RAM disks are faster
![Page 25: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/25.jpg)
APC (PHP)
Bytecode caching In memory user cache Insanely fast Not centralized or shared
![Page 26: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/26.jpg)
Memcache
If you’re not using this you’re crazy Easy to set up and use Insanely fast over the network Scales to insane heights Failover, widely supported, etc. Centralized and shared across site
![Page 27: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/27.jpg)
Mogile FS
File and data store Runs over WebDAV Scales out infinitely (in theory) Serialize data, store in file Centralized and shared across site
![Page 28: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/28.jpg)
Amazon S3
File and data store Runs over HTTP Scales out infinitely (in theory) Serialize data, store in file Centralized and shared across site Costs money Widely supported in all languages Check out ThruDB
![Page 29: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/29.jpg)
Are you using caching? Why not? If so, what’s your strategy?
Discussion!
![Page 30: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/30.jpg)
Content Storage/Delivery
What are your storage needs? Is it critical YOU store them? How costly is it to store in-house? Can you do it for free? (YAY! Mooching!)
![Page 31: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/31.jpg)
i can has free storage?
YouTube for video Scribd for documents Flickr for images
![Page 32: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/32.jpg)
Cloud Services (S3)
Simple to get up and running No hardware maintenance Costs money, but not as much as you think
![Page 33: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/33.jpg)
NFS
Simple to set up and get running Costs money, requires colocation, etc. Does. Not. Scale. Did I mention it doesn’t scale? Stop gap solution at best
![Page 34: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/34.jpg)
Mogile FS
Somewhat complicated to set up Costs money, requires colocation, etc. Scales exceptionally well Used at Digg, LiveJournal, others Check out File_Mogile by Digg (PEAR)
![Page 35: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/35.jpg)
Roll Your Own
File storage IS your business Highly specialized and customized Costs money, requires colocation, etc. Last resort
![Page 36: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/36.jpg)
CDN
Completely outsource it Costs a ton of money Out of your control Scales and scales and scales
![Page 37: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/37.jpg)
What are you using for storage? What’s worked for you? What’s failed epically?
Discussion!
![Page 38: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/38.jpg)
Parallel Data Requests
Access your data in parallel Make data access asynchronous (WHAT?!) Loosely couple your data access layer All for the low, low price of FREE!*
*Offer only available for hardcore nerds looking for street cred.
![Page 39: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/39.jpg)
HTTP
Parallel Asynchronous Non-blocking Loosely coupled Free foot massages!
![Page 40: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/40.jpg)
![Page 41: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/41.jpg)
HTTP
![Page 42: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/42.jpg)
Gearman
Parallel Asynchronous Scales well
![Page 43: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/43.jpg)
Which format to use for exchange? Anyone doing this already? Amazon, Google, Yahoo!
Discussion!
![Page 44: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/44.jpg)
Near time processing
Does this need to be done NOW? Offload to background processes Offloading must be a no op Feeds, Facebook, crawling, etc.
![Page 45: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/45.jpg)
Cron
Run every minute or two Simple Great for batch jobs Not decentralized, locking issues
![Page 46: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/46.jpg)
Gearman
Fire and forget Simple Scales well Digg Images Nearly instant Decentralized No guarantees
![Page 47: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/47.jpg)
Queues
Grid Engine by Sun Starling by Twitter Others?
![Page 48: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/48.jpg)
Amzon’s EC2
Near limitless computing resources Remember; don’t share Awesome for bots, crawling, etc.
http://open.blogs.nytimes.com/2007/11/01/self-service-prorated-super-computing-fun/
![Page 49: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/49.jpg)
What’s low(er) priority? Where would you implement this?
Discussion!
![Page 50: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/50.jpg)
Partitioning Data
Horizontal v. Vertical Not all data lives in a single place Hash records to partitions App smart / logical sharding
![Page 51: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/51.jpg)
Horizontal
Usersid int(11)username char(15)password char(15)email char(45)
Usersid int(11)username char(15)password char(15)email char(45)
Usersid int(11)username char(15)password char(15)email char(45)
192.168.0.1 192.168.0.3192.168.0.2
![Page 52: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/52.jpg)
Hashing your dataoh hai! were’s mai dataz?!
![Page 53: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/53.jpg)
How?
Put 10,000 users per partition Partition users alphabetically Partition home listings by zip code Partition products by SKU
![Page 54: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/54.jpg)
Vertical
Usersid int(11)username char(15)password char(15)email char(45)
UsersPrfid int(11)fname char(50)lname char(50)url char(255)
UsersStgid int(11)cmts_pg tinyint(2)cmts_lvl tinyint(1)cmts_prf tinyint(1)
192.168.0.1 192.168.0.3192.168.0.2
![Page 55: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/55.jpg)
Why?
Avoid altering large tables Save time during insert Many small tables v. one large table Lazy loading of rarely used data
![Page 56: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/56.jpg)
Natural partitions in your data? How would you hash your data?
Discussion!
![Page 57: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/57.jpg)
Reduce HTTP Requests
Bundle JavaScript and CSS Use sprites for images Reduce images / outside objects
![Page 58: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/58.jpg)
Reduce HTTP Requests
Bundle JavaScript and CSS Use sprites for images Reduce images / outside objects
![Page 59: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/59.jpg)
Avoid inline JS/CSS
External = Cached
Inline = Not Cached
![Page 60: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/60.jpg)
Compression / Minify
Enable Gzip compression sitewide Use minification software on JS jQuery/Prototype Minified
![Page 61: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/61.jpg)
Learn to Love HTTP/1.1
Cache-Control: public/private Connection: close Expires: Thu, 28 Feb 2008 16:00:00 GMT
![Page 62: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/62.jpg)
Conclusions
Share nothing, decentralize, redundancy Caching, caching, caching, caching Reduce, recycle and reuse
![Page 63: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/63.jpg)
Resources
High Performance Web SitesEssential Knowledge for Front-End Engineersby Steve Souders
Serving JavaScript Fasthttp://www.thinkvitamin.com/features/webapps/serving-javascript-fastby Cal Henderson, Director of Development, Flickr.com
![Page 64: Improving The Performance of Your Web App](https://reader036.vdocuments.net/reader036/viewer/2022062512/55494016b4c9050a4d8b5030/html5/thumbnails/64.jpg)
Questions?!