scaling for web - architecting hootsuite for 5+ million users
TRANSCRIPT
![Page 1: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/1.jpg)
Scaling for WebArchitecting HootSuite for 5 Million+ Users
![Page 2: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/2.jpg)
About Us
Beier CaiChief [email protected]@beiercai
Bill MonkmanLead Developer, [email protected]@bmonkman
![Page 3: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/3.jpg)
Hoo’s HootSuite
![Page 4: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/4.jpg)
Some Numbers
5+ million Members
10+ million Social Networks
25+ million Streams
350+ million Messages Scheduled
1.3+ Billion Messages Sent
2+ Billion Page Views per month
2,000 Web Requests/sec
![Page 5: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/5.jpg)
![Page 6: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/6.jpg)
Technologies
PHP
MySQL
![Page 7: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/7.jpg)
Technologies
PHP
Memcached
MySQL
![Page 8: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/8.jpg)
Technologies
PHP
Gearman
Memcached
MySQL
![Page 9: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/9.jpg)
Technologies
PHP
Gearman
Memcached
MySQL MongoDB
![Page 10: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/10.jpg)
Technologies
PHP
Gearman ZeroMQ
Memcached
MySQL MongoDB
![Page 11: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/11.jpg)
Technology Stack
LAMP (Linux, Apache, MySQL, PHP)
Scala
Akka
Python
Clojure
MongoDB
Redis
Memcached
Gearman
ZeroMQ
RabbitMQ/AMQP
jQuery/Backbone
EC2 Cloud / CloudStack
Jenkins / CruiseControl
CloudFront CDN
HAProxy/Nginx/Varnish
Zend
Statsd/Graphite
LogStash
ElasticSearch/Lucene
Nagios/Munin/Sensu
ZooKeeper
![Page 12: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/12.jpg)
Optimization
• Push processing up to the front-end, make your users’ computers into a distributed processing network. e.g. stream rendering
• Cache as much as possible, invalidate only when necessary.
• Sometimes features have to take a back-seat to performance/scalability.
• Look at profiler data.
• Watch query execution time, optimize when necessary with indexes, partitions, de-normalization, etc.
• Know what your weak spots are and keep a close eye on them.
![Page 13: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/13.jpg)
Optimization
![Page 14: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/14.jpg)
Optimization Case Study
![Page 15: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/15.jpg)
Optimization Case Study
![Page 16: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/16.jpg)
![Page 17: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/17.jpg)
Scaling Dev Team
• Passion
• Commitment
• Dedication
• Responsibility
• General Awesomeness
Besides technical competency, we also value
![Page 18: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/18.jpg)
Scaling Dev Team
Passion
• Commitment
• Dedication
• Responsibility
• General Awesomeness
Besides technical competency, we also value
![Page 19: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/19.jpg)
Scaling Dev Team
![Page 20: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/20.jpg)
Team Work
Entrepreneurial
• Communication
• Respect
• Trust
![Page 21: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/21.jpg)
Coding Practice
• Clean and Readable
• Maintainable
• Reusable
• Scalable
• Commented
Have a well defined coding standard within your dev team, and follow it!
![Page 22: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/22.jpg)
Monitoring, Profiling & Logging
![Page 23: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/23.jpg)
Monitoring, Profiling & Logging
![Page 24: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/24.jpg)
Case Study
![Page 25: Scaling for Web - Architecting HootSuite for 5+ Million Users](https://reader036.vdocuments.net/reader036/viewer/2022062704/555823a2d8b42a5e468b50e2/html5/thumbnails/25.jpg)
Case Study