how soundcloud scales...¥alexa ranking (233) ¥ employees (190) - 80 engineers ¥ 4 office...
TRANSCRIPT
How SoundCloud scalesAlexander Grosse
@klangberater
Freitag, 5. April 13
“Youtube for Audio”
Freitag, 5. April 13
Fakten
Freitag, 5. April 13
• Alexa Ranking (233)
Fakten
Freitag, 5. April 13
• Alexa Ranking (233)• Employees (190) - 80 engineers
Fakten
Freitag, 5. April 13
• Alexa Ranking (233)• Employees (190) - 80 engineers• 4 office locations (Berlin, London, San Francisco,
Sofia)
Fakten
Freitag, 5. April 13
• Alexa Ranking (233)• Employees (190) - 80 engineers• 4 office locations (Berlin, London, San Francisco,
Sofia)• >35 million registered users
Fakten
Freitag, 5. April 13
• Alexa Ranking (233)• Employees (190) - 80 engineers• 4 office locations (Berlin, London, San Francisco,
Sofia)• >35 million registered users• partners (BBC, CNN)
Fakten
Freitag, 5. April 13
• Alexa Ranking (233)• Employees (190) - 80 engineers• 4 office locations (Berlin, London, San Francisco,
Sofia)• >35 million registered users• partners (BBC, CNN)• > 10.000 Apps
Fakten
Freitag, 5. April 13
Freitag, 5. April 13
How SoundCloud scales
Freitag, 5. April 13
?Freitag, 5. April 13
Organization
Technology
Freitag, 5. April 13
What is the foundation?
Freitag, 5. April 13
Freitag, 5. April 13
Principles
Freitag, 5. April 13
Top-Down vs. Bottom-Up
Principles
Freitag, 5. April 13
Top-Down vs. Bottom-Up
Trust
Principles
Freitag, 5. April 13
Top-Down vs. Bottom-Up
Trust
How to react to mistakes
Principles
Freitag, 5. April 13
Top-Down vs. Bottom-Up
Trust
How to react to mistakes
Continuous Improvement
Principles
Freitag, 5. April 13
Top-Down vs. Bottom-Up
Trust
How to react to mistakes
Continuous Improvement
Innovation
Principles
Freitag, 5. April 13
• Flat Hierarchies? Everybody has them...
• Engineering teams responsible for architecture (not architecture teams)
• Influence on the product
Top-Down vs. Bottom-Up
Freitag, 5. April 13
• Deployment
• Change Management
• Process
• Time for “technical debt”
Trust
Freitag, 5. April 13
• What will be changed if a mistake happens?
Reaction to mistakes
Freitag, 5. April 13
• R&D Department?
• Only Product?
Innovation
Freitag, 5. April 13
SoundCloud‘s basic principles
Freitag, 5. April 13
• No dedicated QA roles
SoundCloud‘s basic principles
Freitag, 5. April 13
• No dedicated QA roles
• “You build it - you run it”
SoundCloud‘s basic principles
Freitag, 5. April 13
• No dedicated QA roles
• “You build it - you run it”
• Team Building (Rotation, constant changes)
SoundCloud‘s basic principles
Freitag, 5. April 13
• No dedicated QA roles
• “You build it - you run it”
• Team Building (Rotation, constant changes)
• Hacker Time
SoundCloud‘s basic principles
Freitag, 5. April 13
• No dedicated QA roles
• “You build it - you run it”
• Team Building (Rotation, constant changes)
• Hacker Time
• Continuous Improvement
SoundCloud‘s basic principles
Freitag, 5. April 13
• No dedicated QA roles
• “You build it - you run it”
• Team Building (Rotation, constant changes)
• Hacker Time
• Continuous Improvement
• Move fast and break things
SoundCloud‘s basic principles
Freitag, 5. April 13
No QADoes that mean we don‘t test?
Freitag, 5. April 13
“You build it - you run it”
Freitag, 5. April 13
Flat Hierarchy
Freitag, 5. April 13
Innovation
Hacker Time
Freitag, 5. April 13
Freitag, 5. April 13
Continuous Improvement
RetrospectivesPost Mortems (Projekte)
Freitag, 5. April 13
Mistakes...
Freitag, 5. April 13
Scrum? Kanban?Process?
Freitag, 5. April 13
What is the most important thing you have to do so that this works?
Freitag, 5. April 13
Freitag, 5. April 13
???
Freitag, 5. April 13
???
Initial Culture
Freitag, 5. April 13
???
Initial Culture
Hiring!!!
Freitag, 5. April 13
Freitag, 5. April 13
SoundCloud Hiring Process
Freitag, 5. April 13
How does it look like in reality?
Freitag, 5. April 13
Organisation
Freitag, 5. April 13
HTML5 Mobile (native) other V1
System Engineering (Berlin / SF)
Search T&S Payment Partner Tools
Data Activ. Delivery Platform
API
Freitag, 5. April 13
How do we structure teams?
cross functional?
Freitag, 5. April 13
Do teams stay the same?
Change!
Freitag, 5. April 13
Technology
Freitag, 5. April 13
Ruby/Scala/Go/Clojure/Javascript
Freitag, 5. April 13
Scale Horizontally/Vertically?
Freitag, 5. April 13
Rails->Mysql
RabbitMQ->Cassandra
http://backstage.soundcloud.com/2012/08/evolution-of-soundclouds-architecture/
Freitag, 5. April 13
• 7 Master, 9 slaves, 900GB data
• Additional slaves in EC2
• Master Peak is 15K IOPS writes, 45k reads
• one slave lags intentionally 1 hour behind
• Je nach Konsistenzanforderung wird Slave oder Master gewählt (für reads - unterstützt read-only mode)
MySQL
Freitag, 5. April 13
• 3 Cluster
• 88 Nodes
• 15 TB of data
• up to 70K reads/sec
Cassandra
Freitag, 5. April 13
HadoopElastic Search
Proprietary Spam Detection
Freitag, 5. April 13
ElasticSearch
80 Mio Docs (gaining 1 Mio every 6 days)
175 QPS (gaining 15 QPS every month)
Freitag, 5. April 13
Hosted in S3/EC2and physical DCs
Freitag, 5. April 13
Outages
Freitag, 5. April 13
Freitag, 5. April 13
Post Mortems
Freitag, 5. April 13
• Move fast and break things
• Don‘t get slow because you fear to make mistakes
Summary
Freitag, 5. April 13
backstage.soundcloud.com
Freitag, 5. April 13
Alexander Grosse
alexandergrosse.com@klangberater
We are hiring!
www.soundcloud.com/jobs
Freitag, 5. April 13