hacker news meetup april 2014
DESCRIPTION
How Songkick's technology team does continuous delivery and uses experiments to develop our products.TRANSCRIPT
![Page 1: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/1.jpg)
Text
What do you know?How Songkick uses continuous delivery and experimentation to learn from our users
![Page 2: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/2.jpg)
About me: Dan Crow@crowquine
CTO, Songkick
!
Visiting Professor, University of Leeds co-Founder Blurb Apple, Google, Unicru, Verb
![Page 3: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/3.jpg)
Songkick• The live music startup
• One of original Silicon Roundabout startups
• Second largest live music service globally
• Growth:
![Page 4: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/4.jpg)
Back in 2012• Simplified product
• Simplified code
• Moved to SOA
• Moved to continuous integration
![Page 5: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/5.jpg)
Results• Releases per month went way up
!
!
!
!
!
• Team happiness too
![Page 6: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/6.jpg)
Text
Continuous Delivery
![Page 7: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/7.jpg)
Aim• Fast feedback from users
• Get code live as fast as possible
• Minimize developer context switching
• Lots of small changes, instead of few large releases
![Page 8: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/8.jpg)
Our principles• Create a continuous flow of work
• Fast feedback trumps exhaustive testing
• Waiting for builds kills focus
• Developers own product quality
• Business metrics tell you what is happening
![Page 9: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/9.jpg)
What we do• Develop on main
• Jenkins CI server
• Automated tests on staging, 10 minute maximum
• Push live on green
• Features accumulate behind feature flippers
• Manual testing live in production
![Page 10: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/10.jpg)
Feature flippers• Group changes together
• Control who can see changes: • No-one • VPN only • percentage of traffic • everyone
• Admin UI controls
![Page 11: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/11.jpg)
Lessons learned• Speed is essential
• Small changes create speed
• Small changes easier to debug
• Immediate feedback leads to: • better product • happier team
![Page 12: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/12.jpg)
It’s about culture, people• CD is cultural change, not a technical one
• Fast and small is (initially) scary
• Roll forwards becomes possible
• Minimize process once everyone is comfortable
• Developers own quality and testing
![Page 13: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/13.jpg)
Text
Experiments
![Page 14: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/14.jpg)
Everything is an experiment• Your startup
• Your products
• Each feature
• Each change
Embrace the scientific method
![Page 15: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/15.jpg)
How we experiment• Answer a question
• Figure out the lowest cost way to answer the question • Example: sharing experiment
• Run as many experiments as possible
• Always throw away the code and design
![Page 16: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/16.jpg)
Throw away the experiments• 80% of experiments fail, so do lots of experiments
• If the experiment fails • remove the changes asap
• If the experiment succeeds • remove the changes asap • rebuild the code
![Page 17: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/17.jpg)
Mechanics• Document the question and the expected results
• A/B testing using feature flippers
• Variety of packages to analyse results: • Google Analytics • skab • MixPanel
![Page 18: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/18.jpg)
It’s about culture, people• Always find the MVE
• Be comfortable with rough quality
• Get used to experiments failing
• Experiment often and fast
• Trust your users, not your instincts
• Trust your tests
![Page 19: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/19.jpg)
Results• Run multiple experiments at once
• Experiments typically last 2-5 days
• Most web features now subject to tests
• Sometimes short-circuit formal A/B testing
![Page 20: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/20.jpg)
Limitations• Your UI changes often
• Can get stuck in local minima
• Everyone needs to understand statistical significance
• Need lots of traffic to get fast results
• Mobile development cycles are too slow
![Page 21: Hacker News Meetup April 2014](https://reader034.vdocuments.net/reader034/viewer/2022051608/5455a21bb1af9f33608b4900/html5/thumbnails/21.jpg)
Obligatory hiring appeal
Songkick are hiring: [email protected]
@crowquine
!
Silicon Milkroundabout #7, May 10-11