socialcam - ammon bartam
TRANSCRIPT
![Page 1: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/1.jpg)
![Page 2: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/2.jpg)
Testing made Socialcam successful!•16 Million Downloads
•#1 Free App•Acquired by Autodesk
![Page 3: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/3.jpg)
What is A/B testing?The scientific method applied to product development
![Page 4: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/4.jpg)
Intuition is a bad way to solve complex problems!
![Page 5: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/5.jpg)
How to run A/B test?•Ask a question
•Construct a hypothesis
•Test your hypothesis with an experiment
•Analyze the results and draw a conclusion•Repeat
![Page 6: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/6.jpg)
1) Ask a question =Decide what you're trying to improve
![Page 7: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/7.jpg)
•Pick a single metric
•Low hanging fruit first
•Get the team on board
•Have good stats before you start
![Page 8: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/8.jpg)
At Socialcam we started with the account creation flow
![Page 9: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/9.jpg)
2) Construct a hypothesis =Build things that might improve your metric
![Page 10: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/10.jpg)
•Simplify, remove steps
•Small optimizations or large shifts
•Number of variants depends on usage
![Page 11: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/11.jpg)
At Socialcam, we:•Added email / password login
•Added twitter login•Tried many designes of the initial view in the app
![Page 12: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/12.jpg)
3) Test your hypothesis with an experimentChoose analytics and testing systems, deploy!
![Page 13: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/13.jpg)
3rd party products•https://www.optimizely.com
•http://apptimize.com
![Page 14: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/14.jpg)
Write-up of our system athttp://blog.socialcam.com/mobile-ab-testing-made-easy
![Page 15: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/15.jpg)
4) Analyze the data and draw a conclusionPick a winner!
![Page 16: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/16.jpg)
Humans are bad at statistics!You have to check statistical significance
![Page 17: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/17.jpg)
Beware repeated significance testing errors!
![Page 18: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/18.jpg)
5) Repeat!
![Page 19: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/19.jpg)
At Socialcam, we got the account creation rate to 75%
![Page 20: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/20.jpg)
When You Can't A/B Test•When you don't have any users
•When external services are involved•A totally new product
![Page 21: Socialcam - Ammon bartam](https://reader035.vdocuments.net/reader035/viewer/2022081513/554dcc0fb4c905c2488b54bf/html5/thumbnails/21.jpg)
Takeaways•Testing made Socialcam blow up
•It’s the scientific method