7 ways you are doing your a/b testing wrong by côme courteault
TRANSCRIPT
#TFWORKSHOP
7 WAYS YOU ARE DOING YOUR A/B TESTING WRONG
CÔME COURTEAULT GROWTH HACKER AT THEFAMILY
@C2PRODS
A/B TESTING
A/B TESTING
#1 NOT ENOUGH PEOPLE
You must A/B test on a lot of people
Check the χ² test(evanmiller.org/ab-testing/chi-squared.html)
GOOD
BAD
#2 NOT BEING RADICAL
Don’t start by testing shades ofred for the uppermost button!In the beginning, focus on
changes that have impact.
#3 TIME 2 days are not enough. 2 months are too much.
Run your tests for one week.
#4 NOT PAYING ATTENTION TO THE CONTEXT
Shit happens in the world. Holidays, too.Pay attention to what’s going on, this can
have an effect on the way your users behave.
#5 NOT HAVING AN HYPOTHESIS
Don’t run tests just because you can.Think of an hypothesis. Why are
you running this test?
#6 NO FOLLOW-UP TESTS
Double down on what works.Kill what doesn’t.
A/B testing is never over!
#7 IGNORING SMALL GAINS
A/B testing doesn’t always bring a 1000% increase!
Even a 10% gain on 100 000 users can prove worthy ;-)
PLEASE Build an awesome product before
running A/B tests.But after, use A/B testing to improve
your onboarding and your retention.Run multivariate tests only when you can (i.e. when you have bandwidth)
Don’t forget to care personally!
LET’S TALK!
SEE YOU SOON! LOVE.
THEFAMILY.CO ___
___