exploring the full range of a:b testing opportunity - sascon 2015
TRANSCRIPT
@matt_lacey12 #SASCon
EXPLORING THE FULL RANGE OF A/B TESTING OPPORTUNITY
Matt LaceyHead of Optimisation
@matt_lacey12 #SASCon
Team of leading conversion optimisation specialists
Head office in Manchester, with clients across UK and Europe
Work with brands to analyse, research, test and optimise site experiences
ABOUT PRWD
@matt_lacey12 #SASCon
“We’ll make the big changes and you can just do the optimisation.”
“Conversion Optimisation & Continuous Improvement are
part of our DNA.”
@matt_lacey12 #SASCon
CONVERSION OPTIMISATION IS NOT JUST TESTING . . .
. . . BUT IT IS AN ESSENTIAL ELEMENT.
@matt_lacey12 #SASCon
Four Options:
1. Control – (4 months FREE)
2. Variation A – (SAVE £22)
3. Variation B – (SAVE 38%)
4. Variation C – remove ‘(cancel anytime)’ from monthly
WHICH TEST WON?
12.5% overall increase in
subscriptions for Variation A (SAVE £22)
@matt_lacey12 #SASCon
1. Control - Annual £39.99 (Save £44)
2. Variation – £3.33 per month (£39.99 annually)
WHICH TEST WON?
77% overall increase in annual
subscriptions for the Variation
@matt_lacey12 #SASCon
“We had gotten really good at optimising, tuning and iterating, but in the process we had lost sight of the purpose of those activities: testing a clear business hypothesis in the service of the company’s vision.” Eric Ries, Author the Lean Startup
@matt_lacey12 #SASCon
GET REVISING SIGN-UP TABLE
“ Price doesn’t put me off, although the basic package is more than enough for me.
I think the site has already got everything I need.
Moderated user research participant
@matt_lacey12 #SASCon
@matt_lacey12 #SASCon
GET REVISING SIGN-UP TABLE
185% increase in premium subscriptions
with a small drop in free subscriptions. This was seen to be a good trade-off.
@matt_lacey12 #SASCon
UI TWEAKING
BUSINESS HYPOTHESIS
TESTING
OPPORTUNITY
@matt_lacey12 #SASCon
THANKS FOR LISTENING
MATT LACEY Head of Optimisation
@matt_lacey12