tripling your conversion rates with a/b testing
TRANSCRIPT
Tripling Your Conversion Rates With A/B TestingLars Lofgren
@larslofgren
2X and 3X in 9-12 months
KISSmetrics homepage conversion to signups.
Nov 2013 = 2.85% Aug 2014 = 8.71%
• Why not just launch stuff? Why A/B test?
• The Growth March
• Prepping for Your March
• 7 Rules of A/B Testing
We’ll cover...
But Lars! I already know how to increase
conversions!
My win-rates at KISSmetrics after 65 A/B tests.N
umbe
r of W
ins
0
10
20
30
40
Confirmed Loser Marginal Confirmed Winner
At best, 28% of your tests will win.
If most of our tests fail…we can’t depend on any
single test.
The Growth March
Growth is not a project, it’s a relentless grind.
A/B testing is about playing the long-term.
You’re in a quantity game. More tests = more wins.
Prepping for Your March
Get your data in order.
Need to tie test segments through entire funnel.
Get your team in place.
If your tests involve multiple teams, you won’t get far.
The Bare-Bones Growth Team
• PM, background in marketing or product
• Designer that loves iteration and results
• Front-end engineer, full-stack if you can spare
Get your levers prioritized.
Don’t waste tests on assets that impact only a small
number of people.
First, fix any broken steps in your funnel.
Start at the top of your funnel and work down.
Which of your assets drive the most conversions?
Get research for testing.
Bad ideas for testing.
You won’t find majorwins from random“growth hacks”.
You need a stream of insights from your
customers.
1. Interviews with customers from last 30 days.
2. Surveys for people that abandoned funnels.
3. Qualaroo pop-ups at funnel bottlenecks.
4. Heatmaps on struggling pages.
5. Usability tests in signup and lead flows.
Great places for testing research.
1. Get your data in order.
2. Get your team in place.
3. Get your levers prioritized.
4. Get research for testing.
The Growth March Prep
7 Rules of A/B Testing
A/B tests aren’t for picking winners. They’re for
avoiding losers.
False positives will destroy your gains.
Cycle through as many tests as possible while only
launching confirmed winners.
1. Above all else, the control stands.
2. Get 2000+ people through the test within 30 days.
3. Wait at least 1 week before checking data.
4. Only launch variants at 99% statistical significance.
Same test, 4 weeks later.
5. If the test drops below 10%, ditch it.
6. If no new winner after a month, ditch variant.
7. Get the next test ready while you wait.
1. Above all else, the control stands.
2. Get 2000+ people through the test within 30 days.
3. Wait at least one week before checking data.
4. Only makes changes at 99% certainty.
5. If test drops below 10%, ditch it.
6. If no winner after 1 month, ditch variant.
7. Build your next test while waiting on data.
7 Rules of the Growth March
Questions?Lars Lofgren@larslofgren