lars lofgren
TRANSCRIPT
![Page 1: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/1.jpg)
Tripling Your Conversion Rates With A/B TestingLars Lofgren
![Page 2: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/2.jpg)
@larslofgren
![Page 3: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/3.jpg)
2X and 3X in 9-12 months
![Page 4: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/4.jpg)
KISSmetrics homepage conversion to signups.
Nov 2013 = 2.85% Aug 2014 = 8.71%
![Page 5: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/5.jpg)
I Will Teach double confirmation page
July 2015 = 63% June 2016 = 82%
![Page 6: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/6.jpg)
I Will Teach popup monthly lead counts
Aug 2015 = 1800 Aug 2016 = 7100
![Page 7: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/7.jpg)
Win rates at KISSmetrics after 65 A/B testsN
umbe
r of T
ests
0
10
20
30
40
Confirmed Winner Marginal Confirmed Loser
52%20%28%
![Page 8: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/8.jpg)
Win rates at I Will Teach after 96 A/B testsN
umbe
r of T
ests
0
15
30
45
60
Confirmed Winner Marginal Confirmed Loser
60%
23%17%
![Page 9: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/9.jpg)
If most of our tests fail…we can’t depend on any
single test.
![Page 10: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/10.jpg)
You’re in a quantity game. More tests = more wins.
![Page 11: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/11.jpg)
We don’t use A/B tests to pick winners. We use them to avoid losers as we stack
confirmed winners.
![Page 12: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/12.jpg)
Cycle through as many tests as possible while only
launching confirmed winners.
![Page 13: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/13.jpg)
8 Rules of A/B Testing
![Page 14: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/14.jpg)
1. Above all else, the control stands.
![Page 15: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/15.jpg)
2. Get 2000+ people through the test within 30 days.
![Page 16: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/16.jpg)
3. Wait at least 1 week before checking data.
![Page 17: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/17.jpg)
4. Only launch variants at 99% statistical significance.
![Page 18: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/18.jpg)
Fast forward a week…
![Page 19: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/19.jpg)
5. If the test drops below 10%, ditch it.
![Page 20: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/20.jpg)
6. If no new winner after a month, ditch variant.
![Page 21: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/21.jpg)
7. Get the next test ready while you wait.
![Page 22: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/22.jpg)
8. If you have another goal, say so before starting
![Page 23: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/23.jpg)
Cycle through as many tests as possible while only
launching confirmed winners.
![Page 24: Lars lofgren](https://reader031.vdocuments.net/reader031/viewer/2022021814/58f2d08a1a28abef6d8b459b/html5/thumbnails/24.jpg)
Q&ALars Lofgren
bit.ly/triple-conversions