a/b testing in big companies
TRANSCRIPT
![Page 1: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/1.jpg)
A/B & multivariate testing in big companies
@laurencevealewww.iqcontent.com
![Page 2: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/2.jpg)
A/B & Multivariate Testing...
Easy, right?
![Page 3: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/3.jpg)
Wrong!
![Page 4: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/4.jpg)
What’s so hard?
• Complex CMS & code
• Politics
• Risk aversion
• Red tape: Policies & procedures
![Page 5: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/5.jpg)
“Oh Fuck”
![Page 6: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/6.jpg)
New dog, old tricks
![Page 7: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/7.jpg)
What to test
Ad copySite contentEmail Radical design change (ab)Page elements (mvt)
![Page 8: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/8.jpg)
"The headline is the most important element. It is the telegram which decides the reader whether to read the copy"
![Page 9: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/9.jpg)
Buy now
Buy now
Buy now
Buy now
Add to cart
Add to cart
Add to cart
Add to cart
![Page 10: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/10.jpg)
Call to action
What’s around it (context & contrast)
Predominant brand colours
Size, shape
Labels
Buy now
![Page 11: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/11.jpg)
Where to test firstTop entry pages
with high bounce rates. Pages with good traffic and clear
purpose & conversions
Buy now
![Page 12: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/12.jpg)
Biggest loser, most dramatic results
![Page 13: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/13.jpg)
Where to test
Checkout flow
1 23 €
![Page 14: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/14.jpg)
What to test
Expectation - does the page people land on match the promise made?
![Page 15: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/15.jpg)
What to test
Clarity of page purpose visual hierarchy, clarity and purpose of content
![Page 16: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/16.jpg)
What to test
Urgency: are we influencing this?
Buy now only 3 left
![Page 17: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/17.jpg)
What to test: trust
![Page 18: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/18.jpg)
What to test
Distractions
![Page 19: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/19.jpg)
![Page 20: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/20.jpg)
results are only as good as the
hypothesis being tested
(not opinions)
![Page 21: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/21.jpg)
Forming an hypothesis?2
![Page 22: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/22.jpg)
I’m on TwitterFollow me on twitterYou should follow me on twitterYou should follow me on twitter here.
![Page 23: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/23.jpg)
![Page 24: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/24.jpg)
UX
![Page 25: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/25.jpg)
![Page 26: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/26.jpg)
Getting all this donein a big company
![Page 27: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/27.jpg)
Getting all this done
Find allies - motivated & enthusiastic(they’ll help you get stuff done)
![Page 28: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/28.jpg)
Getting all this done
Include everyone who needs to be,
don’t do sneaky AB(this gets buy-in)
![Page 29: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/29.jpg)
Getting all this done
Start small (remember the Biggest Loser)
![Page 30: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/30.jpg)
Getting all this done
Take time on designing the test
(results are only as good as the hypothesis)
![Page 31: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/31.jpg)
Tell a good story
Use the results to start a good story...
(not just about this test, but about testing-driven optimisation)
![Page 32: A/B Testing in Big Companies](https://reader034.vdocuments.net/reader034/viewer/2022042717/55d4ea2bbb61eb62158b4685/html5/thumbnails/32.jpg)
Give credit
To everyone involved