beyond a/b testing: how to take your split tests to the next level
TRANSCRIPT
![Page 1: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/1.jpg)
800-920-7227 | www.pinpointe.com
Presented by:Michael Pelosi
![Page 2: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/2.jpg)
800-920-7227 | www.pinpointe.com
ALeaderinBusiness-to-BusinessEmailMarketing
Brought to you byPinpointe
800-920-7227 | www.pinpointe.com
![Page 3: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/3.jpg)
800-920-7227 | www.pinpointe.com
DigitalMediaStrategistatPinpointe On-Demandwww.pinpointe.com
Phone:617-461-1209|[email protected]/Facebook:@Pinpointe
Featured SpeakerMichael Pelosi
ErynBranhamPinpointeOn-Demand,[email protected]
800-920-7227
Moderator
800-920-7227 | www.pinpointe.com
![Page 4: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/4.jpg)
BEYOND A | BTESTING
A S K A Q U E S T I O N , F I N D A N A N S W E R , A N D P L A N F O R T H E F U T U R E
![Page 5: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/5.jpg)
800-920-7227 | www.pinpointe.com
WHAT IS A | B TESTING?
A/B Testing, or split testing, tests one component of your marketing campaign (subject line, call-to-action, etc.) against another within a
certain time frame and across a fixed amount of users.
![Page 6: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/6.jpg)
800-920-7227 | www.pinpointe.com
TEST EVERYTHING (ALMOST)
• The preferences and tastes of your customers change as you offer more products and services.
• Don’t just throw darts at the dartboard and hope they stick. A/B testing gives a framework for picking winners.
• Consistent A/B testing builds a more data driven email and digital marketing culture.
![Page 7: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/7.jpg)
800-920-7227 | www.pinpointe.com
REASONS NOT TO A/B TEST
• Your campaign is performing well. If it’s not broken, don’t fix it. Or you just may break it.
• More data is does not lead to better decisions. It only leads to more analysis.
• If something just isn’t working, don’t spend the time testing. Spend the time fixing. A/B testing is all about providing a lift for a metric of interest.
![Page 8: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/8.jpg)
800-920-7227 | www.pinpointe.com
A TOUCH OF RANDOM IN THE AIR
• A/B testing is powerful, but it’s not a crystal ball.
• Customer behavior can be unpredictable and sometimes random.
• There’s always an unseen error in A/B test results.
• The results from an A/B test today may not be the same as the results from the same test tomorrow.
![Page 9: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/9.jpg)
800-920-7227 | www.pinpointe.com
HOW WE THINK PEOPLE READ THEIR EMAILS
![Page 10: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/10.jpg)
800-920-7227 | www.pinpointe.com
Can you go to the grocery store on your
way home?
![Page 11: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/11.jpg)
800-920-7227 | www.pinpointe.com
4 SIMPLE STEPS TO TEST
Select A and B
![Page 12: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/12.jpg)
800-920-7227 | www.pinpointe.com
4 SIMPLE STEPS TO TEST
Select how many subscribers will receive A and how many will receive B. Then select for how long the test will run for.
![Page 13: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/13.jpg)
800-920-7227 | www.pinpointe.com
4 SIMPLE STEPS TO TEST
Conduct an A/A test to establish a baseline.
![Page 14: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/14.jpg)
800-920-7227 | www.pinpointe.com
4 SIMPLE STEPS TO TEST
Develop the hypothesis, wait for the result, and analyze the data.
![Page 15: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/15.jpg)
800-920-7227 | www.pinpointe.com
#1. SELECT A & SELECT B
•Test small differences to start.
•Have a good business reason for selecting B.
![Page 16: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/16.jpg)
800-920-7227 | www.pinpointe.com
EXAMPLE ONE: EMAIL SUBJECT LINECONTROL SUBJECT LINE (A)
VARIATION SUBJECT LINE (B)
We Could Really Use Your Input on This Problem
The 7 Strategies We Used to Solve Our Problem
![Page 17: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/17.jpg)
800-920-7227 | www.pinpointe.com
MORE SUBJECT LINESCONTROL SUBJECT LINE (A)
VARIATION SUBJECT LINE (B)
“This is just a service I can’t live without.”
“7 services curated just for you”
![Page 18: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/18.jpg)
800-920-7227 | www.pinpointe.com
MAKING ONLY ONE CHANGE AT A TIME
If you’re testing a subject line, everything after the subject line has to stay the same. The same goes for email layout, landing
pages, etc.
![Page 19: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/19.jpg)
800-920-7227 | www.pinpointe.com
SUBJECT LINE A
“Our biggest summer sale: 25% off select brushes”
![Page 20: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/20.jpg)
800-920-7227 | www.pinpointe.com
SUBJECT LINE B
“Brush Bonanza is Here! 25% Off Premium Brushes”
![Page 21: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/21.jpg)
800-920-7227 | www.pinpointe.com
EMAIL A
![Page 22: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/22.jpg)
800-920-7227 | www.pinpointe.com
EMAIL A
![Page 23: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/23.jpg)
800-920-7227 | www.pinpointe.com
SUBJECT LINE TESTING IDEAS
1. Subject line length (character differences, short versus long subject lines).
2. Using numbers in the subject line vs. no numbers in the subject line.
3. Using “testimonials” or “quotes.”
4. Using emojis.
5. Using urgent words versus a relaxed words.
6. Using personalized lines versus generic lines.
![Page 24: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/24.jpg)
800-920-7227 | www.pinpointe.com
MORE EXAMPLES
![Page 25: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/25.jpg)
800-920-7227 | www.pinpointe.com
EXAMPLE 2: PLAIN TEXT VERSUS IMAGE BASED EMAILS
![Page 26: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/26.jpg)
800-920-7227 | www.pinpointe.com
![Page 27: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/27.jpg)
800-920-7227 | www.pinpointe.com
![Page 28: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/28.jpg)
800-920-7227 | www.pinpointe.com
![Page 29: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/29.jpg)
800-920-7227 | www.pinpointe.com
![Page 30: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/30.jpg)
800-920-7227 | www.pinpointe.com
![Page 31: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/31.jpg)
800-920-7227 | www.pinpointe.com
![Page 32: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/32.jpg)
800-920-7227 | www.pinpointe.com
EXAMPLE 3: CALL TO ACTION BUTTONS
![Page 33: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/33.jpg)
800-920-7227 | www.pinpointe.com
THE CTA RESEARCH
• Best Color: Blue
• Best Size for Mobile: 44 x 44 pixels
• Not Much Other Research: Perfect Place for Your Case Studies
![Page 34: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/34.jpg)
800-920-7227 | www.pinpointe.com
EMAIL CTA: BUTTON PANEL
![Page 35: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/35.jpg)
800-920-7227 | www.pinpointe.com
MORE EXAMPLES
![Page 36: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/36.jpg)
800-920-7227 | www.pinpointe.com
MORE EXAMPLES
![Page 37: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/37.jpg)
800-920-7227 | www.pinpointe.com
MORE EXAMPLES
![Page 38: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/38.jpg)
800-920-7227 | www.pinpointe.com
CTA PLAIN TEXT
•Because plain text emails offer larger lifts, A/B test image CTA versus link CTA.
•Use actionable words that are clearly links.
![Page 39: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/39.jpg)
800-920-7227 | www.pinpointe.com
CTA LINK EXAMPLES PLAIN TEXT• Buy now• Buy today• Claim your spot• Claim your spot today• 50% off your purchase• Happy Birthday! Enjoy our gift to you.• Download our whitepaper • Tell us you’ll be there!• Give us a follow on Facebook to stay in touch
![Page 40: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/40.jpg)
800-920-7227 | www.pinpointe.com
PLAIN TEXT CTA IN EMAILS
Multiple CTAs, One Email
![Page 41: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/41.jpg)
800-920-7227 | www.pinpointe.com
MORE PLAIN TEXT EXAMPLES
Multiple CTAs, One Email
![Page 42: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/42.jpg)
800-920-7227 | www.pinpointe.com
#2. SELECT A SAMPLE SIZE AND TIME FRAME
• Your split testing experiment has to have enough people.
• The experiment needs to go on long enough to get valid results.
![Page 43: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/43.jpg)
800-920-7227 | www.pinpointe.com
HOW MANY PEOPLE WILL I NEED?
• For email testing, your sample size needs to be your open rate.
• Landing pages, CTAs on your website, etc., it’s based on traffic.
• In a typical A/B test for email, 50% of subscribers get A, 50% get B.
![Page 44: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/44.jpg)
800-920-7227 | www.pinpointe.com
GENERAL SAMPLE SIZE TABLE
![Page 45: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/45.jpg)
800-920-7227 | www.pinpointe.com
HOW TO CONDUCT AN EMAIL A/B TEST
1. Select variation A and variation B.
2. Select how many subscribers you will need.
3. Send out the emails randomly to the subscribers in a 50/50 split.
4. Wait some time period until everyone has had a chance to open the email and select a winner.
![Page 46: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/46.jpg)
800-920-7227 | www.pinpointe.com
FATAL A/B TESTING FLAW
Never stop an A/B test early. For landing pages and website based tests, 2 to 4 weeks is ideal.
Once you develop a time frame, don’t look until the test is over.
![Page 47: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/47.jpg)
800-920-7227 | www.pinpointe.com
![Page 48: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/48.jpg)
800-920-7227 | www.pinpointe.com
![Page 49: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/49.jpg)
800-920-7227 | www.pinpointe.com
DAYS 1-9. STOP HERE AND A WINS
![Page 50: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/50.jpg)
800-920-7227 | www.pinpointe.com
AT DAY 14, B WINS
![Page 51: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/51.jpg)
800-920-7227 | www.pinpointe.com
AT DAY 30, NO REAL RECENT WINNER
![Page 52: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/52.jpg)
800-920-7227 | www.pinpointe.com
#3. BEFORE A/B, A/A TEST THE ASSET
• What if it wasn’t a change in subject line or layout that resulted in a change?
• What if it was the behavior of your customers instead?
• In an A/A test, you perform a 50% / 50% split within your subscribers, but they both get the A variation.
![Page 53: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/53.jpg)
800-920-7227 | www.pinpointe.com
WHY?
A/B testers started to become suspicious of high conversion rates given by split testing software (300%
conversion just by changing the color of a button!).
When an A/A test was performed, the detected lift was still pretty high.
![Page 54: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/54.jpg)
800-920-7227 | www.pinpointe.com
EMAIL CTA BUTTON TEST
![Page 55: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/55.jpg)
800-920-7227 | www.pinpointe.com
EXAMPLE SPLIT TEST RESULTS
Blue Conversion Rate = 60%Red Conversion Rate = 50%
LIFT FROM BLUE = 10%
![Page 56: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/56.jpg)
800-920-7227 | www.pinpointe.com
EMAIL CTA BUTTON TEST: A/A
![Page 57: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/57.jpg)
800-920-7227 | www.pinpointe.com
A/A TEST RESULTS
Red Button Conversion: 47.75%Red Button Conversion: 63.03%
Lift: 16%
![Page 58: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/58.jpg)
800-920-7227 | www.pinpointe.com
BLUE IS NOT THE WINNER
![Page 59: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/59.jpg)
800-920-7227 | www.pinpointe.com
THE RESULTS FROM AN A/A TEST SHOULD SHOW NO DIFFERENCE BETWEEN A AND A.
![Page 60: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/60.jpg)
800-920-7227 | www.pinpointe.com
#4. DEVELOP A HYPOTHESIS, ANALYZE THE DATA
• Your hypothesis is your assumption as to how the test will go. Typically, you’ll state: “There is no difference between A and B.”
• Set a desired lift in the desired metric for analyzing the data. “I want to see a 20% lift, + or – 5% for this test to be successful.”
![Page 61: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/61.jpg)
800-920-7227 | www.pinpointe.com
THE UNCOMFORTABLE TRUTH
There are literally millions of variations you could make that would
improve or ruin your desired metric. You’re tests are only as good as
your assumptions as to what should be tested. Don’t get caught up in
testing. Get caught up in gathering data and intelligence to make your
email / website / landing page / subject lines better for your audience.
![Page 62: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/62.jpg)
800-920-7227 | www.pinpointe.com
SPLIT TEST TO CREATE A BETTER EXPERIENCE
The ultimate goal of split testing is to not have to split test anymore until there is a
valid reason for doing so.
![Page 63: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/63.jpg)
800-920-7227 | www.pinpointe.com
PRACTICAL SPLIT TESTING CASES
![Page 64: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/64.jpg)
800-920-7227 | www.pinpointe.com
NETFLIX LANDING PAGE A
Chrome Browser landing page. Netflix giving me, based on past data, what I want.
![Page 65: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/65.jpg)
800-920-7227 | www.pinpointe.com
NETFLIX LANDING PAGE B
Firefox Browser Landing Page Result. Netflix B variation is a scroll through of their original programming. Variation A indulged my current interests. Variation B is about future value.
![Page 66: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/66.jpg)
800-920-7227 | www.pinpointe.com
MORE SENSIBLE SPLIT TESTING
CONTROL EMAIL
Same offer, two different email addresses, eight hours apart. From Macy’s.
![Page 67: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/67.jpg)
800-920-7227 | www.pinpointe.com
BUTTON TESTING
![Page 68: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/68.jpg)
800-920-7227 | www.pinpointe.com
WHAT LARGER TESTS LOOK LIKE
![Page 69: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/69.jpg)
800-920-7227 | www.pinpointe.com
BIGGER CHANGES IN EMAIL
12: 52 PMGmail
![Page 70: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/70.jpg)
800-920-7227 | www.pinpointe.com
1:56 PMBusiness Email
![Page 71: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/71.jpg)
800-920-7227 | www.pinpointe.com
LANDING PAGE TEST SEQUENCE
PROBLEM: After clicking a button in our email, the subscriber arrives at a landing page. We
want to increase conversions on that landing page by 10%.
![Page 72: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/72.jpg)
800-920-7227 | www.pinpointe.com
VARIATION A
![Page 73: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/73.jpg)
800-920-7227 | www.pinpointe.com
THE METHOD
• Sampling Method: With 5,000 subscribers, we must randomly sample 726 customers in our database. 363 receive A, and 363 receive B in a 50/50 split.
• Length of Time: Check results five days from the time the email is sent out.
![Page 74: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/74.jpg)
800-920-7227 | www.pinpointe.com
VARIATION B
![Page 75: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/75.jpg)
800-920-7227 | www.pinpointe.com
THE RESULTS ARE IN
![Page 76: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/76.jpg)
800-920-7227 | www.pinpointe.com
WINNER
![Page 77: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/77.jpg)
800-920-7227 | www.pinpointe.com
BUT WAIT….WE NEED AN EXTRA STEP
Always review the quality of the data to make sure everything was fair
in the experiment. If not, you may be using bad data.
![Page 78: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/78.jpg)
800-920-7227 | www.pinpointe.com
THE IDEAL SCENARIO
![Page 79: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/79.jpg)
800-920-7227 | www.pinpointe.com
BUT DATA CAN BE MESSY
![Page 80: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/80.jpg)
800-920-7227 | www.pinpointe.com
PROBLEMS AND FIXES• Different open rates lower the validity of your data.
• You can’t use the results even though there was a clear winner.
• You can re-run the test with a new sample 50/50 split.
![Page 81: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/81.jpg)
800-920-7227 | www.pinpointe.com
EASY SPLIT TESTING IN PINPOINTE
![Page 82: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/82.jpg)
800-920-7227 | www.pinpointe.com
EASY SPLIT TESTING IN PINPOINTE
![Page 83: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/83.jpg)
800-920-7227 | www.pinpointe.com
EASY SPLIT TESTING IN PINPOINTE
![Page 84: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/84.jpg)
800-920-7227 | www.pinpointe.com
RECAP
1. 4 Simple Steps to Split Testing.
2. Have a business justification for the best testing results (you’re only as good as your assumptions).
3. Always establish a baseline metric with an A/A test.
4. Don’t create data for the sake of creating data.
5. Don’t spend all your time testing. Spend all your time improving. Split testing is only a tool.
![Page 85: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/85.jpg)
800-920-7227 | www.pinpointe.com
QuestionsAskMichael…DigitalMediaStrategistPinpointe [email protected]
Twitter/Facebook:@Pinpointewww.pinpointe.com
800-920-7227 | www.pinpointe.com
![Page 86: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/86.jpg)
800-920-7227 | www.pinpointe.com
We Thank YouWeappreciateyoutakingthetimetowatchthistoday!
800-920-7227 | www.pinpointe.com
![Page 87: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/87.jpg)
800-920-7227 | www.pinpointe.com
Check Out Our Blogwww.pinpointe.com/blog
Getmoregreatmarketingcontent.
Easilycreateautomatedcampaignsequences.www.pinpointe.com/get-started - Selectprogram&entercodePPTWEBNR
FREEMONTHofPinpointeService(limit100Kemails)
TryPinpointe’sEmailMarketingSolutionforFREE
800-920-7227 | www.pinpointe.com
![Page 88: Beyond A/B Testing: How to Take Your Split Tests to the Next Level](https://reader033.vdocuments.net/reader033/viewer/2022052709/58ac45bf1a28ab99028b52bf/html5/thumbnails/88.jpg)
800-920-7227 | www.pinpointe.com
Follow
formarketingtips&info@pinpointe
www.facebook.com/Pinpointewww.linkedin.com/company/pinpointe
Pinpointe
800-920-7227 | www.pinpointe.com