always be testing: how eroi + taco bell built a testing agenda (and what happened next)
TRANSCRIPT
![Page 1: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/1.jpg)
1
ALWAYS BE TESTING
How eROI + Taco Bell Built a Testing Agenda(and What Happened Next)
![Page 2: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/2.jpg)
Cher FullerSENIOR DIGITAL STRATEGIST
@cherfuller
Christine LiE-COMMERCE INNOVATION
@li_tacobeta
FULL BLEED ANCHORS
MARGINS
MARGINS
2
![Page 3: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/3.jpg)
3
FULL BLEED ANCHORS
MARGINS
MARGINS
“Without data, you’re just another person with an OPINION.”
-W. Edwards Deming, data scientist
![Page 4: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/4.jpg)
Here are some other people with opinions….
4
FULL BLEED ANCHORS
MARGINS
MARGINS
![Page 5: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/5.jpg)
5
FULL BLEED ANCHORS
MARGINS
MARGINS
Product testing TOLEDO, OH.
![Page 6: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/6.jpg)
6
FULL BLEED ANCHORS
MARGINS
MARGINS
Digitally engaged brandsswim in OCEANS OF DATA.
![Page 7: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/7.jpg)
7
FULL BLEED ANCHORS
MARGINS
MARGINS
More data is being collected and analyzed today than at ANY OTHER TIME IN HISTORY.
![Page 8: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/8.jpg)
8
FULL BLEED ANCHORS
MARGINS
MARGINS
2.5 billion gigabytesof data are collected
every day.
90% of globally stored data was collected in
the past two years.
IBM: What is Big Data? https://www-01.ibm.com/software/data/bigdata/what-is-big-data.html
![Page 9: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/9.jpg)
9
A
B A
B
By A/B TESTING, right?
![Page 10: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/10.jpg)
1 2 3 4 5
10
FULL BLEED ANCHORS
MARGINS
MARGINS
But A/B testing is step 5.
![Page 11: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/11.jpg)
1 2 3 4 5
11
FULL BLEED ANCHORS
MARGINS
MARGINS
But A/B testing is step 5.
You need to start at step 1.
![Page 12: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/12.jpg)
12
FULL BLEED ANCHORS
MARGINS
MARGINS
To swim this ocean of data,first TEST THE WATER.
![Page 13: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/13.jpg)
13
Build a Learning Agenda
FULL BLEED ANCHORS
MARGINS
MARGINS
![Page 14: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/14.jpg)
14
FULL BLEED ANCHORS
MARGINS
MARGINS
A LEARNING AGENDA asks specific questions in a specific order to set the framework for a testing strategy.
![Page 15: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/15.jpg)
LEARNING AGENDA
FULL BLEED ANCHORS
MARGINS
MARGINS
15
HIERARCHYWHY
Information hierarchy strongly influences user engagement.
By testing how prominently we present messages, we can measure what our users really care about—and why.
BENEFITS CONTENT TYPESWHY
To understand consumer incentive, we must pinpoint the benefit to online ordering.
Without this incentive, online ordering loses its value and risks shifting to in-store purchases.
WHY
Test engagement within several modules against a control to learn if specific types of content are more engaging than
others.Once we understand the value of certain modules, we can
make strategic decision around when/where to place them in triggered emails.
Hierarchy
Benefits Content Types
Information hierarchy strongly influences user engagement.
By testing how prominently we present messages, we can measure what our users really care about—and why.
Create an outlineThe success of a deck relies on a solid outline, which should happen well before you create a new deck.Drafting a deck outline will not only prevent hours of unused work, but it will be the nucleus for team alignment and effective storytelling.
The deck outline can also serve as a helpful means to define who speaks to what points, and function as speaking notes!
To understand consumer incentive, we must pinpoint the benefit to online ordering.
Without this incentive, online ordering loses its value and risks shifting to in-store purchases.
Gangsta’s ParadiseMINUTE AFTER MINUTE, HOURI’m the kinda G that little homieswant to be like.They keep spending most their lives living in the gangsta’s paradise. They keep spending most their lives living in the gangsta’s paradise. They keep spending most their lives living...They keep spending most their lives living in the gangsta’s paradise.They keep spending most their lives living in the gangsta’s paradise.They keep spending most their lives living in the gangsta’s paradise.
TOO MUCH TELEVISION WATCHING GOT ME CHASING DREAMS.I’M EDUCATED FOOL WITH MONEY ON MY MIND.
Test engagement within several modules against a control to learn if specific types of content are more engaging than others.
Once we understand the value of certain modules, we can make strategic decision around when/where to place them in triggered emails.
![Page 16: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/16.jpg)
16
Before starting any learning agenda, ask...
1. What are you testing?2. Why are you testing it?3. What do you want to learn?4. What will you do with your insights?
FULL BLEED ANCHORS
MARGINS
MARGINS
P
StHa
Before starting any LEARNING AGENDA, ask…
What do you want to learn?
What will you do
with your insights?
What are you
testing?
Why are you
testing it?
What are you testing?
What do you want to learn?
Why are you testing it?
What will you do with your insights?
1
2
3
4
![Page 17: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/17.jpg)
17
SEE TEXT ON SLIDEFULL BLEED
ANCHORS
MARGINS
MARGINS
P
StHa Why does a brand care about the DATA STORY?
![Page 18: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/18.jpg)
18
Where We Sometimes Misstep
FULL BLEED ANCHORS
MARGINS
MARGINS
![Page 19: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/19.jpg)
19
FULL BLEED ANCHORS
MARGINS
MARGINS
NOT EVERYONE speaks data.We need to translate it.
![Page 20: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/20.jpg)
20
WHAT WE WILL COVER TODAY
FULL BLEED ANCHORS
MARGINS
MARGINS
Make the complexSIMPLE. A LEARNING AGENDA
might test:
![Page 21: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/21.jpg)
21
FULL BLEED ANCHORS
MARGINS
MARGINS
TELL STORIES,not reports.
![Page 22: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/22.jpg)
22
So We RevisualizedThe Learning Agenda.
FULL BLEED ANCHORS
MARGINS
MARGINS
![Page 23: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/23.jpg)
23
FULL BLEED ANCHORS
MARGINS
MARGINS
BEFORE AFTER
![Page 24: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/24.jpg)
24
SEE TEXT ON PAGE
FULL BLEED ANCHORS
MARGINS
MARGINS
P
StHa This learning agendahelped Taco Bell’s brand teamTAKE TRULY DATA-DRIVEN ACTIONS.
![Page 25: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/25.jpg)
25
We RevisualizedOur Performance Reports Too
FULL BLEED ANCHORS
MARGINS
MARGINS
![Page 26: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/26.jpg)
26
FULL BLEED ANCHORS
MARGINS
MARGINS
Digital Strategists
Performance Analysts
![Page 27: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/27.jpg)
27
FULL BLEED ANCHORS
MARGINS
MARGINS
P
StHa PERFORMANCE REPORTSinfluence brand decisions
![Page 28: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/28.jpg)
PERFORMANCEReports
![Page 29: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/29.jpg)
29
FULL BLEED ANCHORS
MARGINS
MARGINS
qualitative data branding operations quantitative data
performanceAudienceBrand positionMessage / toneVisual style directionContent strategy
![Page 30: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/30.jpg)
30
Data Storytelling Opens the Door to Bigger (!) Creative Possibilities
FULL BLEED ANCHORS
MARGINS
MARGINS
![Page 31: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/31.jpg)
TOP Emails 2016
31
FULL BLEED ANCHORS
MARGINS
MARGINS
headline
CTA
headline
CTA
copy
1
2
3
4
5
![Page 32: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/32.jpg)
CYBER MONDAYDelivered 42% more clicksthan average
And a 24% higher click-to-open rate than average
FULL BLEED ANCHORS
MARGINS
MARGINS
32
![Page 33: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/33.jpg)
FULL BLEED ANCHORS
MARGINS
MARGINS
NAKED CHICKEN CHALUPABy placing the secondary messagefirst, we gained +10% total clicksand 2x higher conversions.
![Page 34: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/34.jpg)
34
To Recap
FULL BLEED ANCHORS
MARGINS
MARGINS
![Page 35: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/35.jpg)
35
FULL BLEED ANCHORS
MARGINS
MARGINS
TO RECAPData is cool
![Page 36: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/36.jpg)
36
FULL BLEED ANCHORS
MARGINS
MARGINS
TO RECAPFirst ask WHY, then WHAT
![Page 37: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/37.jpg)
37
FULL BLEED ANCHORS
MARGINS
MARGINS
TO RECAPTell stories, not reports
![Page 38: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/38.jpg)
38
FULL BLEED ANCHORS
MARGINS
MARGINS
TO RECAPCollaborate!
![Page 39: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/39.jpg)
39
FULL BLEED ANCHORS
MARGINS
MARGINS
TO RECAPInformed creative > Pretty things
![Page 40: Always be Testing: How eROI + Taco Bell Built a Testing Agenda (and What Happened Next)](https://reader031.vdocuments.net/reader031/viewer/2022030311/58ef2d821a28ab19318b4603/html5/thumbnails/40.jpg)
40
Any questions?
FULL BLEED ANCHORS
MARGINS
MARGINS