ELIMINATE THE GUESSWORKfrom quick fix to structural success

ROEL WILLEMS

10+ year - digital analytics & conversion optimisation

Current: Senior Digital Analyst, Conversion optimization specialist - Ahold Delhaize

Former: Senior Consultant / Product manager Analytics and conversion

- OrangeValley

consumer behaviour from a data perspective

ELIMINATE THE GUESSWORK

➤ From test to testing

➤ Quick fix(es) to structural approach

➤ 3 key principes

FROM TEST TO TESTING

VOORBEELDCHALLENGE: WHERE AND WHAT TO TEST?

?HOMEPAGE CHECKOUT

VOORBEELDOPLOSSING: DO WE NEED MORE IDEAS?

VOORBEELDNEAR LIMITLESS OPPORTUNITY

“… starting isn’t a problem, continuity is the challenge.

“74% of the survey respondents who reported having a structured approach to conversion also stated they had improved their sales.

-source: Econsultancy Conversion Rate Optimization Report

FROM QUICK FIX(ES) TO STRUCTURAL APPROACH

3 KEY PRINCIPLES

VOORBEELDAD HOC TO STRUCTURAL APPROACH

-source: Adapted from Trochim, William M. The Research Methods Knowledge Base, 2nd Edition.

Result, but not reliable Reliable, but no result Reliable results

1. ELIMINATE GUESSWORK

VOORBEELDAD HOC / OPINION-BASED

“Most of the world will make decisions by either guessing or using there gut. They will be either lucky or wrong.

-Suhail Doshi

VOORBEELDLUCKY OR WRONG?

VOORBEELDFOCUS ON STRUCTURE & PROCESS

Ad hoc Opinion-based

Testing discipline Fact-based

VOORBEELDOPLOSSING: ITERATIVE APPROACH, LOOK FOR RELEVANCY

How can we stimulates visitor behaviour?

Can we make it more easy to use?

Does an element influence visitor behaviour?

VOORBEELDCASE

2. LEARN FROM FAILURE

VOORBEELDTESTING DISCIPLINE

“➤ Only one third of the ideas tested at Microsoft improved the

metric(s) they were designed to improve;

➤ at Google, only “about 10 percent of controlled experiments were leading to business changes;

➤ Netflix considers 90% of what they try to be wrong;

➤ Etsy “nearly everything fails” and “it's been humbling to realize how rare it is to succeed on the first attempt;

➤ “No matter how much you think it’s a no-brainer, how much research you’ve done, or how many competitors are doing it, sometimes, more often than you might think, experiment ideas simply fail.” - Skyscanner.

-source: exp-platform.com Online Controlled Experiments and A/B Tests

“It ain’t what you don’t know that gets you in trouble, it’s what you know for sure that ain’t so.

-Mark Twain

VOORBEELDDON’T TRUST IN INTUITION, … BUT VALIDATE ALTERNATIVES

VOORBEELDOPLOSSING: ACTION ORIENTED BIAS

➤ More experiments ≠ more effect

➤ Focus on information discovery, test valid alternatives

-source: Andrew Anderson

VOORBEELDCASE

3. FOCUS ON EFFICIENCY

VOORBEELDEFFICIENCY

ÍOPLOSSING: SINGLE BACKLOG

Backlog

Team

Visitors

√success / - failure

ÍOPLOSSING: ACCELERATE YOU OPTIMISATION PROGRAM? - MORE IDEAS?

Backlog

Team

Visitors

√success / - failure

ÍOPLOSSING: ACCELERATE? - MORE RESOURCES?

Backlog

Team

Visitors

√success / - failure

VOORBEELDOPLOSSING: MOVING FROM: SINGLE LANE TESTING…

ORHOMEPAGE CHECKOUT

VOORBEELDOPLOSSING: … TO INCREASE EFFICIENCY (AND THUS INFORMATION DISCOVERY)…

HOMEPAGE

CHECKOUT

CATEGORIE PRODUCT

BASKET THANKS

√

VOORBEELDOPLOSSING: …TO: MULTI-LANE TESTING

HOMEPAGE

CHECKOUT

CATEGORIE PRODUCT

BASKET THANKS

√

TAKE AWAY

VOORBEELDOPTIMISING THE ORGANISATION

Eliminate the guesswork learn from failure focus on efficiency

“Want to be a better marksman? Stop looking for silver bullets, improve you aiming.

Roel Willemsinfo@roelwillems.com

@roelwillems linkedin.com/in/roelwillems

VOORBEELDCONTACT