Thriving on the journey of continuous optimization

Clearhead Blu Dot

Optimizely

Today’s Speakers

Robin Pam Product marketing!

Matty Wishnow Founder and CEO!

Mike Wodtke Director of e-Commerce!

Housekeeping

This webinar is being recorded Slides and recording will be e-mailed There will be time for Q&A at the end Submit questions via the Q&A box below

1997

1998

1994

2000

2006

2010

2015

Great customer experiences are the result of continuous iteration

and testing

++

test everything?who says you can’t?

optimizely + clearhead + blu dotmay 27, 2015

test everything? who says you can’t? | May 2015 2

hi. i’m matty,founder & ceo at

hi. i’m mike,director of e-commerce at

test everything? who says you can’t? | May 2015

test everything? who says you can’t? | May 2015 3

Optimizely's #1 North American Solution Partner

Proud nominee of: Opticon Partner of the Year, Developer of the Year, Most Dramatic Business Impact & Experience of the Year

Founded by clients who uniquely understand how to help solve client problems

Proud partners to over two dozen clients including Adidas, Tory Burch, FreshDirect, Cost Plus, Warner Music Group, Blu Dot & many more

Company-owned stores in: New York City (Soho), Los Angeles, San Francisco, Austin, and MinneapolisOutlet coming Summer 2015

Partner stores in: Sydney, Australia and Monterrey, Mexico

Over 100 independent retailers throughout the world

Design Awards: Cooper-Hewitt National Design AwardProduct Design Finalist (2013) The Chicago Atbenaeum Good Design Award (2010 & 2011) ICFF 2010 Editors Award for Body of Work AIGA Certificate of Excellence in Design for Blu Dot Catalog (2009)

founded in 1996 in Minneapolis with the goal of making modern design

accessible to everyoneAustin-based validation and optimization agency

test everything? who says you can’t? | May 2015 4

our back story. 2013.

0 a/b testing tools

0 tests run

conversion rate

online revenue

growth

test everything? who says you can’t? | May 2015 5

our A-HA!moment

a cb

test everything? who says you can’t? | May 2015 6

the tests that stick with you

font changesdelivery promotion

checkout flowreviews on PDP navigation

add to cart / cart redirecthome page redesign

color swatch display header redesign highlight search

mobile navigationinterstitial upsell live chat

inline email sign up financing options

experiment:

test everything? who says you can’t? | May 2015 7

product color optionshypothesis: I believe if we add color selections to category pages we'll increase clicks and conversions.

We believe adding the little color circles on the category level pages will increase people's awareness of additional color options and therefore increase the number of people clicking on products and requesting swatches or buying.

control swatch count

colors as text

only swatches

functional swatches

order conversion index 100 +154 +158 +133 +117 confidence

90%

visitor PDP conversion rate 51.8% 51.8% 51.9% 48.1% 48.5% confidence

99%

visitor add to cart index 100 101 106 92 100

experiment:

test everything? who says you can’t? | May 2015 8

catalog request modalhypothesis: I believe if we replace the email modal capture with a Free Catalog modal capture then conversion to catalog requests will go up.

Currently we feature a email capture modal when visitors come to our website.

control catalog modal

modal conversion rate

% difference

0.83% - - - - -

1.86% 124.28%

confidence99%

newsletter signup

conversion index100 29 confidence

99%

newsletter signup

conversion rate modal

% difference

0.83% - - - - - n/a

catalog request conversion index 100 875

catalog request conversion rate

modal % difference

n/a 1.86% - - - - -

n N

experiment:

test everything? who says you can’t? | May 2015 9

zip & coupon redesignhypothesis: I believe that coupon and zip code entry fields are competing with the modest checkout button and are negatively impacting conversion.

These are extra steps that require friction and embolden customers to leave this page right when they are considering clicking "checkout." The goal of this page is to get customers into the funnel.

I believe that if we redesign the coupon field to be less disruptive on the page, then we will see an increase in conversion rate and revenue.

control zip and coupon redesign

conversion to checkout page

% difference34.75%

- - - - -43.75%

25.9%confidence

99%

orderconversion rate

% difference20.85%

- - - - -24.55%

17.8%confidence

90%

conversion to apply coupon

code clicks % difference

0.56% - - - - -

1.57% 180.4%

confidence91%

zip code formconversions % difference

9.52% - - - - -

0.00% - - - - -

test everything? who says you can’t? | May 2015 10

goal & problem identification

1

2

3

goals

a

b

c

d

problemsvalidated by data

1

2

3

4

hypothesesprioritized by data

test everything? who says you can’t? | May 2015 11

e-commerce resourcing & prioritization

acquisition & retention marketing Norm 40% | Blu Dot 45%

design & UX Norm 10% | Blu Dot 10%

merchandising & content Norm 15% | Blu Dot 5%

features & development Norm 20% | Blu Dot 10%

analytics Norm 10% | Blu Dot 5%

optimization Norm 5% | Blu Dot 25%

average company Blu Dot

25%

5%

optimizationoptimization

test everything? who says you can’t? | May 2015 12

test everything?

catalog modal: tested & optimized

homepage redesign: tested & optimized

security badge: tested & optimized

color options: tested & optimized

product gridon mobile:

tested & optimized

inline email on mobile:

tested & optimized

test everything? who says you can’t? | May 2015 13

diminishing returns?

benefit

time

win rate

return on investment

conversion rate

number of tests run

test everything? who says you can’t? | May 2015 14

what’snext?

personalization

PDP redesign

+

test everything? who says you can’t? | May 2015 15

5 takeaways:

1 Optimization doesn't have to be just another thing. It can be the thing.

2 Even with limited traffic and resources, you can still be a mature and agile optimization organization.

3 Make sure your hypotheses are prioritized based on the opportunity and correlate to problems you have qualified. Otherwise, you’re just dabbling.

4 Imagine a world where you dedicated more resources to optimization than to “table stakes”

5 Testing everything gives you confidence — statistical confidence and the confidence of knowing that your organization is always, increasingly likely to succeed.

thanks!

matt wishnow, founder & ceo matt@clearhead.me

clearhead.me

+ Blu Dotclearhead

mike wodtke, director of e-commercemwodtke@BluDot.com

BluDot.com

+clearhead

Q&A

Thank You!