Blending big & LITTLE dataOVERview effect

Ben  Babcock  Director  of  Research,  Jet.com

BENJAMINBabcock

A bit about me …

Research UX  Research UX  Research Dir.  Research

XBOX  gamesHardwareMobile

Amazon.comDevicesGrocery

Jet.comAll

Start  Up Big  Corp Start  Up  à Big  Corp Start  Up

• Jet is a new online marketplace (ecom site)

• Jet sells directly to consumers• Wide range of merchants sell on Jet

• It’s different … prices drop as you shop?!• 1st 12 months, 5+ million customers, 1b sales

Prices drop as you shop?!

Startups VS Big Companies ... what’s the big diff?

Startups Medium Big Corp

Who is our customer?

Love/hate about us?

Last time you spoke with a customer?

Your team improving customer experience?

Where do you go to learn about customers?

Make knowing your customer a thing (pop quiz)consistentspecific

specificprioritized

todaylast  week

metricsprojects

[team]CX  dashboard

inconsistentno  details

no  ideaall  love,  right?

neverWhy  would  I?

not  our  thinghow?

do  we  do  that?that  one  person?

NOT goodGood

Learn from our CS team, and give them tools

It goes both ways … (live listen-ins)

• Can do field studies quickly and easily, not multi week projects• Inexpensive (save money & time)• Do these often (100% participation)

FIELD STUDIES

FLOATING FIELD STUDY• Printed guide (NDA, etc)• Gift cards or $$• 15 min sessions• Creepy

• Customers come in every Thurs (rolling)• Live shopalong study (1hr, naturalistic)

• Everyone is invited (new hire orientation)• Observe, discuss, summarize, action

• Real-time iteration (test, iterate)

In-person studies …Real  customers

Free  Food  +  user  testing  =  FunDiscover,  insight,  action,  update,  repeat

Thursday

Popcorn & Insights

Live  notes  stream  via  Slack

Live  session Summary  Report

Live  stream

Typical insights handoff process

Faraway  (Remote)

Remote testing• Fast and easy - results in mins• Unmoderated & Moderated• Benchmarking, feature launches, etc• Your customers, or their panel

Sat  Survey

• Our field studies and in person studies yield a gold mine of insights and pain points• These findings fall into one of 3 buckets …

Insights in hand, we rapidly iterate

3 types of findings1) D’uh, just launch it!

(common sense)2) I think I know, but

(let’s be cautious)3) Yikes, let’s take

(some time to think)we  test,  using  A/B  testing  methodology

we  launch  immediately  to  improve  site  experience

parallel  path  the  stuff  that’s  not  as  easy,  needs  more  time

A/B testing (quick intro)

Buy Box

A B

?✔

C B

?✔

A B

✔

C

• Small, autonomous, vertical teams• Self-selection of projects• Unfettered access to data/research• Empowered to push it live!• Still have our day jobs, though

Squads

Fini. Thanks for tuning in!Ben  Babcock  Dir.  UX  Research,  Jet.com