Teresa Torres www.ProductTalk.org !!

{@ttorres !!

How to Choose Your Next Experiments

@ttorres2Photo  Credit:  @Librarygroover  on  Flickr  |  CC  A:ribution  2.0  

Imagine you are tasked with building a public transit app for your city.

Imagine  you  are  tasked  with  building  a  public  transit  app  for  your  city.  What  would  you  do?    Get  the  audience  to  throw  out  suggestions.  Capture  them  on  a  whiteboard.

@ttorres3Photo  Credit:  @jakecaptive  on  Flickr  |  CC  A:ribution  2.0  

It’s easy to generate a lot of ideas.

It’s  pre:y  easy  to  generate  ideas.    What  would  you  do  next?    Get  audience  reply.  We  need  to  decide  where  to  invest.  If  everybody  agrees  on  the  top  idea,  you  can  start  pursuing  that  one.  But  how  often  does  that  happen?  More  often  than  not  people  disagree.  Everyone  has  their  favorite.  And  more  often  than  not  that  leads  to  …

@ttorres

The most common Lean Startup mistake is to

test everything.

4

…the  most  common  Lean  Startup  mistake  -­‐‑  testing  everything.    Instead  of  making  a  decision,  we  turn  to  testing  to  avoid  conflict.    We  think  this  is  the  Lean  way.    But  this  is  an  expensive  mistake.  It  requires  building  out  MVPs  for  each  idea  and  finding  the  right  testing  environment.    This  can  be  a  lot  of  work.

@ttorres

The Lean Startup gives us tools to test our

judgment not replace it.

5

The  Lean  Startup  gives  us  tools  to  test  our  judgment  not  replace  it.    If  we  try  to  test  every  idea  we  have,  we’ll  run  out  of  money  before  we  deliver  any  value.    This  is  true  for  venture-­‐‑backed  startups  as  well  as  profitable  enterprises.    If  you  stop  shipping,  your  revenue  and  funding  will  dry  up.    In  order  to  keep  moving  forward,  we  have  to  make  judgment  calls.    We  can’t  assume  that  we’ll  have  time  to  test  everything.  So  how  do  we  decide  what  to  test?    

@ttorres

Your job is not to generate good ideas.

6

Here’s  the  fundamental  challenge.    We  think  our  job  is  to  generate  good  ideas.    We  assume  we  are  one  good  idea  away  from  a  successful  product.    This  is  evident  in  the  way  that  entrepreneurs  (and  consumers)  lament  that  they  thought  of  Uber  a  decade  ago.    Who  among  us  hasn’t  had  a  similar  thought?    We  think  the  value  is  in  having  the  good  idea.    Nothing  could  be  further  from  the  truth.    This  belief  that  the  value  is  in  having  the  idea  leads  to  us  arguing  over  whose  idea  is  be:er.    We  fall  in  love  with  our  ideas.    It  creates  conflict  and  instead  of  trying  to  resolve  that  conflict,  we  say,  “Let’s  just  test  it.”  How  many  of  you  have  had  this  experience?    If  our  job  isn’t  to  have  good  ideas,  what  is  it?  

@ttorres

Your job is to define the shared context so that good ideas emerge.

7

Our  job  is  to  define  the  shared  context  so  that  good  ideas  emerge.    When  we  disagree,  more  often  than  not,  it’s  because  we  have  a  different  interpretation  of  the  problem.    We  are  starting  from  different  perspectives.    Our  goal  should  be  to  define  enough  of  a  shared  context  so  that  when  an  idea  does  emerge  -­‐‑  from  anywhere  in  the  organization  -­‐‑  everyone  can  look  at  that  idea  and  say,  yes  this  idea  is  worth  testing  or  no  this  idea  is  not  worth  testing    How  do  we  do  that?  

@ttorres8

This  is  the  framework  that  I  use  in  my  coaching  practice  to  develop  shared  context.    The  right  side  of  this  graphic  is  gong  to  look  familiar.  Steps  4,  5,  and  6  look  a  lot  like  the  Build  -­‐‑>  Measure  -­‐‑>  Learn  loop.    We  take  our  top  ideas,  we  identify  the  assumptions  they  depend  on,  we  collect  evidence  through  experimentation,  and  we  decide  whether  to  invest  in  that  idea  or  jump  to  the  next  idea.    What  might  not  look  familiar  are  steps  1,  2,  and  3.    Steps  1  and  2  in  particular,  are  the  steps  that  create  shared  context.    That’s  where  we  are  going  to  spend  most  of  our  time  today.    They  do  a  couple  of  things:    First,  they  alleviate  the  pressure  from  you  as  a  founder  or  a  product  manager  from  having  to  generate  all  the  good  ideas  yourself.  They  help  you  engage  the  whole  company  in  sourcing  ideas.    And  finally,  they  make  it  clear  which  ideas  are  worth  testing  and  which  should  be  skipped  over.    !

@ttorres

Map the Challenge?

9

That’s  a  big  promise.    Let’s  start  with  the  first  step,  mapping  the  challenge.    What  do  I  mean  by  that?    You  are  probably  already  familiar  with  the  most  common  way  of  mapping  challenges.  

@ttorres

Customer Journey “a diagram that illustrates the steps your customers go through in engaging with your customer.” - HBR

10 Source: https://hbr.org/2010/11/using-customer-journey-maps-to

Customer  journeys.    As  HBR  defines,  a  customer  journey  is  a  diagram  that  illustrates  the  steps  your  customers  go  through  in  engaging  with  your  company.    How  many  of  you  use  customer  journeys  at  your  company?  Great.  Why  do  customer  journeys  help?    They  create  shared  context  by  visually  mapping  the  different  touch  points  a  customer  has  with  our  product.    It  allows  us  to  all  look  at  the  same  view  of  how  a  customer  engages  with  us.  We’ll  look  at  why  this  ma:ers  in  just  a  minute.    But  first,  I  don’t  like  this  particular  definition.    It’s  too  product-­‐‑centric  and  not  customer-­‐‑centric  enough.    Let’s  modify  it.  

@ttorres

Customer Journey “a diagram that illustrates the steps your target customer goes through when trying to accomplish a task”

11

Instead,  I  prefer  to  define  a  customer  journey  as  a  diagram  that  illustrates  the  steps  your  target  customer  goes  through  when  trying  to  accomplish  a  task.  I  made  two  modifications.    First,  I  changed  your  customers  to  your  target  customers.  Now  we  can  start  to  think  about  a  customer  journey  for  our  transit  problem  even  though  we  don’t  have  any  customers.    And  I  switched  the  focus  from  the  customer  engaging  with  a  product  or  company  to  them  trying  to  accomplish  a  task.    This  shifts  the  focus  to  our  customer  and  what  they  are  trying  to  do  and  not  to  us  and  what  we  are  trying  to  get  them  to  do.  This  is  a  critical  distinction.  

@ttorres12Photo  Credit:  @Librarygroover  on  Flickr  |  CC  A:ribution  2.0  

Your turn: Map your most recent public transit journey.

Now  that  we  have  a  new  definition  for  our  customer  journey,  let’s  try  to  create  one.    Take  a  few  minutes  to  draw  out  a  customer  journey  related  to  public  transit.    It  doesn’t  have  to  be  perfect.  Don’t  over  think  it.    Take  a  few  minutes  and  sketch  out  your  most  recent  experience.  Don’t  worry  about  product  or  service  ideas  yet.    Just  map  the  journey.

@ttorres

Star your biggest pain point.

13

Now  take  a  look  at  your  whole  journey  and  put  a  start  next  to  the  step  in  your  journey  that  you  think  is  your  biggest  pain  point  related  to  taking  public  transit.  

@ttorres

Share your journey and pain point with your neighbor.

14

Now  turn  to  your  neighbor  next  to  you  and  share  your  journey  and  pain  point  with  them.    (3  minutes)  Now  switch  neighbors  and  share  again.    (3  minutes)  How  many  of  you  had  the  same  pain  point  as  your  neighbors?    How  many  of  you  had  the  same  map  as  your  neighbor?  

@ttorres

We each bring a unique perspective to the challenge.

15

Hopefully,  you  just  experienced  the  first  takeaway  from  this  talk.    We  each  bring  a  unique  perspective  to  the  challenge.  Based  on  our  experience  and  knowledge.  The  same  is  true  for  everyone  on  your  team.    The  same  is  true  for  your  customers.  Extreme  users  will  have  a  completely  different  view  of  the  challenge  than  the  average  customer.    

@ttorres

You want to uncover multiple perspectives.

16

When  I  talk  about  mapping  the  challenge,  the  goal  is  to  uncover  as  many  different  perspectives  as  you  can.    The  more  perspectives  you  uncover,  the  be:er  you  will  understand  the  challenge.    Eventually,  you’ll  align  around  one  shared  perspective  that  will  drive  your  minimum  viable  product.    !Okay,  let’s  circle  back.  How  does  this  help  us  decide  which  experiments  to  run?    !

@ttorres

What happened when you identified your

biggest pain point?

17

Think  back  to  just  a  few  minutes  ago.  What  happened  when  I  asked  you  to  draw  a  star  next  to  your  biggest  pain  point?    Did  you  start  to  think  of  ways  to  solve  it?    This  is  the  power  of  mapping  out  the  challenge.    We  start  to  identify  points  of  intervention.    We  start  to  see  how  we  might  impact  the  journey.    We  start  to  focus  our  ideas  around  a  single  point.    !

@ttorres18

Let’s  go  back  to  this.    Remember  if  you  skip  steps  1  and  2  and  you  just  start  generating  ideas,  you  come  up  with  all  kinds  of  ideas.    You  have  no  idea  which  ones  will  have  an  impact  and  which  ones  won’t.    Refer  to  our  list  from  the  start.    But  if  instead,  we  start  by  mapping  the  challenge,  then  identify  our  points  of  intervention  (in  this  case  our  biggest  pain  point),  we  automatically  start  to  source  ideas  around  that  specific  point.    We  get  focused  ideation.  We  get  a  set  of  ideas  on  how  to  solve  the  same  problem  not  a  variety  of  problems.    

@ttorres

What happens when you don’t map the challenge?

19

You’ve  probably  experienced  it.  It  looks  something  like  this.    I  want  to  build  a  feature  that  tells  me  when  the  next  bus  is  coming.    You  want  to  build  a  feature  that  allows  you  to  buy  a  ticket  from  your  phone.    We  have  no  shared  context.  We  have  no  idea  which  idea  to  pursue.    So  we  decide  to  build  an  MVP  for  each.    But  the  problem  with  this  scenario  is  that  there  are  probably  more  than  2  people  at  our  company  with  ideas.    This  approach  doesn’t  scale.    We  can’t  MVP  everybody’s  pet  idea.    Although  some  companies  try.

@ttorres

When we map the challenge, we build

shared understanding.

20

As  we  explore  multiple  perspectives  of  the  challenge,  we  start  to  converge  on  our  company  perspective.  As  we  hear  about  many  different  pain  points,  we  start  to  understand  where  we  can  have  the  biggest  impact  on  the  whole  journey.    We  identify  our  points  of  intervention   When  we  map  the  challenge,  we  build  shared  understanding.    This  allows  us  to  focus  ideation  to  a  single  problem  Ruling  out  all  of  our  other  ideas.  Fast-­‐‑tracking  just  those  ideas  that  help  solve  our  biggest  pain  point.  

@ttorres

What do we do if we still have

too many ideas?

21

This  is  a  common  problem.    Even  when  you  narrow  ideation  to  just  your  biggest  pain  point,  you  will  probably  have  far  too  many  ideas  to  test.    Let’s  try  it.  Suppose  our  biggest  pain  point  is  purchasing  a  train  ticket.  Let’s  do  a  li:le  brainstorming  based  on  our  own  public  transit  experience.    !

@ttorres

How might we improve the

ticket purchasing process?

22

Capture  ideas  on  the  whiteboard.  !Okay,  so  once  again  we  have  too  many  ideas.    What  are  we  supposed  to.    We  can’t  possibly  build  MVPs  for  all  of  these  ideas.    How  do  we  decide  which  to  build?    This  leads  us  to  the  second  most  common  Lean  Startup  mistake  …

@ttorres

The second most common Lean Startup mistake is to

test your ideas.

23

Don’t  test  your  ideas.    What?    Isn’t  that  what  the  Lean  Startup  teaches?    No.  We  have  way  too  many  ideas.  We  can’t  possibly  test  all  of  them.    

@ttorres

Don’t test your ideas, test the

underlying assumptions.

24

We  shouldn’t  be  testing  our  ideas.  We  should  be  testing  the  assumptions  that  have  to  be  true  for  those  ideas  to  work.    This  is  an  important  distinction  for  two  reasons:    1. We  can  usually  test  assumptions  much  quicker  than  we  can  test  ideas.    2. Many  ideas  share  the  same  assumptions,  allowing  us  to  build  support  or  refute  many  ideas  at  once.    Let’s  take  a  look  at  some  of  the  ideas  we  generated.    Go  back  to  the  whiteboard,  enumerate  some  of  the  assumptions.    Talk  through  how  to  test  them  -­‐‑  note  how  it  doesn’t  require  building  as  much,  note  how  what  we  learn  rules  out  or  supports  whole  sets  of  ideas.    You  can  prioritize  your  experiments  by  testing  your  riskiest  assumptions  first.  And  remember  if  you  test  your  assumptions  instead  of  your  ideas,  this  will  accelerate  your  cycles  through  the  build  -­‐‑>  Measure  -­‐‑>  Learn  loop  !

@ttorres25

Explore Multiple Perspectives

We’ve  come  a  long  way.  Let’s  sum  up.    First,  you  want  to  always  start  by  mapping  the  challenge.    Explore  as  many  perspectives  as  you  can  before  converging  on  your  shared  company  perspective  of  the  challenge.

@ttorres26

Key to finding your MVP

Second,  identify  one  or  two  points  of  intervention.    These  are  the  points  in  your  journey  where  you  think  you  can  have  the  biggest  impact,  create  the  most  value,  reduce  the  most  pain.    This  is  the  key  to  finding  your  MVP.  

@ttorres27

Limit your ideasto just the points where you can havethe biggest impact.

Third,  limit  your  ideas  to  just  the  points  where  you  can  have  the  biggest  impact.    Ignore  everything  else.    

@ttorres28

Test your assumptions.Not your ideas.

And  finally,  don’t  test  your  ideas.    Test  the  assumptions  that  have  to  be  true  in  order  for  your  ideas  to  work.    Prioritize  your  riskiest  assumptions.  This  will  lead  to  fewer  tests  and  faster  iterations  through  the  Build  -­‐‑>  Measure  -­‐‑>  Learn  cycle.

Teresa Torres Product Consultant & Coach !!

{

ProductTalk.org@ttorres

Let’s keep the conversation going.

Thank  you.    I’d  love  to  keep  the  conversation  going.  I  blog  at  ProductTalk.org  and  you  can  find  me  on  twi:er  @:orres  Please  don’t  hesitate  to  reach  out.