product management bootcamp · 2016-09-05 · not validate what you already think. 31. do customers...

Chris BradleyPublicate

PRODUCT MANAGEMENT BOOTCAMP

HI…

@briscradley

chris@publicate.it

WELCOMEHI…

About you…What you do…Whats the ONE thing you want to learn today?

LEARNING OBJECTIVESHI…

‣ Understanding the correct procedures and processes for validating an idea and de-risking a product or business

‣ Learn lean practices in; ‣ Customer development ‣ Testing and validating assumptions using experiments and

MVP ‣ Measuring with metrics

STAGES OF A BUSINESSHI…

EMPATHY VIRALITY SCALEREVENUESTICKINESS

WHO HAS AN IDEA?HI…

Pitch it…. 60 secondsForm 3-4 groups

PT 0 - ASSUMPTIONSOver 95% of all businesses started FAIL within the first 2 years, and even more fail after that…

Holy Shit…

WHY DO SO MANY FAIL?ASSUMPTIONS

‣ The product is built on false assumptions‣ $$$ spent in the wrong direction‣ No real Value delivered‣ Too much $$$ spent to make the right changes‣ Painful death

ASSUMPTIONS

”I HAVE NOT FAILED, I’VE JUST FOUND 10,000 WAYS THAT

WON’T WORK”Thomas Edison

ASSUMPTIONS

YOU ARE NOT THE ORACLE. EVERYTHING IS AN ASSUMPTION

UNTIL VALIDATED72%

THE LEAN CANVASASSUMPTIONS

THE LEAN CANVASASSUMPTIONS

1

8

9

7

5

6

3

4 2

EXERCISE

TIMING

DELIVERABLE

KEY OBJECTIVE(S)

ACTIVITY

Learn how to use the lean canvas

1. Create a lean canvas & fill it in with all your initial assumptions and ideas for each segment based on the idea pitched

30 min

A Lean Canvas full of initial assumptions

SWOT ANALYSIS

PT 1 - EMPATHY“Empathy is the capacity to understand what another person is experiencing from within the other person's frame of reference, i.e, the capacity to place oneself in another's shoes.”

SOLUTION OR PROBLEM?EMPATHY

CUSTOMER DEVELOPMENT

EMPATHY

WHY IS IT IMPORTANT TO

UNDERSTAND YOUR USERS?

INTERVIEWING USERS

PROCESS…INTERVIEWING USERS

Problem Interviews

Design Test(s)

Interviews

Initial Wireframes

Interviews

Wireframe Variations

Interviews

MVP

STEP 1

Interviews

INTERVIEWING USERS

UNDERSTANDING LONG TERM HUMAN OR BUSINESS NEEDS AND USING TECHNOLOGY TO IMPROVE THE OUTCOME

WHAT IS THE JOB PEOPLE ARE GOING TO HIRE YOUR PRODUCT TO DO?

INTERVIEWING USERS

INTERVIEWING USERS

‣ Store pictures in a safe place, like a shoebox under a bed - ????? ‣ Display favourite photos in prominent place like the mantelpiece

- ????? ‣ Collect scrap book of ideas and inspiration - ????? ‣ School kids want to pass notes and messages without people

seeing them - ????? ‣ Leave reviews and tips for other travellers in a guest book - ?????

OLD JOBS, NEW SOLUTIONS

INTERVIEWING USERS

WHO ARE MY USERS? WHAT ARE THE STEPS

INVOLVED IN GETTING THE JOB DONE?

WHAT ARE THEY TRYING TO GET DONE AT EACH

STEP?

WHAT NEEDS TO HAPPEN AT EACH STEP TO COMPLETE THE JOB

SUCCESSFULLY?

WHAT ARE THE CURRENT TOOLS AND BEHAVIOURS THEY ARE EMPLOYING TO GET THE JOB DONE?

WHAT IS THE JOB TO BE DONE?

EXAMPLE

INTERVIEWING USERS

PROBLEM INTERVIEW

PROBLEM INTERVIEWINTERVIEWING USERS

‣ Aim for face to face interviews

‣ Pick a neutral location ‣ Try not to record the

interview ‣ Prepare your script

properly

‣ Introduction ‣ Demographics ‣ Tell a problem story ‣ Test the problem ‣ Test the solution ‣ Ask for something

BEFORE THE INTERVIEW AT THE INTERVIEW

WHY DO I NEED TO KNOW THIS?

INTERVIEWING USERS

INTERVIEWING USERS

Do you think the company’s benefits are good?

Tell me your thoughts about the company’s benefits.

ASK OPEN-ENDED QUESTIONS.

27

INTERVIEWING USERS

What did you do the last time you had a technical issue?

What do you usually do when you have a technical issue?

FOCUS ON SPECIFIC INSTANCES.

28

INTERVIEWING USERS

What are your frustrations with the performance review process?

How do you feel about the performance review process?

AVOID LEADING QUESTIONS.

29

INTERVIEWING USERS

INTERVIEWING USERS

YOUR MISSION IS TO FIND THE TRUTH NOT VALIDATE WHAT YOU ALREADY THINK.

31

DO CUSTOMERS HAVE THE PROBLEM YOU THINK? IS IT PAINFUL ENOUGH?

INTERVIEWING USERS

• Did the customer successfully rank the problems?

• Is the customer already trying to solve the problem?

• Was the customer engaged and focused?

• Did they agree to a followup meeting?

• Did they offer to refer you to others?

• Did they offer to pay immediately?

31+

WHAT ARE YOU TESTING?

INTERVIEWING USERS

INTERVIEWING USERS

WHERE DO I FIND MY CUSTOMERS?

SHOPPING MALL

AT THE OFFICE

ONLINE FORUMS GROCERY SHOPPING

COFFEE SHOPS

ON THE STREET

REFERRAL NETWORK

SOCIAL NETWORKS

INTERVIEWING USERS

GET OUT OF THE BUILDING

MARKET RESEARCH

EMPATHY

MARKET RESEARCH

BENEFITS, NOT FEATURES

THE MARKET

GAP IN THE MARKET

MARKET IN THE GAP

COMPETITIVE ANALYSIS

SIMPLE COMPETITIVE ANALYSIS‣ What do your customers perceive as the most important value? (benefits) ‣ Outline and research competitors. ‣ Identify the key points of differentiation. ‣ Compare your business agains these key points. ‣ Look for the gap. ‣ What else can you collect?

COMPETITIVE ANALYSIS

GREEN = GAP

MARKET SIZING

WHAT IS MARKET SIZING?ANALYSE THE SCALE OF OPPORTUNITYHIGHLIGHTS ASSUMPTIONS TO BE TESTEDCAN BE DONE WITH LITTLE TO NO RESEARCH, OR LOTS OF RESEARCH SOMETIMES DONE AS A “GUT” CHECK

40

MARKET SIZING

CONCESSION ORDERING APP FOR NBA GAMES

‣ Number of teams: 30

‣ Number of games per team: 82

‣ Total games = (30 x 82) / 2 = 1,230

‣ Average available seats in NBA stadium: 18,000

‣ Average capacity per game: 85%

‣ Average attendance at NBA game = (0.85 x 18,000) = 15,300

‣ Average spend per attendee: $100 ($80 ticket, $20 concession)

‣ Total spend per game = 100 x 15,300 = $ 1.5M

‣ Total for all games = 1.5M x 1,230 = $1.8B

BOTTOMS UP!!!!

How do we work out the potential market size for this APP? - $$

41

MARKET RESEARCH

CONCESSION ORDERING APP FOR NBA GAMES

‣ Number of teams: 30

‣ Number of games per team: 82

‣ Total games = (30 x 82) / 2 = 1,230

‣ Average available seats in NBA stadium: 18,000

‣ Average capacity per game: 85%

‣ Average attendance at NBA game = (0.85 x 18,000) = 15,300

‣ Average spend per attendee: $100 ($80 ticket, $20 concession)

‣ Total spend per game = 100 x 15,300 = $ 1.5M

‣ Total for all games = 1.5M x 1,230 = $1.8B

BOTTOMS UP!!!!

MARKET RESEARCH

MARKET SIZING - TOP DOWN & BOTTOM UP

Search Benchmarks Key Word Planner

PT 2 - STICKINESSDo people see the value, and are they going to pay you for it?Yes! - I has businessNo! - I need to pivot

LETS BUILD IT! OR NOT…

STICKINESS

STICKINESS

YOUR JOB IS NOT TO BUILD A PRODUCT, IT’S TO DE-RISK A

BUSINESS MODEL

PROCESS…EMPATHY

Problem Interviews

Design Test(s)

Interviews

Initial Wireframes

Interviews

Wireframe Variations

Interviews

MVP

YOUR HERE

Interviews

PRODUCT/MARKET FIT

TESTING AND VALIDATING YOUR IDEA

PRODUCT MARKET FIT

2.5% 13.5% 34% 34% 16%

PRODUCT MARKET FIT

PRODUCT/MARKET FIT

Build

Measure

Learn

PRODUCT/MARKET FIT

Iterate your way to Product/Market Fit

YOUR USERS

THE ROAD TO PRODUCT MARKET FIT

MVP 53

MVP

WHAT IS AN ASSUMPTION?

54

MVP

YOU LOVE A GOOD EXPERIMENT…‣Hypothesis or assumption for testing‣Method that is going to test the assumption‣Line in the sand - what does success look like?‣Run it‣Evaluation & decision

55

EXPERIMENT FRAMEWORKTHE LEAN CANVAS

EXPERIMENT & ASSUMPTIONS - NEW PRODUCT

MVP 57

DROPBOXProblem: It’s hard to manage your files across systems and computers.

Customer: People who work across multiple platforms. Ex. Office workers who need to access files at home.

Riskiest assumption: If we provide an extremely easy to use product, people will try it.

Experiment: Video demonstrating ease

MVP 58

EXPERIMENT & ASSUMPTIONS - EXISTING PRODUCT

MVP 59

AIRBNB

Improve the number of nights a property was rented.

Professional quality photos will increase number of nights

the place is booked for.

Hire a few photographers to professionally photograph

a few properties and compare booking rates.

Metric

Hypothesis

Experiment

MVP 60

RESULTMVP 61

MVP

YOU DO NOT NEED TO CODE TO TEST

SOMETHING.

62

NON CODE EXAMPLES MVP

‣ Fictitious case study ‣ PPC / Ad Click throughs‣ Email lists‣ Landing pages‣ Designs wireframes‣ None technical prototypes

Example

63

LOOK AT ALL THE SHINY TOOLSMVP

Mailchimp Unbounce Launch Rock Balsamiq AxureSketch

64

LUNCH TIME!

TESTING AND VALIDATING YOUR IDEA

MVPS

67MVPS

THE LEAST AMOUNT OF WORK YOU CAN DO TO

LEARN THE MOST OF SOMETHING.

MINIMAL VIABLE PRODUCT

MVPS

MVP

68

69MVPS

WHY MVP?

70MVPS

REDUCE RISK & MAXIMIZE SUCCESS FASTER FEEDBACK

REDUCED OVERHEAD MEASURABLE PROGRESS

WHY MVP?

MVP

CONCIERGE‣ Delivering a service

manually to the customer. ‣ Ex: Food on the table

71

MVPS

WIZARD OF OZ‣ Everything appears to be

real to the customer, but on the back end it is manual.

‣ Example: Zappos

MVPS

LANDING PAGE‣ Used to see how much interest there is in the idea. ‣ Remember: you need to drive traffic to the site!

MVPS

VIDEOS‣ Used to demonstrate a hard

concept. ‣ Can be pieced together without

the technology being completely in place.

‣ Ex: Dropbox

MVP

CROWD FUNDING‣ Estimate early demand ‣ Prove traction before building ‣ £££ before building ‣ Pebble / Air umbrella

75

EXERCISE

TIMING

DELIVERABLE

KEY OBJECTIVE(S)

ACTIVITY

Design your first experiment

1. Outline your riskiest Assumption

2. Design the minimum method to test it

3. Define your line in the sand

5 min

20 min

Blueprint for your first experiment

5 min

MVP

YOU LOVE A GOOD EXPERIMENT…‣Hypothesis or assumption for testing‣Method that is going to test the assumption‣Line in the sand - what does success look like?‣Run it‣Evaluation & decision

77

EXPERIMENT FRAMEWORKTHE LEAN CANVAS

SOLUTIONSMVP 79

ANALYSE THE RESULTS‣ Affinity mapping ‣ Identify trends ‣ Look for common goals/

themes ‣ Remember: People have no

idea what they want. It’s your job to figure it out.

SOLUTIONIZE 80

SOLUTIONIZE

BRIANSTORM POSSIBLE SOLUTIONS‣Aim for speed, not refinement‣Level of detail depends on stage in process‣Get out as many possible solutions as you can‣ If you are struggling for ideas you probably haven't understood the problem and the customer well enough‣All solutions are assumptions, so don’t be precious

1

5

2

6

3

7

4

8

EXERCISE

TIMING

DELIVERABLE

KEY OBJECTIVE(S)

ACTIVITY

Outline the problem and come up with 8 solutions

1. Reframe / crystallise the problem

2. Do Crazy 8’s to come up with 8 possible solutions

5 min

15 min

Your crazy 8 sheet completed

DECIDING WHAT GOES INTO AN MVPMVP

‣ Customer research‣ Create and test prototypes - Minimum test possible‣ Park everything that is not valuable‣ ONE THING!‣ DO ONE THING!‣ DO ONE THING WELL!‣ Backlog everything else‣ But… no seriously, NO!

83

USABILITY TESTINGMVP 84

MVP

STORYBOARDING & WIRE FRAMING

A PANEL OR SERIES OF PANELS OF ROUGH SKETCHES OUTLINING THE SEQUENCE OF EVENTS THAT A CUSTOMER WILL EXPERIENCE WHILE USING YOUR PRODUCT DURING A SPECIFIC ACTIVITY.

ANALYSIS

USABILITY TESTING

Invision Usability hubSilverback

Visit people & Pick up the phone - EVERY DAY

‣ 3-5 for 95% bugs‣ What do you want

to test‣ Ask to complete

specific actions‣ Observe and shut

up

MVP 87

INFORMATION MANAGEMENT

STICKINESS

WHAT IS A PRODUCT MANAGER?

TECHNOLOGYBUSINESS

DESIGN

Product Manager

Credit: http://www.mindtheproduct.com/2011/10/what-exactly-is-a-product-manager/

WHAT IS A PRODUCT MANAGER?

WHAT DO YOU DO ALL DAY?10:00am Coffee & email, put out any fires

11:00am Dev team standup to review progress on feature in development

12:00pm Lunch at desk, acceptance testing new features in staging

1:00pm Meeting with merchandising customers to review requirements

for new sales tools

2:00pm Writing user stories for sales tools & backlog management

3:00pm Meeting with CEO, CMO, Head of Product to review KPIs for the week on

launched projects

4:00pm Read through support emails and contact users for further questions

5:00pm Meet with UX Designer to explain requirements for sales tools

6:00pm Start creating experiments for new sales tools

INFORMATION IS YOUR MAP TO TREASURESTICKINESS

SCRUM / AGILEPROCESS

ARE WE THERE YET?MVP 93

FINDING PRODUCT/MARKET FIT

“THE ONLY METRICS THAT ENTREPRENEURS SHOULD INVEST ENERGY IN COLLECTING ARE THOSE THAT HELP THEM MAKE DECISIONS.”

Eric Ries

M FOR MEASURE THE RIGHT THING & METRICS

STICKINESS

ONE METRIC THAT MATTERS

STICKINESS

KPIS

WHAT IS A

KEY

PERFORMANCE INDICATOR?

*Chart by Wilson Joseph from The Noun Project

STICKINESS

A GOOD METRIC ISUnderstandable Comparative A rate or ratio Behaviour Changing

If you’re busyexplaining thedata, you won’tbe busy actingon it.

Comparison iscontext.

The only way to understand performance(MPH).

The information enables you to make a decision.Otherwise whats the point?

STICKINESS

CUSTOMERS WILL... VISIT & ARRIVE

INTERACT & ENJOY COME BACK

SPREAD THE WORD GENERATE REVENUE

STICKINESS

MEASURE YOUR CUSTOMER JOURNEY.

AARRRACQUISITIONACTIVATIONRETENTIONREFERRALREVENUE

STICKINESS

ACQUISITIONGETTING NEW PEOPLE TO YOUR PRODUCT.

WAYS TO CLASSIFY A NEW ACQUISITION?

WHERE ARE THESE PEOPLE COMING FROM (SOURCES)?

Spends at least 1 minute on a pageVisits at least 2 pagesClicks at least 2 things

Social MediaWord of Mouth

BlogsSEO

Paid Advertising

STICKINESS

SOURCE

CLICKSIMPRESSIONS

CTRCONVERSIONSAVERAGE CPC

COST PER CONVERSIONBOUNCE RATE

KEY TERMINOLOGYSTICKINESS

ACQUISITIONSTICKINESS

ACTIVATIONWHEN NEW USERS SIGN-UP, USE, AND ENJOY YOUR PRODUCT.

WAYS TO CLASSIFY A NEW ACTIVATION?

Signed-UpFilled Out a Form

Used Specific Features

STICKINESS

PAGES PER VISIT

TIME PER CONVERSION

TASK / ACTIVITY COMPLETIONTIME ON SITECLICKS PER CONVERSION

KEY TERMINOLOGYSTICKINESS

ACTIVATIONSTICKINESS

RETENTIONUSERS THAT CONTINUE USING YOUR PRODUCT OVER AND OVER AGAIN.

WAYS TO CLASSIFY A REPEAT USER?

How often do they come back?How often do they perform specific actions?

How long is each visit to your product?

STICKINESS

TIME SINCE LAST VISIT

EMAIL OPEN RATESESSION LENGTHUSER LOYALTY

KEY TERMINOLOGYSTICKINESS

CHURN RATE

RETENTIONSTICKINESS

REFERRALUSERS TELLING FRIENDS TO ALSO BECOME USERS OF YOUR PRODUCT.

WAYS TO ENCOURAGE & INCREASE REFERRALS?

Social Media FeaturesBuilding Virality In

Affiliates

STICKINESS

AFFILIATES & AFFILIATE NETWORKS

% OF USERS REFERRINGVIRALITY

PAID & ORGANIC GROWTH

KEY TERMINOLOGY

# OF REFERRED USERS

STICKINESS

REFERRALSTICKINESS

REVENUETHE METHOD AND AMOUNT OF MONEY YOU’RE MAKING

THINGS TO CONSIDER ABOUT REVENUE?

How many users are paying?How long did it take for them to make a

purchase?How much money did they spend?

STICKINESS

TIME TO FIRST TRANSACTION

COST PER ACQUISITIONSUBSCRIPTIONS / RECURRING TRANSACTIONS

BREAK-EVEN

KEY TERMINOLOGYSTICKINESS

LTV

REVENUESTICKINESS

MORE SHINY TOOLS, ITS ABOUT THE MIX…

GOOGLE ANALYTICS KISSMETRICS INTERCOM

ANALYSIS

MIXPANEL WOOPRA

USER FLOWSTICKINESS

FUNNEL ANALYSISSTICKINESS

COHORT ANALYSISSTICKINESS

NPSMEASUREMENT

HAVE YOU GOT PRODUCT MARKET FIT?STICKINESS

35% - retention rate2% - churn rate40% - would be hate life if you died

Benchmark what makes sense

PT 3 - SUMMARYHas it sunk in?And…Did we get to the bottom of your questions?

SUMMARY

Build

Measure

LearnEmpathy

Stickiness

Metrics & research

Q&APM BOOTCAMP

TOP STARTUP MISTAKES TO AVOIDSUMMARY

‣Building a business on assumptions‣Falling in love with your solution‣Trying to be something for everyone‣Not focusing on monetisation strategy - Unicorns aren't real‣Not looking after yourself‣Money management‣Single founder‣Poor team / culture