product management bootcamp · 2016-09-05 · not validate what you already think. 31. do customers...
TRANSCRIPT
Chris BradleyPublicate
PRODUCT MANAGEMENT BOOTCAMP
WELCOMEHI…
About you…What you do…Whats the ONE thing you want to learn today?
LEARNING OBJECTIVESHI…
‣ Understanding the correct procedures and processes for validating an idea and de-risking a product or business
‣ Learn lean practices in; ‣ Customer development ‣ Testing and validating assumptions using experiments and
MVP ‣ Measuring with metrics
STAGES OF A BUSINESSHI…
EMPATHY VIRALITY SCALEREVENUESTICKINESS
WHO HAS AN IDEA?HI…
Pitch it…. 60 secondsForm 3-4 groups
PT 0 - ASSUMPTIONSOver 95% of all businesses started FAIL within the first 2 years, and even more fail after that…
Holy Shit…
WHY DO SO MANY FAIL?ASSUMPTIONS
‣ The product is built on false assumptions‣ $$$ spent in the wrong direction‣ No real Value delivered‣ Too much $$$ spent to make the right changes‣ Painful death
ASSUMPTIONS
”I HAVE NOT FAILED, I’VE JUST FOUND 10,000 WAYS THAT
WON’T WORK”Thomas Edison
ASSUMPTIONS
YOU ARE NOT THE ORACLE. EVERYTHING IS AN ASSUMPTION
UNTIL VALIDATED72%
THE LEAN CANVASASSUMPTIONS
THE LEAN CANVASASSUMPTIONS
1
8
9
7
5
6
3
4 2
EXERCISE
TIMING
DELIVERABLE
KEY OBJECTIVE(S)
ACTIVITY
Learn how to use the lean canvas
1. Create a lean canvas & fill it in with all your initial assumptions and ideas for each segment based on the idea pitched
30 min
A Lean Canvas full of initial assumptions
SWOT ANALYSIS
PT 1 - EMPATHY“Empathy is the capacity to understand what another person is experiencing from within the other person's frame of reference, i.e, the capacity to place oneself in another's shoes.”
SOLUTION OR PROBLEM?EMPATHY
CUSTOMER DEVELOPMENT
EMPATHY
WHY IS IT IMPORTANT TO
UNDERSTAND YOUR USERS?
INTERVIEWING USERS
PROCESS…INTERVIEWING USERS
Problem Interviews
Design Test(s)
Interviews
Initial Wireframes
Interviews
Wireframe Variations
Interviews
MVP
STEP 1
Interviews
INTERVIEWING USERS
UNDERSTANDING LONG TERM HUMAN OR BUSINESS NEEDS AND USING TECHNOLOGY TO IMPROVE THE OUTCOME
WHAT IS THE JOB PEOPLE ARE GOING TO HIRE YOUR PRODUCT TO DO?
INTERVIEWING USERS
INTERVIEWING USERS
‣ Store pictures in a safe place, like a shoebox under a bed - ????? ‣ Display favourite photos in prominent place like the mantelpiece
- ????? ‣ Collect scrap book of ideas and inspiration - ????? ‣ School kids want to pass notes and messages without people
seeing them - ????? ‣ Leave reviews and tips for other travellers in a guest book - ?????
OLD JOBS, NEW SOLUTIONS
INTERVIEWING USERS
WHO ARE MY USERS? WHAT ARE THE STEPS
INVOLVED IN GETTING THE JOB DONE?
WHAT ARE THEY TRYING TO GET DONE AT EACH
STEP?
WHAT NEEDS TO HAPPEN AT EACH STEP TO COMPLETE THE JOB
SUCCESSFULLY?
WHAT ARE THE CURRENT TOOLS AND BEHAVIOURS THEY ARE EMPLOYING TO GET THE JOB DONE?
WHAT IS THE JOB TO BE DONE?
EXAMPLE
INTERVIEWING USERS
PROBLEM INTERVIEW
PROBLEM INTERVIEWINTERVIEWING USERS
‣ Aim for face to face interviews
‣ Pick a neutral location ‣ Try not to record the
interview ‣ Prepare your script
properly
‣ Introduction ‣ Demographics ‣ Tell a problem story ‣ Test the problem ‣ Test the solution ‣ Ask for something
BEFORE THE INTERVIEW AT THE INTERVIEW
WHY DO I NEED TO KNOW THIS?
INTERVIEWING USERS
INTERVIEWING USERS
Do you think the company’s benefits are good?
Tell me your thoughts about the company’s benefits.
ASK OPEN-ENDED QUESTIONS.
27
INTERVIEWING USERS
What did you do the last time you had a technical issue?
What do you usually do when you have a technical issue?
FOCUS ON SPECIFIC INSTANCES.
28
INTERVIEWING USERS
What are your frustrations with the performance review process?
How do you feel about the performance review process?
AVOID LEADING QUESTIONS.
29
INTERVIEWING USERS
INTERVIEWING USERS
YOUR MISSION IS TO FIND THE TRUTH NOT VALIDATE WHAT YOU ALREADY THINK.
31
DO CUSTOMERS HAVE THE PROBLEM YOU THINK? IS IT PAINFUL ENOUGH?
INTERVIEWING USERS
• Did the customer successfully rank the problems?
• Is the customer already trying to solve the problem?
• Was the customer engaged and focused?
• Did they agree to a followup meeting?
• Did they offer to refer you to others?
• Did they offer to pay immediately?
31+
WHAT ARE YOU TESTING?
INTERVIEWING USERS
INTERVIEWING USERS
WHERE DO I FIND MY CUSTOMERS?
SHOPPING MALL
AT THE OFFICE
ONLINE FORUMS GROCERY SHOPPING
COFFEE SHOPS
ON THE STREET
REFERRAL NETWORK
SOCIAL NETWORKS
INTERVIEWING USERS
GET OUT OF THE BUILDING
MARKET RESEARCH
EMPATHY
MARKET RESEARCH
BENEFITS, NOT FEATURES
THE MARKET
GAP IN THE MARKET
MARKET IN THE GAP
COMPETITIVE ANALYSIS
SIMPLE COMPETITIVE ANALYSIS‣ What do your customers perceive as the most important value? (benefits) ‣ Outline and research competitors. ‣ Identify the key points of differentiation. ‣ Compare your business agains these key points. ‣ Look for the gap. ‣ What else can you collect?
COMPETITIVE ANALYSIS
GREEN = GAP
MARKET SIZING
WHAT IS MARKET SIZING?ANALYSE THE SCALE OF OPPORTUNITYHIGHLIGHTS ASSUMPTIONS TO BE TESTEDCAN BE DONE WITH LITTLE TO NO RESEARCH, OR LOTS OF RESEARCH SOMETIMES DONE AS A “GUT” CHECK
40
MARKET SIZING
CONCESSION ORDERING APP FOR NBA GAMES
‣ Number of teams: 30
‣ Number of games per team: 82
‣ Total games = (30 x 82) / 2 = 1,230
‣ Average available seats in NBA stadium: 18,000
‣ Average capacity per game: 85%
‣ Average attendance at NBA game = (0.85 x 18,000) = 15,300
‣ Average spend per attendee: $100 ($80 ticket, $20 concession)
‣ Total spend per game = 100 x 15,300 = $ 1.5M
‣ Total for all games = 1.5M x 1,230 = $1.8B
BOTTOMS UP!!!!
How do we work out the potential market size for this APP? - $$
41
MARKET RESEARCH
CONCESSION ORDERING APP FOR NBA GAMES
‣ Number of teams: 30
‣ Number of games per team: 82
‣ Total games = (30 x 82) / 2 = 1,230
‣ Average available seats in NBA stadium: 18,000
‣ Average capacity per game: 85%
‣ Average attendance at NBA game = (0.85 x 18,000) = 15,300
‣ Average spend per attendee: $100 ($80 ticket, $20 concession)
‣ Total spend per game = 100 x 15,300 = $ 1.5M
‣ Total for all games = 1.5M x 1,230 = $1.8B
BOTTOMS UP!!!!
MARKET RESEARCH
MARKET SIZING - TOP DOWN & BOTTOM UP
Search Benchmarks Key Word Planner
PT 2 - STICKINESSDo people see the value, and are they going to pay you for it?Yes! - I has businessNo! - I need to pivot
LETS BUILD IT! OR NOT…
STICKINESS
STICKINESS
YOUR JOB IS NOT TO BUILD A PRODUCT, IT’S TO DE-RISK A
BUSINESS MODEL
PROCESS…EMPATHY
Problem Interviews
Design Test(s)
Interviews
Initial Wireframes
Interviews
Wireframe Variations
Interviews
MVP
YOUR HERE
Interviews
PRODUCT/MARKET FIT
TESTING AND VALIDATING YOUR IDEA
PRODUCT MARKET FIT
2.5% 13.5% 34% 34% 16%
PRODUCT MARKET FIT
PRODUCT/MARKET FIT
Build
Measure
Learn
PRODUCT/MARKET FIT
Iterate your way to Product/Market Fit
YOUR USERS
THE ROAD TO PRODUCT MARKET FIT
MVP 53
MVP
WHAT IS AN ASSUMPTION?
54
MVP
YOU LOVE A GOOD EXPERIMENT…‣Hypothesis or assumption for testing‣Method that is going to test the assumption‣Line in the sand - what does success look like?‣Run it‣Evaluation & decision
55
EXPERIMENT FRAMEWORKTHE LEAN CANVAS
EXPERIMENT & ASSUMPTIONS - NEW PRODUCT
MVP 57
DROPBOXProblem: It’s hard to manage your files across systems and computers.
Customer: People who work across multiple platforms. Ex. Office workers who need to access files at home.
Riskiest assumption: If we provide an extremely easy to use product, people will try it.
Experiment: Video demonstrating ease
MVP 58
EXPERIMENT & ASSUMPTIONS - EXISTING PRODUCT
MVP 59
AIRBNB
Improve the number of nights a property was rented.
Professional quality photos will increase number of nights
the place is booked for.
Hire a few photographers to professionally photograph
a few properties and compare booking rates.
Metric
Hypothesis
Experiment
MVP 60
RESULTMVP 61
MVP
YOU DO NOT NEED TO CODE TO TEST
SOMETHING.
62
NON CODE EXAMPLES MVP
‣ Fictitious case study ‣ PPC / Ad Click throughs‣ Email lists‣ Landing pages‣ Designs wireframes‣ None technical prototypes
Example
63
LOOK AT ALL THE SHINY TOOLSMVP
Mailchimp Unbounce Launch Rock Balsamiq AxureSketch
64
LUNCH TIME!
TESTING AND VALIDATING YOUR IDEA
MVPS
67MVPS
THE LEAST AMOUNT OF WORK YOU CAN DO TO
LEARN THE MOST OF SOMETHING.
MINIMAL VIABLE PRODUCT
MVPS
MVP
68
69MVPS
WHY MVP?
70MVPS
REDUCE RISK & MAXIMIZE SUCCESS FASTER FEEDBACK
REDUCED OVERHEAD MEASURABLE PROGRESS
WHY MVP?
MVP
CONCIERGE‣ Delivering a service
manually to the customer. ‣ Ex: Food on the table
71
MVPS
WIZARD OF OZ‣ Everything appears to be
real to the customer, but on the back end it is manual.
‣ Example: Zappos
MVPS
LANDING PAGE‣ Used to see how much interest there is in the idea. ‣ Remember: you need to drive traffic to the site!
MVPS
VIDEOS‣ Used to demonstrate a hard
concept. ‣ Can be pieced together without
the technology being completely in place.
‣ Ex: Dropbox
MVP
CROWD FUNDING‣ Estimate early demand ‣ Prove traction before building ‣ £££ before building ‣ Pebble / Air umbrella
75
EXERCISE
TIMING
DELIVERABLE
KEY OBJECTIVE(S)
ACTIVITY
Design your first experiment
1. Outline your riskiest Assumption
2. Design the minimum method to test it
3. Define your line in the sand
5 min
20 min
Blueprint for your first experiment
5 min
MVP
YOU LOVE A GOOD EXPERIMENT…‣Hypothesis or assumption for testing‣Method that is going to test the assumption‣Line in the sand - what does success look like?‣Run it‣Evaluation & decision
77
EXPERIMENT FRAMEWORKTHE LEAN CANVAS
SOLUTIONSMVP 79
ANALYSE THE RESULTS‣ Affinity mapping ‣ Identify trends ‣ Look for common goals/
themes ‣ Remember: People have no
idea what they want. It’s your job to figure it out.
SOLUTIONIZE 80
SOLUTIONIZE
BRIANSTORM POSSIBLE SOLUTIONS‣Aim for speed, not refinement‣Level of detail depends on stage in process‣Get out as many possible solutions as you can‣ If you are struggling for ideas you probably haven't understood the problem and the customer well enough‣All solutions are assumptions, so don’t be precious
1
5
2
6
3
7
4
8
EXERCISE
TIMING
DELIVERABLE
KEY OBJECTIVE(S)
ACTIVITY
Outline the problem and come up with 8 solutions
1. Reframe / crystallise the problem
2. Do Crazy 8’s to come up with 8 possible solutions
5 min
15 min
Your crazy 8 sheet completed
DECIDING WHAT GOES INTO AN MVPMVP
‣ Customer research‣ Create and test prototypes - Minimum test possible‣ Park everything that is not valuable‣ ONE THING!‣ DO ONE THING!‣ DO ONE THING WELL!‣ Backlog everything else‣ But… no seriously, NO!
83
USABILITY TESTINGMVP 84
MVP
STORYBOARDING & WIRE FRAMING
A PANEL OR SERIES OF PANELS OF ROUGH SKETCHES OUTLINING THE SEQUENCE OF EVENTS THAT A CUSTOMER WILL EXPERIENCE WHILE USING YOUR PRODUCT DURING A SPECIFIC ACTIVITY.
ANALYSIS
USABILITY TESTING
Invision Usability hubSilverback
Visit people & Pick up the phone - EVERY DAY
‣ 3-5 for 95% bugs‣ What do you want
to test‣ Ask to complete
specific actions‣ Observe and shut
up
MVP 87
INFORMATION MANAGEMENT
STICKINESS
WHAT IS A PRODUCT MANAGER?
TECHNOLOGYBUSINESS
DESIGN
Product Manager
Credit: http://www.mindtheproduct.com/2011/10/what-exactly-is-a-product-manager/
WHAT IS A PRODUCT MANAGER?
WHAT DO YOU DO ALL DAY?10:00am Coffee & email, put out any fires
11:00am Dev team standup to review progress on feature in development
12:00pm Lunch at desk, acceptance testing new features in staging
1:00pm Meeting with merchandising customers to review requirements
for new sales tools
2:00pm Writing user stories for sales tools & backlog management
3:00pm Meeting with CEO, CMO, Head of Product to review KPIs for the week on
launched projects
4:00pm Read through support emails and contact users for further questions
5:00pm Meet with UX Designer to explain requirements for sales tools
6:00pm Start creating experiments for new sales tools
INFORMATION IS YOUR MAP TO TREASURESTICKINESS
SCRUM / AGILEPROCESS
ARE WE THERE YET?MVP 93
FINDING PRODUCT/MARKET FIT
“THE ONLY METRICS THAT ENTREPRENEURS SHOULD INVEST ENERGY IN COLLECTING ARE THOSE THAT HELP THEM MAKE DECISIONS.”
Eric Ries
M FOR MEASURE THE RIGHT THING & METRICS
STICKINESS
ONE METRIC THAT MATTERS
STICKINESS
KPIS
WHAT IS A
KEY
PERFORMANCE INDICATOR?
*Chart by Wilson Joseph from The Noun Project
STICKINESS
A GOOD METRIC ISUnderstandable Comparative A rate or ratio Behaviour Changing
If you’re busyexplaining thedata, you won’tbe busy actingon it.
Comparison iscontext.
The only way to understand performance(MPH).
The information enables you to make a decision.Otherwise whats the point?
STICKINESS
CUSTOMERS WILL... VISIT & ARRIVE
INTERACT & ENJOY COME BACK
SPREAD THE WORD GENERATE REVENUE
STICKINESS
MEASURE YOUR CUSTOMER JOURNEY.
AARRRACQUISITIONACTIVATIONRETENTIONREFERRALREVENUE
STICKINESS
ACQUISITIONGETTING NEW PEOPLE TO YOUR PRODUCT.
WAYS TO CLASSIFY A NEW ACQUISITION?
WHERE ARE THESE PEOPLE COMING FROM (SOURCES)?
Spends at least 1 minute on a pageVisits at least 2 pagesClicks at least 2 things
Social MediaWord of Mouth
BlogsSEO
Paid Advertising
STICKINESS
SOURCE
CLICKSIMPRESSIONS
CTRCONVERSIONSAVERAGE CPC
COST PER CONVERSIONBOUNCE RATE
KEY TERMINOLOGYSTICKINESS
ACQUISITIONSTICKINESS
ACTIVATIONWHEN NEW USERS SIGN-UP, USE, AND ENJOY YOUR PRODUCT.
WAYS TO CLASSIFY A NEW ACTIVATION?
Signed-UpFilled Out a Form
Used Specific Features
STICKINESS
PAGES PER VISIT
TIME PER CONVERSION
TASK / ACTIVITY COMPLETIONTIME ON SITECLICKS PER CONVERSION
KEY TERMINOLOGYSTICKINESS
ACTIVATIONSTICKINESS
RETENTIONUSERS THAT CONTINUE USING YOUR PRODUCT OVER AND OVER AGAIN.
WAYS TO CLASSIFY A REPEAT USER?
How often do they come back?How often do they perform specific actions?
How long is each visit to your product?
STICKINESS
TIME SINCE LAST VISIT
EMAIL OPEN RATESESSION LENGTHUSER LOYALTY
KEY TERMINOLOGYSTICKINESS
CHURN RATE
RETENTIONSTICKINESS
REFERRALUSERS TELLING FRIENDS TO ALSO BECOME USERS OF YOUR PRODUCT.
WAYS TO ENCOURAGE & INCREASE REFERRALS?
Social Media FeaturesBuilding Virality In
Affiliates
STICKINESS
AFFILIATES & AFFILIATE NETWORKS
% OF USERS REFERRINGVIRALITY
PAID & ORGANIC GROWTH
KEY TERMINOLOGY
# OF REFERRED USERS
STICKINESS
REFERRALSTICKINESS
REVENUETHE METHOD AND AMOUNT OF MONEY YOU’RE MAKING
THINGS TO CONSIDER ABOUT REVENUE?
How many users are paying?How long did it take for them to make a
purchase?How much money did they spend?
STICKINESS
TIME TO FIRST TRANSACTION
COST PER ACQUISITIONSUBSCRIPTIONS / RECURRING TRANSACTIONS
BREAK-EVEN
KEY TERMINOLOGYSTICKINESS
LTV
REVENUESTICKINESS
MORE SHINY TOOLS, ITS ABOUT THE MIX…
GOOGLE ANALYTICS KISSMETRICS INTERCOM
ANALYSIS
MIXPANEL WOOPRA
USER FLOWSTICKINESS
FUNNEL ANALYSISSTICKINESS
COHORT ANALYSISSTICKINESS
NPSMEASUREMENT
HAVE YOU GOT PRODUCT MARKET FIT?STICKINESS
35% - retention rate2% - churn rate40% - would be hate life if you died
Benchmark what makes sense
PT 3 - SUMMARYHas it sunk in?And…Did we get to the bottom of your questions?
SUMMARY
Build
Measure
LearnEmpathy
Stickiness
Metrics & research
Q&APM BOOTCAMP
TOP STARTUP MISTAKES TO AVOIDSUMMARY
‣Building a business on assumptions‣Falling in love with your solution‣Trying to be something for everyone‣Not focusing on monetisation strategy - Unicorns aren't real‣Not looking after yourself‣Money management‣Single founder‣Poor team / culture