Become a Web 2.0 Jedi:How to Use the Force of Product & Marketing
Metrics to Optimize Your Business
Web 2.0 ExpoApr 3, 2009#w2e #metrics
Startup Metrics for Pirates:AARRR!
Web 2.0 Expo SFApril 2009
Dave McClure, Founders Fundhttp://www.foundersfund.com/
http://500hats.typepad.com/http://slideshare.net/dmc500hats/
Startup Metrics: The Basics
• Measure Stuff. Keep It Simple.• 5 Steps: Startup Metrics for Pirates (AARRR!)• 3 Metrics Frameworks + 3 Roles (CEO, Dev/Product, Marketing)• Iterate & Optimize. Feedback Loop. (Loop. Loop.)• One Step at a Time. The “Ass vs. Face” Issue.
Appendix (we won’t have time)• Activation• Retention• Acquisition• Referral• Revenue
Web 2.0: Hell Yes, Good Times.
1. # Users, Bandwidth = Bigger.2. Lower Startup Costs = Badder.3. PPC, E-Com $ Growing = Uncut.
Collect Usage Metrics in Real-Time Decisions Based on Measured User Behavior
The Startup Metrics Religion
• Progress ≠ Features (Less = More)• Focus on User Experience• Measure Conversion; Compare 2+ Options• Fast, Frequent Iteration (+ Feedback Loop)• Keep it Simple & Actionable
Optimize 4 Happiness (both User + Business)
• Define States of User + Business Value• Prioritize (Estimate) Relative Value of Each State• Move Users: Lower Value -> Higher Value• Optimize for User Happiness / Business $$$• Achieve Low Cost + High Value @ Scale
$$$
Discover MeaningKeywords, Images, Call-to-Action
Top 10 - 100 words• Your Brand / Products• Customer Needs / Benefits• Competitor’s Brand / Products• Semantic Equivalents• Misspellings
Relevant images• People• Products• Problems• Solutions
Call-to-Action• Words• Images• Context• Button/Link• Emotion
Result• Positive?• Negative?• Neutral (= Death)• A/B test & Iterate
Startup Metrics for Pirates: AARRR!
• Acquisition: users come to site from various channels• Activation: users enjoy 1st visit: "happy” experience• Retention: users come back, visit site multiple times• Referral: users like product enough to refer others• Revenue: users conduct some monetization behavior
AARRR!
(note: If you’re in a hurry, watch 5 min video after slide 23)
AARRR!: 5-Step Startup Metrics Model
Website.com
Revenue $$$
Biz DevAds, Lead Gen, Subscriptions, ECommerce
ACQUISITION
SEOSEM
Apps & Widgets
Affiliates
PR Biz Dev
Campaigns, Contests
Direct, Tel, TV
Social Networks
Blogs
Domains
Retention
Emails & Alerts
System Events & Time-based
Features
Blogs, RSS, News Feeds
Q: What’s My Business Model?
Can be one of the following:1. Get Users (= Acquisition, Referral)2. Drive Usage (= Activation, Retention)3. Make Money (= Revenue*)
* ideally profitable revenue
Note: eventually need to turn Users/Usage -> Money
Just Gimme the GOOD Metrics.Users, Pages, Clicks, Emails, $$$...?
Q: Which of these is best? How do you know?• 1,000,000 one-time, unregistered unique visitors• 500,000 visitors who view 2+ pages / stay 10+ sec• 200,000 visitors who clicked on a link or button• 20,000 registered users w/ email address• 2,000 passionate fans who refer 5+ users / mo.• 1,000 monthly subscribers @ $5/mo
the good stuff.
3 Core Models:biz model, conversion dashboard, mktg channels
• Define 1-Page Biz Model: customer segments + desired actions / behaviors• Identify critical Conversion Events & Dashboard for each segment & prioritize• Test & develop Marketing Channels; measure Volume (#), Cost ($), Conv (%)
Optimize product & marketing using Fast Iteration Cycles & A/B Testing
Role: Founder/CEOQ: Which Metrics? Why?A: Focus on Critical Few Actionable Metrics
(if you don’t use the metric to make a decision, it’s not actionable)
• Hypothesize Customer Lifecycle• Target ~3-5 Conversion Events (tip: Less = More)
• Test, Measure, Iterate to Improve
The 1-Page Business Model(Users + Conversions + Priorities)
Q1: What types of people use your website? • Visitor = Average User / Buyer• Contributor = Content Contributor / Seller• Distributor = Passionate Fan (unpaid) / Affiliate (paid)
Q2: What actions could they take to help you or them ?
TeachStreet 1-Page Business Model:Teachers & Students
Teachers StudentsActivati
on•Claim Profile
•Add Class•Contact Teacher
•View 3 PagesRetenti
on•Visit 1x/mo for 3
mo’s•Visit 1x/mo for 3
mo’s Referral •Request Review •Suggest
Teacher
Role: Product / Engineering
Q: What Features to Build? Why? When are you “Done”?A: Easy-to-Find, Fun/Useful, Unique Features that
Increase Conversion (stop iterating when increase decelerates)
• Wireframes = Conversion Steps• Measure, A/B Test, Iterate FAST (daily/weekly)• Optimize for Conversion Improvement
– 80% on existing feature optimization– 20% on new feature development
Example Conversion Dashboard(note: *not* actuals… your mileage may vary)Stage Conversion Status Conv.
%Est. Value
(*not* cost)
Acquisition Visitors -> Site/Widget/Landing Page(2+ pages, 10+ sec, 1+ clicks = don’t
abandon)
60% $.05
Activation “Happy” 1st Visit; Usage/Signup(clicks/time/pages, email/profile reg, feature
usage)
15% $.25
Retention Users Come Back; Multiple Visits(1-3x visits/mo; email/feed open rate / CTR)
5% $1
Referral Users Refer Others(cust sat >=8; viral K factor > 1; )
1% $5
Revenue Users Pay / Generate $$$(first txn, break-even, target profitability)
2% $50
Role: Marketing / SalesQ: What channels? Which users? Why?A: High Volume (#), Low Cost ($), High Conv (%)
• Design & Test Multiple Marketing Channels + Campaigns• Select & Focus on Best-Performing Channels & Themes• Optimize for conversion to target CTAs, not just site/landing page• Match/Drive channel cost to/below revenue potential
• Low-Hanging Fruit: – Blogs– SEO/SEM– Landing Pages– Automated Emails
Example Marketing Channels• PR• Contest• Biz Dev• Direct Marketing• Radio / TV / Print• Dedicated Sales• Telemarketing
• Email• SEO / SEM• Blogs / Bloggers• Viral / Referral• Affiliate / CPA• Widgets / Apps• LOLCats ;)
MAARRRketing PlanMarketing Plan = Target Customer Acquisition Channels
• 3 Important Factors = Volume (#), Cost ($), Conversion (%)• Measure conversion to target customer actions• Test audience segments, campaign themes, Call-To-Action (CTAs)
[Gradually] Match Channel Costs => Revenue Potential • Increase Vol. & Conversion, Decrease Cost, Optimize for Revenue Potential• Avg Txn Value (ATV), Ann Rev Per User (ARPU), Cust Lifetime Value (CLV)• Design channels that (eventually) cost <20-50% of target ATV, ARPU, CLV
Consider Costs, Scarce Resource Tradeoffs• Actual $ expenses• Marketing time & resources• Product/Engineering time & resources• Cashflow timing of expense vs. revenue, profit
One Step at a Time.
1. Make a Good Product: Activation & Retention2. Market the Product: Acquisition & Referral3. Make Money: Revenue & Profitability
“You probably can’t save your Ass and your Face at the same time… choose carefully.” – DMC
Summary
• Measure Stuff. Keep It Simple.• 5 Steps: Startup Metrics for Pirates (AARRR!)• 3 Items: Biz Model, Conversion, Mktg Channels• Iterate & Optimize. Feedback Loop. (Loop. Loop).• One Step at a Time.
Links & Resources
Additional References:
• “Influence: The Psychology of Persuasion”, Robert Cialdini (book)
• “Putting the Fun in Functional”, Amy Jo Kim (etech 2006 preso)
• “Futuristic Play”, Andrew Chen (blog)
• “Don’t Make Me Think”, Steve Krug (book)
• “Designing for the Social Web”, Joshua Porter (book, website)
• “Startup Lessons Learned”, Eric Ries (blog)
• “Customer Development Methodology” Steve Blank (presentation, blog)
• “Startup-Marketing.com”, Sean Ellis (blog)
Dan OlsenDan OlsenCEO, YourVersionCEO, YourVersion
Web 2.0 ExpoWeb 2.0 ExpoApr 3, 2009Apr 3, 2009
How to Use the Force of Metrics How to Use the Force of Metrics to Optimize your Web 2.0 Productto Optimize your Web 2.0 Product
The Force of Metrics is Real
The Force of Metrics? Really?
Copyright © 2009 Olsen Solutions LLCCopyright © 2009 Olsen Solutions LLC
Approaching Your Product and Approaching Your Product and Business as an Optimization ExerciseBusiness as an Optimization Exercise
Given reality as it exists today,Given reality as it exists today,optimize our resultsoptimize our resultssubject to our resource constraints.subject to our resource constraints.
Profit = Profit = RevenueRevenue - Cost - Cost
Unique VisitorsUnique Visitors x x Ad Revenue per VisitorAd Revenue per Visitor
Impressions/VisitorImpressions/Visitor x Effective CPM / 1000 x Effective CPM / 1000
Visits/Visitor x Pageviews/Visit x Impressions/PVVisits/Visitor x Pageviews/Visit x Impressions/PV
New VisitorsNew Visitors + Returning Visitors + Returning Visitors
Invited VisitorsInvited Visitors + Uninvited Visitors + Uninvited Visitors
# of Users Sending Invites x Invites Sent/User x Invite Conversion Rate# of Users Sending Invites x Invites Sent/User x Invite Conversion Rate
Copyright © 2009 Olsen Solutions LLCCopyright © 2009 Olsen Solutions LLC
Define the Equation of your BusinessDefine the Equation of your Business“Peeling the Onion”“Peeling the Onion”
Create ratios from primary metrics: X / YCreate ratios from primary metrics: X / Y Example: Example: How good is your registration page?How good is your registration page? Okay:Okay: # of registered users per day# of registered users per day Better:Better: registration conversion rate =registration conversion rate =
# registered users / # uniques to reg page# registered users / # uniques to reg page
Need third-party analytic tools PLUS home-grown metricsNeed third-party analytic tools PLUS home-grown metrics Track each metric as daily time seriesTrack each metric as daily time series
Copyright © 2009 Olsen Solutions LLCCopyright © 2009 Olsen Solutions LLC
How to Track Your MetricsHow to Track Your Metrics
DateDateUnique Unique VisitorsVisitors
Page Page viewsviews
Ad Ad RevenueRevenue
New User New User Sign-upsSign-ups ……
4/24/084/24/08 10,10010,100 29,60029,600 2525 490490
4/25/084/25/08 10,50010,500 27,10027,100 2424 480480
……
Copyright © 2009 Olsen Solutions LLCCopyright © 2009 Olsen Solutions LLC
Analyzing the Value of Product Ideas Importance and Satisfaction
To offer a good customer value proposition:Meet High Importance needs with High Satisfaction
98
8784
8679 847055 80
7280
75
4150
55
60
65
70
75
80
85
90
95
100
40 50 60 70 80 90 100
Satisfaction
Impo
rtan
ce
See Anthony Ulwick’s “What Customers Want”
BadBad
GreatGreat
Copyright © 2009 Olsen Solutions LLCCopyright © 2009 Olsen Solutions LLC
Prioritizing Product Ideas by ROIPrioritizing Product Ideas by ROI
Investment (developer-weeks)
Retu
rn (V
alue
Cre
ated
)
Idea C
Idea B
Idea D
Idea A
Idea F
1
1
2 3 4
2
3
4?
Marrying UI Design with Metrics Think of “Ease of use”
as distinct from the functionality
User Interface elements matter Position Layout Size Color Font Text copy Interaction design Navigation
Copyright © 2009 Olsen Solutions LLCCopyright © 2009 Olsen Solutions LLC
Improving Signup Page ConversionImproving Signup Page Conversion
Daily Signup Page Yield vs. TimeNew Registered Users divided by Unique Visitors to Signup Page
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10
Dai
ly S
ignu
p Pa
ge Y
ield
Changedmessaging
Added questionsto signup page
Started requiringregistration
Copyright © 2009 Olsen Solutions LLCCopyright © 2009 Olsen Solutions LLC
OpenAccount
Sign in
Account Selection
Register
56%
44%
Forget Password
Registration Process
45% drop off(20% of total)
36% overall drop off for
this step
70%(32% of Total)
17% drop off (10% of total)
20% drop off(6% of total)
30%(14% of Total)
80%(26% of Total)
55%(24% of Total)
64%of Total
Improving Sign In/Registration FlowImproving Sign In/Registration Flow
Change Password
83%(46% of Total)
Abandonment Rate (7 Day Moving Average)
0%
10%
20%
30%
40%
50%
60%
70%
80%
10/7
/02
10/1
4/02
10/2
1/02
10/2
8/02
11/4
/02
11/1
1/02
11/1
8/02
11/2
5/02
12/2
/02
12/9
/02
12/1
6/02
12/2
3/02
12/3
0/02
1/6/
03
1/13
/03
1/20
/03
Aba
ndon
men
t Rat
e (7
Day
Mov
ing
Ave
rage
)
Steps 1-2
Copyright © 2009 Olsen Solutions LLCCopyright © 2009 Olsen Solutions LLC
Redesigned User Flow Improved Redesigned User Flow Improved Registration Conversion RateRegistration Conversion Rate
37% improvement in conversion rate
ReleasedNew Design
Copyright © 2009 Olsen Solutions LLCCopyright © 2009 Olsen Solutions LLC
Optimization through Iteration:Optimization through Iteration:Continuous ImprovementContinuous Improvement
Measurethe metric
Analyzethe metric
Identify top opportunities
to improve
Design & develop the enhancement
Launch theenhancement
Learning
Gaining knowledge:
• Market
• Customer
• Domain
• Usability
Copyright © 2009 Olsen Solutions LLCCopyright © 2009 Olsen Solutions LLC
Summary: Using the Force of MetricsSummary: Using the Force of Metrics
Define what success meansDefine what success means Equation of your businessEquation of your business Customer value propositionCustomer value proposition
Instrument your site and track key metricsInstrument your site and track key metrics Identify opportunities and prioritize by ROIIdentify opportunities and prioritize by ROI Launch, learn, and iterateLaunch, learn, and iterate
[email protected]@yourversion.com@danolsen@danolsen
www.yourversion.comwww.yourversion.comMay the Force May the Force of Metrics be of Metrics be
with you!with you!
Dogster, Inc. Case StudyTed Rheingold
blog.dogster.com / @tedr
How actionable metrics informed us we were chasing our tail the wrong key results yet still guided us to the primary objective … bacon making money
Dogster, Inc.Sites & Services• Dogster.com, Catster.com, Snuzzy.com• Togethertag.com
5 year old profitable business. No VC, Angel onlyPrimary Revenue Source• Directly sold ad inventory
Secondary Sources• User subscriptions• Virtual currency• Together Tag (Physical dog & cat ID tags w/ always-on web
connecting service)
Marketing Initiative Feb ‘07Goal: 2,000 New Registrants a Day
Marketed sites via AdSense and BlogAdsAt launch we linked ads to our homepagesQuickly made a general landing page with intro info and big link to registerThen made dynamic landing with relevant text and registration form.This significantly increased percentage of ad clickers that became registrants
LANDING PAGE Welcome
Landing Page Cute Overload
LANDING PAGE Daily Candy
LADING PAGE Dog Info
REGISTRATION PAGE -Activate
Quantitative Success Doesn’t Mean Qualitative Success
We hit key result of 2k registrants a dayBUT only 60% were activatingOnly 8% we’re making profile pages vs. 80% that found site on their ownReal members were generating a $4 ARPU, these were worth $0.10
Actionable Results
Within 3 months we halted all ad spending.
We were sitting on a treasure trove of what topics engaged pet people the most.
We could drive a lot of uniques to our sites and advertisers love uniques
HOMEPAGE - Current
CRAZY EGG Searchers
Dogster Search Traffic
Ted Rheingold Top Dog, Dogster, Inc.
blog.dogster.comtwitter: @tedr
Web2Expo - 2009
Questions? #metrics@davemcclure @danolsen @tedr
Copyright © 2008 Olsen Solutions LLCCopyright © 2008 Olsen Solutions LLC
Kano Model: User Needs & SatisfactionUser SatisfactionUser Satisfaction
User DissatisfactionUser Dissatisfaction
Performance Performance (more is better)(more is better)
Delighter (wow)Delighter (wow)
NeedNeednot metnot met
NeedNeedfully metfully met
Must HaveMust Have
Needs & features Needs & features migrate over timemigrate over time
Copyright © 2008 Olsen Solutions LLCCopyright © 2008 Olsen Solutions LLC
Is the site up when I want to use it?
Is the site fast enough?
Does the functionality work?
Does the functionality meet my needs?
Olsen’s Hierarchy of Web User Needs(adapted from Maslow)
Customer’s Perspective What does it mean to us?
Uptime
Page Load Time
Absence of Bugs
Feature Set
Usability & Design
Decreasing
Dissatisfaction
Increasing S
atisfaction
How easy to use is it?
Copyright © 2008 Olsen Solutions LLCCopyright © 2008 Olsen Solutions LLC
Adding Metrics and Optimization to your Product Process
PlanPlan
DesignDesign
DevelopDevelop
BusinessObjectives
ProductObjectives
Prioritized Feature List
Scoping
Requirements & Design
Code Test Launch
Site Level
Feature Level
OptimizeOptimize Metrics & User Feedback
Copyright © 2008 Olsen Solutions LLCCopyright © 2008 Olsen Solutions LLC
Using Metrics to Optimize the Using Metrics to Optimize the Equation of your BusinessEquation of your Business
What are the metrics for your business?What are the metrics for your business? Where is current value for each metric? Where is current value for each metric? How many resources to “move” each metric?How many resources to “move” each metric?
Developer-hours, time, moneyDeveloper-hours, time, money Which metrics have highest ROI opportunities?Which metrics have highest ROI opportunities?
Retu
rnRe
turn
InvestmentInvestment
Retu
rnRe
turn
InvestmentInvestmentRe
turn
Retu
rnInvestmentInvestment
Metric AMetric AGood ROIGood ROI
Metric BMetric BBad ROIBad ROI
Metric CMetric CGreat ROIGreat ROI