david binetti, votizen

(c) 2009 David Binetti

The Sacred, The Profane, and the Pivot

A Brief Case Study of Applied CDM, AARRR Metrics, and Lean Principles

Monday, March 7, 2011


DefinitionsSacred

Steve Blank’s Customer Development Model (CDM). I follow it religiously.

ProfaneDave McClure swears every third word. Oh, and his Acquisition, Activation, Retention, Referral, Revenue (AARRR) model is awesome.

PivotEric Ries’s concept of strategic shifts predicated on learnings. (Sorry, no pithy witticism for this.)



CDM (Brief Version)

A) Establish Hypotheses

Test -> (Iterate or Exit)

B) Sell

Test -> (Iterate or Exit)



CDM (Discovery - Phase A)Category Hypothesis

Product State what you are selling.

Problem State the customer problem you are solving.

Channel State how you are going to sell your product.

Demand State how you are going to generate demand for your product.

Market Type State the market type (existing, re-segmented, new) you’re in.

Competitive State the competition you face.



CDM (Validation - Phase B)

I could explain in detail, but basically boils down to: prove you can sell by actually selling before you scale.



Metric Description

Acquisition Users visit

Activation Users sign up

Retention Users come back

Referral Users like and refer

Revenue User pay something for something

AARRR Model

(AKA, Metrics for Pirates)



AARRR Model (cont.)The Model measures:

Conversion rates for each step

Expected value at each step (not included here.)

A ‘one-slide pitch deck’.

Have these numbers, and you need nothing else.



Slight Differences

The CDM has you look at and test the whole business model.

The AARRR assumes the model rolls up in the metrics.



How They Complement

The CDM teaches you to test assumptions, but isn’t as specific on how to measure results.

The AARRR gives metrics to know what’s good, but not how to identify what’s broken.



Case: Votizen

Social Network for Registered Voters

Civic Participation Made Simple and Meaningful

Voter-Focused (as opposed to politician-focused) approach to $5B Political Market.

Independent, non-partisan, simple and fun!



CDM (Brief)Category Hypothesis

Product Social Network for Registered Voters (“LinkedIn for Politics”)

Problem Traditional campaign techniques #fail in a web 2.0 world.

Channel The internet is the primary source of voter information.

Demand Social networks are both the product and the demand-generator.

Market Type Resegmented; displace current market for direct-mail & robocalls.

Competitive Hyper-fragmented.



Metric Description

Acquisition Creates account on system. (Precursor-requirement)

Activation Certifies voting record (Authenticates - gives voter the power)

Referrals Forwards to friends (Acquisition model)

Retention Uses system to affect change (Validates lifetime value)

Revenue Private! (Sorry, not appropriate for a public document.)

AARRR (Brief)


AppliedStarted with MVP - Minimum Viable Product.

Just enough to get feedback (cost: $1206)

Metric MVP

Acquisition 5%

Activation 17%

Referrals -

Retention -

Revenue -



Improve

Tried a bunch of A/B testing on home page.

Found an optimal message.

Learning: When given the option to explore, people explored. When given a direction, they took the direction.

If you want something done, make it part of registration flow.


Applied

Significant, immediate jump in initial metrics. (Spend ~$10K)

Metric MVP V.1

Acquisition 5% 17%

Activation 17% 90%

Referrals - 4%

Retention - 5%

Revenue - -



Improve Retention and Referrals

Same Process, Different Results

A/B, Feedback, Analytics, etc.

Incremental improvement on retention and referrals.

Was improving, but marginal value of efforts were asymptotic.

Time for a pivot!



The PivotEric Ries concept: shift direction while grounded in learning

Implementation of “back to start of CDM”

I took learning and created new MVP.

So, $10K sunk cost -- buh-bye! (ugh)

Needed to fire current developers, hire new skills. (double-ugh)



The Pivot (cont.)

But, not a waste!

Time, money and effort brought learning.

Look beyond the features to the problem.

“I always wanted to get more involved. This makes it so much easier. Thanks!”


AppliedNew product launched as MVP - @2gov on Twitter

Contact all public officials at a single address.

Results? Much better R&R, and better conversion.

Metric MVP V.1 New ProductAcquisition 5% 17% 42%

Activation 17% 90% 83%

Referrals - 4% 54%

Retention - 5% 21%

Revenue - - -


AppliedNext step: test revenue assumptions

Provide a channel for voters to support causes and candidates in which they believe.

Another pivot! - No data yet

Metric MVP V.1 New Product

Next IterationAcquisition 5% 17% 42% ?

Activation 17% 90% 83% ?

Referrals - 4% 54% ?

Retention - 5% 21% ?

Revenue - - - ?



TakeawaysSet up your CDM hypotheses so that you know what you are trying to accomplish and how to tell if you are finished or need to start over.

The AARRR approach serves as a great ‘startup dashboard.’ Numbers don’t lie (as long as you don’t lie to yourself.)

Should you need to iterate, then pivot -- don’t flail. Measure success by lessons learned, not by the results themselves (oh, and restate CDM.)



Learn from the Source

CDM: http://www.steveblank.com

AARRR: http://www.slideshare.net/Startonomics/startup-metrics-for-pirates-presentation

Pivot: http://www.startuplessonslearned.com/2009/06/pivot-dont-jump-to-new-vision.html


http://www.steveblank.com

http://www.steveblank.com

http://www.slideshare.net/Startonomics/startup-metrics-for-pirates-presentation






http://www.startuplessonslearned.com/2009/06/pivot-dont-jump-to-new-vision.html







Feedback Wanted!

Would love feedback!

Want to get better, so criticism is preferred.

“If you don’t have anything bad to say, then don’t say anything.”

[email protected]


david binetti, votizen

Technology