customer feedback: the missing piece of the agile puzzle

Customer FeedbackThe missing piece of the Agile Puzzle

June 22, 2012Agile Roots 2012

Salt Lake City, Utah

Who am I?

• This guy

• Maciej (“Ski”) Skierkowski• @skierkowski

Why are we here?

• We are learning and we are sharing• I learned a lot and I’ve got a lot more to learn

Why are we really here?

Building products is easy,building products customers want is hard.

• I will share a few…– basic rules– basic mechanisms

Rule #1: Your opinion is wrong

• Problem: you think you know what the customer wants, but you are probably wrong.

• Customers will use your product in a different way than you expected

• They might not use the product at all

Rule #1: Your opinion is wrong

• The litmus test– “I think customers need…” (ur doin it wrong)– “Customer [xyz] needs…” (ur doing it right)

Rule #1: Your opinion is wrong

• Solution: build a machine that validates (or rejects) your assumptions about customer needs

Rule #2: Don’t ask what people want

Problem: Customers ask for features. Your are in the business of solving problems, not building features; you need to identify the problems to solve, not the features to build.

Rule #2: Don’t ask what people want

Example 1Consider a feature request such as “GitHub should let me FTP up a documentation site for my project.” What this customer is really trying to say is “I want a simple way to publish content related to my project…” – Tom Preston-Werner from “Ten lessons from GitHub’s first year”

Rule #2: Don’t ask what people want

Example 2… it’s not just about features.

• Friend: “How much are you willing to pay for an iPhone”?

• Me: “No more than $250”• (3 months later)• Me: “I just bought an iPhone for $450”

Rule #2: Don’t ask what people want

Example 3…Customers: “we need custom SSL”

• That was a no-brainer, but we had more questions:– How much are people willing to pay for it?– How much demand is there?– Do they need to update certificates?

Rule #2: Don’t ask what people want

Solution: when someone asks for a feature, ask “why”, and do it at least 3 times

Rule #3: Fake it till you make it

Rule #3: Fake it till you make it• Problem: If you build it first…– Will people use it?– Will they pay for it?– How much are they willing to pay for it?

• Solution…• First build a façade to allow users to express

interest• Once you have traction/evidence, then build

the real thing

Rule #4: Make it cheap, throw it out

• Problem: you invest in building a product but nobody wants it

• Don’t get too attached• Take shortcuts• Build only what you need. Cutting is king.• Throw it out if people aren’t using it; it’s just

dead code.

Rule #5: Scalability is not a problem, make it one

• Problem: Engineers want to make it scalable… but nobody is using it

• The real problem is getting people to use your product and to pay for it. Scalability is just premature optimization.

• Scalability is a great problem to have.• (e.g. PHP Fog hitting 2^15-1 apps)

• Keep an eye out for “quality” too.

Rule #6: Overcommit and deliver

• Keep resources fixed, keep schedule fixed, and keep the functionality

• Leverage creativity and 3rd party services to deliver without compromise.– Pusher, Loggly, Airbrake, Sendgrid, Mailchimp,

Google Forms, GitHub, RedisToGo, …

Rule #7: Don’t bother with feature lists

• If you can’t remember it, it’s probably not worth building

• The things you need to build are the ones your customers won’t shut-up about.

• If you have a big backlog/icebox… delete it. Nobody will miss it.

Rule #8: Not all users are customers

• Problem: Listening to the wrong customer will lead you to solving the wrong problems.

• Learn why customers use your product• Learn to identify users who are your target

customers

Recap of the rules

1. Your opinion is always wrong2. Don’t ask what people want3. Fake it till you make it4. Make it cheap, throw it out5. Scalability is not a problem, make it one6. Overcommit and deliver7. Don’t bother with feature lists8. Not all users are customers

Rules => Mechanisms

• Rules give us structure• Product team delivers value to customers• Customers tell us what is valuable… but how?

Outstanding Support

• Everyone provides support– Executive team (CEO, Director)– Developers– Support team

• A business differentiator• Very easy to identify problems for existing

customers

Net Promoter Score

• The survey:– [0 to 10] Would you recommend this service?– Why?

• Powerful free response• Score doesn’t tell you much, but lots of great

candid feedback about product• Identifies with both existing and lost

customers

Live Chat

• “Hi, anyone there”• Makes the company personal• Immediate gratification and solutions for

support issues

Interviews

• Every week interview a customer for 15-30 minutes over the phone or over coffee

• Common questions: Name, Job, story, their business, what they love/hate, etc.

• Special question: Ask for advice• Share info with the company• Helps make the customer a promoters• Makes customers the advisors

Surveys

• Use sparingly• “Data driven decision making” or “decision

driven data making” (hint: it’s the latter)• Great for targeted data (e.g. marketing

material)• Terrible for testing your value proposition

Usability study

• Don’t make this formal• Make it cheap and easy• Hire a good designer/usability person• Everybody focuses on usability• KEEP IT SIMPLE!!

Discussion Forums

• http://community.phpfog.com/• Surprised at engagement• Customers helping other customers• Great way to engage with groups of

passionate people

Third party services

• Customer information is everywhere• Services we monitor and engage– Quora– Blogs and comments– Stack overflow– Google Alerts– Hacker News

Workflow analytics

• Tools– Google Analytics– KISS Metrics for flow measurement

• Great for optimization• Bad for testing hypothesis

Retention Emails

• “I noticed you haven’t logged in over the past 2 weeks”

• You got the user over the biggest hurdle (getting their attention) but never got them to the finish line (converting them to a customer).

List of mechanisms

• Outstanding support• Net Promoter Score (NPS)• Live Chat• Interviews• Surveys• Usability studies• Discussion forums• Third party services• Workflow analysis• Retention emails

Thank You

• @Skierkowski

• http://blog.skierkowski.com/