nps mrsnz conference paper 2007

© Research Solutions – October 2007 1

Net Promoter Score ~ Answer to our research prayers or curse of the industry?

NNPPSS

How many of you have heard of the Net Promoter Score?



IntroductionIntroduction

Case StudiesCase Studies

ConclusionsConclusions

AA

BB

CC



IntroductionIntroduction

Case StudiesCase Studies

ConclusionsConclusions


Customer SatisfactionCustomer Satisfaction

Customer EngagementCustomer Engagement


TRI*MTRI*M Gallup C11Gallup C11


CustomerCustomerSymmetrySymmetry

Conversion Conversion ModelModel

CB WorldCB World

ImplicitImplicit


PreferencePreference

Share of WalletShare of Wallet

ChurnChurn


Net Promoter Score (NPS)

Just another loyalty measure?

The Ultimate Research Solution?

OR

Well, let’s have a look, shall we?

And in case you wondered, this is not a copy of Mark Molenaar’s winning SAMRA conference paper Would you recommend this paper to your friends?.


Source: Satmetrix

In Search of the Ultimate Question

What is the ultimate question?

The ONE that truly helps our clients to understand where their brand sits and WHAT they should do to either maintain or change that position. Basically an easy predictor for future growth.


“Would you recommend brand X to a friend?”

Source: Satmetrix

0

1

2

3

4

5

6

7

8

9

10

0

1

2

3

4

5

6

7

8

9

10

0

1

2

3

4

5

6

7

8

9

10

The net promoter score is a calculated metricNPS= (% rating 9 or 10) – (% rating 0 to 6)

Let’s look at NPS.

The principle is quite simple really – “would you recommend brand X to a friend on a 0-10 scale?”- Those rating this question 9-10 are called promoters.- Those rating this question 0-6 are called detractors.- Those rating this question 7-8 are called passive.

Plus, NPS asks respondents “what is your primary reason for this rating?” to gain insights into their attitudinal motives.


In Search of the Ultimate Question

• The Net Promoter Score was introduced in December 2003 by noted Bain loyalty consultant Frederick Reichheld

• Bain consultancy claims that the NPS is the “one number you need to grow” - the ultimate question

• Reichheld states that compared with other survey-based questions asked of customers, this score is the most effective across industries at predicting a firm's growthand has successfully moved a wide range of service companies towards the use of the metric– Because customers will have to make a personal sacrifice by risking

their reputation in making a recommendation to someone else

Basically, if more people are champions of your service or product (promoters) than are neutral or detractors of it, your company will grow and vice versa.

Reichheld’s claim is backed up by research conducted amongst over 400 companies in more than a dozen industries. Well, actually just over 50 companies were included in the actual analysis due to certain criteria they had to meet.

However, the results are impressive:- NPS leaders are said to outgrow their competitors in most industries by an average of 2.5 times;- And it’s reported that a 12-point increase in NPS leads to the doubling of a company’s growth rate on average.


“The one number you need to grow.”(Reichheld)

Source: Satmetrix

High NPS =high growth rate

Low NPS =low growth rate

I’m not a statistician, but remember that correlation analysis does NOT imply cause and effect.

It is interesting to note that Reichheld arrived at the “recommend to a friend”question by testing a batch of around 20 customer satisfaction and retention questions within a sample of customers across a range of products and industries, and then correlating their responses with future purchase or referral behaviour. The “recommend to a friend” question was found to have the best correlation, out of the 20 questions tested, 80% of the time (Satmetrix Systems White Paper, 2004).


Achieving success with Net Promoter requires much more than simply calculating your NPS

A successful Net Promoter Discipline includes 5 elements:1. Metrics proven to link to growth, i.e. NPS

2. Leadership practices that instill customer focus, passion and values

3. Organizational strategies to ensure processes

4. Integration with core business processes

5. Operational systems to support the initiative

Overall, a commitment to building customer-centric focus into the business

Source: Satmetrix

Sounds too good to be true? And all this for free? Well, after all, there is no licence fee involved.

Well, that’s not entirely true. It has a number of commercial implications for Bain & Company and Satmetrix:• There’s a book – The Ultimate Question (Reichheld 2006);• A website;• International conferences and more.

And, of course, to gain full potential of NPS and its implications one should use Bain & Company and Satmetrix for their specialised consulting.


Let’s look at the Net Promoter Discipline Roadmap

Source: Satmetrix

Net PromoterFast Start Edition™

Baseline Performance

Net Promoter Standard/Enterprise Edition™

Standard Products

Role-based Reporting

Operationalize

Supporting Transactional Systems

Expand

Now

Future

Gain a basic understanding of the Net Promoter dynamics within your business using a staged approach

Deploy an operational system;move from measurement tomanagement

Implement account and functional-level action plans; incorporatecustomer feedback in decisions

Support insight with additional customer touch-point analysis

Does this roadmap look familiar to you? Not that much different from other consultancy and/or research programmes.


Some Quotations of Senior Corporate Executives….

The last step is how we measure success, and this is something we’ve taken across the company called Net

Promoter Score…. We’re doing it in commercial finance, consumer finance, healthcare, NBCU, every business….

[I]t’s not just a kind of metric for the sake of having a metric. It gets embedded into the culture in terms of how we

interface with customers. (Jeffrey Immelt [2005], Chairman and Chief Executive Officer, General Electric)

We will focus our organisation on what we call Net Promoter score, which

goes much beyond the pure customer satisfaction index. (René Obermann

[2005], Chief Executive Officer, T-Mobile International AG)

So what’s driving growth?... [W]e measure the customers experience using a system called Net Promoter. The

higher the score, the more customers are delighted with the product and service experience and would recommend it to a friend… [I]t’s all about Net Promoter. (Steve Bennett

[2005], President and Chief Executive Officer, Intuit)

The average U.S. company has net promoter scores of between 5 and 10.

We, on early reads, believe that we have scores sort of in the 40-50 range, which certainly would be a point that [is]

quite acceptable. (Glenn Renwick [2005], President and Chief Executive Officer, The Progressive Corporation) All companies should ask their customers what Fred

[Reichheld] calls “the ultimate question”. (Ken Chenault [quoted in Reichheld 2006c], Chairman and Chief Executive

Officer, American Express)

Source: Keiningham, Cooil, Andreassen, and Aksoy Paper in the Journal of Marketing, July 2007

Quite powerful stuff, isn’t it? But understandably so. Simple one-number metrics are popular in the boardroom, where life is too short to have to learn how to interpret complex measures.

And it’s not just the Americans who love it. In particular when you do research for a global company, you will ultimately be asked about using NPS.

But you might say, what about those who simply don’t recommend something? Some cultures don’t do it at all, e.g. Asia; others wouldn’t feel comfortable recommending particular things, say in regards to their financial affairs, etc.


Let’s compare NPS with other loyalty measures ~ Case Studies from around the world

1) U.S. ~ American Customer Satisfaction Index (ACSI)

2) Norway ~ Norwegian Customer Satisfaction Barometer

3) Switzerland ~ Customer Commitment

4) New Zealand ~ Customer Delight Index, Preference, Churn

5) New Zealand ~ Share of Wallet, Customer Satisfaction, Customer Commitment

According to Reichheld the ‘net promoter’ score is independent of other measures such as satisfaction and loyalty. However, it is virtually impossible to imagine a scenario in which the results of customers’ overall satisfaction levels, their likelihood to repurchase and their likelihood to recommend the firm were not highly correlated. And a multi-industry study conducted by TNS Global shows that satisfaction scores from 1-10 and NPS scores from 0-10 highly correlate indeed – with R squares of around 0.97 or higher.


American Customer Satisfaction Index

#1

The following 3 case studies were in fact part of Reichheld’s original study in 2003, which led to his claim that NPS is a better predictor of future growth than ACSI.

The study looks at 3-year average growth rates (1999-2002) and 2-year average NPS (2001-2002) – note that this means that Net Promoter Score was correlated to past growth rates, NOT future growth rates.

So actually, Reichheld’s original study does not help to identify whether NPS levels are linked to current or future changes in revenue growth. And remember cause and effect.


Compared to the American Customer Satisfaction Index, it seems that NPS is more highly correlated to 3-year growth for life

insurances (for which there is comparable data)


Net Promoter Score American Customer Satisfaction Index


But ASCI is slightly more highly correlated to 3-year growth for personal computer manufacturers

(for which there is comparable data) than NPS is




And ASCI is much more highly correlated to 3-year growth forairlines (for which there is comparable data) than NPS is





These 3 charts of industries examined were presented in the Harvard Business Review

article that introduced Net Promoter.

Based on this data, it’s hard to believe that NPS is so much more effective than ASCI.


Norwegian Customer Satisfaction Barometer

#2

This is based on a study by Keiningham, Cooil, Andreassen, and Aksoy in 2007, who have done 2 things:

• Replicated the original Bain research vs. ASCI as shown before;• Used NCSB data to measure correlations of a variety of commonly used

satisfaction/loyalty metrics.

The Norwegian Customer Satisfaction Barometer was created in 1994-1995 to uncover the degree of satisfaction with various services by interviewing defined companies’ existing customers. Around 15,000 plus customers (representing 21 firms of which 17 where used for the following correlations) were included in this study.


Replicating NPS analysis and looking at 5 industries (representing 17 companies) within the NCSB reveals that…


NPS is ahead, but no strong correlation to any measure

in Banking…

Note:• Correlation is shown as r, not r2;

• P-value = probability value, which checks whether results occur by chance or not = the lower the score, the better.



Satisfaction (top 2 boxes) is the winner

in Gasoline…



Satisfaction (top box) is the winner

in Home Furnishing…



Recommendation (mean and top 2 boxes) is the winner

in Security Systems…



No correlation whatsoever

in Transportation…


Overall, none of these loyalty/satisfaction measures are significantly correlated to

revenue!!



Customer Commitment

#3

This study was conducted in the Insurance Market in Switzerland in 2006, using the Conversion Model, a proven and validated predictor of future behaviour (both amongst customers and non-customers).

The Conversion Model incorporates…• Satisfaction with current choice and favourability to other choices;• Involvement, i.e. how important is this;• Attraction of other alternatives;• Ambivalence between alternatives.

The Conversion Model classifies users as committed and uncommitted. One would assume that these segments are somewhat similar to the NPS segments, e.g. committed users are similar to promoters.


Conversion Model study in Switzerland showed little overlap between CM and NPS subgroups

Source: Stefan Ruf, Trustmark CFI presented at Ignite 2007

25%9%6%Uncommitted

11%20%27%Committed

Detractor‚Middle'Promoter

Not used for NPS

Net Promoter Segments

CM

Seg

men

ts


And their mindsets are significantly different ~


0 20 40 60 80 100

Attribute 1Attribute 2............................................................

Promoters, Committed

Do detractors look like that?

0 20 40 60 80 100

Detractors, Committed


Brand XBest Other Brand

0 20 40 60 80 100

Even ‚Middle‘ (not classified by NPS) is correct: CM Committed


These people are committed…

All committed customers clearly prefer brand X, but some simply don’t recommend this brand (or maybe any brands in this industry).


And their mindsets are significantly different ~These people are UNcommitted…


Brand XBest Other Brand

0 20 40 60 80 100

Detractors, Uncommitted


0 20 40 60 80 100

Promoters, Uncommitted

Do real promoters look like that?


One would expect promoters to be much more positive to their brand than shown above. After all, they intend to recommend this brand. But will they really recommend or are they just saying they will? Remember, they are actually uncommitted.


Does NPS segment the wrong people?They might be promoters or detractors, but

their mindsets and attitudes give them away as committed or uncommitted.



Customer Delight Index, Preference, Churn

#4

This case study looks into a market where we have 2 extremely strong competitors who basically share the market amongst them.

We look into correlation between the customer delight index and preference, churn and NPS. The CDI score is based on three variables – overall satisfaction, satisfaction vs. expectations, satisfaction vs. ideal.


Some correlation between CDI and NPS~ Consumer tracking study in New Zealand

-20%

-15%

-10%

-5%

0%

5%

10%

15%

20%

Apr 04/

May 04

Jun 04

/July

04

Aug 04/Se

p 04

Oct 04/

Nov 04

Dec 04/Jan 05

Feb 05

/Mar 0

5

Apr 05/

May 05

Jun 05

/July

05

Aug 05/Se

p 05

Oct 05/

Nov 05

Dec 05/Jan 06

Feb 06

/Mar 0

6

Apr 06/

May 06

Jun 06

/July 06

Aug 06/Se

p 06

Oct 06/

Nov 06

Dec 06/Jan 07

% v

aria

tion

vs a

vera

ge o

ver p

erio

d

-100%-80%-60%-40%-20%0%20%40%60%80%100%

NPS

(muc

h gr

eate

r var

iatio

n th

an o

ther

var

iabl

es)

CDIChurnPreferenceNPS

• CDI does appear to correlate (though not strongly) with both NPS and preference, with R-squared figures of…

– .42 for CDI vs. NPS– .55 for CDI vs. preference for Brand X

• CDI is NOT strongly correlated to churn, with R-squared of…– .10 for CDI vs. churn

Consumer sample

© Research Solutions

The chart shows VARIATION from the mean over the period for each measure, as a percentage of that mean. NPS has a much greater variation over time than other variables. That is, in August/September 2005, the NPS was almost DOUBLE the average.


But it’s a different story amongst the Business sample…

• CDI does NOT appear to correlate with any of these three measures, with R-squared figures of…

– .13 for CDI vs. churn– .12 for CDI vs. NPS– .03 for CDI vs. preference for Brand X

Business sample


-25%-20%-15%-10%

-5%0%5%

10%15%20%25%

Jun 04

/July

04

Aug 04/Se

p 04

Oct 04/

Nov 04

Dec 04/Ja

n 05

Feb 05

/Mar 0

5

Apr 05/

May 05

Jun 05

/July

05

Aug 05/Se

p 05

Oct 05/

Nov 05

Dec 05/Ja

n 06

Feb 06

/Mar 0

6

Apr 06/

May 06

Jun 06

/July

06

Aug 06/Se

p 06

Oct 06/

Nov 06

Dec 06/Ja

n 07

Feb 07

/Mar 0

7

% v

aria

tion

vs a

vera

ge o

ver p

erio

d

-400%

-300%

-200%

-100%

0%

100%

200%

300%

400%

NPS

(var

iatio

n is

MUC

H la

rger

)

CDIChurnPreferenceNPS

Chart shows VARIATION from the mean over the period for each measure, as a percentage of that mean. That is, in August/September 2004, the NPS was THREE TIMES the average.


Some correlation to Customer Delight amongst Consumers, but not amongst

Businesses.But wouldn’t we expect promoters to be

delighted as well?© Research Solutions


Share of Wallet, Customer Satisfaction, Customer Commitment#5

This client operates in 4 distinctly different markets:

- Market 1: Client is clearly the market leader with 1 or 2 other competitors;- Market 2: Shared market leader in an extremely price-driven market;- Market 3: Market leader in a very small market with little competition;- Market 4: Small niche competitor in a very big market with numerous competitors.


Does NPS deliver share of wallet?


90%

69% 76%

38%

87%

63%73%

39%

80%

51%66%

34%

0%

20%

40%

60%

80%

100%

Market 1 Market 2 Market 3 Market 4

% o

f usa

ge in

the

past

yea

r tha

t wer

e w

ith B

rand

X

Promoters Neither Detractors

To some extent – moving from detractor to promoter adds ~10% (except in Market 4)

The objective for this analysis was to provide a view of the value of increasing NPS scores, in terms of numbers of additional sales to be gained (and hence, incremental revenue).

Share of wallet is based on asking respondents what percentage of their current spend is with each of the brands in the market.


Does SERVICE deliver share of wallet?More strongly in Market 2, but not at all in Market 4!


87%

30% 36%

74%

42%

68%

38%

84%

57%64%

35%

88%

65% 70%

38%

89%

67%80%

37%

0%

20%

40%

60%

80%

100%


% o

f usa

ge in

the

past

yea

r tha

t wer

e w

ith B

rand

X

1 - Poor 2 3 - Good 4 5 - Excellent

Rating for overall service experience

Even in an extremely price-driven market service delivery is very important – see Market 2.


But we found that COMMITMENT is a much stronger predictor of share of wallet


93%77%

53%67%

90%74% 74%

86%86%

50%36%

20%

65%

33% 31% 38%46%

30% 27%14%

0%

20%

40%

60%

80%

100%


% o

f usa

ge in

the

past

yea

r tha

t wer

e w

ith B

rand

X

Committed to Brand X Uncommitted because they're unhappyShared - Brand X & other brand(s) Casual - don't care who they useCommitted to a competitor

And commitment is a proven predictor of future behaviour.


What happens if we shift NPS scores in different markets?

Market 11.Moving rest into promoters

– For every 1% move in NPS we estimate an extra 0.34 sales per 1000 currently sold – i.e. One extra sale per 3000 currently

2.Moving detractors into rest– For every 1% move in NPS

we estimate an extra 1.06 sales per 1000 currently sold – i.e. One extra sale per 950 currently

NPS can be increased by either moving people from passive into promotersOR moving people from detractors into passive

Estimates based only on best data available ~ Sample not big enough for calculations in Market 3 © Research Solutions


– For every 1% move in NPS we estimate an extra 1.11 sales per 1000 currently sold – i.e. One extra sale per 900 currently




– For every 1% move in NPS we estimate no gain at all –currently the rest give us a higher share of wallet than the advocates do!



Note that the sample in Market 3 was not big enough to accurately calculate share shifts.

For the other 3 markets we did some calculations based on the % of promoters, detractors and passives in each market and also their current share of wallet.


Overall, very little gain by increasing the NPS scores.Highest potential is in more competitive markets, with

fewer market barriers.BUT data suggests that improving the overall service

rating would have more impact in Market 2 than improving the NPS.



So is it really the Ultimate Question?

Source: Morgan/Rego when cited by Stefan Ruf at Ignite 2007

Morgan and Rego did some longitudinal analysis in 2006 based on data used from ACSI. Their results show that NPS does not have any value in predicting future business performance.


“Would you recommend NPS to a client as the ultimate measure of

growth?”

0

1

2

3

4

5

6

7

8

9

10

0

1

2

3

4

5

6

7

8

9

10

0

1

2

3

4

5

6

7

8

9

10


You make up your own mind, but in summary…

• NPS is a simple, easy to understand loyalty measure– And it delivers ONE number

• BUT it’s only one dimensional and not a measure that incorporates competitor situation, ratings or attractions

– And therefore is NOT a true brand measure

• And to call it the “one number you need to grow”, the most effective across industries at predicting a firm's growth, is hard to comprehend

Molenaar’s SAMRA conference paper concludes that the “recommend to a friend” question is literally measuring the same dimension as customer satisfaction. As researchers we know, however, that customer satisfaction is a poor predictor of future behaviour. So why would we want to go back to the Stone Age of market research?

True brand measures take into account that respondents can have a relationship with more than one brand, and therefore may recommend more than one brand (or none).


Discuss why they want to include it and what outcomes they expect.

Ask the recommended question in parallel with your current accepted loyalty measures.

Analyse your results, show correlations, discuss strength and limitations.

So what if your clients ask you to use NPS?


Thank you!

Horst Feldhaeuser, Research Solutions, [email protected]

NNPPSS

References:Keiningham T.L., Cooil B., Andreassen T.W., Aksoy L. 2007. A longitudinal examination of ‘Net Promoter’ on firm growth. Journal of Marketing. Vol. 71 (July 207), 39-51.Molenaar M. 2007. Would you recommend this paper to your friends? Proceedings of the SAMRA Convention, Cape Town, South Africa.Morgan N.A. & Leotte Rego L. 2006. The value of different customer satisfaction and loyalty metrics in predicting business performance. Marketing Science 25 (5): 426-439. Reichheld F.F. 2003. The one number you need to grow. Harvard Business Review - On Point Article, Dec 2003.Reichheld F. 2006. The ultimate question: Driving good profits and true growth. Harvard Business School Press.Ruf S. 2007. Commitment and the Net Promoter score. A comparison: Con-version Model™ (CM) vs. Net Promoter Score (NPS). Conversion Model™Conference, Cape Town, South Africa. Satmetrix. 2006. Getting Started With The Net Promoter®. Satmetrix Net Promoter Fast Start Edition™, www.satmetrix.com.Tronchin E. 2007. Net Promoter score: The one number he needed to grow to promote his career. TNS Customer Equity Company Position Paper, 13 June 2007.