pls structural equation modeling for customer satisfaction

Kai Kristensen, IMPS´03

PLS structural Equation Modeling for Customer Satisfaction

-Methodological and Application Issues-

Kai Kristensen, J. Eskildsen, H.J. Juhl, P. ØstergaardCentre for Corporate PerformanceThe Aarhus School of Business, Denmark

2


Agenda

• The EPSI Rating Model• Latent Structure• Manifests

• A few recent results• The Danish car market• External validity

• Practical problems and observations• The choice of scale• Reliability:The choice of manifests• Explanatory power• Missing values• Multicollinearity

• Some results from a simulation study

3


The EPSI Rating Model: Latent structure

Perceived Quality“Software”

Perceived Quality“Hardware”

Expectations

Image

LoyaltyPerceived Value Customer Satisfaction

4


EPSI Rating model

• Generic model with 7 latent constructs· 4 latent exogenous constructs (Image,

expectation, quality of ”hardware” and ”software)

· 3 endogenous constructs (perception of value, satisfaction and loyalty)

• Each construct is determined by 3-6 manifest measurements.

• The model is estimated by use of PLS (Partial Least Squares estimation techniques.

5


Examples of manifest measurements

• Image: General perception of company image with regard to:

• Reliability• Being customer

focussed• Giving value for money• Innovation in products

and services• Overall image

• Satisfaction:• Overall satisfaction• Comparison to ideal• Disconfirmation

• Loyalty:• The customer's intention to

repurchase, • Intention of cross buying

(buy another product from the same company),

• Intention to recommend the brand/company to other consumers.

6


Example from the Danish car industry

74

88

77 77

7275 7474

84

77 77

7274 73

50

55

60

65

70

75

80

85

90IM

AG

E

EX

PEC

TA

TIO

NS

QU

AL

ITY

OF

"HA

RD

WA

RE

"

QU

AL

ITY

OF

"SO

FTW

AR

E"

VA

LU

E

SAT

ISFA

CT

ION

LO

YA

LT

Y

Inde

x

2001 2002

7


Individual brands

7473

75

81

74

70

75

68

76

67

70

73

79

72

68

77

71

76

50

55

60

65

70

75

80

85

Peugeot VW Ford Toyota Opel Citroen Mazda Fiat Other

Inde

x

SATISFACTION 2001 SATISFACTION 2002

8


Inner coefficients for the 2002 model

UNSTANDARDISED INNER COEFFICIENTS

IMAGEEXPECTA-

TIONSQUALITY OF

"HARDWARE"

QUALITY OF "HUMAN WARE" VALUE

SATISFAC-TION LOYALTY

IMAGEEXPECTATIONS

QUALITY OF "HARDWARE"QUALITY OF "HUMAN WARE"

VALUE 0,36 -0,06 0,35 0,32 SATISFACTION 0,44 -0,03 0,23LOYALTY 0,29 -0,11 0,24 0,27 0,53

T-VALUES FOR INNER COEFFICIENTS

IMAGEEXPECTA-

TIONSQUALITY OF

"HARDWARE"

QUALITY OF "HUMAN WARE" VALUE


IMAGEEXPECTATIONS

QUALITY OF "HARDWARE"QUALITY OF "HUMAN WARE"

VALUE 14,72 -3,38 11,30 11,21 SATISFACTION 20,93 -1,80 9,06 10,82 5,54 LOYALTY 7,10 -4,08 5,22 6,10 13,84

9


The impact of drivers on satisfaction and loyalty 2001 & 2002

0,53

0,01

0,31

0,22

0,47

-0,03

0,26

0,29

0,57

0,01

0,33

0,33

0,54

-0,13

0,38

0,42

-0,2 -0,1 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7

Image

Expectations

Quality of "hardware"

Quality of "software"

Dri

vers

Impact

Satisfaction 2001 Satisfaction 2002 Loyalty 2001 Loyalty 2002

10


External validity: Relation to actual service performance

Other

VW

Toyota

Peugeot

Opel

Mazda

FordFiat

Citroen

y = -36,036x + 103,99R2 = 0,4554

70

72

74

76

78

80

82

84

60,0% 65,0% 70,0% 75,0% 80,0% 85,0% 90,0%

Percentage cars with defects

Inde

x on

"H

uman

war

e" in

200

2

11


External validity: Relationship between satisfaction and complaints

OtherFiat

Mazda

Citroen

Opel

Toyota

Ford

VW

Peugeot

y = 6E+17x-11,008

R2 = 0,643

0

0,0005

0,001

0,0015

0,002

0,0025

0,003

0,0035

0,004

65 67 69 71 73 75 77 79 81 83 85

Average satisfaction 2001 & 2002

Com

plai

nts:

Pro

port

ion

of p

opul

atio

n m

entio

ned

on se

lfrep

ortin

g ho

mep

age


Practical problems and observations

The Choice of Scale

13


The experiment

• In order to test the effect of scale choice on the results of customer satisfaction studies a controlled experiment was set up.

• Under totally identical conditions two samples were drawn from the population. The only difference between the samples was that in the first sample a 5-point scale was used and in the second a 10-point scale was used.

• The questionnaires were the standard customer satisfaction questionnaires used for a given company.

• The size of the samples was 545 for the 10-point scale and 563 for the 5-point scale.

14


Mean value of latent variables

Ten points Five pointsMean Mean

Expectations 73,3 75,1 0,13Products 64,2 64,3 0,88Service 66,9 66,4 0,70Value 54,4 54,4 0,96Satisfaction 65,2 65,2 0,97Loyalty 57,5 58,7 0,36Image 63,6 64,0 0,74

Data source

Significance, two sidedVariable

15


Conclusion: Mean values

•There is no significant difference between the mean values of the aggregate variables.

•This means that the choice of scale has no influence on the level of the customer satisfaction index or the loyalty index.

16


Standard deviation of aggregate variables

Ten points Five pointsStd Deviation Std Deviation

Expectations 19,2 20,1 0,476Products 19,1 20,5 0,274Service 21,2 23,4 0,014Value 19,7 22,4 0,005Satisfaction 19,3 21,5 0,013Loyalty 21,7 23,6 0,054Image 18,1 19,5 0,069

Data source

Variable Significance

17


Conclusion: Latent variable standard deviations

•As expected the standard deviation of the 10-point scale is smaller than the standard deviation of the 5-point scale with Image, Expectations and Products as possible exceptions.

•The difference is on the average app. 10%.•The reason for this difference is, that the underlying distributions are discrete.

18


5- and 10-point scales

Shape of the distribution

19


Satisfaction: Distribution 10 point scale

0 20 40 60 80 100Satisfaction

Mean: 65.2Std. dev.: 19.2

0.0280.072

0.250

0.439

0.213

20


Comparison of observed and theoretical distributions. (10-point scale)

0

0,1

0,2

0,3

0,4

0,5

-20 20-40 40-60 60-80 80-100

ObservedNormalBeta 10

21


Satisfaction: Distribution 5 point scale

0 20 40 60 80 100Satisfaction

0.0360.075

0.281

0.369

0.240Mean: 65.2Std. dev.: 21.4

22


Comparison of observed and theoretical distributions. (5-point scale)

0

0,1

0,2

0,3

0,4

-20 20-40 40-60 60-80 80-100

ObservedNormalBeta 5

23


A comparison of satisfaction distributions

05

1015202530354045

%

0-20 20-40 40-60 60-80 80-100

10-point5-point

24


Satisfaction: A general comparison of the distribution of 5- and 10-point scales

Kolmogorov-Smirnov Z Asymp. Sig. (2-tailed)

Image 1,08 0,19Expectations 2,53 0,00Products 1,41 0,04Service 1,63 0,01Value 1,95 0,00Satisfaction 1,77 0,00Loyalty 1,53 0,02

25


Conclusion

•In general the standardized distributions are not identical with Image as a possible exception. This is to be expected due to the discrete underlying distributions.

•The beta distribution or the doubly truncated normal distribution seem to give the closest approximation to the distribution but even here we have a significant difference in both cases.

26


5- and 10- point scales

Are demographics and scale interacting?

27


Variables and factors for the analysis of variance

•Dependent variables:•All aggregate variables

•Explanatory variables•Data Source (5-point, 10-point)•Age (-25, 26-35, 36-45, 46-55, 56-65. 66-)•Education ( 8 groups from high school to university)

•Gender (Male, Female)•Location (Copenhagen, Sealand, Funen, Jutland)

28


Analysis of variance (10% significance)

Image Expectation Product Service Value Satisfaction Loyalty

Main AgeLoca-tionEduca-tionGender

Location Location AgeLoca-tionEduca-tionGender

NONE AgeLocationEducation

Loca-tionEduca-tion

Two-wayinter-action

NONE Age x source

NONE NONE Age x source

NONE NONE

29


Conclusion

•In the case of Image, Product, Service, Satisfaction and Loyalty there is no effect from the data source.

•Only in the case of Expectation and Value we can trace an effect. In these cases there is a tendency that the age groups are using the scales differently.

•Based on this our general conclusion is, that the demographic interpretation of customer satisfaction studies will not be seriously affected by the choice of scale.

•When it comes to satisfaction there seems to be a universal main effect of Age, Location and Education.• Satisfaction is increasing with age and decreasing with

education.• Satisfaction is decreasing with the degree of urbanization.

30


Conclusions concerning scales

•In general terms a 10-point scale is preferable to a five point scale:•Smaller variance.•Closer approximation to a continuous variable.•10-point scales are used by all the major national customer satisfaction studies.

•In general it is possible to compare studies using 5 and 10-point scales since the mean values (on a 100-point scale) are not affected.

•Demographics have a small but not very important effect on the results from the scales.



Reliability and prediction

32


Internal reliability and validity of the results: The car example

IMAGEEXPECTA-

TIONS

QUALITY OF

"HARDWARE"

QUALITY OF

"HUMAN WARE" VALUE


R-SQUARE FOR LATENT VARIABLES 0,56 0,76 0,55COMPOSITE RELIABILITY 0,92 0,95 0,89 0,86 0,97 0,90 0,91AVERAGE VARIANCE EXPLAINED BY LATENT VARIABLES 0,69 0,86 0,73 0,68 0,91 0,75 0,84

R-SQUARE FOR LATENT VARIABLES 0,62 0,72 0,56COMPOSITE RELIABILITY 0,92 0,94 0,91 0,90 0,97 0,90 0,91AVERAGE VARIANCE EXPLAINED BY LATENT VARIABLES 0,71 0,84 0,77 0,75 0,91 0,74 0,84

2001

2002

( )=

=

λ=

λ + Θ

∑

∑

p2i

i 1p

2i i

i 1

AVE =

= =

⎛ ⎞λ⎜ ⎟

⎝ ⎠ρ =⎛ ⎞

λ + Θ⎜ ⎟⎝ ⎠

∑

∑ ∑

2p

ii 1

c 2p p

i ii 1 i 1

33


Reliability and choice of manifests

• Automobiles• Reasonable reliability: No reason for changes.

• Petrol stations• High reliability: No reason for changes.

• Banks:• In the satisfaction construct the “comparison to ideal” may cause a

problem. Much lower level than the two other questions.• Supermarkets

• In the satisfaction construct the “comparison to ideal” may cause a problem. Much lower level than the two other questions.

• The value for money indicator and the assortment indicator may cause a problem since they reflect the type of supermarket.

• The question about opening hours which is classified as belonging to the service block should possibly be re-classified

34


Reliability and choice of manifests: Conclusions

•For most of the areas covered by the Danish Customer Satisfaction Index the manifest questions are working well.

•The only area where we have observed a necessity for changes is Supermarkets.

•Other conclusions may apply when we have discussed the problem of missing values.

35


Explanatory power

•The general observation is, that the explanatory power of the model is rather good.

•There is no problem in obtaining an R2 beyond .65 for the satisfaction construct as required by the ECSI Technical Committee. In general R2 is somewhere between .70 and .80.

•The degree of explanation for value and loyalty is usually a little lower.



Missing values and multicollinearity

37


Missing values

Supermarkets Banks Automobiles Petrol Stations

•Below 10% for all items

•Relative comparisons are problematic. 40-50% missing values

•19 out of 22 items have missing values below 5%

•13 out of 22 have missing values below 10%.•8 have missing values between 10% and 20%

38


Multicollinearity (Latent variables)

•In general the degree of multicollinearity is rather high.

•Banks: Correlations between .54 (expectation and service) and .82 (product and service).

•Petrol stations: Correlations between .42 (expectations and service) and .69 (product and service).

•Automobiles: Correlations between .48 (expectations and service) and .85 (product and service).

•Mobile telephones: Correlations between .44 (expectations and service) and .76 (product and service).

•Supermarkets: Correlations between .52 (expectations and image) and .71 (image and product).


Simulation study

A study of some of the implications of the empirical findings

40


Background

• To get insight into the consequences of some of the empirical problems based on a true model which is very close to the actualmodels observed. Our model is reflective for all latent variables.

• To formulate some simple rules of thumb.• To supplement and verify the simulation study conducted by

Cassel, Hackl and Westlund (1999, 2000). These authors investigated the effect of the following factors on the estimation of an EPSI like model with formative exogenous and reflective endogenous latent variables:

• Skewness of manifest variables• Multicollinearity between latent variables• Misspecification (omission of relevant regressors or regressands, or

manifests within a measurement model)• Sample size• Size of the path coefficients

41


Simulation setup

• STAGE 1(Screening): Orthogonal main effect plan with 7 factors in 27 runs with 25 replications for each run. Each replication has a number of observations varying between 50 and 1000.

• Exogenous distribution (Beta vs. Normal)• Multicollinearity between latent exogenous variables• Indicator validity (bias)• Indicator reliability (standard deviation within a block)• Structural model specification error• Sample size• Number of indicators in each block

• STAGE 2: Full factorial design with 4 factors in 54 runs with 25replications for each run

• Multicollinearity, reliability, sample size and number of indicators

42


Stage 2 factor levels and response variables

•FACTOR LEVELS:•Multicollinearity: ρ={0.2;0.8}.•Reliability: σ={1; 10; 20}.•Sample size: n={50; 250; 1000}.•Number of indicators: p={2; 4; 6}.

•RESPONSE VARIABLES:•Absolute bias for indices•Standard deviation for indices•Bias for path coefficients•R2, AVE and RMSE.

43


The simulation model (A simplified customer satisfaction model)

4G3G

1G

2G

β =12 .50

γ =11 .50

γ =21 .25

γ =22 .75

1y

2yy21λ

y11λ

x

2x

3x

4x

x11λ

x21λ

x42λ

3y 4y

y32λ y

42λx32λ

1ζ

2ζ

y1ε

y2ε

y3ε

y4ε

x1ε

x2ε

x3ε

x4ε

12φ

122.5*Beta(4,3)

44


Simulation results

F1 F2 F3 F4Multicollinearity Indicator reliability Sample size # indicators F1*F2 F1*F3 F1*F4 F2*F3 F2*F4 F3*F4

g1 ** ** ** ** ** **g2 ** ** ** ** ** **g3 ** ** ** ** ** **g4 ** ** ** ** ** **stdg1 ** ** ** ** **stdg2 ** ** ** ** **stdg3 ** ** ** ** ** **stdg4 ** ** * ** ** **gamma21 ** ** ** ** * ** **gamma22 ** ** ** ** **gamma11 ** ** ** ** ** **beta12 ** ** ** ** ** **stdgam21 ** ** ** ** ** ** ** ** **stdgam22 ** ** ** ** ** ** ** ** **stdgam11 ** ** ** ** ** ** ** ** **stdbet12 ** ** ** ** ** ** ** ** **rsq1 ** ** ** **rsq2 ** ** ** ** **ave1 ** ** **ave2 ** ** *ave3 ** ** * *ave4 ** ** ** *avetot ** ** *rmse ** ** ** ** ** ** ** **

45


Bias of indices: Multicollinearity and indicator reliability

Multicollinearity

phi=0.8phi=0.2

Mea

n ab

solu

te b

ias

,80

,60

,40

,20

0,00

G1

G2

G3

G4

Indicator reliability

sigma=20sigma=10sigma=1

Mea

n ab

solu

te b

ias

,8

,6

,4

,2

0,0

G1

G2

G3

G4

46


Bias of indices: Sample size and number of indicators

Sample size

n=1000n=250n=50

Mea

n ab

solu

te b

ias

,8

,6

,4

,2

0,0

G1

G2

G3

G4

Number of indicators

642

Mea

n ab

solu

te b

ias

,8

,6

,4

,2

0,0

G1

G2

G3

G4

47


Example of mean relative bias for Gamma 21

Sample size

n=1000n=250n=50

Mea

n R

elat

ive

bias

Gam

ma2

1 (%

)

8,0

7,5

7,0

6,5

6,0

5,5

5,0

4,5

4,0


642

Mea

n R

elat

ive

bias

Gam

ma2

1 (%

)

8,0

7,5

7,0

6,5

6,0

5,5

5,0

4,5

4,0

48


Example of mean relative bias for Beta 12

Sample size

n=1000n=250n=50

Mea

n R

elat

ive

Bias

Bet

a12

(%)

-4,0

-5,0

-6,0

-7,0

-8,0

-9,0

-10,0

-11,0


642

Mea

n R

elat

ive

Bias

Bet

a12

(%)

-4

-5

-6

-7

-8

-9

-10

-11

49


Standard deviation of Gamma 21 as a function of multicollinearity and indicator reliability

Multicollinearity

phi=0.8phi=0.2

Mea

n St

d. d

ev. G

amm

a 21

,060

,040

,020

0,000



Mea

n St

d. d

ev. G

amm

a 21

,06

,04

,02

0,00

50


Standard deviation of Gamma 21 as a function of sample size and number of indicators

Sample size

n=1000n=250n=50

Mea

n St

d. d

ev. G

amm

a 21

,06

,04

,02

0,00


642

Mea

n St

d. d

ev. G

amm

a 21

,060

,040

,020

0,000

51


Degree of explanation.



Mea

n R

SQ

1,0

,9

,8

,7

,6

RSQ1

RSQ2


642

Mea

n R

SQ

1,00

,90

,80

,70

,60

RSQ1

RSQ2

52


Average variance extracted.


642

Mea

n AV

ETO

T

1,00

,90

,80

,70

,60



Mea

n AV

ETO

T

1,0

,9

,8

,7

,6

53


A couple of rough rules of thumb concerning the absolute bias of the indices

•Let σ be the standard deviation of the manifest variables, n the sample size, and p the number of indicators, then:

· BIAS(kσ,n,p) = k BIAS(σ,n,p).· BIAS(σ,kn,p) = (1/√k) BIAS(σ,n,p).· BIAS(σ,n,kp) = (1/k) BIAS(σ,n,p).

54


Conclusion to the simulation study

• Basically our results support The Cassel, Hackl, Westlund results where comparable:

• Misspecification is in general a serious problem with severe parameter bias.• Skewness of distribution is of minor importance to the PLS estimates.• Multicollinearity between the latent variables is without importance for the

estimated indices. It has a significant but small impact on the bias of the path coefficients. It has a significant effect on all standard deviations.

• Size of the sample has no influence on the bias of the path coefficients. It has a large effect on all standard deviations.

• In addition:• Indicator reliability has an enormous influence on all measured responses,

i.e. bias, standard deviation and fit measures. Furthermore several cases of two-factor interaction with both multicollinearity, sample size, and thenumber of indicators were found.

• Likewise the number of indicators has a strong impact on all responses, and also a strong two-factor interaction with sample size and reliability.

55


General conclusion

• PLS provides reasonably robust estimates of a customer satisfaction index in a usual practical setting where the sample size is n=250, the standard deviation around σ=20, and the average multicollinearity around ρ=.60.

• In a usual practical setting the bias of the indices is low and usually not larger than .50 (on a 100 point scale).

• The parameter estimates are in general biased. The bias can be both positive and negative depending on the model structure. Therelative bias will in a usual practical setting be in the area of 10-20%.

pls structural equation modeling for customer satisfaction

Documents