Effects of unequal indicator intercepts on manifest composite differences
Holger Steinmetz and Peter Schmidt
University of Giessen / Germany
Introduction
Importance of analyses of mean differences
For instance:- gender differences on wellbeing, self-esteem, abilities, behavior- differences between leaders and non-leaders on intelligence and personality traits
- differences between cultural populations on psychological competencies, values, wellbeing
Usual procedure: t-test or ANOVA with manifest composite scores
Latent variables vs. manifest variables
Manifest mean = indicator intercept + factor loading * latent mean
→ Will unequal intercepts lead to wrong conclusions regarding composite differences?
Intercepts and latent means
eXBBY 10
X Y
Y
X
B1
B0
Intercepts and latent means
iiiix
xi
i
i
x1
x4
x2
x3
Intercepts and latent means
)()()( iiii EExE
xi
i
i
x1
x4
x2
x3
Intercepts and latent means
x1
x4
x2
x3
iiixM )(
xi
i
i
M(xi)
Intercepts and latent means
x1
x4
x2
x3
iiixM )(
xi
i
i
M(xi)
Group differences in intercepts and factor loadings
xi
M(xi)M(xi)
M(xi)
x1
x4
x2
x3
x1
x4
x2
x3
Group A Group B
Group differences in intercepts and factor loadings
xi
M(xi)M(xi)
M(xi)
x1
x4
x2
x3
x1
x4
x2
x3
Group A Group B
Group differences in intercepts and factor loadings
xi
M(xi)M(xi)
M(xi)
x1
x4
x2
x3
x1
x4
x2
x3
Group A Group B
Meaning of (unequal) intercepts
Associated terms used in the literature- Item bias- Differential item functioning- Measurement/factorial invariance ("strong factorial invariance", "scalar invariance")
Meaning- Response style (acquiescence, leniency, severity)- Response sets (e.g., social desirability)- Connotations of items- Item specific difficulty
The study
Partial invariance
Research question: Is partial invariance enough for composite mean difference testing?
- Pseudo-differences
- Compensation effects
Procedure (Mplus):
- Step 1: Specification of two-group population models with latent mean and intercept differences; 1000 replications,
raw data saved
- Step 2: Creation of a composite score
- Step 3: Analysis of composite differences
- Step 4: Aggregation (-> sampling distribution)
The study
Design (population model):- Two groups- One latent variable- 4 vs. 6 indicators- All intercepts equal vs. one vs. two intercepts unequal in varying directions (+.30 vs. -.30)
- Latent mean difference: 0 vs. .30- Loadings kept equal with ‘s = .80; latent variance = 1- N = 2 x 100 vs. 2 x 300- Latent models as comparison standard for each condition
Dependent variables- Average composite mean difference - Percent of significant composite differences („% sig“)
Full scalar invariance(Latent mean difference = .30)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Avg. composite difference
%sig
N = 2 x 100 N = 2 x 300
4 Indicators 6 Indicators 4 Indicators 6 Indicators
Pseudo-Differences(Latent mean difference = 0; unequal intercept(s)
4 Ind. 6 Ind.
N = 2 x 300 N = 2 x 100 N = 2 x 300
2 Intercepts unequal (.30)
4 Ind. 6 Ind. 4 Ind. 6 Ind. 4 Ind. 6 Ind.
N = 2 x 100
1 Intercept unequal (.30)
0.00
0.10
0.20
0.30
0.40
0.50
0.60
Avg. composite difference
%sig
Pseudo-Differences(Latent mean difference = 0; unequal intercept(s)
4 Ind. 6 Ind.
N = 2 x 300 N = 2 x 100 N = 2 x 300
2 Intercepts unequal (.30)
4 Ind. 6 Ind. 4 Ind. 6 Ind. 4 Ind. 6 Ind.
N = 2 x 100
1 Intercept unequal (.30)
0.00
0.10
0.20
0.30
0.40
0.50
0.60
Avg. composite difference
%sig
Compensation effects(Latent mean difference = .30; negative intercept
difference)
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
Avg. Composite difference
%sig
4 Ind. 6 Ind.
N = 2 x 300 N = 2 x 100 N = 2 x 300
2 Intercepts unequal (-.30)
4 Ind. 6 Ind. 4 Ind. 6 Ind. 4 Ind. 6 Ind.
N = 2 x 100
1 Intercept unequal (-.30)
Compensation effects(Latent mean difference = .30; negative intercept
difference)
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
Avg. Composite difference
%sig
4 Ind. 6 Ind.
N = 2 x 300 N = 2 x 100 N = 2 x 300
2 Intercepts unequal (-.30)
4 Ind. 6 Ind. 4 Ind. 6 Ind. 4 Ind. 6 Ind.
N = 2 x 100
1 Intercept unequal (-.30)
Summary
Full latent variable models have more power than composite analyses
Pseudo-differences- Even one unequal intercept increases the risk to find spurious composite differences
- High sample size increases risk- Number of indicators reduces the risk – but not substantially
Componensation effects- Even one unequal intercept reduces the size of the composite difference to 50%
- In small samples little chance to find a significant composite difference (power = .25 - .40)
- Two unequal intercepts drastically reduce the composite difference: The power in the „best“ condition (2x300, 6 Ind.) is only .50
Conclusons
Most comparisons of means rely on traditional composite difference analysis
These methods make assumptions that are unrealistic (i.e., full invariance of intercepts)
Even minor violations of these assumptions increase the risk of drawing wrong conclusions
Advantages of SEM:- Assumptions can be tested- Partial invariance implies no danger- Greater power even in small samples
Thank you very much!