analyzing observed composite differences across groups: is partial measurement invariance enough?...
Post on 22-Dec-2015
216 views
TRANSCRIPT
Analyzing observed composite differences across groups: Is partial measurement invariance
enough?
Holger Steinmetz Faculty of Economics and Business Administration
Department of Human Resource Management, Small Business Enterprises, and EntrepreneurshipUniversity of Giessen / Germany
Introduction
Importance of analyses of mean differences
For instance:- gender differences on wellbeing, self-esteem, abilities, behavior- differences between leaders and non-leaders on intelligence and personality traits
- differences between cultural populations on psychological competencies, values, wellbeing
Usual procedure: t-test or ANOVA with observed composite scores
Latent means vs. observed means
Partial invariance as legitimation for the composite difference test
Research question: Effects of unequal intercepts and/or factor loadings across groups on composite differences
Relationship between latent and observed means
iiix
x1
x4
x2
x3
Relationship between latent and observed means
iiiix
xi
i
i
x1
x4
x2
x3
Relationship between latent and observed means
)()()( iiii EExE
xi
i
i
x1
x4
x2
x3
Relationship between latent and observed means
x1
x4
x2
x3
iiixE )(
xi
i
i
E(xi)
Group differences in intercepts and factor loadings
xi
E(xi)
E(xi)
x1
x4
x2
x3
x1
x4
x2
x3
Group A Group B
Group differences in intercepts and factor loadings
xi
E(xi)
E(xi)
x1
x4
x2
x3
x1
x4
x2
x3
Group A Group B
Group differences in intercepts and factor loadings
xi
E(xi)
E(xi)
x1
x4
x2
x3
x1
x4
x2
x3
Group A Group B
The study
Partial invariance: Some loadings / intercepts are allowed to differ
Research question: Is partial invariance enough for composite mean difference testing?
- Pseudo-differences
- Compensation effects
Procedure (Mplus):
- Step 1: a) Specification of two-group population models with varying differences in latent mean, intercepts and loadings
b) 1000 replications, raw data saved
- Step 2: Creation of a composite score
- Step 3: Analysis of composite differences
- Step 4: Aggregation (-> sampling distribution)
The study
Population model:- Two groups- One latent variable
Conditions:- 4 vs. 6 indicators- Latent mean difference: 0 vs. .30- Intercepts: equal vs. one vs. two intercepts unequal in varying directions (-.30 vs. +.30)
- Loadings: equal (‘s = .80) vs. one vs. two loadings = .60- Sample size: 2x100 vs. 2x300
Dependent variables- Average composite mean difference - Percent of significant composite differences
Group A Group B
x1
x4
x2
x3
x1
x4
x2
x3
x5
x6
x5
x6
=.00
=-.30
=.80
=.60
=.30
Pseudo-DifferencesEffects on the average composite difference
4 Ind. 6 Ind.
N = 2 x 300
4 Ind. 6 Ind.
0.00
0.05
0.10
0.15
0.20
0.25
0.30
1 intercept unequal
2 intercepts unequal
N = 2 x 100
Pseudo-DifferencesEffects on the probability of significant differences (Type
I error)
0.00
0.10
0.20
0.30
0.40
0.50
0.60
4 Ind. 6 Ind.
1 intercept unequal
2 intercepts unequal
All intercepts equal
N = 2 x 100
Pseudo-DifferencesEffects on the probability of significant differences (Type
I error)
0.00
0.10
0.20
0.30
0.40
0.50
0.60
4 Ind. 6 Ind. 4 Ind. 6 Ind.
1 intercept unequal
2 intercepts unequal
All intercepts equal
N = 2 x 300N = 2 x 100
Compensation effectsEffects on the average composite differences
1 intercept unequal
2 intercepts unequal
All intercepts equal
Loadingsequal
1 Loadingunequal
4 Indicators
2 Loadingsunequal
0.00
0.05
0.10
0.15
0.20
0.25
0.30
Effect of unequal loadings
Effect of unequal intercepts
Compensation effectsEffects on the average composite differences
1 intercept unequal
2 intercepts unequal
All intercepts equal
Loadingsequal
1 Loadingunequal
4 Indicators
2 Loadingsunequal
Loadingsequal
1 Loadingunequal
6 Indicators
2 Loadingsunequal
0.00
0.05
0.10
0.15
0.20
0.25
0.30
Compensation effectsEffects on the probability of significant differences
(Power)
Loadingsequal
1 Loadingunequal
N = 2x300 / 6 Indicators
2 Loadingsunequal
1 intercept unequal
2 intercepts unequal
All intercepts equal
Loadingsequal
1 Loadingunequal
N = 2x100 / 4 Indicators
2 Loadingsunequal
0.00
0.10
0.20
0.30
0.40
0.60
0.90
0.50
0.70
0.80
Summary
Pseudo-differences- Even one unequal intercept increases the risk to find composite differences
- High sample size increases risk (up to 60% with two unequal intercepts)
- Unequal factor loadings have only a low influence- Number of indicators reduces the risk – but not substantially
Compensation effects- Just one unequal intercept reduces the size of the composite difference to 50%
- With a “small” sample size little chance to find a significant composite difference (power = .25 - .40)
- Two unequal intercepts drastically reduce the composite difference: The power in the „best“ condition (2x300, 6 Ind.) is only .50
Conclusons
Most comparisons of means rely on traditional composite difference analysis
Researcher must not use supported partial invariance as a legitimation for using all items of the scale as a composite
Recommendations- Use SEM:
a)Testing latent mean differences under partial invariance possible
b)Greater power even in small samples
- use only those items that were invariant in tests of invariance
- Increasing number of items (will, however, probably violate the factor model)
