Modeling item response profiles using factor models,
latent class models, and latent variable hybrids
Dena Pastor
James Madison University
Purposes of the Presentation
• To present the model-implied item response profiles (IRPs) that correspond to latent variable models used with dichotomous item response data
• To provide an example of how these models can be used in practice
Item Response Profiles (IRPs)
1 2 3 40
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Item Number
Pro
port
ion
Res
pond
ing
Cor
rect
ly
Elevation Differences
IRPs for classes of examinees with the same pattern, but differences in elevation
Latent Variable Model
PARALLELNON-PARALLEL
C
Latent Class Model
C is a latent categorical
variable with as many levels as #
of classes
C is a nominal latent variable
C is a ordinal latent variable
Exploratory Process
• In latent class modeling a variety of models are fit to the data with differing numbers of classes– 1-class model, 2-class model, 3-class
model, etc.
• Use fit indices and a priori expectations to determine the number of classes to retain
• Can allow latent categorical variable to be nominal and examine resulting profiles; can also constrain latent categorical variable to be ordinal
Alternative Model for Parallel Profiles
Do we have 3 classes, with no variability within class?
OR
Do we have 1 profile with systematic variability within class?
F
Factor ModelF is a latent continuous
variable
Different Models for Different IRPs
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
1 profile…
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
…+ within profile variability
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
2 parallel profiles…
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
…+ within profile variability
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
2 non-parallel profiles…
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
…+ within profile variability
LCM: 1 class
Factor Model
LCM: 2 classes (C is
ordinal)
Semi-parametric
Factor Model
LCM: 2 classes(C is nominal)
Factor Mixture Model
Latent Variable Hybrids
Deci
sion
sM
od
els
IRP
s
1
Number of profiles?(number of
classes)
no
Latent class model(LCM)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
yes
Factor model(FM)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
Systematic variability
within profiles?
1+
Nature of profile
differences?
Parallel
LCM with
parallel
profiles
Semi-parametric factor model
(SPFM)
Systematic variability
within profiles?
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
no
yes
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
Non-parallel
Factor mixtur
e model(FMM
)
LCM with non-
parallel profiles
Systematic variability
within profiles?
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
no
yes
Sem
i-p
ara
metr
ic f
act
or
mod
el
(SP
FM
)
F C
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
2 classes
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
FC
1 class: Factor Model!
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
F C
2 classes, w/in class factor variance = 0
2 classes
Fact
or
mix
ture
mod
el
(FM
M)
F C
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
1 class: Factor Model!
F C
CLatent class model(LCM)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 40
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
F C
2 classes, w/in class factor variance = 0
1
( 1| ) ( 1| )K
i k ik
P u P u c
Marginal probability of getting an item correct is sum across classes of probability of getting item correct conditional on class membership
Conditional probability differs across models
exp( )
1 (exp( ))ki ki k
ki ki k
F
F
~ (0, )k kF N F C
Factor mixtur
e model(FMM
) exp( )
1 (exp( ))i i k
i i k
F
F
~ ( , )k k kF N F CSemi-
parametric factor model
(SPFM)
Cexp( )
1 (exp( ))ki
ki
Latent class model(LCM)
C
Latent class model(LCM)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
C
C
IRPPath diagram
Latent Variable Distribution
C is ordinal
C is nominal
Semi-Parametric Factor Model
(SPFM)
IRP Path diagramLatent Variable
Distribution
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
F C
F C
Measurement Invariance
Same measurement model parameters (thresholds, loadings) for each class
Quantitative differences between
classes
Factor Mixture Model(FMM)
IRP Path diagramLatent Variable
Distribution
F C
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 2 3 4
F C
Measurement Non-Invariance
Different measurement model parameters (thresholds, loadings) for each class
Qualitative differences between classes
Example
• 9 dichotomously scored items measuring 3 aspects of psychosocial research:
1. Confidentiality
2. Generalizability
3. Informed Consent
• Sample 2,259 incoming freshmen tested in low-stakes conditions prior to start of classes
Exploratory Model Selection
• Exploratory model selection approach to answer the question, “What type and number of latent variables are most salient for our data?”
• Reasons to believe that IRPs would differ in pattern and/or elevation because students differ in:• Completion of psychosocial coursework
• Effort they put forth on test
Model Fit Indices
Model LL # paras BIC SSA-BIC LMRFM 1f -12628 18 25395 25338 NA
2f -12576 26 25352 25270 NA 3f -12547 33 25348 25243 NA
LCM 1c -12938 9 25946 25918 NA 2c -12649 19 25445 25384 0.00 3c -12591 29 25406 25314 0.07 4c -12538 39 25377 25253 0.01 5c -12520 49 25418 25262 0.14
SPFM 1f2c -12618 21 25399 25332 0.01
FMM 1f2c -12539 35 25348 25237 0.00
2-class FMM
0.44
0.56
ˆ 0.27
ˆ 3.96X
Y
26. Which ethical practice is not considered by Marty?a) She failed to obtain
informed consent from her participants
b) She failed to randomly select participants
c) …d) …
Factor Variability Within Each Class
Visually Conveying Loading Information
Item Content Item Number Class X Class YConfidentiality 17 0.26 0.36
34 0.28 0.7439 0.16 0.81
Generalizability 25 0.46 -0.0327 0.49 -0.0736 0.46 0.27
Informed Consent 23 0.51 0.3026 0.88 0.3532 0.44 0.31
Standardized Loadings
ˆ 0.27
ˆ 3.96X
Y
X Y
Validity Evidence for 2-class FMM Solution
• Students with higher SAT-V scores, who reported put forth more effort on the test, and who have completed psychosocial coursework more likely to be in Class X
• Positive relationship between SAT-V, coursework completion and factor scores in that class (negative relationship with effort)
• Negative relationship between number of missing responses and factor scores in Class Y
X Y
Correspondence Between Models
A & B from
LCM, X from FMM
C & D from
LCM, Y from FMM
X & Y from FMMwith
intervals
Parting Thoughts…
• These models are like potato chips…– It was so much easier to settle on a brand of
chip when I had a limited number of brands to choose from
– But I also like having more brands because it increases my chances of finding the brand that is right for me
– With all these brands, it is possible that some are selling essentially the same chip….but which ones?
– When two brands are essentially the same chip, what criteria do I use to choose between the two brands?
Pastor, D. A., & Gagné, P. (2013). Mean and covariance structure mixture models. In G. R. Hancock & R. O. Mueller (Eds.), Structural Equation Modeling: A Second Course (2nd Ed.). Greenwich, CT: Information Age.
Pastor, D. A., Lau, A. R., & Setzer, J. C. (2007, August). Modeling item response profiles using factor models, latent class models, and latent variable hybrids. Poster presented at the annual meeting of the American Psychological Association, San Francisco.