recent advances in latent variable modelingmodel # par’s logl bic chi-2 df p rmsea cfi classic 31...

47
Recent Advances in Latent Variable Modeling Bengt Muth´ en [email protected] Mplus: www.statmodel.com Presentation at the joint Stats/RMME colloquium, UCONN November 20, 2020 I thank Tihomir Asparouhov for comments and Noah Hastings for assistance. Bengt Muth´ en Advances in Latent Variable Modeling 1/ 47

Upload: others

Post on 06-Mar-2021

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Recent Advances in Latent Variable Modeling

Bengt Muthen

[email protected]: www.statmodel.com

Presentation at the joint Stats/RMME colloquium, UCONNNovember 20, 2020

I thank Tihomir Asparouhov for comments and Noah Hastings for assistance.

Bengt Muthen Advances in Latent Variable Modeling 1/ 47

Page 2: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Overview

Many different models with a common theme - longitudinal analysis:

Modeling ideas of multilevel factor analysis and longitudinalstructural equation modeling

Latent transition analysis

Multilevel time series analysis

Bayesian analysis of count, nominal, and binary logit outcomes

Drawing on material in Mplus Web Talks atwww.statmodel.com

Bengt Muthen Advances in Latent Variable Modeling 2/ 47

Page 3: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Multilevel Factor Analysis Origins

Yij = ν +YBj +YWij (1)

= ν +ΛB fBj + εBj︸ ︷︷ ︸+ΛW fWij + εWij︸ ︷︷ ︸, (2)

with covariance structure V(Yij) = ΣB +ΣW where each covariancematrix has a factor model structure, Λ Ψ Λ′+Θ.

Cronbach (1976). Research on classrooms and schools: Formulation ofquestions, design, and analysis. Stanford University, School of Ed

Harnqvist (1978). Primary mental abilities of collective and individuallevels. Journal of Educational Psychology

Goldstein & McDonald (1988). A general model for the analysis ofmultilevel data. Psychometrika

McDonald & Goldstein (1989). Balanced versus unbalanced designsfor linear structural relations in two-level data. British Journal ofMathematical and Statistical Psychology

Bengt Muthen Advances in Latent Variable Modeling 3/ 47

Page 4: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Random Intercept View of Two-Level Factor Analysis

The two-level factor analysis model (one factor on each level),

yij = ν +λB fBj + εBj + λW fWij + εWij

expressed in terms of level 1 and level 2 (within and between),

Level 1 : yij = νj +λW fWij + εWij ,

Level 2 : νj = ν +λB fBj + εBj .

Two key aspects:Two-level factor analysis can be viewed as a random interceptmodel where the intercepts have a factor structureBecause the random intercepts appear for the factor indicators y,measurement non-invariance is allowed

Bengt Muthen Advances in Latent Variable Modeling 4/ 47

Page 5: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Two-Level Factor Analysis in a Model Diagram

A random intercept is represented in model diagrams as a factor(latent variable) with unit loading (slope) for the observedoutcomeE.g. students within classrooms or schools. Achievement testing

Typically fewer between-level factors than within-level factorsfb interpretation different from fw interpretation

Within

Between

fb

y1 y6

fw1 fw2

y6

y2 y3 y4 y5

y1 y2 y3 y4 y5

Bengt Muthen Advances in Latent Variable Modeling 5/ 47

Page 6: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Going Deeper Into Multilevel Factor Analysis

Within

Between

fb

fw

y1 y2 y3

Cross-sectional: Students within schools

But the case of time points nested within individuals is also multilevel -so why not apply this model to longitudinal data?

Bengt Muthen Advances in Latent Variable Modeling 6/ 47

Page 7: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

What Multilevel Factor Modeling Can Teach UsAbout Single-Level Modeling: Longitudinal Model for T=2

==

=u11 u12 u13 u u u21 22 23

fw1 fw2

fb

u1 u2 u3

Within

Between

fb

u1 u2 u3

fw

Bengt Muthen Advances in Latent Variable Modeling 7/ 47

Page 8: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Longitudinal Factor Analysis

Hamaker et al. (2016). Using a few snapshots to distinguish mountainsfrom waves: Weak factorial invariance in the context of trait-stateresearch. Multivariate Behavioral Research. - “CUTS” (Common andUnique Trait State) model, based on multilevel FA

Argues that factors obtained from cross-sectional factor analysis arepartly determined by between- and partly by within-person covariancestructure (Cattell, 1978): “An uninterpretable blend” (R & B, 2002)

Multilevel-based longitudinal factor analysis allows differencesbetween between-person, trait-like factor structure and within-person,state-like factor structure

T=2 is sufficient to determine the within-person factor structure. Noneed for intensive longitudinal data to estimate a factor model for eachperson (N=1 analysis; Cattell’s P-technique)

Relates to latent state-trait work in Marsh-Grayson (1994) in SEM, Eid(1996) in MPRO, Dumenci-Windle (1996) in MBR, Geiser-Lockhart(2012) in Psych Methods, Geiser et al. (2015) in Behav Res, andGeiser (2020): Longitudinal Structural Equation Modeling with MplusA Latent State-Trait Perspective. Guilford Press.

Bengt Muthen Advances in Latent Variable Modeling 8/ 47

Page 9: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Longitudinal Factor AnalysisBased on Multilevel Thinking

==

=u11 u12 u13 u u u21 22 23

fw1 fw2

fb

u1 u2 u3

Model extended to include auto-regression for the fw factors

Corresponds to a cross-sectional model where students within schoolsinfluence each other

This multilevel-based longitudinal model is quite different from the”naive” longitudinal model which uses only the bottom part

Bengt Muthen Advances in Latent Variable Modeling 9/ 47

Page 10: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

46 Years Ago: University of Wisconsin - Madison 1974-75

Wheaton, B., Muthen, B.,Alwin, D., & Summers, G.(1977). Assessing reliabilityand stability in panel models.In D. R. Heise (Ed.),Sociological Methodology1977 (pp. 84 - 136). SanFrancisco: Jossey-Bass, Inc.

Bengt Muthen Advances in Latent Variable Modeling 10/ 47

Page 11: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

“Wheaton et al” 1977 Structural Equation Model of theStability of Alienation 1996-1971

a1 p1 a2 a3p2 p3

ses

f1 f2 f3

ed sei

Anomia and Powerlessness as indicators of alienation

Interest in stability estimates 1966 to 1967 and 1967 to 1971 whiletaking measurement error into account

Bengt Muthen Advances in Latent Variable Modeling 11/ 47

Page 12: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

A Random Intercept Version of the “Wheaton et al” Model

a p

ses

ed sei

a1 p1 a2 a3p2 p3

f1 f2 f3

WithinBetween

Bengt Muthen Advances in Latent Variable Modeling 12/ 47

Page 13: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Random Intercept Model Features

a p

ses

ed sei

a1 p1 a2 a3p2 p3

f1 f2 f3

WithinBetween

Still a single-level, wide model

Two random intercepts representing ”between-level” variation(stable over time) instead of many correlated residuals

SES is a ”between-level” variable (does not change over time)and influences only the random intercepts

The relationships between the factors over time representwithin-person dynamics

Bengt Muthen Advances in Latent Variable Modeling 13/ 47

Page 14: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Model Fit Results: Classic vs New (N = 932)

Model # par’s logL BIC Chi-2 df p RMSEA CFI

Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Findings for the New, Random Intercept model:Despite using fewer parameters (25 vs 31), it has better logL,better BIC, and better chi-square, RMSEA, and CFIOn Level-2 (Between), SES has a significant negative effect onthe random intercepts as expected. This means that SES explainsmeasurement non-invariance across peopleOn level-1 (Within), Positive effect of f1 on f2 (1966 to 1967) butno significant effect of f2 on f3 (1967 to 1971)

Some across-time equalities can be relaxed for even better fit

Bengt Muthen Advances in Latent Variable Modeling 14/ 47

Page 15: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Overview

Modeling ideas of multilevel factor analysis and longitudinalstructural equation modeling

Latent transition analysisMultilevel time series analysis

Bayesian analysis of count, nominal, and binary logit outcomes

Bengt Muthen Advances in Latent Variable Modeling 15/ 47

Page 16: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Hidden Markov - Latent Transition Analysis

u: observed categorical variable (latent class indicator)c: latent categorical variable (latent class variable)

u1 u3u2

c1 c2 c3

u4

c4

u5

c5

u11

c1 c2 c3

u12 u31 u32u21 u22

Bengt Muthen Advances in Latent Variable Modeling 16/ 47

Page 17: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

LTA Features

u11

c1 c2 c3

u12 u31 u32u21 u22

1 Measurement probabilities: P(Ut|Ct) - LCA for each time point,measurement invariance across time

2 Initial status probabilities: P(C1)

3 Transition probabilities: P(C2|C1), P(C3|C2)

Extensions:

Covariates: Influencing latent class probabilities and transitionprobabilities

Multiple-group analysis: Measurement invariance

Mover-Stayer modeling

Bengt Muthen Advances in Latent Variable Modeling 17/ 47

Page 18: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

What’s Missing in These Models? Random Intercepts

u1 u3u2

c1 c2 c3

u4

c4

u5

c5

u11

c1 c2 c3

u12 u31 u32u21 u22

Single indicator per time pointA statistical perspective

Multilevel modeling: Level 1 = time, level 2 = subjectRandom effects, especially random intercepts

A substantive perspectiveTrait theory in psychologyBetween-subject differences that are stable over time

Multiple indicators per time pointA psychometric perspective

Measurement non-invarianceMultilevel factor analysisMultilevel latent class analysis

Bengt Muthen Advances in Latent Variable Modeling 18/ 47

Page 19: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Hidden Markov Modeling with a Random Intercept

u

1 1 1

u1 u2 u3

c1 c2 c3

Altman (2007). Mixed hidden Markov models. Journal of the AmericanStatistical Association, 102, 201-210.

Relapsing-remitting multiple sclerosis patients, N=39

Symptoms worsen and then improve in alternating periods ofrelapse and remission

Outcome: Number of lesions in the brain (count variable)

T=24 (monthly scans for two years)

2 latent classes (hidden states): Relapse versus remission

Bengt Muthen Advances in Latent Variable Modeling 19/ 47

Page 20: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Single-Indicator LTA (Hidden Markov)

u

1 1 1

u1 u2 u3 u1 u2 u3

c1 c2 c3 c1 c2 c3

The Hidden Markov model on the left can use the fastbackward-forward Baum-Welch algorithm for ML estimation

The random intercept model on the right loses this simplicity - the U’sare no longer independent conditional on the Cs

ML estimation needs numerical integration with much heaviercomputationsBartolucci et al. (2012). Latent Markov Models for LongitudinalData

Bengt Muthen Advances in Latent Variable Modeling 20/ 47

Page 21: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Random Intercept LTA (RI-LTA)

u11

c1 c2 c3

u12 u31 u32u21 u22

f

λ1 λ1 λ1 λ2λ2λ2

Muthen & Asparouhov (2020). Random intercept latent transitionanalysis (RI-LTA). Forthcoming in Psychological Methods.

Papers and Mplus scripts:http://www.statmodel.com/RI-LTA.shtml

Bengt Muthen Advances in Latent Variable Modeling 21/ 47

Page 22: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Mplus Software Improvements for LTA and RI-LTA

RI-LTA can be time consuming due to numerical integration andneeding many random starts to find the global maximum.Mplus Version 8.4 released last year:

Significant speed improvements for computationally demandingmixture models such as with LTA and RI-LTA using a new three-stagerandom starts search and using specialized algorithms drawing onBaum-Welch ideas

Asparouhov & Muthen (2019). Random Starting Values andMultistage Optimization (Technical Report: http://www.statmodel.com/download/StartsUpdate.pdf)

”A 20 hours computation in Mplus 8.3 can be done in Mplus 8.4in less than 15 minutes, by utilizing the advantages of thethree-stage estimation, the Baum-Welch algorithm, as well asupdated hardware (i9-9900k Intel CPU)”

Substantially simplified output for mixture models with multiple latentclass variables

Bengt Muthen Advances in Latent Variable Modeling 22/ 47

Page 23: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Regular LTA Fits Worse than RI-LTA Most of the Time

Analyses of 4 data sets from the LTA literature (data athttp://www.statmodel.com/RI-LTA.shtml):Table 1: Model fitting results

Life Satisfaction (non-stationary) Mood (stationary)N=5147, T=5, R=1, J=5 N=494, T=4, R=2, J=2

Model # par’s logL BIC Model # par’s logL BIC

Regular LTA 11 -15326 30745 Regular LTA 7 -2053 4150

RI-LTA 12 -15267 30637 RI-LTA 9 -2018 4093

Reading proficiency (non-stationary) Dating and sexual risk behavior (stationary)N=3574, T=4, R=5, J=3 N=2933, T=3, R=3, J=5

Regular LTA 35 -21793 43873 Regular LTA 49 -16202 32796

RI-LTA 40 -20329 40985 RI-LTA 52 -16043 32502

1

Bengt Muthen Advances in Latent Variable Modeling 23/ 47

Page 24: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Reading Proficiency. Kaplan (2008)

Early Childhood Longitudinal Study (ECLS-K), N = 3574

4 time points: Fall and Spring of Kindergarten and Fall and Spring ofGrade 1

5 binary items representing mastery of:

Basic reading skills of letter recognitionBeginning soundsEnding letter soundsSight wordsWords in context

3 latent classes corresponding to 3 stages of learning:

Low alphabet knowledge, early word reading, early readingcomprehension

Kaplan (2008). An overview of Markov chain methods for the study ofstage-sequential developmental processes. Developmental Psychology

Bengt Muthen Advances in Latent Variable Modeling 24/ 47

Page 25: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Reading Data Latent Class Probabilities

Regular LTA

Fall K Spring K Fall 1st Spring 1st

ClassProbabilities

1 0.694 0.235 0.142 0.0412 0.284 0.635 0.627 0.1543 0.023 0.130 0.232 0.805

RI-LTA

Fall K Spring K Fall 1st Spring 1st

ClassProbabilities

1 0.948 0.161 0.040 0.0102 0.049 0.818 0.880 0.0173 0.003 0.022 0.080 0.973

Bengt Muthen Advances in Latent Variable Modeling 25/ 47

Page 26: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Reading Data Transition Probabilities

Regular LTA

1 2 3

TransitionProbabilities Fall K -– Spring K

1 0.338 0.649 0.0122 0.001 0.652 0.3483 0.000 0.000 1.000

RI-LTA

1 2 3

TransitionProbabilities Fall K -– Spring K

1 0.170 0.820 0.0102 0.000 0.819 0.1813 0.000 0.000 1.000

Bengt Muthen Advances in Latent Variable Modeling 26/ 47

Page 27: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Transition Probabilities Influenced By Covariate: RI-LTA

u11

c1 c2 c3

u12 u31 u32u21 u22

f

x

Mplus Web Talk No. 2 at www.statmodel.com

Bengt Muthen Advances in Latent Variable Modeling 27/ 47

Page 28: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Overview

Modeling ideas of multilevel factor analysis and longitudinalstructural equation modeling

Latent transition analysis

Multilevel time series analysisBayesian analysis of count, nominal, and binary logit outcomes

Bengt Muthen Advances in Latent Variable Modeling 28/ 47

Page 29: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

What Single-Level Modeling Can Teach UsAbout Multi-Level Modeling

u11

c1 c2 c3

u12 u31 u32u21 u22

f

λ1 λ1 λ1 λ2λ2λ2

How do you do analyze relations between variables at different timepoints using two-level modeling?

Multilevel time series analysis (time and individual)Other random effects can be added: Random slopes, randomvariances, random AR, random transition probabilitiesMany time points required - intensive longitudinal data (EMA,ESM, daily diary)

Bengt Muthen Advances in Latent Variable Modeling 29/ 47

Page 30: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Dynamic Structural Equation Modeling (DSEM)

Single-level, wide Two-level, long, time series (DSEM)

u11

c1 c2 c3

u12 u31 u32u21 u22

f

λ1 λ1 λ1 λ2λ2λ2

Within (time)

u1 u2

f

λ λ

ct-1 ct

u1t u2t

Between (subject)

1 2

DSEM data are in long format:2 outcomes per time point results in 2 columns of data (not 2*T)

Across-time effects specified by using lags: C ON C&1 (lag 1)Bengt Muthen Advances in Latent Variable Modeling 30/ 47

Page 31: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Dynamic Structural Equation Modeling (DSEM)

Two-level analysis: randomeffects varying across subjectsCross-classified analysis (oftime and subject): randomeffects varying across subjectsand time

fb φ

Between

φfwtfwt-1

Within

DSEM using Bayesian analysis:

Asparouhov, Hamaker & Muthen (2017). Dynamic Latent ClassAnalysis. Structural Equation Modeling: A MultidisciplinaryJournal, 24:2, 257-269Asparouhov, Hamaker & Muthen (2018). Dynamic structuralequation models. Structural Equation Modeling: AMultidisciplinary Journal, 25:3, 359-388

Papers at http://www.statmodel.com/TimeSeries.shtml

Bengt Muthen Advances in Latent Variable Modeling 31/ 47

Page 32: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Bayesian Analysis: Advantages over ML

Bayes with non-informative priors - a powerful computingalgorithm :

Analyses are often less computationally demanding, for example,when maximum-likelihood requires high-dimensional numericalintegration due to many latent variables (factors, random effects)In cases where maximum-likelihood computations areprohibitive, Bayes with non-informative priors can be viewed as acomputing algorithm that would give essentially the same resultsas maximum-likelihood if maximum-likelihood estimation werecomputationally feasible

New types of models can be analyzed where themaximum-likelihood approach is not feasible (e.g. multilevel timeseries models with many random effects)

Bayes with informative parameter priors - a better reflection ofhypotheses based on previous studies

Bengt Muthen Advances in Latent Variable Modeling 32/ 47

Page 33: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Random Effects in Two-Level Time Series Modeling

Time nested in level 2 units (persons, firms, etc)Random effects allowing parameter variation across level 2units:

Random intercept (level)Random slopes on covariates such as timeRandom varianceRandom auto-regressionRandom amplitude

Bengt Muthen Advances in Latent Variable Modeling 33/ 47

Page 34: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Modeling Cycles: Dummies, Splines, Sine-Cosine

0

5

10

15

20

25

30

35

40

45

50

55

60

65

70

75

80

85

90

95

100

105

Time

-7 -6.5

-6 -5.5

-5 -4.5

-4 -3.5

-3 -2.5

-2 -1.5

-1 -0.5

0 0.5

1 1.5

2 2.5

3 3.5

4 4.5

5

B2a_

Y, m

ean

= 24 hrs

Biological cycles24-hour cycles: Circadian rhythm such as heart rate

Behavioral cyclesWeekly drinking pattern

Environmental cyclesMonthly temperature fluctuations

Bengt Muthen Advances in Latent Variable Modeling 34/ 47

Page 35: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Cyclic Formulas Using Sine-Cosine

f (t) = A cos (2π ω t+φ)

=−A sin φ︸ ︷︷ ︸β1

sin (2π ω t)︸ ︷︷ ︸x1

+A cos φ︸ ︷︷ ︸β2

cos (2π ω t)︸ ︷︷ ︸x2

Amplitude = A =√

β 21 +β 2

2

Phase = φ = tan−1(−β1/β2)

ω is a frequency index defined as cycles per unit. Usingω = 1/24 = 0.04167 gives 24-hour cycles

Multiple f (t) components with different cycles per unit can beused. Spectral analysis finds the components of the cycles

Two-level or cross-classified analysis with random effects

Bengt Muthen Advances in Latent Variable Modeling 35/ 47

Page 36: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Very Long Longitudinal Data: T= 1096

Electricity consumption of firms measured daily (and hourly)over 3 years: T=1096 (Schultzberg, 2018)

Intervention: change in tariffN=184 intervention group (N= 800 Control group; not used here)Pre-intervention data for 1 year, post-intervention data for 2 yearsSine-cosine cross-classified model

1

51

101

151

201

251

301

351

401

451

501

551

601

651

701

751

801

851

901

951

100

1

105

1

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

B2a_

Y1, m

ean

B2a_Y1, mean (autocorr = 0.910(0.030))

1

51

101

151

201

251

301

351

401

451

501

551

601

651

701

751

801

851

901

951

100

1

105

1

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

B2a_

Y2, m

ean

B2a_Y2, mean (autocorr = 0.821(0.030))

In the left part of the figure, the curve after the intervention(marked by a vertical line) shows the predicted development inthe absence of the intervention

In the right part of the figure, significant drop in amplitude afterthe intervention (marked by a vertical line)

Bengt Muthen Advances in Latent Variable Modeling 36/ 47

Page 37: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Intervention Modeling in Multilevel Time Series Analysis:Propensity Score Analysis

No randomization: Matching on propensity scores obtained frompre-intervention random effects to evaluate post-interventionoutcomes

Schultzberg (2019). Using high-frequency pre-treatmentoutcomes to identify causal effects in non-experimental data

Within (time)

Between (firms)

tx

yt yt-1

Bengt Muthen Advances in Latent Variable Modeling 37/ 47

Page 38: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Very Long Longitudinal Data

Schultzberg & Muthen (2018). Number of subjects and time pointsneeded for multilevel time series analysis: A simulation study ofdynamic structural equation modeling. Structural Equation Modeling.25:4, 495-515

Bengt Muthen Advances in Latent Variable Modeling 38/ 47

Page 39: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Intervention Modeling in Multilevel Time Series Analysis:Randomized Studies

Intervention interacting with pre-intervention random effects ininfluencing post-intervention random effects (for whom is theintervention effective?)

Within

Between

yt-1pre yt

pre yt-1post yt

post

tx

Bengt Muthen Advances in Latent Variable Modeling 39/ 47

Page 40: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Randomized Trial

Geschwind et al. (2011). Mindfulness training increases momentarypositive emotions and reward experience in adults vulnerable todepression: A randomized controlled trial. Journal of Consulting andClinical Psychology

Experience sampling method (ESM): T= 60 pre, 60 post (each periodhas 10 beeps/day via digital wristwatch for 6 days), N=119

Muthen et al. (2020). In preparation: DSEM with daily cycles inpositive affect modeled by sine-cosine curve with random effects

Bengt Muthen Advances in Latent Variable Modeling 40/ 47

Page 41: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Randomized Trial DSEM Results

Within

Between

yt-1pre yt

pre yt-1post yt

post

tx

Preliminary findings for intervention effects on the random effects ofthe post-intervention time series for momentary positive emotions:

Positive effect on level (intercept); higher effect on level forpersons with higher pre-intervention levelPositive effect on level for persons with high pre-interventionauto correlation

Bengt Muthen Advances in Latent Variable Modeling 41/ 47

Page 42: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Overview

Modeling ideas of multilevel factor analysis and longitudinalstructural equation modeling

Latent transition analysis

Multilevel time series analysis

Bayesian analysis of count, nominal, and binary logitoutcomes

Bengt Muthen Advances in Latent Variable Modeling 42/ 47

Page 43: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Bayesian Analysis of Count, Nominal,and Binary Logit Outcomes

”...has long been recognized as a hard problem”Polson, Scott, Windle (2013). Bayesian inference for logisticmodels using Polya-Gamma latent variables. JASA

Asparouhov, Muthen (2020). Expanding the Bayesian structuralequation, multilevel and mixture models to logit,negative-binomial and nominal variables

Allows many latent variables (factors, random effects) in ageneral modeling frameworkCount modeling using negative binomial (NB-2) and PoissonMplus Version 8.5 released earlier this week

Polson et al. (2013) considered an autoregressive N=1 timeseries model for count data

Asparouhov-Muthen (2020) consider multilevel (N > 1) versions

Bengt Muthen Advances in Latent Variable Modeling 43/ 47

Page 44: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Application to Longitudinal Count Modeling

Example with T=10: Maximum-likelihood estimation requires11 dimensions of integration; intractable. Bayes ok at N ≥ 500

ν

η1 η2 η10

y1 y2 y10...

...

Yit ∼ NB2(νi +ηit,α) (3)

νi ∼ N(ν ,v1) (4)

ηit = ρηi,t−1 + εit, for t = 2, ...,T (5)

εit ∼ N(0,v2). (6)

The model has 5 Negbin parameters: ν (random intercept mean),ρ (AR), α (dispersion), v1 and v2 (variance/residual variance)

Bengt Muthen Advances in Latent Variable Modeling 44/ 47

Page 45: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Random Intercept Cross-Lagged Panel Model (RI-CLPM)

Extensions of Hamaker et al. (2015) to counts and categoricaloutcomes (in preparation)

z1

y1

wy1

wz1

y

z

z3

y3

wy3

wz3

z2

y2

wy2

wz2

Bengt Muthen Advances in Latent Variable Modeling 45/ 47

Page 46: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

UCONN 1995 Origins: Traits, States, and Errors

Kenny & Zautra (1995). The trait-state-error model for multiwavedata. Journal of Consulting and Clinical Psychology

TRAIT-STATE-ERROR MODEL 55

e.

Figure 2. Bivariate trait-state-error model. Paths from state factors (a, b, c, and d) are denoted as ax

Oyx, axy, and ayy in the text. Numerals 1 through 4 indicate the four lags; e = error; u = disturbance.

1989). For four-wave data, the resulting model with both ofthese modifications would have zero degrees of freedom be-cause an additional six degree parameters (three error variancesand three loadings) are estimated. Such a model is just-identi-fied and there is no chi-square test.

Bivariate Model

The major interest in this article is the causal relationshipbetween two variables. There are two variables, X and Y, eachof which is measured at each wave. Both X and /are assumedto be caused by a trait, Tx or Ty, a state factor, Sx or Sy, and bymeasurement error, Ex or Ey. So we have

X = Tx + Sx + Ex and

Y= Ty + Sy + Ey.

For each variable, there are the four parameters described in theprevious section: trait variance V( Tx) and V( Ty); state distur-bance variance V(UX) and V(Uy)', error variance V(EX) andV(Ey); and two autoregressive parameters (a^) and (ayy).There are five additional bivariate parameters or parametersthat describe the association between X and Y. The first threeinvolve correlations between the X and Y variables at the traitC( Tx, Ty), state C( Ux, Uy), and error C(EX, Ey) levels. Thenext two are lag-1 regression coefficients of state variables: theeffect of Y on X (axy) and the effect of X on Y (ayx). There are atotal of 13 parameters, and the model's degrees of freedom aren(2n+ 1) — 13, where n is the number of waves. The model ispresented in pictorial form in Figure 2.

In principle, one could allow for lag-2 effects; that is, one'scurrent standing is affected by the previous time and the time

before that. This article does not explore this complexity be-cause it seems theoretically unlikely.

There are three nonlinear constraints on the parameters.Each state variance at Time 1 equals the disturbance varianceof the state divided by 1 minus the multiple correlation squaredof the state factor. The state covariance at Time 1 equals a func-tion of the covariance of the state disturbances and the lag co-efficients.3 A practical estimation strategy is to re-estimate themodel and constrain the parameters to their value in the previ-ous solution. The estimation stops when the parameters do notchange very much.

Covariates

If there are covariates that operate at the trait level (e.g., gen-der or ethnicity), their correlation can be estimated and testedat that level. Although one could introduce them into the struc-tural equation model, there is another alternative. One firstcomputes the mean score for each variable (Xand Y) across then time points. One then correlates a covariate with this meanscore. However, this correlation is attenuated because of the factthat the mean score estimates but does not equal the trait factor.To disattenuate the correlation, one multiplies the previous cor-relation by the trait standard deviation (taken from the struc-tural equation model run) divided by the standard deviation ofthe mean of the measures. (This ratio of standard deviationsequals the square root of the reliability of mean score.)

3 That function is as follows:

c- x _ a»fl*yV(Ux) + • y) + C( U*,

Figure 2. The bivariate case

Bengt Muthen Advances in Latent Variable Modeling 46/ 47

Page 47: Recent Advances in Latent Variable ModelingModel # par’s logL BIC Chi-2 df p RMSEA CFI Classic 31 -19455 39121 90.04 13 0.00 0.080 0.979 New 25 -19425 39021 30.65 19 0.04 0.026 0.997

Recent Paperswww.statmodel.com/recentpapers.shtml

Asparouhov & Muthen (2019). Nesting and equivalence testing for structural equationmodels. Structural Equation Modeling

Asparouhov & Muthen (2019). Random starting values and multistage optimization(Technical Report)

Asparouhov & Muthen (2019). Bayes parallel computation: Choosing the number ofprocessors (Technical Report)

Asparouhov & Muthen (2019). Bayesian estimation of single and multilevel models withlatent variable interactions. Structural Equation Modeling

Asparouhov & Muthen (2019). Advances in Bayesian model fit evaluation for structuralequation models. Structural Equation Modeling

Asparouhov & Muthen (2019). Latent variable centering of predictors and mediators inmultilevel and time-series models. Structural Equation Modeling

Asparouhov & Muthen (2020). Comparison of models for the analysis of intensivelongitudinal data. Structural Equation Modeling

Asparouhov & Muthen (2020). Expanding the Bayesian structural equation, multileveland mixture models to logit, negative-binomial and nominal variables

Hamaker & Muthen (2020). The fixed versus random effects debate and how it relates tocentering in multilevel modeling. Psychological Methods

Muthen & Asparouhov (2020). Latent transition analysis with random intercepts(RI-LTA). Forthcoming in Psychological Methods

Bengt Muthen Advances in Latent Variable Modeling 47/ 47