1

SEM and Longitudinal DataAutoregressive models and

missing data

UTD

06.04.2006

2

1.1,1.2) Overview and Goals• 1) Model specification for autoregressive, cross-lagged,

growth curves and ALT/hybrid models.• 1a) Single group models;• 1b) Multiple-group models.• 1c) MIMIC models.• 1d) Treatment of missing values.• 2) Use of AMOS graphics to set up and test

autoregressive, cross-lagged, growth curves and ALT/hybrid models.

• 3) Interpretation of parameters in the different types of models in substantive examples (authoritarianism and anomie in Germany).

• 4) Comparison of the approaches.

3

Autoregressive models (called also quasi-simplex or markov models)

Cross-lagged models

Growth curves models

ALT/hybrid models

4

Xt1 Xt2

res1

Autoregressive Model•each variable X at t2 is a function of its lagged measure at t1 and residual (or any variable at t(n) is a function of a variable at t(n-1) and not of any variable before (like a variable at time t(n-2)).•stability coefficients indicate degree of stability of interindividual differences

One has to differentiate between unstandardized stability coefficients and standardized stability coefficients.

5

Cross-lagged autoregressive models•each variable X at t2 function of its lagged measure at t1 and residual•stability coefficients indicate degree of stability of inter-individual differences

Xt1

Yt1

Xt2

Yt2

res2

res1

a

b

cross-lagged autoregressive model (Finkel 1995)• cross-construct regression weights: X predicting Y, controlling for former values of Y

c

d

6

latent growth curve models•for analysing individual change processes using single/ multiple indicators

• assumption: a latent trajectory characterizing the sample (or subgroups) can be found • individual change as function of intercept and slope factors for each time period

Xt3Xt1Xt2

res1

res

2res

3

11

1

Intercept

1 F

Slope

0

individual change as function of intercept and slope factors for each time period

7

ALT/hybrid models

Xt3Xt1Xt2

res1

res

2res

3

1 F

Slope

0

11

1

Intercept

An ALT/hybrid model is a combination of both autoregressive and growth curves models. It includes the stability coefficients of the autoregressive model, and also the slope and intercept latent variables.

8

1.3,1.4) Autoregressive models with t waves on the latent level

Xt1 Xt2 Xt

Err2 Errn

b21 bt t-1

For the structural model the first order autoregressive structure without intercepts is

Xit = bt t-1*Xit-1 + Err t , i=1,…

9

1.5 (Interpretation of parameters:• Stability is generally defined as a relation between

individuals in two subsequent times t-1 and t.

• Perfect stability means that the relation between individuals does not change. For example, if we have a sample of two individuals, and one individual has a value twice as high as the second one has, then we would expect this relation not to change in the next time point under perfect stability. In such a case, the standardized regression (stability) coefficient between time1 and time 2 will be 1.

10

Exercise:

• 1) In a sample of 3 individuals we measure attitude with one indicator. The values the three respondents provide are:1,1,2. In the second time point the first individual has a value of 2, and the stability coefficient is 1. What are the values that the two other respondents provide in the second time point?

11

• Positive stability (standardized coefficient close to +1): the order remains the same between individuals.

• Negative stability (standardized coefficient close to –1): the order of individuals inverts.

• Low stability (standardized coefficient close to zero): the order just mixes, low become high, high become low.

• No stability at all: standardized regression coefficient is zero.

• Standardized and unstandardized regression coefficients have different functions. For test propositions we can compare only the unstandardized coefficients, just like in other statistical tests. For interpretation, mostly standardized coefficients are used.

12

1.6) Parameterization of Autocorrelation:Autoregressive models with three waves and multiple

indicators

auto_04

AU01W3R

e7

1

1

AU02W3R

e81AU03W3R

e9

f3c

1

auto_03

AU01W2R

e4

AU02W2R

e5

AU03W2R

e61 1

f3b

1

auto_02

AU01W1R

e1

AU02W1R

e2

AU03W1R

e31 1

f3a

1

a1

r1 r21 1

a2

f2b1f2a1 f2c

A model with autocorrelations

13

Autoregressive model with method factors

• Every item is determined by: • 1) a latent variable of interest;• 2) measurement error;• 3) specific/unique factor: each item has a specific meaning

that frequently will be invariant across time and groups.• We interpret each item’s content as being constituted by two

facets: the common factor (the latent variable) and the unique factor (the item meaning).

14

Autoregressive models with three waves and multiple indicators- an alternative approach

A model with uniqueness factors

aut1

AU03W1R

e1

1

1

AU04W1R

e2

1

1

aut3

AU03W3R

e5

1

1

AU04W3R

e6

1

1

aut2

AU03W2R

e3

1

1

AU04W2R

0

e4

1

1

Ex2: autoregressive model of authoritarianismwith MTMM and Socratic effect

res1 res2

1 1

1

M1

1

M2

15

1.7 (Socratic Effect

• H1) In addition to common factors, there are specific factors that contribute significantly to the autocorrelations of the indicators.

• The consistency process underlying the socratic effect does not primarily change the relations between items, but rather the relations between items and their corresponding latent variables.

• This effect is postulated for short term panel studies.

16

Socratic Effect (2)• If there is a learning and a memory effect, then the

following hypothesis should hold:

• H2.1) The standardized coefficient linking an item to its corresponding latent variable is significantly lower in the first wave than in the second wave in short-wave panel-studies (what is a short wave?…).

• We further hypothesize that the loading does not increase beyond wave 2.

• H2.2) The standardized loading of the items on the latent variable do not increase beyond the second wave.

17

Socratic Effect (3)• The same should hold for the uniqueness factors:

• H3.1) Standardized factor loadings of the items on the uniqueness factors are significantly lower in the first wave than in the second wave.

• H3.2) The standardized factor loading on the uniqueness factor does not increase significantly from the second wave on.

18

Socratic Effect (4)

• Logically we can derive from H2 and H3:

Because respondents are sensitized by the first interview, their responses, in general, should be more consistent.

H4.1) The variance of random measurement error is greater in the first wave than in the second.

H4.2) The random measurement error does not increase significantly from the second wave on.

19

Socratic Effect (5)• H5) In a short-wave panel (for example only a few

weeks) the inter-temporal consistency of a latent variable is nearly perfect, that is, the unstandardized and the standardized stability coefficients are close to unity.

• Hypotheses H2-H5 form the core of the socratic effect.• H7) The observed interwave correlation of an item

between t-1 and t is larger than the correlation between t1 and t2. Therefore we expect the stability from the second time point to be higher than between the first and the second.

• Empirical evidence was inconclusive. • Saris/van der Putte proposed an alternative model which

had the same fit for the same data (but simpler) based on a true score model (1988).

20

(1) Srole (1956, p. 716; see Scheepers et al. 1992):

• anomic individuals choose authoritarian stances in order to recover orientation

(2) McClosky & Schaar (1965)

• authoritarian individuals are hampered to interact effectively

• less opportunities to escape from social isolation

• resulting in anomia

AuthoritarianismAnomia

Anomia Authoritarianism

Cross-lagged effects: the issue of causality, standard and non-standard model specification.

21

Anomia Authoritarianism

(3) reciprocal relationship: not necessarily unplausible

Research questions for longitudinal analysis: a)• are authoritarian attitudes stable over time?• are anomic attitudes stable over time?• does anomia cause authoritarianism, does authoritarianism cause anomia or do we get evidence for both processes?

b) Research questions which we will relate to tomorrow:• if we get evidence for individual change of authoritarian and/ or anomic attitudes: is there an increase or a decrease?• do we get evidence for individual differences concerning such a development?• is there a relationship between the initial level of authoritarianism/ anomia and its dynamic?

22

Cross-lagged autoregressive modelsautoregressive model • each variable X at t2 function of its lagged measure at t1 and residual• stability coefficients indicate degree of stability of interindividual

differences

cross-lagged autoregressive model (Finkel 1995)• cross-construct regression weights: X predicting Y, controlling for former values of Y

Xt1

Yt1

Xt2

Yt2

res2

res1

a

b

c

d

23

A Cross-lagged model with auto-correlations

auto_04

AU01W3R

e7

1

1

AU02W3R

e81AU03W3R

e9

f3c

1

anom_02

AN01W1R

e10

AN02W1R

e11

auto_03

AU01W2R

e4

AU02W2R

e5

AU03W2R

e61 1

f3b

1

auto_02

AU01W1R

e1

AU02W1R

e2

AU03W1R

e31 1

f3a

1

a1

r1 r21 1

a2

anom_03

AN01W2R

e12

AN02W2R

e13

anom_04

AN01W3R

e14

AN02W3R

e15

b1 b2

r3 r4

1

aut_1 aut_2

f2b1f2a1 f2c

f4a

11

1 f4b

1

1

1

1

1

f4c

1

1

24

A cross-lagged model with multiple indicators and uniqueness factors

auto_04

AU01W3R

e7

1

1

AU02W3R

e81AU03W3R

e9

f3c

1

anom_02

AN01W1R

e10

AN02W1R

e11

auto_03

AU01W2R

e4

AU02W2R

e5

AU03W2R

e61 1

f3b

1

auto_02

AU01W1R

e1

AU02W1R

e2

AU03W1R

e31 1

f3a

1

a1

r1 r21 1

a2

anom_03

AN01W2R

e12

AN02W2R

e13

anom_04

AN01W3R

e14

AN02W3R

e15

b1 b2

r3 r4

1

aut_1 aut_2

f2b1f2a1 f2c

f4a

11

1 f4b

1

1

1

1

1

f4c

1

1

UAu1 UAu2 UAu3

UAn1 UAn2

25

1.9 (Measurement invariance over time

• To establish equal meaning of the construct over time and over groups, different criteria for measurement invariance have been proposed.

• These criteria form prerequisites for a comparison of latent means over time and groups.

26

t2

=1

=1

t1

Item a

Item b

Item c

Item d

Item e

Item f

Measurement Invariance:

Equal factor loadings across groups and/or time points

t2

=1

=1

t1

Item a

Item b

Item c

Item d

Item e

Item f

Group A Group B

27

• Configural Invariance

• Metric/measurement Invariance

• Scalar Invariance

• Invariance of Factor Variances

• Invariance of Factor Covariances

• Invariance of latent Means

• Invariance of Unique Variances

Steps in testing for Measurement Invariance

28

• Configural Invariance

• Metric Invariance

• Equal factor loadings

• Same scale units in both groups/time points

• Presumption for the comparison of latent means

• Scalar Invariance

• Invariance of Factor Variances

• Invariance of Factor Covariances

• Invariance of latent Means

• Invariance of Unique Variances

Steps in testing for Measurement Invariance

29

• Configural Invariance

• Metric Invariance

• Scalar Invariance

• Equal item intercepts

• Same systematic biases in both groups/time points

• Presumption for comparison of latent means

• Invariance of Factor Variances

• Invariance of Factor Covariances

• Invariance of latent Means

• Invariance of Unique Variances

Steps in testing for Measurement Invariance

30

• Configural Invariance

• Metric Invariance

• Scalar Invariance

• Invariance of Factor Variances

• Invariance of Factor Covariances

• Invariance of latent Means

• Invariance of Unique Variances

Steps in testing for Measurement Invariance

31

• Concept of ‘partial invariance’ introduced by Byrne, Shavelson & Muthén (1989)

• Procedure

• Constrain complete matrix

• Use modification indices to find non-invariant parameters and then relax the constraint

• Compare with the unrestricted model

• Steenkamp & Baumgartner (1998): Two indicators with invariant loadings and intercepts are sufficient for mean comparisons

• One of them can be the marker + one further invariant item

Full vs. Partial Invariance

32

1.10) Multi-group analysis of means and intercepts over time.

• ETA=Alpha(g) +Beta (g) *Eta (g) + Gamma (g) *Ksi (g) +Zeta (g)

• X = constant + FA (g) + E (g)

• Where alpha and constant are vectors of constant intercept terms. We assume that Zeta is uncorrelated with Ksi, and E is uncorrelated with A. We also assume as before that Ex(E)=Ex(Zeta)=0 (Ex is the expected value operator), but it is not assumed that Ex(A) is zero. The mean of A, Ex(A), will be a parameter denoted by k.

33

Getting a model with means identified• As Sörbom (1978) has shown, in order to estimate the

means, we must introduce some further restrictions:• 1) setting the mean of the latent variable in one group-the

reference group- to zero. The estimation of the mean of the latent variable in the other group is then the mean difference with respect to the reference group.

• 2) setting the measurement models invariant across groups, since it makes no sense to compare the means of constructs having a different measurement model in the groups. One intercept per construct has to be set to zero.

34

Getting a model with means identified (2)

• 3) invariant stabilities (not necessary for identification).

• 4) When we have only one group: setting the intercepts and factor loadings invariant over time as we will do in the examples.

• 5) Two more new methods are dealt with by Little et al. (2006).

35

aut1

In_Au3

AU03W1R

0;

e1

1

10

AU04W1R

0;

e2

1

1

aut3

In_Au3

AU03W3R

0;

e5

1

10

AU04W3R

0;

e6

1

1

aut2

In_Au3

AU03W2R

0;

e3

1

10

AU04W2R

0;

e4

1

1

Ex6a: estimation of means and intercepts

0;

res10;

res2

1 1

36

2,15; ,43

aut1

,51

AU03W1R

0; ,46

e1

1,00

10

AU04W1R

0; ,30

e2

1,00

1

,08

aut3

,51

AU03W3R

0; ,42

e5

1,00

10

AU04W3R

0; ,29

e6

1,00

1

,38

aut2

,51

AU03W2R

0; ,45

e3

1,00

10

AU04W2R

0; ,32

e4

1,00

1

,84 ,96

Ex6a: estimation of means and intercepts

,22

,21

,23,04

,00

,01

0; ,08

res10; ,07

res2

1 1

37

More about stability

• A structural stability refers to longitudinal consistency in multi-dimensional measurement properties, and provides a basis for comparison of factor means and relations over time (Meredith 1993). That is, it refers to measurement invariance between the two time points.

• Differential stability represents the correlation of individual latent scores assessed at two separate occasions (corresponds to our earlier stability concept).

38

More about stability (2)

• Latent mean stability describes changes in the latent construct means over time.

39

1.12 (MIMIC: autoregressive models and cross-lagged effects

• Uses:• Predicting exogenous variables in the autoregressive

models but possibly also on other endogenous variables.• Auxiliary variables may improve FIML in case they

have no missing data, as they can help predicting missing values in the model.

• Characteristics: the newly introduced exogenous auxiliary/background/instrumental variables may be dichotomous or interval. Some of them must be observable.

• Distributional assumptions are not violated when background variables are dummies and used as exogenuous variables.

40

auto_04

AU01W3R

e7

1

1

AU02W3R

e81AU03W3R

e9

f3c

1

auto_03

AU01W2R

e4

AU02W2R

e5

AU03W2R

e61 1

f3b

1

auto_02

AU01W1R

e1

AU02W1R

e2

AU03W1R

e31 1

f3a

1

a1

r1 r21 1

a2

f2b1f2a1 f2c

Age Education East/West

r01

41

1.13) Robustness and non-normality with multiple-group analyses in SEM models (Satorra 2001)

• Satorra studied three chi square test statistics with non-normal data and structural equation models with multiple-groups.

• The normal theory-NT (ML) approach proposed by Joereskog 1971 has been complemented by Satorra with robust statistics and results on asymptotic robustness. It was built in in LISREL.

• His conclusion: for non-normal data the various test statistics like chi square and CR (critical ratio) have different powers (power is type 2 error).

42

Robustness and non-normality(2)• Non normality is not a problem with large sample

sizes (1000 and more), and results are robust. However, performance of chi square statistics in any SEM model in small and medium samples with non-normal data is rather poor.

• Differences in power do not arise when asymptotic robustness (non-bias) holds.

43

Stability – different definitions

  Type of Stability Phenomenon Coefficient Interpretation

1. Positional Stability The relative position of individuals within a sample

AR coefficients- standardized and unstandardized

Perfect positional stability means coefficients are all 1. No stability:0. medium: around 0.5

2. Absolute Stability The absolute mean In AR model the mean over time.In LGC model coefficient from latent slope across time.

Whether level or mean or coefficients change. May change when positional stability is 1. However, if there is absolute stability, there is also positional stability.

3. Dynamic Stability Equilibrium of a dynamic model

Eigenvalues are all non-zero (positive or negative)

A dynamic stability means long-term growth or decline.

44

2.11 Beyond SEM: General Latent Variable Modeling/

Muthen 2002• The generalized approach: it views all kinds of latent

variable models as specific cases of one general model.

• Its goals: • 1) better integration of Psychometric modeling into

mainstream statistics. • 2) to show how many statistical analyses implicitly

utilize the idea of latent variables in the form of random effects, components of variation, missing data, mixture components and clusters.

45

• The generality of the model is achieved by considering not only continuous latent variables, but also categorical latent variables.

• This makes it possible to unify and to extend the following analyses: classical SEM, growth curve modeling, multi-level modeling, missing data modeling, finite mixture modeling, latent class modeling and survival modeling.

• The general framework is represented by a square D, and is a combination of 3 special cases (ellipses):

• A) continuous latent variables: includes measurement error, measurement invariance in conventional SEM, latent variables in growth modeling and variance components in multi-level modeling.

• B) categorical latent variables: includes latent class analysis and latent class growth analysis.

• C) latent profile models and models that combine continuous and categorical latent variables such as growth mixture modeling.

• D) new types of models including modeling with missing data on a categorical latent variable in randomized trial (like modeling under MNAR ).