hlm sswr 2004

8/8/2019 HLM SSWR 2004

http://slidepdf.com/reader/full/hlm-sswr-2004 1/27

Four Innovative Applicationsof Hierarchical Linear

Modeling (HLM)

Shenyang Guo, Ph.D.

University of North Carolina at Chapel [email protected]

8/8/2019 HLM SSWR 2004


Acknowledgment

Support for this research is provided by theDiscretionary Grants Program of Children¶sBureau to Shenyang Guo (PI). The projectaims to develop innovative quantitativemethods for child welfare research.

8/8/2019 HLM SSWR 2004


O verview of HLMj Why HLM? The need to study multilevel influences

on an outcome variable, and to run growth curveanalysis.

j Central problem: intraclass correlation.j Conceptually one may view HLM as running

regression model several times, or at 2 or 3 levels.j O ther names:multi-level analysis, mixed-effects

model, random-effects model, growth curve analysis,random-coefficient regression model, covariance

components model.j The key idea is to estimate random effects. Inaddition to traditional regression coefficients, HLMestimates a set of random effects associated with eachhigh-level unit, which can be used to control for autocorrelation.

8/8/2019 HLM SSWR 2004


Four innovative models

1. Latent-variable analysis of HLM2. O mnibus score of CBCL & TRF using

latent-variable HLM3. Meta analysis using HLM4. Modeling multivariate change

8/8/2019 HLM SSWR 2004


Latent-variable analysis (1)

j Latent variables: variables that are not directlyobserved. Under this framework, any observedvariable is an indicator, and can be viewed as alatent true-score plus measurement error.

j Statistical models for analyzing latent variables:structural equation modeling: (1) measurementmodel ± relations between indicator and latent

variable; (2) structural model ± relations amonglatent variables.

j In HLM, a latent-variable analysis consists of two parts: measurement model, and structural

model involving explanatory variables.

8/8/2019 HLM SSWR 2004


Latent-variable analysis (2)

Examplej Sampson, Raudenbush, & Earls (1997, Science277 (15): 918-924) applied this approach to analyzingmultilevel influences of collective efficacy, in which

they view collective efficacy as a latent variable. Their three-level HLM treats ten items collected from allsurvey respondents as level 1, and conceptualizesthat these items are commonly determined by a latenttrue score ³collective efficacy´ plus measurement

errors. Their model then explores how informantswithin neighborhoods (i.e., level 2) vary randomlyaround the neighborhood mean of ³collectiveefficacy´, and how neighborhoods across whole studyarea (i.e., level 3) vary randomly about the grandmean of ³collective efficacy´.

8/8/2019 HLM SSWR 2004


O mnibus score of CBCL & TRF (1)j Disentangle multiple raters¶ measurement error

from clients true change (Guo & Hussey, 1999,Social Work Research 23(4): 258-269).

j Ratings are likely to be collected by multiple

raters (e.g., Achenbach instruments: CBCL, TRF,& YSR).

j Attritions can also occur in raters.j None of the prior studies (before 1999) ever

controlled for raters¶ impact on ratings, thoughmany used multiple raters to collect ratings.

j A theoretical framework to investigate multiplesources of measurement error: Cronbach¶s

Generalizability Theory.

8/8/2019 HLM SSWR 2004


The Need: Hypothetical DataTwo raters¶ ratings on a single subject

1a 1b 1c

0

10

20

30

40

50

60

70

0 2 4 6 8 10

Time

Y

Rater A

Rater B

0

10

20

30

40

50

60

70

0 2 4 6 8 10

Time

Y

0

10

20

30

40

50

60

70

0 2 4 6 8 10

Time

Y

0

10

20

30

40

50

60

70

0 2 4 6 8 10

Time

Y1d

O mnibus score of CBCL & TRF (2)

8/8/2019 HLM SSWR 2004


Problem Type of

Data

Causes of the Problem Solution Major Results

Divergent ratingsmade by caregiver,teacher, and youthself

Both cross-sectional &longitudinal

Three versions of ratingforms: CBCL, TRF, &YSR

A three-levelHLM with latent-variable analysis;or Guo & Hussey¶sthree-level HLM

y Both models estimate a ³truescore´ (an omnibus score basedon multiple ratings) for eachstudy child.

y Both facilitate a multivariateanalysis identifying significantpredictors of true-scoredifferences in the study sample.

Attrition of raters or missing ratings

Longitudinal A teacher may change jobin a longitudinal study andmake herself no longer amember of the TRFcollection team. A caregiver may miss oneor more CBCL collections. A youth may miss one or more YSR collections.

Guo & Hussey¶sthree-level HLM

y The model estimates a ³truechange trajectory´ (an omnibustrajectory based on all availableratings) for each study child;

y The model facilitates an inter -individual analysis that identifiessignificant predictors of the overalltrajectory of the study sample.


Problems & solutions

8/8/2019 HLM SSWR 2004



Level 1:

ijk ijk jk ijk D R I ET !

Level 2: jk jk k jk k k jk r X ATE X ATE 00 20 100 )2 _ ()1 _ (! F F FT

Level 3:k k k k uW CHILDW CHILD 0000 200 100000 )2 _ ()1 _ (! K K K F

F00 k is the omnibus score

8/8/2019 HLM SSWR 2004



Model 1

Level 1: ijk ijk jk ijk I T !

evel 2: jk k jk r 000! FT

evel 3: k k u 0000000 ! K F


8/8/2019 HLM SSWR 2004



Model 2Level 1:

ijk ijk jk ijk I T ! Level 2:

jk k jk r 000! FT

Level 3 :k k qk k uWqCHILDW CHILD 000000 100000 ) _ (...)1 _ (! K K K F


8/8/2019 HLM SSWR 2004



Illustrating examplej Acknowledgment to Dr. Richard Barth and Ms.

Ariana Wall at UNC for their help.j Data: National Survey of Child and Adolescent Well-

being (NSC AW).j We focus on externalizing and internalizing scores.

Each child has four such scores: two from caregiver (CBCL), and two from teacher (TRF). The task: howto create one score?

j Variables employed in level 3 of Model 2: agegender, race, social behavior, MB A reading score,MB A math score, count of risky behaviors of delinquency, count of risky behaviors of substanceabuse, and count of risky behavior of suicidalattempt.

8/8/2019 HLM SSWR 2004



__________________________________________________________________________ Ec Et Ic It

_______ _______ _______ _______ Externalizing rated by caregiver (E c) 1.000Externalizing rated by teacher (E t) .405** 1.000Internalizing rated by caregiver (I c) .639** .142** 1.000Internalizing rated by teacher (I t) .174** .476** .196** 1.000

Mean (S.D.) 60.8 (11.52) 58.7 (9.60) 57.3 (11.91) 56.0 (9.69)

__________________________________________________________________________ ** p < .01

Correlation coefficients and descriptivestatistics on disagreement betweencaregiver and teacher¶s scores (N=448)

8/8/2019 HLM SSWR 2004



Evaluation Schemes:C1 Caregiver's scores only.5Ec + .5Ic

C2 Teacher's scores only.5Et + .5It

C3 All 4 scores from both versions with equal weights.25Ec + .25Ic + .25Et + .25It

C4 Similar to C3 but heavier weights giving to caregiver's sc.35Ec + .35Ic + .15Et + .15It

C5 Similar to C3 but heavier weights giving to teacher's score.15Ec + .15Ic + .35Et + .35It

C6 Similar to C3, a 5 0/ 50 split between caregiver and teacher scores but heavier weights giving to externalizing scores

.35Ec + .15Ic + .35Et + .15It

8/8/2019 HLM SSWR 2004


8/8/2019 HLM SSWR 2004


O mnibus score of CBCL & TRF (1 0 )

Evaluations ___________________________________________________________________________

Scheme Mean S.D. Minimum Maximum Correlation Coefficient ____________________________with Omnibus 1 with Omnibus 2

__________ ________ ________ ________ ________ ______________ ______________

Omnibus 1 60.56 3.29 52.30 69.21Omnibus 2 60.56 4.78 49.34 72.81 .707C1 59.03 10.61 32.00 82.50 .855 .700C2 57.32 8.29 39.50 80.00 .747 .406C3 58.18 7.63 39.00 78.25 1.000 .707C4 58.52 8.49 37.00 79.35 .966 .731C5 57.84 7.39 39.40 77.15 .955 .620

C6 58.79 7.90 39.30 77.60 .979 .702C7 57.57 7.69 37.20 79.35 .978 .681C8 53.00 8.22 32.00 74.00 .929 .656C9 63.36 8.21 41.50 82.50 .929 .657C10 57.74 8.09 37.50 76.50 .961 .698C11 58.62 7.81 39.00 81.00 .958 .658

___________________________________________________________________________

All correlation coefficients are statistically significant (p<.01)

8/8/2019 HLM SSWR 2004


O mnibus score of CBCL & TRF (11)Use the score as a dependent variable

______________________________________

Scheme Employed the score crea tedby the scheme

as an ou tcome variable

R2

______________________________________

Omnibus 1 .388

Omnibus 2 .987

C1 .431

C2 .147

C3 .388

C4 .434

C5 .293

C6 .388

C7 .361

C8 .337

C9 .337

C10 .383

C11 .332

______________________________________

8/8/2019 HLM SSWR 2004


8/8/2019 HLM SSWR 2004


Meta analysis using HLM (2)

Based on these data, calculate effect size:

And variance of the effect size:

Square root of V j is called ³standard error of d j´

jCj E j j S Y Y d /)(!

)](2/[)/()( 2C j Ej jC j EjC j Ej j nnd nnnnV !

8/8/2019 HLM SSWR 2004


Meta analysis using HLM (3)j General model

Level 1: d j = H j + e j

Level 2: H j = K0 + u j

or combined model: d j = K0 + u j + e j

where d j ~N( K0, ( j) with ( j = X+ V jj In this model, we only have one subscript j to

indicate study. This is a special case of two-

level model, in which subscript i is omitted,because we don¶t have original data at the studysubject level.

j V-known model: unlike previous HLM, this

model has known variance V j,or S.E.(d j)= V j.

8/8/2019 HLM SSWR 2004


Meta analysis using HLM (4)j Use HLM DOS version to run the v-

known model.j Data look like this:

1 .0 30 .0 16 2. 000

2 .12 0 .0 22 3. 000

3 -.14 0 .0 28 3. 000

««

19 -. 0 70 .0 30 3.000

Format of the raw data file: (a11,3f11.3)See HLM 5 manual pp. 221-226.

8/8/2019 HLM SSWR 2004



Ex peri m ental Studies of Teacher Ex pectancy E ffects on Pupil IQ

Study

E ffectSize

E sti m ated ¡

StandardE rror of

d ¡

Wee ks of Prior

Contact

1. Rosenthal et al. (1974) 0.030 0.126 2.0002. Conn et al. (1968) 0.120 0.148 3.0003. Jose & Cody (1971) -0.140 0.167 3.0004. Pellegrini & Hicks (1972) 1.180 0.373 0.0005. Pellegrini & Hicks (1972) 0.260 0.369 0.0006. Evans & Rosenthal (1969) -0.060 0.105 3.0007. Fielder et al. (1971) -0.020 0.105 3.0008. Claiborn (1969) -0.320 0.219 3.0009. Kester & Letchworth (1972) 0.270 0.164 0.000

10. Maxwell (1970) 0.800 0.251 1.00011. Carter (1970) 0.540 0.302 0.00012. Flowers (1966) 0.180 0.224 0.00013. Keshock (1970) -0.020 0.290 1.00014. Henrickson (1970) 0.230 0.290 2.00015. Fine (1972) -0.180 0.158 3.00016 reiger (1970) -0.060 0.167 3.00017. Rosenthal & Jacobson (1968) 0.300 0.138 1.00018. Fleming & Anttonen (1971) 0.070 0.095 2.000

19. insburg (1970) -0.070 0.173 3.000

8/8/2019 HLM SSWR 2004



Running HLM, we obtain the following findings:The estimated grand-mean effect size is0.084, implying that, on average, experimentalstudents scored about .084 standard deviation

units above the controls.However, the estimated variance of theeffect parameter is =.019. This correspondsto a standard deviation of .138 (i.e., .019 =

.138), which implies that important variabilityexists in the true-effect sizes. For example, aneffect one standard deviation above theaverage would be .084+.138=.222, which is of

nontrivial magnitude.

8/8/2019 HLM SSWR 2004


I n a cross-sectional study, we usecorrelation coefficients to see the level of association of an outcome variable with othervariables.

I n a longitudinal study, we have a similartask, that is, we need to model multivariatechange: whether two change trajectories(outcome measures) correlate over time?For details of this method, see MacCallum,

R.C., & Kim, C. (2000). Modelingmultivariate change , in Little, Schnabel, &Baumert edited, Modeling Longitudinal andMultilevel Data. Lawrence Erlbaum

Associates, pp.51-68.

Multivariate change (1)

8/8/2019 HLM SSWR 2004


W hat kind of questions can beanswered?

W hether benefits clients gained from anintervention over time negatively correlatewith the intervention s side effects?

W hether clients change in physical healthcorrelates with their change in mental health?

W hether a program s designed change inoutcome (e.g., abstinence from alcohol orsubstance abuse) correlates with clients levelof depression?

Use software MLn/MLwiN to estimatethe model. I t s possible to use SAS ProcMixed.

Multivariate change (2)

hlm sswr 2004

Documents