meta-analysis using hlm 6.0
DESCRIPTION
Meta-Analysis using HLM 6.0. Yaacov Petscher Florida Center for Reading Research. Why use HLM?. Nested structure Necessity of special models Variation at both subject and study levels. “Special Case” of HLM. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/1.jpg)
Meta-Analysis using HLM 6.0
Yaacov Petscher
Florida Center for Reading Research
![Page 2: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/2.jpg)
Why use HLM?
• Nested structure
• Necessity of special models– Variation at both subject and study levels
![Page 3: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/3.jpg)
“Special Case” of HLM
• If ES are based on n ≥ 30, we assume approximate normal distribution with sampling variance assumed to be known
• V-known models run in Interactive Mode– Time to brush up on your DOS command
code!
![Page 4: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/4.jpg)
Standardized Mean Difference
• No raw data for us– Must rely on stats to be converted to single
metric
• Many types of statistics that may be usedZ t M/SD
χ² F p-value
r r²
![Page 5: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/5.jpg)
Level-1 (Within-Studies) Model
jjj ed
jd = any standardized effect measure from study j
j = the corresponding population parameter
je = sample error associated with d
![Page 6: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/6.jpg)
Level-2 (Between-Studies) Model
jsjsj uW 0
0 = grand mean effect size
s = regression coefficients
sjW = study characteristics (moderators)
ju = level 2 random error
![Page 7: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/7.jpg)
Combined Model
s
jjssj euWd 0
![Page 8: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/8.jpg)
Estimation
s
sjsjjjj Wd )ˆˆ)(1( 0*
Empirical Bayes Estimator
)/( jj V where
![Page 9: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/9.jpg)
EB Estimates
• Level 1– May be used as a shrinkage estimator to
identify potential outliers • Shrinkage in the direction of the grand mean
• Level 2– Supplying the grand mean provides an
estimate of the conditional shrinkage • Shrinkage towards a value that is conditional on
the amount of prior contacts (WEEKS)
![Page 10: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/10.jpg)
The following example will be run using data from Raudenbush & Bryk
Chapter 7 data (pg 211)
The Effect of Teacher Expectancy on Pupil IQ
![Page 11: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/11.jpg)
Study Week s ES Std Error Unconditional Model Conditional Model1 2 0.03 0.125 0.05 0.092 3 0.12 0.147 0.10 -0.063 3 -0.14 0.167 0.00 -0.064 0 1.18 0.373 0.22 0.415 0 0.26 0.369 0.11 0.416 3 -0.06 0.103 -0.01 -0.067 3 -0.02 0.103 0.02 -0.068 3 -0.32 0.22 -0.03 -0.069 0 0.27 0.164 0.16 0.4110 1 0.8 0.251 0.25 0.2511 0 0.54 0.302 0.16 0.4112 0 0.18 0.223 0.11 0.4113 1 -0.02 0.289 0.06 0.2514 2 0.23 0.29 0.11 0.0915 3 -0.18 0.159 -0.03 -0.0616 3 -0.06 0.167 0.03 -0.0617 1 0.3 0.139 0.19 0.2518 2 0.07 0.094 0.07 0.0919 3 -0.07 0.174 0.02 -0.06
Empirical Bayes Estimates
![Page 12: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/12.jpg)
Some Calculations
• Need the Conditional Variances– Since d in this model is Fisher’s r to Z transformation,
the formula is
)3(
1
n
vi
Since we’re not given n we need to calculate another way…..ideas?
![Page 13: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/13.jpg)
YAY!
• Simply square your standard errorStudy ES Std Error
1 0.03 0.1252 0.12 0.1473 -0.14 0.1674 1.18 0.3735 0.26 0.3696 -0.06 0.1037 -0.02 0.1038 -0.32 0.2209 0.27 0.16410 0.8 0.25111 0.54 0.30212 0.18 0.22313 -0.02 0.28914 0.23 0.29015 -0.18 0.15916 -0.06 0.16717 0.3 0.13918 0.07 0.09419 -0.07 0.174
Study ES Std Error Vj1 0.03 0.125 0.01562 0.12 0.147 0.02163 -0.14 0.167 0.02794 1.18 0.373 0.13915 0.26 0.369 0.13626 -0.06 0.103 0.01067 -0.02 0.103 0.01068 -0.32 0.220 0.04849 0.27 0.164 0.026910 0.8 0.251 0.063011 0.54 0.302 0.091212 0.18 0.223 0.049713 -0.02 0.289 0.083514 0.23 0.290 0.084115 -0.18 0.159 0.025316 -0.06 0.167 0.027917 0.3 0.139 0.019318 0.07 0.094 0.008819 -0.07 0.174 0.0303
![Page 14: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/14.jpg)
Data File Prep Considerations
• Since meta-analysis in HLM is a V-known model, only one data file is used
Study ES Vj Week s1 0.03 0.0156 2.0002 0.12 0.0216 3.0003 -0.14 0.0279 3.0004 1.18 0.1391 0.0005 0.26 0.1362 0.0006 -0.06 0.0106 3.0007 -0.02 0.0106 3.0008 -0.32 0.0484 3.0009 0.27 0.0269 0.00010 0.8 0.0630 1.00011 0.54 0.0912 0.00012 0.18 0.0497 0.00013 -0.02 0.0835 1.00014 0.23 0.0841 2.00015 -0.18 0.0253 3.00016 -0.06 0.0279 3.00017 0.3 0.0193 1.00018 0.07 0.0088 2.00019 -0.07 0.0303 3.000
![Page 15: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/15.jpg)
Data File Prep Considerations, cont
• Four key features to data prep (assume using SPSS)– Column 1 = ID in character format– Column 2 = ES estimates – Column 3 = Variance estimates– Column 4-n = Potential level-2 predictors
![Page 16: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/16.jpg)
Formatting
• Variable View– Column 1
• String, width = 2, decimal = 0
– Columns 2-n• Numeric, width = 12, decimal = 3
– Save as a Fixed ASCII (.dat) file– Hold onto your output, you’re gonna need it!
• You should now have a list of the variables and associated Format statements
![Page 17: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/17.jpg)
HLM in Batch Mode
• Bring up your computer’s Command Prompt– Typically found in “Accessories”
• By default, you should see C:\>– If not, type c: and hit enter
• At this point you want to locate HLM– Type dir, hit enter
• cd program files, enter• cd HLM6, enter• hlm2, enter
– We’re now ready to begin!
![Page 18: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/18.jpg)
HLM2- MDM File CreationC:\Program Files\HLM6\hlm2
Will you be starting with raw data? yIs the input file a v-known file? yHow many level-1 statistics are there? 1How many level-2 predictors are there? 1 Enter 8 character name for level-1 variable number 1: Zes
Enter 8 character name for level-2 variable number 1: weeks Input format of raw data file (the first field must be the character ID) format: (a2, 3f12.3) What file contains the data: e:\test.dat
Enter name of MDM file: e:\test.mdm 19 groups have been processed
C:\Program Files\HLM6>
![Page 19: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/19.jpg)
Specifying UC HLM ModelC:\HLM6> hlm2 e:\test.mdm
SPECIFYING AN HLM MODEL Level-1 predictor variable specification
Which level-1 predictors do you wish to use? The choices are: For ZES enter 1
Level-1 predictors? (Enter 0 to end) 1
Level-2 predictor variable specification
Which level-2 variables do you wish to use? The choices are: For WEEKS enter 1
Which level-2 predictors to model ZES? Level-2 predictor? (Enter 0 to end) 0
![Page 20: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/20.jpg)
ADDITIONAL PROGRAM FEATURES
Select the level-2 variables that you might consider for Inclusion as predictors in subsequent models. The choices are: For WEEKS enter 1
Which level-2 variables to model ZES? Level-2 variable? (Enter 0 to end) 0Do you wish to use any of the optional hypothesis testing procedures? n
OUTPUT SPECIFICATION
Do you want a level-2 residual file? nHow many iterations do you want to do? 10000Do you want to see OLS estimates for all of the level-2 units? n Enter a problem title: lvl1 Enter name of output file: e:\lvl1.lis
![Page 21: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/21.jpg)
Results for UC Model ----------------------------------------------------------------------------------------- Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. ----------------------------------------------------------------------------------------- For ZES, B1 INTRCPT2, G10 0.084376 0.052039 1.621 18 -----------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------- Random Effect Standard Variance df Chi-square P-value Deviation Component ------------------------------------------------------------------------------------------------ ZES, U1 0.13896 0.01931 18 36.25115 0.007 ------------------------------------------------------------------------------------------------
Significant variability exists in true-effect sizes
![Page 22: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/22.jpg)
EB Estimation Level 1
s
sjsjjjj Wd )ˆˆ)(1( 0*
Since there are no predictors at Level 1, the last term is omitted, leaving us with
0* ˆ)1( jjjj d
![Page 23: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/23.jpg)
Using Excel to Calculate EB
)/( jj V ES Variance
0.03 0.01560.12 0.0216
-0.14 0.02791.18 0.13910.26 0.1362
-0.06 0.0106-0.02 0.0106-0.32 0.04840.27 0.02690.80 0.06300.54 0.09120.18 0.0497
-0.02 0.08350.23 0.0841
-0.18 0.0253-0.06 0.02790.30 0.01930.07 0.0088
-0.07 0.0303
ES Variance lambda0.03 0.0156 0.5487360.12 0.0216 0.467877
-0.14 0.0279 0.4052121.18 0.1391 0.1201550.26 0.1362 0.122453
-0.06 0.0106 0.641697-0.02 0.0106 0.641697-0.32 0.0484 0.2818990.27 0.0269 0.4139790.80 0.0630 0.2317040.54 0.0912 0.1724080.18 0.0497 0.276448
-0.02 0.0835 0.1853280.23 0.0841 0.184287
-0.18 0.0253 0.429078-0.06 0.0279 0.4052120.30 0.0193 0.4958120.07 0.0088 0.682569
-0.07 0.0303 0.385583
Using the variance component fromlvl 1 Model, create Lambda usingthe formula function
![Page 24: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/24.jpg)
0* ˆ)1( jjjj d
ES Variance lambda Ebdelta0.03 0.0156 0.548736 0.050.12 0.0216 0.467877 0.10
-0.14 0.0279 0.405212 -0.011.18 0.1391 0.120155 0.210.26 0.1362 0.122453 0.10
-0.06 0.0106 0.641697 -0.01-0.02 0.0106 0.641697 0.02-0.32 0.0484 0.281899 -0.030.27 0.0269 0.413979 0.160.80 0.0630 0.231704 0.250.54 0.0912 0.172408 0.160.18 0.0497 0.276448 0.11
-0.02 0.0835 0.185328 0.060.23 0.0841 0.184287 0.11
-0.18 0.0253 0.429078 -0.03-0.06 0.0279 0.405212 0.030.30 0.0193 0.495812 0.190.07 0.0088 0.682569 0.07
-0.07 0.0303 0.385583 0.02
Supplying the grand mean ESinto the Excel formula function allows us get the EB estimates
See the difference in results?
![Page 25: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/25.jpg)
-0.40
-0.20
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
ES Ebdelta
![Page 26: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/26.jpg)
Specifying CL2 HLM ModelC:\HLM6> hlm2 e:\test.mdm
SPECIFYING AN HLM MODEL Level-1 predictor variable specification
Which level-1 predictors do you wish to use? The choices are: For ZES enter 1
Level-1 predictors? (Enter 0 to end) 1
Level-2 predictor variable specification
Which level-2 variables do you wish to use? The choices are: For WEEKS enter 1
Which level-2 predictors to model ZES? Level-2 predictor? (Enter 0 to end) 1
![Page 27: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/27.jpg)
ADDITIONAL PROGRAM FEATURES
Select the level-2 variables that you might consider for Inclusion as predictors in subsequent models. The choices are: For WEEKS enter 1
Which level-2 variables to model ZES? Level-2 variable? (Enter 0 to end) 0Do you wish to use any of the optional hypothesis testing procedures? n
OUTPUT SPECIFICATION
Do you want a level-2 residual file? nHow many iterations do you want to do? 10000Do you want to see OLS estimates for all of the level-2 units? n Enter a problem title: lvl2 Enter name of output file: e:\lvl2.lis
![Page 28: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/28.jpg)
Results for CL2 Model
------------------------------------------------------------------------------------------------- Standard Approx. Fixed Effect Coefficient Error T-ratio d.f. P-value ------------------------------------------------------------------------------------------------- For ZES, B1 INTRCPT2, G10 0.408572 0.087146 4.688 17 0.000 WEEKS, G11 -0.157963 0.035943 -4.395 17 0.000 -------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------ Random Effect Standard Variance df Chi-square P-value Deviation Component ------------------------------------------------------------------------------------------------ ZES, U1 0.00283 0.00001 17 16.53614 >.500 ------------------------------------------------------------------------------------------------
![Page 29: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/29.jpg)
EB Estimation Level 2
Since then = 0 and we’re left with 0ˆ )/( jj V
jj WEEKS)(ˆˆ 10*
ES Weeks EBLvl20.03 2 0.090.12 3 -0.06
-0.14 3 -0.071.18 0 0.410.26 0 0.41
-0.06 3 -0.07-0.02 3 -0.07-0.32 3 -0.070.27 0 0.410.80 1 0.250.54 0 0.410.18 0 0.41-0.02 1 0.250.23 2 0.09
-0.18 3 -0.07-0.06 3 -0.070.30 1 0.250.07 2 0.09
-0.07 3 -0.07
Using G10 and G11, we can calculatethe EB estimates for Level 2
![Page 30: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/30.jpg)
-0.40
-0.20
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
ES EBLvl2
![Page 31: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/31.jpg)
EB Estimation Level 2
• Since our Level-2 predictor takes on one of four different values, the shrinkage is towards one of the four points.
![Page 32: Meta-Analysis using HLM 6.0](https://reader036.vdocuments.net/reader036/viewer/2022062519/56815209550346895dc04a23/html5/thumbnails/32.jpg)
End(for now)