understanding student achievement: the value of administrative data
DESCRIPTION
Understanding Student Achievement: The Value of Administrative Data. Eric Hanushek Stanford University. Big Issues in School Policy Debates. Relating analysis to policy interests Confidence in causation Generalizability. Analytical designs. Random assignment experiments - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/1.jpg)
Understanding Student Achievement: The Value of Administrative Data
Eric HanushekStanford University
![Page 2: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/2.jpg)
Big Issues in School Policy Debates
Relating analysis to policy interests
Confidence in causation
Generalizability
![Page 3: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/3.jpg)
Analytical designs
Random assignment experiments Natural experiments “Data solutions”
Trade-offs Credibility Expense Questions that can be addressed
![Page 4: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/4.jpg)
UTD Texas Schools Project
Multiple cohorts followed 1993-2002
Annual achievement in grades 3-8 (TAAS math and reading)
Each cohort > 200,000 students in over 3,000 schools
Augmented with district data
![Page 5: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/5.jpg)
Examples of Topics Teacher quality variations Charter schools
Not discussed School choice and mobility Special education Teacher mobility Racial composition Peer achievement
![Page 6: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/6.jpg)
Existing Evidence on Teacher Quality
Substantial variation in teacher quality
Observable characteristics of teachers explain little of the variation
Salary and other factors affect teacher transition probabilities
No evidence on transitions and teacher quality
![Page 7: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/7.jpg)
Questions Addressed
What is variation in teacher quality? Measurable characteristics?
Do urban schools lose their best teachers? Quality by transitions
Do districts hire the best teachers?
![Page 8: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/8.jpg)
Basic model
standardized gain
( , )
isg
j
G
f X S
j j
![Page 9: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/9.jpg)
Measurement Error and Calculation of Variance of Teacher Quality
Observe teachers in two years:
Correlation across years:
(1) (2),j j
12( ) var( ) / var( )E r
![Page 10: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/10.jpg)
Estimated Variance in Teacher QualityLonestar District
Within districtWithin school
and year
unadjusteddemographic
controlsunadjusted
demographic controls
Teacher-year variation
0.210 0.179 0.109 0.104
Adjacent year correlation
0.500 0.419 0.458 0.442
Teacher quality variance / (s.d.)
0.105(0.32)
0.075(0.27)
0.050(0.22)
0.047(0.22)
![Page 11: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/11.jpg)
Kernal Density Estimates of Teacher Quality Distribution: Standardized Average Gains by Teacher Move Status
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
-2 -1.8 -1.6 -1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8
Relative teacher quality (s.d.)
Stays at Campus Campus Change District Change Out of Public Education
![Page 12: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/12.jpg)
Conclusions on Teacher Quality Very large differences among teachers
Differences within schools much larger than between schools
Conventional measures not good index of quality (master’s degree, certification test)
Observable characteristics First year of experience Teacher-student race match
Common assumptions about market for teachers not correct Best do not leave Districts with advantages do not use them
![Page 13: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/13.jpg)
Popularity of charter schools
3,000 charter schools
40 states plus DC since 1991
1 percent of total students
10 percent of size of private school market
7+ percent rate of closure
![Page 14: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/14.jpg)
Evaluation issues
Most analysis of entry and participation
No reliable information on performance
Difficulty of selection issue
Very political
![Page 15: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/15.jpg)
Evaluation approaches
Model selection process [Heckman (1979)]
Instrument for attendance [Neal(1997)]
Intake randomization [Howell and Peterson (2002)]
![Page 16: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/16.jpg)
Difficulties with traditional approaches
Difficult to find factors affecting attendance but not achievement
Cannot handle treatment heterogeneity
![Page 17: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/17.jpg)
Empirical framework Mean differences in individual value-added
Identify charter school from individual entry-exit Consider time varying effects associated with
charter school movements
Heterogeneity across schools
Consumer responsiveness to quality
![Page 18: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/18.jpg)
Charter enrollment
1997 2001
4th grade 0.2 % 0.8%
7th grade 0.2% 0.9%
![Page 19: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/19.jpg)
Participation rates by race/ethnicity
1997 2001
Blacks 0.8% 2.2%
Hispanics 0.1% 0.6%
Whites 0.0% 0.4%
Low income 0.3% 0.8%
![Page 20: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/20.jpg)
Charters by vintage (analytical)
1997 1998 1999 2000 2001 2002 Total
one 17 10 70 83 43 47 270
![Page 21: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/21.jpg)
Charters by vintage (analytical)
1997 1998 1999 2000 2001 2002 Total
one 17 10 70 83 43 47 270two 2 16 9 69 78 40 214
![Page 22: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/22.jpg)
Charters by vintage (analytical)
1997 1998 1999 2000 2001 2002 Total
one 17 10 70 83 43 47 270two 2 16 9 69 78 40 214Three 0 2 15 8 68 73 166Four 0 1 2 15 8 66 92Five+ 0 0 1 3 17 22 43
![Page 23: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/23.jpg)
Charter school effect
Charter -0.17
Age 1 -0.33
Age 2 -0.25
Age 3 -0.08
Age 4 0.00
Age 5 or more 0.02
![Page 24: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/24.jpg)
Demographically Adjusted School Quality
Residual-Based Quality Measure
Charter Non-Charter
-1.57744 .788794
0
2.32064
![Page 25: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/25.jpg)
Do parents make good decisions?
Parents cannot see value added Considerable mobility/exiting
Models: Exit=f(quality, age, year, race, grade)
![Page 26: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/26.jpg)
Parental Choice(linear probability of exit)
Student characteristics
Student + peer
characteristics
Student + peer
characteristics + peer
achievement
School quality
0.002 0.006 0.006
School quality x charter
-0.152 -0.142 -0.138
![Page 27: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/27.jpg)
Parental Choice(linear probability of exit)
Student characteristics
Student + peer
characteristics
Student + peer
characteristics + peer
achievement
Student + peer
characteristics + peer
achievement
School quality
0.002 0.006 0.006
School quality x charter
-0.152 -0.142 -0.138
high income-0.187
low income-0.096
![Page 28: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/28.jpg)
Conclusions on Charter Schools
Difficult start-up period Mean performance regular ≈ charter
after two years Heterogeneity in both markets Parents react to quality in charter
market Low income reaction one half upper
income
![Page 29: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/29.jpg)
Administrative data
Pros Broader generalizability Understanding heterogeneity Perhaps less costly
Cons Requires structure (e.g., linearity, time pattern of
achievement) Regulatory problems (confidentiality) Data quality issues
![Page 30: Understanding Student Achievement: The Value of Administrative Data](https://reader038.vdocuments.net/reader038/viewer/2022110102/568134d5550346895d9c002c/html5/thumbnails/30.jpg)
Papers on Teacher Quality and Charter Schools
www.hanushek.net or www.nber.org
Hanushek, Eric A., John F. Kain, Daniel M. O'Brien, and Steve G. Rivkin. 2005. "The market for teacher quality." National Bureau of Economic Research, Working Paper No. 11154, (February).
Hanushek, Eric A., John F. Kain, Steve G. Rivkin, and Gregory F. Branch. 2005. "Charter school quality and parental decision making with school choice." National Bureau of Economic Research, (March).