Strategies for estimating the effects of teacher credentials

Helen F. Ladd Based on joint work with Charles Clotfelter

and Jacob Vigdor

CALDER Conference, Oct. 4, 2007

Basic value-added model • Definition:

Ait = a Ait-1 + b TQit + c Xit + errorit

where A = student achievement (i.e. test score) ; and TQ = teacher qualifications

X = control variables

• Justification: Education is a cumulative process a = estimate of persistence of knowledge from one year to the next.

a =1 => complete persistence (no decay)a = 0 => no persistence (100 percent decay)

b = estimate of the effects of the qualifications of the student’s teacher in year t on her achievement in year t.

(Model assumes a and b are constant across years)

Three papers – NC data

• Cross sectional data – fifth graders “Teacher-Student Matching and the Assessment of Teacher Effectiveness”

• Longitudinal data – fourth and fifth graders, multiple cohorts of students“How and Why Do Teacher Credentials Matter for Student Achievement?””

• Course-specific achievement in high school courses – multiple cohorts “Teacher Credentials and Student Achievement in High School: A Cross-Subject Analysis with Student Fixed Effects”

Note. Student achievement is normalized by grade, year and subject so that the mean is 0 and the SD = 1.

Challenges for all three papers

Data – Identification of each student’s teacher Elementary schools – EOG tests

High schools – E0C tests In both cases, we start with proctor of the test but we keep the observation only if we are quite confident that the proctor is the relevant teacher. `(> 75 % match rate in both elementary schools and high schools)

Middle schools – identification not feasible Estimation – Non-random sorting of teachers and students among class rooms.

“Positive” sorting => upward biased coefficients of teacher credentials

Cross sectional model – 5th graders

Strategies to reduce bias of estimates:

• Add an extensive set of student covariates Rich set available in NC data – e.g. education level of parents, T.V. watching

• Include school fixed effects Rules out bias from teacher-student sorting across schools

• Restrict sample to schools with evenly balanced classroom

Reduces bias from sorting across classrooms within schools.

Coefficients of teacher experience - Math (all coefficients are statistically significant.)

Years of experience

(Base = 0 years)

Student covariates

School fixed effects

Restricted sampleWith school fixed effects

1-2 0.058 0.051 0.066

3-5 0.082 0.078 0.080

6-12 0.086 0.076 0.085

13-20 0.077 0.089 0.113

20-27 0.093 0.096 0.103

> 27 0.104 0.090 0.130

Observations 60,656 60,656 25,711

Longitudinal – grades 4 and 5Achievement levels (Ait)

or achievement gains (Ait- Ai,t-1 )

Models 1-3 (of 5) No fixed effects 1. Levels (with prior year achievement) . Upward

biased coefficients because of teacher student matching; potential bias from lagged achievement

With school fixed effects2. Levels. Better but problem of matching within schools

remains and potential bias from lagged achievement; direction of bias unclear (see earlier paper)

3. Gains. Downward bias from misspecified persistence variable

Longitudinal Data (cont.) Models 4 and 5 (preferred)

Full use of the longitudinal aspect of the data With student fixed effects

4. Levels (but no lagged achievement). Lower bound estimates of teacher credentials

5. Gains. Upward bound estimates of teacher credentials

Teacher experienceCoefficients from models 4 and 5

All are statistically significant

Base= no experience

Math Reading

1-2 years 0.057 / 0.072 0.032 / 0.043

3-5 years 0.072 / 0.091 0.046 / 0.064

6-12 years 0.079 / 0.094 0.053 / 0.071

13-20 years 0.082 / 0.102 0.062 / 0.820

21-27 years 0.092 / 0.118 0.067 / 0.096

28+ 0.084 / 0.109 0.062 / 0.089

High school cross-subject analysis Subjects – algebra 1, English I, biology, geometry, ELP

Strategy – at least three test scores for every student; include student fixed effects

Equivalent to estimating:

(Ais-Ai*) = b (TQis-TQi*) + error terms.

Where A* is the mean for the student.

Consider one potentially problematic error term: (eis-ei*).

Think of e as unmeasured student ability. Potential concern if ability for a given student differs by subject AND teachers are distributed in a systematic way by the relative ability of students

Based on empirical tests reported in the paper, we have reasonable confidence in our approach.

Coefficients of teacher experiencein high school courses

Years of experience

(base = 0 years)

Model with student fixed effects

1-2 0.050

3-5 0.061

6-12 0.061

13-20 0.059

21-27 0.062

More than 27 0.043

Cf. rising coefficients with with teacher fixed effects