human capital policies in education: further research on teachers and principals 5 rd annual calder...

Human Capital Policies in Education: Further Research on Teachers and Principals

5rd Annual CALDER ConferenceJanuary 27th, 2012

Principal Quality and the Persistence of School Policies

Sarah Cannon, NorthwesternDavid Figlio, Northwestern/CALDER

Tim Sass, Georgia State/CALDER

Introduction

• Considerable energies over the past few years in quantifying the effects of principals– Increasing evidence that principals, like teachers, vary

dramatically in quality• On the other hand, while we know about some

management attributes of effective principals, there are few policy/practice choices that they have in common

• Our question: Do principals bring their policies and practices with them as they change schools, and might this help to explain whether they continue to be effective in their new environments?

Research questions

• (1) Is principal quality transferrable? That is, do principals who are successful in school A continue to be successful in school B?

• (2) Do principals who change schools tend to take their policies and practices with them?

• (3) Can the answer to (2) help to explain our findings in (1)?

• Data: merged student and educator longitudinal files in Florida with repeated censuses of surveys of school policies and practices (1999-2000; 2001-02; 2003-04; 70%+ response rate each wave)

Preliminary conclusions

• (1) Is principal quality transferrable? That is, do principals who are successful in school A continue to be successful in school B? Yes – but the spillover is modest, and decreases further when the old school and new school are more different.

• (2) Do principals who change schools tend to take their policies and practices with them? Yes – and it appears that they are about as likely to port their policies when the schools are different as they are when the schools are similar.

• (3) Can the answer to (2) help to explain our findings in (1)? We think so.

Roadmap of the presentation

• (1) Document the degree to which principals who were successful in school A maintain their success in school B, and show how this pattern changes as the “distance” between schools A and B grows.

• (2) Document the degree to which principals who move schools carry their old policies and practices with them, and show how this pattern changes as the “distance” grows.

• (3) Document whether the results found in (2) differ depending on the principal’s measured success in school A.

How to measure effective principals?

• Many possible ways to go; we want to see whether results are consistent across different measures of principal effectiveness

• Because we are looking at whether principal quality in school A transfers to school B, we can’t identify off of school changes, but rather look at measures of principal effectiveness in a given school in a given year

How to measure effective principals?

• Estimate “value-added” model of student achievement, including principal-by-year fixed effects; test score is normed SSS score in Florida

• Three dimensions of principal VA modeling:– Gain on LHS versus achievement level on LHS with lagged

achievement on RHS– Control for teacher time-varying characteristics– Complete sample of schools versus only schools connected

by teachersEight different measures of principal effectiveness in school

A, correlated but not extremely highly

Principal transitions and data

• 34 percent (809) of Florida schools in 2003-04 had a different principal in 2001-02

• 32 percent (258) of THESE schools had a new principal who was a principal at a different school in Florida in 2001-02

• 98 percent (252) of THESE schools had survey data in both 2001-02 and 2003-04

• 84 percent of THESE schools had principals in 2003-04 who completed the survey in 2001-02

• Our sample: 212 school transitions in which the school answered the survey before/after transition and the new principal answered in his/her previous school

Do high VA principals do better in their new school?

Principal VA measure

1 2 3 4 5 6 7 8

Coeff on prior VA in old school

0.068(0.042)

0.101(0.033)

0.066(0.042)

0.101(0.033)

0.083(0.044)

0.104(0.034)

0.081(0.044)

0.104(0.045)

Note: regressions also control for the current school’s prior VA measure in the previous year.

How different are old and new schools?

Are patterns similar for principals with different fixed effects?

PFE measure 1

Are patterns similar for principals with different fixed effects?

PFE measure 2

Are fixed effects similarly persistentwhen old and new schools differ?

Principal VA measure

1 2 3 4 5 6 7 8

Coeff on prior VA in old school

0.236(0.078)

0.296(0.059)

0.237(0.079)

0.299(0.059)

0.237(0.080)

0.297(0.061)

0.237(0.080)

0.299(0.061)

Coeff on prior VA x absolute differ-ence in %FRL

-0.753(0.259)

-0.780(0.183)

-0.773(0.264)

-0.794(0.185)

-0.693(0.262)

-0.775(0.186)

-0.705(0.267)

-0.784(0.189)

Note: regressions also control for the current school’s prior VA measure in the previous year as well as absolute difference in %FRL and absolute difference x school’s prior measure.

Are fixed effects similarly persistent when old and new schools differ?

• Point estimates are larger (in 6 of 8 cases) when the principal moves to a school serving a more affluent population

• However, the difference in coefficients on the interaction is never statistically significant between principals moving to lower %FRL schools and those moving to higher %FRL schools focus just on the interaction with absolute change in %FRL today for convenience

Do principals bring their old policies and practices to their new schools?

• Survey asked 65 questions about instructional policies and practices

• Because of a school budget constraint, and the fact that these are often variations of a theme, we combine these questions into domains

• Domains are weighted averages of individual policy responses, weighted by the variation in the question response

Policy and practice domains

• Policies to improve low-performing students• Lengthening instructional time• Reduced class size for specific subjects• Narrowing of curriculum• Systems of scheduling and class organization• Policies to improve low-performing teachers• Teacher resources• Teacher incentives• Teacher autonomy• Principal control• School climate

Nine domains asked of all principals

• Policies to improve low-performing students• Lengthening instructional time• Reduced class size for specific subjects• Narrowing of curriculum• Systems of scheduling and class organization• Policies to improve low-performing teachers• Teacher resources• Teacher incentives• Teacher autonomy• Principal control• School climate

Example: scheduling systems

• Component questions:– Block scheduling– Common prep periods– Subject matter specialist teachers– Organizing teachers into teams– Looping– Multi-age structure– Other scheduling systems

Example: Policies to improve low-performing students

• Component questions:– Require grade retention– Require summer school– Require before/after school tutoring– Require in-school supplemental instruction– Require tutoring– Require Saturday school– Require other policy

Do principals bring their old policies and practices to their new schools?

Policy/practice domain Coefficient on principal’s domain value at former school

Improve low-performing students 0.050 (0.073)

Lengthening instructional time -0.092 (0.089)

Systems of scheduling/class organization 0.112 (0.081)

Improve low-performing teachers 0.058 (0.067)

Teacher resources 1.063 (0.306)*

Teacher incentives 0.240 (0.075)*

Teacher autonomy 0.300 (0.067)*

Principal control 0.180 (0.076)*

School climate 0.120 (0.082)

Note: models also control for the current school’s domain values in the previous survey.

Do principals bring their old policies and practices to their new schools?

Policy/practice domain Coefficient on principal’s domain value at former school

Coefficient on former value x absolute difference in %FRL

Improve low-performing students 0.062 (0.125) -0.050 (0.424)

Lengthening instructional time -0.078 (0.147) -0.070 (0.492)

Systems of scheduling/class organization 0.032 (0.144) 0.376 (0.501)

Improve low-performing teachers 0.176 (0.107) -0.430 (0.296)

Teacher resources -1.323 (0.466)* 9.091 (1.458)*

Teacher incentives 0.431 (0.118)* -0.800 (0.397)*

Teacher autonomy 0.409 (0.107)* -0.554 (0.387)

Principal control 0.283 (0.116)* -0.455 (0.405)

School climate 0.363 (0.135)* -0.995 (0.465)*

Note: models also control for the current school’s domain values in the previous survey.

Are high value added principals more flexible than low VA principals?

• We investigate whether principals with higher fixed effects at their previous school change their policies more when they move to a school serving a different clientele than do those with lower fixed effects

• We examine the significance level of the three-way interaction between principal’s prior policy value x absolute difference in %FRL between old and new schools x principal’s FE in prior school

• 4 interaction terms positive, 5 negative, only 1 statistically significant (school climate, positive)

Tentative conclusions

• This project is still a work in progress• Our preliminary findings are that:– Principals tend to import their policies and practices

from one school to another– This is the case no matter how different the school

clientele is between the two schools• These findings may help to explain the relatively

weak persistence of measured principal effects when principals change schools (especially when the old and new schools are quite different)