Do Differences in Schools' Instruction Time Explain International Achievement Gaps? Evidence from Developed and Developing

Countries

Victor LavyHebrew University, University of Warwick, and

NBER

October 2013

Large differences across countries in instructional time in public schooling institutions

• Children aged 15:

– Belgium, France and Greece – Over 1,000 hours per year in secondary schools– England, Luxembourg and Sweden - 750 hours, Spain - 833

• Children aged 7-8:

– England, Greece, France and Portugal - over 800, Spain – 959 – Finland and Norway - less than 600

• These differences also reflected in number of lessons (measured in hours) per week in different subjects, for example children aged 15:

– Denmark: math - 4.0, reading - 4.7, science - 2.8

– France: math - 3.4, reading - 2.5, science - 3.6

– Austria: math - 2.7, reading - 2.4, science - 2.2

– Spain: math - 3.1, reading - 3.2, science - 2.8

Table 1 - Means and Standard Deviations of Instructional Time in OECD, Eastern European, and Developing Countries, 2006

Proportion of pupils by weekly instruction time

Subject Mean Value Std. Dev < 2 Hours 2-3 Hours 4-5 Hours 6 Hours +             

Panel A: 22 OECD Countries

All Subjects 3.38 (1.48) 13.16 40.43 36.45 9.97

Math 3.53 (1.38) 8.72 39.54 43.14 8.60

Science 3.06 (1.57) 21.14 42.72 25.53 10.61

Reading 3.54 (1.44) 9.61 39.02 40.66 10.71                         

Panel C: 13 Developing Countries

All Subjects 3.23 (1.71) 22.86 34.72 27.51 14.90

Math 3.48 (1.69) 18.72 30.73 34.06 16.50

Science 2.97 (1.74) 29.03 37.17 18.53 15.27

Reading 3.24 (1.65) 20.85 36.27 29.94 12.95                         

Mean 513.4 485.6 413.5 3.38 3.05 3.23

Standard Deviation84.4 86.9 75.1 1.02 0.88 1.22between pupils

Standard Deviation38.8 40.9 46.7 1.08 1.28 1.19within pupils

Table 1B : Descriptive Statistics - Test Score and Instructional Time Test scores Instructional time

OECD Develop

Eastern Europe

Developing

Eastern Europe

OECD Develop

Developing

Questions of Interest:

• Can these differences explain differences across countries in achievements in different subjects?

• Can differences in the average productivity of instructional time explain the performance differences between pupils from different countries?

• What characteristics of schools can explain variations in the average productivity of instructional time?

Scientific Background

• There is convincing evidence about the effect of several inputs in the education production function: – Class size, Teachers’ training and certification, Remedial education,

Teachers quality, Computer aided instruction, School choice, Tracking, Gender and ability peer effects, Students’ incentives, Teachers’ incentives….

• There is limited evidence on effect of classroom instructional time.

• Important evidence because Instructional Time can be increased relatively easily.

• There is much scope for such an increase in many countries.

Related literature

• Effects of the length of the school year:– Grogger (1997), Eide and Showalter (1998), in the US, found

insignificant effects– Rizzuto and Wachtel (1980), Card and Krueger (1992), Betts and

Johnson (1998) used State data in US, positive effects– Lee and Barro (2001) examine effect in cross section of countries,

find no effects

– Wößmann (2003), cross-country data, negligible effect – Pishke (2008), use German short school years in 1966-67 as a

natural experiment, find increased repetition, fewer students attending higher secondary school tracks, no adverse effect on earnings and employment later in life.

– Dolton and Marcenaro-Gutierrez (2009), focus on effect of teachers’ salaries but include teaching hours per year. Report inconclusive evidence. With PISA data it is negative or zero.

This Study

• Investigates the causal relationship between instructional time (IT) and pupils' knowledge in math, science and reading.

• Examines factors that explain part of the variation across countries in the average productivity of IT.

• PISA 2006 data for 50+ countries, measured skills of 15-year-olds, variation in

IT across subjects.

• Exploits within student variation in t-scores and IT across subjects.

• Estimate pupil fixed effect models, implicitly also control for family, school, community and country fixed effects.

• Investigate whether the estimated effect of IT varies by certain characteristics of the school: accountability, autonomy, environment, labor market for teachers.

Preview of Findings

• Instructional Time has a positive and significant effect on the academic achievements of 15 years old pupils in OECD countries

• Almost identical results from the Israeli data of pupils in 5 th grade

• Estimates from Eastern European countries are very similar

• In a sample of developing countries much lower effect of Instructional Time, half of the OECD estimate

Preview of Findings

• Overall, the effect is larger for girls, for pupils from disadvantaged backgrounds and for immigrants.

• Effect of instructional time is larger when:– More school accountability measures are adopted – More school autonomy in hiring teachers and determining their

wages, – More school autonomy in using their budget

• Effect of instructional time does not vary with:– School autonomy in pedagogy – Quality of inputs such as computers and other school facilities

Data• PISA “- Program for International Student Assessment, 2000, 2003 and 2006 (58

countries)

• Random sample of 15-year-olds, between 15 years 3 months and 16 years 2 months of age at the time of the test , mostly near end compulsory school

• Measures student performance in reading, mathematics and science literacy, pencil-and-paper tests, both multiple-choice questions and questions requiring students to construct their own responses

• The material is organized around texts and sometimes includes pictures, graphs or tables setting out real-life situations, about seven hours of test material

• From this, each student takes a two-hour test, with the actual combination of test materials different for every student

• The average score among OECD countries is 500 points and the standard deviation is 100 points.

Identification of the Effect of Instructional time• The effect of IT is usually confounded by the effects of unobserved

correlated factors. – If self-selection and sorting of students across schools are affected by school

resources– If there is a correlation between school IT and other characteristics of the school that

may affect students’ outcomes.

• One possible method to account for both sources of confounding factors in the estimation of IT is to rely on within-student variations in IT across subjects:– Examine whether within student differences in t-scores and in school IT are

systematically correlated.

– The basic idea for identification is that the student’s characteristics, average ability, and the school environment are the same for all three subjects except for the fact that some subjects have more instructional time than the other subjects do.

– It could be that at the school level, such variation is not purely random but the cause of such selection does not vary within each student.

– Threats for identification: student subject specific ability and other inputs correlated with subject specific instructional time.

Based on this approach I estimate following equation:

Aijk = µi + γ Hjk + β Xijk + δ Sk + (εj + ηk)+ uijk

Where:

Aijk is the achievment of the ith student in the jth school in the kth subject

Hjk is the instructional time in the jth school in the kth subject

Xijk is a vector of pupil characteristics

Sj is a vector of subject dummies

εj and ηk are unobserved characteristics of the pupil and the school

uijk is the remaining unobserved error term

Table 2 - OLS Regressions of Test Scores on Instructional Time, OECD Sample

Mathematics Science Reading

  (1)   (2)   (3)   (4)   (5)   (6)   (7)   (8)   (9)

I. Continuous Hours:

Hours 21.69 27.98 24.45 26.24 38.36 33.92 4.56 15.43 12.48

(1.03) (1.19) (1.10) (0.80) (0.90) (0.85) (1.00) (1.32) (1.19)

Country dummies

Individual characteristics

Table 3 - Estimated Effect of Instructional Time on Test Scores, OECD SampleWhole Sample

OLS Student FE(1)   (2)  

       

A. Mathematics + Science + ReadingRegression I. Hours of instruction 19.58 5.76

(0.72) (0.37)

Number of students 460,734

      

B. Mathematics + ScienceRegression I. Hours of instruction 25.48 7.14

(0.73) (0.55)

Number of students 307,156

The effect size of the instructional time

• The standard deviation of the within student distribution of instructional time is 1.0

• The standard deviation of the within student test scores distribution is 38.0

• One standard deviation change in the within student distribution of hours will cause an increase of a 5.76 points

• This change is equal to 0.15 of a standard deviation of the within student test score distribution or 0.07 of a standard deviation of the between student test score distribution.

Robustness checks

• Estimated effect of hours is higher in sample of private school

• Results are unchanged when a control is added for subject specific lack of qualified teachers

• Results are very similar when controls are added for extent to which admission to school depends on academic ability

• No significant differences by samples stratified by the importance of academic ability for schools' admission policies [test scores are prerequisite or high priority for admission, test scores are considered for admission, test scores are not considered for admission]

• No variation in effect across samples stratified by school tracking practices

Table 3 - Estimated Effect of Instructional Time on Test Scores, OECD Sample

Whole Sample Sample Divided by School Admission Policy

Academic Record is Irrelevant

Academic Record Taken into Account

  (1)     (2)     (3)

                     

A. Mathematics + Science + Reading

Hours of instruction 5.76 6.01 6.21

(0.37) (0.50) (0.89)

Number of students 460,734 266,769 86,370

Table 4 - Estimated Effect of Instructional Time on Test Scores

by School Tracking Policy

Track in School Track In Class No Tracking

   

Hours of instruction

6.61 6.17 5.17

(0.53) (0.56) (0.68)

Number of students 212,169 201,138 160,188

Table 5 - Estimated Effects of Instruction Time on Test Scores, with Controls Included in the Regressions for Special Science Activities in School and for Scarcity of Teachers in Each

SubjectControl Added For

Special Science School Activities

Scarcity of Teachers in Each Subject

Hours of instruction 5.59 5.75

(0.39) (0.37)

Number of students 460,734 224,508

            

Table 6 - Estimated Effect of Instructional Time on Test Scores, by Gender, OECD SampleBoys Girls

OLSStudent

FE OLS Student FE  (1)   (2)   (3)   (4)               

 

Hours of instruction 20.25 4.99 18.62 5.62(0.86) (0.40) (0.77) (0.41)

Number of students 224,508 236,226

4.83 6.54 6.37 7.62

(0.42) (0.44) (0.88) (0.95)

Hours of Instructional Time

Table 7 - Heterogeneity in Student Fixed Effect Regressions of Test Scores on Instructional Time, OECD Sample.

High Parental Education

Low Parental Education

Immigrants - First

Immigrants - Second

All Boys Girls

High Parental Educatio

Low Parental Educatio

Immigrant 1st Gen.

Immigrant 2nd Gen.

(1) (2) (3) (4) (5) (6) (7)

38.20 38.89 37.25 41.20 33.37 26.35 35.68(1.28) (1.42) (1.38) (1.56) (1.25) (3.32) (2.70)

6.07 5.15 6.49 5.03 6.67 5.53 7.26(0.56) (0.59) (0.59) (0.66) (0.62) (2.07) (1.88)

Number of Students 177,015 84,612 92,403 78,006 99,009 3,525 5,604

Table 6 - Estimates using OLS and Pupil Fixed Effects, Samples of Eastern European and Developing Countries

Eastern European Countries

OLS

Fixed Effects

All Boys Girls

High Parental Educatio

Low Parental Educatio

Immigrant 1st Gen.

Immigrant 2nd Gen.

(1) (2) (3) (4) (5) (6) (7)

36.60 38.17 35.24 43.27 29.64 58.13 51.54

(1.20) (1.36) (1.24) (1.38) (1.23) (5.34) (4.15)

2.99 2.39 3.29 3.41 2.60 18.59 11.11

(0.80) (0.87) (0.90) (0.94) (0.88) (4.65) (3.91)

Number of Students 238,938 108,927 130,011 76,970 82,322 1,642 2,210

Table 6 - Estimates using OLS and Pupil Fixed Effects, Samples of Eastern European and Developing Countries

Developing Countries

OLS

Fixed Effects

Results from primary and middle schools in Israel

Math &

Science

Math &

English

Science &

English

All 3 Subject

s

Math &

Science

Math &

English

Science &

English

All 3 Subject

sSample (1) (2) (3) (4) (5) (6) (7) (8)

0.075 0.082 0.058 0.071 0.037 0.090 0.010 0.036(0.008) (0.011) (0.008) (0.007) (0.010) (0.017) (0.010) (0.010)0.055 0.060 0.060 0.058 0.041 0.036 0.015 0.029

(0.010) (0.016) (0.012) (0.007) (0.012) (0.024) (0.015) (0.009)FE

Table 8 - OLS and Pupil Fixed Effects in Israel Using Various Combinations of Pooled Subjects

5th Grade 8th Grade

All OLS

Interaction effects of productivity of school instruction time with structural characteristics of the school

• Effect of instructional time is larger when:

– More school accountability measures are adopted

– More school autonomy in hiring teachers and determining their wages

– More school autonomy in using their budget

– School governing board influence budget and staffing

Interaction effects of productivity of school instruction time with structural characteristics of the school

• Effect of instructional time does not vary with:

– School autonomy in pedagogy

– Quality of inputs such as computers and other school facilities

– Governing board influence on curriculum and evaluation methods

Index(1) (2) (3) (4) (5)

.335 5.017 2.744 1.962 2.452(.472) (.447) (.840) (.903) (.912)

.216 5.153 2.106 2.158 2.317(.411) (.432) (.889) (1.135) (1.134)

.294 5.501 .345 -1.230 -.934(.456) (.458) (.819) (1.015) (1.010)

.461 5.222 1.385 .498 -.168(.498) (.517) (.744) (.784) (.806)

.437 5.398 .771 .303 .408(.496) (.508) (.753) (.778) (.778)

.150 5.834 .099 .435 .442(.989) (.395) (.393) (.399) (.400)

Achievement data are posted publicly (e.g. in the media). (Binary Variable).

Achievement data are used in evaluation of the principal's performance (Binary

Achievement data are used in evaluation of teachers' performance (Binary Variable).

Students are Grouped by Ability within their Classes -all or some subjects- (Binary

Hours Main

Index Means

Table 10 - Estimated Effects of School Characteristics Interacted with Instructional Hours, OECD Countries.

Hours X

Separate Spec. Joint Spec.Hours

X

Quality of Educational Resources: Index, (Range -3.45 to 2.1)

Hours X

Students are Grouped by Ability into Different Classes -all or some subjects-

Index(1) (2) (3) (4) (5)

-.058 5.925 1.224 .842 .938(.946) (.380) (.398) (.433) (.435)

.052 5.830 -.247 -.451 -.561(.964) (.386) (.399) (.427) (.429)

.363 4.981 2.599 1.199(.481) (.523) (.763) (.883)

.706 3.759 2.974 1.834(.455) (.711) (.843) (.925)

.162 5.973 -.588 -.199(.368) (.429) (.968) #####

.219 6.018 -.837 -.802(.413) (.464) (.831) (.922)

Hours Main Effect 4.676 3.255(.713) (.964)

School Governing Board Influences Staffing (Binary Variable).

Hours Main

Index Means

Table 10 - Estimated Effects of School Characteristics Interacted with Instructional OECD Countries.

Hours X

Separate Spec. Joint Spec.Hours

X

School Governing Board Influences Assessment (Binary Variable).

Hours X

School Governing Board Influences Instructional Content (Binary Variable).

School Governing Board Influences Budget (Binary Variable).

School Responsibility for Resource Allocation: Index, (Range -1.1 to 2.0)

School Responsibility for Curriculum & Assessment: Index (Range -1.4 to 1.3).

Conclusions

• Instructional time has positive significant effect on test scores.

• The OLS results are highly biased upward but the within student

estimates are very similar across groups of developed and middle-income countries and age groups.

• Effect size of one more hour of instruction:– 0.15 of standard deviation of the within student standard

deviation in test scores.

• The estimated effect of instructional time in the developing countries is only half of the effect size in the developed countries.

Conclusions

• There is significant association between characteristics of the work environment of teachers and of the education system in OECD countries and the average productivity of instructional time.

• These correlations point to some directions of how productivity can be improved in some of the developed and in less developed countries.

• For example:– Enhance school accountability measures – Increase school autonomy in hiring teachers/determining wages– Increase school autonomy in using own budget– Allow school governing board influence budget and staffing