topic 3 c15 economic policy analysis education: school inputs and pupil performance kjell g....

Topic 3C15 Economic Policy Analysis

Education: School inputs and pupil performance

Kjell G. Salvanes

November 10 and November 17, 2003

2

School quality is again on top of the policy agendaTopics:

• Relationship between school inputs (class size, eduation of teachers) and student performance (scores, wages)– Do we need more resources or better teachers?

– For which student outcomes does resources matter for?

– Does it matter for all students?

• Educational attainment at high school and university level is another issue– Are compulsory school laws necessary?

3

Topics cont’

• Does privatisaton of schools/universities matter?• How will increased university fees matter?• Selective schools or comprehnesive schools?

• How should we evaluate whether school inputs, compulsory school laws and educational policy in general matter for student outcomes?

4

Todays lecture

• The impact of school resources and student performance

• Methodological issues

• The impact of compulsory school laws on educational attainment wages

5

School resources and student performanceIs there a connection?

TIMSS performance and spending (in US purchasing power, primary schools)

(countries ranked by TIMSS aggregates)

0

1000

2000

3000

4000

5000

6000

7000

8000

Kor

ea

Japa

n

Bel

gium

1N

ethe

rland

s

Aus

tria

Aus

tral

ia

Sw

eden

C

zech

Rep

ublic

Irel

and

Sw

itzer

land

Hun

gary

U

nite

dS

tate

s U

nite

dK

ingd

omG

erm

any

Nor

way

Fra

nce

Tha

iland

Den

mar

k

Spa

in

Gre

ece

Italy

Por

tuga

l

Isra

el

spen

din

g/s

tud

ent

6

School resources and student performance

• What are we trying to measure– We have two schools – one using a high level of

resources per students (small classes) and one little resoruces.

– Pick two identical students and put one in each of the schools and test performance after a year.

• We cannot do this and we end up comparing results for students in schools with for instance large and small classes.

• How can we estimate the causal effect of school resources on student performance?

7

School resources and student performance

• Problems– Too little variation in e.g. class size:

• Between 18 and 30 students per class– Other factors may be important in explaining

differences in student performance and which is correlated with class size

– Teachers use small classes for less able students– Parents choose neighbourhood based on school quality

(class size)– School with small class size may also have other

benefits (attracting better teachers etc)

8

Methods used to evaluate the impact of school resources

• Experiments– Randomly assign students to different types of schools

• Cannot do usually

• Collect data and evaluate by estimating something like:– Achievement = preparation+ families + peers

+schools– 1) Natural experiments – Instrumental variables– 2) Matching

9

Causal effect vs correlation

• Consider the realtionship between student performance Yi and School resources Si

Yi=a+(b+vi)Si+ui

Si=1 denotes a small class size, b+vi is the unobserved returns to be in a class with much resources, and ui represents all other individual resources determining performance.

10

Different measures

• The expected (average) performance outcome for those in a small class (Si=1):

E(YiS=1-Yi

S=0|S=1)=b+E(vi|S=1)This measure is called treatment of the treated.

|The second term reflects the way pupils are selected into small classes: if those who benefit most from small classes there is a positive correlation between their characterisics, vi, and small classes, S=1:

E(vi|S=1)>0

11

Different measures

• Compare those in small classed to those in large classes.

E(Yi|S=1)- E(Yi|S=0)

=b+E(vi|S=1)

+ E(ui|S=1)- E(ui|S=0)

The last term is the selection bias

12

Different measures

• The point is the students in large classes may be different from the students in the small classes in a systematic way such that performance differences are attributed to these differences in stead of class size.

• Rich /highly educated parents have their children in schools with more resources and small classes.

13

Methods to solve these problems

• Experiments– Construct the assigment such that there is no systematic

relationship between class size and students background variables:

E(ui|S=1)= E(ui|S=0)

– Hence there is no selection bias

• However:– Expensive,

– Unethical

14

Other methods

• Natural experiements or IV– Use information that allocates students to

schools with large and small resources to avoid selection problems

– Problems:– Depending on which instrument is being used

to decide allocation into different schools, the results may only apply for a certain group of students

15

Other methods

• Matching– Basically the method is to compare individuals in small

and large class sizes that are identical on observable characteristics Xi

– I.e. assume that for a set of observed characteristics X (family background etc), we have that

E(ui|Xi,Si)=jXi

This means that both the allocation rule deciding whether you og to a small or large school or not and the impact of that experience depend on observable characteristics.

Measuring heterogeneity in returns to education in Norway using educational

reforms

Arild Aakvik*Kjell G. Salvanes

Kjell Vaage*

*University of BergenNorwegian School of Economics and SSB

November 10 and November 17, 2003

17

Approaches and results in papers on the reading list

• Krueger : ”Experimental estimates of education production functions”– Class size and test scores– Method: Experiment ; STAR project in Tennessee;

random assignment of pupils and teachers after kindergarten to small (13-17)/regular (22-25) schools , stayed for 4 years

• Results:– Effect after one year on standarized tests– The advantage is kept throughout the 4 years.

18

Approaches and results in papers on the reading list cont’

• Dearden, Ferri & Meghir• Method: condition on a lot of background

variables• Measure educational attainment and wages

on class size, British data• Impact on women’s wages• No impact on men’s wages and eduational

attainment

19

Approaches and results in papers on the reading list cont’

• Dustmann, Rajah, van Soest– Data: England and Wales– Method: Controll for background variables– Measure effect of class size on educational

attainment and wages– Find strong impact of class size on the decision

to stay on in school after 16 and on wages

20

Measuring the Effect of a School Reform on Educational Attainment and Earnings

Arild Aakvik*

Kjell G. SalvanesKjell Vaage*

*University of BergenNorwegian School of Economics, IZA-Bonn and SSB

21

Background

• Controversy regarding returns to education especially due to selection concerns and heterogeneity in returns

• The decision to take more education is a complex process. – ability, financial constraint and preferences are usually

unobserved for the researcher; endogeneity problem– heterogeneity in the return heterogeneity arises if

individuals select into education based on their comparative advantages of education

22

• A natural but mainly unexploited resource of information to overcome these problems are the educational reforms in the European countries in the postwar period.

• The focus in the present paper is to exploit some interesting features of one of the school reforms in Norway - the school reform extending the mandatory years of schooling from 7 to 9 years.

• The reform took 10 years to implement and we observe same birth cohorts going through both compulsory school systems.

• Use additional reforms to identify a Roy model

23

• We utilize a flexible framework and a very rich data set to study different return parameters of education, both in a linear and non-linear fashion

• we allow the effect of education to vary both in terms of observed and unobserved factors.

• This model is termed a random coefficient model where we estimate returns to different levels of education (Roy model)

24

Overview• The reform

• The reform as an instrument

• + additional identification strategy

• The data

• Effects on educational attainment

• Two model of estimating returns– Continuous in education– Using a flexible Roy model for education levels

25

Aims of the reform

• Increase the minimum level of education

• Smooth the transition to higher education

• Enhance equality of opportunities along the socio-economic and geographical dimensions

26

The school reform

• From 7 to 9 years of compulsory schooling Old system:

New system:• Implemented from 1959-1974 (1961-1970)• Impl. at municipality level, decided locally• Social experiment:10 cohorts (1948-1957)

passing through 2 different school systems • Targeted to certain groups

27

Reforms in other countries

• Similar reforms in Sweden (Meghir & Palme, 1999, 2001), UK (Blundell et al. 1997), France, Germany, etc.

• The reform went further in Norway in terms of unification and in promoting equality of opportunity (Leschinsky and Mayer, 1990)

28

Effects of the school reform?

• Are there different educational outcome for individuals in the pre vs. post reform system?

• Did the reform help the targeted groups in attaining higher education?

• Can we use this (potential) variation to estimate the returns to education, i.e. can we use the reform as an instrument?

• Using upper secondary reforms/college reform as additional instruments (distance to higher education)

29

The reform as an instrument

• Is the reform correlated with the variable for which it serves as an instrument, i.e. did it lead to increased educational attainment? For all? For some?

• Is the reform uncorrelated with earnings (except indirectly through the schooling variable), or does it pick up other characteristics of the municipalities?

30

Reform implementation and municipality characteristics

• Implementation decided at municipality level, costs reimbursed by the Government• Government’s strategy: reform implementation according to a representative set of municipalities • No signs of selection on municipality observables in our data

31

Data

• SN’s administrative registers: earnings, cohort and county indicators, work experience, education (highest obtained)

• National census of population and housing: residing municip. during school, family income from 1970

• Males in full-time job• Education and earnings measured in 1995• Reform dummy • Availability of high school, college, university in

the municipality

32

Construction of reform indicator

• Use census-data on parents’ residence in 1960 and 1970 to assign schooling municipality

• Combine with register-data at municipality level• Problems: (i) 20% of the munic. used > 1 year (ii) Commuting between residence and school (iii) Special arrangement for the earliest cohorts (iv) School reform coincides with municipality

reform

33

Construction of reform indicator (continued)

• SN-data on individual reform assignment, but only for the group that left school after compulsory schooling (16%)

• Our strategy: Combine Municipality Register and SN data, dropping cohorts - but not municipalities! - with missing or uncertain information

• Use fraction of pupils on reform in the municipality as the reform indicator

34

School choice• Continuous (7-20 years)

• Categorical (7 different levels)1) Pre/post reform compulsory school (7/9 years)

2) Upper secondary school 1 year; mainly vocational

3) Upper secondary school 2-3 years; mainly vocational

4) Upper secondary school 2-3 years; gymnasium

5) University I, post upper secondary school, 1-2 years

6) University II, post upper secondary school, 3-4 years

7) University III, master level, university degree, 5+ years

35

O Probit Models of school choice

• Switching regression• Covariates:

- Age cohort dummies- Municipality variables

- Parental education- Family income (percentiles)

• Derive generalised residuals (i) for the earnings equation

36

Observed pre and post reform education

• Birth cohorts 1948-57.• • Levels Pre-reform Post-reform Change Change in %• ________________________________________________________________

• 1 Pre/post comp. 0.213 0.135 -0.078 -36.6• 2 Vocational I 0.167 0.180 0.013 7.8• 3 Vocational II 0.249 0.303 0.054 21.2• 4 Upper secondary 0.043 0.060 0.017 39.5• 5 University I 0.134 0.135 0.001 0.8• 6 University II 0.092 0.093 0.001 1.1• 7 University III 0.099 0.090 -0.009 -9.1• ________________________________________________________________

37

Predicted pre and post reform education Conditional on cohort, region and family income & education

• Birth cohorts 1948-57.• • Levels Pre-reform Post-reform Change Change in %• _______________________________________________________________•  • 1 Pre/post comp. 0.195 0.141 -0.054 -27.8• 2 Vocational I 0.159 0.183 0.024 15.1• 3 Vocational II 0.248 0.307 0.058 23.7• 4 Upper secondary 0.044 0.060 0.016 38.3• 5 University I 0.139 0.133 -0.006 - 4.5• 6 University II 0.098 0.089 -0.008 - 8.9• 7 University III 0.114 0.084 -0.030 -26.7• _______________________________________________________________

38

Earnings equations, sources of possible biases

• Unobserved individual heterogeneity

- ability - financial constraints

• Heterogeneity in returns - self selection to education level

based on comparative advantage

• Non-linearity in returns to education

UNIVERSITY OF BERGENUNIVERSITY OF BERGEN

39

Earnings equations, specifications

• Instrumental Variable ( LATE):

log yi = Xi + Si + i + Ui

log yi = Xi + Si + i + Ui

• Random Coefficient Model ( ATE):

log yi = Xi + (iSi + i + Ui

log yi = Xi + Sii Si + i + Ui

40

The Roy model

• Run the Randdom coefficient model for each education level

E(log yi)=Xi + Si + i

We can then estimate the return to education by comparing the different estimated model parameters for a given x is simply calculated fromΔATE(x) =xi(βl-βl-1)+(l- l-1) i

ΔTT(x) =xi(βl-βl-1)+(l- l-1) i

41

Earnings equations,estimated coefficients

Earnings equations, full time men, cohorts 1948-57

OLS IV RCM

Education 0,0754451 * 0,1010064 * 0,0546282 *

0,0003998 0,0018642 0,0023033Lambda 0,0472469 * 0,0123857 *

0,0033657 0,0035105Education*Lambda -0,0131717 *

0,0003857Tenure 0,0104852 * 0,0105042 * 0,0105124 *

0,0005041 0,0005039 0,000504Tenure2 -0,0004829 * -0,000483 * -0,0004832 *

0,0000234 0,0000234 0,0000234Experience 0,0446061 * 0,0453716 * 0,0456544 *

0,0016455 0,0016466 0,0016658Experience2 -0,0007032 * -0,000714 * -0,0007207 *

0,0000409 0,0000409 0,0000413R-Squared 0.22 0.22 0.23Number of Obs, 134884 134884 134884

42

Result from the Roy model• Table 6.2. Returns to education in percent.• =======================================================• No selection Selection• ------------------- -------------------• ATE TT ATE TT• -------------------------------------------------------• 1• 2 -00.2 01.2 04.8 01.5• 3 08.3 08.8 08.8 09.2• 4 20.7 21.1 22.1 21.7• 5 27.0 26.8 23.8 27.4• 6 21.8 21.9 15.7 22.7• 7 44.6 42.3 31.7 43.3• -------------------------------------------------------•

43

Main findings

• The reform enhanced educational attainment for low achievers

• Pupils from low income families were picked up by the reform (?)

• OLS gives biased estimates of the returns to edu.

44

Main findings

• Non-linearity in returns to education

• Selection on unobservables appears to be important

• Appears to be hard to obtain gains from inducing a very high proportion to university education