Equitable Progress? The Role of School Quality in

Shaping Learning Gaps: A Comparative Study of Two

Developing Countries.

Caine Rolleston

Abhijeet Singh

• Large gaps between countries on common tests point to school system effects

• But also large gaps in educational performance by SES in some developing countries

• SES gaps due to both background and school quality effects (Heyneman-Loxley debate) - which are correlated

• Several YL papers address issues of the role of school quality compared to home backgrounds within/across countries

• E.g. inequality by backgrounds AND school quality especially high in Peru, more equitable in Vietnam (Glewwe, Krutikova & Rolleston 2015)

• A key equity question surrounds whether less advantaged pupils are ‘sorted’ into poorer quality schools, potentially widening SES gaps

• Ideally, independent data on learning progress and school quality required

• Availability of data from R4 and Ethiopia & Vietnam school surveys provide new evidence

• How much of SES gaps are due to ‘sorting on school quality’? • What is the impact of differences in school quality on learning

progress?

MOTIVATION AND RESEARCH QUESTIONS

Young Lives longitudinal survey of children, households &

communities every 3 years since 2002

• 12,000 index children

• Ethiopia, India, Peru, Vietnam

• 20 sentinel sites in each country

• Qualitative component for a sub-sample

• Tested in maths at each round with common items

allowing IRT scaling of scores

School surveys implemented since 2010

• Focus on learning & learning progress

• School and teacher effectiveness

• Index children and their class peers

• Tested in curricular maths at beginning and end of

school year (common items/scale within country)

YOUNG LIVES STUDY

AGES: 1 5 8 12 15

YOU

NG

ER C

OH

OR

T

Following 2,000 children

OLD

ER C

OH

OR

T

Following 1,000 children

AGES: 8 12 15 19 22

Round 1 Round 2 Round 3 Round 4 Round 5 2002 2006 2009 2013 2016

YL HOUSEHOLD DATA

Same age children at

different time points

• 2012-2013 school year

• 7 regions

• 30 sites

• Site-level school census

• All pupils in all G4 & G5

classes (~12,000 total)

• Start and end of school

year surveys

• Child, class, teacher,

principal and school

data

• Allows identification of

learning progress over

school year

ETHIOPIA SCHOOL SURVEY

VIETNAM SCHOOL SURVEY

• 2011-12 School year

• 5 provinces

• Grade 5 pupils in classes

attended by YL index

pupils

• Plus class peers to a total

of 20 per class

• Total 3,600 pupils

• Start and end of school

year survey (as Ethiopia)

DISTRIBUTION OF MATHS SCORES (ETHIOPIA)

.000

5.0

01

.001

5.0

02

.002

5

Den

sity

200 400 600 800Score

Test 1 Test 2

Maths Scores at First and Second Round Tests

• Mean Test 1 = 500, SD= 100. Mean Test 2=530 (gain 0.3 SD)

• School survey data for non-index children only used to estimate school value-added independently from household sample

• Approx. 90 schools in each country• Approx. 2,300 pupils Vietnam and 11,500 Ethiopia • Estimate school fixed effects for maths using beginning and

end of year test data (lagged value-added model) separately by country (IRT scales)

• Using squared & cubic terms plus pupils’ backgrounds not due to schooling

• Linked maths scores from household tests on cross-country scale (IRT)

• Household wealth on common scale & comparable controls• Insert school FE into regressions using household data• Allows identification of wealth gaps

DATA AND METHODS

• Normalised scale within each country, not directly comparable• Conditioned on child backgrounds not due to schooling

0.1

.2.3

.4

kde

nsity s

choo

l_fe

-2 -1 0 1 2 3School Value Added Estimate

Ethiopia

0.1

.2.3

.4

kde

nsity s

choo

l_fe

-4 -2 0 2 4School Value Added Estimate

Vietnam

SCHOOL VALUE-ADDED BY COUNTRY

-100

-50

050

10

0

Sch

ool V

alu

e-A

dd

ed (

90%

CI)

0 10 20 30 40 50School Value-Added Rank

SCHOOL VALUE-ADDED VIETNAM

• Fixed-effects on test-score scale (mean 500 at baseline)

-10

12

3

Ethiopia Vietnam

1 2 3 1 2 3

2009 2013

MATHS SCORES 2009 & 13 BY WEALTH TERCILE

• 1: Poorest 2: Middle Wealth 3: Least Poor

0.1

.2.3

.4.5

-2 0 2 4 6Math scores

Ethiopia 2009 Vietnam 2009

Ethiopia 2013 Vietnam 2013

Distribution of Test Scores 2009 and 2013

• Maths scores from household surveys R3 and R4• Standardised using Ethiopia R3 mean and SD• Ethiopian pupils typically exposed to fewer years of schooling

0.2

.4.6

01

23

4

-2 0 2 4 -2 0 2 4

Ethiopia Vietnam

Most Poor Least Poor

Math

score

s (

201

3)

Math scores (2009)

Graphs by country

Maths scores

Learning divergence by wealth groups

• Kernel density R3 (2009)• Stronger relationship with prior score in Ethiopia, broadly similar gaps by wealth• Pupils typically make more gains over time in Vietnam (not linked to prior scores)

LEARNING GAPS

• Most poor group is reference category

• Compare wealth gaps (middle and least poor):

1. No controls, dummy variables only

2. Add backgrounds – parental education etc.

3. Add prior test scores

4. Add school effectiveness in mathematics

5. Add site fixed effects

• Differences in scores in 2013 strongly correlated with wealth

• Average gain score is 0.86 SD over 4 years, similar to gap in Ethiopia in 2013 between poorest and least poor (4 years’ schooling) more like 3 years in Vietnam

No Controls + Backgrounds + Prior Scores + School Eff + Site FE

Middle Wealth Group 0.3914 0.1567 0.1548 0.1590 0.1398

(6.59)*** (2.52)** (2.56)** (2.64)*** (4.69)***

Least Poor Group 0.7406 0.2835 0.2289 0.2146 0.0913

(12.15)*** (3.87)*** (3.21)*** (3.01)*** (1.42)

School Effectiveness 0.0747 -0.0315

(2.85)*** (-1.04)

Constant 2.4685 2.6394 2.8336 2.7113 2.1602

(57.22)*** (2.22)** (2.83)*** (2.71)*** (7.44)***

Observations 1,317 1,315 1,307 1,307 1,276

R-squared 0.101 0.239 0.285 0.290 0.200

Number of sites 19

VIETNAM

• Backgrounds reduces wealth gaps by >half• Prior scores small effect – wealth gaps not strongly linked to earlier test score gaps (some

catch-up)• School effectiveness no change - little evidence of sorting• Site fixed effects reduces gaps for the rich group – suggests clustering of progress in sites• No evidence of school sorting/differences in school effectiveness within sites• Relatively equitable regarding ‘school choice’

-1-.

50

.51

mea

n o

f scho

ol_

fe

1 2 3 4 5 6 7 8 9 10 11 12 13 15 16 17 18 19 20

VARIATION IN SCHOOL EFFECTIVENESS BY SITE

• Similar wealth gaps & reduction with backgrounds

• Reduced further by half when prior scores added (and no longer significant) - wealth gaps linked to earlier test score gap persistence

• No further reduction when school effectiveness is added – little evidence of sorting

• Similar results with site fixed effects – little evidence of site-level clustering

• Strong effects of differences in ‘school effectiveness’ including within sites

• Again ‘school-choice’ not leading to inequity

VARIABLES No Controls + Backgrounds + Prior Scores + School FE Site FE

Middle Wealth Group 0.3700 0.1886 0.0833 0.1141 0.1017

(4.40)*** (2.16)** (1.09) (1.52) (1.52)

Least Poor Group 0.8447 0.2549 0.1325 0.1487 0.1242

(9.95)*** (2.23)** (1.33) (1.52) (1.14)

School Effectiveness 0.1826 0.1566

(5.38)*** (2.34)**

Constant 0.4863 1.9980 1.2013 1.1298 2.1691

(7.75)*** (1.67)* (1.65) (1.57) (3.40)***

Observations 863 863 843 843 839

R-squared 0.105 0.213 0.422 0.442 0.301

Number of sites 17

ETHIOPIA

• School VA estimate 0.157 – a one standard deviation change in school effectiveness increases 2013 score by 0.157 SDs (conditional on Xs)

• Approx. equivalent to moving from 25th to 55th

percentile of school effectiveness distribution

• Apparently effect at least as large as effect of moving from poorest to least poor wealth tercile

• Much less evidence of variation in effectiveness in Vietnam

IMPACT OF SCHOOL QUALITY -ETHIOPIA

-2

-1

01

mea

n o

f sch

oo

l_fe

Addis AbabaAmhara Oromiya SNNP Tigray Somali Afar

-.5

0.5

11.5

mea

n o

f sch

oo

l_fe

government private community faith-based ngo

VARIATION IN SCHOOL QUALITY (ETHIOPIA)

-.5

0.5

1

mea

n o

f sch

oo

l_fe

shift school full-day school Mixed school

VARIATION IN SCHOOL QUALITY (ETHIOPIA)

FINDING OUT MORE…

www.younglives.org.uk

• methodology

• datasets (ESDS International)

• publications

• child profiles and photos

• e-newsletter