school effectiveness in ethiopia challenges and opportunities
TRANSCRIPT
School Effectiveness in
Ethiopia: Challenges &
Opportunities
Zoe James & Caine Rolleston
UKFIET
16th September 2015
• Analysis conducted as part of World Bank
Public Expenditure Review for Education– YL data alongside EMIS, SDI etc
– Focus on efficiency and effectiveness of spending in
Ethiopia’s education sector
• Feeding into ESDP V process
• Draws together household and school data
to review changes in education system over
course of YL & existing challenges
INTRODUCTION
4 country, dual-cohort study
12,000 children in 4 countries over 15 years
Ethiopia, India (Andhra Pradesh), Peru,
Vietnam
Two age cohorts in each country:
- 2,000 children born in 2001-02
- 1,000 children born in 1994-95
From infancy to parenthood
Pro-poor sample: 20 sites in each country selected to reflect country diversity, rural-
urban, livelihoods, ethnicity etc; roughly equal
numbers of boys and girls
YOUNG LIVES: OVERVIEW
AGES: 1 5 8 12 15
YOU
NG
ER C
OH
OR
T
Following 2,000 children
OLD
ER C
OH
OR
T
Following 1,000 children
AGES: 8 12 15 19 22
Round 1 Round 2 Round 3 Round 4 Round 5 2002 2006 2009 2013 2016
VISUALISING THE HOUSEHOLD DATA
Same age children at
different time points
Qualitative nested sample
and surveys of children
in their schools
• 2012-2013 school year
• Site-level school census
• All pupils in all G4 & G5 classes
• Start and end of year survey
(W1 & W2)
• Child, class, teacher, principal and
school data
• Assessments in maths and reading
comprehension
• 94 schools, 280 classes, 11982 pupils W1, 10068 pupils W2
• Allows identification of learning progress over school year
ETHIOPIA SCHOOL SURVEY 2012-13
YOUNG LIVES SITES EXTENDED TO SOMALI & AFAR
CONTEXT: ETHIOPIA HAS MADE GREAT PROGRESS IN
TERMS OF ON-TIME ENROLMENT & PROGRESSION
Enrolled in 1st cycle primary by
age 7
Older Cohort
(born 1994/95)
Younger Cohort
(born 2001/02)
Total 27.7 50.0
Location Urban (R2) 43.2 71.2
Rural (R2) 18.2 39.5
Poverty Least poor quintile
(R2)
47.2 74.7
Poorest quintile
(R2)
15.8 32.4
CHANGING PATTERNS OF YEARS OF SCHOOLING
No years of schooling by age 11
Older Cohort
(born 1994/95)
Younger Cohort
(born 2001/02)
Total 19.5 4.2
Location Urban (R2) 3.1 0.6
Rural (R2) 30.4 6.5
Poverty Least poor quintile
(R2)
3.1 0.0
Poorest quintile
(R2)
33.0 8.5
BUT…. ISSUES PERSIST IN TERMS OF GRADE
REPETITION, DROP-OUT AND PUPIL ABSENCE
Has dropped
out, %
Mean % days
of absence
W1-W2
Total 17.3 4.6
Pastoralist
livelihood
Pastoralist 24.0 6.9
Non-pastoralist 17.2 4.4
Location Urban 16.1 3.7
Rural 20.4 7.1
Poverty Least poor quintile 13.0 3.8
Poorest quintile 19.2 6.9
Pastoralist children, rural children, and poor children are particularly disadvantaged, raising serious equity concerns
PUPIL ABSENTEEISM OVER 2012-13 SCHOOL YEAR
0
2
4
6
8
10
12
14
16A
D1
AD
2
AD
3
AM
4
AM
5
AM
6
AM
7
OR
8
OR
9
OR
10
OR
11
SN1
2
SN1
3
SN1
4
SN1
5
SN1
6
TI1
7
TI1
8
TI1
9
TI2
0
SO2
1
SO2
2
SO2
3
SO2
4
AF2
5
AF2
6
AF2
7
AF2
8
AF2
9
AF3
0
% d
ays
abse
nt
• Pupils are absent for an average of 4.6% of time • 1.9% of days in Addis Ababa and 9.6% of days in Amhara• In one Afar site, students were absent for 15% of the time1
Source: Aurino et al (2015)
ASSESSMENT DESIGN AND COMPETENCY LEVELS
• Development of test items in relation to MLCs & textbooks• Inclusion of items from grades 1-3 • Assessments are indicative of skills required across grade range 1-
5 (not comprehensive curricular assessments)
• Competency levels arrived at using actual difficulty of item in test data, and grade-level expectations
• Items from W1 and W2 pooled (given relatively small number of items) and allocated to competency levels 1-4 with level 0 denoting a score below level 1
• Pupils who score correctly on 2/3 of items in a particular competency level are considered to be at that level providing they also reach required competency of levels below this
WHILST CHILDREN MAKE PROGRESS,
COMPETENCY LEVELS ARE LOW - MATHSCompetency level Number of Pupils Percentage of
Pupils
Cumulative
Percentage
0 (Below Level 1) 853 8.66 8.66
1 (Early Foundational) (~ G1) 2,121 21.54 30.21
2 (Foundational) (~G2-3) 5,152 52.33 82.53
3 (Emerging) (~G3-4) 1,473 14.96 97.49
4 (Grade level) (~G4-5) 247 2.51 100.00
Total 9,846 100.00
020
4060
8010
0
perc
ent
Rural Urban
5 4 5 4
Level 0 Level 1
Level 2 Level 3
Level 4
WHAT DOES LEVEL 2 / FOUNDATIONAL MATHS
LOOK LIKE?
Example multiple choice questions:
- Put numbers in ascending order: 19, 6, 2, 11
- How many minutes in 1 hour?
- Tamiru has 5 Birr. His mother takes 4 Birr.
How many Birr does Tamiru have?
- Which is half of 6?
SIGNIFICANT VARIATION EXISTS BETWEEN SITES SUCH THAT IN
SOME SITES, MOST CHILDREN ARE IN LEVEL ZERO FOR MATHS
Note: Grade 4 only
02
04
06
08
01
00
per
cent
AD
1
AF
30
SN
14
AD
3
AM
4
AD
2
SN
15
SN
12
SO
23
AM
6
TI1
9
OR
10
TI2
0
AF
25
AM
7
OR
9
AM
5
SN
16
OR
8
OR
11
SN
13
SO
22
TI1
7
TI1
8
AF
28
SO
24
Level 0 Level 1
Level 2 Level 3
Level 4
DISTRIBUTION OF SCORES: PUPILS DO MAKE PROGRESS.0
00
5.0
01
.001
5.0
02
.002
5
Den
sity
200 400 600 800Score
Test 1 Test 2
Maths Scores at First and Second Round Tests
- Test 1 mean = 500 - Progress ~0.3 SD
DISTRIBUTION OF W2 MATHS SCORES IN SELECTED SITES
0
.001
.002
.003
Den
sity
300 400 500 600 700 800Score
Addis Amhara Urban (Typical)
SNNP Rural Somali Rural
Somali Jijiga Afar Rural
Maths Scores R2: Selected Sites
PUPILS BACKGROUNDS AND HOUSEHOLD CHARACTERISTICS
REMAIN VERY IMPORTANT FOR PROGRESS, IN ADDITION TO
ACHIEVEMENT AT W1
Characteristics Mathematics Reading Comprehension
Ch
ild/
HH
Girl -
Orphan - -
Pastoralist -
No-one in household literate -
3+ meals a day
Reads books at home
HH assets
Edu
cati
on
al
exp
eri
ence
Absenteeism - -
Repetition - -
Drop-out -
Attended pre-school
Learns in mother tongue
SCHOOLS DO ADD ‘VALUE’ IN TERMS OF PUPIL
LEARNING IN MATHS
• In maths urban schools do not seem to add more value than rural schools
-100
-50
050
100
Scho
ol V
alue
-Add
ed
0 20 40 60 80 100School Rank
Urban Rural
AND ALSO IN READING – BUT VARIATION
EMERGES BETWEEN RURAL AND URBAN
SCHOOLS IN THIS SUBJECT
• Whilst in reading, urban schools do seem to have an advantage
-100
-50
050
100
Scho
ol Va
lue-A
dded
0 20 40 60 80 100School Rank
Urban Rural
SCHOOL, CLASS AND TEACHER FACTORS ASSOCIATED
WITH HIGH AND LOW ‘VALUE-ADDED’ CLASSES
Characteristic Maths Reading
High VA Low VA Sig High VA Low VA Sig
School teaches only shift classes 0.58 0.73 * 0.49 0.87 ***
Class Assets -0.01 -0.40 * -0.06 -0.30
Teacher score on maths test % 63.13 55.79 ***
Teacher education University 0.15 0.03 ** 0.21 0.01 ***
EMERGING IMPLICATIONS?
• RURAL CHILDREN, PASTORALIST CHILDREN & CHILDREN FROM THE POOREST HOUSEHOLDS CONTINUE TO LAG BEHIND
• WHILST PROGRESS HAS BEEN MADE, LEARNING LEVELS REMAIN LOW
• PUPILS BACKGROUNDS & EXPERIENCES AT HOME CONTINUE TO MATTER AS THEY PROGRESS THROUGH PRIMARY
• SCHOOLS DO ‘ADD VALUE’, BUT THERE IS A LOT OF HETEROGENEITY BETWEEN SCHOOLS
• WHAT CAN WE LEARN FROM THOSE RURAL SCHOOLS WHICH REALLY SEEM TO BE EFFECTIVE?
FINDING OUT MORE…
www.younglives.org.uk
• methodology
• datasets (ESDS International)
• publications
• child profiles and photos
• e-newsletter