improving education management in madagascar ( agemad )
DESCRIPTION
Results of an Impact Evaluation. Improving Education Management in Madagascar ( agemad ). Préparée par Jee Peng et Cornelia. Présentée par. Paul RANDRIANIRINA. ACCRA, GHANA. Mai 11, 2010. Primary Education in Madagascar: Much Progress, but still Many Challenges. - PowerPoint PPT PresentationTRANSCRIPT
IMPROVING EDUCATION MANAGEMENT IN MADAGASCAR
(AGEMAD)
ACCRA, GHANAMai 11, 2010
Results of an Impact Evaluation
Préparée par Jee Peng et Cornelia
Paul RANDRIANIRINA
Présentée par
Primary Education in Madagascar: Much Progress, but still Many Challenges
Signs of progress: Primary completion rate doubled from 35% in1999 to 71% in 2008
Evidence of weak sector performance: ½ of each cohort of 1st graders does not finish the primary cycle; Repetition rate still high at 18% in 2005 (30% in 2000) Low quality: in 2004-5 PASEC, average test score of 50% in Maths
and Malagasy and 32% in French; deteriorated since 1997-98
Multiple systemic causes : Inconsistencies in teacher allocation across schools; Ineffective management of pedagogical processes at
school and classroom levels
Les défis de l’AGEMAD
Améliorer l’allocation des ressources entre les écoles. Assurer que les ressources allouées sont transformées en résultats
au niveau des élèves.
Identifier des interventions qui permettent de renforcer la gestion du système.
Tester les interventions et évaluer leur impact afin d’éclairer les choix sur des généralisations possibles
Une démarche en 4 étapes :
1. Identifier acteurs du système éducatif = ceux qui ont des responsabilités à assurer
2. Déterminer quelles tâches ils ont à accomplir3. Elaborer des outils de travail pour qu’ils
accomplissent ces tâches: outils (procédures, tableaux de bord, statistiques) rationalisés, adaptés , conçus et testés en collaboration avec les utilisateurs
4. Clarifier les conditions incontournables pour la bonne exécution du système de gestion.
Tighter Management to Improve AccountabiltyConceptual Intervention Framework & IE
Design: Workflow tools to clarify tasks and internal
accountabilities; Facilitation of meetings between school and community; Better information flows within school and between
school and community; Structured training for teachers and school heads
Leading to: improvement in actors’ behavior through
bottom up and top down accountabilitybetter managed school
increased school quality higher student learning
Key Questions for Policymakers
What is the impact of tighter management of processes on school functioning and student performance?
At what administrative level are management interventions the most effective (school, district or inspection level)?
Impact Evaluation Design (1)
Method: Randomized experimental design over 2 school years
Interventions: Specify actors’ responsibilities & their mutual
accountability processes through: Management Tools and Guides for key tasks (e.g.
pedagogical, administrative) Training
Focus attention on results by clarifying goals through: Report cards: School, district and inspection report
cards School meetings: Facilitated school meetings &
development of school improvement plans based on school report cards
School & District Report Cards for Better Information Flow
Report cards for school directors, sub-district and district levels officers:
Complement the tools and processes Draw attention to schooling outcomes Include comparative data, allowing a school
to compare its outcomes with those of other schools
Serve as basis for dialogue and accountability
Impact Evaluation Design (2)
9
303 Schools AGEMAD
TREATMENT 3
303 SchoolsZAP AGEMADTREATMENT 2
15 CISCO AGEMAD
89 ZAP AGEMAD
80 ZAP CONTROL
303 SchoolsCONTROL
15 CISCOCONTROL
84 ZAPNON-AGEMAD
303 SchoolsCISCO AGEMADTREATMENT 1
Collecting Data
Actors’ Behavior (direct effects): Questionnaire from impromptu school visits in 1,200
schools, with information for 4,000 teachers Questionnaires for District and Community admin. level Collection and analysis of tools used in 40 schools (850
tools)
Schooling outcomes (indirect effects): Test scores from standardized tests in 3 subjects National year-end school census data: flow rates,
repetition, CEPE pass rate
Timeline: 2 school years, 2005-2007 Baseline survey/test and post-intervention survey/test
What tasks are deemed essential?
Teacher: Takes daily roll call Prepares daily lesson plan Prepared bi-monthly lesson plans
Monitors student learning Has tested pupils during the past two months
Helps lagging students Discusses student learning issues with the director
School director: Keeps a register of enrollments
Signs off on daily roll call Analyzes student absences on a monthly or bi-monthly basis
Reviews pupils’ test results Takes stock of teacher absences
Informs sub-district or district officer about teacher absences
Follows up with teachers on lesson planning
Results: Effects on Actors’ Behavior
AGEMAD schools
Control schools
Teacher absence (%) 8.7 9.2
Teacher completes all key tasks (%) 63.0* 42.4
All teachers in school perform their key tasks (%)
42.9** 22.1
Well managed schools (%) 36.5** 15.2
*significant at 5% level; **significant at 1% level
13
Results: Effects on Schooling Outcomes
AGEMADschools
Controlschools
Student attendance (%) 90.7* 86.6
Repetition (%) 17.5* 22.6
Drop out (%) 5.5 6.1
Success rate at CEPE exam 73.0 61.9
Student test results (post-test)
Math 51.2 49.4
Malagasy 50.9 48.5
French 30.0 29.4
All subjects 43.5 41.9
*significant at 5% level
Policy Implications Prioritize school-level actors
“Cascade” training model alone, as currently defined, doesn’t work Though results are encouraging, better management essentially entails
changing peoples’ behaviors, which takes time and effort Mainstream IE results into MoE activities
Need a champion from the start Need early involvement of a national team, with good technical support Necessary to sustain change in actors’ attitudes & behaviors
Use existing structures and mechanisms for scale up: Tools, guides and training modules integrated into teacher training Tool distribution, training and facilitated school meetings funded
through the local catalytic funds based on regional, district and school performance plans and needs
Develop leaders to drive change in management practices Discussion underway on collaboration in leadership training between
Madagascar MoE and partner organization in another country
Stay tuned…Publications forthcoming
Africa Human Development Working Paper Series«Améliorer la gestion de l'enseignement primaire à Madagascar - Résultats d'une expérimentation randomisée »
Journal Article undergoing peer review «Managing for results in primary education in Madagascar: Evaluating the impact of selected workflow interventions »
It takes a village…
Government commitment: Stable counterpart team (15 staff from MoE with coordinator)
Partner commitment: Financial and technical assistance from AFD (via two staff) WB team lead by Jee-Peng Tan and Cornelia Jesse, consisting
of Gérard Lassibille and Trang van Nguyen (with in-country field coordinators)
Local NGO Aide et Action to assist with training
Financing: WB, AFD, MoE, EFA-FTI (EPDF), Irish Aid, Ministry of Foreign Affairs of Norway
Timeline: 2004 – 2007 Total number of people involved: 50
The Perils of Data Collection…