Tom Pellens, Rachel Outhred, Zara Majeed, Andrej Kveder, Michele Binci, Johanna Wallin, Fatimah Kelleher, Adrian Beavis and Shefali Rai
Draft Baseline Technical Report prepared by EDOREN on behalf of UNICEF GEP3
Evaluation of UNICEF Girls’ Education Project Phase 3 (GEP3)
September 2016
EvaluationoftheGirl’sEducationProject3
DisclaimerEDORENisaconsortiumofleadingorganisationsininternationaldevelopmentandeducation:OxfordPolicyManagement(OPM),andtheInstituteofDevelopmentStudies(IDS)attheUniversityofSussex,andissupportedbyUKAid.EDORENcannotbeheldresponsibleforerrorsoranyconsequencesarisingfromtheuseofinformationcontainedinthisreport.AnyviewsandopinionsexpresseddonotnecessarilyreflectthoseofOPM,IDSandEDORENoranyothercontributingorganisation.
EDOREN No2,16MafemiCrescent Tel +2348107278718EducationData,Research&EvaluationinNigeria Utako Tel +2348176678243 Abuja,Nigeria Email [email protected] Website www.edoren.org©EDOREN i
Acknowledgements
Weareverygratefultothegovernmentdepartmentsandstaffinthefiveprojectstates,aswellasatthenationallevel,whofacilitatedthebaselineresearchandsharedtheirperspectiveswiththeevaluationteam;inparticular,StateMinistriesofEducation(SMoEs),StateUniversalBasicEducationBoards(SUBEBS),StateAgenciesforMassEducation(SAMEs),CollegesofEducation,andtheGenderUnitoftheFederalMinistryofEducation(FME).
WewouldliketothanktheUKDepartmentforInternationalDevelopment(DFID),whichcommissionedthisindependentevaluation–particularlyDFIDEducationAdviserMsLauraBrannellyandtheDFIDEvaluationAdvisersforNigeria,MsKristenHopkinsandMrLaurieThompson,forusefuldiscussionsandadviceontheevaluationdesignandimplementation.
Ourthanksalsogototheimplementingorganisation,theUnitedNationsChildren’sFund(UNICEF):TerryDurnnian(SeniorEducationSpecialist),CharlesAvelino(EducationSpecialist)andAlassaneOuedraogo(MonitoringandEvaluationOfficer)gavemuchoftheirtimetotheevaluationteaminordertoparticipateintheevaluationdesignandtoshareinformationontheGirlsEducationProjectPhase3(GEP3)programme.SpecialthanksareduetotheGEP3stateteams–inparticular,thestateprojectcoordinators,andUNICEFstaffintheGEP3states,forassistanceinaccessinginformationandarrangingmeetingswithkeyinformants.Inaddition,wethanktheReadingandNumeracyActivity(RANA)implementationteamandmonitoringandevaluation(M&E)supportstaffforcoordinatingtheRANAroll-outwiththeevaluationteamandprovidinginformationontheRANAproject.
ForadviceonthedevelopmentoftheHausapupillearningassessmentwearegratefultoProfessorMalamiBuba,fromSokotoStateUniversity.WearealsogratefultoDrDavidJohnsonoftheDepartmentofEducationatUniversityofOxford,whodesignedtheoriginalTeacherDevelopmentNeedsAssessments(TDNAs)forGEP,whichservedasabasisforthedevelopmentoftheteachercompetencyandskillsassessment,aswellastoAdrianBeavis,forhisworkonpre-literacyitemdevelopmentandteachermotivationinstrumentdevelopment.
ThisevaluationisbeingcarriedoutwithfullindependencebytheEDOREN(EducationData,ResearchandEvaluationinNigeria)project.TheEDORENteamresponsibleforthedesignoftheevaluationandtheimplementationandanalysisofthebaselinedataconsistedof:TomPellens(TeamLeaderandEvaluationSpecialist);RachelOuthred(ProjectManager,SeniorEducationSpecialist,QualitativeDesignLeadandLeadforInstrumentDevelopment);ZaraMajeed(InstrumentDevelopmentandQuantitativeAnalysisSupport);AndrejKveder(SurveyTechnicalLead);MicheleBinci(Econometrician);FatimahKelleher(QualitativeLead);JohannaWallin(QualitativeResearcher);VictorSteenbergen(QualitativeResearcher);andShefaliRai(Researcher).Thecoreteamwassupportedbyagroupofspecialistadvisers:AdrianBeavis,DavidMegill,JusteNitiema,OladeleAkogun,StuartCameronandSouroviDe.QualityassuranceofthetechnicalreportwascarriedoutbyMonazzaAslam.Thekeyinformantinterviews(KIIs)forthebaselinetheoryofchangeassessmentwereconductedwiththeassistanceofDrHafsatLawalKontagora.
EvaluationoftheGirl’sEducationProject3
DisclaimerEDORENisaconsortiumofleadingorganisationsininternationaldevelopmentandeducation:OxfordPolicyManagement(OPM),andtheInstituteofDevelopmentStudies(IDS)attheUniversityofSussex,andissupportedbyUKAid.EDORENcannotbeheldresponsibleforerrorsoranyconsequencesarisingfromtheuseofinformationcontainedinthisreport.AnyviewsandopinionsexpresseddonotnecessarilyreflectthoseofOPM,IDSandEDORENoranyothercontributingorganisation.
EDOREN No2,16MafemiCrescent Tel +2348107278718EducationData,Research&EvaluationinNigeria Utako Tel +2348176678243 Abuja,Nigeria Email [email protected] Website www.edoren.org©EDOREN ii
ThequantitativebaselinefieldworkwasconductedbytheOxfordPolicyManagement(OPM)Nigeriaofficesurveyteam,whichincluded:FemiAdegoke(SurveyManager);ArogendadeEkundayo(FieldManager);BabatundeAkano(DataManager);andBukolaAdekolu(SurveyAdministrator).Thefieldworkwascompletedsuccessfullythankstothehardworkofthefieldenumerators.Ourspecialthanksthereforegoto:GodwinAdams,KogentDauda,FahadLawal,MuhammadKassim,LadidiLilianBulus,GloriaOlisenekwu,RachealLinusAvindia,MatthewOmaye,DanielOmokagboAyala,ArmayauSallau,MiriamUkemenam,MercyAbalaku,GabrielPanshakPandukur,EuniceAtajiri,MohammedLawanKagu,MusaAhmedMutallab,SandraWilson,CelestineAninna,AbdulhamidAbubakar,OluwaseyiMoses,LukmanAbubakarAbdullahi,DanlamiUseniNakoto,PreciousChrysanctus,JosephineMalum,GloriaObiageliUtti,IyaboAbulganiyu,ElizabethBello,HajaraNakkudu,MariamBako,AbubakarShehu,RaliyaSuliaman,JummaiChabo,AnnahJoseph,VeronicMichael,GraceBala,RuthAbbahJonah,AgathaTobias,BridgetBakar,OluwaseunTemtopeBalogun,HenriettaHabu,JanetJammaiJulius,MercyMusa,LilianNakoto,RuthVincent,CharityAndrew,TabithaChaamba,FatimaHamza,RuthAgono,BlessingDavo,AishaMusa,FalaluMa’sudMuhammed,SalomeSanda,MarleeyahSuleiman,AgathaLeo,ZainabLawal,AishaAbu,PaulSusanGaladima,HassanaGarba,SaratuTebu,SarayaDaudaLoya,NazeerMuhamman,HabibaMuhammed,HassanuHashim,LarabaIliyaandRotmwaIshayaBangson.
ThequalitativefieldworkwasconductedbyateamofnationalqualitativeresearchersworkingwiththeEDORENqualitativeresearchteam,andwascompletedduetotheirdedicationandgreatefforts.InthisregardwewouldliketothankYusufIbrahim,DrAdamuBabikkoi,JiddereMusaKaibo,HalimaSanni,DrAmminuDukku,NafinatuAbdullahi,FeliciaYakubuandMadaraAdamu.
Needlesstosay,duringthefieldworkstateeducationofficers,localgovernmentofficers,communityleaders,integratedQur’anicschool(IQS)proprietors,headteachers,teachers,pupilsandparentsplayedacrucialrolebygivingtheirtimesoastoallowtheimpactevaluationbaselineresearchtobeconducted.Wewouldliketoextendabigthankyoutothemall.
Forfurtherinformation,contacttheGEP3EvaluationProjectManager,DrRachelOuthred([email protected]),ortheEDORENCountryoffice([email protected]).ThecontactpersonatUNICEFisTerryDurnnian([email protected]),andtheresponsibleDFIDadviserisLauraBrannelly([email protected]).
EvaluationoftheGirl’sEducationProject3
©EDOREN iii
Executivesummary
Thisreportpresentsthefindingsofquantitativeandqualitativebaselinedatacollectionundertakenby
EducationData,ResearchandEvaluation inNigeria (EDOREN)aspartofamulti-yearevaluationof the
GirlsEducationProjectPhase3 (GEP3).This is the full technical reportof thebaselineevaluation,whichdescribes themethodology for the evaluation and the full set of quantitative and qualitative findings indetail.Itiscomplementedbyasynthesisdocumentthatsummarisesandcollatesthebaselineevidenceforalargeraudience.GEP3 is an eight-year project (2012–20) that seeks to improve school access, retention and learning
outcomesforgirlsinfivenorthernNigerianstates.ItismanagedbytheUNChildren’sFund(UNICEF)andfundedbytheUKDepartmentforInternationalDevelopment(DFID).Between2014and2017theprojectispiloting a series of interventions in primary schools and Integrated Qur’anic schools (IQS). Those mosteffective in improvingeducationoutcomes for girlswill be scaledup.Akey focusof theevaluation is toinformdecisionsrelatedtothescale-upofGEP3’sinterventions.Thescopeoftheevaluationinvolves:1. ahigh-levelexplicationandexaminationofGEP3’stheoryofchange(ToC);2. animpactevaluationofGEP3’searlylearningintervention;and3. aperformanceevaluationofGEP3’ssupporttoIQSs.
Thebaselineevaluationhasthreecoregoals:first,tocapturetheeducationsituation,andperceptionsofthissituation,atthestartofthere-designedGEP3,sothatchangescanbemeasuredduringfollow-updatacollection rounds, andproject attributionor contribution canbe assessed; second, to answer evaluationquestions about the relevance of the project; and third, to provide evidence to informGEP3’s ToC andprojectimplementation.
ExaminationofGEP3’sToC
Methodology
ThebaselineassessmentofGEP3’sToCfocusesontwoevaluationquestions:
• HowplausibleisGEP3’sToCinthecontextoftheGEP3states?• HowappropriateareGEP3’sinterventionsintermsoftheimplementationstrategy?Theplausibility of the ToC is examinedby reviewing stakeholders’ understandingof intendedoutcomes,interrogating the logic of the outcome chain, and identifying factors that have a key bearing on theachievement of the stated outcomes. GEP3’s implementation strategy is assessed on three dimensions:stakeholderinvolvement, implementationcapacityandtheextenttowhichtheprojectfocusesonunder-served groups. This assessment is based on key informant interviews (KIIs) conducted largelywithGEP3state-levelstakeholdersinAugust2015.KIswerepurposivelyselected,withthesupportoftheGEP3stateteams,basedontheirknowledgeoftheproject.
Keyfindings
The plausibility of GEP3’s ToC is supported by its coherent logic, synergies across interventions, and
stakeholders’commonunderstandingofitsmainobjectives–althoughtherearecaveatshere.Synergiesbetween the project’s interventions could enhance its impact, but also present risks, in thatimplementation difficulties on one dimension of the project could undermine its performance on otherfronts. A second key caveat is thatwhile stakeholders understandGEP3’smain objectives, the project’soperational design is not always understood. For instance, at the time of the interviews government
EvaluationoftheGirl’sEducationProject3
©EDOREN iv
stakeholdersinvolvedinteachertrainingdidnotyetunderstandwhatcapacitiesneedtobedeveloped,inwhatways, andwhen. This couldmake it difficult for them to assessGEP3’s performanceor be held toaccountfortheresultsachieved.
Stakeholdersnoted that theproject’sobjectiveof increasinggirls’enrolment is feasible,particularly in
primaryeducation,butthatimprovementsinretentionandlearningoutcomeswillbehardertoachieve.
Stakeholderswereoftheviewthatenrolmentdrivesandcashtransfersarebotheffectivestrategiesthataddressimportantbarrierstogirls’accesstoeducation,andthattheycomplementoneanother.Retentionis held to bemore challenging as it is shaped by supply-side factors, such as the presence of sufficientteachers,thequalityofteaching,andthequalityoftheschoolenvironment.Stakeholderswerelesslikelyto recognise learning as a key expected project outcome. When they did, they noted that learningoutcomesaremoredifficult to improve thanaccess,owing to the lowqualityof teaching,particularly inrural areas, and insufficient investment in the supply of quality education. There was wide consensusamongstKIs that tackling thequalityof teachingandteachers iscentral to improving learningoutcomes.ThisisconsistentwiththefocusonthisvariableinGEP3’sToC.
Awiderangeofstakeholdersendorsedthe importanceof theearly learning intervention.Stakeholderssupporttheemphasisonliteracyduringearlygradesasafoundationalskill.However,thereisnoconsensusamong stakeholders about the extent towhich the use of amother tongue is a necessary condition toimprovelearningoutcomes.ThebaselinefindingsoftheearlylearningevaluationconfirmthatHausaisthelanguage of the immediate environment in Katsina and Zamfara, but they also highlight that Hausaknowledgeamongteachersis low.Thisneedstobetakenintoaccountinthedesignoftheearlylearningintervention.
GEP3’s interventions to support school governance are considered to be largely promising, although
thereare some risks related to the targetedempowermentof School-BasedManagementCommittees
(SBMCs).SBMCempowermentisbothapivotalintermediaryoutcomeinGEP3’sToC,aswellasoneofitsmostprecariouslinks.TheexpectationsplacedonSBMCsintermsofrolesandresponsibilitiesareveryhigheventhoughtheseorganisationsareoftenstartingfromaverylowbase,particularlyinIQSs.Atthestatelevel, High-Level Women Advocate (HiLWA) members engage with decision-makers and schoolcommunitiesandhavethepotentialforindirectinfluence.TheGirlsEducationSteeringCommittee(GESC)alsohasthepotentialtosupportgovernance,totheextentthatitisactive,sufficientlylocallyownedandaddresses key challenges. GEP3’s support to the educationmanagement information system (EMIS) andAnnualSchoolCensus(ASC)willplausiblyimprovedatareliability,althoughthiswillnotnecessarilyleadtodatabeingusedinthepolicy-makingprocess.
The findings point to some specific risks to the scale-up process. The scale-up process is reliant ongovernmentfunding.However,thereissignificantuncertaintyabouttheextenttowhichthisfundingwillbeforthcoming.ThisisdespitecertainsupportivemeasuresbeingintroducedaspartoftherecentdesignofGEP3,notablyanincreasedemphasisonhigh-leveladvocacyandafocusonamoremanageablesetofoutcomes. The scale-up of GEP3’s support to IQSs is particularly uncertain because the institutionalmandate over IQSs is unclear, the number ofwell-established IQSs available for scale-up is limited, andthere are supply-side constraints, particularly with regards to facilitators. One further constraint is thatstakeholdersdonothaveaclearunderstandingofhowscale-upisexpectedtotakeplace.
Overall,thebaselinefindingspointtoanumberofriskstothecausallinksintheToC.Criticalassumptionsrelatedtothemanagementandresourcingoftheeducationsystemarehighlyuncertain,inparticularthereleaseofgovernmentfunding,school investmentandhumanresourcecapacityremainingonaparwithincreasedenrolment,effectivemonitoringattheschoolandinterventionlevels,andSBMCs’abilitytoplaythewide-rangingrolethatisbeingsoughtaspartoftheproject.
Baseline findings on implementation capacity are mixed. The project makes an appreciable effort toinvolve a variety of stakeholders. However, their involvement in operational planning is uneven across
EvaluationoftheGirl’sEducationProject3
©EDOREN v
interventions. Government capacity building is embedded in project design, but an effective system isneeded to identify and address themost important capacity development needs.Monitoring capacity isrightly emphasised in GEP3’s re-design but its operationalisation has yet to advance. With regards toequity,severalGEP3interventionsaredesignedtobeequityenhancing,butitisunclearwhetherthemostvulnerablehouseholdsandgroupsareabletobenefitfromandactivelyparticipateinGEP3interventions.Notably,GEP3isnolongeroperatinginLocalGovernmentAreas(LGAs)withthehighestgendergapinallstates.
ImpactevaluationofGEP3’searlylearningintervention
GEP3’searlylearninginterventionaimstoimprovetheearlylearningskillsofchildreninprimaryGrades
1to3(Primary1(P1)–Primary3(P3))inthemothertongue,whilealsopreparingchildrentolearnwithEnglish as a language of instruction by the time they transition to Grade 4. A key measure of theintervention’ssuccesswillbe improved literacyskills.The interventionwillbe implementedoverathree-yearperiod (2016–2018) insixLGAs inZamfaraandKatsina (threeperstate).The interventionhas threekeycomponentsatschoolandcommunitylevel:theprovisionofapackageofHausa-mediumteachingandlearning materials to schools; early grade professional development for teachers and head teachers(includingmonthlyschoolvisits);andasetofcommunityawarenessandengagementactivitiestosupportearlygradeliteracy.
Methodology
The evaluation of the early learning intervention uses a theory-based approach and is designed as a
clusteredrandomisedcontrolledtrial(RCT),stratifiedbyLGAandtypeofschool(primaryschoolvs.IQS),and randomised at the school level. The intervention’s ToC was used as a framework to formulate theevaluationquestions.TheRCTdesignallowstheevaluationteamtomeasuretheattributableimpactoftheearly learninginterventiononlearningoutcomesbycomparingoutcomechangesinatreatmentgroupofschoolswith those ina control group that is statistically similaronaverage.TheRCTdesign is combinedwith the overarching theory-based evaluation approach to measure not only changes in final outcomevariablesbutalsonetchangesinintermediaryoutcomevariablesalongtheassumedcausalchain.Thiswillallowtheevaluationteamtounpackhowchangetakesplace.
Inorder tomeasure changes inoutcomesapanel surveywasdesigned fordata collectionatbaseline,
midlineandendline.Datawillbecollectedateachofthesethreestagesinasampleof120publicprimaryschoolsand120IQSsdrawnfromthesixinterventionLGAsacrossKatsinaandZamfara.Halfofthepublicprimary schools and IQSs form part of the treatment group, while the other half serves as the controlgroup. Baseline data collection was conducted in October–November 2015 during the first term of the2015–2016 school yearbefore the startof implementationof theearly learning intervention.Within thesample schools, male and female pupils and teachers were randomly sampled to form part of a panelsurvey.Sevendatacollection instrumentswereadministeredwithineachschool:pupilEnglishandHausaliteracy assessments, pupil and teacher questionnaires, a teacher knowledge and skills assessment, ateacherclassroomobservationandaheadteacherquestionnaire.Theliteracyassessmentswerecarefullydesignedandpilotedtoensurethatitemdifficultymatchedpupils’ability.
Analysisofthebaselinedataindicatedthatrandomisationhadworkedtocreatecomparabletreatment
and control groups. We assessed whether the randomisation had achieved its intended purpose bychecking whether key outcome variables and school-, teacher- and pupil-level characteristics differedbetweenthetreatmentandcontrolgroupsatbaseline.Thegreatmajorityofvariablesinvestigateddidnotshowanystatisticallysignificantdifferencebetweenthetwogroups.
Keyfindings
EvaluationoftheGirl’sEducationProject3
©EDOREN vi
Thebaselinefindingshighlightthattheearlylearninginterventionwillbeimplementedinachallenging
context.Thevastmajorityofschoolsarelocatedinruralareasandhavepoorinfrastructure.Whilealargershare of IQSs than public primary schools have access to drinking water and electricity, they have lessrooms on average and have less access to books and functional toilets for girls. Teachers in IQSs arepredominantlymale. The baseline also found that 40% of IQSs only have one teacher who teaches theintegrated curriculum subjects, which raises questions about the applicability of the school-level peermentoringapproachintheseIQSs.
Head teachers make some effort to address teacher attendance, but are less likely to take action to
improvethequalityofinstruction.Almost90%ofheadteachersinpublicprimaryschoolsreportedtakingactiononteacherattendance.ThecorrespondingfigureforIQSswasjustover50%.Overhalfoftheheadteachers interviewedhadnotobserveda single lessonduring theprevious school term.Similarly,halfofheadteachersdonothavemeetingswithteachersormeetthemlessthanonceamonth.
Anotableshareofearlygrade teachersdonot teach languages.Roughlyhalfof teachers reported thattheyteachonlyonesubject,ofwhichjustover40%teachasubjectotherthanHausaorEnglish.GiventhattheReadingandNumeracyActivity (RANA)willemphasiseearlygradereading, thispoolof teachersmaynot be suitable candidates for the intervention. This needs to be taken into account when selectingteachersfortraining.
Theteacherssurveyeddemonstratedverylimitedknowledgeandskillsinrespectofmostofthedomains
covered by the teacher assessments. Less than 3% of teachers were able to display competence inidentifying lowperformers,evidencing judgementsanddiagnosingpupilperformance, interpretingwordsandphrases,andwritingskills.AlthoughallteachersreportedthattheyspeakHausa, lessthan40%wereabletodisplaycompetenceinprimaryGrade1and2-levelHausa.ThisissignificantforaninterventionthatfocusesonteachinginHausaasteachers’subjectknowledgehasimportantimplicationsforthequalityofteaching.Teachers’verypoorskills in identifying lowperformersanddiagnosingpupils’performancearelikely to present key hurdles to improving teaching quality given the wide recognition in the educationliteraturethatchildrenlearnbestwhenteachingistargetedtowhatthechildisreadytolearn.Therearesomeminorvariationsinperformanceacrossdifferentgroupsofteachers,butknowledgeandskillslevelsareconsistentlylowwithinallgroups.
Thesefindingsregardingteachers’knowledgeandskillshaveafewkeyimplicationsfortheearlylearning
intervention.Theverylowlevelsofknowledgeandskillsamongstteachersatbaselineindicatesthatthereis substantial scope for improvement in this area. However, it also presents challenges, in that there isoftenalackofbasicfoundationalskillsthattheprojectcanbuildon.Thefindingsshouldserveasausefulinput to the RANA implementation team as they highlight some of the key areas of weakness that theintervention would need to address. They also provide an indication of the scale of the challengesconfrontingtheproject,whichwillhaveimplicationsforimplementationdecisions–forinstancerelatedtothefrequencyandcontentoftraining,andthefocusoftheschoolvisits.Classroom observations indicate that the quality of early grade instruction is low across all groups of
teachers.Theextentofpupil-centred learningobservedatbaselinewas low,aswas theextent towhichteacherslinkthelessontopreviouslearningandlearningobjectives.Incontrast,timeontaskwashigh,atanaverageof96%ofatotallesson.However,thisfindingshouldbeinterpretedwithcautionasitislikelythatthepresenceofobserversintheclassroomledtoanincreaseintimeontask.Thebaselinealsosoughttomeasuregender-sensitiveteachingpracticesusingclassroomobservationdata,butextremecomplianceeffectswereobservedacrossall items,raisingquestionsaboutthevalidityandreliabilityofthemeasure.Teachers’ responses to the assessments indicate that teachers are aware of objectives to target girls inclass,astheystatethatitisimportanttofocusongirls,buttheirresponsesalsopointtothepersistenceofdeeplyingrainedgenderbiases.
EvaluationoftheGirl’sEducationProject3
©EDOREN vii
Theavailabilityanduseof teachingand learningmaterialswasobserved tobevery low. This suggeststhatthedistributionoflearningmaterialsbytheRANAinterventioncouldfillanimportantgap.However,itwillbevital formaterialstobepitchedatanappropriate level, taking intoaccountthevery lowlevelsofknowledgeamongstbothteachersandpupils.Hausa-basedteachingisprevalentintheearlygrades.Hausawasusedinallclassesobservedatbaseline.In roughly half of classes, a second languagewas used in addition to Hausa. However, as noted above,manyteacherslackcompetenceinbasicHausa.The baseline sought to assess teacher motivation and attendance as these variables are likely to
influence the extent to which improved knowledge and skills amongst teachers translate into better
teaching.Teachermotivationwas relativelyhighon threedimensions: theeffortput into,andperceivedimportance, of teachers’ work; their enjoyment of teaching; and interaction between teachers. Scoresrelated to pressure andwork-related tension and teachers’ perceived self-efficacywere low. Therewaslimited variation across teachers. Absenteeism was higher in IQSs than in public primary schools, withfacilitators reporting that theywere absent for an averageof eight days over theprevious 60days. Thecorrespondingfigureforpublicschoolswasthreedays.ThebaselinefindingsindicatethatinbothEnglishandHausa,veryfewpupilshavelevelsofknowledge
that are appropriate for their grade. In both subjects, the vast majority of P2 pupils assessed haveknowledgeand skills expectedofpre-school children (pre-literacy skills). Furthermore, inboth cases, thepeak of the distribution of performance falls well below the cut-off point between pre-literacy andemergingliteracy.Thissuggeststhatsubstantialeffortwouldbeneededtoachieveasignificantincreaseintheshareofpupilsmovingfrompre-literacytoemergingliteracy.Pupils’knowledgeofphonic isparticularly low.Thepsychometricanalysis foundthat itemsthat requireknowledgeofphonicsrankasthemostdifficultitemsinboththeHausaandEnglishassessments.Correctlysoundingoutlettersandidentifyingsimilarsoundswasmoredifficultforpupilsthanwritingorreadingfullpassages.Learningoutcomesdiffer by age and gender.Older pupils in P2 performbetter than younger ones andboys perform better than girls. Gender differences in performance are small in the younger years, butincreaseoncegirlsreachpuberty(around12yearsofage).Regressionanalysis indicatesthatpupilcharacteristics,socio-economicbackgroundandschooltypeare
associatedwithlearningoutcomes,butteachercharacteristicsarenot.Thelatterfindingmayreflectthefactthatcompetencylevelsareverylowacrossallteachers.Itislikelythatastrongercorrelationwouldbedetected if there were greater variation in competency levels across teachers, and if their ability toinfluencepupils’learningoutcomeswashigher.Thisalsoimpliesthatiftheearlylearninginterventionleadsto substantial improvements in teachers’ knowledge and skills we should see a more significant linkbetweenteachercharacteristicsandlearningoutcomesatendline.
EvaluationofGEP3’ssupporttoIQSs
GEP3’s support to IQSs (IQSS) seeks to improve education outcomes at these schools, with the aim ofprovidinganacceptablealternativeformofqualitybasiceducationforgirls.Theinterventionseekstoraiselearning outcomes in basic literacy and numeracy for pupils at IQSs (especially girls), improve retentionamong girls, and (to a lesser extent) increase girls’ enrolment. The project targets registered Islamiyya,Qur’anicTsangayaEducation(IQTE)centresthatofferanintegratedcurriculumandinwhichatleast40%ofpupilsaregirls.TheseIQSslargelyoperateascommunity-basedinitiatives,butarewillingtobuildlinkswithgovernmentforthepurposesofmonitoringandtechnicalsupport.GEP3’sIQSSinvolvestheprovisionof training and mentoring of IQS facilitators, training for head teachers, the distribution of classroom
EvaluationoftheGirl’sEducationProject3
©EDOREN viii
teachingandlearningmaterials,capacitybuildingforCentre-BasedManagementCommittees(CBMCs)andtheprovisionofmini-grants.
Methodology
Theevaluationapproachdrawsontheprinciplesofcontributionanalysisandreliesonastrongmixof
quantitative and qualitative methods. It does not make use of a comparison group to make causalinferences but rather seeks to make credible causal claims about the intervention’s contribution toeducationoutcomesbyverifyingthechainofexpectedresultsandassumptionsasperacredibleToC,aswell as assessing alternative explanations for the outcomes observed. The evaluation will assess threecontributionclaims:• GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs;• GEP3’sIQSScontributestoanimproved,girl-friendlyenvironmentwithintheschools;and• more effective teaching of formal subjects and an improved, girl-friendly environment contribute to
improvedlearninglevels,particularlyamonggirls.
ThequantitativecomponentoftheevaluationconsistsofrepresentativesamplesurveysamongGEP3IQSsinthe12GEP3LGAsinBauchiandNiger.Thebaselinesurveytookplaceinasampleof60IQSs,stratifiedbyLGA.Itinvolvedtheadministrationofthesameseveninstrumentsusedfortheearlylearninginterventionevaluation, as well as a pupil numeracy assessment and CBMC questionnaire. The baseline qualitativeresearchtookplaceinsixIQSsthatwerepurposivelysampledusingtypicalandextremecasesamplingandthat were also included in the quantitative survey. The same set of IQSs will be visited at midline andendline.
Keyfindings
ContributionClaim1:GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs
The landscape of IQSs is complex. Variations in school structures, pupil–teacher ratios, facilitators’qualifications,schoolleadershipandmanagement,andthenumberandhoursforwhichintegratedsubjectsaretaughtperschoolsuggestthattheoutcomesofinterveningintheseschoolsarelikelytodifferfromoneanother. Some IQSsmaybemobile,whichwouldhave implications for continued teachingand learning,particularly for girlswho are unlikely tomovewith theMallam.1 The extent of integration varies acrossIQSs,andisloweronaverageinBauchithaninNiger.Acrossthetwostates,headteachersreportedthatanaverageof threehoursperweek is spentonteaching the integratedcurriculum,which iswellbelowtheeight hours recommended by official guidelines. A quarter of IQSs only have one facilitator, which willconstrainamentoringprocessbasedonschool-basedpeerinteraction.Onecrucialsupportivefactoristhatintegrationdoesappeartohavegainedacceptanceamongparentsandthecommunity.Facilitators’knowledgeandskillsinkeydomainsassociatedwitheffectiveteachingareverylow.Only2%of facilitators were able to display competence in identifying low performers. Less than 1% hadcompetence in writing skills and none of the facilitators assessed were competent in evidencingjudgementsanddiagnosingpupilperformance.Only34%offacilitatorswereabletodisplaycompetenceinGrade1and2-levelHausa.Mostfacilitatorsdisplayalackofunderstandingofwhatstepstheycantaketoimprovepupils’performance.Aswiththeearly learning intervention,thesefindings indicatethatthere is
1ThetermMallamcanhavedifferentinterpretations.WeconsidertheMallamtobetheheadofthereligiousschool,responsibleforreligiouseducation.
EvaluationoftheGirl’sEducationProject3
©EDOREN ix
tremendous scope for improvement in teachers’ performance, but also that it will be crucial for theinterventiontobecarefullytargetedtotheverylowlevelsofskillsandknowledgeamongstfacilitators.Facilitators scoredverypoorlyona composite indexof effective classroompractices. Inparticular, thedepthofpupil-centredlearningwasfoundtobelowandtherewaslittleeffortamongfacilitatorstolinkthelesson to previous learning and learning objectives. Time on task was generally high: in 65% of lessonsobservedpupilsspent100%ofthelessonontask.However,thismayhavebeeninfluencedbythepresenceofobservers.Thereismixedevidenceonfacilitators’attitudesandclassroompracticestowardsgirls.Thefacilitatorsinthesixcasestudyschoolsexpressedpositiveattitudestowardsgirls’education,althoughtheseresponsesarepronetosocialdesirabilitybias.Indeed,bothqualitativeandquantitativedatapointtothepersistenceofgenderbiases–forinstancerelatedtogirls’abilities.Thefindingsindicatethatfacilitatorsareawareofand sometimes practice gender-sensitive techniques, such as actively engaging both girls and boys.However, thesepractices seem tobeperformed in a tokenisticway—in linewithdevelopmentprojects’sensitisationefforts—andgender-biasedclassroompracticescontinuetotakeplaceinIQSs.Aswiththeearlylearningintervention,thebaselineevaluationsoughttoassessfacilitators’motivation
and attendance. On average, facilitators think that their role is important and they enjoy working asteachers, but they have poor perceptions of their teaching efficacy. The qualitative research found thatfacilitatorsfeelintrinsicallymotivatedbywhattheydo,andthatschoolstakeholders,includingpupils,thinkthatfacilitatorsasgenerallydedicated.However,italsoindicatesthatthefactthatmostfacilitatorsarenotpaidmakesitdifficulttoattractqualifiedfacilitatorsandholdthemaccountable.Thequantitativefindingsconfirmthataverysmallshareoffacilitatorsarepaidasalaryorastipend(3%inBauchi,33%inNiger)andpointtoapositivelinkbetweenreceivingremunerationandteachermotivation.Roughly75%offacilitatorsreported that they had been absent at least once in the previous threemonths. Furthermore, the casestudy findings indicate that few IQSshave set timetables, and formal subjects are taughtasandwhenafacilitatorisavailable.Theavailabilityanduseofteachingandlearningmaterialsisverylimited.AlmostnoHausamaterialsareavailable,althoughitisimportanttonotethatHausaisnotthemothertongueofallpupils.InBauchi,93%of sampledchildren reported speakingHausaathomebut the corresponding figure inNiger is just54%,with43%speakingNupe.SchoolleadershipinIQSsiscomplex,whichraisessomechallengingquestionsaboutwhopreciselyshouldbe targeted by the pedagogical leadership component of the IQS intervention. IQSs have a variety ofdifferent leadership roles (Mallam, head teachers, proprietor). These may be held by one person ordifferentpeopleandthewayinwhichresponsibilitiesaresplitacrosstheseindividualsvariesacrossIQSs.Theprofessionalandacademicqualificationsofheadteachers inIQSsaregenerally low,and,onaverage,below those of facilitators. This raises questions about head teachers’ technical capacity to exercisepedagogical leadership over facilitators. The appointment of a head teacher is not necessarily based onability and qualifications but has to dowith social statuswithin the community and perceptions aroundleadershipmoregenerally.
ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlyschoolenvironmentinIQSs
CBMCs have some potential to bring about improvements in the learning environment in IQSs. Themanagement of IQSs is seen as collective, without clearly defined and delegated roles, which providesopportunitiesforcommunity-basedmanagementstructureslikeCBMCs.Thebaselinefindingsalsoindicatethatmost CBMCs are established and active, as indicated by regularmeetings andmonitoring activities.CBMCs have considerable community representation, although women are under-represented and veryfewmembersarechildren.This indicatesthatatpresentCBMCsareunlikelytoprovideaforuminwhichgirls’voicesandneedscanbeheard. Inall IQSs included in thequalitativestudytheMallamholdsakeypositionintheCBMC,suchaschairperson.ThisraisesquestionsabouttheextenttowhichCBMCscanplayaneffectiveroleinholdingschoolleaderstoaccount.
EvaluationoftheGirl’sEducationProject3
©EDOREN x
CBMCsdomakeanefforttoraiseresourcesfromthecommunity,butthisappearstoofferlimitedscope
inregardtobridgingresourcegapsinIQSs.ThequalitativeresearchindicatesthatCBMCsfeelthatalackof fundshinders theirability to ‘solve’problemsat IQSs.Theyalsonote thatwhileparentsarewilling tocontributeresources,theyoftenhavelimitedcapacitytodoso.ThesefindingshighlightthecaseforCBMCtrainingtogobeyondanarrowfocusoncommunityresourcemobilisationtoencompassthemobilisationofresourcesfromalternativesources.
CBMCmembersseemtolargelyunderstandtheirrolesandresponsibilitiesbutoftenlackthecapacityto
enact them. Key areas in which CBMCs are under-performing are school planning and financialmanagement.CBMCmembersdomakeaneffort tomonitor thequalityof the teachingand learning, inparticular pupils’ and facilitators’ attendance. CBMC members assert that the main constraint thatunderminestheirperformanceisalackoffunds,particularlytoimproveinfrastructureandpayfacilitators.
Most IQSs do not offer a girl-friendly school environment at present. The vast majority face majordeficiencies inphysical infrastructure.Only3%of the schoolshaveaccess toawater source,20%of theIQSsdonothaveaphysicalclassroomstructureforthestudents,andlessthan25%havefunctioningtoiletsforpupils,withonly8%havingfunctioningtoiletsforgirls.Only3%ofIQSshaveamothers’group,teacher–student association or pupil groupwhere students can discuss their concerns. Gender-biased classroompracticesandattitudesprevail,whichcanaffectgirls’learningopportunitiesandself-confidence.However,there appears to be an openness among school leaders to try and address these issues. ResourcesmobilisedbyCBMCsareinvestedinschoolsandgirl-friendlyinvestmentsareconsidered,buttheamountsraisedappearinsufficientgiventhepoorconditionoftheschoolenvironment.
Theextentofgovernmentengagementwith IQSs is limited.Themajorityof IQSs in thesamplehadnotbeen visited by a government official during the previous term. Government officials say that they facechallenges in reaching all the schools on a regular basis. Communities generally view support from thegovernmentnegatively,consideringgovernmentactorstobeunreliable.
ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlyschool
environmentcontributetoimprovedlearninglevels,particularlyamonggirls
Thebaselineevaluationidentifiessomekeyfindingsthatarerelevanttothiscontributionclaim.Girlsandboys are both equally likely to attend IQSs. However, both face challenges in regard to attending andremaining fully engaged with the learning process because of out-of-school responsibilities, which arelinked to household poverty. A large share of childrenwho study in IQSs attend another school aswell.There were significant cross-state differences, with 72% of pupils in Bauchi reporting attending otherschools,ascomparedtoonly13%pupilsinNiger.InBauchi,mostofthesechildrenwereattendingapublicprimary school (94%), as compared to 55% in Niger. This indicates that interventions at local primaryschoolsmayconstituteanalternativeexplanationforanyimprovementsinlearningoutcomesatIQSs.
Pupils’ learningoutcomesarevery low. InbothHausaandEnglishover90%ofGrade2pupilsassesseddisplayed literacy levels associatedwith pre-school pupils. Furthermore,most of these pupils are a longway from the cut-off for emerging literacy skills (those associated with the P1 curriculum). Numeracyoutcomeswerebetter,with69%ofpupilsdisplayingemergingnumeracyskills,comparedto21%withpre-numeracyskills.However,only11%ofpupilshadnumeracyskillsassociatedwiththeP2curriculum.Boysandgirlsbothperformpoorly,althoughgirls’performancedeclines,comparedtothatofboys,ataroundpuberty(roughly12yearsofage).Continuedbarrierstogirls’accessto,andretentionin,schoolingexist.Attitudestowardsgirls’educationarenotalwayssupportive,althoughthisappearstobechanging.Changingattitudesalone,however,maynot be sufficient to bring about change in behaviour as poverty is often cited as a critical reason whyparentsdonotsend theirchild toanyschool,publicprimaryor IQS.Bothboysandgirlsoftenengage insomeformofincome-generatingactivity,whichparentstendtoviewasafinancialandpracticalnecessity.
EvaluationoftheGirl’sEducationProject3
©EDOREN xi
Thisadverselyaffectsschoolattendance.Earlymarriageisalsocitedasakeyreasonforlowenrolmentandretentionamonggirls.
Recommendations
ThebaselinefindingspointtocertainkeyrecommendationsforGEP3.ThebaselineassessmentofGEP3’sToC highlights that continued political engagement is required to ensure that government funds aremobilisedforprojectscale-up.Theprojectshouldspecifyandcommunicateitsoperationalobjectivesandstrategies to government andother implementing partners, strengthen itsmonitoring of assumptions inthe ToC, and ensure that monitoring information is used to facilitate learning and accountability. Inparticular,emphasisshouldbeplacedonmonitoringGEP3’ssupporttoSBMCs/CBMCs,giventheirpivotalroleintheproject’sToC.
Baseline findingson theearly learning interventionhighlight that theRANA interventionwillneed tobecarefully tailored to thevery low levelsofknowledgeandskillsamongst teachers toensure that trainingcontentandmaterialsarepitchedattherightlevel,andthattrainingissufficientlyintensivetofillthelargegapsinteachers’subject,pedagogicalandcurriculumknowledge.Teachercapacitydevelopmentalsoneedsto incorporate actions to change teachers’ awareness of their own potential for influencing learningoutcomes,andtheirunderstandingofhowbesttosupportpupils’learning.Thepeermentoringapproachneedstobeadaptedfor the largenumberof IQSsthatonlyhaveoneteacherwho iseligible forsupportunder the intervention. RANA also needs to sufficiently strengthen the capacity of IQS governmentstakeholders to ensure that they effectively monitor and provide support supervision. Finally, thecomplexityoftheIQSleadershipmodelindicatesthatspecialattentionneedstobegiventoensuringthatappropriateindividualsatIQSsaretargetedfortheprogramme’spedagogicalleadershiptraining.
ThefindingsontheIQSSinterventionhighlighttheneedfortheprojecttostrengthenitsmonitoringoftheintervention. The IQS context is diverse, flexible and evolving. In order to adapt the intervention to thiscontext,quicklearningandfeedbackbasedonmonitoringdataisneeded.Theselectionoffacilitators,headteachersandCBMCmembersfortrainingrequirescloseattention,verificationandmonitoring.FacilitatortrainingandmentoringneedstobecarefullyadaptedtotheverylowlevelsoffacilitatorcompetencyandtheIQScontext.Similarly,learningandteachingmaterialsneedtobetailoredtotheverylowlevelsofskillsandknowledgeamongstbothfacilitatorsandpupils,andtothelanguageoftheusers(particularlyinNiger,where Nupe is widely spoken). Training and mentoring on gender-sensitive class practices needs to gobeyond a focus on gender-sensitive teaching practices to try and tackle entrenched gender biases, forinstancerelatedtogirls’abilitytolearn.Womenandgirls’participationinshapingeducationandtheschoolenvironmentneedsfurther investigation:atpresent,bothareunder-representedonCBMCs,which limitsthe scope for thesebodies toact as forums forwomenandgirls’ participation in the sector. Finally, thebaseline findings highlight the need for GEP3 to promote the mobilisation of resources beyond thecommunityandtoadvocateforfacilitatorstobeadequatelyremuneratedbythegovernment.
EvaluationoftheGirl’sEducationProject3
©EDOREN xii
Tableofcontents
Acknowledgements i
Executivesummary iiiExaminationofGEP3’sToC iiiImpactevaluationofGEP3’searlylearningintervention vEvaluationofGEP3’ssupporttoIQSs viiRecommendations xi
Listoffigures,tablesandboxes xiv
Listofabbreviations xviii
1 Introduction 11.1 Background 11.2 Objectivesofthebaseline 11.3 Organisationofthereport 2
2 BaselineassessmentofGEP3’sTheoryofChange 32.1 Objectiveofthisreport 32.2 Methodology 32.3 PlausibilityofGEP3’sToC 42.4 AppropriatenessofGEP3’simplementationstrategy 35
3 Baselineevaluation–Earlylearningintervention 433.1 GEP3’searlylearningintervention 433.2 Methodology 493.3 Comparisonofbaselinecharacteristicsbetweeninterventionandcontrolschools 873.4 Analysisofthedata 89
4 BaselineoftheIQSSevaluation 1374.1 GEP3’sIQSS 1374.2 Methodology 1414.3 PresentationoftheupdatedToC 1674.4 Analysisofthedata–ContributionClaim1:GEP3’ssupporttoIQSscontributestomore
effectiveteachingofformalsubjectsinIQSs 1774.5 Analysisofthedata–ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-
friendlylearningenvironment 2214.6 Analysisofthedata-ContributionClaim3:Moreeffectiveteachingofformalsubjectsandan
improved,girl-friendlyschoolenvironmentcontributetoimprovedlearninglevels,particularlyamonggirls 248
Bibliography 270
AnnexA GEP3TheoryofChangeDiagram 273
AnnexB BaselineTheoryofChangeassessment 274B.1 Focusofkeyinformantinterviewsperstate 274B.2 OverviewofkeyinformantsinterviewedforthebaselineTheoryofChangeassessment 275
AnnexC Mapsofsampledschools 280
EvaluationoftheGirl’sEducationProject3
©EDOREN xiii
AnnexD CalculationofWeightsandFinitePopulationCorrectionFactorsforGEP3School,TeacherandPupilData 284
D.1 CalculationofGEP3Weights 284D.2 FinitePopulationCorrectionFactorsfortheGEP3Analysis 286
AnnexE EthicalReview 288
AnnexF TeachermotivationsubscalesforGEP3 290F.1 Distributionofthefiveteachermotivationsubscales 290
AnnexG Randomisationchecksoftreatmentandcontrolgroup 292G.1 SchoolLevelBalance 292G.2 PupilLevelBalance 297
AnnexH QualitativeEvaluationMatrixforSupportforIQS 301
AnnexI Evaluationgovernance,managementandindependence 302I.1 Governanceandmanagement 302I.2 Stakeholderinvolvementinframeworkdevelopment 303
AnnexJ RobustnessChecksonAgeforMainRegressionModel 304ExclusionofAgeVariable 305
AnnexK GEP3AssetIndex 308
AnnexL EarlyLearningDescriptiveStatistics 314
AnnexM IQSchoolSupport(IQSS)DescriptiveStatistics 326
AnnexN CBMCIQSSDescriptiveStatistics 342
EvaluationoftheGirl’sEducationProject3
©EDOREN xiv
Listoffigures,tablesandboxes
Figure1: DiagramdepictingToCfortheearlylearningintervention......................................................46Figure2: TimelineofdatacollectionandtheRANAintervention...........................................................52Figure3: Schoolentryprotocol...............................................................................................................58Figure4: Teachersampling......................................................................................................................59Figure5: Pupilselection–Headteacher’soffice.....................................................................................60Figure6: Pupilselection–classroom......................................................................................................61Figure7: Instrumentdevelopmentprocess............................................................................................62Figure8: GEP3surveyteamstructure.....................................................................................................67Figure9: Teacherknowledgemodel.......................................................................................................78Figure10: Theorisedcausalpathway(takenfromtheTDP).....................................................................82Figure11: Schoolsizebasedonaveragenumberofteachers,byschooltype.........................................91Figure12: Prevalenceofheadteachermanagementactions...................................................................92Figure13: Actionstakenandagegroups..................................................................................................93Figure14: SchoolstructureinIQSs............................................................................................................95Figure15: InfrastructureinKatsinaandZamfara......................................................................................96Figure16: Teacherqualificationandeducationbyschooltype................................................................99Figure17: Percentageofteachersachievingwithinthelower,middleandupperbandsofcompetence
acrosstheteacherknowledgeandskillssubscales................................................................102Figure18: Percentageofobservedteachersengagingindifferentkindsofteachertalk.......................107Figure19: Percentageofobservedteachersengagingindifferenttypesofteacheraction...................107Figure20: Percentageofclassroomswherepupilsengagedindifferenttypesofpupilaction..............108Figure21: Percentageofteachersscoringzeroinpedagogical,curriculumandsubjectknowledge.....110Figure22: Meanpupil-centredteachingscorebyyearsofexperienceasateacher..............................112Figure23: Percentageoftimeontask.....................................................................................................112Figure24: PercentageofclassroomsusingEnglish,HausaandArabic...................................................114Figure25: FrequencyofHausauseinclass.............................................................................................114Figure26: Comparisonofmotivationsubscales......................................................................................116Figure27: Motivationbyagegroups.......................................................................................................116Figure28: DistributionofagegroupsinIQSsandpublicprimaryschools.............................................119Figure29: HWItertilecategorisationbystate.........................................................................................120Figure30: Categoriesoffactorsthatinfluencelearningoutcomes........................................................121Figure31: DistributionofHausaliteracyproficiency..............................................................................123Figure32: DistributionofEnglishliteracyproficiency.............................................................................125Figure33: Hausaliteracybygenderandschooltype..............................................................................127Figure34: Hausameanscalescorebyage(95%confidenceinterval)....................................................128Figure35: MeanHausascalescorebyageandgender(95%confidenceinterval).................................128Figure36: Hausabywealth.....................................................................................................................129Figure37: InterventionlogicdiagramoftheIQSS...................................................................................141Figure38: TimelineforIQSSactivitiesanddatacollectioninGEP3pilotIQSs........................................146Figure39: Teachersampling....................................................................................................................152Figure40: Flowchartofdatacollectionrelatingtofacilitators...............................................................156Figure41: DiagramdepictingToCoftheIQSS.........................................................................................169Figure42: Causalpackageandunderlyingassumptionsof‘moreeffectiveteaching’............................172Figure43: Causalpackageandunderlyingassumptionsof‘improved,girl-friendlyschoolenvironment’...
.............................................................................................................................................174Figure44: Causalpackageandunderlyingassumptionsof‘improvedlearningoutcomes’....................176Figure45: Numberofintegratedsubjectstaughtperfacilitator............................................................181Figure46: Facilitatortrainingcontent,bystate......................................................................................182Figure47: Religioussubjectstaughtinschools,bystate........................................................................184Figure48: Percentageoffacilitatorsachievingwithinthelower,middleandupperbandsofcompetence
acrosstheteacherknowledgeandskillssubscales................................................................185
EvaluationoftheGirl’sEducationProject3
©EDOREN xv
Figure49: Percentageofobservedfacilitatorsengagingindifferentteachertalkactivities..................189Figure50: Percentageofobservedclassroomswherepupilsengagedinvarioustypesofpupilaction190Figure51: Numberoflanguagesusedbythefacilitatorduringonelesson............................................191Figure52: PercentageoffacilitatorsusingHausa,Arabic,Englishandotherlanguagesinclass............191Figure53: Percentageoffacilitatorsscoringzeroinpedagogical,curriculumandsubjectknowledge..193Figure54: Depthofpupil-centredteaching,bystate..............................................................................194Figure55: Percentageoftimeontaskinobservedlessons....................................................................195Figure56: Lengthoflessonsobserved,bystate......................................................................................195Figure57: MotivationsubscalesforIQSfacilitators................................................................................196Figure58: Availabilityanduseofresourcesinobservedlessons............................................................199Figure59: Contentofheadteachertraining,bystate............................................................................200Figure60: Frequencyofheadteachermeetingswithteachers,bytrainingstatusofheadteacher......201Figure61: Frequencyofheadteachermeetingswithteachers,bystate...............................................202Figure62: Availabilityofschoolrecords,bystate...................................................................................203Figure63: AveragenumberofmembersinCBMCs.................................................................................223Figure64: ContentofCBMCtraining.......................................................................................................224Figure65: ActiontakenbyCBMCstoaddresspupildropout,bystate...................................................226Figure66: AmountmobilisedbyCBMCs.................................................................................................227Figure67: ReceiptofGEPschoolgrant,bystate.....................................................................................227Figure68: Infrastructurerepairneeds,bystate......................................................................................230Figure69: Investmentcategories,bystate.............................................................................................231Figure70: Sourcesofmonitoringvisitstoschools,bystate...................................................................232Figure71: Ageofpupilsbygender..........................................................................................................250Figure72: Languagesspokenathome,bystate.....................................................................................251Figure73: Languageofinstruction,bystate...........................................................................................252Figure74: DistributionofHausaliteracyproficiency..............................................................................254Figure75: Englishliteracyproficiencydistribution.................................................................................255Figure76: Distributionofnumeracyproficiency.....................................................................................257Figure77: MeanHausascalescorebyageandgender(95%confidenceinterval).................................258Figure78: MeanEnglishscalescorebyageandgender(95%confidenceinterval)...............................258Figure79: Meannumeracyscalescorebyageandgender(95%confidenceinterval)...........................259Figure80: MeanEnglishscalescores,bystate........................................................................................259Figure81: MeanHausascalescores,bystate.........................................................................................260Figure82: Meannumeracyscalescores,bystate...................................................................................260Figure83: GEP3evaluationgovernancearrangements..........................................................................302Figure1: DiagramdepictingToCfortheearlylearningintervention......................................................46Figure2: TimelineofdatacollectionandtheRANAintervention...........................................................52Figure3: Schoolentryprotocol...............................................................................................................58Figure4: Teachersampling......................................................................................................................59Figure5: Pupilselection–Headteacher’soffice.....................................................................................60Figure6: Pupilselection–classroom......................................................................................................61Figure7: Instrumentdevelopmentprocess............................................................................................62Figure8: GEP3surveyteamstructure.....................................................................................................67Figure9: Teacherknowledgemodel.......................................................................................................78Figure10: Theorisedcausalpathway(takenfromtheTDP).....................................................................82Figure11: Schoolsizebasedonaveragenumberofteachers,byschooltype.........................................91Figure12: Prevalenceofheadteachermanagementactions...................................................................92Figure13: Actionstakenandagegroups..................................................................................................93Figure14: SchoolstructureinIQSs............................................................................................................95Figure15: InfrastructureinKatsinaandZamfara......................................................................................96Figure16: Teacherqualificationandeducationbyschooltype................................................................99
EvaluationoftheGirl’sEducationProject3
©EDOREN xvi
Figure17: Percentageofteachersachievingwithinthelower,middleandupperbandsofcompetenceacrosstheteacherknowledgeandskillssubscales................................................................102
Figure18: Percentageofobservedteachersengagingindifferentkindsofteachertalk.......................107Figure19: Percentageofobservedteachersengagingindifferenttypesofteacheraction...................107Figure20: Percentageofclassroomswherepupilsengagedindifferenttypesofpupilaction..............108Figure21: Percentageofteachersscoringzeroinpedagogical,curriculumandsubjectknowledge.....110Figure22: Meanpupil-centredteachingscorebyyearsofexperienceasateacher..............................112Figure23: Percentageoftimeontask.....................................................................................................112Figure24: PercentageofclassroomsusingEnglish,HausaandArabic...................................................114Figure25: FrequencyofHausauseinclass.............................................................................................114Figure26: Comparisonofmotivationsubscales......................................................................................116Figure27: Motivationbyagegroups.......................................................................................................116Figure28: DistributionofagegroupsinIQSsandpublicprimaryschools.............................................119Figure29: HWItertilecategorisationbystate.........................................................................................120Figure30: Categoriesoffactorsthatinfluencelearningoutcomes........................................................121Figure31: DistributionofHausaliteracyproficiency..............................................................................123Figure32: DistributionofEnglishliteracyproficiency.............................................................................125Figure33: Hausaliteracybygenderandschooltype..............................................................................127Figure34: Hausameanscalescorebyage(95%confidenceinterval)....................................................128Figure35: MeanHausascalescorebyageandgender(95%confidenceinterval).................................128Figure36: Hausabywealth.....................................................................................................................129Figure37: InterventionlogicdiagramoftheIQSS...................................................................................141Figure38: TimelineforIQSSactivitiesanddatacollectioninGEP3pilotIQSs........................................146Figure39: Teachersampling....................................................................................................................152Figure40: Flowchartofdatacollectionrelatingtofacilitators...............................................................156Figure41: DiagramdepictingToCoftheIQSS.........................................................................................169Figure42: Causalpackageandunderlyingassumptionsof‘moreeffectiveteaching’............................172Figure43: Causalpackageandunderlyingassumptionsof‘improved,girl-friendlyschoolenvironment’...
.............................................................................................................................................174Figure44: Causalpackageandunderlyingassumptionsof‘improvedlearningoutcomes’....................176Figure45: Numberofintegratedsubjectstaughtperfacilitator............................................................181Figure46: Facilitatortrainingcontent,bystate......................................................................................182Figure47: Religioussubjectstaughtinschools,bystate........................................................................184Figure48: Percentageoffacilitatorsachievingwithinthelower,middleandupperbandsofcompetence
acrosstheteacherknowledgeandskillssubscales................................................................185Figure49: Percentageofobservedfacilitatorsengagingindifferentteachertalkactivities..................189Figure50: Percentageofobservedclassroomswherepupilsengagedinvarioustypesofpupilaction190Figure51: Numberoflanguagesusedbythefacilitatorduringonelesson............................................191Figure52: PercentageoffacilitatorsusingHausa,Arabic,Englishandotherlanguagesinclass............191Figure53: Percentageoffacilitatorsscoringzeroinpedagogical,curriculumandsubjectknowledge..193Figure54: Depthofpupil-centredteaching,bystate..............................................................................194Figure55: Percentageoftimeontaskinobservedlessons....................................................................195Figure56: Lengthoflessonsobserved,bystate......................................................................................195Figure57: MotivationsubscalesforIQSfacilitators................................................................................196Figure58: Availabilityanduseofresourcesinobservedlessons............................................................199Figure59: Contentofheadteachertraining,bystate............................................................................200Figure60: Frequencyofheadteachermeetingswithteachers,bytrainingstatusofheadteacher......201Figure61: Frequencyofheadteachermeetingswithteachers,bystate...............................................202Figure62: Availabilityofschoolrecords,bystate...................................................................................203Figure63: AveragenumberofmembersinCBMCs.................................................................................223Figure64: ContentofCBMCtraining.......................................................................................................224Figure65: ActiontakenbyCBMCstoaddresspupildropout,bystate...................................................226Figure66: AmountmobilisedbyCBMCs.................................................................................................227
EvaluationoftheGirl’sEducationProject3
©EDOREN xvii
Figure67: ReceiptofGEPschoolgrant,bystate.....................................................................................227Figure68: Infrastructurerepairneeds,bystate......................................................................................230Figure69: Investmentcategories,bystate.............................................................................................231Figure70: Sourcesofmonitoringvisitstoschools,bystate...................................................................232Figure71: Ageofpupilsbygender..........................................................................................................250Figure72: Languagesspokenathome,bystate.....................................................................................251Figure73: Languageofinstruction,bystate...........................................................................................252Figure74: DistributionofHausaliteracyproficiency..............................................................................254Figure75: Englishliteracyproficiencydistribution.................................................................................255Figure76: Distributionofnumeracyproficiency.....................................................................................257Figure77: MeanHausascalescorebyageandgender(95%confidenceinterval).................................258Figure78: MeanEnglishscalescorebyageandgender(95%confidenceinterval)...............................258Figure79: Meannumeracyscalescorebyageandgender(95%confidenceinterval)...........................259Figure80: MeanEnglishscalescores,bystate........................................................................................259Figure81: MeanHausascalescores,bystate.........................................................................................260Figure82: Meannumeracyscalescores,bystate...................................................................................260Figure83: GEP3evaluationgovernancearrangements..........................................................................302Box1: RANA’sintermediateresultsandsub-components..................................................................43Box2: RANAmonitoringtools.............................................................................................................48Box3: EvaluationquestionsforevaluationofGEP3’searlylearningintervention.............................49Box4: P2ortheequivalentlevel.........................................................................................................54Box5: P1–P3orequivalentlevel.........................................................................................................54Box6: EstimationofMDEs..................................................................................................................56Box7: Explanatoryfactorsincludedinthepupilscorrelationmodel................................................130Box8: EvaluationquestionsforevaluationofGEP3’ssupporttoIQSs.............................................142Box9: KeystepsincontributionanalysisappliedtotheevaluationofGEP3’sIQSS.........................145Box10: EstimationofMDEs................................................................................................................149
EvaluationoftheGirl’sEducationProject3
©EDOREN xviii
Listofabbreviations
ASC AnnualSchoolCensus
BSAME BauchiSAME
CAPI Computer-assistedpersonalinterview
CBMC Centre-BasedManagementCommittee
CS CompositeSurvey
CSO Civilsocietyorganisation
DAC DevelopmentAssistanceCommittee
DFID DepartmentofInternationalDevelopment(UK)
ECCDE Earlychildcaredevelopmentandeducation
EDOREN Education,Data,ResearchandEvaluationinNigeria
EGRA/EGMA EarlyGradeReadingAssessment/EarlyGradeMathsAssessment
EMIS Educationmanagementinformationsystem
ERC EthicsReviewCommittee
ESSPIN EducationSectorSupportProgrammeinNigeria
FGD Focusgroupdiscussion
FME FederalMinistryofEducation
FOMWAN FederationofMuslimWomen’sAssociationsofNigeria
FTTSS FemaleTeacherTraineeScholarshipScheme
G4G GirlsforGirls
GEP3 Girl’sEducationProjectPhase3
GESCs Girls’EducationSteeringCommittees
GPE GlobalPartnershipforEducation
GPI GenderParityIndex
HiLWA High-LevelWomen’sAdvocates
HWI HouseholdWealthIndex
ICC Intra-clustercorrelation
IQQS IQSsupport
IQS IntegratedQur’anicschool
IQTEs Islamiyya,Qur’anicTsangayaEducationCentres
ITC Inter-temporalcorrelation
JSS JuniorsecondarySchool
KI Keyinformant
KII Keyinformantinterview
LGA LocalGovernmentArea
LGEA LocalGovernmentEducationArea
EvaluationoftheGirl’sEducationProject3
©EDOREN xix
M&E Monitoringandevaluation
MA Mothers’Association
MDAs Ministries,departmentsandagencies
MDE Minimumdetectableeffect
MTSS Medium-TermSectorStrategy
NCE NationalCertificateinEducation
NGN NigerianNaira
NHREC NationalHealthResearchEthicsCommittee(ofNigeria)
NIPEP NigeriaPartnershipforEducationProject
OECD OrganisationforEconomicCo-operationandDevelopment
OPM OxfordPolicyManagement
P2 Primarygrade2
PEA Politicaleconomyanalysis
PIU ProgrammeImplementationUnit
QCO Qualitativeclassroomobservation
RANA ReadingandNumeracyActivity
RARA ReadingandAccessResearchActivity
RCT Randomisedcontroltrial
RLP RANALiteracyPackage
SAME StateAgencyforMassEducation
SAVI StateAccountabilityandVoiceInitiative
SBMC School-BasedManagementCommittee
SESOP StateEducationSectorOperationalPlan
SMoE StateMinistryofEducation
SUBEB StateUniversalBasicEducationBoard
SPARC StatePartnershipforAccountability,ResponsivenessandCapability
SSCE SeniorSecondaryCertificateExamination
SWAGS StrengtheningWomenandGirls’Spaces
TDNA TeacherDevelopmentNeedsAssessment
TDP TeacherDevelopmentProgramme
TEGIN TransformingEducationforGirlsinNigeria
ToC Theoryofchange
TPD Teacherpracticediscussion
UBEC UniversalBasicEducationCommission
UNICEF UnitedNationsChildren’sFund
WCDP WholeCentreDevelopmentPlan
WSDP WholeSchoolDevelopmentPlan
EvaluationoftheGirl’sEducationProject3
©EDOREN 1
1 Introduction
1.1 Background
ThisreporthasbeenpreparedbyEducationData,ResearchandEvaluationinNigeria(EDOREN).EDORENiscontractedbytheUKDepartmentforInternationalDevelopmentinNigeria(DFIDNigeria)toevaluatetheGirlsEducationProjectPhase3(GEP3),inlinewiththeGEP3EvaluationFramework(EDOREN2015).
GEP3isaneight-yearproject(2012–20)thatseekstoensurethatmoregirlscompletebasic
educationandacquireskillsforlifeandlivelihoodinfivenorthernNigerianstates:Katsina,Sokoto,
Bauchi,NigerandZamfara.ItismanagedbyUNICEF,inpartnershipwithfederalandstategovernments,andisfundedbyUKAidDFID.Theprojectaimstoimproveaccess,retentionandlearningoutcomesforgirls.Between2014and2017,aseriesofinterventionsarebeingpilotedinpublicprimaryschoolsandintegratedQur’anicschools(IQSs).Thosemosteffectiveinimprovingeducationoutcomesforgirlswillbescaledup.During2014–2015,EDOREN,incollaborationwithUNICEFandDFIDNigeria,developedanevaluationframework,inparallelwithGEP3’sredesign.Theevaluationframeworkdefinedtheobjectives,scope,questions,designandpartnerresponsibilitiesoftheevaluationactivities,withafocusontheperiod2014–2017,toinformdecision-makingregardingscale-up.
ThescopeofEDOREN’sevaluationworkconsistsofthreesetsofevaluationquestionsandpieces
ofanalysis:
1. ahigh-levelexplicationandexaminationofGEP3’stheoryofchange(ToC);
2. animpactevaluationofGEP3’searlylearningintervention;and
3. aperformanceevaluationofGEP3’ssupporttoIQSs.
1.2 Objectivesofthebaseline
Theobjectivesofthebaselineanalysisare:
• tocapturetheeducationsituation,andperceptionsofthissituation,atthestartoftheredesignedGEP3project,withafocusonspecificinterventions,sothatchangescanbemeasuredduringfollow-updatacollectionrounds,andprojectattributionorcontributioncanbeassessed;
• toanswerspecificevaluationquestionsabouttherelevanceoftheproject;and• toprovideevidencetoinformtheGEP3ToCandprojectimplementation.
InlinewiththescopeofEDOREN’sevaluationworkthebaselineanalysisisorganisedintothreeparts:i)abaselineassessmentofGEP3’sToC,drawingmainlyonKIIswithstate-levelstakeholdersofGEP3;(ii)analysisofthequantitativebaselineresultsofinterventionandcontrolschoolsinLocalGovernmentAreas(LGAs)inwhichGEP3’searlylearninginterventionisbeingimplemented;and(iii)analysisofthequantitativeandqualitativebaselineresultsforIQSsthatareeligibleforGEP3support.
EvaluationoftheGirl’sEducationProject3
©EDOREN 2
1.3 Organisationofthereport
Thebaselinereportisorganisedintotwovolumes:asynthesisevaluationreportandatechnicalreport.Thesynthesisevaluationreportconsolidatesthemainfindingsandsupportingevidencecollectedthroughvariousevaluationmethods.Itisdesignedtobeaccessibletoallreaders.ThepresenttechnicalreportoftheGEP3’sbaselineanalysis,whichcomplementsthesynthesisbaselinereport,coversthemethodologyandtechnicalanalysisindetail.Itisintendedforthoseinterestedinthedesign,methods,detailedstatisticalresults,anddetailedqualitativeanalysisoftheevaluation.ItisstructuredaccordingtothethreecomponentsofEDOREN’sevaluationactivities:
Chapter2presentsthebaselineassessmentoftheGEP3ToC.Afterasummaryofthemethodologyused,itexaminestheplausibilityofGEP3’sToC.NextitassessestheappropriatenessofGEP3’simplementationstrategy.
Chapter3describesthemethodologyandbaselineresultsofGEP3’searlylearninginterventionevaluation.ItstartswithanoverviewoftheGEP3earlylearningintervention.Next,theevaluationandbaselinedatacollectionmethodologyispresented.Thisisfollowedbyacomparisonofbaselinecharacteristicsbetweeninterventionandcontrolschools,tocheckforbalanceaspartoftherandomisedcontroltrial(RCT)design.Thefollowingsectiondetailstheanalysisofthebaselinedata.
Chapter4providesapresentationofthemethodologyandbaselineresultsoftheevaluationofGEP3’ssupporttoIQSs.AnoverviewofGEP3’ssupporttoIQSsisfirstprovided,afterwhichtheevaluationdesignanddatacollectionmethodsarediscussed.Next,theanalysisofthequantitativeandqualitativedatacollectionispresentedaccordingtothecontributionclaimsthatareformulatedbasedontheinterventions’ToC.
Theannexesofthisreportprovideadditionaldetailedmethodologicalandanalyticalinformation.
EvaluationoftheGirl’sEducationProject3
©EDOREN 3
2 BaselineassessmentofGEP3’sTheoryofChange
2.1 Objectiveofthisreport
ThisreportpresentsthebaselineassessmentoftheGEP3ToC,inlinewiththeGEP3evaluationframework.ThereportfocusesontwomainevaluationquestionsaspartoftheevaluationofGEP3’srelevance:2
• HowplausibleisGEP3’sToCinthecontextoftheGEP3states?• HowappropriateareGEP3’sinterventions,intermsofimplementationstrategy,giventhe
contextoftheGEP3states?
AsdiscussedintheGEP3evaluationframework,afirststepintheevaluationprocessistofurtherexplicateandexaminetheToC–inparticularbyinterrogatingcausalpathwaysandinfluencingfactorsthatenableGEP3interventionstobringaboutintendedoutcomes.GEP3’sToCdiagramisincludedinAnnexA.3
TheassessmentpaysparticularattentiontoidentifyingfactorsthatarelikelytoinfluencewhetherGEP3achievesitsoutputsandoutcomes.Thisallowsimplementingpartnerstopayattentiontothesefactorsduringprojectimplementation,inordertoimprovetheproject’slikelihoodforsuccessandforM&EtoassessandaccountforhowGEP3isachievingitsoutcomes.
2.2 Methodology
TheassessmentofGEP3’sToCisbasedonKIIsthatwereundertakenmostlywithstate-levelstakeholdersofGEP3.Anumberofnational-levelstakeholdersoractorsworkinginsectorsrelevanttoGEP3interventionswerealsointerviewed.Furthermore,theassessmentmakesuseofadocumentreview,althoughanextensivereviewofsecondarydatasourcesorliteraturehasnotbeencarriedout.
KIswerepurposivelyselectedtocovergovernmentstaffinmanagementpositionsintheeducationministries,departmentsandagencies(MDAs)thatconstituteGEP3’spartnership,aswellrepresentativesofgovernmentornon-governmentagenciesinvolvedinGEP3implementation.Furthermore,somenon-governmentinformantsthatcouldprovidefurtherinsightsaboutspecificinterventionswereselected.4KIIswerespreadacrossGEP3interventions,puttinganemphasisondifferentinterventionsindifferentGEP3states.AnnexBprovidesanoverviewofthefocusthatwasplacedontheinterventionsineachstate.ToidentifythepotentialKIs,alonglistofKIswasdraftedincollaborationwithGEP3stateteams,whichweresubsequentlydiscussedwitheachoftheStateProjectCoordinatorsduringacall.BasedonthisdiscussionandKIavailability,afinallistwasselectedbytheevaluationteam.
Fieldworktookplacefrom10Augustuntil27August2015,withtworesearchersspendingapproximatelytwodaysineachoftheGEP3statecapitals(Katsina,Gusau,Sokoto,Minnaand
2Theevaluationframeworkidentifiedanumberguidingsub-questions.SeeAnnexA.3ThediagramispartofUNICEF’s(2015a)ToCdocumentfortheproject.4Non-governmentinformantswereselectedbasedontherichandconcreteknowledgeitwasassumedtheycouldprovideaboutaspecificintervention.Theywereselectedbytheresearchteamincoordinationwith,andsometimesattheproposalof,GEP3stateteamsandDFID.
EvaluationoftheGirl’sEducationProject3
©EDOREN 4
Bauchi).Theresearchteamconducted,onaverage,11face-to-faceKIIsperstate.KIIswithnational-levelexpertswereconductedinAbujaorbyphone.ThelistofpeopleinterviewedaspartoftheKIIsispresentedinAnnexB.Theresearchteamconductedatwo-dayparticipatorymethodsreviewbeforethestartoftheinterviewperiod.TheteammetwithDFIDandUNICEFforadebriefaftertheinterviewperiod(inAbujaon28August).
Interviewsfollowedsemi-structuredinterviewguidesthatwereadaptedtotheinterviewedstakeholderandtheinterventionstobecoveredineachstate.Whiletheinterviewguidesensuredthatpre-setbasiclinesofinquirywerepursued,thesemi-structurednatureoftheinterviewsallowedforparticularsubjectstobefurtherexploredandexpanded,inlinewiththeexploratorynatureofthebaselineToCassessment.
Followingthemethodologypresentedintheevaluationframework,themainlinesofinquiryconcerned:
• theeducationcontextofthestate;• thestakeholders’understandingofprojectand/orinterventionobjectivesandpathwaysto
change;• factorsinfluencingtheoutcomesoftheprojectand/oritsspecificinterventions;• thestakeholders’understandingofimplementationstrategies,andinputsandprocessesusedas
partofthisimplementation.
Interviewswereaudiorecordedafterreceivingconsentfromtheintervieweesandinterviewnotesweretranscribedwiththesupportoftheaudiorecordings.DatawereorganisedandanalysedbytheresearchteamusingNvivoqualitativedatamanagementsoftware.
Themethodologyhadcertainlimitations:
• Becauseoftheexploratorynatureoftheinquiry,aprioritywasgiventobreadthoverdepth,byinterviewingalargenumberofstakeholdersinordertoallowtheinputofmultipleperspectives.Furthermore,stakeholdermeetingswerealsousedtodiscusstheevaluationplans.Thishaslimitedin-depthexaminationofspecificinterventionsandcausalpathways.
• DuetotheGEP3redesignandthe2015elections,someKIswerenewtoGEP3’sredesignedinterventionsandtheirimplementationinspecificstates.5Furthermore,someinterventions,suchastheearlylearninginterventionandGirlsforGirls(G4G)groups,hadyettobeoperationalised.Thisalsolimitedthedepthofthedatacollection.
• Duetotimeconstraints,theinterviewstargetedstate-levelornational-levelstakeholders.Therefore,itwasnotpossibletocompleteandtriangulatetheinformationobtainedwithstakeholdersatlocalgovernmentlevel,althoughtheseindividualsareimportantstakeholdersintheimplementationoftheGEP3interventions.
2.3 PlausibilityofGEP3’sToC
ThischapterexaminestheplausibilityofGEP3’sToCbyreviewingstakeholders’commonunderstandingofitsintendedoutcomes,byinterrogatingthelogicofitsoutcomechain,andby
5ThreeoutoffiveoftheGEP3statecoordinatorshadrecentlystartedworkingintheirspecificstate,althoughallofthemhadexperienceofworkingaspartofGEP3inotherstates.TheDirectorofPlanning,ResearchandStatisticsinKatsinahadonlybeeninhispositionfortwomonths.HewasassistedintheinterviewbytheGEP3deskofficer.TheKIfortheCollegeofEducationinZamfarahadonlybeentheGEP3focalpersonforonemonth.
EvaluationoftheGirl’sEducationProject3
©EDOREN 5
makingmoreexplicitfactorsthatarenecessaryfor,ormayhinder,theachievementoftheoutcomes.Thechapterfirstprovidesabird’s-eyeassessmentofGEP3’sultimateoutcomes,itslogicandmainproposedinfluencingfactors.Next,adetaileddiscussionofdifferentinterventionsperGEP3outputispresented.ThechapterendswithanassessmentoftheplausibilityofGEP3’spilot-to-scaleapproach.
2.3.1 Generalassessment
ThekeyprojectstakeholdersaregenerallywellawareoftheenvisionedultimateoutcomeofGEP3,intermsofimprovinggirls’education.Higherlevelgovernmentpartners6,UNICEFfieldofficersandGEP3stateteamswereaskedaboutwhatthemainresultsarethatGEP3intendstoachievebytheendoftheproject.Increasedgirls’schoolenrolmentwaswellrecognisedasamainintendedoutcome,andwascommonlylinkedtothetargetofgetting1millionadditionalgirlsintoschool.TheKIsrecognisethatGEP3’sultimatesuccessisalsodependentongirls’retentionandimproved
learning.Theseoutcomeswerementionedlessfrequently,particularlyamonggovernmentKIs,whichsuggeststhatGEP3isforemostassociatedwithimprovinggirls’accesstoschool,representedbyitstargetofgetting1millionadditionalgirlsintoschool.
TheUNICEFKIsinthefieldofficesexpectthatincreasedgirls’enrolmentwillbeeasiertoachieve
comparedtoretentionorlearningbecausethelatterrelyoneducationsupply-sidefactors,suchassupplyofqualifiedteachers,whichisrecognisedasakeychallenge(alsobygovernmentKIs,seebelow).Consequently,teachercapacitydevelopmentiswidelysupportedasakeyareaofintervention,notonlybecauseoftherecognitionofcurrentteachercapacitybeinganimportantchallengebutalsobecauseteachingqualityisconsideredcentraltoimprovinglearningoutcomes.Furthermore,communityownershipthroughtheworkoftheSchool-BasedManagementCommittees(SBMCs)iscommonlyheraldedasthesolutiontodemand-sideandsupply-sideconstraintsandtogovernanceproblems.GEP3’semphasis,sincetheredesign,onteachingquality
andcommunityempowermentthroughSBMCsisthereforewellalignedwithstakeholders’
prioritiesandtheirunderstandingofhowthequalityofeducationcanbeimproved.Aswillbediscussedinthenextsections,theeffectivenessofthisstrategyisdependentonawiderangeofassumptions.
Whilegender-sensitiveteachingpractices,theinvolvementofMothers’Associations(MAs)andinvestmentinthegirl-friendlinessofschoolenvironmentsarerecognisedbyKIsaswaystoimprovethequalityofeducation,particularlyforgirls,theKIIsprovidelittleevidenceabouthowthiswillbeoperationalised,whichintheendwilldeterminehowplausibleitisthatthisoutcomewilloccur.
WhileKIshavearelativelygoodknowledgeofGEP3’soverallstrategy,andappreciatethefactthattheredesignhasimprovedtheproject’sfocus,itsoperationaldesignislesscommonlyunderstood.Attributesofintervention-specificoutputsandoutcomes—suchastheprioritytargetgroups,thetiming,andthenatureoftheoutcome—areoftennotcleartoKIs.Thismaybepartlyduetotheredesignandthefactthatsomeinterventionsstillneedtobeoperationallydefinedandcommunicated.Whenoutputandoutcomeattributesarenotclear,performancewillbedifficulttomeasureandassess.
TheintermediateoutcomesofGEP3’sdifferentinterventionsarehighlyinterlinkedandsynergetic,
whichsupportstheassumptioninGEP3’sToCthatcombinedinterventionswillsupportthemost
6Thequestionwasaskedatdirectorandhigherlevels(permanentsecretaryorsecretary)intheStateMinistriesofEducation(SMoEs)andStateUniversalBasicEducationBoards(SUBEBs).
EvaluationoftheGirl’sEducationProject3
©EDOREN 6
vulnerablegirlsmoreeffectively.Forexample,thecashtransferprogrammebuildsonthesensitisationconductedbytheenrolmentdrivesandthecapacityoftheSBMCstosupportthecashtransferenrolmentandmonitoring.Theoutcomesofteachercapacitybuildingdependonthesupportivesupervisionbytheheadteacherandtheschoolenvironmentbeingimprovedviamini-grantsandcommunityinvolvement.Thesesynergiespotentiallyreinforcetheresultsoftheinterventions,althoughtheyalsoconstituteariskofinterventionproblemsspreadingacrossthe
programme.Inthisregard,theSBMCempowermentinterventioniskeysincemostofGEP3’sschool-levelinterventionsassumefunctioningSBMCs.ThefactthatsupporttoSBMCswasmaintainedinallGEP3focusschoolsaspartoftheprojectredesignisthereforeimportant.7AsrecognisedintheGEP3ToCdocument,theToCalsoincludesapotentially‘negative’relationshipbetweenincreasedaccessanddeterioratingteachingandlearningquality.Totheextentthatsupply-sideinvestmentdoes
notkeepupwithincreasedenrolment,teachingandlearningqualitymaybenegativelyaffected.KIsindicatethatthisriskismaterialising(seebelow).Finally,itisworthnotingthatthecompletepackageofinterventionsisnotimplementedinallGEP3states.Therefore,combinedinterventionscanstrengthenresultsinthosecontextsforwhichthedifferentinterventionsareplanned.
WithregardstoGEP3’ssupporttoIQSs,KIsaresupportiveofintegrationandGEP3’scontributiontothis.TheKIsworkingwithIQSsexpectGEP3tocontributetogirls’enrolment,retentionandmainstreamingintheformaleducationsystem.ImprovedlearningoutcomesarementionedbysomeKIs.ItwillbeusefultoclarifytowhatextentmainstreamingisanimpactthatGEP3considersas
partofitsscope.IfmainstreamingisconsideredtofallwithinthescopeofGEP3,thiswouldrequirepayingattentiontobarrierstomainstreaming.Povertyismentionedasonesuchbarrier.
‘Therearemanyparentsthathavetheinterestathearttosendchildrentoschool,buttheymaynotbeabletocaterforsendingtheirchildrentosecondaryschool.PovertyisoneofthefactorsthatmotivatesparentstosendtheirchildrentoQur’anicschoolbecauseitisfree.TheseparentsmaystillbetoopoortofacilitatemainstreamingoncethechildrenhavepassedthroughBasicorPost-Basic.’(GovernmentKI,Bauchi)
Totheextentthatpubliceducation,inparticularsecondaryeducation,isnotfullyfreeofcharge,povertymayinhibitatransitionfromIQSstothepubliceducationsystem8.SomeKIsclaimtherearenoinstitutionalbarrierstomainstreaming,andsuggestthatIQSpupilsareabletotransitiontoaformalschoolwithoutadministrativehindrance.Nonetheless,itrequiresfurtherinvestigationtodeterminehoweasyitisforpupilstodosoinpractice.
ThefactorsthatinfluenceGEP3’sToCarediscussedinmoredetailinthenextsections.Inthissection,wehighlightthemainfactors:
• KIsconfirmthefindingsofEDOREN’spoliticaleconomyanalysis(PEA)thatacentralfactoraffectingthesuccessoftheprojectisstategovernmentsfulfillingtheircommitmentintermsof
financialcontributionstotheproject,especiallythereleaseoffundsbythestateexecutivegovernor.WhiletheGEP3ToCacknowledgesthisandhasincludedinterventionstomitigatethisriskfactor,theKIsindicatethisremainsanimportantconcern.AreviewbytheStatePartnershipforAccountability,ResponsivenessandCapability(SPARC)andtheStateAccountabilityandVoiceInitiativeSAVI(2015)confirmsthatthepublicfinancialmanagementsituationintwooutofthree
7AccordingtotheGEP3workplanrefresherSBMCtrainingisplannedinallfiveGEP3states.8WhilepublicbasiceducationisofficiallyfreeinNigeria,parent–teacherassociationlevies,thecostofschooluniforms,writingmaterialsandotheradhoccostsaremajorcausesofnon-enrolment,absenteeismanddropout(HumphreysandCrawfurd,2014)
EvaluationoftheGirl’sEducationProject3
©EDOREN 7
GEP3statesexaminedisstillpoor.9Furthermore,someoftherecommendationsofEDOREN’sPEA(2014)remainunaddressed.StategovernmentexpenditureonGEP3interventionscanstillnotbesystematicallyaccessed,anditisnecessarytorelyonadhocrequeststogovernmentcounterpartstoprovideinformationonexpendituresonGEP3activities.Thelevelofresourceprovisionisnotexplicitlylinkedtotheachievementofspecifictargets.However,positiveelementsarethatthenewlyelectedgovernorsseemtoprioritiseeducationintheirdiscourseandhaveundertakensomestepstosupporteducationpolicy-making.GEP3’sfocusonanarrowersetofinterventionsfollowingitsredesignhasallowedformoreeffectivemanagementandfollow-upofstatecommitments.Furthermore,UNICEF,togetherwithstatepartners,hasconductedhigh-leveladvocacyinrelationtothestategovernors,whichismentionedbyKIsasimportantinordertoimprovethelikelihoodoffundingrelease.
• KIsagreethatteachingqualityiscentraltoimprovinglearningoutcomes.Therefore,teacher-relatedsupportingfactors,suchasnumberofteachers,deploymentandtransfersofteachers,andmotivationofteachersareconsideredkeytothesuccessoftheprogramme.Monitoringofteachersishighlightedasbeingespeciallyimportant.
• Asmentionedabove,theroleofthecommunityinthedemand,supplyandgovernancesideoftheschoolsystemisanotherfactoremphasisedbyalargenumberofKIs.ThecommunityisexpectedtodirectlycontributetoschoolinputsandtotheschoolenvironmentinlinewiththesloganoftheUniversalBasicEducationCommission(UBEC)‘Educationforallistheresponsibilityofall’.ThroughtheSBMCs,thecommunityisalsoassumedtotakeownershipofschooldevelopmentandoversight.Hence,theKIsconfirmthekeyassumptioninGEP3’sToCthattheeffectivenessoftheSBMC/Centre-BasedManagementCommittees(CBMCs)hasanimpactonGEP3’soutcomes.Aswillbediscussedbelow,thispresentstwomainrisks.First,thehighexpectationswithregardtoSBMCsmaynotmatchtheircapacity,asthecapacitybuildingprocessisexpectedtobegradualandSBMCsoftenstartfromalowinitialcapacitylevel.Second,communityownershipandresourcemobilisationmaybepromotedasasubstitutefor,ratherthanacomplementto,astategovernmentthatisaccountableforprovidingandresourcingaqualityeducationsystem.
• EffectivemonitoringishighlightedbyseveralKIsaskeytothesuccessoftheproject.ThiscorrespondstobothmonitoringandoversightofschoolactivitiesbytheSBMC,headteacherorlocal/stategovernmentofficers,aswellasmonitoringofGEP3projectimplementation.SomeKIslinkthistoaneedforinterventionstohaveastrongerresultorientation:holdingpeopletoaccountforresults.KIsacknowledgethatthemonitoringsystemsareweak,althoughinvestmentshavebeenmadeinthepasttostrengthenmonitoringandqualitycontrol.10Therefore,GEP3rightlyemphasisesmonitoringinitsToC.However,effectivemonitoringrequiresacommonunderstandingoftheobjectives,rolesandresponsibilities,andapproachofthemonitoring,andsufficientresourcesforimplementation.
• InthecaseofIQSs,additionalkeyinfluencingfactorsapply.KIsworkinginthefieldofintegratedQur’aniceducationarguethatthebuy-inoftheproprietorforintegrationandprojectinterventionisvitalinorderforinterventionstoleadtoresults.However,thebaselinefindingsoftheIQSsupport(IQSS)evaluationindicatethatschoolleadershipinIQSsiscomplex.ThemanagementoftheIQSisnotanindividualresponsibilitywithclearlydefinedanddelegatedresponsibilities.Therefore,itwillbeimportanttotakeintoaccountthevoicesofkeycommunity
9SPARCandSAVI(2015),‘StateoftheStates–PoliticalEconomy,ProspectsforReformandFiscalConditions’,unpublishedPowerPointpresentation,13July2015.10InSokoto,schoolqualityassuranceofficershavebeenscreenedandtrainedwithsupportoftheNorthernEducationInitiativeproject.InZamfara,accordingtoaKI,theSUBEBrecentlysuppliedqualityassuranceofficerswithnewmotorcyclesandfuelallowances.InBauchi,theSUBEBexpandeditszonalofficesfromthreeto10toensuretherearemorequalityassuranceunitsclosertotheschools.
EvaluationoftheGirl’sEducationProject3
©EDOREN 8
figures,giventheinfluencetheymayhaveonthesuccessofintegration.Furthermore,theselectionofIQSsthatactuallyandcontinuouslyofferformaleducationtogirlsisaprerequisiteinordertoimprovegirls’education.WhilethisisforeseenintheeligibilitycriteriausedbyGEP3,11thebaselinedatacollectionindicatesthatinpracticenotallGEP3IQSsareintegratedorhavegirlsenrolled.ThequalitativeresearchalsohighlightsthatsomeIQSsmaystillbemobile(whichneedsconfirmationatmidline).GiventhatgirlsareunlikelytofollowtheMallamwhentheschoolmoves,theireducationwillpotentiallybeinterrupted.12
Furthermore,theprojectoutcomesmaybeinfluencedbyawidevarietyofcontextual,non-project-relatedfactorsspecifictoeachstate.Thepresenceofotherprojectscontributingtothesameoutcomesisonesuchfactor.Theseprojectsvaryfromstatetostate.Forexample,theNigerianPartnershipforEducationProject(NIPEP)aimstosupportsimilarinterventionsasGEP3inKatsinaandSokoto.Thiscancreatesynergiesbetweenprojects,butmayalsoconflateoutcomes.Similarly,theTeacherDevelopmentProgramme(TDP)issupportingin-serviceteachertraininginKatsina,ZamfaraandNiger.13TotheextentthatTDPprovidessupporttoteachersfromGEP3focusschoolssynergiescanbecreated,whileatthesametimeitwillbedifficulttoattributechangesineducationoutcomestoGEP3alone.
GovernmentpoliciesandstructuresmayalsoleadtovariationinGEP3outcomesfromstatetostate.Forexample,NigerStatehashadapolicyofimplementingWardDevelopmentProjects,todecentraliseinvestmentinbasicsocialservices–including,amongotherthings,ineducation.14Intermsofgovernmentstructures,statesmayhavespecificMDAstosupportgirls’educationortheintegrationofbasiceducationintheQur’anicschools.Forexample,theZamfaragovernmenthasestablishedtheFemaleEducationBoard,whichismanagingboardingschoolstargetedatgirls.ThepreviousgovernmentinBauchicreatedtheDirectorateofTsangayaEducation,whichprovidedallowancestoproprietors.SuchinstitutionaldifferencesmayinfluenceGEP3’sresultsindifferentways.SuchinstitutionsmaycomplementGEP3interventionandsupportGEP3achievingitsresults.Thepaymentofproprietorallowancesisonesuchexample.Ontheotherhand,totheextentthattheseinstitutionscompeteforthesamestateresourcestheymayinfluencethecapacityofGEP3’sdirectgovernmentpartnerstoimplementinterventions.ThenationalandstateelectionsthatwereheldinMarchandApril2015respectivelymayaltergovernmentpoliciesandstructures.Atthetimeoftheinterviews,itwasyettobeseenwhichGEP3stateswouldchangetheireducationpolicy,structuresorbudgetprioritisationasaresultoftheelections.
Finally,GEP3’soutcomeswillbeaffectedbyabroadersetofsocioeconomic,demographic,cultural
andpoliticalfactors.InsecurityandconflictwerementionedinBauchi,ZamfaraandKatsinaashavingaffectedschooling.InBauchi,thishasresultedinthestateeducationsystemhavingtocopewithaninflowofinternallydisplacedpeople.InZamfaraandKatsina,truancyamongbothpupilsandfemaleteachersoccursduetofearsregardingsecurity,especiallyintheoutskirtsofthetwostates.KIsinNigerpointedoutthat,giventhesizeofNigerState,15improvingschoolinfrastructureandaccessibilityisproportionallymoredifficultcomparedtosmallerstates.
11IQSsareeligibleforGEP3supportduringthepilotphasewhen40%ofpupilsaregirls.12ThetermMallamcanhavedifferentinterpretations.WeconsidertheMallambeingtheheadofthereligiousschool,responsibleforthereligiouseducation.13Initsfirstphase,from2014,TDPoperatesinJigawa,ZamfaraandKatsina.ThereareplanstoextendittoKano,KadunaandNigerstatesinlate2016.14Thepolicyawards100,000NigerianNaira(NGN)permonthforeducationdevelopmentatwardlevel.15NigerStateisthelargeststateinNigeriaintermsofsurfacearea.Onaverage,Nigerhasfourprimaryschoolsper100km2,comparedtonineper100km2inKatsina.
EvaluationoftheGirl’sEducationProject3
©EDOREN 9
2.3.2 Output1:Increasedenrolmentandretentionofgirlsinbasiceducation
UnderOutput1GEP3seekstoincreasegirls’enrolmentandretentioninbasiceducationbyaddressingbarrierswhichaffectdemand.Threeinterventionsthatcontributetothisresultaresupported(seedetailsbelow):
• enrolmentdrives:annualcampaignstoidentifyout-of-schoolgirlsandtoencourageschoolenrolmentandretention;
• acashtransferprogramme:unconditionalcashtransfersprovidedtofemalecaregiversofallgirlsbetweentheagesofsixand15livinginselectedprimaryschoolcatchmentcommunities;
• G4Ggroups:girls’groupsestablishedinprimaryschoolsandIQSsundertheguidanceofamatronselectedfromtheschool’sMAorfromamongthefemaleteachers.
Inaddition,theprojectintendstoinfluencesupplyinordertomeetincreaseddemand,withthesupportofhigh-levelpoliticalengagement.
Whilethisoutputisformulatedtocoverbasiceducation,theemphasisoftheinterventionshasshiftedtoprimaryschooleducationaftertheGEP3redesign.Enrolmentdrivesdocoverthewholerangeofbasiceducation,includingearlychildcaredevelopmentandeducation(ECCDE)andjuniorsecondaryschool(JSS)education,butthecashtransferprogramme—eventhoughitcoversgirlsbeyondprimaryschoolage—isorganisedaroundaselectionofprimaryschools.TheG4Ggroupswillsimilarlybeoperationalisedinprimaryschools.Hence,resultsaremorelikelytobeachievedin
primaryeducationratherthanacrosstheentirebasiceducationcycle.
2.3.2.1 Enrolmentdrives
Enrolmentdrivecampaignsareannualeventsthathavebroadacceptanceandsupportamongstakeholders.TheyaremeanttobeledbySBMCsandMAs,withthesupportofcommunityleadersandreligiousleaders.AllKIsperceivethesedrivesasaneffectivemechanismforincreasing
enrolment,byraisingparentalawarenessabouteducation.Thisappreciationfollowsfromtheirmultidimensionalnatureintermsofinvolvingawiderangeofstakeholdersatdifferentlevels(state,localgovernmentandcommunity),usingmultiplecommunicationstrategies,andmultiplemessagesthatcanspeaktoadiverseaudience.Whilethecampaignscoverbasiceducationingeneral,theinclusionofgirlsandotherdisadvantagedgroupsisemphasised.16Thefactthattheenrolmentdrivesareabletopenetratethecommunities,withtheactiveinvolvementofthecommunitiesthemselves,ishighlightedasakeyfeatureinregardtotheireffectiveness.
Theinterventionisbasedontheassumptionthatdemandforeducationistheresultofpeople’sknowledgeaboutthebenefitsofeducation,andtheextenttowhichtheyhavetherightattitudesandinterestswithregardstoeducation.Theexpectationisthattheprovisionofinformationwillinfluencetheirawareness,attitudesandbehaviour.InthediversecontextoftheGEP3states,thelevelofawarenessof,andattitudestowards,girls’educationvaries.SeveralKIspointoutthattheculturalacceptanceof‘Western’educationandtheappreciationofthevalueofeducationhaveimprovedconsiderably.Theseattitudesvaryregionallyandbetweenruralandurbanareas.17Therefore,asawarenessandattitudeschange,enrolmentdriveswillneedtobecomemore
16 KIs pointed out that last year’s enrolment drive emphasised the enrolment of children from ethnic minorities andphysicallydisadvantagedchildren.17 In Bauchi, KIs highlighted the difference between the Northern, Central and Southern region. Acceptance of, and aninterestin,formaleducationishigherintheSouthernregion,whichtraditionallyhasbeeninfluencedbyChristianmissions,andtheCentralregional,whichisinfluencedbytheBauchiurbancentre.
EvaluationoftheGirl’sEducationProject3
©EDOREN 10
contextualised–targetingtherightmessagetotherightpeople.SomeKIsindicatedthatthisisalreadytakingplace,intheformofintensifyingcampaignsinareasoflowenrolment.
Changesinawarenessandattitudesdonotnecessarilyresultinchangesinbehaviour.Inparticular,povertyorotherconstraintsmayinhibitactualchangesinbehaviour.GEP3acknowledgesthisinitsdesignbypilotingacashtransferprogrammetotacklethefinancialbarrierposedbypoverty.Asawarenessandattitudeschange,thefinancialbarriermaybecomerelativelymoreimportantand
theneedtoexpandaninterventiontargetingthisbarriermayincrease.However,weassessthattheimplementationofacashtransferprogrammeacrossGEP3statesatscaleisunlikely(seebelow).
KIsacknowledgethattheobjectiveoftheenrolmentdrivesisnotonlytoincreaseenrolmentbutalsotoretaingirlsinschool,whichisconsideredtobethemoredifficultobjectivetoachieve.Retentionwillbeinfluenced,amongotherthings,byparents’andgirls’perceptionsregardingthequalityofeducation,whichwilldependonschoolsbeingabletomeettheincreaseddemandwithadequate
facilitiesandteachers.TheGEP3ToChasidentifiedasakeyassumptionthatgovernmentcansupplyprimaryschoolandteacherstomeetincreasingdemand.InallstatesKIsperceivedthatincreasedenrolmentisnotsufficientlyaccommodatedintermsofkeyresourcessuchasteachersandclassrooms,althoughsomeseethisasaproblemthatismainlymaterialisinginurbanareas.
‘However,aweaknessofenrolmentdriveisthattheyoverenrol.ThisistheoneareawheresomeSBMCmemberscomplain.Therearemorechildrenenrolledthanthefacilities,infrastructures,materialsorteachersprovided.Insomeschoolsonlyoneteacher,theheadteacherorArabicteacher.Theremaybeover200pupilswithjusttwoclasses;theinfrastructureisjustnotthere.Parentsarecomplaining:youasksustosendchildrentoschoolbutwherearetheteachers,wherearetheclasses?’(Non-governmentKI,Zamfara)
‘Ineducationallyadvancedcommunitiestheschoolsbecomeoverstretched.(…)InBauchinowtheclassesareoverstretched;thereisaclassthathas150pupils,180even200pupilsinoneclassbothintheprimaryandjuniorsecondary.InsideBauchi,Toro,Azare,andothersthatareintheurbanareasmostoftheschoolsareoverstretched.’(GovernmentKI,Bauchi)
Thismaycreateabacklashinregardtoparents’andgirls’responses,aswellasinregardtoSBMCs’effortsinrelationtoenrolmentdrives.Themostseriouschallengeliesinsufficientteachersupplyinruralareasandinfrastructuresupplyinurbanareas.Furthermore,astheenrolmentdrivesexpandtheirreachonthedemandside—forexample,bytargetingECCDEandtheenrolmentofdisabledchildren—theriskofthesupplysidenotmeetinghigherdemandincreases.
Besidestheresponseonthesupplyside,KIshighlightedthefollowingotherkeyfactorsthatwillinfluencewhetherenrolmentdrivesachievetheirintendedresults:
• TheroleoftheSBMCsandMAsareconsideredcentraltothesuccessoftheenrolmentdrives.Theirroleisnotonlyseenintermsofchannellingthecampaignmessagesdirectlyintothecommunities,butalsointermsoffacilitatingsupply-sideresponsestotheincreaseddemandbymobilisingcommunityresources,signallingsupply-sidebottleneckstolocalorstategovernmenteducationofficialsand/orpotentiallyholdinggovernmenttoaccounttoresolvesuchbottlenecks.Aswillbediscussedbelow,themultiplerolesthatSBMCsareexpectedtotakeonmaynotbeinlinewiththeiremergingandgradualcapacitybuildingandempowermentprocess,particularlyinruralareas.Hence,theirfunctionalityandcapacityisakeyassumptionforsuccessfulenrolmentdrives.GEP3hasincludedSBMCcapacitybuildingaspartofitsdesignandToC,whichstrengthenstheplausibilityofthisassumptionbeingmet.Nonetheless,the
EvaluationoftheGirl’sEducationProject3
©EDOREN 11
assumptionwillrequireclosemonitoring,whichisforeseeninGEP3’sdesign,viatheSBMCeffectivenessmonitoring.
• KIsconfirmthepositivecontributionthattraditionalleadersandfemalerolemodelsmakeinregardtogirls’enrolment.Traditionalleadersareconsideredcredibleopinionleaderswhocanaddresssocio-culturalnorms,whilefemalerolemodelsareassumedtoemanatethebenefitsofgirls’education.KIsvalueGEP3-supportedactionsthatstrengthenthisfactor,suchasincorporatingtraditionalleadersinzonalmeetings,thecontributionoftheHigh-LevelWomenAdvocates(HiLWA)groupandtheexpansionoffemaleSBMC/MAmembersintheorientationmeetingoftheenrolmentdrives.
Intermsofcontextual,non-GEP3-relatedfactorsthatwillinfluencetheimpactoftheenrolmentdrives,thesecuritysituationisariskfactor.KIsindicatethatinBauchi,theBokoHaraminsurgencyloweredschoolenrolmentin2014insomeareaswhereattacksoccurred.Enrolmentisalsoaffectedbytheimmigrationofinternallydisplacedpersonsfromneighbouringstates.ThesecuritysituationwasalsoidentifiedasaninfluencingfactorinZamfaraandKatsina.Finally,itisimportanttorecognisethathighfertilityinnorthernNigeriawillincreasetheschoolagepopulationandsubsequentlymaycontributetoincreasedenrolment.
2.3.2.2 Cashtransferprogramme
TheunconditionalcashtransferprogrammethatisbeingpilotedinNigerandSokotoaimstoreducethepoverty-relatedbarrierthatispreventinggirls’enrolmentandretention.KIshaveacommon
understandingofthisobjective:theygenerallyperceivetheprogrammeasaneffectivestrategy
andindicatethathighinterestintheprogrammeinthetargetedcommunitiescanbecountedondespiteinitialmistrustamongsomeparentsabouttheintervention’sintentions,whichresultedinsomeeligiblegirlsnotbeingregistered.TheSokotoandNigerPIUsarguethatthehighcommunityinterestisalsodemonstratedbythedemandfortheprogrammethatisnowcomingfromnon-eligiblecommunities.
ThePIUsarewellinformedaboutthekeyfeaturesofthecashtransferprogramme,suchasthegeographicalcategoricaltargetingofthebeneficiariesanditsunconditionality.Theyareespeciallysupportiveoftheunconditionalnatureofthecashtransferbecauseconditionalitycouldunderminetheincome-smootheningeffectbydenyingtransferstothosewhofailtosatisfyconditions,suchasschoolattendance–potentiallyinthosetimeswhenthetransferisneededmost(forexample,whenthechildfallsill).Theeffectivenessoftheunconditionalnatureofthecashtransferis,however,contestedbyotherKIs,whofavoursomeformofconditionalitybasedonthegirlscomingtoschool,ontheassumptionthatthiswillincentiviseattendanceasfamilieswouldlosethecashtransferwhenacertainattendancethresholdisnotmet.WhiletheKIsconfirmthatthecashtransferfocusesonalleviatingthepovertybarriertogirls’education,theyalsoacknowledgetheimportanceof
sensitisationandtheroleoftheSBMCsandheadteachersinfollowinguponattendance.
‘Itisthesensitisationthatmatters,sothatpeopleunderstandthateventhoughthereisnoconditiontheysendtheirchildrentoschool(…).TheinvolvementofSBMCandtraditionalleadersinthecashtransferprogrammehelpedalotinthesensitisationofthecommunities.’(Non-governmentKI,Sokoto)
EvaluationoftheGirl’sEducationProject3
©EDOREN 12
Furthermore,thePIUsclaimthatthenon-paymentofseveraltransfersduringthefirstyearofimplementation18didnotsubstantiallyaffectschoolparticipation,which,ifthisholdstrue,indicatesthatfactorsotherthanincomemaybeinfluencingtheparticipationdecisionandmay,therefore,influencetheachievementofoutcomes.However,nodatawereprovidedtosupportthisclaimandUNICEFhasindicatedthatthePIUshavenotdocumentedenrolmentandschoolattendanceofcashtransferbeneficiaries.
KIsrecognisethesynergybetweenthecashtransferprogrammeandtheenrolmentdrives.Thelatterreinforcesthesensitisationaroundthecashtransferprogramme.InSokoto,thecashtransferprogrammewaslaunchedintheweekoftheenrolmentdrive.Inaddition,theon-the-spotschoolenrolmentatthetimeofthecashtransferpaymentisalsomentionedasaneffectivefeaturecontributingtogirls’enrolment.
TheKIshighlightedthefollowingkeyfactorsthatinfluencetheoutcomesofthecashtransferprogramme:
• Similartoenrolmentdrives,thecommitmentandcapacityofSBMCmemberstosupporteffectivesensitisationandattendancemonitoringarehighlightedasimportantcontributionstothesuccessofthecashtransferprogramme.Inaddition,SBMCsareconsideredtoprovidesupply-sideinputstoincreasedenrolmentduetothecashtransferprogramme.Inturn,thecashtransferprogrammemayinfluencetheroleoftheSBMCsbystrengtheningtheirfunctionintheschoolandcommunity(withanassociatedriskofunderminingitiftheycannottransparentlyandadequatelyimplementthatrole).19
• Again,similartoenrolmentdrives,supply-sidefactorsliketheavailabilityandattendanceofteachersareconsideredkeytomotivatingparentstosendtheirchildrentoschool.
• Communitycommunicationandsensitisationregardingtheeducationalobjectiveofthecashtransferisalsoseenasessential,giventheunconditionaldesign.20Thismaychangehowparentsperceiveeducation.However,thisislikelytobeaffectedbyparents’perceptionsofthequalityofsupply-sidefactorsandthecredibilityofthepeopleinvolvedinthesensitisation.ThelatterislinkedagaintotheinvolvementofSBMCsandtraditionalleaders.
Twointerrelated,centralinfluencingfactorsthatGEP3haslessinfluenceonarethebroaderlegalandpolicyframeworkwithregardstosocialprotectioninGEP3states,aswellaspoliticalleadershipchangesandsubsequentpolicyprioritychanges.Currently,nosocialprotectionlegalframeworkisinplaceinGEP3statesthatcanprovidecontinuityandstabilitytoacashtransferprogramme.Thismeansthatthepolicypurposeofacashtransferprogrammemayshiftbecauseofchangesinstate-levelleadershiporfundingprioritiesatfederalanddonorlevel.AccordingtoKIs,thediscontinuationoftheeducationcashtransferpilotsinBauchiandKatsinawasinfluencedbychangesinpoliticalleadershipandare-prioritisationofthepurposeofcashtransfers:topovertyalleviationratherthaneducation.InNigerandSokoto,thefirstyearofoperationsoftheGEP3cashtransferpilothasbeenaffectedbytheelectionsandthetransitioninpoliticalleadership.Asrecognisedby18 At the time of the interview, only one payment out of four was transferred in Sokoto, while two payments wereoutstandinginNiger.19TherolesandresponsibilitiesoftheSBMCintheGEP3cashtransferprogrammeinclude:1)conductingmobilisationandsensitisationactivities;2)assistinghouseholdswithappeals,complaintsandchangemanagementprocedures;3)communicatingdetailsofthepaymentdaytobeneficiaryhouseholds;4)beingpresentonthepaymentdaystoassistcommunication,identityverificationandsubmissionofappeals,complaintsandchangemanagementrequests;and5)ifinstructed,conductinghomevisitsofgirlsthatarenotenrolledornotattendingregularly(GEP3-CTPOperationalManual).20Benhassineetal.(2015)foundevidence,basedonarandomisedcontroltrialinMorocco,thata‘labelled’cashtransfer,whichmakestheeducationobjectiveofthecashtransferclear,isenoughtoimproveenrolment,withnoneedforexplicitconditions.
EvaluationoftheGirl’sEducationProject3
©EDOREN 13
theGEP3ToCandconfirmedbytheKIs,statefundingsupportforcashtransfersdependsonthepolicyprioritiesofthenewgovernors,thefederalgovernmentmechanismanddonorpriorities.
2.3.2.3 G4G
AccordingtotheGEP3ToC,G4Gsseektoempowerschoolgirlsandtoaddresstheirretentionbyprovidingthemwithaspaceinwhichtomeetandaddressissueswhichaffecttheirschoolparticipation.Thisisdoneundertheguidanceoffemaleteachers/matronswhocanactasrolemodels.TheG4GsaretobeestablishedundertheumbrellaoftheSBMC/CBMC,toensuresustainability,undertheassumptionthatSBMCs/CBMCstakeownershipofthegroupsandpromotetheirimplementationbeyondtheterminationofexternalsupport.
ItisplannedthattheG4Ginterventionwillbeimplementedin2016–2017,andsoitisyettobeoperationalised.Itwouldstrengthentheplausibilityoftheintervention’sToCtoclearlyspecifyandagreeontheintervention’sobjectives,itstargetpopulationandtheconceptualisationofgirls’
empowerment.Withregardtotheobjectives,atthetimeoftheKIIsitwasnotclearwhethertheinterventionaimstoachievetheretentionofin-schoolgirlsor/andthe(re)enrolmentofout-of-schoolgirls.TheGEP3ToCdocument(UNICEF2015a)presentsG4Gasaninterventionthatseekstoaddressgirls’retention,whiletheGEP3SBMCstrategypaper(UNICEF,2015g)introducesitspurposeasbeingtocreateaplatformfortheempowermentofgirlstoexercisetheirrighttoenrolandremaininschool.21Closelyrelatedisthespecificationofthetargetpopulation.Itneedstobeclearwhetherout-of-schoolgirlsareadirecttargetpopulationofG4G(andhenceinvitedtobemembersoftheG4G),anindirecttargetpopulation(reachedthroughin-schoolG4Gmembers),ornotatargetgroupatall.TheGEP3ToCdocumentandGEP3SBMCstrategypaperseemtosuggestthatout-of-schoolgirlsareatargetpopulation,indicatingthatG4Gwill‘developanactionplanforcommunitysensitisationand[promote]theenrolmentofout-of-schoolgirls’andthatmembersofG4Ggroupswill‘serveaspeermobilisers,identifyingtheirpeerswhoareneverenrolledordroppedoutofschool’.22Furthermore,theageofthetargetpopulationwillrequireconsiderationasbarrierstoeducationvaryaccordingtothegirls’lifecycle.
Finally,girls’empowermentisamultidimensionalconcept,rangingfrommoretangibleconditionslikeaccesstoresourcestomoreintangible,subjectivevalues,feelingsandperceptions–bothattheindividualandthestructurallevel.Insum,thefocus,scope,conceptualfoundationandstrategyoftheinterventionneedstobeclarifiedtoenhancetheplausibilityofitsToC.
SomefactorsthatKIsmentionedasinfluencingthelikelysuccessofG4Gare:
• TheschoolneedstohavefemaleteachersorMAmemberstoactasmatronsandrolemodelsforthegirls.KIsconfirmthatthedeploymentoffemaleteacherstoruralschoolsischallenging.TheStrengtheningWomenandGirls’Spaces(SWAGS)projectdealtwiththischallengebyliaisingwiththeEducationSecretariesatthelocalgovernmentlevelinordertotransferfemaleteacherstoalimitednumberofpilotschools.However,itseemslessfeasibletoimplementthisprocessatscaleacrossallruralschools.
• Lowqualityteachingandunqualifiedteachersattheschoolmaypotentiallyaffectgirls’retention.EmpiricaltestingwouldberequiredtodeterminetheextenttowhichaG4Gcouldcounterthenegativeeffectonretentionofalowqualityteachingenvironment.
21Aspartofthereviewofthisdocument,UNICEFspecifiedthattheobjectiveoftheG4Gistheretentionofgirlsenrolledinprimaryschools.22Aspartofthereviewofthisdocument,UNICEFindicatedthatout-of-schoolgirlsarenotatargetpopulationoftheintervention.
EvaluationoftheGirl’sEducationProject3
©EDOREN 14
• HeadteachersneedtobeconvincedabouttheprogrammesothattheirleadershipcanfacilitatethefunctioningoftheG4G.Equally,thebuy-inby,andfunctioningof,theSBMC/MA,undertheumbrellaofwhichtheG4Gwillbeestablished,isrequired.Parents–particularlyofout-of-schoolgirls–shouldalsobeconvincedaboutthisapproach.
• AKIquestionedhowgirlswillbemotivatedtoparticipateinG4Gmeetings.Vocationalactivitiesaresuggestedasameanstomotivategirls.However,totheextentthatsuchvocationalactivitiesrequireresourcestooperate,resourcemobilisationwillneedtobeincorporatedaspartoftheG4Gactivities,tomakethemsustainable.23
Animportantcontextualfactorthatwillinfluencegirls’participationinG4Gmeetingsistheaccessibilityandsecurityoftheschool.ItcanbeexpectedthattheoutcomesofG4Gwillvarybetweenmoreruralandurbanareas,giventhedifferencesinschoolaccessibility.
2.3.3 Output2:Improvedcapacityofteacherstodelivereffectivelearningforgirls
InterventionsunderGEP3’sOutput2seektoensurethat,oncegirlsareenrolled,improvedteachingqualityandschoolmanagementresultinbetterlearningoutcomes.Particularemphasisisputonearlygradeinstructionandlearning,becauseearlyacquisitionofliteracyandnumeracyskillsisconsideredfoundationalforfuturelearning.Thefollowinginterventionsareimplementedwiththeaimofachievingthisoutcome:
• Teachercapacitydevelopment:trainingandmentoringofprimaryschoolandIQSteachers,with,atthecore,a1.5-yearcycleofmonthlycluster-leveltraining/mentoringmeetingsinitiatedbyfive-dayinductionworkshop;
• Headteachercapacitydevelopment:cycleofthree-daytrainingsessionspertermoveratwo-yearperiodforprimaryschoolsandIQSheadteachers;
• Earlylearninginterventions:earlygradeteacherin-serviceprofessionaldevelopmentinprimaryschoolsandIQSs,includingcluster-leveltraining,weeklyschool-basedpeer-mentoring,on-sitemonthlymonitoring,headteachertraining,andteachingandlearningmaterialdistribution.
TheoveralllogicandobjectivesofthisoutputarewidelyunderstoodandsupportedbyKIs.Lowlearningoutcomesandinadequateteachingqualityarecommonlyrecognisedaskeyproblems.AwiderangeofKIsargued,inlinewithGEP3’sToC,thattacklingthequalityofteachingandteachersiscentraltoimprovinglearningoutcomes,whileacknowledgingthatthisneedstobesupportedbyanimprovementinthebroaderlearningenvironment.Thecapacityoftheheadteacherisrecognisedasanimportantsupportingfactor.Furthermore,KIsvalidatedtheimportanceofearlygradelearningasameansofacquiringbasicliteracyandnumeracy,andpavingthewayforsuccessfullearningoutcomesinlatergrades.
Beforepresentinganintervention-leveldiscussionbelow,itisimportanttohighlightthatingeneralmonitoringandsupervisionareconsideredkeysupportingfactorstotranslatethecapacity
buildingofteachersandheadteachersintoschool-levelbehaviourchange,andultimately
improvedlearningoutcomes.ThisisrecognisedinGEP3’sToC,whichsupportsitsplausibility.SomeKIsexpressedtheirappreciationforGEP3’sincreasedemphasisontheissuesinceitsredesign.
‘Weneedbettermonitoring.Whenyoutrainfacilitators,givethemsupportandmotivatethem,itisbettertosuperviseormonitorthemtoseeifworkisbeingdoneornot.Monitoringand
23ResourcemobilisationisindicatedintheTransformingEducationforGirlsinNigeria(TEGIN)Girls’ClubManualascontributingtotheeffectiverunningandsustainabilityofagirls’club.
EvaluationoftheGirl’sEducationProject3
©EDOREN 15
supervisionisveryimportant.Sometimesmonitoringhappens;especiallythosethatarepaidNGN7,500,theysupervise.’(Non-governmentKI,Sokoto)
‘GEP3introducedmonitoring.Aftertrainingtheteachersaremonitoredintheirschools;howaretheyimpartingtheknowledgethattheyacquiredduringtraining?Thiswillhelpalottogetthelearningoutcomes.Whathappensintrainingisdifferentfromwhathappensintheclassroom.’(GovernmentKI,Katsina)
TheplausibilityofGEP3’sToCwithrespecttomonitoringcanbestrengthenedbybuildingacommon
understandingamongstakeholdersabouttheobjectivesofthedifferenttypesofmonitoring
activities.DifferenttypesofmonitoringandsupervisionarepresentinGEP3’sToCindifferentways–eachwiththeirownlogicandobjectives.Thesetypesofmonitoringandsupervisionarecurrentlynotwelldistinguished,commonlyunderstoodorspelledout.SeveralKIspointouttheimportanceofSBMCmembersmonitoringteachers,butKIsdonothaveacommonunderstandingaboutthefocusandscopeofthismonitoring.Forexample,itremainsambiguouswhetherSBMCmembersaremandatedtomonitorteachingpracticeandoutcomes,ratherthantofocusonmonitoringteacherattendance.24Headteachermonitoringiscommonlyunderstoodasincludingpedagogicalsupervisionandmentoring,butitisnotclearhowthiswillworkforIQSproprietorswhomaynothaveapedagogicalbackgroundorinterest.Trainersofteachersandheadteachersaremeanttoconductfollow-upmonitoringthatissupposedtoincludeamentoringelement.However,howthiswillbeoperationalisedremainsvague.Also,localandstategovernmenteducationstaffhaveamonitoringfunction,whichisgenerallyunder-resourced.Howtheirmonitoringfunctionalignswiththebefore-mentionedmonitoringrolesisalsounclear.
Likemanyinterventions,monitoringandsupervisioncanbeeffectivewhentherightconditionsaremet.Forexample,KIsconfirmthatthemonitoringpotentialofaCBMCiscircumscribedbythewillingnessoftheIQSproprietortosupportsuchmonitoring.Thesupervisoryleadershipofaheadteacherisconditionedbyhis/herowncapability,credibilityandpowertoact.ManyKIshavehighexpectationsofthemonitoringroleofSBMCsbutalsoacknowledgetheirstillemergingcapacity.Suchpotentialconstraintsneedgreaterrecognitionandreview,inordertobeabletoadaptschoolmonitoringtotheactualconditionsintheschools.
2.3.3.1 Teachercapacitydevelopment
Asmentionedabove,thereisawideconsensusamongtheKIsthatdevelopingteachercapacityin
primaryschoolsandIQSsishighlyrelevant,andthisiswellacceptedasanobjectiveoftheproject.Theimportanceofemphasisinggender-sensitiveteachingpracticesaspartofGEP3,inordertoimproveoutcomesforgirls,isacknowledged,althoughoneKIhighlightedthatitismorerelevantinupperprimarygrades,whengenderstereotypingbecomesmorepronounced.OtherelementsthatseveralKIsemphasiseasameanstoimprovinglearningoutcomesparticularlyforgirlsare:1)increasingtheirinterestinschoolingbypromotingthebenefitsofeducationviafemaleteacherrolemodels,careertalksorvocationalskillstraining;and2)improvingthegirl-friendlinessoftheschoolenvironmentbyensuringappropriatewaterandsanitationandsecurity.
Inordertofullyassesshowplausibleitisthatthisteachercapacitydevelopmentwillleadtoresults,itisimportanttoconsidertheattributesofitsapproachandintendedoutcomes,suchaswhatcapacitiesneedtobedeveloped,whosecapacitiestheseareandhowandwhenthesewillbe24AccordingtoUNICEF,SBMCsaremeanttoonlymonitorteacherattendance,notpedagogicalpractices,whichistheresponsibilityofheadteachersandteachertrainers.However,theGEP3ToCdocument(UNICEF2015a)indicatesthat‘they[SBMCs]willalsoexerciseoversightovertheperformanceoftheschoolandthequalityoftheeducationprovided(…)’.
EvaluationoftheGirl’sEducationProject3
©EDOREN 16
developed.TheGEP3StrategyPaperonSchool-basedTeacherDevelopment(UNICEF2015h)outlinessomeoftheseelements,suchasafocusonpedagogicalandcontentknowledge.However,KIsaregenerallynotwellawareofsuchattributesoftheredesignedteachercapacitydevelopment
approach.EvengovernmentstakeholdersinvolvedintheIQSfacilitatortrainingscheduledforearlySeptember2015hadonlyavagueunderstandingofthefeaturesofthistraining.GovernmentKIsinBauchiinvolvedintheschool-basedteacherdevelopmentdidnotknowwhattheredesignedschool-basedteacherdevelopmentapproachwouldlooklike.
WhiletheGEP3school-basedteacherdevelopmentstrategypaperandGEP3ToCpostulateasafirstassumptionthatwhereteacherdevelopmentiswelltargetedandrelevantteachers’performancewillimprove,whatthistargetingentailsremainsunclearinthestrategydocuments.TargetingspecificgroupsofteachersrarelycameupduringtheKIIsascriticalinordertoimproveteachingorlearning,althoughonceprobedKIsdosupporttargetingearlygradeteachers.Thismaybeduetothefactthatthecapacityneedsarecommontomostteachers,or,asoneKIpointedout,thatteachertraininghastraditionallybeenaboutallteachers‘fairly’benefittingfromtraining,ratherthanimplementingtrainingwithapurposeandresultinmind,suchastrainingthoseteachersthatmaycontributemosttoimprovingpupilslearningoutcomes.SeveralKIsacknowledgethatinthepast,theselectionoftrainingparticipantshasresultedinunqualifiedpeoplebeinginvitedandsuchselectionlackedmonitoring.25Otherspointoutthatthe‘right’or‘trainable’teachersneedtobetrainedbutwhattheattributesofsuchrightandtrainableteachersareremainsundefined.Therefore,itwouldincreasetheplausibilityoftheToCasregardsachievingitsoutcomesiftheapproachtotargetingwaselaboratedon,intermsofwhichteachersitwouldbebesttotrainandwithwhatresultinmind.
Ingeneral,theKIsinterviewedonthetopicfavouredachild-centred,activelearningapproachtoteaching,withafocusonbasicliteracyandnumeracy.Thecommunicationandlanguageskillsofteachersarealsomentionedasanelementtoincludeinteachertraining.Monitoringand
supervisoryfollow-upvisitsareconsideredkeytoachievingresults.
Aclusterapproachforfollow-upteacherinteractionisvaluedasameansforcontinuouslearningandsharingofexperiences.However,KIsquestionwhetherresourceswillbeavailablefortraineesandmentorstoattendregularmeetings,andwhetherthementorswillbesufficientlypreparedtoeffectivelyfacilitatethemeetings.Aone-offtrainingapproachisconsideredtobeinsufficienttoachievechangesinteachingpractices,whichtheGEP3teacherdevelopmentapproachhastakenintoaccountbyprogrammingfollow-upmentoring.
TheGEP3ToCindicatesthatIQSfacilitatorcapacitydevelopmentwillbeadaptedtoteachingandlearningintheIQSenvironment.TherearelikelytobelargevariationsinthebackgroundsoftheIQSfacilitatorsandtheIQSsinwhichtheyareteaching.Somefacilitatorsareteachersintheformaleducationsystemwhileothersarecommunitymemberswithlittleinthewayofteachingqualifications.26Teachinghours,availabilityofmaterials,teachingfacilities,pupilcharacteristics,27typeofIQS28andproprietorsupportarelikelytovaryconsiderably.Therefore,thecapacitybuilding
25ThiswasconfirmedbytheIQSSbaselinesurvey,whichfoundthataroundone-fifthoftrainedfacilitatorsthatwerefoundinIQSseligibleforthesurveywerenotteachingattheschool.26 InBauchi,KIs indicated that facilitatorsworking inSUBEB-managed IQSsaregenerally selected fromteachersunderaLGEA contract. In Sokoto, facilitators are identified from the IQS community or neighbouring community, with SAMEcarryingoutanexaminationofthefacilitatorpresentedbytheproprietor.27ReadingandAccessResearchActivity(RARA)researchamongIQSsinBauchiandSokotohasfoundthatIQSpupilsbelongtoawiderangeofagecategories,includingadultlearners.28Qur’anicschoolsvaryconsiderablyintermsoftheQur’aniceducationprovided,thetypeofpupilstheyattract,andhowthepupilsrelatetotheirtraditionalreligiousteacher,themallam.Forexample,IQSsmaybeboardingschoolsthatattract
EvaluationoftheGirl’sEducationProject3
©EDOREN 17
approachdoesnotonlyneedtobeadaptedtotheIQScontext,butalsoneedstotakeintoaccount
largevariationsinthatcontext.
‘Thepedagogy,approach,andthemethodaredifferentinnon-formalcomparedtoformal.UNICEFtrainingusedtojoinfacilitatorandteachertraining,butinrealityitisdifferent.Thesefacilitatorswouldjustgo,hearandlearn,buttheycannotgototheirlevelanduseitbecausethestrategyisdifferenttotally.’(GovernmentKI,Katsina)
TheGEP3IQSstrategypaperjustifiablyemphasisestheimportanceofmentoringinthecapacitybuildingprocess.Thismaymakeitpossiblefortheneededtailoredapproachtobeadopted.However,KIsacknowledgethattheeffectivenessofsuchmentoringassumesadequateresourcesformentormeetingstobeheld,sufficientmentorcapacityandfacilitatorsbeingsufficientlymotivatedtobementored.Asmentionedabove,italsoneedstobeclarifiedwhatsuchmentoringwillactuallyentail,andwhowillbeinvolvedandinwhatrespect.
SomeotherkeyfactorshighlightedbyKIsasaffectingtheresultsofteachercapacitydevelopmentonteachingandlearning,areasfollows:
• Theinitialskillsandknowledgeofteachersarelikelytoaffecttowhatextenttheteachersareabletotranslategainedknowledgeandskillsintopractice.SeveralKIsclaimthatsometeacherswhoareselectedfortrainingare‘untrainable’becausetheydonothavetheminimumunderstandingneededtocomprehendthetraining.ThebaselinefindingsinbothprimaryschoolsaswellasIQSsdemonstratethatteachershaveverylimitedknowledgeandskillsinthemajorityoftheareasrequiredtofunctioneffectivelyasateacher.Inparticular,teachersdisplaylowcompetencyinHausawritingandcomprehension.TheGEP3ToCacknowledgesthistosomeextent,bystatingthattheselectionofteacherswill,amongotherthings,bebasedonaconceptualunderstandingofteaching.Hence,teachertrainingpedagogyandmaterialswillneedtobeappropriateforverylowinitialteacherskillsandknowledgelevels.
• TeachercommitmentandmotivationisanotherinfluencingfactorhighlightedbyseveralKIs,whichislinkedtoteacherattendance.Lowcommitmentandmotivationisassociatedwithlowandirregularremuneration,poorlivingandsocialamenitiesinruralareas,andapoorteachingenvironment.AccordingtoKIs,primaryteachersarenotalwaysremuneratedontimeinallGEP3states29andtheirsalaryisconsideredtoolowtoincentivisedeploymenttoruralareaswhereamenitiesandschoolfacilitiesarepoor.Furthermore,severalKIspointedtoteacherappointmentanddeploymentbeingpoliticised,whichalsoaffectscommitment.
• RemunerationappearstobeparticularlyproblematicforIQSfacilitators.IQSfacilitatorsaregenerallypaidamonthlyallowance,whichisnotonaparwiththestandardofNGN7,500permonthinanyoftheGEP3states.30TheallowanceseemstovarybetweenNGN3,000andNGN5,000,andtheactualpaymentoftheallowancedependsontheavailabilityoffunding.ExperiencefromtheEducationSectorSupportProgrammeinNigeria(ESSPIN)indicatesthatthenon-paymentofallowanceseventuallyresultsinfacilitatorsstoppingteaching.ThebaselinefindingsoftheIQSSevaluationconfirmthataverysmallshareoffacilitatorsarepaid,witha
mostlyitinerantboysandthatfocusonrecitationoftheQur’an,whileothersmayoperateasadayschooltargetingboysandgirlsfromthecommunity.29 Informationabouttheregularityof thepaymentof teachers’pay ismixedandseeminglycontradictory. InBauchi, thepayment of civil servantswas not regular in 2015. In Katsina and Zamfara, a SPARC/SAVI political economy assessmentdatedJuly2015indicatesthatthegovernmentowesuptosixmonths’ofsalariestoworkers.However,KIsinthesestatesindicatethatnosalariesareowedtoteachers.30 It isnotclearwherethisstandardcomesfrom.OneKI indicatedthatthis istheamountrecommendedintheNationalBenchmark,buttheNationalBenchmarkforIntegratingBasicEducationintoQur’anicSchoolsinNigeriarecommendsNGN10,000.
EvaluationoftheGirl’sEducationProject3
©EDOREN 18
positiveassociationbetweenreceivingremunerationandteachermotivation.TheGEP3ToCassumptionthatgovernmentwillpayteachersregularlyandappropriatelytosupportmotivationinregardtoeffectiveperformanceisthereforeunlikelytoholdinthecaseofIQSs.
• Teacherattendanceandmotivationis,however,alsolinkedtosupervision,monitoringand
accountability,which,accordingtosomeKIs,isamoresignificantfactorinfluencingteacherbehaviourthanremuneration.TheseKIsmakethecomparisonwithprivateschoolsinGEP3states,whereteacherpayisgenerallylowerthaninpublicschoolsbutteachingandlearningoutcomesareperceivedtobebetterbecauseteachersareheldtoaccountinrelationtodeliveringeffectiveteaching,andareadequatelysupervised.31ImprovementofteachermonitoringandsupervisionisincorporatedwithinGEP3’sToCviaimprovementofheadteachercapacityandSBMCgovernance.However,asdiscussedabove,thelikelihoodofitactingasastrongsupportivefactorinGEP3’sToCcanbestrengthenedbyclarifyingitsobjectivesandrecognisingitlimitationswhencertainconditionsarenotinplace.ThequalitativeresearchfindingsoftheIQSSevaluationsuggeststhataccountabilityrelationshipsareweakenedinthecaseofIQSfacilitatorsbecauseofalackofremuneration.
• Accesstoteachingandlearningmaterialsandtheschoolenvironmentarelikelytoaffecttheextenttowhichimprovedteacherknowledgeandskillscantranslateintoeffectiveteachingandlearningoutcomes.MostofthesefactorsareincorporatedintoGEP3’sdesignthroughdifferentinterventions.Itisnotensured,though,thatthesedifferentinteractingfactorssupporteachotheratthesametimeattheindividualschoollevel.
• Pupilattendanceandmotivationareinfluencingfactorsinregardtoimprovinglearningoutcomes.32Output1interventionsaremeanttocontributetogirls’schoolattendance.Eventhoughthecashtransferprogrammeisunconditional,KIsbelievethatgirls’attendanceispositivelyinfluencedbecauseofSBMCs’follow-uponnon-attendanceandthesensitisationimplementedaspartoftheprogramme.Girls’motivationhasalreadybeenbrieflydiscussedabove.
• InIQSs,additionalfactorsneedtobeconsidered.Theattitudeoftheproprietorandthecommunitytowardsintegrationwillinfluencetheextenttowhichabetterskilledfacilitatorisabletoputimprovedteachingskillsintopractice.Asthequotebelowemphasises,thisrequiresbothcontinuoussensitisationoftheproprietorandbuildinghis/hertrust.Iftheproprietordecidestodiscontinueteachingoftheintegratedcurriculumwithintheschool,theopportunitytoimprovelearningoutcomeswithintheschoolwillbeterminated.‘Howdoestheproprietoraffecttheteachingandlearning?Heistheowneroftheschool,soifhedoesn’tacceptnobodywillevengotoschool.(…)Yousensitisethemandcreateawarenessandgivethemthatsupporttotheschool.Someofthemdon’thavetoiletorwaterpointanditisveryimportantfortheschools.Youstillgivethemthestipendmonthly.Itisbecauseofthesupportthatiswhytheintegrationistakingplace.Ifthereisnosupportnothingwillhappen.Theyalsoneedtotrustthatyouarenotgoingtotaketheirschoolfromthem.’(GovernmentKI,Katsina)
Furthermore,theactualavailabilityoffacilitatorstotrainandmentorisanobviousbutimportantassumptionoftheToC.ThiswillvarybyIQSandstate.KIsinallstatesotherthanSokotoindicatethatonaverageonefacilitatorteachesformalsubjectsineachIQS,whichislessthanthenumberassumedintheGEP3logframe.AccordingtobaselinedatafromBauchi,Niger,KatsinaandZamfara
31Itshouldbenotedthatnon-religiousprivateschoolsaremorelikelytobelocatedinurbanareas,whichmaybeanotherexplanatoryfactorfordifferencesineducationoutcomesbetweenpublicandprivateschools.32 Other factors, like the child’s health, cognitive ability and socio-economic background, can also influence learningoutcomes.Suchfactorsarenotdiscussedhere.
EvaluationoftheGirl’sEducationProject3
©EDOREN 19
mostIQSshaveonetotwofacilitators.33AnotherIQS-specificfactoristhatpupilsmayattendboth
anIQSandapublicprimaryschool.ThiswasparticularlyobservedinthebaselineIQSsampleinBauchi,where72%ofpupilsreportedattendinganotherschool.ThiscouldcontributetothedifferencesinlearningoutcomesthathavebeenobservedbetweenpupilsofIQSsandpublicprimaryschools(USAID,2013).34However,thisdifferenceisnotwellunderstood.KIsspeculatethatitcouldbeexplainedbythelevelofsupervision,thedegreeofinteractionbetweenpupilsandproprietors,thematurityofthepupils,orpupils’motivation.
Theabovefactorsareembeddedintheinstitutionalandpolitical-economycontextofeachstate,whichdetermineteacherrecruitment,deploymentandremunerationpolicies,fundingdecisions,
andinstitutionalrolesandresponsibilities.Forexample,Katsina’srestrictivecivilservantrecruitmentpolicyinthepasthasaffectedteachersupplyinthestate,whichislikelytoinfluencethepoolofteachercandidatestobetrainedandtheschools’pupil–teacherratio.Teacherdeploymentmayalsohavebeenaffectedbythe2015elections.Giventhatteacherdeploymenthasbeensubjecttopoliticalinfluenceinthepast,politicallyinspiredteachertransfersmaytakeplace,whichagainmayaffecttheteachersthataretrained.Institutionalarrangementsmayalsoaffectteachingandlearningoutcomes.TheGEP3IQSsaremanagedmostlybySAME,exceptinBauchiwheremostGEP3IQSsfallunderSUBEB.Thecapacityofbothinstitutionsdiffer,amongotherthingsintermsofaccesstofundingandtrainedteachers,aswellasintheirexperiencewithintegrationandthenon-formalsector.Thisaffectsthecontextandconditionsinwhichtrainedfacilitatorsoperateandmentoringhappens,whichcouldinfluenceresults.35
2.3.3.2 Headteachercapacitydevelopment
AccordingtoGEP3’sToC,headteachercapacitydevelopmentaimstoimprovethecapacityofheadteacherstosupporteffectiveteachingandschoolmanagement,contributingtoimprovedlearningintheclassroomandagender-sensitiveschool.Atthetimeoftheinterviews,thisinterventionwasyettobeoperationalised.Therefore,theattributesofthisinterventionanditsoutcomes(what,who,how…)wereyettobedefinedbasedonanassessmentofheadteachercapacityneeds.
Animportantattributethatrequiresclarificationistheexpectedfocusofthecapacitybuilding.KIsemphasisethedifferentrolesexpectedoftheheadteacher,suchasschoolmanagement,pedagogicalleadership,andsupervisionofcurriculumdelivery.Holdingteachersaccountableisalsomentionedasimportant,butitremainsundefinedhowandwhythisshouldtakeplace,aswellashowtheheadteacherisincentivisedtoholdteachersaccountable.Leavingthesefactorsundefinedlowerstheplausibilityasregardstheinterventionachievingresultsbecausethecapacitydevelopmentmaynotsufficientlytargetthekeymechanismsthroughwhichheadteacherscancontributetoeffectiveteachingandlearning.GEP3isscheduledtoconductaparticipatoryreviewofitscurrentapproachtoheadteachertraining.
TheToCofthisinterventioninthecontextofIQSsrequiresfurtherclarification.TheGEP3ToCsuggeststhattheinterventiontargetsheadteachertrainingingovernmentschools,whiletheIQSstrategypapersetstargetsforheadteachertraininginIQSs.KIsworkingintheIQSsectorindicate
33InBauchiandNiger25%ofIQSshaveonlyoneteacherofnon-religioussubjects,while37%havetwo.InKatsinaandZamfara,40%ofIQSonlyhaveasingleteachertoteachintegratedcurriculumsubjects.34RARAresearch in Islamiyya,Qur’anicTsangayaEducationcentres (IQTEs) foundthat IQTE learnershave relativelyhighlearningoutcomescomparedtoearlygradeprimaryschoollearners.35 RARA research (USAID, 2014) explored the association between learning outcomes and the agencies supporting theIQTEs. Higher performance was observed in IQTEs supported by SAME compared to SUBEB. However, it was notdemonstratedifsuchdifferencesweresignificant.Thecausalroleofthisinstitutionalfactorisalsouncertain.
EvaluationoftheGirl’sEducationProject3
©EDOREN 20
thataheadteacherpositionthatisseparatefromtheproprietororMallamrarelyexistsandthattheIQSproprietortakesontheroleofheadteacher.TheroleoftheproprietorvariesstronglyfromoneIQStoanother:focusingonschooladministrationinsomewhilebeingstronglyinvolvedintheteachinginothers.Hence,capacitydevelopmentofproprietorswillneedtobetailoredand/or
flexibletotheIQScontextandalignedwiththecapabilities,interestsandroutinepracticesoftheproprietors,inordertoobtaintheirbuy-in.
Severalinfluencingfactorsmentionedaspartoftheteachercapacitydevelopmentinterventionequallyaffecttheresultsoftheheadteachercapacitydevelopment.TwofactorsareworthhighlightingwithregardstothefunctioningoftheheadteachersinthecontextofGEP3states.First,KIsemphasisethatheadteacherappointmentisnotnecessarilytiedtoqualificationsandcanbe
subjecttopoliticalallegiance.Thismayaffecttheircompetencyandmotivationtocarryouttheposition,theirlikelihoodoftransferattimesofpoliticaltransitionandtheircredibilityinregardtoholdingteacherstoaccount.
‘Theappointmentofheadteachersissometimesbasedonpoliticaldemands.Politiciansdemandthatanallyismadeheadteacher.Youmayfindthattheheadteacherhaslowerqualificationsthantheteachers.Thismakesheadteacherslosetheircapacitytolead.’(GovernmentKI,Zamfara)
Second,theirabilitytoinfluenceteacherappointmentorbudgetallocationtoimprovethelearning
environmentiscircumscribedbythewiderpublicserviceandfinancialmanagementsysteminthe
state.TheEDORENliteraturereviewofbasiceducationinNigeria36indicatesthatheadteachershaveverylittlepowerandfewresourcestoaffectchange.Therefore,itisanuncertainassumptionthatheadteachershavethecapabilitytomakeschools,forexample,moregirl-friendly(inthewaythatKIsmostlyinterpretagirl-friendlyenvironment):thatis,providingseparatetoilets,installingwaterpointsandappointingfemaleteachers.GEP3doesaddressthisassumptionbystrengtheningtheSBMCs’capacitytomobiliseresourceswhichcansupportheadteachersinregardtobringingaboutchange.ThisassumesstrongcoordinationandalignmentofinterestsbetweenheadteachersandSBMCs.
2.3.3.3 Earlylearningintervention
TheearlylearninginterventionhassetasitsmainoutcometheimprovementofHausaliteracyinGrades1to3.37ThemaincausalassumptionunderlyingitsToCisthatliteracylearningoutcomes,particularlyinthemothertongue(whichisassumedtobeHausaformostpupils),willimproveinearlygradesifteachingimprovesintermsof:a)ateachingprocessthatemphasisesliteracyasafoundationalskillandhasafocusonactivelearningandtimeontask;andb)theuseofthepupils’mothertongueastheprimarymediumofinstruction.
Asmentionedabove,awiderangeofKIsendorsetheimportanceofearlygradelearning,whichprovidessupportfortheweightthatthisinterventionhasreceivedintheredesignedGEP3programme.Furthermore,KIssupporttheemphasisonliteracyduringearlygradesasafoundationalskill.AswasstatedbyoneoftheKIs:‘oncepupilsareabletoreadandspeaklearningwillgoeasily’.ThiswasalsoconfirmedinanIQSsetting.OneKIinvolvedinIQSteachingstatedthat‘thefirststepweareofferingistoshowthemhowtoread,howtowrite’.
36HumphreysandCrawfurd(2014).37HausaisthemainlanguagespokeninnorthernNigeria.
EvaluationoftheGirl’sEducationProject3
©EDOREN 21
Withregardstotheuseofthemothertongueastheprimarymediumofinstructioninearlygrades,thereisnooverallconsensusamongKIstowhatextentthisisanecessaryconditionforimproving
learningoutcomes.SeveralKIsargueinfavour–thatis,tostrengthentheuseofthemothertongueinearlygradesbasedonthefollowingarguments:thepupilwillbetterunderstandconceptswhentheyareexplainedinthemothertongue;communicationintheclassisenhanced;thetransitionbetweenhomeandschoolissmoothened;andtheexposuretophoneticsandphonologyinthemothertonguefacilitatestheirunderstandinginotherlanguageslikeEnglish.OtherKIsseetheuseofthemothertongueaspotentiallyusefulbutdonotattributelowlearningoutcomestolanguageorconsideritanecessaryconditiontoimprovelearningoutcomes,emphasisingthatotherelementsinthelearningenvironmentaremoreinfluential,suchasteachercompetencies.Comparisonismadewithprivateschools,whereEnglishisusedthroughoutandlearningoutcomesareperceivedtobebetter.
Questionsarealsoraisedaboutthedesirabilityandacceptanceofusingthemothertongueinearly
grades.SomeKIsdoubtthattheuseofthemothertonguewillbeacceptedinmetropolitanareasorareaswithhigherliteracylevels.Theyalsorefertothepotentialunintendedoutcomethatwhenmothertongueinstructionismorestronglyenforced,moreeducated,wealthyparentsmaysendtheirchildrentoprivateschools,reinforcingthedichotomybetweenprivateandpublicschools.Also,parentsmaynotsupporttheuseofthemothertonguesincetheymayperceivetheuseofEnglishasaproxyforthequalityofeducation.Furthermore,someKIssuggestthattheuseofthemothertongueinearlygradesmayresultinitcontinuingtobeusedinupperprimarygrades.Finally,someKIsarguethatHausaisnotthemothertongueofallpupils,andHausadialectsvaryfromstatetostate.However,thebaselinedatainKatsinaandZamfara,wheretheearlylearninginterventionispiloted,confirmthatHausaisthelanguageoftheimmediateenvironment,whichvalidatestheToCassumptionforthisintervention.Also,merelybecausetheyspeakalanguagedoesnotmeanthatteachersareliterateinit,orthattheyareabletouseteachingandlearningmaterialsinthatlanguage,eveniftheyareavailable.Thebaselinedatapointtothisreality.WhileallteacherssurveyedreportbeingabletospeakHausa,Hausaliteracylevelsamongteachersarelow.
‘Theimpedimentisthatsometeachersarenotversedinusingthelanguageforinstruction.Thereisadifferenceinspeakingalanguageandusingit.TherearedifferenttermsinHausafromstatetostate.TeachinginHausaisdifficult.’(Non-governmentKI,Katsina)
ImplementingtheearlylearninginIQSswillrequirecarefuldesign.Asmentionedbefore,thecapacitybuildingapproachneedstobeadaptedtotheIQScontextandshouldtakeintoaccountlargevariationsinthatcontext.WhatconstitutesearlygradesfromtheperspectiveoftheRANAteachingandlearningmethodologyneedstobeassessedagainsthowgradeprogressionisorganisedinIQSs.Othervariationsoftheorganisationoftheselevelsarelikelytoexist.Furthermore,themothertongueorlanguageofimmediatecommunityistherecommendedmediumofinstructioninBasic,whileEnglishandthelanguageoftheimmediateenvironmentisrecommendedinPost-Basic.TotheextentthattheRANAapproachincludesgrade-specificfeaturesthatarenotwelladaptedtotheIQScontexttheresultsoftheinterventionmaybeaffected.Also,theRANAschool-basedpeer-mentoringwillneedtobeadaptedtothecontextofahighpercentageofIQSsonlyhavingasinglefacilitator,asthebaselinedemonstrates.Finally,thedesignoftheinterventionwillneedtosuitthewaytheproprietorhasorganisedthedeliveryofthecurriculumintheIQS.
2.3.4 Output3:Improvedgovernancetostrengthengirls’education
AccordingtoGEP3’sToC,interventionsunderthisoutputaimtoinfluencekeyareasofgovernanceinstatebasiceducationsystemswhichaffectgirls’basiceducationinthelongterm.Theoutput
EvaluationoftheGirl’sEducationProject3
©EDOREN 22
addressesbothschool-levelgovernancethroughtheempowermentofSBMCs/CBMCsaswellasmostlystate-levelgovernancebysupportinginterventionsthatinfluenceplanning,programmingandfundingforgirls’education.ThemaininterventionsunderOutput3are:38
• capacitybuildingSBMCs/CBMCs:trainingandfollow-upmonitoringofSBMC/CBMCmembersfrompublicprimaryschoolsandIQSs,includingtheprovisionofschoolgrantsmanagedbySBMC/CBMC;
• supporttotheAnnualSchoolCensus(ASC)andtheeducationmanagementinformationsystem(EMIS):trainingandtechnicalsupporttostate-levelEMISstaffinASCplanning,dataentry,dataverification/cleaning,databasemanagementanddataanalysis;
• supporttotheGirls’EducationSteeringCommittees(GESCs):coordinationsupportforcontinuationoftheGESCsineachstateandatnationallevel;and
• supporttoHiLWA:supporttotheadvocacygroupHiLWAatthestateleveltoengagewithdecision-makerstoinfluenceactionstoimprovewomen’sandgirls’participationintheeducationsector.
2.3.4.1 EmpowermentofSBMCs/CBMCs
HighexpectationsregardingSBMCs.GovernmentKIsconsiderSBMCsakeyinstrumenttopromote
communityownershipofschools,withtheexpectationthatstrengthenedcommunityownershipwillalleviatedemand,supplyandgovernanceproblemsattheschoollevel.GovernmentKIshighlightedthatcommunitiesshouldnotrelysolelyonthegovernmenttoensureschooldevelopment.TheyseeSBMCsasameanstoshareresponsibilityforeducationwiththecommunity.Non-governmentKIsalsoseeSBMCs‘inthelimelight’,asoneintervieweeputit.TheyappreciateSBMCsasbeingawaytostrengthencommunityownershipoftheschool,highlightingtheirroleinschoolmonitoring,sensitisingparentsandschoolmanagement,observingthatSBMCsare‘inthedriver’sseatwhenitcomestowhattheschoolreallyneeds’,‘domorethangovernmenttomaintainschools’,and‘dothemonitoringthatpeoplefromtheministryareunabletodo’.
GiventhepivotalrolethatSBMCsareassignedintheschoolsysteminGEP3states,andthefactthatalmostallGEP3-supportedinterventionsrelystronglyontheircontribution,itishighlyrelevanttocontinuestrengtheningtheircapacityandsupporttheirempowermentasameanstoimproveschoolgovernanceaspartoftheredesignedGEP3.
WhiletheroleofSBMCsinschool-levelgovernanceisbroadlysupportedbyKIs,theirroleinthegovernanceofthewiderstateeducationsystemremainstobeclarified:inparticular,theirrolein
regardtoholdingstateandlocalgovernmentstoaccountforservicedelivery.ThisremainedlargelyunaddressedbygovernmentKIs,althoughsomeappreciatedSBMCsasbeinganefficient,directchanneloftheinformationneededforstate-levelplanningandaction.Non-governmentKIsperceiveSBMCsasagrassrootsvoicethatcanputpressureonthegovernmenttofulfilitscommitments.GEP3doesnotexpectoraimtosupportagovernanceroleforSBMCsbeyondtheschoollevel.39Thisassumesthatbyholdingactorsattheschoollevel(e.g.headteachersandteachers)toaccountSBMCscanimprovetheperformanceoftheschool.However,thisislikelytohaveitslimitationsbecauseseveralaspectsofschools’performance(suchashumanresourcemanagement,provisionoflearningmaterials,schoolinfrastructure)dependonstateandlocalgovernmentaction.TheremunerationoftheIQSfacilitatorsisacaseinpoint.TheIQSSbaselinedataindicatethatlocal38Output3alsoincludesadvocacyfortheFemaleTeacherTraineeScholarshipScheme(FTTSS).Thisinterventionhasnotbeenthefocusofthisassessmentbecauseitisbeingphasedout.39ThiswasconfirmedbyUNICEFaspartoftheirreviewofafirstdraftofthisreport.
EvaluationoftheGirl’sEducationProject3
©EDOREN 23
stakeholdersconsideritdifficulttoholdunpaidfacilitatorstoaccount.Hence,theCBMCs’effectivenessinregardtoimprovingfacilitatorperformancedependsongovernmentactionthattheCBMCmayormaynotbeabletoinfluence.
TheempowermentofSBMCsisbothapivotalintermediaryoutcomeinGEP3’sToCaswellasone
ofitsmostprecariouslinks.ThereisagapbetweenthehighexpectationswithregardtoSBMCs,intermsoftheirmanyrolesandresponsibilities,40andthecapacitybuildingandempowerment
processthatwillbegradualandthatstartsfromanoftenlowinitialcapacitylevel,particularlyinruralareas.KIspointoutthat,particularlyinruralareas,manySBMCmembershavelittleeducationandstilllackanunderstandingoftheirrolesandresponsibilities.TheplannedtrainingandmentoringismeanttoaddressthisbutKIsunderlinethatsuchtrainingandmentoringneedstoberegularandcontinuousinorderforittobuildthenecessarySBMCcapacity.TheyalsoindicatethatonlyalimitednumberofSBMCmembersreceivetrainingandthatthecapacitybuiltisnotnecessarilycascadeddowntoallmembers.Monitoringisagainhighlightedasimportant,whichGEP3isrightlyemphasisingviaitsSBMCEffectivenessMonitoringprocess.Giventheimportanceofmentoringandsupportivesupervision,itwouldbevaluabletopayattentiontothisaspectintheSBMCEffectivenessMonitoring.
‘TheSBMCshavebeendoingalotofjobs.Theyarewellmotivated.Theproblemis,especiallyintheruralareas,gettingtheSBMCstofullygrasptheirjobs.Someofthemwhentheygettheinitialtraining,theygettheSBMCsupport,thatwillbetheendofit.Theyhardlythinkofmotivatingtogetsomeinternalsupport.Theyaretherewaitingforthenextsupporttocome.ThereisneedformoretrainingbecausemostoftheSBMCsarenotfullyaware,especiallyintheruralareas.Butintheurban/semi-urbanareasyoufindSBMCsdoingsomewonderfuljobs.Peoplecomingingivingmoralandfinancialsupport,evenbuildingsomeblocksinsomeschools.Gradually,itismovingtowardssomeoftheruralareas.’(Non-governmentKI,Zamfara)
Ownershipisgeneratedthroughparticipationandactionthataddressestheneedsofthecommunity.Itisfacilitatedthroughnetworksthattransferknowledgeandideas,andthatgeneratesocialcapital.41Therefore,theactiveparticipationofSBMCsininterventionssuchasenrolment
drives,thecashtransferschemeandgrantmanagementcancontributetoownershipbuildingtotheextentthatitsbuildsonandnurturesbottom-upenthusiasmforaction,targetsissuesthatcommunitieswanttoseeaddressed,andtakesintoaccountlocalcapacityconstraints.StateSBMC
structures,trainingandparticipationincoordinationmeetings(e.g.forenrolmentdrives)offer
opportunitiesforSBMCtobuildsocialcapital.
TheestablishmentofMAsaspartoftheSBMCisappreciatedbytheKIs.TheyemphasisetheeffectiveroleofMAsatthehouseholdlevelinregardtosensitisingindividualsaboutgirls’enrolmentduringenrolmentdrivesandfollowingupontruancy.WhiletheinterviewsdidnotspecificallyinquireabouttheroleofwomeninSBMCs,thefactthatKIsmainlysituateMAs’andwomen’srolesintermsofsensitisingathouseholdandcommunitylevelsignalsaneedforfurtherinquiryregardingtheirroleatschoollevel–andinparticularregardingtheirabilitytoinfluencea
40 Some of the SBMC responsibilities mentioned by KIs are: supporting enrolment drives; supporting cash transferinterventionprocesses(registration,payment,attendancefollowup);supportG4G;supportingschoolmaintenance/repair;resource mobilisation for school investment; mobilising additional resources to motivate and support teachers; schoolgrantmanagement;developingandimplementingWholeSchoolDevelopmentPlans(WSDPs);keepingrecordsonWSDPsandresourcesmobilised/spent;teacherandpupilattendancemonitoring;curriculumimplementationmonitoring;holdingteachers accountable for performance; follow-up on pupil absenteeism; overall school oversight; maintaining therelationshipbetweenthecommunityandtheschool;andholdinggovernmenttoaccountforservicedelivery.41BurnsandWorsley(2015).
EvaluationoftheGirl’sEducationProject3
©EDOREN 24
girl-friendlylearningenvironment,throughwholeschooldevelopmentplanningand/orholdingheadteachersandteachersaccountableforimplementinggender-sensitiveschoolpractices.
Channellingfinancialschoolsupport,suchastheprovisionoftheGEP3grants,throughtheSBMCsisconsideredbyseveralKIsasanimportantfactorcontributingtoSBMCempowerment.TheyseeitasawaytomakeSBMCsfeelresponsibleandaccountablefortheiractions,andtheyexpectittotriggeradditionalresourcemobilisationwithinthecommunity.
‘Thefinancialsupportencouragesschoolsespeciallyintheruralareas,becauseifyouaskthemtousetheirownmoneytheywillstartaskingquestions.Butifyougivethemthatinitialsupport(thegrant)itwillpropthemup.Unfortunatelysomecommunitieswhenyougivethemthegrantstheyjustsitdownifthereisnomentoringfollow-uptofindoutwhattheyhavedonewiththegrants.Sincethegrantcamein,whatcontributionoftheirownhavetheymadeintheirowntimetotheschoolsbecausethegrantissupposedtoencouragethemtocontributetotheschool.Tobuildupschool–communityrelationsandparticipationofthecommunityinrunningtheschool,financialsupportincombinationwithtrainingandmentoringneedtoworktogether.’(Non-governmentKI,Zamfara)
However,thisdependson:theSBMCseffectivelyaccessingandusingthefinancialsupportfortheschool;communitiesbeingawareofthesupportprovided;SBMCshavingtheabilitytomanagethefinancialsupport;andthefinancialsupport,itsuseandresultsbeingtransparentlymonitored.Furthermore,thebaselinefindingsoftheIQSSevaluationindicatethatwhileCBMCsinIQSsundertaketheeffortofmobilisingadditionalresources,povertylimitstheirabilitytodoso.GEP3grantsseemtohavebeenthemainsourceoffinancialsupportchannelledthroughSBMCs,althoughfederalgovernmentself-helpgrants(providedviaUBEC)arementionedasapotentialalternativesource.ThesearecurrentlychannelledthroughtheLocalGovernmentEducationArea(LGEA).TotheextentthatchannellingfinancialsupportthroughSBMCsisindeedanimportantcontributoryfactorinregardtotheireffectiveness,itwillbeimportanttoclearlydefinetherolesandresponsibilitiesofSBMCsandLGEAsinregardtoschoolfinance.
OtherkeyfactorshighlightedbyKIsthataffecttheresultsofSBMCcapacitydevelopmentandempowerment,areasfollows:
• ThecompositionoftheSBMCs.SBMCmembersneedtobeproperlyselectedfromwithinthe
communitysothattheycanadequatelyengendercommunityownership.Itisarguedthattheinclusionofwell-respectedcommunitymembersincreasesanSBMC’scredibilityandpowertoinfluencebothschoolandcommunity.Thepresenceofwell-educatedmemberswhoareawareoftheeducationsystemcontributestotheSBMCseffectivelytakingontheirmultipleroles.Thisismoreplausibleinurbanthaninruralareas.Effectiverepresentationofwomencanbea
problemduetosocio-culturalnormsinhibitingjointmale-femalemeetings.TotheextentthatMAsareinvolvedinallSBMCfunctions,femalevoicecouldbechannelledviathisstructure.GEP3intendstobringgirls’voicesintoschoolgovernancebylinkingG4GgroupstoSBMCs.However,thisobjectiveneedstobemadeexplicitintheinterventiondesignbecausethisobjectivewasnotincludedinpastexperienceslikeSWAGS.Finally,theassignmentofpositionswithintheSBMCneedstosupportkeepingteachersandheadteachersatarm’slength.KIsmentionedthatitisbestthatteachersorheadteachers,whoformpartoftheSBMCs,nottakeontheroleofchairmanortreasurerasthiscouldunderminetheSBMC’sschoolmonitoringfunctionandcommunityownershipofinvestmentintheschool.
• SBMCacceptanceandsupportintheschoolandcommunity.KIsindicatethatheadteachersandteachersgenerallyaccepttherolethattheSBMCplaysintheschoolsinceitismeanttobea
EvaluationoftheGirl’sEducationProject3
©EDOREN 25
partnerinschooldevelopment.However,totheextentthatSBMCstakeanauthoritativeapproachtoschoolmonitoringandlackcredibility,thisacceptancemayfade.Asmentionedabove,acceptancebythecommunitydependsonthewaySBMCmembersareselectedandhowfartheyupholdtransparency.OneKIpointedoutthatthepoliticalaffiliationofsomeoftheSBMCmemberscanaffectthesupportfor,andfunctioningof,theSBMC.
• Accesstotheschool.Thedistancebetweentheschoolanditscommunities,andaccessibilitybyroad,influenceshowactiveSBMCmemberscanbeintheschool.KIsconfirmthatitislikelythatharder-to-reachcommunitiesmaybelessrepresentedintheSBMC.Thispointstoatrade-offbetweentheeffectivenessofanSBMCandequitablerepresentationontheSBMC.Inaddition,thesecuritysituationaffectsthefunctioningofSBMCs.
• Institutionalisationofgovernmentsupport.TheestablishmentofSBMCsisanationalpolicy,whichhasbeendomesticatedtodifferentdegreesatstatelevel.Forexample,accordingtooneKI,NigerhaslegalisedtheestablishmentofastateSBMCassociation,whilethisisnotyetthecaseinSokoto.Thedegreethatgovernmentsupportisinstitutionallyembeddedand‘goesbeyondpaper’,asoneKIputit,contributestohowcredibleSBMCsandtheirstateassociationsareinperformingtheirroles.However,thecreationofaninstitutionalenablingenvironmentmaystifleownershiptotheextentthatitimposesactionsnotwantedbytheschoolcommunity,insteadofenablingcommunityself-organisedaction.
CBMCsinIQSsfaceamorechallengingsituationthantheircounterpartsinformalschools.Theircapacitybuildingandempowermentthereforeneedstobemetwithevenmorerealisticexpectationsinregardtograduallyimprovingresults.Ingeneral,KIsconsiderthatCBMCsareabletotakeupsimilarrolesandresponsibilitiesasSBMCsbutthisissubjecttomorefactorsthatneedtobeaccountedfor.Thekeyfactorsmentionedareasfollows:
• TheCBMCneedstobeactuallyestablished.WhileinthecaseofformalschoolsitisplausibletoassumethatSBMCshavebeenestablished,giventheirlegalinstitutionalisationandrelativelylongtrackrecordofsupport,thisisnotthecaseforCBMCs.TheGEP3baselinedatacollectionindicatesthatactualintegrationinGEP3cannotbeassumed:hencetheestablishmentoftheCBMCcanalsonotbeassumed.TheestablishmentofCBMCsisaprerequisiteinorderfortheircapacitytobebuilt.
• TheCBMCneedstobeacceptedbytheproprietor.SinceCBMCsoperateinaprivateschoolenvironment,theiracceptancebytheproprietoriskey.‘SometimestheproprietorsarenothappyastheyseetheCBMCsestablishedencroachingontheirpowersandreducingtheirauthorityintheschools,thoughnotinallcases.Thereisaneedforrealorientationbecauseit’samisunderstandingonthepartoftheproprietors.Ifthisiswellexplainedtothem,theywillgivetheirsupport.’(Non-governmentKI,Sokoto)
• Theproprietor’sacceptanceislikelytodependontheaddedvaluethattheCBMCcanbringtotheschool,particularlyintermsofresourcemobilisation,theproprietor’sacceptanceofintegrationingeneral,andthewaythattheproprietorisinvolvedandhasavoiceinitsfunctioning.OneKIwasscepticalabouttheCBMCs’resourcemobilisationcapacitybeyondthetermoftheGEP3grantsinceoneofthemaindriversforparentstosendtheirchildrentoIQSsispoverty.ThisisimportantbecausetheCBMCs’resourcemobilisationfunctionisrelativelymoreimportantthanisthecaseforSBMCssinceIQSshavelessaccesstogovernmentfunding.
• Theacceptanceofintegrationbythecommunity.TheCBMCcanbebothanimportantsensitisingforceforintegrationwithinthecommunityaswellasbeingaffectedbylowacceptanceofintegrationbythecommunity.Ifthereislowacceptanceofintegrationamong
EvaluationoftheGirl’sEducationProject3
©EDOREN 26
parentsitwillbedifficulttomobiliseresourcesfromthemorhavethemparticipateasCBMCmembers.
• EffectiveCBMCmonitoringdependsontheroleoftheproprietorintheCBMCandonsupply-
sideconditionsbeingmet.SeveralKIsindicatedthattheproprietormaybethechairmanoftheCBMC.ThiswillinfluencethecommunityoversightfunctioninrespectoftheIQSsince‘howcanyoumonitoryourself?’–asoneKIquestioned.Inaddition,CBMCmonitoringisunlikelytobeeffectivewhenfacilitatorsarenotpaidandlearninginputsarenotprovidedbecauseitisdifficulttoholdfacilitatorstoaccountinsuchacontext.
• ThetypeofIQSinvolvedwillaffectthefunctionalityoftheCBMC.IftheIQSisofanomadicnature,thelinkwiththecommunityinwhichittemporarilyresidesmaybeweak,whichwillaffecttheestablishmentoftheCBMC.Similarly,ahighpercentageofboardingpupilsthatcomefromoutsideofthecommunitymayaffectthislink.KIsdidnotseethelatterasaproblem,arguingthatpupilsfromoutsideofthecommunityaregenerallywellacceptedin,andintegratedinto,thecommunity.
2.3.4.2 State-levelinterventionssupportinggovernanceinthestatebasiceducation
system
GEP3supportsseveralinterventionsabovetheschoollevel—mostlyatthestatelevel—toinfluenceorinformpolicy-making,planning,programmingandbudgetingaroundgirls’education.HiLWAseekstoengagewithdecision-makersandschoolcommunities,withtheaimofinfluencingactionstoimprovewomen’sandgirls’participationintheeducationsector.TheGESCispromotedasaforumforcoordination,supervision,advocacyandpoliticalengagementamongabroadgroupofstakeholdersofgirls’education.ThecapacitydevelopmentofstateEMISteamsissupportedtoimprovetheirabilitytoprovidereliableASCdataforGEP3outcomereportingandstateplanning.Inaddition,GEP3teamsandUNICEFstaffprovideassistanceandlobbystateandfederalgovernments.
HiLWA
AprerequisitestepinHiLWA’sToCisthatHiLWAneedstoberecognisedandthatHiLWAisabletoconnectwithstakeholdersatthestateandcommunitylevel.BasedonKIIsinZamfaraandBauchi,thevisibilityandrecognitionofHiLWAseemstovarydependingonwhenHiLWAwasestablishedandwasabletobecomeactive.Forexample,inZamfarastakeholdersaremoreawareofHiLWAcomparedtoBauchiwhereHiLWAbecameactivelateron,in2014.HiLWA’sdegreeofactivityhasbeeninfluencedbyGEP3’sredesignandbytheelections.
IntermsofHiLWA’sabilitytoconnectwithstakeholders,itsinvolvementinenrolmentdrivesand
engagementwithSBMCstructuresandinitiatives42offersanimportantlinktotheschool
communities.ItsengagementwithSBMCspresentsanimportantopportunitybecauseitallowsHiLWAtoconnectwithschoolcommunitiesandMAsbeyondtheannualenrolmentdrives.Thispotentiallyallowsittomorefrequentlyexerciseitrolemodelfunction,aswellastogathercommunity-basedevidenceoneducationissueswithwhichitcanapproachstate-leveldecision-makers.
TheKIsindicatethatHiLWAmembersareinapositiontoconnectwithhigh-leveldecision-makers,althoughHiLWAmembersstatedthatgettinganaudiencewiththesedecision-makersisachallenge.HiLWAmembersvisitedthegovernorinSokotoandthegovernor’swifeinZamfaraandBauchi.InBauchiHiLWAmetwiththeheadofthecivilservicetodiscussgreaterwomen’sparticipationin42HiLWAmembersinZamfaraindicatedthattheyparticipatedinSBMCworkshops.
EvaluationoftheGirl’sEducationProject3
©EDOREN 27
governmentfunctionsattheopportunemomentwhenthegovernmentmadethedecisiontorecruitadditionalteachers.Suchopportunisticactions,i.e.beingonthelookoutforopportunespacesforactionandstakeholderstoinfluence,isagoodapproachinadynamicandcomplexpolicyenvironment.Nonetheless,someimportantpolicyprocesses,likeperiodicplanningandbudgeting,followastructuredcycle.ItisnotclearhowHiLWAaimstobeinvolvedinsuchprocessesbeyondindirectlyliaisingwithstakeholdersthathaveaninfluenceonsuchprocesses.
BeyondtheprerequisitefirststepofstakeholdersbeingawareofandexposedtoHiLWA,factorsthataffectHiLWA’sToCandwhetheritachievesitsresults,areasfollows:
• HiLWA’scredibilityinregardtobeingabletohaveinfluenceandactingasrolemodels.Thisdependsonthecredibilityofitsmembers.TheKIsindicatethatthemembershavehighcredibilitygiventheirformerhigh-levelpositions.43HiLWA’scredibilitywillalsobeinfluencedbytheextenttowhichstakeholdersunderstandHiLWA’sroleandstructure.SomeKIswonderedwhetherHiLWAisanon-governmentalorganisation(NGO)oraproject.Transparencyinterms
ofroleandstructureavoidspotentialconfusion,whichmaydamageHiLWA’scredibility.• HiLWA’scapacitytoinfluence.ThispartlydependsonHiLWA’scredibilityandtheopportunities
forinfluencethatitseizes,butalsoontheextenttowhichHiLWAmembersareinapositiontodirectlyhaveinfluence.KIsindicatedthatitisdifficultforwomenthatarestillinhigh-levelpositionstobecomeHiLWAmembersbecauseofthetimecommitmentrequiredandbecauseHiLWAdoesnotofferadirectbenefit.ThecasesofBauchiandZamfaraindicatethatmemberswhohavebeeninhigh-levelpositionareoftenretiredornotintheirpositionanymore.Therewillbeatrade-offbetweenseekingmemberswithactiveengagementinHiLWA,whichrequiresatimeinvestment,andaimingtoincludememberswithahighinfluencingcapacitybecauseoftheircurrentposition(butwhowillhavelimitedtime).
• ThemotivationoftheHiLWAmembers.KIssuggestthatHiLWAmembersarewellmotivatedtobepartofHiLWA.Motivationwillbeimportantbecause,asoneHiLWAmemberargued,resultswillbe‘aquestionofkeeping[continuing]topush’.Theirmotivationislikelytodependon‘somethingcomingoutofit’,andonmembersfeelingsupportedandrecognised.
• AvailableresourcestosupportHiLWAactivities.ItseemsthatGEP3isseizingtheopportunitytocarryHiLWAmembersalongthroughsynergieswithotherinterventions,suchasenrolmentdrivesorSBMCworkshops.Thisisanefficientuseofresources.However,oneHiLWAmemberraisedresourcelimitationsinregardtotheirlogisticsandoperations(e.g.transportation)asachallenge,andindicatedthattheyareusingtheirpersonalfunding.
• Politicalandinstitutionalenvironment.ThetransitionperiodinrelationtotheelectionshasaffectedHiLWA’sscopeofwork:inparticular,itsgovernmentadvocacyactivities.Inaddition,theinstitutionalenvironmentwillcircumscribeHiLWA’seffectivenessininfluencingwomenparticipation.Forexample,theBauchiHiLWAmembersprovidedtheexampleofthefactthatBauchihasaquota-basedcivilservantemploymentsystemacrossitsregions.Hence,regionswithquotasthatarefilledupwillhavefeweropportunitiestoincreasewomen’semployment.Hence,theexistinginstitutionalisedemploymentpracticesmaylimitHiLWA’sroomformanoeuvreunlesssuchinstitutionsthemselvesarechanged.Finally,thereisariskthatHiLWAwillitselfbeaffectedbypoliticalaffiliation,whichmayundermineitscredibility.
GESC
SimilartoHiLWA,theprerequisitesfortheGESCtosupporteducationgovernanceisforstakeholders43Forexample,inZamfaramembershipincludestheformercommissioner,aformermemberofparliament,andthewifeofaformergovernor.InBauchi,theHiLWAchairanddeputychairareformerpermanentsecretaries.
EvaluationoftheGirl’sEducationProject3
©EDOREN 28
tobeawareofitsexistence,tobeinvitedand,participateintheGESCmeetings,andhaveacommonunderstandingofitsmandate.MostKIsareindeedawareoftheGESCatstateorfederallevel,44althoughsomenon-governmentKIsthataresupposedtobeGESCmembershadnotheardofit.ThemandateoftheGESCisgenerallyunderstoodasexercisingaGEP3oversightfunction,andactingasanadvocatefor,andcoordinating,actionstoimprovegirls’education.TheextenttowhichtheGESCisunderstoodashavingapoliticalengagementmandateislessclear,particularlyinregardtodiscussingandadvocatingaboutfundingchallenges.
InallstatesKIsconfirmthattheGESCisestablishedinthestateandhasbeenactiveinthepast.DuetotheprojectredesignandelectionstheGESChasmetinfrequentlyintherecentpast.45Inseveralstates,GESCseemstohavebeenreactivated,withmeetingsscheduledinAugustandSeptember2015.TherewasnouniformunderstandingamongKIsaboutwhethertheGESChasanannualworkplanbutsomeKIsindicatedthatthiswillbepartofthereactivationoftheGESC.
InBauchi,theGESCformedalobbysubcommitteeduringthepastgovernmentwiththeobjectiveoftakinguppoliticalengagementonfundingissues.Suchanexplicitpoliticalengagementfunctionwaslesspronouncedinotherstates.Inonestate,oneKIarguedthatfundingissueswerenotdiscussedinthecommittee,whileanotherKIindicatedthatstakeholdersarefreetovoicefundingconcerns.
‘ThecommitteehasthePTAchairman,theSBMCchairperson,CSOs[civilsocietyorganisations],womengroups.So,thereareindefinitepeoplethatarealwaysfreetovoiceoutissuesofbudget.Itisnotonlygovernmentofficialssothereisnowayyoucandenythem.Budgetsarediscussedatthecommitteemeetingbecausemembersofvariousagenciesarepresentandmaywantcertaininformationandclarifications.’(Non-governmentKI,Bauchi)
GiventhatreleaseoffundsbystategovernmentshasbeenoneofthemainchallengesofGEP3,the
plausibilityofGESCcontributingtoGEP3’soutcomeswouldbestrengthenedoncemonitoringand
politicalengagementaroundthisissueismoreexplicitlyincorporatedintoitsmandateandis
plannedfor.
TheToCforGESCseemstobethatifawiderangeofstakeholdersperiodicallymeet,discussandcoordinateactionstosupportgirls’education,thenthiswillinfluencegovernmentplanningandbudgetingdecisions,becausestakeholdersaremadeawareoftheissuesandareinapositionofinfluencetoact.Themechanismforactioncouldbethatonceanissueisraised,stakeholdersfeelpeerpressuretoact(externalmotivation),orthatstakeholdershavesufficientownershipthattheyfeelintrinsicallymotivatedtoact.SomeKIs,however,questiongovernmentownershipoftheGESC,indicatingthatitis‘alien’driven,orthattheGEP3teamiskeytomakingthemeetinghappen.LackofgovernmentownershipislikelytoaffectGESC’seffectiveness.AnotherKIpointedoutthattheGESCcouldtriggerademonstrationeffectbyshowcasingideasabouthowtoimprovegirls’educationthatcanbepickedupbythewidergroup.
AkeyfactorthataffectstheGESCToCiswhoisinvitedtobecomeamemberofthecommittee.Themembershipshouldallowforinfluenceandeffectivecoordinationofactions.Sincethekeycentreofpowerforfundingisthegovernor,theextenttowhichmembershaveaccessto,andinfluenceover,thegovernormattersforGESCeffectiveness.Thereislikelytobeatrade-offbetweenhavingahigh-levelpoliticalmembership(likecommissioners)thatpotentiallyhavehighinfluence,butwhose
44TheKIIsdidnotinquireaboutthefederal-levelGESCduringstate-levelinterviews.OnlyoneKIbroughtuptheLGA-levelGESC,buttheinterviewsdidnotexplicitlyinquireaboutthis.45TheinterviewteamwasnotabletoverifywhetherGESCmeetingshavebeentakingplaceandmembershavebeenparticipating,basedonmeetingminutes.
EvaluationoftheGirl’sEducationProject3
©EDOREN 29
commitmenttoattendingregularmeetingsisuncertain,versusamoretechnical,administrativemembershipthathaveamorecertaincommitmenttothemeetingsbutmayhavelessinfluence.Atthispoint,theredoesnotseemtobeacommonunderstandingamongKIsaboutwhoissupposedtochairtheGESC:thecommissionerorthepermanentsecretaryoftheMinistryofEducation.
SeveralKIsappreciatedthemulti-sectorandmulti-stakeholdercharacteroftheGESC.Forexample,oneKIpointedoutthathavingadirectorofaradiocooperationintheGESChelpstogetsupportatthetimeofenrolmentdrives.Finally,theinfluentialroleoftraditionalleaderswasraisedinseveralinterviews.
Besideawareness,degreeofactivity,ownershipandmembershipoftheGESC,otherfactorsthatarelikelytoinfluenceitsToCare:
• Thepoliticalenvironment.Asmentionedbefore,theelectionsandtransitioningovernmenthasaffectedthefunctioningoftheGESC.Furthermore,politicalconflictsmayaffectthefunctioningofthecommittee,whichisaparticularriskwhenitsmembershaveamorepoliticalprofile.OneKIsuggestedthatinthepastpoliticalmisunderstandingsbetweenthegovernorandthecommissionerhaveaffectedGESCactivitiesinZamfara.
• Fundingavailability.TheeventualtranslationofGESCactionsintoactualinfluenceonplanningandbudgetsassumesthatfundingisavailable.SeveralKIsconfirmedthatthegovernorhasdiscretionaryfundingavailablethatrequiresadvocacytoaccess.However,thisenvelopewilldependontheoverallbudgetenvelopethatisavailableviafederaltransferssincemoststateshavelittleinternallygeneratedrevenue.
SupporttotheASC
ThecapacitydevelopmentofstateEMISteamsaimstoimprovetheirabilitytoprovidereliableandregularASCdataforGEP3outcomereportingandstateplanning.AfirststepinthechangepathwayforEMIScapacitydevelopmentisforstakeholderstohaveacommonunderstandingofthecapacityneeds.TheKIsconfirmsomeofthecapacityneedsthatarepresentedintheGEP3strategypaper
onEMIS:inparticular,relatedtocoordination,enumeration,dataverification,andaccessanduseofdatabaseandentrysoftware.TheEMISteamsseemtohaveinvestedstronglyinthedataverificationsystems,whichisanecessaryconditionforthereliabilityofthedata.Itwillbeimportantforthis
systemtobewellinstitutionalisedandresourced,toensuresustainabledataquality.
Whilestrengthenedcapacityinthegenerationofreliabledataisanecessaryfirststep,atalaterstagecapacitytobetterfacilitatetheaccesstoandusageofdataneedstobeconsideredsincetheusageofthedataisrequiredfortheASCtocontributetoplanning,demonstratingresultsorotherpurposes.Ingeneral,KIstakeitasagiventhatonceEMISdatareliabilityisadequate,datacanbeaccessedandwillbeusedinpolicy-makingprocesses.However,thisisnotnecessarilythecase.Theinterfacebetweenknowledgeandpolicyisinfluencedbyfactorssuchasthepoliticalcontext,theinterestoftheactorsinvolvedindataproductionandpolicy-making,andtheimplementationofintermediaryfunctionstostrengthentheknowledge-policyinterface.46
AccesstotheASCdataisaconditionforusageofdata.Government-levelaccessisfacilitatedby
havingStateMinistryofEducation(SMoE)andSUBEBteamscollaborateonthestateEMISunitsandbyhavingavarietyofstakeholdersrepresentedonthestateEMIScommittees.TheinvolvementofawiderangeofstakeholdersontheEMIScommitteemaycontributetotheownership,andsubsequentusage,ofthedata.Accessbynon-governmentactors,whichcan46Jonesetal.(2013).
EvaluationoftheGirl’sEducationProject3
©EDOREN 30
contributetosustainabledemandfordataandmonitoringoftheeducationsector,officiallyreliesonformaldatarequeststothePermanentSecretaryoftheSMoE.Thisprocessdoesnotseemtoworkeffectively:EDORENsentaformalrequestforthe2013ASCreportanddatatoallGEP3SMoEs47attheendofSeptember2015andnoresponsewasreceived.TheGEP3strategypaperonEMIScapacitybuilding(UNICEF2015c)indicatesthatallannualASCreportswillbepublishedandprintedandcopiesplacedinthepublicdomain.AccordingtooneKI,ASCdataarealsodisseminatedtothelocalgovernments,andthemediaareinvolvedindisseminatingthemtoawideraudience.48TheextenttowhichASCdataareaccessibleattheschoollevelisnotclear,butKIsindicatethatschoolskeepacopyoftheASCquestionnaire,whichgivesthemprimaryaccesstothedata.However,thisdoesnotallowthemtocomparetheirschool’ssituationwiththatofotherschools.Oneoftheshort-termobjectivesofGEP3supporttoEMIS,accordingtotheGEP3StrategyPaper(UNICEF2015c),istoensurethatEMISdatafeedbacktoLGAsandschoolstoinformplansinsupportofgirls’educationandmonitoringprogress.
KIsindicatethatASCdataareusedforplanningpurposes,suchasStateEducationSectorOperationalPlans(SESOPs).OnegovernmentKIalsopointedoutthatthedatacouldbeusefulfortargetingenrolmentdrivestoLGAswiththehighestgirl’senrolmentgaps.OtherusagesmentionedbytheKIsareplanningandmonitoringfordonor-supportedprojects,assessmentsbyspecialcommittees49orforresearchpurposes.GEP3directlyinfluencestheuseofEMISdatabyincludingthedatainitsdonorreporting.
TheGEP3StrategyPaper(UNICEF2015c)alsoreferstotheEMISdatabeingmadeavailableforinclusioninthepreparationofthestateMedium-TermSectorStrategy(MTSS)andAnnualSectorPerformanceReview.Ingeneral,nospecificactionsseemtobeforeseenaspartofGEP3’sstrategytoactivelyfacilitatetheuseofEMISdata.Whilethisisconsistentwiththefactthatthedatafirstneedtobeimprovedintermsofreliabilityandtimeliness,beforeemphasisinguse,andKIIsseemtoindicatethatusageistakingplace,itwouldbeworthexaminingwhetherthedemandsideofthe
EMISneedsstrengtheningintheGEP3ToCinorderachieveitseffectiveuseinevidence-based
planning,resourceallocationandpolicyorientationtoimprovegirls’education.
GEP3seekstoimprovegirls’representationineducationdatabyincludingIQSdatainEMIS.ThisisplausibleforthoseIQSsthatarewellalignedwiththeformaleducationsystemviaSUBEBs(theseIQSsarealreadyincludedintheEMIS),butdoesnotseemplausibleatthismomentforthemore
informalIQSsthatgenerallyfallunderSAME’sresponsibility.WhileUNICEFhasclarifiedthatintegratingIQSdataintoEMISdoesnotnecessarymeanintegratingthesedatawithformalschooldata,basicrequirementsforqualityeducationdatacollectionarenotyetinplace.AreliableIQSlistingwouldberequiredasastartingpoint,beforedatacollectioninmostIQSscantakeplace–particularlythosemanagedbySAME.TheGEP3baselineexperienceindicatesthatinformationontheexistenceofIQSsandtheirintegrationstatusisincomplete.Therealsoseemtobefluctuationsinintegrationstatus,whichwouldrequirecontinuousupdatingofthislisting.Furthermore,ifenrolmentdataareadesiredindicator(whichislikely)acensuswouldhavetocapturethewidevarietyofenrolmentsituations50acrossavarietyofgradecategorisationspresentinIQSs.Also,ifthe
47TherequestwasaddressedtothePermanentSecretary,withacopysenttotheDirectorofPlanningResearchandStatistics.LetterdeliverywasconfirmedbyDHL.48TheZamfaraEMISunitindicatedthatmediaannouncementswereaired,tointroducetheASCbeforeandduringtheconductingoftheASC.Equally,duringdisseminationtheyinvitethemediatobroadcastabouttheASC.49TheZamfaraEMISunitprovidedtheexampleofaSecondarySchoolAssessmentCommitteethatwasassignedtolookatthesecondaryschooleducationinthestateandtoadviseonhowitcouldbeimproved,byrequestingtheASCdata.50RARAresearchonIQSsencounteredawidevarietyofenrolmentsituations:pupilsjustenrolledinQur’anicclasses,pupilsenrolledinQur'anicclassesandintegratedclasses,pupilsenrolledinQur’anicclasses,integratedclassesandformal
EvaluationoftheGirl’sEducationProject3
©EDOREN 31
dataneedtobeusedtomeasureadditionalaccesstoformaleducationthedatacollectionneedstotakeintoaccountpossibledoublecountingwiththeformaleducationsectorwhenpupilsattendbothformalandnon-formalschools.KIspointoutthatdatacollectionformscouldnotbesimplytransferredfromtheformalschoolASCsincetheywouldneedtobeadjustedtothegradeclassificationandcontextofIQSs.51GiventhecapacityconstraintsfacedbySAMEitisunlikelythattheycanimplementthisrigorouslyatthemoment.
FactorsthatmayaffectthelikelihoodofGEP3supporttotheASCresultinginimprovedproductionanduseofreliableASCdataareasfollows:
• ResourceavailabilityforASCandEMISunitsforsustainableoperationsandcontinuedcapacitybuilding.Staffcapacitiesarestillbeingbuiltandfacilities,suchascomputers,arelimited.KIsconfirmedthatreleaseofgovernmentfundsforASChasbeendifficult.
• MotivationoftheEMISunitstaff.Dataqualitydependsonthemeticulousimplementationofdatacollection,verificationandmanagementprotocols.Wellmotivatedstaffareimportant,andthisinturndependsonseveralfactors,includingappreciationofworkdone,thefacilitiesavailabletodotheworkandtheremunerationthattheyreceive.Duringtheinterviews,EMISstaffvoicedconcernsaboutthelackofmodernhardware/softwaretocarryouttheworkanddelaysinpayment,whichtheyindicateddiscouragesstaff.
• Qualityofthedataentryandmanagementsoftware.EMISstaffhighlightedthesoftwareusedasoneofthemajorchallengestotheirworkinthepast.ThestaffarepositiveaboutthesoftwarerecentlyintroducedbyUNICEF,althoughitsintroductioncreateddelays.
• Anenablingschoolenvironmentandsystem.Poorqualityrecord-keepingatschoollevelwilltranslateintopoorqualitydatabeingrecordedintheASCquestionnaires.WhiletrainedenumeratorsareresponsiblefortheASCdatacollection,theyrelyontheheadteachersandtheschoolrecordstoprovidesomeofthedata,suchaspupildata.Hence,oneEMISKIpointedoutthatthoseattheschoollevelconcernedwithrecord-keepingneedtobetrainedandsensitisedtokeepqualityrecords.TheKIsalsomentionedthatshiftsintheacademiccalendarcancreateplanningproblemsfortheASC.Additionally,insecurityorclimateincidencesmayinhibitaccesstotheschools.
• Interestofgovernmentstaffandpolicy-makersinusingEMISdatainpolicy-makingprocesses.EMISdatamaybeignoredinpolicy-makingprocessesevenwhenthedataarereliableandtimely.OneKIpointedoutthatpolicy-makersneedtobesensitisedontheuseofEMISdatainplanninganddecision-making.
2.3.5 Pilot-to-scale-upapproach
Inordertoreachlargenumbersofgirls,GEP3followsapilot-to-scale-upapproach.GEP3’sToCindicatesthatakeyassumptionunderlyingtheoverallprojectapproachisthatstategovernmentsarewillingandabletoimplementlarge-scalechangesforgirls’education,oncerelevantandcompellingevidenceofthevalueofnewapproachesisclear.TheGEP3logframeconcretisesthescale-upambitionbyformulatingtargetsforprimaryschoolsandIQSstobereachedduringscale-upandadecreasingratioofprojectfinancingcomingfromdonorfundingcomparedtothestategovernmentcontribution.
schools,andpupilsjustenrolledinintegratedclassesinIQSs.Pupils’attendanceisalsonotassumedtobeasregularasinformalschools,withpupilsnotattendingforextendedperiodsoftimewithoutbeingconsidereddropouts.51Forexample,teachinginanIQSmaynottakeplaceintraditionalclassrooms,whichwouldrequireadjustinghowinfrastructureiscapturedintheASCform.
EvaluationoftheGirl’sEducationProject3
©EDOREN 32
Abasicprerequisiteforthepilot-to-scale-upassumptiontoholdisforstategovernmentstobeawareoftheexpectationtoscale-up.WhileKIsaregenerallyawareofthescale-upobjective,there
was,atthetimeoftheinterviews,noclearunderstandingaboutwhatscale-upwouldlooklike,
andonlylimitedplanninghappeningforit.AsoneKIputit,‘thediscussionofscale-upisafloat’.GovernmentKIscouldnotprovideinformationabouthowmanyschoolsareexpectedtobereachedduringscale-up,despitetargetsbeingincludedinthelogframe.OnlyfewKIshadconcreteideasabouthowscale-upcouldbestbeimplemented.Forexample,onegovernmentKIsuggestedthatinterventionsshouldfirstbeexpandedtomoreLGAs,withlimitedschoolsbeingtargetedperLGA(asisthecaseduringthepilotphase)because‘othercommunitiescanwitnessinterventionsasapilot’.AnotherKIarguedforadditionalsupporttoberesult-based,makingthecasethatadditionalSBMCsupportshouldbelinkedtoresults.
StateGEP3teamsandUNICEFstaffinterviewedrecognisedthatplanningforscale-upisimportant
andneedstostartwellaheadof2017,takingintoaccountthetimelinesofplanningandbudgetingprocesses.Scale-upplans,targetsandbudgetsneedtobeincorporatedintostateplanningprocesses,suchasSESOPsand/orMTSSs.SomeKIsindicatethatscale-upplanningisalreadytakingplaceandisbeingincorporatedintothereviewoftheMTSS.Similarly,scale-upneedstobetakenintoaccountinannualstatebudgetprocesses.52OneKIarguedthatthefoundationsofscale-upneedtobeintegratedintothe2016budget,ratherthanscale-upfundingbeingincludedonlyinthe2017budget,whenscale-upisscheduledtostart.Forexample,investmentsintheeducationsupplysideshouldbeintensifiedin2016,soastobeabletoaccommodatethescale-upofGEP3interventionsfrom2017onwards.
‘IfGEP3wouldwanttoscaleupin2017,theprojectwouldneedtoengageinApril2016whentheaggregatestateresourceenvelopestarttobedrafted.Itisgoodtobe‘around’whenthishappens.AtminimumGEP3shouldgetinvolvedwhenannualbudgetcallcircularisissuedtosectors.SectorbudgetsshouldbefinishedbySeptember.However,budgetcyclerarelyhappensasplanned.’(Non-governmentKI,National)
InlinewithEDOREN’sPEAstudy(2014),KIsconfirm,however,thatincorporatingscale-upprojectionsinplanningandbudgetprocessesisnotasufficientconditionforfundingbeingavailableforscale-up,sincethereleaseoffundsdoesnotnecessarilyfollowbudgets.Nonetheless,whilenotasufficientcondition,integratingscale-upinsectorplansandbudgetsisasupportingconditionandmakesscale-upmoreplausible.
FundingavailabilityisconsideredbymostKIstobethemainfactorconditioningGEP3scale-up.Atthetimeoftheinterviews,KIshadnoclearunderstandingabouthowthescale-upwouldbefunded.SomegovernmentKIsarguethatfundingwillbeavailablefromthestatebudget,andsupporttheirclaimbymakingreferencetointerventionsalreadybeingscaleduporsustainedwithstatesupport,suchasenrolmentdrivesorFTTSS.53ThechangeingovernmentthathastakenplacehasalsomadesomeKIsoptimisticthatfundingforscale-upwillbeavailable,asgovernorshaveexpressedaninterestineducation.However,otherKIsexpresseddoubtsaboutfundingthatreliesstronglyonstategovernment.Theyforeseethatstatebudgetswillbeconstrainedasincreasingfundingneedswillemergewithagrowingschoolpopulation,whilestateresourcesremainleanand
52OneKIpointedoutthatforscale-upfundingtobeincludedinthe2017budget,theprojectwouldneedtoengagewiththebudgetprocessinApril2016,whentheaggregatestateresourceenvelopestartstobedrafted.Anothercriticalmomentofengagementiswhentheannualbudgetcallcircularisissuedtothesectorministries.SectorbudgetsaretobecompletedbytheSeptemberbeforethebudgetyear.53 Government partners in Katsina and Niger indicate that the FTTSS has continuedwith state support, despite GEP3’sfinancialsupportbeingstopped.
EvaluationoftheGirl’sEducationProject3
©EDOREN 33
highlydependentonfederalsubvention.
‘Letmespeakonanoptimisticnote.Wemaynotsaythatit[thescale-up]isnotimplementableandmaynotbepositive.Itispositivebecauseitisagoodintention.However,ifyoulookatglobalprojections,particularlyforoilwhichisthemainstayofoureconomyinNiger,itisnottoogoodfor2016upto2017.However,onanotherpositivelight,Nigerstatehasbeencollaboratingwithotherinstitutionstoimproveagriculturevaluechain.Ifwedothatatleastthestate’sGDPwillvastlyincreaseandthenourcapacitytoimplementprogrammeswillalsoimprove.”(GovernmentKI,Niger)
Somepotentialfundingfromnon-statesourcesforscale-upareproposed,suchasfundingfromtheGlobalPartnershipforEducation(GPE)inSokotoandKatsina,orUBEC-matchedgrantfunding.SomeKIsexpectalternativenon-governmentfundingsourcestobemobilised:forexample,mobilisationofcommunityresourcesviaSBMCs,contributionsbyphilanthropists,orspecialstatefunds,liketheSokotoEducationDevelopmentFund.
MostKIsseethereleaseoffundsasbeingdependentonthepoliticalwillofthegovernor.Factorsthatinfluencethispoliticalwillare(accordingtoKIs):high-leveladvocacy,governmentownershipofandinvolvementintheproject,alignmentbetweentheprojectandgovernmentpolicies/structures,andevidenceabouttheresultsoftheinterventions.However,afewKIsdidnotbuyintothisargument,arguingthatbudgetreleaseisnotcausedbylackofpoliticalwillbutsimplybecause
resourcesarenotavailable.
‘Thedebateaboutwhybudgetsarenotreleasedisthewrongdebate.Youcannotimplementifyoudon’thavetheresourcesinthefirstplace.Theonlysolutionistodoproperbudgeting.’(Non-governmentKI,National)
WiththesupportoftheUNICEFfieldofficeeducationteams,GEP3isengaginginhigh-leveladvocacytoinfluenceplanningandbudgetingforgirls’educationatthestatelevel.Therecentelectionsrequirereinvestinginadvocacyinrelationtothegovernor,whichalsooffersan
opportunity,sincestateexecutivesaresettingouttheirpolicypriorities.Atthetimeoftheinterviews,UNICEF,togetherwithGEP3stakeholders,hadpaidcourtesyvisitsinSokoto,NigerandBauchi.KIsindicatethattheroleofexternalactors(suchasdonors),traditionalrulers,andSBMCscanbeparticularlyeffectiveforadvocacy.GESCandHiLWAcanalsoactasadvocacyplatforms(seeabove).TheKIssuggestthatpolicy-makersareresponsivetoevidenceintheirdecision-making.Inthisregard,GEP3’sToCisconfrontedwithachallengeintermsofthetimingofevidencebecoming
availableversusthetimingregardingwhentheevidencemaybeneeded.Asdiscussedabove,planningandbudgetingforscale-upwillneedtostartbefore2017,whilecertainitemsofevidence,liketheEDORENevaluationresults,willonlybecomeavailablein2017.
Theplausibilityofscale-upvariesdependingontheinterventionsinvolvedandthestatecontextualfactorsrelatedtotheseinterventions.Scale-upofteacherandSBMCcapacitybuildinginterventionsarerelativelymoreplausiblebecausetheneedfortheseinterventionsiswellrecognisedandacceptedascentraltoimprovedschooling.Thereisaninstitutionalandlegalframeworkthatsupportstheinterventions,andKIscanpointtospecificfundingsourcesthatcouldfinancescale-up–thatis,UBECfundingandGPE.Statecontextualfactorsneedtobetakenintoaccountaswell.Forexample,scale-upoftheGEP3earlylearninginterventioninKatsinawilldepend
EvaluationoftheGirl’sEducationProject3
©EDOREN 34
onthestate’scommitmenttoasimilarJollyPhonicsearlylearningintervention.54
Similarly,enrolmentdrivesarewellacceptedandsupported.Sinceenrolmentdrivesalreadycovertheentirestate,scale-upislikelytoexpressitselfinthewayenrolmentdriversareimplemented,ratherthanexpandingtheirreachtomoreschoolcommunities.Twofactorsmayinfluencegovernmentcontinuingwithstate-wideenrolmentdrives:first,ifthegovernmentcannotinvestsufficientlyinteachersandotherschoolsupply-sideelementstomeettheincreaseddemandcreatedthroughtheenrolmentdrives,governmentmayscaledowntheintensityoftheenrolmentdrivesorfocusthemwheresupplycanmeetdemand;second,theneedforenrolmentdrivesmaydecreaseinareaswhereawarenessaboutgirls’educationissufficientlyraisedandthenumberofout-of-schoolgirlsislow.
Totheextentthatthecashtransferprogrammecannotbesustainedbyotherdonorprojects(liketheGPE),thecashtransferprogrammeisalessplausiblecandidateforscale-upbecauseitiscapital
intensive,isnotwellinstitutionalised,andisseenasadonorpriority.Cashtransferswouldrequireconsiderablecapitalexpenditurefromthestate,iffundedthroughthestatebudget.Atitscurrentscale,thecashtransferscorrespondtoapproximately13percentoftheaverage2009–2013actualannualcapitalexpenditureinNiger,55whilethe10,700girlscurrentlybenefittingrepresentaround4percentofthefemaleprimaryschoolpopulation,whichisonlypartofthecashtransfertargetpopulation.56Evenalimitedexpansionofthetargetpopulationwouldrequireaconsiderableproportionofthestatecapitalexpenditures.Statecapitalbudgetsarehighlydependentonfederalallocations,andarelesslikelytobereleasedcomparedtorecurrentexpenditures.57Sincefederalallocationsaredependentonoilrevenues,thegovernment-fundedscale-upofthecashtransferwouldessentiallybeafunctionoftheoilprices,asoneKIputit.Lowoilpricesthereforemakethescale-upofthecashtransferprogrammelessplausible.
‘Theinterventionthatisgoingtobealittlebitdifficulttoscaleupisthecashtransferprogrammebecauseitiscapitalintensive.ThestatemadeacommitmentatthecommencementoftheprojectofcounterfundingofNGN21,000,000to12,500students.Untiltodaythismoneyhasnotbeenreleased.Thisisbecauseofthelimitedresourcesatthedisposalofgovernment.Ifgovernmenthasnotbeenabletoredeemitscommitmentforthefirstphaseuptillnow,itwillbeveryunwiseforonetotalkaboutscale-upofthatproject.’(GovernmentKI,Niger)
Inaddition,comparedtoSBMCsandteachercapacitydevelopment,cashtransfersarenotbasedonanestablishedinstitutionalorpolicyframework,whichmakesthemmoresensitivetoadhocpolicychanges.AccordingtoKIs,acombinationofthesefactorshascontributedtothecashtransferprogrammesinKatsinaandBauchibeinghalted.Finally,oneKIalsoquestionedwhetherscalingupdemand-sideinterventionslikecashtransfersmakessensewhensupply-sideinvestmentscannotkeepup.
Thescale-uptheIQSsupportisparticularlyuncertain.WhileKIsaregenerallysupportiveofthe
54JollyPhonicsiscurrentlyimplementinganearlylearningintervention,withthesupportoftheSUBEB,acrossZamfara.ItstartsimplementationinKatsinainthe2015–2016academicyear.55Weestimatethecapitalexpenditurecostofthecashtransfer(i.e.thecostofthebeneficiarytransferpayments)atNGN214millionperyearforonestate(NGN5,000xfourquartersx10,700beneficiaries).Theaverageactualannualcapitalexpenditureintheperiod2008–2013wasapproximatelyNGN1.7billion(source:SPARCPFMdatabase).56Inthe2014–15schoolyear275,359girlswereenrolledinprimaryschoolinNiger(ASC2014–2015).Thecashtransferinterventiontargetsallgirlsbetweenfiveand16yearsofage.57TheSPARCPFMdatabasedemonstratesthatintheperiod2009–2013theratiobetweenactualcapitalexpendituretobudgetfluctuatedbetween3%and89%,withanaverageof48%,whiletheratiobetweenactualrecurrentexpendituretobudgetonlyfluctuatedslightlyarounda92percentaveragefigure.
EvaluationoftheGirl’sEducationProject3
©EDOREN 35
integrationofbasiceducationinQur’anicschools,theinterviewsindicatethatscale-upislikelytobeaffectedbytheinstitutionalcontextofintegration,SAME’saccesstoresources,thenumberofwell-establishedIQSsavailableforscale-up,andsupply-sideconstraints–particularlywithregardtofacilitators.Withregardtotheinstitutionalcontext,KIsdonothaveacommonunderstandingabout
whichinstitutionisresponsibleforintegration.SomeKIsarguethatSUBEBisresponsible,giventheirmandatetoprovideuniversalbasiceducationtoallchildren,whileothersconsiderSAMEtoberesponsiblebecauseofitsmandateregardingnon-formaleducation.ThiscreatesadispersedlandscapeofIQSsmanagedbydifferentagencies58thatischaracterisedbylackofcoordination.Thiswillmakescale-upinstitutionallyandpoliticallycomplex.
‘DifferentactorsworkintheIQSsector:SAME,ArabicandIslamicEducationBoardandSUBEB.Allhaveanelementofthisactivitygoingoninschools.Thereshouldbeacomingtogethersoyoucanscale-upwithinalltheseschools.Thereisaneedforacollaborativepartnership.’(GovernmentKI,Sokoto)
Furthermore,SAME,whichmanagesthelargestnumberofIQSs,hasrelativelylimitedanduncertainaccesstofunding.ThemainfederalUBECfundingsourceisnotaccessibletoSAME.Furthermore,SAMEKIsindicatethattheirstatefundingisuncertainfromoneyeartoanother.Hence,scale-upfundingforIQSsupportisthereforehighlyunclearifscale-upismeanttorelyonSAME-managedIQS.
Inaddition,theGEP3logframehassetthetargetforIQSscale-upat500IQSsperstate,butatthetimeoftheinterviewthisnumberoftargetIQSsdoesnotseemtobeinplaceinallstates,atleastwhenthecurrentIQSeligibilitycriteriaaremaintained.59Evenwherethenumbersareavailableonpaper,itisuncertainwhethertheIQSsarewellintegratedinpractice.Selectingandsustainablyfundingfacilitatorsatscaleinlinewithagreedstandards(inparticularly,astipendofNGN7,500)alsoseemsunlikely.KIsindicatethatitisalreadychallengingtoselectqualifiedfacilitatorsinagreementwiththeproprietoratlimitedscale;andpaymentoffacilitatorstipendshasbeenhighlydependentonuncertainbudgetreleases.
2.4 AppropriatenessofGEP3’simplementationstrategy
BesidestheplausibilityofitsToC,therelevanceofaprojectisafunctionoftheappropriatenessofitsimplementationstrategyforthecontextinwhichitoperates.AnappropriateimplementationstrategyisaprerequisiteforthecausalpathwaysintheToCtooccur.Aspartofthisassessment,weconsiderthefollowingdimensionsofimplementationappropriateness:
• Involvementofstakeholders,particularlygovernmentstaff,intheimplementationprocess;• Capacitytoimplementtheprojectinaqualitymanner;and• Equityinthereachofthetargetpopulation.
ThisassessmenthasbeenconstrainedbytheoperationalchangesaspartoftheprojectredesignatthetimeoftheKIIs.Theoperationalisationofsomeinterventions,suchastheearlylearning
58Furthermore,insomestatesadditionalagencieshavebeencreatedthatalsoplayaroleinintegration,liketheIslamicEducationBoardinSokoto,andtheOfficeoftheDirectorGeneralforTsangayainBauchi.59InBauchi,SUBEBsupports526centresregisteredforintegration,whileBauchiSAME(BASAME)manages175centres.MostofthesecentresareoftheTsangayatype,withattendancemostlybyboys(source:SUBEBandBASAME).InNiger,SAMEmanages450IQSs,whileSUBEBmanagesalimitednumberofmodelIQTEs(source:SAMEandSUBEB).InKatsinaSAMEsupportsaround600IQSs,someofwhichtargetAlmajiri,whicharemostlyitinerantboys(source:SAME).InSokoto,SAMEhasabout630IQSs,whileSUBEBmanagesninecentres(source:SAME)
EvaluationoftheGirl’sEducationProject3
©EDOREN 36
interventions,hadyettobedefined.TheGEP3stateteamshadrecentlybeenrestructured,whichaffectedtheirunderstandingoftheoperationaldetailsoftheGEP3strategyintheirrespectivestates.
2.4.1 Stakeholderinvolvement
Stakeholderinvolvementinprojectdesign,decision-making,planningandimplementationcontributestoownership,whichinturnincreasesthelikelihoodofsoundoperationalperformanceandsustainabilityofaproject.GEP3’soperationalplansupportsstakeholderownershipbyemphasisingimplementationwithandthroughstateeducationpartners.Furthermore,byinvolvingstateactorstheircapacitiescanbebuilt.
Overall,KIsvaluethefactthatGEP3operatesfromwithinstategovernments,whichisconsideredtocontributetogovernmentinvolvementandcoordinationbetweentheGEP3stateteamandgovernmentstaff.
‘GEP3isdesignedinaverygoodformtobeproperlyimplementedwithcooperationofthestategovernment.Itisimportantthatitissupportedbygovernment.WithGEPoperatingundergovernmentroofyouarecarryinggovernmentalong.YoucaneasilygetaholdofPermSec.Governmentcapacityintermsofplanningisalsobuilt,forexample,creatingworkplans.’(Non-governmentKI,Sokoto)
GovernmentpartnerswereinvolvedintheGEP3projectredesignandtheGESCisdesignedasaspacetocontinuetheirengagementinprojectplanning,deliberationanddecision-making.FocalpersonsanddeskofficersatpartnerMDAsareappointedtocoordinateandsuperviseimplementation.Furthermore,thefocalpersonisasignatoryontheprojectaccount.Hence,itisforeseenthatstructuresandsystemsenablegovernmentinvolvement.
Projectownershipandimplementation,aswellastheextenttowhichstructuresandsystemsareactuallyoperational,areinfluencednotonlybywhetherbutalsobyhowgovernmentpartnersareinvolved.Governmentinvolvementistosomeextentexternallyratherthaninternallydriven,withKIsqualifyingGEP3’sgovernmentinvolvementas‘carryinggovernmentalong’,andstatingthat‘involvementhappenstotheextentthatleadershipissensitised’.Also,asmentionedabove,KIsquestiongovernmentownershipoftheGESC.Operationalisationofstructuresandsystemsthat
facilitategovernmentinvolvementareinfluencedbychangesinstaffingandthepoliticalcontext.Forexample,therestructuringoftheGEP3stateteamsaspartoftheredesignandnewgovernmentappointmentsasaresultofthe2015electionshaveinfluencedtheactivitiesoftheGESC.Theelectionsalsoaffectedtheoperationalisationofthecashtransferprogrammeimplementationunitsasgovernmentoperationalsupportwasnotprovidedasplannedduetothetransitioningovernment.
GovernmentinvolvementandownershipislikelytovaryacrossGEP3interventions.Theenrolment
drivescancountonbroadstakeholderengagementandenthusiasm,alongtrackrecordofgovernmentimplementation,andanannualprocessofplanningandlearningfrompastaction.KIshave,overall,agoodunderstandingoftheoperationalprocessesoftheenrolmentdrives.
‘TheUNICEFapproachofinvolvingallstakeholdersfromgrassrootsleveltotoptopartakeinenrolmentdrivehasshowntremendouswillingnessofalltosupportthisintervention.’(GovernmentKI,Sokoto)
EvaluationoftheGirl’sEducationProject3
©EDOREN 37
Similarly,theoperationalprocessesoftheASCandcashtransferprogramme,supportedbyjointSUBEB–SMoEimplementationunits,arecommonlyunderstood.Whileexternalconsultantshaveprovidedsupportduringdesignandimplementation,governmentstaffhavebeencloselyinvolvedbothinthedesignandinleadingtheimplementation.
Thedesignandplanningofotherinterventionsislesscoordinatedwithgovernmentpartners.TheoperationalisationofGEP3’ssupporttoIQSslacksclarityandcoordination.KIsworkingwithIQSswerenotveryawareoftheplanningofIQSfacilitatortrainingatthetimeoftheinterviews,eventhoughthiswasscheduledforwithinonemonthoftheinterviews.ThebaselinesurveysuggeststhatGEP3-focusIQSsarenotwellidentified.60InBauchi,stakeholdersdonothaveacommonunderstandingofhowmanyGEP3-focusIQSaremanagedbySUBEBandhowmanybytheBauchiStateAgencyforMassEducation(BASAME).Weakmonitoringandinstitutionaluncertaintyabouttherolesandresponsibilitiesforintegrationarelikelytobefactorscontributingtothissituation.TheprovisionofgrantsisanotherinterventionthatKIsareuncertainabout,assomedonotseemtoknowwhethergrantswillstillbeprovided.Also,withregardstoSchool-basedTeacherDevelopment,severalKIsinBauchiwerenotawarethatanewapproachisconsidered,ortheyindicatedthattheywerewaitingforDFIDtoprovidethemwitharedesignedapproach.
‘LastyearinOctoberwewereinvitedtoaone-daystakeholders’meetinghereinBauchiwithtwoconsultantsfromDFIDandtalkedabouttheredesignedGEP3andthefocusoftheredesign.Theconsultantstoldusthatschool-basedteacherdevelopmentwillnolongeranswertothatnamebecauseofotherelementsofteachertraining;sowearenowwaitingforDFIDtogiveustherevisedortheredesignedapproachwhichisyettobeavailabletous.’(GovernmentKI,Bauchi)
Atthetimeoftheinterviews,KIsinKatsinaandZamfarahadlittleinformationaboutthe
implementationoftheearlylearninginterventionorG4G,beyondthefactsuchinterventionswereplannedfor.Governmentinvolvementintheimplementationoftheearlylearninginterventionwillbeparticularlyimportantbecauseasimilarintervention,JollyPhonics,hasalreadybeenrolledoutacrossZamfaraandislikelytoberolledoutacrossKatsinain2016.
KIsconsidertheinvolvementofabroadrangeofstakeholdersatdifferentlevels(community,local,
stateandnationallevel)astrengthofGEP3’simplementationstrategy,inparticularthe
participationofSBMCsandMAs.TheempowermentofSBMCstatestructuresisapromisingmechanismforfacilitatingcontinuousandgraduallearning,andforprovidingvoiceandasenseofself-efficacytotheSBMCsandMAs.NGOsandCSOsarealsoinvolvedinGEP3implementation–inparticularinSBMCcapacitybuildingandmonitoring.However,KIIsindicatethatatlocalandstateleveltheirinvolvementisproject-driven,focusingmostlyonservicedeliveryandperiodicrepresentationinmeetingswheninvited,ratherthanstructurallystrengtheningthevoiceofcivilsocietyingirls’education.
Governmentinvolvementismeanttostrengthengovernmentcapacity.Thisisparticularlyrelevant
inthecaseofmonitoring.AlthoughGEP3’smonitoringfunctioninvolvesbothgovernmentaswellasnon-governmentpartners,itsrelianceongovernmentcapacityissubstantial.Thisisarisksincegovernmentmonitoringcapacityisconsideredweak(seebelow).Nonetheless,government
involvementinGEP3monitoringisrightlyconsideredimportant.Notonlydoesitofferanopportunitytobuildmonitoringcapacity,italsoenablesgovernmentstafftodirectlyobservethe
performanceoftheinterventions.Inorderforthistotranslateintolearningandimprovement,thespacesandprocessesneedtobeinplaceforgovernmentstafftoreflectonandaccountforresults.6020%oftheGEP3-focusIQSscontactedaspartofthebaselinesurveywerefoundtobenon-integrated,non-existingornottohavegirlsaspupils.
EvaluationoftheGirl’sEducationProject3
©EDOREN 38
Itseemsthatforenrolmentdrivestheannualstatecoordinationmeetingsoffersuchaspace.KIsindicatethatthefutureheadteacherandSBMCtrainingwillalsobeadaptedbasedonassessmentsandmonitoringresults.Thisagainoffersanopportunityforinvolvinggovernmentstaffinmonitoring,aswellasinjointreflectiononresults.
2.4.2 Governmentcapacity
Workingthroughgovernmentsystemsofferstheopportunitytobuildcapacitythroughaction,aspartofGEP3’sobjectivestoimprovegovernanceandachievesustainabilityandscale.However,italsoexposestheprojecttooperationalrisksrelatedtoweaknessesininstitutional,organisationalandindividualcapacitieswithingovernment.
‘Aweaknessforworkingthroughgovernmentmightbeinthebureaucraticnaturewhereyoucannotjettisonbureaucracy.Thestrengthofworkingthroughgovernmentisthatyouarebuildingtheinstitutionalandhumancapacity,whichleadstosustainability.’(Non-governmentKI,Niger)
KIsindicatethatindividualcapacityintermsofpeople’sskillsandknowledgeisweakeratcommunityandlocallevelcomparedtostatelevel.Asmentionedbefore,someSBMCmembersareilliterate,whichneedstobetakenintoaccountinthetasksrequiredfromthem.KIsalsosignalthatlocal-leveldeskofficersdonothavetherequiredcapacityyettocarryoutalltasksrequiredofthem,althoughsomearguethatthisisnotduetoskilllevelsbutratherbecauseoftheresourcesavailable.Regularfollow-upaspartofcapacitybuildingishighlightedbymanyKIsaskeybutthisisespeciallyaffectedbychallengesinresourceavailability.Aneffectivesystemofidentifyingwhereindividualcapacityneedsarehighestisthereforeessentialinordertobeabletoprioritiselimitedresources.TheSBMCEffectivenessMonitoringofferssuchasystematSBMClevel,totheextentthatitcapturesallrolesandresponsibilitiesrequiredoftheSBMC,suchastheirroleinthecashtransferprogramme.GEP3’sOperationalPlanalsoforeseestheestablishmentofacapacitydevelopmentreportingsystemthataimstoidentifycapacitydevelopmentneedsatLGEAlevel.Atthetimeoftheinterviews,theextenttowhichthiswasdevelopedwasnotyetclear.
IndividualcapacitybuildingismorelikelytobesuccessfulatstatelevelsinceGEP3isabletomore
directlyprovidesupportandmobilisespecifictechnicalexpertise.Forexample,theinterviewedmembersofthestatecashtransferPIUwerepositiveaboutthecapacitybuildingprovidedbytheexternalconsultantsduringtheinterventiondesign.The2015GEP3AnnualReviewpointstoimprovementinstateEMIScapacityduetomorepurposefultechnicalinputs,theinvolvementoftheGEP3M&Eleadandexternalpartnerships.TeacherFacilitatorsandMasterTrainersimplementingteacherdevelopmentaresourcedfromgovernmentandtheCollegesofEducation.KIsindicatethatthisisastrengthbecauseitensuresthattheyhaverelevantexperience.However,theTDPbaselineevaluation(Deetal.,2015)indicatesthatalthoughTeacherFacilitatorshavestrongbackgroundsinclassroomteachingandschooladministrationtheyhavethesamepedagogicallimitationsintheirskillsasaredemonstratedbyteachersthemselves.
Inadditiontoknowledgeandskills,individualcapacityisinfluencedbythemotivationoftheimplementationstaffandtheirretentionaftercapacityhasbeenbuilt.SomeKIsstatethatdelaysinpaymentstogovernmentstaffinvolvedinGEP3interventionsdiscouragestheirparticipationin
interventionimplementation.Inaddition,thetransferofgovernmentstaffoncetheircapacitieshavebeenbuiltaffectstheirabilitytoactasaneffectiveprojectresource.AccordingtooneKI,UNICEFhasinsistedthatthosethathavebeentrainedshouldnotbetransferredindiscriminately.Nonetheless,itremainsachallenge,inparticularatatimeofleadershipchange.
EvaluationoftheGirl’sEducationProject3
©EDOREN 39
‘Lettherebemotivationasstaffgetdirectpaymentimmediatelyworkisdone.Lettherenotbedelayinpayment.CurrentlytherearedelaysofthreemonthsforpeoplegoingfromASCenumeratoruptodataentryofficers.Thatdiscourages.’(GovernmentKI,Niger)
‘Anotherproblemalsoisthatpeoplecomeandgo.Thiscreatesaproblemwhenyoubuildcapacity.(…)Sothetrainingtheyweregivenforsometimetobeabletodeliver,attheendofthedayheispostedelsewhere.’(Non-governmentKI,Bauchi)
Governmentorganisationalcapacityvariesacrossinterventions.ThecashtransferschemeandEMISaremanagedbyspecialunits,jointlystaffedbySUBEBandSMoEs,whichfacilitatescoordination.Theseunitshavereceivedintensiveexternalsupporttoimproveproceduresandsystems.61TheKIIsindicatethattheseunitsarewellawareoftheirrolesandresponsibilities.Althoughtheyarestillconfrontedbychallengesinthereleaseoffunds,whichaffectstheiroperation,theirorganisationalcapacityislikelytosupporteffectiveimplementation.
‘InBauchiandKatsinatheydonothaveoperationalmanualsintermsofimplementationofcashtransferprogramme.TheSokotoProgrammeImplementationUnithasmanualsintermsofrecord-keeping,payment,mobilisationandsensitisation.Wehaveworkingdocumentsthatguideallactivities.InKatsina,oncetheconsultantlefttheydidnothaveanydocument.’(GovernmentKI,Sokoto)
However,organisationalcapacityatlocalgovernmentlevelmaynotbeonaparwithstatelevel
capacity,whichposesarisktoimplementation.
‘TheSBMCsneedmoretrainingbecausesomearenotliterate.Therewasonlyonetraining,andyoushouldcontinuouslyre-trainthem.Identifyingthebeneficiarieshadsomechallenges.So,theyhadtofallbackontheSBMCstogofromhousetohouseinordertoconfirmthebeneficiaries.So,theyneedtrainingontechnicalitiesofhowtheygoaboutobtaininginformation.’(GovernmentKI,Sokoto)
Similarly,enrolmentdrivesfollowwell-understoodprocedures,whichareperiodicallyreviewedandsubjecttosharingofexperience.Amulti-tiercoordinationprocessinvolvingawiderangeofstakeholderssupportstheirimplementation.TheearlylearninginterventionandG4Gwillbeimplementedwiththesupportofnon-governmentactors.Thisislikelytosupporttheorganisationalcapacityforimplementationduringthepilot,andismeanttostrengthengovernmentorganisationalcapacityforscale-up.OnepointofattentionisthattheirimplementationtakesplaceinthecontextofsimilarinterventionshavingbeenconductedinthepastorbeingcarriedoutatthesametimeinKatsinaandZamfara.Ifoperationalproceduresdiffersubstantiallyfrompastexperience,thismayresultinconfusionamonggovernmentstaff.
Monitoringandfollow-upsupportsupervisionareconsideredbymostKIsaskeyoperational
processesaspartofprojectimplementation.ItisthereforerelevantthattheGEP3redesignhasemphasisedtheproject’smonitoringfunctionandhaskeptM&EofficersaspartoftheGEP3stateteams.GiventhatthenewGEP3stateteamshadrecentlystartedoperationsatthetimeoftheKIIs,itwasnotyetclearhowthiswouldbeoperationalised.GEP3partlyreliesongovernmentcapacitytomonitorGEP3interventions.Whilestategovernments,withthesupportofpartners,haveinvestedinqualityassuranceunitsatstate,zonalandLGAlevel,KIspointtoinadequateandinfrequent
61KIshighlightthat incontrasttocashtransferschemes introducedinBauchiandKatsina inthepast,thePIUs inSokotoandNigerhaveaccesstomanualsandprocedurestosupporttheiroperations.ThePIUsalsoappreciatetheuser-friendlysupportsoftware.Similarly,theinterviewedEMISmembersvaluethenewlyintroducedASCdatasoftware.
EvaluationoftheGirl’sEducationProject3
©EDOREN 40
implementationofinterventionmonitoringandfollow-upaftertraining.Themainreasonforthisthatwashighlightedisthatresourcesarenotavailableorarenotreleasedformonitoringactivities.Otherreasonsmentionedare:thequalificationsofthemonitors(particularlyatlocalgovernmentlevel),accessibilityoftheschoolsandpoliticalmeddling.TheKIIsindicatethatmonitoringisparticularlyweakinIQSsbecauseSAMEanditslocalgovernmentstructureshaverelativelylimitedaccesstoresourcestoconductthemonitoring.TheGEP3baselinesurveyconfirmsthatintegrationwithinGEP3-focusIQSsisnotwellmonitoredandthatIQSsreceivefewexternalmonitoringvisits.
Finally,GEP3’simplementationislikelytobeinfluencedbyinstitutionalcapacityconstraints.Amajorissueisthelackofclarityregardingrolesandresponsibilitiesfortheintegrationofformal
educationinQur’anicschoolsbetweenSAMEandSUBEB.Also,thelackofapolicyframeworkforcashtransferschemesmayaffecttheimplementationofthisintervention,particularlyintimesofleadershiptransition.However,SBMCsupport,theASCandenrolmentdrivesarewellanchoredinpolicyandstatestrategy.Theearlylearningintervention,G4GandHiLWAareproject-driven,sotheirembeddednessingovernmentinstitutionalstructureshasyettobeestablishedanddemonstrated.
2.4.3 Equity
EquityineducationisaconsiderationthatisattheheartofGEP3.Theprojectintendstoreducethedisparitiesbetweengirlsandboysineducation.VulnerabilitywasaconsiderationintheselectionofthesixGEP3LGAswithineachstateatthestartofGEP3,usingthegendergapasatargetingcriterion.62Error!Referencesourcenotfound.presentstheaverageGenderParityIndex(GPI)amongGEP3LGAsandintheentireGEP3states.Furthermore,itdetailshowthesixGEP3LGAsperstatearedistributedacrosstheGPIquintiles.ThefirstquintileincludestheLGAswiththe20%lowestGPI,whilethefifthquintilerepresentstheLGAswiththe20%highestGPIwhenallLGAsinthestatearerankedbytheirGPI.InBauchiandNiger,GEP3ismostlyoperatinginLGAsthataremost
vulnerableintermsofagendergap,whileinKatsina,ZamfaraandSokotoGEP3isimplementedin
LGAswithrelativelylowergenderdisparitiescomparedtotherestofthestateLGAs.Theevaluationteamdoesnothaveaccesstotherawschool-levelASCdatatoreviewtowhatextentthemostvulnerableschools,intermsofgendergap,areincludedamongtheGEP3focusschools.Theinitialschoolselectiondidtargetmostlyruralschools,whichcanbeexpectedtobemorevulnerableintermsofgenderdisparityasbarrierstogirl’seducationaregenerallyhigher.
Table1: GPI–averagesandquintiledistribution,perLGA
GEP3
states
AverageGPIa NumberofGEP3LGAsperGPIquintile
GEP3LGAs AllLGAs Q1 Q2 Q3 Q4 Q5
Katsina 0.78 0.75 -- 1 3 2 --
Zamfara 0.56 0.55 -- -- 1 3 2
Sokoto 0.65 0.61 -- 2 1 -- 3
Niger 0.69 0.78 2 2 1 1 --
Bauchi 0.75 0.85 2 2 1 1 --
Source:ASC2014–2015aAverageofLGA-levelGPI,calculatedbydividinggrossprimaryschoolenrolmentofgirlsbygrossprimaryschoolenrolmentofboysintheLGA
62Cocco-Klein,S.(2014)NoteonGEP3targetingandimplementationstrategy.UNICEF.
EvaluationoftheGirl’sEducationProject3
©EDOREN 41
SeveralGEP3interventionsaredesignedtobeequityenhancingbeyondtheoverallgenderequity
focus:thatis,ashavinganemphasisonreachingthemostvulnerablepopulations.Thecashtransferprogrammetargetsschoolswiththehighestproportionofout-of-schoolgirls.ThesupporttoIQSsismeanttoexpandaccesstoqualitybasiceducationformarginalisedchildreninrurallocations,since,accordingtotheGEP3valueformoneystrategypaper,Qur’anicschoolsareoftenlocatedincommunitiesthatareunder-served.
Thedegreetowhichthemostvulnerablegroupswillactuallybereachedwillbeinfluencedbythe
operationalisationoftheinterventionstrategies.AccordingtotheinterviewedrepresentativesofthecashtransferPIU,thecashtransferprogrammehasreachedthetargetedpopulation.Empiricalverificationisrequiredastowhetherthemostvulnerablehouseholdsareabletoreceivethetransferasplanned.OneKIpointedoutthatcashtransferprogrammesinNigeriaarecurrentlynotrights-based:beneficiariesdonotseeitastheirrighttoreceivethetransferandgenerallydonotcomplainiftheydonotdoso.Thismaybeparticularlythecaseforthemostvulnerablegroups.
WithregardtoIQSs,theIQSselectioncriteriaincludedtherequirementthat40%ofthepupilsreceivingtheintegratedcurriculumaregirls.ThisshouldensureGEP3supportbenefitsgirls.However,asindicatedbefore,baselinedatacollectionsuggeststhatthisselectioncriterionisnotalwaysmet.
Theenrolmentdriveimplementationstrategyisadaptedfromyeartoyearbasedonlearningandstakeholderfeedback.Equityconsiderationsseemtobepartofthisadaptation.InZamfara,oneKIpointedoutthatinthe2014enrolmentdrivespecificattentiontoethnicminoritiesanddisabledchildrenwasincluded.InBauchi,the2014enrolmentdrivetargetedLGAswiththelowestenrolment.Localgovernmentcoordinationmeetingshaveexpandedtheleveloffemalerepresentation.SomeKIsacknowledgetherisk,though,thatenrolmentdrivescouldbereachingcommunitiesclosetotheschool,orwhereSBMCmembersreside,morethanremotecommunitieswheretheymaybeparticularlyneeded.
Theremotenessofcommunitiesandtheschoolislikelytoaffectwhichcommunitymembersare
representedontheSBMC.OneKIpointedoutthatthiswillparticularlyplayaroleduringtherainyseason,whenrepresentativesfromsomecommunitiesmaynotbeabletoattendSBMCmeetings.AspointedoutbyanotherKI,remoteschoolsaredisadvantagedsincetraineementorsmayprovidementoringatclusterlevelratherthanvisitindividualschools.ThisisparticularlyburdensomeforSBMCmembersofschoolsthatarelocatedataremotedistancefromtheclusterpoint.
HumphreysandCrawfurd(2014)pointoutthatequityconsiderationsneedtobeaddressedwhenSBMCsareempoweredandexpectedtomobiliseresourcesfortheschool.Theyindicatethatsystemsneedtobeputinplacetomakesurethatpoorcommunitiesthatareunabletomobiliseresourcesarenotfurtherdisadvantagedbyreceivingnofinancialsupport.
TheKIIsraisedtwoequityconsiderationswithregardstoteacherdevelopment.First,severalKIspointedtotheriskofincreasingthedisparitiesbetweenruralandurbanschoolsbyreinforcingmother-tongueinstruction.Bettereducatedparents,mostlyinurbanareas,arelikelytodemandinstructioninEnglish.Totheextentthatmothertongueinstructionisreinforcedinsomeschools—likelytoberuralschools—thismayexacerbatethedifferencebetweenurbanschoolsteachinginEnglishandattractingthechildrenofeducatedparentsversusruralschoolsteachinginthemothertongueandattractingthechildrenofless-educatedparents.Second,KIsindicatedthattheteachertrainingmaterialsaremostlyinEnglish.Thismaydisadvantageteacherswithlowercapacity–particularlyIQSfacilitators.
EvaluationoftheGirl’sEducationProject3
©EDOREN 42
EvaluationoftheGirl’sEducationProject3
©EDOREN 43
3 Baselineevaluation–Earlylearningintervention
3.1 GEP3’searlylearningintervention
3.1.1 Objectivesandexpectedresults
GEP3’searlylearninginterventionaimstoimprovetheearlylearningskillsofchildreninthe
primarygradesonetothree(P1–P3)inthemothertongue,whilealsopreparingchildrentolearnwithEnglishasalanguageofinstructionbythetimetheytransitiontogradefour.Akeymeasurementofsuccesswillbeimprovedliteracyskills.63
Tothisend,UNICEFhascontractedFHI360toimplementtheRANAproject.Theproject’sprimaryfocusisonimprovingreadingoutcomesforgirlsandboysingradesonetothree.Improvementinnumeracywillbeindirectlytargetedbyintegratingnumeracyintoreadinginterventions.64TheRANAprojecthasidentifiedthreeintermediateresults,eachofwhichhastwosub-components(seeBox1).
Box1:RANA’sintermediateresultsandsub-components
1. IntermediateResult1:Readingandnumeracyinstructionimproved1.1. Relevantmaterialsdevelopedanddistributed1.2. Teacherin-serviceprofessionaldevelopmentimproved
2. IntermediateResult2:Increasedengagementofcommunitiesinreadingactivities2.1. Communityawarenessincreased2.2. Schoolcommunitiesengaged
3. IntermediateResult3:Sustainablereadingdeliverysystemsimproved3.1. Readingpolicyimproved3.2. Interventionssustainablydeveloped
3.1.2 Interventionstrategy
RANAwillbeimplementedoverathree-yearperiodinsixLGAsthatareamongGEP3’spilotLGAs
inZamfaraandKatsina(threeLGAsperstate).TheRANALGAshavebeenpurposefullyselectedamongtheGEP3LGAsinviewoftheiraccessibilitytothestatecapital.Theinterventiontargets120publicprimaryschoolsand80IQSsthatareincludedinGEP3’spilotschoollists(60primaryschoolsand40IQSsoutof,respectively,210GEP3primaryschoolsand200GEP3IQSsperstate).Table2presentsthedistributionofthetargetschoolsacrossthesixLGAs.Noscale-uptargetshavebeensetbeyondthethree-yearpilotperiod.
63UNICEF(2015f)StrategyPaperonEarlyLearning(literacyandnumeracy)RANA.64Numeracyinterventionsareplannedtobeginonlyafterthe2016/2017academicyear(RANAQuarterlyReportSeptember–December2015).
EvaluationoftheGirl’sEducationProject3
©EDOREN 44
Table2: NumberofRANApilotschoolsperGEP3LGAandtypeofschool
TypeofschoolKatsinastate Zamfarastate
Batsari Kankia Rimi Bungudu K.Namoda Tsafe
Publicprimaryschools
21 17 22 23 19 18
IQSs 14 12 14 18 12 10
Source:Owncalculations,basedonFHI360reports
TheRANAprojectwillconsistofthefollowingcoreactivities:65
1. Provisionofapackageofteachingandlearningmaterials
• RANAwilldevelopanddistributetheRANALiteracyPackage(RLP):animprovedHausa-basedliteracycurriculum,andteachingandlearningmaterialsforP1,P2andP3inHausa(trainer’sguide,teacher’sguide,scriptedlessons,pupil’sworkbookandreadaloudbooks).
• Gender66andnumeracythemesareintegratedintothereadingcontent,exercisesandmaterials.
2. Teacherin-serviceprofessionaldevelopment
• P1–P3teachersandheadteachers67willbetrainedinHausa-basedliteracyinstructionusingtheRLP,andinteacherprofessionalisation(focusingontopicssuchastimeontask,lessonplanning,andeffectivepreparationandutilisationofmaterials).
• Trainingwilltakeplaceatthelevelofaclusterofschools,ledbyaMasterTrainer.Intotal,20clustershavebeenformedineachstate,witheachclusterconsistingofthreetosixschools.AllclustersinKatsinahaveamixedcompositionofpublicprimaryschoolsandIQSs,whilefourclustersinZamfaraonlyconsistofpublicprimaryschools.
• Peer-mentoring:oneLeadTeacherperschoolfacilitatesweeklymeetingsamongteacherswithintheschool,toshareideas,discussconcernsaboutRLPmaterials,andreviewtheforthcomingweek’slessonsandexercises.68
• MasterTrainerswillconducton-sitemonthlysupervisorysupportandmonitoringvisits.SUBEBand/orLGEAstaffwillconducttwice-monthlyvisits.
3. Communityawarenessandengagementactivities
• RANAstateteams,assistedbyFederationofMuslimWomen’sAssociationsofNigeria(FOMWAN),willsensitisecommunitiesandotherstakeholdersinregardtotheproject,inordertoobtaintheirsupportforearlygradeliteracy.
• SchoolcommunitieswillbeengagedthroughtheSBMC/CBMC,witheachoneappointingaLiteracyChampiontocoordinateliteracyactivitiesintheirschooland,asappropriate,withinthelocalcluster.LiteracyChampionswillbeinvitedtoannualtrainingsessionstolearnaboutactivitiestosupportliteracyacquisition.
65RANAQuarterlyReportOctober–December2015.66Genderisintegratedbyusinggender-sensitivecharacters,illustrationsandadjectives,aswellasthroughteachersensitisation(source:communicationwithFHI360).67InIQSsalltheteacherswhoteachnon-religioussubjectsareinvitedfortraining.IftheIQSdoesnothaveaspecificheadteacherpositiontheproprietorwillbetrained(source:communicationwithFHI360).68Forschoolswithonlyoneortwoteachersinearlygradesthisschool-basedsystemofteachersupportwillneedtobeadjusted,possiblybycreatingmini-clustersofgeographicallyproximatesmallschoolswithinthelargerclusters.
EvaluationoftheGirl’sEducationProject3
©EDOREN 45
• RANAwillprovideaCommunityLiteracyActionGuide,asetoflow-costresources,materialsandexercises,andwillpromotethecreationofcommunity-ledLiteracyActionPlans.
4. State-levelengagementandpolicyimprovementactivities
• State-leveleducationstakeholders,liketheSMoE,SUBEBandSAME,willbeengagedtoencouragestateownershipandsustainabilityoftheproject.
• RANAwilladvocateforachangeinpoliciesofimmediateconcerntotheproject,suchaspoliciesthatsupportearlygradeliteracyor(re-)deploymentofteachers.
TheRLPandteacherin-serviceprofessionaldevelopmentwillberolledouttogradesP1andP2duringthesecondandthirdtermsofthe2015/2016academicyear.AllteachersteachinginP1andP2willparticipateinthetraining.Teachersof96targetedschoolsweretrainedinFebruary2016duringaPhase1roll-out,whiletheteachersintheremaining104schoolsweretrainedinApril2016(Phase2roll-out).P3willbecoveredfromthestartofthe2016/2017academicyear,whichwillinvolvedistributingP3-tailoredmaterialsandinvolvingP3specificteachersinprofessionaldevelopment.TheinterventionpackagewillbesimilarforpublicprimaryschoolsandIQSs.TheRANApilotprojectisscheduledtoendinAugust2018.
RANAisimplementedbyaconsortiumoforganisations.FHI360isresponsibleforprogrammemanagementandtechnicalquality.FOMWANcarriesoutcommunityengagementandmobilisation.TheAchievingHealthNigeriaInitiativeisresponsibleforteachertraining,curriculumandmaterialsdevelopment,andforM&E.RANAeducationfieldteamsaretobeembeddedwithinthestatesandlocaleducationauthorities,whoaremeanttoleadtheinterventions.
RANAwilltakeplaceagainstthebackdropoftheotherGEP3interventionsbeingimplemented(butonlyinpublicprimaryschools69),suchasannualenrolmentdrives,SBMCtrainingandgrantprovisionfrom2015onwards,headteachertrainingin2016,and,potentially,thepilotingofG4GinasubsetofGEP3schools.
3.1.3 InterventionToC
69InKatsinaandZamfaraGEP3willonlyinterveneinthe40RANAIQSsineachstate,outoftheinitial200IQSsselectedforGEP3support.
EvaluationoftheGirl’sEducationProject3
©EDOREN 1
Figure1visualisestheToCoftheearlylearningintervention.Themaincausalassumption
underlyingtheToCisthatliteracylearningoutcomesofgirlsandboys,particularlyintheHausa
mothertongue,willimproveinearlygradesifteachingpracticeimprovesthroughtheuseof
improvedteachingandlearningmaterials(useofRLP)andthepresenceofmoreknowledgeable,
skilledandgender-sensitiveteachers.Numeracywillbeinfluencedindirectlybyincorporatingnumeracythemesintothereadingcontent,exercisesandmaterials.
EvaluationoftheGirl’sEducationProject3
©EDOREN 2
Figure1: DiagramdepictingToCfortheearlylearningintervention
Source:Authors
EvaluationoftheGirl’sEducationProject3
©EDOREN 3
Thecentralassumptiondiscussedaboveisconditionalonschoolstakeholders(suchasparents,communityleaders,IQSproprietors,headteachers,teachersandgovernmentstaff)beingsupportiveofanincreasedemphasisonmothertongueinstructionandliteracyacquisition;andontheteachersadheringtouseofthemothertongueduringinstruction.Tothisend,theprojectwillengagethecommunity,championliteracyinthecommunities,trainheadteachersandadvocateforgovernmenttoimplementenablingpublicpolicies.OtherfactorsthatwillinfluenceimprovedHausa-basedteachingpracticesareregularpupilattendanceandtheclassandschoolenvironmentsbeingconduciveforlearning.ThefocusonHausaassumesthatitisthemothertongueandthelanguagethatpupilsknowandunderstandbest.
Improvementofteacherknowledgeandskillsthroughin-serviceteacherdevelopmentisassumedtobecentraltomoreeffectiveteachingintheearlygrades.70ThisassumesthattheimprovedHausaliteracycurriculum,thematerials,thepedagogicalmethodologyandthein-servicetrainingandmentoringapproacharewelltargetedtotheneedsandcapacitylevelofteachers,aswellastotheirteachingenvironmentinpublicprimaryschoolsandIQSs.Tooptimiseteacherlearning,theprojectenvisionsateacherdevelopmentapproachbasedonprogressiveprofessionaldevelopmentandschool-basedpeer-mentoring,complementedbysupportivesupervisionandmonitoringbyMasterTrainersandgovernmentstaff.ThesuccessofthisapproachdependsonteachersbeingliterateinHausa,weeklypeermeetingsbeingfeasiblegiventhecontextoftheschool(forexample,teachersbeingabletomeetregularlywithotherteachersgiventhesizeoftheschooloritslocation),LeadTeachershavingthecapacity,motivationandavailabilitytoleadthemeetings,andMasterTrainersandgovernmentstaffhavingtheresources,competenciesandincentivestoconducteffectivesupportivesupervision.Throughgender-sensitivetrainingthataddressesequityintheclassroom,teachersareexpectedtotargetgirlsmorespecificallyinclass,whichismeanttoincreasethelikelihoodofgirlsbenefitingoftheeducationprovided.
Inorderforteachertrainingtoresultinimprovedknowledge,skillsandteachingpractices,teachersareassumedtobemotivatedtolearnandtranslatenewknowledgeandskillsintopractice.Peer-to-peerinteraction,supportivesupervisionandotherincentives(e.g.trainingcertificates)aremeanttocontributetothis.Theextenttowhichteachermotivationtranslatesintoimprovedteachingmayalsobeinfluencedbyteachers’remunerationandtheirworkingconditions,amongotherfactors.Furthermore,actualimprovedteacherefficacyandlearningoutcomesmayimproveteachers’perceivedefficacy,whichinturnmayincreasetheirmotivation.
ImprovedteachingandlearningisassumedtobefacilitatedbythedistributionandusageofrelevantteachingandlearningmaterialsinHausa.Thisrequiresthattheteachershavethepedagogicalknowledgeandskillstoeffectivelyusethematerialsduringteaching.Furthermore,itassumesthatthematerialsarewellalignedwiththecurriculumandwiththecompetencylevelsoftheteachersandpupils.
Whilethecentralobjectiveoftheearlylearninginterventionistoimprovepupils’literacyand,toalesserdegree,numeracyskills,itisassumedthatthiswillfacilitatetheacquisitionofEnglishasasecondlanguage,andthetransitiontoEnglishinlatergrades.Furthermore,improvedlearningintheearlygradesisexpectedtocontributetohigherretentionratesbecausechildrenwhoperformwellearlyonareassumedtobemoremotivatedandsupportedbytheirparentstoremaininschoollonger,alinkthatmaybeparticularlyimportantforgirls,whofaceahigherriskofdroppingout(UNICEF,2014d).Apotentialfeedbackloopexistsbetweenincreasedretentionandteachingquality70Wedistinguishbetweenthreetypesofknowledge:subjectknowledge,curriculumknowledgeandpedagogicalknowledge.SeeSection3.2.9.2.
EvaluationoftheGirl’sEducationProject3
©EDOREN 4
inearlygrades.Thisfeedbackloopcanbenegativeorpositive.Ifhigherretentionisnotaccompaniedbyincreasedteachingandschoolresources,itcanleadtohigherpupil-to-teacherratios,whichmaynegativelyaffectteachingquality.Ontheotherhand,teachersmaybeincreasinglymotivatedbythehigherretentionoftheirstudents,whichmaypositivelyinfluenceteachingquality.
3.1.4 PlannedresearchandM&E
InthissectionwedescriberesearchandM&Eactivitiesthatarecomplementarytotheevaluationandthatcouldbeofusefortheinterpretationoftheevaluationfindings.
• PlannedM&EbyRANA
RANAwillconductmonitoringandformativeevaluationbasedonthemonthlyschoolsupportvisitsthatwillbeconductedateachschoolbyMasterTrainers,sometimesincollaborationwithSchoolSupportOfficers.Dimensionsthatwillbemonitoredduringschoolsupportvisitswillincludefidelityofimplementation(properuseofRANA-scriptedlessons),availabilityoflearningmaterials,pupillearning(readingandmathematics)71,basictimeuse,andgenderequity.Datawillbecollectedelectronicallyusingtablets.Box2presentsthesixdatacollectiontoolsthatwillbeusedbytheMasterTrainersandSchoolSupportOfficers.
Box2:RANAmonitoringtools
1. HeadTeacherMeetingGuide:collectsinformationonhowtheRANAprogrammeisbeingimplementedintheschool,whatspecificchallengesarebeingfaced,andwhatisworkingwell.
2. LeadTeacherMeetingGuide:collectsdataonandreviewsweeklyclassobservationswithLeadTeacher.
3. ClassroomObservationTool:capturesdataonactivitiesincludedintheteachers’scriptedlessonsthatteachersdeliverintheclassroom;observationofthetimetakenforeachactivity;anddataontheclassroomenvironment,includingteacherconfidenceandpreparedness,andequalparticipationofboysandgirls.
4. TeacherMeetingGuideandFeedbackTool:facilitatesdiscussionofobservedresultsfromclassroomobservationwiththeteacherandcapturesdataonteacherpracticesthatcannoteasilybeobserved(includinggenderpractices).
5. EGRA:pupilreadingassessmentcoveringlettersoundknowledge(P1andP2),letterwriting(P1),syllableandshortwordreading(P1andP2),listeningcomprehension(P1andP2)andoralreadingfluency(P2).
6. EGMA:pupilmathematicsassessmentcoveringnumberidentification(P1)andquantitydiscrimination(P2).
TheRANAprojecthasestablishedaresultsframework,whichincludesoutcome,outputandprocessindicators.
• PlannedresearchbyUNICEF
71 Master Trainers and School Support Officers will observe classrooms, will conduct early grade reading assessments(EGRA) and early grade mathematics assessments (EGMA) with a small number of students (three to five studentsrandomly selected from each observed class), andwill hold formativemeetingswith teachers, head teachers, and leadteachers.
EvaluationoftheGirl’sEducationProject3
©EDOREN 5
UNICEFisplanningresearchintoparents’attitudesregardingthelanguageofeducationandlearning.Thisresearchisscheduledfor2017/2018.Thetermsofreferenceforthisstudyhaveyettobespecified.
3.2 Methodology
Thissectiondescribesthemethodologythatguidedthedesign,implementationandanalysisofthebaseline.Itstartswithapresentationoftheevaluationquestionsanddesignasaframeworkforthebaseline’smethodologicalchoices.Subsequently,specificmethodologicalissuesarediscussed:thesamplingstrategy,therandomisationprocess,theinstrumentdesign,thesurveyfieldworkandthedataqualityassurance.Thesectionendswithadescriptionofethicalandinclusionissues,andmethodologicallimitations.
3.2.1 Evaluationquestions
TheevaluationofGEP3’searlylearninginterventiontakesatheory-basedapproach.Theintervention’sToCwasusedasaframeworkforformulatingtheevaluationquestions.Theevaluationquestionsinterrogateawiderangeofthecause–effectassumptionsunderlyingdifferentstepsintheToC,inordertobetterunderstandhowchangemaycomeabout.Theevaluationquestionssubsequentlyguidedtheoverallevaluationdesignandbaselinemethodology.TheevaluationquestionsarepresentedinBox3.
Box3:EvaluationquestionsforevaluationofGEP3’searlylearningintervention
1. TowhatextentdoestheearlylearninginterventionimproveHausaliteracyandEnglishlanguagelearningoutcomesamonggirlsandboysintheearlygradesinprimaryschoolsandIQSs?Towhatextentdoestheearlylearninginterventionreducethegapbetweenthelearningoutcomesofthelowestperformingpupilsandtheexpectedlearningoutcomes,asexpressedinthecurriculum?
2. Towhatextentdoesteachers’knowledgeofliteracyandlanguageacquisitioninearlygradesimproveasaresultoftheintervention?
3. Towhatextentdoteachers’skillsinearlygrade,gender-sensitiveinstructionimproveasaresultoftheintervention?
4. Towhatextentandhowdoteachersadjustandchangetheirclassroompracticesasaresultoftheintervention?
5. AremorereadingandlearningmaterialsinHausausedintheclassroomduetotheintervention?Dotheycontributetomoreeffectiveteachingandlearning?
6. Towhatextentdoestheearlylearninginterventionimprovepupilretention,especiallyretentionofgirls?
Inadditiontotheintervention’sToC,thechoiceoftheabovequestionshasbeenbasedonthefollowingadditionalconsiderationsandcontextualfactors:
• InlinewithGEP3’semphasisonqualityoflearningandDFID’scurrentfocusonlearningwithintheeducationsector(DFID,2013),theGEP3evaluationprioritisesmeasuringchangesinlearningoutcomesoverchangesinaccesstoeducation.WhiletheRANAprojectemphasisesreadingasoneaspectofpupil’slearningoutcomes,theevaluationquestionsadoptliteracyasamore
EvaluationoftheGirl’sEducationProject3
©EDOREN 6
comprehensiveconstructtobeexamined,inlinewiththeGEP3strategypaperontheearlylearningintervention.Duetothefactthattheinterventiononlyindirectlytargetsnumeracy,andbecausebudgetconstraintslimitedthenumberofassessmentsthatcouldbeincludedintheevaluation,numeracyisnotprioritisedintheevaluationquestions.
• Fromanequityperspectiveitisimportanttounderstandiflearningistakingplaceforall,andwhetherGEP3isimprovingthelearningoutcomeofthepupilswhoarecurrentlyfallingbelowexpectedlevels,asexpressedinthecurriculum.Hence,theevaluationquestionsincludeafocusonexaminingthelearningoutcomesofthelowestperformingpupils.
• GiventhefactthattheearlylearninginterventionisofthestrategicinterestfortheprojectasawholeandthelimitedevidencebasefortheinterventionthereisstrongagreementamongGEP3managementandDFIDNigeriathattheevaluationshouldemphasisedeterminingtheeffectsoftheinterventiononteachingandlearningoutcomes,andshouldquantifytheattributableimpactoftheinterventionontheseoutcomes.
• Theevaluationfocusesonthelinkagebetweenteacherknowledgeandskills,teacherclassroompracticesandlearning,andtheactualuseoflanguageinthisteaching/learningnexus.TheevaluationtherebyrespondstothehighneedforsuchevidenceidentifiedinHumphreysandCrawfurd(2014).ResearchplannedunderGEP3willassesscommunityengagementactivitiesandparents’attitudestowardthelanguageofinstruction.
3.2.2 Evaluationdesign
Toanswerthequestionsregardingwhetherchangesinlearning,teachingandteacherknowledgeandskillsareattributabletotheearlyintervention,theevaluationisdesignedasaclusteredRCT,stratifiedbyLGAsandtypeofschool(primaryschoolvs.IQS).72ForeachtypeofschoolanequalsizedstudysampleofschoolswasrandomlyselectedfromamongallGEP3schoolsineachofthesixGEP3LGAsinwhichRANAisbeingimplemented.73Subsequently,halfoftheschoolstudysampleineachLGAwasrandomlyassignedtoatreatmentgroupandtheotherhalftoacontrolgroup.TheselectedstudytreatmentschoolswereincludedamongthelargergroupsofschoolstargetedbyFHI360toreceivetheRANAintervention,whilethecontrolschoolswerenotselectedforinterventionimplementation.Thedetailsofthesamplingstrategy,samplesizesandrandomisationprocedurearediscussedinthenextsections.
Theaimoftherandomselectionoftreatmentandcontrolschoolsistoconstructtwogroupsofschoolsthatarestatisticallyequivalentonaverageatthestartoftheevaluation(i.e.atbaseline),intermsofobservableandunobservablefactors.Iftherandomisationsucceedsincreatingsuchequivalentgroups,anydifferencesobservedinoutcomevariablesatalaterstageoftheinterventioncanbeattributedtotheintervention(Section3.3presentsthedataonthebalancebetweenthetreatmentandcontrolgroup).WeoptedtoconducttherandomisationattheschoollevelasthisisalsotheunitatwhichmostofRANA’sactivitiesareimplemented,74andbecauseschoolsoffersufficientunitsofrandomisationtoachievethedesiredstatisticalpower.
WechoseaRCTdesignbecause:a)theevaluationquestionsemphasisethequantificationofattributableimpactandfocusonarelativelywell-delineatedcausalchainbetweenteacher
72Theclusteredaspectoftherandomisationreferstothefactthattherandomisationtakesplaceataclusterorgrouplevelinsteadofatanindividuallevel.Inthiscasethecluster/groupistheschool.73Thereareapproximately35GEP3primaryschoolsand33IQSsperLGA.74Someactivitieswilltakeplaceatthelevelofaclusterofschools(i.e.cluster-basedtraining),whichwillrequireparticularattention,asdiscussedbelow.
EvaluationoftheGirl’sEducationProject3
©EDOREN 7
development,teachingpracticeandlearningoutcomes;b)thedesignisstrongintermsofminimisingselectionbiasandthereforeidentifyingattributableimpact;c)sufficientsampleunitsareavailabletogeneratestatisticallysignificantfindings;d)baselinedataarenotstrictlyrequiredtoassessimpact,whichprovidesflexibilityatendlineinregardtoevaluatingimpactonoutcomesforwhichnobaselinedataareavailable;ande)weconsidereditfeasibletoconstructandmaintainavalidcontrolgroup.Whiletheidentificationandmaintenanceofavalidcontrolgroupisaconcernandisexposedtorisks75,thefactthatRANAispilotedwithinasubsetofidentifiableGEP3schoolsfacilitatedtheconstructionofsampleframesfromwhichtreatmentandcontrolschoolscouldbeselected.Furthermore,sincethecontrolgroupisselectedfromamongGEP3schools(i.e.fromamongschoolsinwhichRANAwouldnotbepiloted),GEP3shouldhaverelativelygoodawarenessof,andcontrolover,whattakesplacesinsuchschools.Thislowerstheriskofthecontrolgroupbeingcontaminated.
TheRCTdesigniscombinedwithanoverarchingtheory-basedevaluationapproach.Theintervention’sToCisusedtoselecttheoutcomesofinterestforwhichchangeswillbecomparedbetweentreatmentandcontrolschools.Theseoutcomescoverintermediaryandfinaloutcomesalongtheassumedcausalchain,whichwillmakeitpossibletounpackhowchangetakesplace.
Aseriesofquantitativemethodsareusedwithinthesampledschoolstocollectdataonoutcomesandotherexplanatoryvariables–inparticulartheadministrationofpupilandteachertests,structuredteacherclassroomobservationsandasurveyofpupils,teachersandheadteachersusingclosed-ended,structuredquestionnaires.Duetobudgetconstraints,EDORENwillnotimplementschool-orcommunity-levelqualitativeresearchduringthe2015–2017evaluationperiod.76Theevaluatorsareawarethatthisisalimitationofthedesign.However,thismaychangeinthe2017–2020scale-upperiod,duringwhichqualitativeresearchcouldbeimplementedforin-depthexplanationofthequantitativefindingsatmidline.ThequantitativemeasurementalongtheToCdoesprovidearichdatasetthatunpackstheToC.Furthermore,qualitativedatageneratedthroughanewroundofstate-levelstakeholderinterviews(scheduledforthefirstquarterof2017)willenrichtheunderstandingofthecontextinwhichRANAtakesplace,andwillprovideinsightsaboutimplementationchallengesandinterventionperformance.Inaddition,RANAM&Edatacancomplementquantitativesurveydata.
Thedatacollectionmethodsare/willbeappliedatbaseline,midlineandendline.BaselinedatacollectiontookplaceinOctober–November2015inthefirsttermofthe2015–2016schoolyearbeforethestartofRANAimplementation(seeError!Referencesourcenotfound.).Thedetailsofthebaselinesurveyimplementationarepresentedintherestofsection3.2.MidlinedatacollectionisscheduledforMay–June2017,whichisthefifthschooltermofRANAimplementation.Thisisbeforetheinterventionisscheduledtofinish.ThetimingofthemidlinehasbeenchosentoallowforevaluationfindingstoinformGEP3scale-updecision-makingin2017.EndlinedatacollectionisnotincludedinError!Referencesourcenotfound.becausetheresources,scopeandtimingofendlinedatacollectionhaveyettobedecided,althoughitstimingispreliminarilysetfor2019.Whileallevaluationquestionscanbeansweredatmidline,endlinedatacollectionwouldofferanswersaboutlonger-termeffects.
75Schoollistshavebeenunreliableinthepastandarealmostcompletelynon-existentforIQSs.Stakeholdershavestressed,andpastexperiencehasdemonstrated,thatitisdifficulttocoordinatewithgovernmentpartnersinordertopreventcontrolschoolsfrombeingaffectedbytheintervention(‘contamination’).ThesecuritysituationinnorthernNigeriaalsoincreasesthedegreeofunpredictability.76Initially,EDORENhadplannedsemi-structured,qualitativeclassroomobservationsandin-depth,unstructuredteacherinterviewstocomplementstructured,quantitativeclassroomobservations.
EvaluationoftheGirl’sEducationProject3
©EDOREN 8
Figure2: TimelineofdatacollectionandtheRANAintervention
Apanelapproachisusedtosamplepupilsfortheevaluation.Thesamepupilsthatarerandomlyselectedatbaselinewillbere-interviewedandre-assessedduringthemidlinesurvey,bothininterventionandcontrolschools.Apanelapproachischosenbecauseitismorecost-effectiveintermsofrequiredsamplesizethanacross-sectionsurvey.Italsoallowsustoefficientlymeasureandcomparetheretentionofpupilsinthesampledcohortsintheinterventionandcontrolgroups.Furthermore,itdoesnotcausecomparabilityproblemswhenbaselineandmidlinesurveystakeplaceduringdifferenttermsintheschoolyear,sincebothtreatmentandcontrolgroupswillonaveragehavegonethroughthesameschoolcyclebetweenbaselineandmidline,andarethereforecomparable.Thedisadvantageofapanelapproachisthatacertaindegreeofattritionofpupilsfromthesamplecanbeexpected.Sincetheperiodbetweenbaselineandmidlineisnotverylong,wethinkthattheattritionlevelwillbeacceptableandwehavetakenthisintoaccountindeterminingthesamplesize.77
Atbaseline,pupilsarerandomlysampledfromamongallpupilsinP2grade(orequivalentinIQSs)whentheyareinthefirsttermofthe2015–2016schoolyear.ThepanelsurveyapproachwillenableustomeasuretheimpactoftheearlylearninginterventiononimprovinglearningoutcomesbetweenthestartofP2andtheendofP3forthecohortofpupils.TheevaluationwillnotbeabletomeasuretheimpactofapupilbeingexposedtotheentirethreeyearsofRANA,fromP1toP3.P2pupilswerechosenasthetargetpopulationatbaselineforthefollowingreasons:
• BecauseofresourceandlogisticalreasonsthedatacollectionfortheevaluationoftheearlylearninginterventionislinkedtothedatacollectionforevaluationoftheIQSS.Becausethisevaluationusesacross-sectionalsurveyP1pupilsintheirfirstschooltermarenotanappropriatestudypopulationbecausetheywouldnothavebeenexposedtoanyteachingatbaselinenoratmidline.
• ThemeasurementofliteracyatthestartofP1—beforebeingexposedtoanyteaching—wouldlikelydemonstratestrongflooreffectsandofferlimitedvaluableinformationinregardtoobservingtherealdifferencesbetweeninterventionandcontrolschoolpupilabilities.
• SinceliteracylevelsatP1wouldlikelytobeverylimitedinthetreatmentandcontrolschoolsitwouldbeimpossibletoverifywhetherthetreatmentandcontrolgroupsarebalancedonthismainoutcomeofinterest.
77Ifanendlinesurveyisconductedin2019inthestudyschoolsitishighlylikelythatalargepercentageofthepupilpanelmayhavedroppedoutortransferredschoolinthefour-yearperiodbetweenbaselineandendline.Ifsufficientresourcesareavailablepupilscanbetrackedtobesurveyedoutsideoftheirformerschools.Iftheresourcestodothisarenotavailable,thesizeoftheoriginalpanelofpupilswhoarestillatthesampledschoolmaynotbesufficienttodrawstatisticallysignificantresults.Insuchacase,weproposetotakeanewcross-sectionalsampleatP2and/oranuppergrade.
Schoolyear2015–2016
Schoolyear2016–2017
Schoolyear2017–2018 2018
Sep–Dec Jan–Mar Apr–Jun Sep–Dec Jan–Mar Apr–Jun Sep–Dec Jan–Mar Apr–Jun Aug
Baseline Midline
Phase1 Phase2RANAroll-out
inP1–P2
RANAroll-outin3
EvaluationoftheGirl’sEducationProject3
©EDOREN 9
Duetothephasedroll-outofRANAinP1–P2intheperiodFebruary–May2016,thepupilsin96oftheschools(Phase1)willhavearoundtwotothreemonthsofadditionalexposuretoRANAinterventionscomparedtotheother104schools(Phase2).Giventhattheschoolsareknownthisinformationcanbetakenintoaccountduringanalysis.
Teacherandheadteachersurveyswillalsotakeapanelsurveyapproach.Thesameteachersandheadteacherswillbesurveyedatbaselineandmidline.Althoughsomeattritioncanbeexpected,weanticipatethatattritionlevelsatmidlinewillbelow.AspartofRANAimplementation,agreementissoughttokeepthetrainedteachersinthelowerprimarygradesandforteachersnottobetransferredtonon-projectschools.78
3.2.3 Samplingstrategyandsampleofthebaselinesurvey
3.2.3.1 Universe
ThestudyuniverseforthebaselineoftheearlylearninginterventionevaluationismadeupentirelyofpublicprimaryschoolsandIQSsselectedforGEP3interventioninthesixLGAsinZamfaraandKatsinawhereRANAisimplemented.SincetheimplementingorganisationpurposefullyselectedthreeRANALGAsoutofthesixGEP3LGAsineachstate,thestudyuniversedoesnotcorrespondtotheentireGEP3schoolpopulation.
TheeligibilityofschoolsforGEP3supportrequiresthataschoolfulfilscertaincriteriasetbytheproject.ThisisparticularlyimportantwhendefiningtheuniversefortheIQSs.AllIQSsincludedintheGEP3needtohavestartedintegration(thatis,non-religious,formalcoresubjects,suchaslanguageandmathematics,aretaughtattheschool),shouldteachbothboysandgirls79andshouldberegisteredwithagovernmentagency.
Thedefinitionoftheuniversealsoextendstoteachersandpupils.ThetargetpopulationofpupilsneedtobeenrolledinP2andneedtostudynon-religioussubjects.Inpublicprimaryschoolstheidentificationofthepupilsisrelativelystraightforward;however,inIQSamoredetaileddefinitionofthepupiluniverseiswarrantedduetonon-standardcompositionandorganisationoftheIQS.Inordertoachievecoherentlydefinedpopulationuniversesacrosstwotypesofschools,itwasnecessarytofurtheroperationalisetheP2equivalentlevelforIQSs(seeBox4).
78BasedonEDOREN’sexperiencewithTDPandESSPINsurveysinNigeria,thisissueofobtainingagreementregardingthenon-transferoftrainedteachersneedstobeapproachedwithcaution.Inpractice,teachertransferisdifficulttocontrol.Therefore,wewilltakeintoaccountattritionintheteachersamplewhendeterminingthesamplesize.79Aminimumof40%ofpupilsaregirls.
EvaluationoftheGirl’sEducationProject3
©EDOREN 10
Box4:P2ortheequivalentlevel
InIQSsweareinterestedinobtaininginformationforpupilsthatareatthesamelevelasP2pupilsinpublicprimaryschools:thatis,pupilsthatareatthe‘equivalentlevel’toP2.Forthepurposeofoursurvey,suchpupilshavebeendefinedas‘pupilsthathavebeenlearningnon-religioussubjectsatprimaryschoollevelatthisschoolformorethanoneyear,butforlessthantwoyears’.Inaddition,theymusthavestartedatthesampledIQSatthelowestprimarylevel.Thesetwoconditionstogetherensurethatthechildrenhavehadexposuretomorethanoneyearofformalschooling,butlessthantwoyears.ThismakesthemcomparabletoP2pupilsinpublicprimaryschools.Theclassesthatsuchpupilsareplacedinarethereforethe‘equivalentlevel’classes,whicharewhatweareinterestedin.ThesemaybecalledStage1,Stage2,BasicLiteracy,Post-basicliteracy,orsomeothernameuniquetotheparticularIQS.Similarlytothepupils,theuniverseofteachersconsideredfortheearlylearninginterventionconsistsofteachersteachingatthedesignatedschoolsandteachingpupilsenrolledinGrades1to3orequivalent(seeBox5).
Box5:P1–P3orequivalentlevel
InIQSsweareinterestedinobtaininginformationforteachersthatteachpupilsthatareatthesamelevelasP1–P3pupilsinpublicprimaryschools:thatis,pupilsthatareatthe‘equivalentlevel’asP1–P3.Forthepurposeofthesurvey,suchpupilshavebeendefinedas‘pupilsthathavebeenlearningnon-religioussubjectsatprimaryschoollevelatthisschoolforlessthanfouryears’.Inaddition,theymusthavestartedatthesampledIQSatthelowestprimarylevel.Thesetwoconditionstogetherensurethatthechildrenhavehadexposuretolessthanfouryearsofformalschooling.ThismakesthemcomparabletoP1–P3pupilsinpublicprimaryschools.Theclassesthatsuchpupilsareplacedinarethereforethe‘equivalentlevel’classes,whicharewhatweareinterestedin.ThesemaybecalledStage1,Stage2,BasicLiteracy,Post-basicliteracy,orsomeothernameuniquetotheparticularIQS.
3.2.3.2 Samplingframe
TheschoolsamplingframeconsistsofallGEP3schoolsintheselectedLGAsofthetwodesignatedstates.ThesamplingframewasconstructedbasedonGEP3schoollistsmaintainedbyUNICEF.Thetotalsamplingframeforthebaselineoftheearlylearninginterventionconsistsof410schools(210publicand200IQSs)insixLGAsintwostates.AlloftheschoolsinthesamplingframewereassumedtobeGEP3compliantandthuseligibleforinclusionintheintervention.
However,duringthefieldworkitbecameapparentthatthequalityofthesamplingframeofIQSswaspoor.TheQur’anicschoolsweremeanttohavestartedintegration–teachingnon-religioussubjects.Duemainlytonon-integration(schoolsnotteachingnon-religioussubjects)manyoftheIQSsincludedinthesamplingframewerefoundtobeineligibletobesurveyed.Thisrequiredthesamplingofreplacementsschools(seebelow).
EvaluationoftheGirl’sEducationProject3
©EDOREN 11
3.2.3.3 Samplingsizeandsamplingparameters
Ineachstate,60primaryschoolsand60IQSsweresampledforstudypurposes,creatingatotalsamplesizeof240schoolsacrossthetwostates;halfofwhichwereassignedtothetreatmentgroupandtheotherhalftothecontrolgroup.Therefore,weoversampledtheproportionofIQSsinthestudy(50%ofschools)comparedtotheproportionofIQSsinwhichtheearlylearninginterventionwillbeimplemented(40%ofschools),toensureasufficientlylargeIQSsamplesize.Weexpecttosurveythesamesampleofschoolsatbaselineandmidline,andweexpectthelikelihoodofattritionwithintheschoolsampletobelow.
Ineachschool,thetargetsamplesizeforpupilsissixgirlsandsixboysfromP2ortheequivalentgrade,resultinginatotaltargetedsamplesizeof2,880pupils(1,440girlsand1,440boys).Thesamplesizeiscalculatedtoallowfordisaggregatedanalysisforgirlsandboys.ThetargetedsamplesizeforteachersisthreeteachersforeachpublicprimaryschoolandtwoteachersineachIQS.ThenumberofteachersislowerinIQSsbecauseweanticipatedonlytwofacilitatorsonaverageperIQS.Hence,thetotaltargetedteachersamplesizeis360primaryschoolteachersand240IQSfacilitators.Asdiscussedbefore,weexpectthesamplesizeforpupilsandteacherstobeloweratthemidlinesurveyduetoattritionofthepanel.Table3presentsthesamplesizesforallschoolsandinstrumentsimplementedwithineachschool.
Table3: Sampleparametersfortheearlylearningintervention
Publicprimaryschools IQSs
States Katsina,Zamfara Katsina,Zamfara
Numberofschoolsperstate 60 60
Totalnumberofschools 120 120
Instrumentsperschool
Headteacherinterview 1 1
Teacherinterview† 3 2
Teachercompetencytest 3 2
Classroomobservation 3 2
Pupilinterview* 6girls,6boys 6girls,6boys
EnglishliteracyassessmentforP2 6girls,6boys 6girls,6boys
HausaliteracyassessmentforP2 6girls,6boys 6girls,6boys†Thesameteacherswillbeinterviewed,testedandobserveddeliveringlessons*Thesamepupilswillbeinterviewedandtested
Thesamplesizewasdeterminedbyevaluatingtheminimallevelofchangeintwokeyoutcomeindicators(thelearningoutcomeandtheteachingknowledgeandskillsoutcome)thatcanbedetectedwithstatisticalconfidence.Thisisreferredtoastheminimumdetectableeffect(MDE).Inarandomisedcounterfactualdesign,theMDEcorrespondstothedifferenceinthemeanoutcomeestimatebetweentheinterventionandcontrolgroups(singledifference)orthedifferencebetweenthebaselineandfollow-upestimatesoftheoutcomefortheinterventionandcontrolgroups(difference-in-differenceestimate).Box6presentstheMDEsforthelearningoutcomeindicatorandteacherknowledgeandskillsindicatorusingadifference-in-differenceestimatebasedonthepupilandteachersamplesizes.ThedetailsofthestatisticalpowerprinciplesusedaregivenintheGEP3EvaluationFramework.
EvaluationoftheGirl’sEducationProject3
©EDOREN 12
Box6:EstimationofMDEs
MDEsforpupillearningoutcomesInlinewiththeGEP3logframe,themainliteracyindicatorisdefinedasthepercentageofgirlsthatachievebasicliteracy(specifiedasHausaliteracy).Usedindeterminingtherequiredsamplesize,theMDEisafunctionofthebaselinevalueoftheoutcomeindicator.Atthetimeofsamplesizecalculationwedidnotknowthebaselinevaluesofthelearningoutcomeindicators,butlearningoutcomesinnorthernNigeriaaregenerallyverylow.WethereforeestimatetheMDEusingplausibleandmaximumscenarios.Furthermore,weassumeanintra-clustercorrelation(ICC)of0.3,arelativelyhighinter-temporalcorrelation(ITC)of0.8,giventhepanelapproach,80%power,anda95%confidencelevel.Wetakeintoaccount30%attritionofsampledpupilsbetweenbaselineandmidline.UsingaplausiblescenariothatthebaselinepercentageofpupilsthatachievebasicHausaliteracyisaslowas10%,weestimatethata3.8percentagepointnetincreasewouldbestatisticallydetectablewhenthedataofprimaryschoolsandIQSs,girlsandboys,areanalysedjointly.WhenprimaryschoolsandIQSsareanalysedseparately,butcombiningthedataofgirlsandboys,weestimatethatthisMDEwouldbe5.3percentagepoints.TheMDEestimatefordisaggregatedanalysisofgirlsandboysforeachtypeofschoolequals5.9percentagepoints.ThelargestMDEforagivensamplesizewouldbeproducedundertheunlikelyscenarioofabaselinevalueof50%ofpupilsachievingHausaliteracy.Insuchacase,theMDEamountsto6.3percentagepointsforcombined‘primaryschoolandIQS,girlsandboys’analysis,8.9percentagepointsforseparateprimaryschool/IQSanalysisbutcombininggirlsandboys,and9.8percentagepointsforafullydisaggregatedanalysis.Bycontrollingforsomecovariatesthatinfluencelearningoutcomes,forexampletheageofthepupil,weexpecttoreducethevarianceoftheestimatesaspartoftheanalysis.Thiswillallowustodetectsmallereffectsizesforagivenpowerandsamplesize.MDEsforknowledgeandskillsoutcomesTheteacherknowledgeandskillsoutcomeindicatoristhepercentageofteachersdemonstratingminimumteachingknowledgeandskills.The2013TDNAassessedteachercompetenciesforprimaryschoolsandIQSteachersinKatsinaandZamfara.80Whileresultsvariedfromstatetostate,a10%baselinevalueseemsplausible(andispotentiallyevenonthehighside).WeassumeanITCof0.8,anICCof0.3,80%power,anda95%confidencelevel.Wetakeintoaccount25%attritionofsampledteachersbetweenbaselineandmidline.Takingintoaccountthesamplesizeandaboveparametersweestimatethatwewillminimallydetectanetincreaseof4.7percentagepointsbetweenbaselineandmidline,assumingabaselinevalueof10%andjointlyanalysingprimaryschoolandIQSdata.TheMDEwillbehigherwithdisaggregatedanalysis,i.e.approximately6.4to6.7percentagepointsforprimaryschoolsandIQSsrespectively.Assuminga50%baselinevaluescenario,whichreturnsthehighestpossibleMDEforagivensamplesize,theMDEwouldbe7.8percentagepointsinthecaseofaggregatedanalysisofprimaryschoolsandIQSsandapproximately11percentagepointsinthecaseofdisaggregatedanalysis.InlinewiththeGEP3logframe,weanticipatetheincreaseinteachercompetenciestoberelativelylargesinceteacherswillbethedirectbeneficiariesofintenseteachertrainingandmentoring.
3.2.3.4 Samplingdesign
80Johnson,D.andHsieh,P.J(2014)
EvaluationoftheGirl’sEducationProject3
©EDOREN 13
Themainsamplingmethodforschoolswasasingle-stagestratifiedsystematicrandomdraw,whileteachersandpupilsweredrawnthroughatwo-stagestratifiedsystematicsample.Measurementswereperformedbothatschoolaswellasatthepupilandteacherlevels,asdescribedingreaterdetailintheinstrumentssection(seeSection3.2.5).
Twotypesofschools(publicandIQS)wereusedasexplicitstrata.Thesamplingdrawwasfullyseparateforeachtypeofschool.SecondaryexplicitstrataweredefinedbytheLGA.AnequalnumberofschoolswasdrawnfromeachLGA.Withineachexplicitstratumanimplicitstratificationwasbasedontypeoflocation(urban,rural)andwithineachtypeoflocationonEuclidiandistancefromAbuja.ThedistancefromAbujawasusedasaproxyforthegeographicaldispersionofthesamplewithineachLGA.ThedistancetoAbujawascalculatedusingGPScoordinatesprovidedwithinthesamplingframe.81AbujawaschosenasanarbitrarylocationthatwasfarenoughawaytoallowforgeographicaldispersionofEuclidiandistanceswithintheLGAs.
Thesystematicrandomsamplingusedarandomstartandarealsamplingstep.Implicitstratificationwasachievedbysortingwitheachexplicitstratum.
IneachLGA20publicand20IQSswererandomlydrawn.Inaddition,twoschoolspertypeperLGAweredrawnasinitialreplacementschools,toprovideapoolofschoolsthatcouldbesurveyedincaseanyoftheschoolsinthemainsamplewerefoundtobeineligibleorinaccessibleduringthefieldworkperiod.Therefore,thesamplingsetuphadprovisiontoselect22schoolsofeachtypeinanygivenLGAinZamfaraandKatsina.
Thereplacementschoolsweremarkedafterthesamplehadbeendrawn.AuniformlydistributedrandomvariablewasusedtosorttheschoolswithineachtypeandLGAandthelasttwoschoolsinthissequenceweredesignatedasreplacementschools.Similarly,therandomvariablewasusedtodesignatethereplacementsequenceamongthedesignatedreplacementschools.
DuringfieldworkmanyoftheIQSswerefoundtobeineligible.Thereasonforineligibilitywasmainlynon-integrationorpoorintegrationofschools.Forthispurpose,additionalreplacementschoolshadtobesampledduringthefieldwork.Thedrawingofthereplacementschoolsmostlymirroredthemainsamplingapproach.InsomeLGAsthenumberofeligibleGEP3IQSsbecamequitelimitedduetohighproportionsofineligibility.Inthesecasesalloftheavailablereplacementswerereleasedtothefieldforthesurveyteamtoselectreplacementsinanorderthatcorrespondedtoapre-assignedrandomsequenceofrelease.
MapsofthesampledschoolsincludedinthestudyarepresentedinAnnexB.
Schoolentry
Thesamplingoftheteachersandpupilswascarriedoutbythesurveyteamsattheselectedschools.Therefore,schoolentrywasimportantinestablishingarapportandestablishingthecredibilityofthedatacollectionatschools.Italsoimprintedtheimportanceofrigorousselectionofalltherespondentsatschoolsandwastheoccasionforpresentingtotheheadteacherthesamplingprocedurestobeusedinpupilandteacherssamplingapproaches.Figure3presentstheschoolentryprotocolthatwasfollowedbythesurveyteams.
81GPScoordinateswereonlyavailableinthesampleframefortheearlylearninginterventionbaseline(KatsinaandZamfara).TheywerenotavailableinthesampleframefortheIQSSbaseline(NigerandBauchi)
EvaluationoftheGirl’sEducationProject3
©EDOREN 14
Figure3: Schoolentryprotocol
07:30Arrivalattheschool
Isschoolopen?
Presentthewrittenconsentformtotheheadteacherandseek
consenttoproceed
NO ContactStateCoordinator
ConsentisGiven NO Contact
StateCoordinator
YES
Sampleteachers
Samplepupils
Proceedwithinterviews
YES
Introduceyourselftotheheadteacherandpresenttheauthorisationletter
Samplingofteachers
Afterconsentwasgivenbytheheadteachertoperformthestudyattheselectedschooltheteamproceededwiththesamplingofteachers.ThesamplingofteachersfollowedtheprocessvisualisedinFigure4.
InordertoensurethatonlyeligiblesampledteacherswereincludedintheGEP3baselinestudy,thesupervisorexplainedtotheheadteacherexactlywhichteacherswereeligibleforinclusioninthesurvey.TheeligibilitycriteriaaredescribedinmoredetailinSection3.2.3.4.
Thesamplingofteacherswasconductedontabletsusingcomputer-assistedpersonalinterview(CAPI)software.ACAPIformwasusedtoinputalltheeligibleteacherspresentintheschoolontheday.TheCAPIselectionformselectedtherelevantteachers.Ingeneral,theheadteacherwasnotconsideredtobeeligibleforteachersampling,eveniftheheadteacherteachesherself/himself.IncaseswheretheIQSdidnothaveanadequatenumberofteachersteachingnon-religioussubjects,theheadteachercouldalsobeincludedintheteachersurvey.
EvaluationoftheGirl’sEducationProject3
©EDOREN 15
Figure4: Teachersampling
TalkwiththeHeadTeacher
Public
Askforteacherattendanceregistersforthedayofthe
interviewforteachersteachinginP1-P3
YES
Check:• Itisforthecurrentday• Itcoversteachersofthecorrectgrade• Itcoversteachersteachingnon-religious
subjects
Areanyattendancerecords
available?NO
RequesttheHeadTeachertoidentifyallteachers(includingvolunteers)teachinggrades
P1-P3
Arerecordscomplete?
YES
EnterthenamesofalltheteachersinthelistsornamedbytheheadteacherintoasamplingForm(CAPI)
NO
Identifythemissinggroupsofteachersormissingnamesofindividualteachers(includeanyvolunteerteachers)
IstheschoolpublicorIQS? IQS
AretheclassestaughtingradesP1-P6?
YES
Askforteacherattendanceregistersforthedayofthe
interviewforallteachersteachingP2orequivalentpupils
NO
Areanyattendancerecords
available?
YES
RequesttheHeadTeachertoidentifyallteachers(includingvolunteers)teachingP2or
equivalentpupils
NO
Foreachteacherspecifywhetherteacherteachesreligiousornon-
religioussubjects
CAPIselectsteachersrandomly:• 3teachersinpublicschools• 2teachersinIQS
Inviteselectedteacherfortheinterview
Istheteachereligibleandhasgiven
consent?
Administertheinterview,classroomobservationandtestto
theteacher
EnterthenamesofallselectedteachersonSchoolCompletion
Form
YESNO
Samplethereplacementteachers
EvaluationoftheGirl’sEducationProject3
©EDOREN 16
Samplingofpupils
Thesamplingofpupilswascarriedoutintwodistinctphases:
1. Consultationwiththeschoolrecordsandtheheadteacher,whichmostcommonlytookplaceintheheadteacher’soffice.
2. Physicalselectionofpupils,whichmostcommonlytookplaceintheclasswherethepupilsweretaught.Ifthepupilswerespreadacrossmultiplestrandsofthesamegrade,allthechildrenforthatthegradeweregatheredtogetherandtheselectionwasperformedonthefullcontingentofpupils.
ThefirststageofthepupilsamplingfollowedtheprocesssetoutinthediagraminFigure5.
Figure5: Pupilselection–Headteacher’soffice
TalkwiththeHeadTeacher
Askforpupilattendanceregistersforthedayoftheinterview
YES
Check:• Itisforthecurrentday• Itcoversthecorrectgrade(P2orequivalent)• Itcoversallarmsofthegrade• Itcoversbothboysandgirls
Areanyattendancerecordsavailable? NO
RequesttheHeadTeachertoidentifyallpupilsattendingP2orequivalentgrade
Arerecordscomplete?
YES
Proceedtotheclassroomwiththelistandusetheteachertohelpidentifyall
thepupilsonthelist
NOIdentifythemissinggroupsofpupilsormissingnamesofindividualpupils
ThepupilswhowereeligibleforassessmentinthisstudywerethoseatP2orequivalentlevel.Itwasimportantforthesupervisortomakesuretheheadteacherthoroughlyunderstoodwhichpupilsweresampled.
Oncetheeligiblepupilswereidentifiedthesecondstageofthepupilselectiontookplaceinclass.Girlsandboyswererandomlyselectedbygirlsandboysseparatelydrawingchitsfromanopaquebag.TheprotocolforthisprocessisdescribedinFigure6.Insmallerschoolstheteamswerenotabletofind12pupilsattendingP2orequivalentgrade.Inthisinstance,allthepupilsthatwereavailableinschoolonthedaywereinterviewed.
EvaluationoftheGirl’sEducationProject3
©EDOREN 17
Figure6: Pupilselection–classroom
Line-upalleligiblepupils–makesureyouhaveallchildrenlined-upifmorethanonestrandofthe
gradeistaught.Splitthemaccordingtogender
Organize2opaquebagsofchits,1forgirlsand1forboys
Eachbagshouldcontain:• 6markedchits• (Numberofboys/girls–6)un-
markedchits
Eachpupilpicksupachitfromthebag
Separateselectedchildreninonepartoftheclassroomandalerttheteacherthatthepupilshavebeen
selected
Inviteselectedchildforthetest
Isthechildeligibleandhas
givenconsent?
Arrivetotheclassroomandexplainselectionprocesstothe
teacher
YES
Administerthetesttothepupil
YES
Prepareforsamplingofreplacementpupils.
Eachbagshouldcontain:• (Numberofreplacedboys/
girls)ofmarkedchits• (Numberofboys/girlsnot
selected–numberofreplacementsneeded)un-markedchits
EnterthenamesofallselectedchildrenonSchoolCompletion
Form
NO
Haveyouexhaustedallinitially
selectedpupils(12)?
NO
3.2.3.5 Weighting
Samplingweightswereconstructedtoreflectthecomplexsamplingdesign.Theweightswereestimatedforalllevelsofanalysisandreflectthemulti-levelnatureoftheschooldata,withpupilsandteachersnestedwithinschools.Theestimatedweightsarepopulationweights,whichsumuptothetotalpopulationsizeasdefinedbytheuniverseandthesamplingframe.Theweightsoftreatmentandcontrolgroupswereestimatedseparatelyandeachsumuptothetotalpopulationlevel.Duetotheveryhighsamplingfractionrate(i.e.thelargeproportionoftheeligibleschoolssampled),finitepopulationcorrectionfactorswerealsoestimatedtoadjustthevarianceestimates.AdetailednoteonweightingcanbefoundinAnnexC.
3.2.4 Randomisationprocess
Randomisationwasperformedonthesampledschoolsinadoubleblindedfashion:neithertheschoolnorthesurveyorganisationwereawareoftherandomassignment.
EvaluationoftheGirl’sEducationProject3
©EDOREN 18
Astratifiedrandomisationapproachwaschosenoverasimplerandomisationinordertoensureabalancebetweentreatmentandcontrolschoolsonkeycharacteristics.RandomassignmentwascarriedoutforeachLGAandtypeofschool.Furthermore,thedistanceoftheschoolfromtheLGAcapitalwasusedforstratificationtoensurebalancebetweentreatmentandcontrolgroups.AscanbeseenfromthemapsinAnnexB,thetreatmentandcontrolschoolsaresimilarlyspreadoutgeographically,whichindicatesthatthegeographicalstratificationworked.
FivestrataonthedistancefromtheLGAcapitalweredefinedbasedontheEuclidiandistancebetweeneachschoolwithintheLGAandtheLGAcapital.Theschoolsweregroupedintofivegroups,accordingtotheestimatedquintilesatthenationallevel.QuintileestimationatthenationallevelensuredthatthedefinitionandthemeaningofeachofthequintileswasequivalentacrossdifferentLGAs.Forthisreason,notallquintileswerepresentineachLGA.Forexample,inasmallerLGAthemostdistantquintilegroupmaybemissing.
Stratifiedassignmentrequiresthatthesamplingunitsarerandomlyallocatedtotreatmentandcontrolschoolswithineachstratum.Itisthereforerequiredthatanevennumberofschoolsbeallocatedtoeachstratum.Inordertoachievethis,asimpleadjustmentprocesswasrun,wherebyschoolsweresystematicallyshiftedbetweentwoneighbouringdistancestratauntilallthestratainalltheLGAshadanevennumberofunits.
Thefinalrandomallocationwasperformedintwophases.Inthefirstphaseeachschoolwasallocatedtooneofthetwodesignatedgroups.Inthesecondphaseeachofthegroupswaslabelledasthetreatmentandcontrolgroup,basedontherandomdraw.
3.2.5 Instrumentdesignandpreparation
3.2.5.1 Instrumentdevelopmentprocess
Theprocessofinstrumentdevelopmentwasbasedonmultipleiterationsandfieldtesting,tocontextualisetheinstrumentsbyreferencetotherealityoftheGEP3schools.ThiswasparticularlyimportantforIQSssincethesearenotformalschoolsettings.ThekeystepsinthisprocessaredepictedinFigure7below.
Figure7: Instrumentdevelopmentprocess
Deskresearch:AdeskreviewoftheexistingevidenceonIQSswascarriedout,toguidethedraftingofmeasurementtoolsandsamplingprotocolswithinthisunder-researchedcontext.Inaddition,we
Deskresearch Pre-piloting Draftinstruments Piloting Finalisationof
paperversions
TranslationCAPIprogrammingCAPIpilotFinalCAPI
instruments
EvaluationoftheGirl’sEducationProject3
©EDOREN 19
reviewedgoodpracticesforeducationmeasurementandexistingeducationsurveyinstrumentsusedinNigeria.
Pre-pilotingandfieldresearch:BasedontheGEP3EvaluationFramework,andinformedbythedeskresearch,asetofexploratorytoolswasdevelopedandtrialledinthefield.InDecember2014preliminaryinstrumentsforinterviewswithheadteachers,CBMCs,pupils,andteachersweretestedinsixschools(IQSsandprimaryschools)inNigerState.Furthermore,thefieldvisitsprovidedabetterunderstandingoftheIQSsystem,thedivisionofrolesandresponsibilitiesbetweenmultipleactors,andtheapproachtobeusedforsensitivequestions,suchasthoserelatingtoschoolexpenditureandfinancing.GroupdynamicsinSBMC/CBMCfocusgroupswerealsoexploredanddocumented.
Draftinstruments:Theinformationgatheredduringthepre-pilotingandfieldresearchwasusedtodevelopdraftinstrumentsappropriatefortheGEP3baselineandapplicabletotheparticularcontextofIQSs.Theinstrumentsdevelopedwere:(i)inlinewiththekeyindicatorsidentifiedintheGEP3ToCandEvaluationFramework;(ii)sensitivetowardsthecontextofIQSs,includingtheschoolstructure,termdates,andterminology;and(iii)sensitivetowardstheculturalcontextofNigeria,andnorthernNigeriainparticular.Atotalofsevenquantitativeinstrumentsweredevelopedforthebaselineoftheearlylearninginterventionevaluation(seeSection3.2.5.2below).
Pilotingexercise:ThedraftinstrumentswerethenpilotedinonepublicprimaryschoolandthreeIQSsinJune2015.Thepilotingexercisehelpedhighlightadditionalchangestoimprovetheinstruments,suchasthefinalisationofresponsecategoriesforclosedresponsequestions,andthelanguagetobeusedtocommunicatewiththeIQSheadteachers.Inaddition,specialattentionwaspaidtotheinteractionwithchildren,todevelopmodulesthataremostlikelytoproducereliableresults.ThefeedbackfromthelocalresearcherswasinvaluableinthisphasebecausetheyhadadditionalinsightsintotheIQSsystemandthecontextofnorthernNigeria.
Thepilotingexercisewasalsousedtotestsamplingprotocolsforchildrenandteachers.Theresearchteamtrialledseveralsamplingmethods,notingthestrengthsandweaknessesofeachmethod.Thisexercisehelpeddevelopfinalsamplingprotocolswhichminimisedtheriskofsamplecontaminationandrespondentanxiety.Thelatterwasespeciallyimportantgiventhenatureoftheinteractionwiththechildren.
Finalisationofinstruments:Theinstrumentswerefinalisedbasedonthepilotingexerciseandadditionalfeedbackfromthelocalresearchers.
Translation:AfterfinalisationtheinstrumentsweretranslatedintoHausa,thelanguagespokenbythemajorityofthepopulationinthefiveGEP3states.ThetranslationwasqualitycheckedbytheEDORENNigeriaofficetoensureitsappropriateness.
CAPIprogramming:ThetranslatedandfinalisedinstrumentswereprogrammedforuseinCAPIontabletsintwolanguages,EnglishandHausa.Theprogrammedquestionnaireswereextensivelycheckedforcoherencewiththepaperversionsbeforebeingtakentothefield.
CAPIpilot:ThefinalCAPIquestionnaireswerepilotedinsixschoolsinAugust2015inNigerstateinordertooptimisethestructureoftheCAPIquestionnaireandtheinterviewflow.Thepilotalsomadeitpossibletofurthertestproperschoolentryandfieldtimemanagement.
EvaluationoftheGirl’sEducationProject3
©EDOREN 20
3.2.5.2 Overviewoftheinstruments
Seveninstrumentsweredevelopedtocollectquantitativedataateachearlylearninginterventionschoolandeachcontrolschool/IQS(seeTable4Error!Referencesourcenotfound.).
Table4: Overviewofthedatacollectioninstruments
Instrument Summary
Pupilquestionnaire
Thepupilquestionnaireprecedesthepupilassessments.Itchecksforpupildisabilityandcollectsdataonpupilhouseholdassets.Inadditionthepupilquestionnairecollectsdataonpupilbackgroundcharacteristics,suchasgender,yearsinschool,ifthepupilattendsanyotherschooletc.Pupilsidentifiedashavingadisabilityarenotgivenassessmentitemsthatrequirethatparticularability,inordertorespondmeaningfully(forexample,apupilwithanidentifiedinabilitytoseeisnotaskedtocompletethereadingitems).
PupilEnglishLiteracyassessmentforP2
TheEnglishLiteracyassessmentcontains13items,witheachitembeingmadeupofseveralsub-items.WhiletheEnglishLiteracyassessmentforGEP3istargetedatadifferentproficiencylevelthantheESSPINassessments,bothESSPINandGEP3havebeencalibratedontothesameEnglishLiteracyscale(theEDORENscale)throughlinkitems.ThismeansrobustcomparisonsbetweenESSPINCompositeSurvey(CS)resultsandGEP3evaluationresultscanbemade.Theassessmenttestsarangeofliteracyknowledgeandskillsacrossthepre-literacy,emergingandbasicliteracyranges.Knowledgeandskillsincludeletterrecognition,phonologicalknowledge,printconcepts,oralliteracy,verbalcomprehension,initialsoundsandletters,readinghighfrequencywords,verbalandwrittengrammar,writinghighfrequencywords,readingfluency,copyingandspellinghighfrequencywords.
PupilHausaLiteracyassessmentforP2
TheHausaLiteracyassessmentisdesignedtotestthesameliteracyknowledgeandskillsastheEnglishLiteracyassessment.Itemsarenotmerelytranslated,butratherparallelitemsaredevelopedtotestsimilarconceptswhenappliedtotheHausalanguage.
Teacherquestionnaire
Theteacherquestionnairecollectsdataonteachers’backgroundcharacteristics,absenteeism,trainingandremuneration.Italsoincludesquestionsonmeetingsandsupervisionfromtheheadteacherandamoduleonteacherperceptionsandattitudes,whichwillbeusedtodevelopameasureofteachermotivation.
Teacherknowledgeandskillsassessment
Theteacherknowledgeandskillsassessmentisdividedintothreesections,collectivelycomprising30items,includingmultiplechoice,shortresponseandlongresponseitems.ThisincludesmarkingpupilresponsestoHausaliteracyquestions,indicatinggradelevelsforitemsbasedoncurriculum,developingananswersheetforareadingtestaimedatGrade2pupils,answeringa
EvaluationoftheGirl’sEducationProject3
©EDOREN 21
3.2.6 Surveyfieldwork
3.2.6.1 Surveyimplementationandmanagement
ThesurveyfieldworkwasimplementedbyOPMNigeriabetween19October2015and20November2015.Itwasprecededbyathree-weektrainingperiodforsurveystaff(seebelow).
ThesurveystaffwererecruitedbyOPMNigeria.Statesurveycoordinatorsandqualityassurancemonitorswereselectedfromapoolofhighlyexperiencedcandidates.TheselectionwasmainlybasedontheirexperiencewithothereducationsurveysinnorthernNigeria.82Otherstaffwereselectedthroughcompetitiverecruitment,whichincludedcanvassingthepotentialcandidates’CVsandface-to-faceinterviews.SpecialattentionwasgiventoexperienceinconductingsurveysinNigeria,withadditionalemphasisgiventoanyeducationresearchexperience.Theselectionprocessalsotookintoconsiderationthegenderbalanceoftheteams,andespeciallytheirlocallanguageproficiency.AllrecruitedinterviewerswereproficientinbothspokenandreadHausa,whichwasthemainlanguageusedforsurveyadministration.OtherlocallanguagerequirementswerealsotakenintoconsiderationwhenselectingteammembersasinsomeareasHausaisnotthemainlanguagethatisspoken.Inselectinginterviewers,specialattentionwasgiventotheknowledgeofArabic,Fulfulde,NupeandGbagyilanguageproficiencyastheselanguagesareusedinthesurveyedareas.
ThesurveywasadministeredusingCAPI.Datawerecollectedusingultra-portablecomputertabletsandtheCSProsoftwareplatform.
ThesurveywassupervisedbytheEDORENSurveyTechnicalLeadthroughoutthesurveyimplementationanddatacleaningprocess.TheEDORENSeniorEducationSpecialistandQuestionnaireDevelopmentExpertattendedthetraining.TheEvaluationTeamLeaderwasinvolvedinanydecisionthataffectedthesurveydesign:forexample,thereplacementofsampledIQSsduetochallengeswiththeIQSsampleframe.
82Statesurveycoordinatorsandqualityassuranceofficerswereinvolved,amongothers,indatacollectionaspartoftheTDPevaluation,ESSPINCS2andChildDevelopmentGrantProgrammeevaluation.SeveralofthemhadparticipatedintheGEP3instrumentpre-testing.
readingcomprehensionquestion,andmakingjudgementsaboutpupils’writing,includinggrammar,spellingandadditionalsupportneeded.
Teacherclassroomobservation
Theclassroomobservationrecordsthefrequencywithwhichtheenumeratorobservesspecificteachertalk,teacherlanguageuse,teacheractionandpupilaction.Theenumeratoralsorecordsteacheractionatthestartofthelesson,actionattheendofthelesson,theresourcesusedandsubjectobserved.
Headteacherquestionnaire
Theheadteacherquestionnaireincludesseveralsectionsrelatingtobackgroundinformationontheschool,basicinformationabouttheschool,schoolmanagementandmonitoring,attendanceandenrolment,attitudestowardsintegrationforIQSsandschoolinfrastructure.
EvaluationoftheGirl’sEducationProject3
©EDOREN 22
3.2.6.2 Surveytraining
Toensurethatallsurveyteamscoulddotheirworkeffectivelywhilstmaintaininghighqualitystandardsallsurveyfieldstaffwereintenselytrainedbetween28Septemberand17October2015inAbuja,includingpracticefieldvisits.Thesurveytrainingwasdividedintotwoparts:thefirstwasforthestatesurveycoordinatorsandqualityassuranceofficers,whichlastedfrom28to30September2015,whilethesecondwasfortheenumerators,from02to17October2015.Thelatterwasfurthersplitintotwobatchessoastoenabletheenumeratorstohaveadeepunderstandingofthesurveyinstrumentsandprotocols.Attheonsetofthetrainingtherewasnodistinctionbetweentheroleofthesupervisororthatoftheenumerators.Alltrainingparticipantsweretrainedasequals.Uponcompletionofthetraining,themostexperiencedandbestperformingenumeratorsweredesignatedasteamsupervisors.Specialsupervisorbriefingsessionswereheldwiththeminordertostrengthentheirunderstandingoftheirrolesandresponsibilities.
Adetailedtrainingmanualwaspreparedforthetraining.Duringthetrainingthemanualwasupdatedbasedonthefeedbackreceivedandclarificationsmadeduringthetraining.
Insummary,themainsubjectscoveredbythetrainingsyllabuswerethefollowing:
• OverviewofGEP3:GEP3’sobjectives,implementationstates,itsduration,anditsrelationshipwithEDORENandUNICEFwaspresentedtotheparticipants,forthemtoreinforcetheirunderstandingoftheprogramme.
• Surveyprotocoloverview:thesurveyandschoolselectionprotocolswereexplainedindetailandincludedin-depthexplanationsofthedifferentinterventiontypesandschooltypesunderGEP3,inorderforthefieldteamstothoroughlyunderstandthecontextoftheirdatacollectionenvironment.
• Communication(reportinglineandfeedback):theflowofcommunicationbetweenstatecoordinators,qualityassuranceofficers,teamsupervisors,andenumeratorswasdiscussedintermsofreportingandroutinefeedback.
• Familiarisationwithsurveyinstruments:participantswereexposedtoeachofthesurveyinstruments,includingthemainquestionnaireandCAPI.
• Mentoringandongoingtraining:thiswasfacilitatedforstatecoordinatorsandqualityassuranceofficers,forthemtogainanunderstandingofhowtoprovidementoringandsupporttointerviewersduringfieldwork,exceptinthecaseofmajorissuesthatwouldexceedtheirknowledgeandauthority.
• Fieldworkmonitoringforms:Inordertomanagefieldworkeffectively,aswellastomonitoractivitiesconducted,monitoringformsweresharedwiththefieldteam.Theseincludedtheschoolcompletionforms,monitoringactivityformsanddailyreporttemplates.
• Qualityassuranceprinciples:Informationaboutfieldethicsandprotocolsthatguaranteedataqualityweresharedwiththefieldteam.
Thetrainingusedseveraldifferenttrainingapproachestoachievethemaximumtrainingeffect,suchasdetailedinstrumentreview,smallgroupworkandmockinterviews.Enumeratorswerealsoexposedtoreal-lifefieldexperienceearlyoninthetraining.Thefirstvisitwasorganisedwithinthefirstfivedaysoftraining,andwasfollowedupbytwomorefieldvisitsduringthetraining.Earlyexposuretoreal-lifefieldsituationscontextualisesthetopicsandapproachesdiscussedinclassandhelpsenumeratorstobettergraspconcepts.ThetrainingwasconductedexclusivelyusingCAPIversionsofthequestionnaires.Enumeratorswerealsotaughtaboutotherfunctionalitiesrelatedto
EvaluationoftheGirl’sEducationProject3
©EDOREN 23
computer-assisteddatacollection,suchashowtosynchronisefilescontainingfilledquestionnairestothecloudandotherITanddatamanagement-relatedissues.
3.2.6.3 Fieldworkorganisation
Afterbeingtrained,thesurveyfieldteamwasoriginallystructuredasfollows:
• fourteamscomprisedofonesupervisorandfourenumerators;• onefieldcoordinatorperstate(statesurveycoordinator);• onequalityassurancestaffmemberperstate;and• OPMNigeriacoreteammemberswerepresentinthefieldtoprovidesupport,supervision,and
mentoring.
However,basedonthespecificneedsandparticularitiesofeachstate,theteamstructurewasprogressivelymodified.Asaresult,oneteamfromBauchiandonefromNiger(seethebaselinesurveyoftheIQSSevaluation)thatwerefacingahighrateofschoolineligibilitywerereallocatedtoZamfaraandKatsinarespectively.Thus,bythethirdweekoffieldworkZamfaraandKatsinawereoperatingwithfiveteamseach.Eachteamwasalsoaccompaniedbyadriver.
TheSurveyManagerandFieldManagerwereresponsiblefortheoverallmanagementoftheGEP3datacollection.TheEDORENSurveyTechnicalLeadprovidedtechnicalsupportandoversight.Statesurveycoordinatorsfocusedonthestatelogistics,schoolpre-schedulingandarrangementoftheschoolvisitswhilethequalitycontrolteamwastaskedwithoverseeingprotocoladherenceandefficiencyofdatacollection.
Figure8: GEP3surveyteamstructure
3.2.6.4 Unitssurveyed
Usingtheeligibilitycriteriamentionedpreviously,theteamsurveyed1,389pupilsand299teachersfrom120publicprimaryschoolsinKatsinaandZamfara,withonlyonereplacementinZamfaraofa
EvaluationoftheGirl’sEducationProject3
©EDOREN 24
schoolthatdidnothaveaP2class(SeeTable5).83Morethan100%oftheschoolsamplesizewasachieved,whilethepercentageofsampledpupilsandteachersfromthesurveyedschoolsequals96%and84%,respectively.Additionally,inZamfaraandKatsina1,264pupilsand178teachersweresurveyed(88%and74%,respectively,ofthetargetedsamplesize)in120IQSs.84Therelativelylowerpercentageofteacherssurveyedcomparedtothetargetedsamplesizewasduetothefactthatsomeschoolsdidnothavethetargetednumberofteachers,whichwasparticularlythecaseintheIQSsinKatsina.
22%oftheoriginallysampledIQSshadtobereplaced,mainlyduetotheIQSsnotbeingconsideredeligibleforthesurveygiventhestudyuniverse.85ReplacementsweremostlyduetotheIQSsnotteachingnon-religioussubjectsyet/anymore,ortheirbeingrecentlyintegrated.86Asexplainedabove,areplacementprocessinlinewiththesampledesign—avoidingtheintroductionofbiasinthisprocess—wasplannedfor,todealwiththechallengeswiththesampleframe.OnlyoneIQSproprietorrefusedtobepartofthesurveyandhadtobereplacedaccordingly.
Table5: Summaryofthesurveyachievement
Publicprimaryschools IQSsKatsina Zamfara Katsina Zamfara
Schools
Targetsampleschools 60 60 60 60
Totalschoolsurveyed 60 60 65 63
Originallysampledschoolssurveyed 60 59 50 44
Replacementschoolssurveyed 0 1 15 19
Totalschoolscontacted 60 61 90 87
Contactedschoolthatwerenotsurveyed–reasons
Recentlyintegrated 8 1
Non-integrated 3 20
Non-existing 3 0
Nogirls 0 1
Other 1 11 2
Percentageofsampleschoolssurveyed 100% 100% 110% 102%
83ItwasreportedthattheschoolenrolledpupilsonlyonceeverytwoyearssotherewerenocurrentP2pupilswhentheteamvisitedtheschool.84EightextraIQSsweresurveyedbutthesehavenotbeenincludedinthestudybecausethenumberofstudytreatmentschoolshadbeenagreedwiththeimplementingpartner.85ForanIQStobeconsideredeligible,ortoqualify,forthesurveyitwasrequiredthat:1)theschoolmusthaveaP2equivalent;2)itmusthaveteachersofnon-religioussubjectteachingP1–P3;3)itmusthavebeenintegratedforapproximatelyayear;and4)itmusthavefemalepupils(wedidnotimposethecriterionthattheIQShadtohave40%girlsenrolledbecausethiswashardtoverifyandmayhavebeentoorestrictivetoattainthesamplesize).86ThelengthofintegrationintheIQSswasparticularlyanissueinBatsariLGAinKatsina,whereapproximately80%ofschoolsintegratedin2015.Inthiscasetheeligibilitycriteriaofone-yearintegrationwasappliedmoreflexiblybysurveyingtheIQSsthathadactuallystartedintegrationinJanuary2015–hencehaving10monthsofintegration.
EvaluationoftheGirl’sEducationProject3
©EDOREN 25
Pupils
Targetsamplepupils 720 720 720 720
Percentageoftargetpupilssurveyed 96% 97% 84% 92%
Teachers
Targetsampleteachers 180 180 120 120
Percentageoftargetteacherssurveyed 83% 83% 59% 89%
Source:Fieldworkreports
3.2.6.5 Interactionwith,andinterferenceby,externalbodies
InteractionbetweenUNICEFandGEP3teamsinthestatesandthesurveyorganisationwasestablishedveryearlyintheprocess.TheGEP3state-levelofficialsparticipatedintheenumeratortrainingfortwodaysinordertosharetheirexperienceaswellastheirexpectationsregardingtheprotocolandconduct.Duringthetrainingperiodstatecoordinatorsestablishedarapport.Ineachstate,thestatesurveycoordinatorsharedthesurveyworkplanwiththeUNICEFfocalpersonandotherfieldmonitoringpersonnel,suchaslocaleducationauthorities,SUBEBs,andSAMEs.Thisensuredthattherelevantgovernmentofficersagreedtoandsupportedthedatacollection,andintroducedthesurveyteamtotheschools.ThesurveyteamreceivedformallettersofintroductionfromtheLGEAandUNICEF.
Insomecases,representativesofgovernmentagencieswerefoundtohavevisitednon-eligibleIQSsbeforethestatesurveycoordinatordid,toupdatetheschoolrecordsand/ortoputinplaceteachersofnon-religioussubjects.ThiswasparticularlyexperiencedinBunguduLGAinZamfara.Interferencefromeducationofficerswasalsofacedbythefieldteam.Inoneoftheschools,forexample,theeducationofficertriedtoinfluencetheteamtosurveyaschoolthatwasfoundtobeotherwisenoteligible.Anotherexample,occurredinanLGAwheretheeducationofficertriedtomakesurethataschoolwassurveyedeventhoughitwasanadulteducationschool.
3.2.6.6 Fieldworkchallenges
Themostrelevantchallengesfacedduringfieldworkaresummarisedbelow:
• TeachinghoursatIQSs:IQSsdonotusuallyfollowtheirtimetablesstrictly.Inaddition,someclassesaretaughtearlyinthemorningandsomelateatnight.Furthermore,mostoftheIQSsinthesurveyedstatesoperateforjusttwohours.Thisrequiredthesurveyteamstomakedetailedenquiresregardingtheday’splanandfortheteamstosetoutveryearlyorlate,oftenadministratingtheheadteacher,teacherandCBMCinterviewbeforethearrivalofthepupilsforsampling.
• Distanceanddifficultterrain:AlargenumberoftheIQSsarelocatedfarawayfromthenearestmetropolitanareasotheteamshadtosetoutveryearlyinordertogettotheschoolattherighttime.Moreover,mostofthecommunitiesdidnothaveaplacewheretheteamcouldspendthenight,whichmeanttheyhadtoreturnlateandhadtostartmovingtoanotherschoolthefollowingdayasearlyaspossible.
• Pre-appointment:SchedulingappointmentswiththeIQSswithoutaphysicalpresencewasalmostimpossible,asmostofthecontactnumberswereusuallynotreachableorwereincorrect.
EvaluationoftheGirl’sEducationProject3
©EDOREN 26
• Harvestseason:Insomecommunities,pupilsdonotattendschoolduringharvestseasonbecausetheyareengagedattheirparents’farms.
• IneligibleIQSs:Asdiscussed,anumberofIQSswereconsideredineligibleforthesurvey.Thesurveyteamidentifiedsomecasesinwhichproprietorswerealteringdocumentstopretendthattheirschoolswereeligible,whentheywerenot.Sometimes,anentirecommunitywasinvolvedinthismanipulation.
• Pupilsampling:InIQSstherewereinstanceswhereteacherstriedtoaddpupilstothesurveywhowereinothergradesorwhoattendedpublicprimaryschoolsfortheirnon-religiouscurriculumandwereonlytakingreligiousclassesattheIQS.
• Changeinschoolname:Duringschoolvisitsitwasobservedthatsomeschoolshadanamethatdifferedfromwhatwascapturedinthesampleframe,eitherduetoachangeintheschoolnameorwronglyspeltnames.
3.2.7 Dataqualityassurance
3.2.7.1 QualityassurancewithinaCAPIquestionnaire
Computer-assisteddatacollectiongenerallyimprovesthequalityofdatacollectedinasurveybyprovidingconsistentdatalinkage,automatedrouting/skips,andconsistencycheckswhichcanprovideinstantfeedbacktothefieldworkers.CAPIfurthermorereducesentryerrorsandfieldworkers’potentialtocheat.Decreasedinterviewtimealsoincreasestheefficiencyoffieldwork.Immediateavailabilityofdataallowsbetterorganisationandclosermonitoringoffieldwork,andenablesdatamanagerstoprovidetimelyfeedbacktothefieldteams,whichhelpstopreventfutureerrors.
Consistentdatalinkagewasmaintainedthroughoutthedatacollectionbyencodingthedatastructureintotheinstrumentsandhardcodedlistsofidentifiers.Thisisparticularlyimportantincomplexsurveyswithmultipleinstruments,suchasthisone.Thisallowsforsoundmappingandlinkingofdifferentlevelsofobservationsorcases,aswellasearlyidentificationofmissingcases.
Automatedskipsandroutingbehaviour,aspartoftheCAPIquestionnaire,ensuredthatonlycorrectquestionswereadministeredandthatnoanswersweremissing.Interviewerswereabletofullyfocusontheinteractionaspectsoftheinterviewandthuswereabletoestablishacloserbondwiththerespondent.Automatedskippingpatternsincreaseinterviewingspeedandallowmorecomplexskippingpatternsthatcansourceinformationfromallpartsofthequestionnaireinordertoadministerveryspecificsetsofquestionstailoredtoveryspecificcases.
Dataconsistencycheckswerebuiltdirectlyintothesurveyinstrumentstoprovidein-interviewfeedbacktothefieldworkers,allowingthemtoaddressinconsistenciesastheyaroseduringinterviews.Consistencychecksrangedfromsimplequestionspecificchecks,likerangechecks,tocomplexcheckscomparinginformationacrossdifferentsections.
CAPIreducessurveyerrorsandeliminateserrorsduetodataentry.TheuseofanelectronicversionofthequestionnaireintheGEP3baselinesurveymeansthatdatawereenteredduringtheinterviewitself,thusmakingseparatedataentryobsolete.TheCAPIquestionnairestrictlyenforcedtheprescribedquestionnaireroutinganddatavalidationforallresponsestosurveyquestions.Theinterviewersthusonlyposedthequestionswhichweremeanttobeaskedandcodedtheresponse
EvaluationoftheGirl’sEducationProject3
©EDOREN 27
onapre-definedscale.Duetothefactthattherewasonlyoneentry(insteadofthethreeinpaper-basedsurveys)thepossibilityofkeyingerrorswasgreatlyreduced.
CAPIalsoensuredtimelyfeedbackfromthefieldtothecentraldatamanagementteam.Thecompletedinterviewfilesweretransmittedfromthefieldviainternetdaily,ifpossible,andwerecentrallyexportedtostatisticalsoftwarepackages.ThesurveyanddatamanagementteaminAbujathushadtheabilitytocloselymonitorfieldworkandtoprovidetimelyfeedbacktofieldworkers,thuspreventingfutureerrorsfromhappening.Automatedroutingandpre-codedanswersincreasedtheinterviewingspeed,reducingrespondentfatigueandtheoveralltimerequiredforthefieldwork.
3.2.7.2 Qualityassuranceinthefield
Aqualityassuranceteam,oneineachstate,wasconstitutedtoperformin-fieldmonitoringandspotchecksonthedatacollectionteam.Duringthedatacollection,thisteamvisited138selectedschoolswheretheycarriedoutmini-surveysthroughliveobservationsandalsoconducted79phone-inbackcheckswiththeheadteacher(Table6).
Table6: Observationsfromearlylearningintervention
STATE/LGALiveobservations Backchecks
IQSs Publicprimaryschools IQSs Publicprimary
schools
Katsina
Batsari 1 4 5 12
Kankia 11 10 6 5
Rimi 9 12 8 7
Total 21 26 19 24
Zamfara
Bungudu 7 13 3 4
KauraNamoda 14 8 1 0
Tsafe 13 13 2 4
Total 34 34 6 8
Theliveobservationsassessedthewayinwhichthesurveyinstrumentswereimplementedinthefield.Sincealltheinformationwasenteredintoaqualityassurancedashboard,thesurveyandfieldmanagercouldmonitoronacontinuousbasistheperformanceoftheteam,theprogressmade,andthechallengesfacedattheschoollevel(i.e.noregister,pupils’arrivaltime,etc.).Therelevantinformationwasalsopassedtothestatesurveycoordinatorsandsupervisorsofthefieldteams,inorderprovidefeedbacktothefieldteamandtotakecorrectivemeasuresalmostimmediately.
Inaddition,afterthefieldteamhadleftaschool,thequalityassuranceteamphonedrandomselectedheadteachersandteachersandundertookashortquestionnaire.Thiswastocheckwhethertheinterviewswerecarriedoutproperly,withrespecttotherespondents,andinaccordancewithbothqualityandethicalstandards.
EvaluationoftheGirl’sEducationProject3
©EDOREN 28
Detailedbelowisasummaryoftheoversightfunctionsthatthequalityassuranceofficerscarriedoutduringthefieldexercise:
• ensuredsamplingandallprocedureswerestrictlyfollowed;• conductedspotchecksforteamsandensuredtheywerewheretheyplannedtobe;• observedongoinginterviewsandaddressedissuesthatrequiredattention;• ensuredthattherewasacleardistinction,amongtheteam,betweentheperiodinwhichthe
programmewasintroducedtotheschoolauthorityandtheactualtimetheschoolbegantoteachnon-religioussubjectsalongsidereligioussubjects;
• performedrandombackchecksvisitstoschoolswherethesurveyhadalreadybeencompleted;and
• gavecontinuousfeedback/refreshertrainingwithstateteamswherechallengeswereencountered.
Thequalityassuranceteamreportedtheirfindingstothestatesurveycoordinators,aswellastothesurveyanddatamanagementteaminAbuja.Togetherwiththestatesurveycoordinatorstheydebriefedthefieldteamsontheirfindingsandbroughtattentiontotheirfindingsinordertoimprovethequalityofthedatacollection.
Italsoneedstobestressedthatsomeofthequalityassurancefunctionsofboththestatesurveycoordinatorsandindependentmonitorswashamperedbytheunanticipatednecessityofactivelyanddiligentlyscreeningtheschools,especiallyIQSs,foreligibility.Thescreeningprocessmostoftenhadtobedoneinpersonandontheground,asthepoortelephoneconnectionsinthemoreremoteareasoftenpreventedcontact.
3.2.7.3 Qualityassurancepost-datacollection
TheCAPIdatacollectionapproachrequiresonlyveryminorpost-datacollectionprocessingandediting.CAPIenforcedverystrictvalidationrulesatinterviewtime,whichwereembeddedintheelectronicsurveyquestionnairesthemselves.Theroutingofthequestionnaireswasalsostrictlyenforced,i.e.itwasmandatoryforallquestionstobefilledandtheinterviewercouldnotproceedwiththeinterviewunlesstheinformationprovidedcorrespondedtoallvalidationrulesimposedonit.
Thepost-datacollectionprocessingdid,however,ensuretheconsistencyofalltheidentifyinginformationforeachinstrument.Asmultipleinstancesofthesameinstrumentwerefieldedateachschoolandtheirwithin-schoolidentificationwasassignedmanuallybytheinterviewers,theconsistencyoftheseidentificationelementshadtobeconsolidated.Fieldworkmonitoringformsanddailyfieldreports,aswellinterviewtimedata,wereusedtoreconcileanyinconsistencies.Theschool-levelidentifierswerepre-loadedandhardcodedintothequestionnairesthemselvesinordertoavoidduplicationormis-assignedIDs.
Allthedatawerealsocheckedforcompletionagainstsecondarysources,suchassurveymanagementandsamplecompletionforms.Thisprocessensuredthatallofthecollectedinformationwascontainedinthefinaliseddatacompilation.
Theactualcompilationofseparatedatarecordswasachievedduringtheactualdatacollectionprocess.ThedataweretransmittedfromthefielddailyandthedatamanagersattheOPMNigeria
EvaluationoftheGirl’sEducationProject3
©EDOREN 29
centralofficeinAbujacompiledtherecordsintoworkingdatabases.Afterthecompletionofthedatacollection,allofthecollecteddatawerecollatedinanalysis-leveldatafiles.Thedatabasewasformattedandreshapedaccordingtothespecificationsforanalysis.
3.2.8 Ethicsandinclusion
Anyevaluationmustabidebyrelevantprofessionalandethicalguidelinesandcodesofconductforindividualevaluators.Evaluatorsmustbemindfulofgenderroles,ethnicity,ability,age,sexualorientation,languageandotherdifferenceswhendesigningandcarryingouttheevaluation(OrganisationforEconomicCo-operationandDevelopment)(OECD)DevelopmentAssistanceCommittee(DAC)Standards,2010).DFIDhasdrawnupitsownprinciplesforensuringhighstandardsofethicsinresearchandevaluations,andthesewereadheredtoaspartoftheGEP3evaluation(DFID,2011).Theseinclude:
i. researchersandevaluatorsareresponsibleforidentifyingtheneedforandsecuringnecessaryethicalapprovalforthestudytheyareundertaking;
ii. researchandevaluationmustberelevantandofhighquality,withaclearvalue;iii. researchersandevaluatorsshouldavoidharmtoparticipantsinstudies;iv. participationinresearchandevaluationmustbevoluntaryandfreefromanyexternal
pressure;v. theconfidentialityofinformation,privacyandanonymityofparticipantsmustbeensured;vi. culturalsensitivitiesmustberespected;vii. UKhumanrightslawmustbeadheredto;viii. allresearchandevaluationresultsmustbepublishedandcommunicated;ix. researchandevaluationmustbeindependentofthoseimplementingtheprogramme;andx. thereshouldbeaparticularemphasisonparticipationbywomenandothersociallyexcluded
groups.
Theseethicalprinciplesareimportant,andrunthroughtheevaluationdesign.Oneareaofparticularimportanceforthisevaluationwasensuringthatchildrenwereaskedage-appropriatequestionsandthatappropriateconsentwassought.UNICEFguidelinesonconductingresearchwithchildrenwerefollowed.87Theevaluationpaidparticularattentiontothemother-tonguelanguagewheninterviewingchildrenofayoungageinordertoguaranteechildren’sunderstandingofthedatacollectionprocess.Furthermore,datacollectionwassensitivetochildren’sdisabilities,imposingspecificprocessesfortheinclusionofdisabledchildrenwhilebeingsensitivetotheirdisability.SurveyprotocolswereapprovedbyOPM’sEthicalReviewCommittee(ERC)(seeAnnexD)andaresearchethicsrequestwassubmittedtotheNationalResearchEthicsCommitteeinNigeria(NHREC).
3.2.8.1 Permits,consent,confidentialityanddatasets
Noincentivesweregiventorespondentsforparticipationinthestudy.Eachschoolthatwaspartofthequantitativebaselinesurveyreceivedasmalleducationalgift(e.g.aninflatableglobe)andparticipantsinthelearningassessmentsreceivedrefreshments.
87Grahametal.(2013)EthicalResearchInvolvingChildren.Florence:UNICEFOfficeofResearch–Innocenti.
EvaluationoftheGirl’sEducationProject3
©EDOREN 30
Consentwassoughtfromboththeheadteacherandindividualsparticipatinginthestudy.Descriptionsofthestudypurpose,therequestedinvolvementoftheparticipantandtheexpectedtimefortheadministrationofeachinstrumentwasprovidedtoeachparticipantpriortoseekingconsent.
Allpersonaldatacollectedaspartofthissurveyareonlyavailabletoauthorisedindividualsforanalyticalpurposesandarehandledusingdataprotectionbestpractices.Eachrespondenthasbeenassignedauniqueidentifier,whichwasusedintheanalysisofthedata.Allcleaned,anonymisedanddocumentedbaselinedatasetswillbemadepublic(subjecttoDFIDapproval)toenablenationalresearchers,researchstudentsandothereducationstakeholderstoaccessandusetheevaluationdatatoconductadditionalanalysisandresearch.
NHRECexemption
AnapplicationoutliningtheGEP3EvaluationEthicalStandards,informationonpermits,consent,confidentialityanddatasets,theprovisionofallparticipantinformationforms,consentformsandinstrumentintroductions,protocolsforworkingwithchildrenandspecificprotocolsfortheparticipationofchildrenwithdisabilitiesinthesurveywassubmittedtotheNHRECon29September2015.On07October2015anoticeofresearchexemptionwasprovidedtoEDOREN.TheexemptionletterisprovidedinAnnexD.
OPMethicscommitteeapproval
ApartialapplicationwassubmittedtotheOPMERCinMay2015forthequantitativeresearchcomponentoftheevaluation.ThissubmissionwasconsideredbytheERCon15May2015.Arequesttore-submitwithadditionaldocumentationwasmadetotheevaluationteam.AfterthefinalisedtoolswereprovidedtotheERC,ethicalapprovalwasgivenon28September2015.PleaseseeAnnexDfortheapprovalletter.
3.2.8.2 Specificprotocolsfortheparticipationofchildrenwithdisabilitiesinthesurvey
Enumeratorswereinstructedduringthetrainingandintheenumeratormanualtomakeanefforttomakeallthepupilsfeelateaseandtohelpthemunderstandthatthiswasnotanexam.Enumeratorswereencouragedtoaskquestionsaboutthepupils’livesordayiftheywereveryshy.Enumeratorsweretolookforaquietandsomewhatisolatedplacetoadministerthetests,avoidingtheheadteacher’sofficebecausethepupilmightbeshyinsuchalocation.
Ifthepupilwasidentifiedashavingadisabilitythatwouldnotaffecttheirabilitytotakethetestenumeratorswereinstructedtomakesuretheywereabletogettothetestlocation.Forexample,apupilwhocannotwalkmaystillbeperfectlyabletotakethetest,butmayhaveneededtobeassisted–byusingawheelchairorliftingthem–toreachtheplacewherethetestwasadministered.
Severalquestionswereincludedinthepupilquestionnairetocheckifthechildhadanydisabilitiesthatcouldimpedetheirabilitytoundertaketheassessment.Theseincludedaskingifthechildcouldhear,speak,writeandsee.Ifasampledchildhadanydisabilitywhichwouldnotaffecttheirabilitytotakethetest,theywerenotexcluded.Ifthepupilcouldnothearenumeratorswereinstructedtoendtheinterview.Inallothercases,enumeratorscontinuedwiththeinterview,andCAPIautomaticallyskippedanyquestionsthatthechildwouldhavebeenphysicallyunabletoanswer.
EvaluationoftheGirl’sEducationProject3
©EDOREN 31
3.2.8.3 Sensitivitytogenderandequity
Sensitivitytogenderandequitywithintheevaluationdesigncanbedividedintofivemaincategories:
• evaluationfocus;• nobiasinthemeasurement;• detectingdisaggregatedoutcomesbygroup;• seekingexplanationsforobserveddifferencesinoutcomes;and• focusingonpupilsatthelowendofthelearningdistribution.
Evaluationfocus:Allaspectsoftheevaluationincludeafocusongenderandmarginalisation.Inaddition,thefocusonIQSsispartlybecauseIQSSismeanttobeaninclusivestrategytoreachouttomarginalisedchildreninrurallocations.AlargenumberofchildreninnorthernNigeria,particularlyfromruralareas,donotattendformalschools,butdoattendacentreofferingQur’aniceducation.Therefore,comparisonsofdifferencesbyschooltypeareaprimaryfocusinevaluatingtheextenttowhichGEP3hascontributedtoimprovedteachingandlearninginIQSs.
Nobiasinthemeasurement:Genderbiascanaffectlearningassessments,andhappenswhenanitemisinterpreteddifferentlybytest-takersand,thus,‘whatwearetesting’isdifferentforgirlsandforboys.Throughtheuseofpsychometricanalysis,thisbiascanbeidentifiedanditemsthatarebiasedinthiswaycanberemoved.ItisessentialfortheGEP3evaluationsthatatestoflearningoutcomesdoesnotdiscriminatebetweengirlsandboys–meaningthatresultsbetweengenderscanbereliablycompared.Inotherwords,throughtheuseofpsychometrictechniques,wecanensurethataboyoragirlwiththesamelevelofliteracy(withallotherthingsbeingequal)willreceivethesamescoreonascaleoflearning.ARaschanalysisoftestitemsusedinotherEDORENevaluationswasundertakenin2014(McGrane,2014)andfoundthattheitemsdonotdiscriminateonthebasisofgender.Newitemsunderwentthesameanalysis.
Detectingdisaggregatedoutcomesbygroup:Inordertoanalyseoutcomesbysocialgroup,dataonpupil,schoolandcommunitycharacteristics(suchasgender,socioeconomicstatus,geographiclocation,etc.)mustbecollectedandlinkedtooutcomedata.Theremustthenbesufficientstatisticalpowertodisaggregateoutcomesbygroup.TheGEP3evaluationcollectedcontextualdataatthepupil,schoolanddistrictlevelswithsufficientstatisticalpowertoanalyselearningoutcomesbygender,whichisofprimaryimportanceinevaluatingtheextenttowhichGEP3hasimprovedlearningoutcomes,especiallyforgirls.
Explainingobserveddifferencesinoutcomes:GirlsinnorthernNigeriafaceamultitudeofbarrierstoachievingliteracy,languageandnumeracyskills.Anydifferencesinlearningoutcomeswereanalysedusingtheapproachesoutlinedabove.However,thereasonsforthesedifferencescannotbeobservedpurelybasedonoutcomedata.Somefactorsareeasilyobservable,suchasaccesstospecificresources.Ifidentifiedinadvance,differencesininputstoeducationbasedongendercanbeinvestigatedthroughthequantitativesurveyandtheimpactonlearningoutcomesbygroupcanbeanalysed.However,otherfactorsarelatentorcanonlybeobservedbygainingacontextualunderstandingofcommunitiesandschools.
Focusingonpupilsatthelowendofthelearningdistribution:Inordertoensureanequityfocusismaintainedtheevaluationofchangesinlearningoutcomeswillbeviewedthroughanequitylens.Purelycomparingmeanincreasesinscalescoresatbaselineandfollow-upsurveyscanobscurefrom
EvaluationoftheGirl’sEducationProject3
©EDOREN 32
viewthelearningofthemostmarginalisedchildren.Theanalysesofincreasesinlearningoutcomeswillalsoincludeananalysisofthegapbetweenthelowestperformingpupilsandtheexpectationsofthecurriculum.Theanalysiswillinclude:(1)thepercentageofpupilsachievingbelow-curriculumexpectations;and(2)thegapbetweentheactualskillsbeingachievedbypupilsatthelowerendofthedistributionandtheskillsexpectedbythecurriculum.
3.2.9 Outcomemeasurementandconstructs
3.2.9.1 Pupilassessmentinstruments
EnglishliteracyandHausaliteracyassessmentswereconstructedfollowingthefivestepssetoutbelow,includingclarifyingconstructs,testtargeting,administration,psychometricanalysis,drawingbenchmarksandsecondarydataanalysis.
Clarifyingconstructs
Thefirststepinundertakingalearningmeasurementexerciseistodefineexactlywhatistobemeasured.Definingexactlywhatitisthatweexpectchildrentoknowandbeabletodoisattheheartofthemeasurementoflearning.Literacyandreadingarequitedifferentconstructs,asarenumeracyandmathematics.Atitssimplest,tobeliteratemeanstobeabletoobservesymbolsorsignsfromapageandtoascertainmeaningsthatarestandardisedtosomeextent.Italsomeansbeingabletoproducetextbywritingthesamesymbolsorsignsinordertorecordmeaning.So,eveninitsmostbasicofforms,literacyisalotbroaderthanreading.Internationalgold-standardassessmentprogrammes,suchasTrendsinInternationalMathematicsandScienceStudy(TIMSS)andProgressinInternationalReadingLiteracyStudy(PIRLS),goevenfurther,incorporatinganelementofsocialcontextintotheirunderstandingsofliteracy.Similarly,theOECD’sdefinitionofnumeracyisbroaderthanbeingabletooperatemathematicalequations–rather,ithingesonthe‘abilitytoaccess,use,interpret,andcommunicatemathematicalinformationandideas,toengageinandmanagearangeofsituationsinlife’(OECD,2014).Thisdefinitionisbroaderthanunderstandingmathematicalconceptsbutalsoincludesbeingabletoapplytheminavarietyofknownandunknownsituations.
TheconstructstobemeasuredthroughthelearningachievementtestsintheGEP3evaluationareEnglishliteracy/language,Hausaliteracy/languageandnumeracy.TheGEP3evaluationusesthebroaderconstructsofliteracyandnumeracy,ratherthanreadingandmathematics,asdiscussionswithstakeholdershavehighlightedthattheprogrammeexpectschildrentoimprovelearninginareasthatfitintothesebroaderareasofliteracyandnumeracy–suchastounderstandtexts,usereadingtounderstandtheworld,drawinferencesandcommunicateinwriting,usemoneyineverydaylife,andreadaclock.
Targeting
Theflawedassumptionthattestedchildrenareabletoreadorwritealreadyhasbeenavoided.TestitemsweredesignedorselectedfromexistingEDORENteststomeasureskilllevelsbelow,atandabovetheskillsassumedtohavebeenreachedgiventhegradethechildisattending.
Themajorweaknessindatameasuringliteracyandnumeracyinlowandmiddle-incomecountriesisthatassessmentsmeasureskillsatlevelsthataretoohighformostofthechildrentakingthetests.ThiswasobservedinthecaseofMali,whereover80%ofstudentsinGrade2couldnotreadasingle
EvaluationoftheGirl’sEducationProject3
©EDOREN 33
wordinfournationallanguages,whileintheNigerianstateofSokoto81%ofthestudentscouldnotreadfullwords(GlobalEducationMonitoring,2012).Theseflooreffectsarenotproblematicifthegoaloftheassessmentistoestablishnationallearninglevels.Forevaluationpurposesitwouldnotbepossibletodetectgrowthinliteracyornumeracyunlessthosechildrenpreviouslyoutofrangeofthetestscomeintotheabilitylevelmeasuredbythetest.Forthosechildrenwhodonotcomeintorange,butwhoseliteracyornumeracylevelsdoimprove,theimpactoftheinterventionwillbeunderestimated.Therefore,theGEP3assessmenttoolsweredesignedtoensureitemdifficultymatchedpupils’ability.Theleveloftheassessmentandpupils’abilitieswereassessedduringthepilotingandadjustedbasedontheresults.
Astheassessmentsusedfortheevaluationareadministeredtoapanelofpupils,theassessmentsneedtobeabletocoverawidevariationinpupilknowledge(fromthelowestperformingpupilsatbaseline,tothehighestperformingpupilsatmidline–thefollowingschoolyear).Thisplacesagreatrequirementonthetesttocrediblymeasureknowledgeoveralargerangeofdifficulty,withoutmakingthetesttoolongforyoungpupils.ThiswillbealleviatedthroughtheuseofRasch(itemresponsetheory)modelling.Thetestswilluselinkitemstolinkthedifficultyofthebaselinetesttothemidlinetestandtoplacetheresultsontothesamescale(metric).TheEDORENmetricsteamhasalsoplacedthislearningontothesamemetricashasbeenusedforESSPINEnglishLiteracyandNumeracy.
Administration
LearningachievementtestsusedaspartoftheGEP3evaluationuseasetofassessmentitemsorquestionstoassesschildren’sliteracyand/orlanguageandnumeracyknowledgeandskills,basedonaone-on-oneindividualoralinteractionbetweenachildandanenumerator.
Psychometricanalysis
TheuseofRaschmodelling(IRT)increasestheamountofanalysisrequiredasmoresophisticatedtechniquesareused,ratherthanaddingupatotalnumberofitemscorrectinthetestandconvertingthenumberintoapercentagescore.Inthisregardtheevaluationteamfollowedthefollowingsteps:thefirststepwastotestthepsychometricpropertiesoftheitemstoensuretheywereusefulmeasuresofwhatpupilsknow.Thesecondstepwastoremoveanyitemsthatdidnotperformwellandwouldbiastheresultsiftheywerecountedintheanalysis.Inathirdsteptheteamrankedtheitemsaccordingtodifficulty.Thiswasdonebyapsychometrician,usingfit-for-purposesoftware.Thesoftwarethenalsorankedpupilsaccordingtotheirabilityandplacedthepupilsandtheitemsontothesamemetric.Thisisaprobabilitymodelaspupilsareplacedonthescaleaccordingtotheprobabilityofapupilansweringthecorrespondingitemcorrectly.
Proficiencybands
Informeddecisionswerethenmaderegardingthestandardsoflearning.Forexample,whatskillsdoweexpectpupilstohavetoconsiderthem‘literate’?Thisdefinesthelearningstandardacrossthemetricandcreatesaproficiencyband.Thepercentageofchildrenthatfallwithinthisrangeisthencalculated.
TheseproficiencybandsweredrawnthroughabenchmarkingworkshopundertakenwithTDP,GEP3,DEEPENandESSPINstakeholdersinNovember2015fortheEnglishliteracyandnumeracyassessments.FortheHausaassessment,thecut-offpointwasdrawntoreflecttheEnglishliteracyproficiencyrangesasmuchaspossible.inotherwords,theskillsthatfallwithineachproficiency
EvaluationoftheGirl’sEducationProject3
©EDOREN 34
rangeforHausaareassimilaraspossibletotheskillswithinthesameproficiencyrangeforEnglishliteracy.
Secondaryanalysis
Theprimaryanalysisproducestwovariablesforeachlearningconstruct,perpupil.Thefirstisascalescore.Thescalescoreisaprecisemeasureofwhere,alongtheachievementscale,thepupilsits.Thescalescoreisusefulforfine-grainedanalysislikemulti-levelmodelling,regressionsandcorrelations.Thesecondvariableiswhichproficiencybandthepupilfallswithin.Thisvariableisusefulfordescribingwhatlearningpupilshaveachievedandhowwhatchildrenactuallyknowandcandohaschanged,overthecourseoftime.
3.2.9.2 Teacherknowledgeandskillstheoreticalmodel
Teacherknowledgeandskillstest
AdaptedfromLeach(2002)andCogill’s(2008)modelsofpedagogy,theknowledgemodelsetoutinthefigurebelowconceptualisestheknowledgethatcontributestoobservableclassroompractices.Withinthemodel,therearethreetypesofknowledgethatateacherdrawsonwithinclassroompractice.Curriculumknowledgereferstoknowledgeofwhatshouldbetaughttoagroupofstudents,knowledgeofthenationalsyllabus,understandingoftheschoolandgrade-levelplanningdocumentsandknowledgeofexaminationscontent.Pedagogicknowledgereferstoknowledgeofthelearnersinthesetting,knowledgeofhowtoprovidetheconditionsthatenablepupilstounderstandandtheselectionoflearningandassessmentmaterials.Subjectknowledgereferstoknowledgeoftheessentialquestionsofthesubject,thenetworksofconcepts,thetheoreticalframeworkandmethodsofinquiry.
Figure9: Teacherknowledgemodel
Fusedknowledgethatcontributestoobservablepractices
Curriculumknowledge
Pedagogicknowledge
Subjectknowledge
EvaluationoftheGirl’sEducationProject3
©EDOREN 35
Testconstruction
Theteacherknowledgeandskillstestwasdividedintothreesections,collectivelycomprising30items,includingmultiplechoice,shortresponseandlongresponseitems.InsectiononeteacherswereaskedtomarkpupilresponsestoHausaliteracyquestionsandtoindicatethegradelevelatwhichtheanswershouldbeknownbypupils,asdefinedbythecurriculum.InsectiontwoteacherswereaskedtofillinmissinginformationonananswersheetforareadingtestaimedatGrade2pupils.Inaddition,teacherswereprovidedwithtwonewspaperarticlesandaskedtofillinmissinginformation,includingbasiccomprehensionquestionsandquestionsthatrequiredtheinterpretationofwordsandphrases.Sectionthreeaskedteacherstoidentifypoorandgoodpupilwork,andtoreviewpupilworkinordertomakejudgementsaboutpupils’writing,includingtheorganisationofthewriting,theuseofgrammar,punctuation,spelling,thepupils’abilitytoself-correctandreflectontheirwritingandthepupils’abilitytoformlettersandusespacesbetweenwords.Teacherswerethenaskedtodescribehowthatmightsupportthepupilsinimprovingtheirwriting.
Teacherresponsestotheitemsweremarkedandcross-checkedusingamarkingmatrixandscoringguide.Themarkingandcross-checkswerecarriedoutbytwosenioreducationspecialists.OnespecialistisapreviousdirectorinNigeria’seducationsystemandtheotherisaninternationaleducationspecialist.
Teacherresponseswereusedtodevelopeightareasofteacherknowledgeandskills:
1. syllabusknowledge;
2. abilitytoidentifylowperformers;
3. abilitytoevidencejudgementsanddiagnose;
4. abilitytobuildonstudentknowledge;
5. teacherwritingskills;
6. teacherGrade2Hausaknowledge;
7. teachercomprehensionskills;and
8. interpretingwordsandphrases.
Dataanalysisandsubscaleanalysis
Themainpurposeofthedataqualityandsubscaleanalysiswastoassessthequalityofthedataonteacherknowledgeandskillswithregardtovalidityandreliability,andtoestablishtwothresholdpointsforeachsubscale.
Thefirst,andlower,thresholdpointismeanttorepresentarudimentarylevelofachievement.Thismeansthatthereappearstobeevidenceofsomeabilitytounderstandandperformsomeoftheteachingtasksmeasuredbythesubscale.Thesecond,andhigher,thresholdismeanttorepresentacompetentlevelofachievement.Thismeansthatthereisevidenceofabilitytoconsistentlyunderstandandperformteachingtasksmeasuredbythesubscale.
ThereliabilityofthescaleswasexaminedusingCronbach’salpha.
Highlevelsofmissingdatawereobserved.Theamountofmissingdataisamajorconcern.First,becauseitmayindicatethatmanyoftheteacherswhoparticipatedintheprogrammewerenotabletodomanyofthetasksaskedofthem.Thesetaskswereintendedtorepresentthecoreelementsof
EvaluationoftheGirl’sEducationProject3
©EDOREN 36
whatteachersshouldknowandbeabletodoinordertoteachereffectively.Sotheconcernis,here,thattheteachersmaynotbeveryskilled.Thismaysuggestthatthetestwas‘toodifficult’.However,thetestwasdesignedaroundnotionsofwhataminimallycompetentteachershouldbeabletoknowanddo.Itisprobably,therefore,unhelpfultobeoverlyconcernedaboutthetest’sdifficulty.Therealissueisthecompetencyleveloftheteachers.
Thesecondconcernisthattherangeofanalyseswaslimitedduetothemissingdata,andhencethecapacitytodrawempiricallystrongconclusionswasimpeded.
Duetothehighdemandplacedononeassessment(tomeasuretherangeofknowledgeandskillsrequiredtodeterminethecompetenceofateacherineightkeyareas)somesubscaleshadfewitemsfromwhichtodraw.Alimitednumberofitemslimitedthereliabilityofascale.Inthecaseof(1)syllabusknowledgeand(2)abilitytobuildonpupilknowledge,thenumberofitemsavailableonwhichtobuildthescale(threeandfourrespectively)werenotsufficienttoyieldreliableresults.Itwasdecidedthatinordertoutilisethegathereddata,theanalysiswouldfocusonprovidingexamplesofitems,descriptionsofthekindsoferrorsteachersmadeandinformationonthepercentageofteacherswhorespondedcorrectlytospecificitems.
Withinthecontextofexploratoryresearchongroups(asopposedtoadiagnosticinstrument)aCronbach’salphaofover0.5–0.6isoftenarguedtobeacceptable(SeeSuhretal.,2009andYuandWatkins,2011).Basedonthis,theremainingsixsubscalesachievedacceptablelevelsofreliability.
Table7: Teacherknowledgeandskillsproficiencybands
SubscalesLowband
Noevidenceofskill
MiddlebandEvidenceof
rudimentaryskill
UpperbandEvidenceofcompetence
Abilitytoidentifylowperformers 0–2 2.5–7 Greaterthan7
Abilitytoevidencejudgementsanddiagnose
0–2 2.5–7.5 Greaterthan7.5andabove
Teacherwritingskills 0–2 2.5–3.5 Greaterthan3.5
TeacherGrade2Hausaknowledge
0–2 2.5–3.5 Greaterthan3.5
Teachercomprehensionskills 0–2 2.5–3 Greaterthan3
Interpretingwordsandphrases 0–1 1.5–2 2.5andabove
Teachermotivationscales
TheTeacherMotivationscalesweredrawnfromapreviousstudyundertakenbyEDOREN–theTDP.88Inthisstudy‘motivation’wasunderstoodas:
88ThissectionistakenfromaninternalOPMdocumentpreparedaspartoftheTDP.ThedocumentistitledNotesonmeasuringteachermotivationfortheTeacherDevelopmentProgrammebaselinesurveyandissignedanddatedSC[StuartCameron]20.8.2014.ThereferencelistprovidedinthisdocumentisincludedintheBibliographyforconvenience,intheeventthatfurtherinformationissought.
EvaluationoftheGirl’sEducationProject3
©EDOREN 37
…amixtureofperceivedefficacyandeffort/importanceattachedtoteachingwork;butisalsomeasuredlessdirectlythroughinterest/enjoymentandpressure/tensionitems.
Accordingtothisdefinitionamotivatedteacherisonewho:
• seesthemselvesaseffectiveandasmakinganeffort;• seestheirworkasimportant;• isinterestedinandenjoystheirwork;and• managesworkpressureandtension.
Theteachermotivationscalesareunderpinnedbyatheoryofactiontoexplainhowaprogrammeoperatestoincreaselevelsofteachermotivation.TheitemsincludedineachsubscalecanbefoundinAnnexE.
Thetheoryofactionisdepictedasacausalchainin
EvaluationoftheGirl’sEducationProject3
©EDOREN 38
Figure10.Accordingtothistheory,theGEP3interventionwillleadto:
• betterpedagogicandcontentknowledgeandskills,andbetterteacher-to-teacherinteraction(subscale6in
EvaluationoftheGirl’sEducationProject3
©EDOREN 39
• Figure10),whichwillinturn,leadto:• betteractualteacherefficacy,whichcanbemeasuredastheextenttowhichlearningoutcomes
improve,whichwillleadto:• increasedperceivedefficacy(subscales1and2in
EvaluationoftheGirl’sEducationProject3
©EDOREN 40
• Figure10)–thatis,theteacherswill(accurately)perceivethattheyhavebecomemoreeffective,whichwillinturnleadto:
• greaterinterestandenjoyment(subscale3),moreeffortbeingmadeandmoreimportanceattachedtotheirwork(subscale4),andlesspressureortensionexperiencedregardingtheirwork(subscale5).
‘Efficacy’canbeseenasperceivedrelativeinfluence,whichcanbedividedintotwoaspects:
• howimportantisteachingeffortvisàvispupilbackground(subscale1)?• howimportantisteachingeffortvisàvisschoolinfrastructureandprofessionalsupport(subscale
2)?
EvaluationoftheGirl’sEducationProject3
©EDOREN 41
Figure10: Theorisedcausalpathway(takenfromtheTDP)
Interestinand
enjoymentofteachingSubscale3
Teacher-to-teacher
interactionSubscale6
>Efficacyasateacher
>
Perceivedefficacyasateacher
Subscales1and2
EffortandimportanceSubscale4
Pressureandwork-relatedtension
Subscale5
Dataquality
Thequalityofadatasetcanbeinfluencedbytheamountofmissingdata.Therewasverylittlemissingdata:onaverage,only1.2%ofdataonthemotivationitemswasmissing.
Itisimportanttoconsiderthedistributionofvaluesforeachvariable.Highlyskeweddata–datathatpileupatoneendofthedistribution–arenotwellsuitedtocertainstatisticalprocedures,includingsomeofthoseplannedtobeundertakenintheanalysisoftheteachermotivationitems.Mostofthedatawerefoundtobehighlyskewed.Whileitisimpossibletoknowforsure,thesedistributionsaresuggestiveofcomplianceeffectsinatleastsomeofthesedata.Thus,cautionisrecommendedintheiruseandinterpretation.
Procedureswereexploredtotransformdatatoimprovetheirdistribution;however,theseprocedureswerenotusedastheydidnotimprovethedistributionmarkedly,andbecausesuchtransformationcanaddconsiderablecomplexitytotheinterpretationandreportingoffindings.
Analysis
ThedatawereanalysedusingexploratoryfactoranalysisinthecomputerapplicationSPSSStatistics.Theanalysiswasundertakenafterreversecodingnegativelywordeditems.Principalaxisfactoring,withorthogonalrotation(Varimax)wasused.Thisisaconservativeapproach,andonethatassiststheinterpretationofresults.ThereliabilityofscaleswasassessedusingCronbach’salpha.
Thevalidityofthescales–theextenttowhichtheyaremeasuringwhattheypurporttomeasure–seemslikelytobeacceptablegiventhattheyarederivedfromandare(moreorless)consistentwith
EvaluationoftheGirl’sEducationProject3
©EDOREN 42
thetheoreticalaccountof‘teachermotivation’thatshapedthedevelopmentoftheitems.However,itisprobablyadvisabletotreatthevalidityofthescaleswithsomecaution.
Thefactoranalysissuggeststhatfivescalesareretrievablefromthesedata.Table8showsthesubscalenameandthenumberofthesubscaleasusedinthepathdiagram(
EvaluationoftheGirl’sEducationProject3
©EDOREN 43
Figure10),alongwiththereliabilityofeachofthese‘new’subscales.89
Table8: Summaryofresultsofthefactoranalysis
Subscalenumberandnameusedinthepathdiagram Reliability
1:Importanceofteachingeffortvs.pupilbackground(perceivedefficacy)0.592:Importanceofteachingeffortvs.schoolinfrastructureandprofessional
support(perceivedefficacy)
3:Interestinandenjoymentofteaching 0.53
4:Effortandimportance 0.62
5:Pressureandwork-relatedtension 0.58
6:Teacher-to-teacherinteraction 0.72
Theresultsofthefactoranalysis,whilenotstrong,permitaninvestigationofthetheoryunderpinningthenotionof‘teachermotivation’,althoughanyfindingsarecarefullyphrasedbecausethesubscalesneedfurtherrefinement(asharperfocus).Theevidenceofcomplianceeffectsinsomeofthedataalsoneedstobeconsidered.
Allfiveelementsofthetheorydepictedinthepathdiagramhaveasubscalethatworkswellenoughtobeused.Here‘wellenough’meansthatthereisevidencethatthesubscaleisplausiblyuni-dimensionalandhasalevelofreliabilitythatisdefensible.
Thecorrelationsbetweenthesubscaleswereinvestigated.Sixofthepairsofcorrelationsareverylow(whichisdesirablegiventhattherotationusedinthefactoranalysiswasorthogonal).Theremainingfourpairshaveamoderatelystrongcorrelation,whichislikelytobeareflectionofthereliabilityofthesesubscales.
AnnexEprovidesasetofgraphsshowingthedistributionofvaluesforeachofthesubscales.
3.2.10 Limitationsandriskstothemethodology
Variouslimitationsandrisksneedtobetakenintoaccountintheabove-presentedmethodology:someofthesecanbeaddressedbycarefulimplementationoftheevaluation,whileothersconstituteriskstothemethodology.Table9reviewsthelimitationsandrisks.
Table9: Limitationsof,andrisksto,theevaluationmethodologyoftheearlylearningintervention
89Cronbach’salphaisusedasthemeasureofreliability.
Limitations/risks Whythisislimitingandwhatwehavedonetoaddressthis
Interventionnotprovidedasplanned
ItisassumedthatthesampledtreatmentschoolswillreceivetheRANAintervention,whilethecontrolschoolswillnot.Furthermore,thedesignassumesthatallprimaryschoolteachersinearlygradesandallIQSteachers
EvaluationoftheGirl’sEducationProject3
©EDOREN 44
teachingtheintegratedcurriculuminearlygradeswillbetrained.Iftheinterventionisnotconductedasplanned—forexample,sometreatmentschoolsorteachersarenot,orareonlypartially,exposedtotheintervention,orcontrolschoolsareincludedintheintervention—theabilitytoestimateimpactandcomparabilitybetweentreatmentandcontrolschoolsmaybereduced.Themitigationofthisriskofpartialcompliance,highlightedintheGEP3EvaluationFramework,requiresclosecoordinationbetweenimplementersandevaluators.Theimplementer,FHI360,haschangedtheinterventionroll-outtoschoolscomparedtoGEP3’sstrategypaper,whichwasinitiallygoingtoinvalidatetheevaluationdesign.Througheffectivecoordinationtheimplementerwaseventuallyabletoalignimplementationroll-outwiththeinitialplan.Nonetheless,adegreeofphasedroll-outisstillimplemented,whichwillresultinaslightnon-randomvariationwithinthetreatmentgroup.Giventhatschoolsthatformpartofthetwophasesofroll-outareclearlyidentifiedthiscanbetakenintoaccountintheanalysis.Withregardstoteachertraining,RANAhasconfirmedthatallearlygradeteachersintreatmentschoolswillbetrainedandmentored.Nonetheless,thebaselineexperiencehasdemonstratedthat,particularlyinIQSs,peoplethatarenotactuallyteachingintheschoolparticipateintraining.Therefore,itwillrequireclosemonitoringtotraintheactualtargetgroup.
Attrition
Sampledschools,teachersorpupilsmaydropoutofthesurveypanelbeforemidline,thuscreatingaproblemofmissingdata.Acertaindegreeofattritionistakenintoaccountinthesamplesizecalculation.GiventhatallstudyschoolsareGEP3schoolsweassumeverylimitedattritionattheschoollevel.OneIQSproprietorincludedinthetreatmentgrouphasindicatedthathemaystopintegrationevenbeforetheinterventionstarts.GovernmentandGEP3staffareconsultingwiththeproprietortomaintainhisinterest.Non-integrationwouldmeanthattheschoolfallsoutsidethetargetpopulation,whichwouldwarrantreplacementoftheschool.Attritionmayposearisktothecomparabilityoftheinterventionandcontrolgroupsifattritionissystematicallydifferentinbothgroups.Ifthecompetenceoftheteacherswhodropoutofthecontrolsampledifferstothatoftheteacherswhodropoutoftheinterventionsample,biasisintroducedinthecomparisonbetweenthetwosamples.Suchattritionishardtocontrolfor,butiftheattritionrateishighasampleofteachersthathavedroppedoutcouldbetrackedoutsideofschoolandsurveyed,iftheresourcestodosoareavailable.
Contamination
ContaminationreferstootherinterventionsaffectingtheRANAoutcomes,whichmaycreateaconfoundingfactorinthemeasurementoftheimpact.Thiscreatesaproblemtotheextentthatsuchcontaminationaffectstheinterventionandcontrolschoolsdifferently.GEP3willimplementotherinterventionsamongpublicprimaryschoolsinbothinterventionandcontrolgroups.Itisimportantthattheprojectdoesnotbiasinterventionimplementationtowardoneofthegroups.WeassumethattheriskinthisregardislowbecausethisisunderGEP3’scontrol.TheTDPandtheJollyPhonicsprojectarealsoprovidingteachertraininginpublicprimaryschoolsinKatsinaandZamfara.WhilesomedegreeofoverlapwithGEP3schoolsispossible,thereiscurrentlynoindicationthatitaffectsinterventionandcontrolgroupsinadifferentdegree,suchascouldcreateabiasinimpactmeasurement.ContaminationofthecontrolgroupmayalsohappenwhenRANAisimplementedincontrolschools.TheidentitiesofcontrolandinterventionschoolswerecommunicatedtotheRANAimplementingagency,andthiswastakenintoaccountintheimplementationroll-out.
EvaluationoftheGirl’sEducationProject3
©EDOREN 45
Spillovers
Spilloversoccurwhenpupilsorteachersfromsampledcontrolschoolsareaffectedbytheintervention,whichwouldcompromisethecontrolgroupasagoodcounterfactual.Spillovercouldhappenifinterventionteachersshareknowledge,skillsormaterialswithcontrolteachers.SincecontrolteacherswillnotbeincludedintheRANAclusters,nodirectinterventionspillovercantakeplace.BecauseoverlapbetweenGEP3andTDPinterventionswillbeavoidedinKatsinaandZamfara,wecanassumethattheRANAinterventionandcontrolteacherswillalsonotbeconnectedthroughTDPcommunitiesofpractice.However,theriskwillremainthatcontrolteacherswillbeincontactwithRANAinterventionteachersduetothegeographicalproximityoftheschools.Similarly,theheadteachersofcontrolschoolsmaypickupideasfromRANAiftheyconnectwithRANAinterventionschools.SuchexposuretoRANAwillbeindirect,whichwilldiluteitsimpact.Furthermore,baselinedatademonstratethatpupilsinIQSsmayalsoattendpublicprimaryschools.TotheextentthatpupilsfromcontrolIQSsattendclassesinpublicprimaryschoolsthatarepartofthetreatmentgroup,spillovercouldhappen.Whiletheriskinthisregardislikelytobesmall,acertainamountofriskremains.Atmidline,exposuretoRANAcanbesurveyedwithinthecontrolgrouptoassessthisissue.
Timingoftheimpactmeasurement
Impactwillbemeasuredatmidlineandendline.Midlineisscheduledforthethirdtermofthe2016–2017schoolyear,inordertoallowfindingstoinformscale-updecision-making.ThiswillexposeteachersandpupilstothreetofourtermsofRANAintervention.Giventhatteachercapacitydevelopmentconsistsofintensetrainingandmentoringthisperiodshouldallowforchangesinteacheroutcomestomaterialise.Sincepupillearningoutcomeschangemoreslowlylongerexposurewouldoffermoretimeforimpacttomaterialise.However,thishastobebalancedwiththetimingoftheevidenceneeds.Usingbaselinedata,weestimatethata3percentagepointnetincreasewouldbestatisticallydetectablewhenthedataofprimaryschoolsandIQSs,girlsandboys,areanalysedjointly.
Externalvalidity
Externalvalidityreferstotherepresentativenessofthefindingsbeyondthesurveysample.ThefindingswillberepresentativeforGEP3schoolswithintheLGAsselectedfortheRANAintervention.BypurposivelyselectingtheLGAsbasedonproximitytherepresentativenessforGEP3schoolsasawholeintherespectivestatesmaybeaffected.However,thetheory-basedapproachenablesustounpickhowandunderwhatconditionsimpactonlearningoutcomesoccurs,whichallowsformoregeneralisablefindings.
Evaluation-driveneffects
Beingpartofanevaluationcanchangebehaviourindependentoftheintervention.Forexample,teachersthatarepartofapanelsurveymayexertgreatereffortthantheywouldiftheywerenotselectedforinterviewing.Ifsucheffectsarecommontobothinterventionandcontrolgroups,theydonotaffectcomparabilitybuttheymayaffectthegeneralisabilityofthefindingstoothersituations.Sincethesurveyswillbeone-offeventsthattakeplacetwiceintwoyears,wedonotexpectastrongimpactonteacherorpupilbehaviour.TherandomisedevaluationdesignhasinfluencedtheformationoftheRANAschoolclusters,sincetheimplementingpartnerhadtoincluderandomlyselectedtreatmentschoolsintheclusters,whichmayhaveinfluencedtheproximityradiusofthecluster.TheextentofthispotentialeffectdependsontheproximityoftheGEP3schoolsintheRANALGAs.ByfocusingtheinterventiononalimitednumberofLGAsandstratifyingthesamplebyLGAs,theconcentrationofschoolsisenhanced.
Mixedmethods Amixedquantitativeandqualitativemethodsapproachoffersstrongaddedvalueintermsofdefiningandexplainingtheimpactofanintervention.Byusing
EvaluationoftheGirl’sEducationProject3
©EDOREN 46
atheory-basedapproachandaddingstate-levelqualitativedatacollectiontothequantitativemethods,theexplanatorypoweroftheevaluationisenhanced.However,theevaluationteamisnotabletocomplementquantitativedatacollectionatschoollevelwithqualitativeresearchatthesamelevel,whichwillconstraintherichnessoftheunderstandingoftheimpact.TheGEP3-ledqualitativeresearchmayprovideanalternativesourceofinformation,andqualitativeresearchcanstillbeconsideredduringthe2017–2020evaluationperiod.
Surveyresponseachievement
Asdiscussedabove,thesurveyteamsucceededinsurveyingahighpercentageoftheintendedschoolandpupilsample.Thepercentageofteacherssurveyedwasrelativelylower–thatis,around80%.Thiswasnotduetoalowresponseratebutratherbecauseinsomecases,particularlyinIQSs,theschoolonlyhadoneortwoteachers.WhilethismayaffecttheintendedMDEsize,weestimatethatthiswillnotjeopardiseavalidimpactjudgementsinceitisexpectedthattheincreaseinteachercompetencieswillberelativelylarge(a30%increaseaccordingtotheGEP3logframe).
Flooreffectsinmeasurement
Highlevelsofmissingdatawereobservedintheteacherassessment.Theamountofmissingdataisamajorconcern.First,becauseitmayindicatethatmanyoftheteacherswhoparticipatedintheprogrammewerenotabletodomanyofthetasksaskedofthem.Thesetaskswereintendedtorepresentthecorebasicelementsofwhatteachersshouldknowandbeabletodoinordertoteacheffectivelyataminimumlevel.Thustheconcernisthattheteachersmaynotbeveryskilled.Whilethissuggeststhatthetestwas‘toodifficult’,thetestwasdesignedaroundnotionsofwhataminimallycompetentteachershouldbeabletoknowanddo.Itisprobably,therefore,unhelpfultobeoverlyconcernedaboutthetest’sdifficulty.Theissuehereisthecompetencyleveloftheteachers.However,theseflooreffectslimittherangeofanalysesabletobeundertakenandlimittheevaluationteam’scapacitytodrawempiricallystrongconclusions.
Socialdesirabilitybias
Anumberofitemsregardingattitudestowardsgirls’educationwereincludedintheteacherquestionnaire.Whilethereliabilityoftheresultswasacceptable,theveryhighproportionsofteachersrespondingpositivelytostatementsregardingtheimportanceofgirls’educationrevealedacomplianceeffect.Thedistributionofresponsesacrossthestronglydisagree,disagree,agreeandstronglyagreeLikertscaleindicatesthatteachersareconformingtoperceivedcorrectresponses,creatingasocialdesirabilitybiasinthemeasurement.Giventhesecomplianceeffects,thedataonteachers’attitudestowardsgirl’seducationwerenotabletobeusedintheanalysis.Thislimitationhighlightstheimportanceofobservationdataandqualitativemethodstoprobesocialnormsandgenderattitudes,particularlyinanevaluationofaprogrammedescribedasaninterventiontargetedatgirls.
Biasinsampling
Aspartofthesamplingprocessbiasesmayoccurwhenunitsareselectedthatdonotformpartofthetargetpopulationorwhensomeunitsofthetargetpopulationarelesslikelytobeselectedthanothers.Ashasbeendiscussed,insomeschools,stakeholderstriedtoinfluencepupilsamplingorschooleligibility.ThiswasparticularlyariskinIQSsgiventheirnon-standardorganisationandthepoorqualityofthesampleframe.Totheextentpossiblethisriskwasmitigatedthroughthedefinitionandimplementationofrigorousandsystematicsamplingproceduresunderclosesupervision.
EvaluationoftheGirl’sEducationProject3
©EDOREN 47
3.3 Comparisonofbaselinecharacteristicsbetweeninterventionandcontrolschools
ThereliabilityofthequantitativeestimationmodelemployedfortheimpactevaluationoftheGEP3earlylearninginterventionreliesontherobustnessoftheRCTthatwasdesignedandimplementedtoidentifytreatmentandacontrolgroups.RCTisanexperimentaldesignthatensuresnosystematicdifferencesexistbetweentreatmentandcontrolobservationsandthus,givenrandomnesshastrulybeenachieved,addressestheissueofselectionbias.ArobustRCTisthereforecriticaltoachieveunbiasedestimatesofintervention(earlylearning)impact.
Forthisreason,wehaverunaseriesofcheckstodeterminewhetherornottherandomisationprotocolwasfollowedcorrectlyandifitachievedtheexpectedresults.Inparticular,ifthesamplingunitsofinterestwereassignedtotreatmentorcontrolgroupsrandomlyandthesamplingstrategyrandomisationprotocolwasmaintainedduringthesurveyfieldworkwewouldexpecttoseenosystematicdifferencesbetweenthetwogroups.Thisentailsthatnostatisticallysignificantdifferencesshouldemergebetweensampletreatmentandcontrolgroupsforarangeofcharacteristicsrelatedtotheobservationsonwhichtherandomisationchecksareperformed.
3.3.1 Categoriesandvariables
TherandomisationcheckswereperformedforthemainRCTsampleandfordifferentsub-samplesofinterest.Atthesametime,checkswererunatschool,teacherandpupillevels.SeparatecheckswereperformedforpublicprimaryschoolsandIQSs.Datawithwhichtoperformtherandomisationcheckswerederivedfromtheheadteacher,teacherandpupildatasets,whichwereproducedwithinformationfromtheirrespectivequestionnaires.Thesetofvariablesonwhichthecheckswererunisspecifictoeachcategoryofinterestandencompassesdifferentthematicareasofinterest.Balancecheckswerealsoperformedonkeyoutcomevariables,includingHausaandEnglishscaledscoresandteacherknowledge.Whilstmultiplecheckiterationsweretestedacrossdifferentcombinationsofvariables,wepresentinthetablesbelowaselectionofthekeyoutcomesandrelatedcharacteristicsatschool,teacherandpupillevel.Thewholesetofresults,includingchecksonsamplesandsub-samplesofinterest,isincludedinAnnexF.
3.3.2 Balanceachieved
Attheschoollevel,dataderivedfromheadteacherswereusedtocheckthebalancebetweentreatmentandcontrolgroupsacrossarangeofschoolcharacteristics,includinggender-specificfactorssuchasthenumberoftoiletsexclusivelyforgirls,boys’andgirls’enrolmentrates,pupilandteachergenderratios(boystogirlsandfemaletomaleteachers),aswellaspupilclassroomratiosandotherschool-specificaspects,includingtheoccurrenceofmajorschoolrepairworks.Asshowninthetablebelow,thelatteristheonlyvariableforwhichwedetectsomesignificantdifferencebetweentreatmentandcontrolgroupswhenlookingatthewholesampleofIQSsandprimaryschools.Alltheothervariablespertainingtothevariouscategoriesofinterestdiscussedaboveshownosignofstatisticallysignificantdifferencesbetweentreatmentandcontrolgroups.Thefullsetofschool-levelvariablesforwhichrandomisationprotocolchecks,andthecorrespondingfindings,areoutlinedinAnnexF.
EvaluationoftheGirl’sEducationProject3
©EDOREN 48
Table10: Balancecheckofschool-levelcharacteristics,publicprimaryschoolsandIQSs
VariablesMean
treatmentgroupMean
controlgroup TotalN+
Headteacherrateofabsenteeism 60% 70% 240
Headteacherobservedalesson 50% 50% 225
Schoolneedsrepairs 90%** 100% 239
Schoolhasseparatefunctionaltoiletforgirls 30% 20% 239
Notes:**,*indicatethatthetreatmentandcontrolmeansaresignificantlydifferentat5%and10%.+Nistotalsample(treatmentandcontrol).
WhenseparatelyinvestigatingIQSsandpublicprimaryschoolsintheRCTsample,theweaklysignificantdifferenceinschoolrepairsisonlyfoundinIQSs,whilstnosignificantdifferenceisdetectedamongsttreatmentandcontrolpublicprimaryschools.Thefactthatthegreatmajorityofschool-levelvariablesarefoundtobebalancedbetweentreatmentandcontrolgroups,andthefactthatthesignificantdifferenceinschoolrepairsisweakandlimitedtoasub-groupoftheschoolsample,indicatesthattherandomisationprotocolhasgenerallyachievedasatisfactorybalancebetweentreatmentandcontrolgroupsattheschoollevel.
Thisisfurtherconfirmedbythebalancediagnosisperformedonteachercharacteristics.Acrossteachers’demographic,qualificationandprofessionalcharacteristicsnostatisticallysignificantdifferenceisdetectedbetweentreatmentandcontrolgroups.Giventhattherandomisedassignmentoftheearlylearninginterventionunderevaluationwascarriedoutattheschoollevel(theclusterlevelofourRCTdesign),thesebalancediagnosticresultsareparticularlyimportantandgiveusconfidenceintherobustnessoftheestimatesofprogrammeimpactthatwilleventuallyemergefromthecomparisonoftreatmentandcontrolgroups.Ofparticularrelevanceisalsothefactthatabalanceisfoundbetweentreatmentandcontrolgroupsacrossteacherknowledgeandskillsindicators,asshowninthetablebelowforaselectionoftheseintermediateoutcomeindicators.Theonlyweaksignificance(10%)isdetectedforteachercomprehensionskills,whichdisappearswheninvestigatingthebalanceoftreatmentandcontrolgroupsinIQSsandpublicprimaryschoolsseparately.
Table11: Balancecheckofteacher-leveloutcomesandcharacteristics,publicprimaryschoolsandIQSs
VariablesMean
treatmentgroup
Meancontrolgroup TotalN+
Abilitytoidentifylowperformers 0.4 0.5 463
Teacherwritingskills 0.2 0.2 463
TeacherHausaknowledge 2.9 2.8 463
Teachercomprehensionskills 2.1* 2.3 463
Notes:**,*indicatethatthetreatmentandcontrolmeansaresignificantlydifferentat5%and10%.+Nistotalsampleacrosstreatmentandcontrol.
EvaluationoftheGirl’sEducationProject3
©EDOREN 49
Additionalbalancingtestsbetweentreatmentandcontrolgroupswerealsocarriedoutforpupillearningoutcomesandpupilcharacteristics,withtheinclusionofvariablesconcerningchildren’shouseholds’factors.Inthiscaseaswell,thegreatmajorityofcharacteristicsinvestigateddonotshowanystatisticallysignificantdifferencebetweenthetwogroups.ThetwolearningoutcomeindicatorsofHausaandEnglishscaledscoresshownostatisticallysignificantdifferencebetweentreatmentandcontrolgroups.Intheoverallsampleofallpupils,onlyfortwovariablesaresomedifferencesinmeansdetectedacrosstheoverallpupilsample,bothreferringtoownershipofassets(chairsandmobilephones).Whendisaggregatingthesamplebetweenmaleandfemalepupils,itispossibletonotethatfurtherdifferencesemerge,althoughonlyforaminorityofvariables.Boththemagnitudeandthedegreeofstatisticalsignificanceinanyofthesedifferencesarelowandshouldnotrepresentaconcern.Smallandstatisticallyweakdifferencesinalimitednumberofvariablesaretobeexpectedandareconsiderablyreducedintheoverallsampleofpupils.Aselectionofvariablesispresentedinthetablebelow,andtherestarepresentedinAnnexF.
Table12: Balancecheckofpupil-leveloutcomesandcharacteristics,publicprimaryschoolsandIQSs
Variables Meantreatmentgroup
Meancontrolgroup TotalN+
Hausaliteracyscaledscore 507.341 509.568 2649
Englishliteracyscaledscore 354.374 356.946 2649
PupilspeaksHausaathome 99% 99% 2623
Pupilhouseholdhasmobilephone 90%** 95% 2580
Notes:**,*indicatethatthetreatmentandcontrolmeansaresignificantlydifferentat5%and10%.+Nistotalsampleacrosstreatmentandcontrol.
3.4 Analysisofthedata
TheanalysisofdatafortheevaluationoftheRANAearlylearninginterventionprogrammeisprovidedbelow.TheanalysisiscategorisedintofourmainsectionsandfollowstheprogrammeToC:
• Theanalysisbeginswithdescriptionsofthecontextwithinwhichteachingandlearningtakesplace,includingschoolleadershipandmanagement,IQSintegration,schoolinfrastructureandresourcesandthegendersensitivityoftheschoolingenvironment.
• Thesecondsectionreportsonteachercharacteristics,knowledge,practicesandmotivations.Itbeginsbydescribingtheteacherswithinthesettingandthendescribeswhattheyknowandwhattheydointheirlessons,fromaneducationalperspective.Theanalysisthenfocusesontheknowledge,skillsandpracticesofteachersfollowingtheinterventionToCandthereforeincludestheToCintermediaryoutcomes,teacherknowledgeinliteracyandlanguageintheearlygrades,improvedteacherinstructionskillsintermsofactivelearningandincreasedtimeontask,theuseofHausa-basedteachingandlearningmaterials,andimprovedHausa-basedteachingintheearlygrades.Inaddition,wediscussteachermotivationandattitudesasimportantassumptionsunderlyingthecausalchain.
EvaluationoftheGirl’sEducationProject3
©EDOREN 50
• Thethirdsectionrelatestopupilsanddescribesthepupilswithinthesettingandtheircurrentlevelsoflearning.
• Theanalysisconcludeswithaninvestigationintotherelationshipsbetweenthecharacteristicswithintheschoolsetting,thecharacteristicsofteachersandteachingandlearning,andtheobservedHausaliteracyskillsofpupils.
Thebaselineanalysisofquantitativedataprovidesabenchmarkthatisausefulcomparisonformidlineandeventuallyendlinedata,andwillbeusedforassessingtheimpactoftheprogrammeunderevaluation,inthiscasetheGEP3earlylearningintervention.Atthesametime,thebaselinealsooffersthepossibilityofundertakingadescriptiveanalysisofassociationsofbaselineindicatorsacrosscategoriesofinterests.Thissectionprovidesasummaryanalysisofthemostrelevantandinterestingdescriptiveassociationsbetweenindicatorsandcategoriesofinterestthatemergedfromtheanalysisofbaselinedata.Itisimportanttobearinmindthattheseassociationsshouldnotbeusedformakingstatisticalclaimsregardingcausalinference,especiallygiventhefactthattherelativelysmallsamplesizesusedinouranalysis(e.g.teacher-andschool-levelsamples),andthefurtheranalysisofsub-samplesacrosscategories(e.g.femalepupilsinprimaryschools)isboundtoyieldmeanestimateswithlargeconfidenceintervals.Thedescriptivedifferencesdiscussedinthissectionshouldthereforeonlybetreatedasasummarypictureofourbaselinesample(weightedfortheextrapolationoftheresultstothepopulationsofschools,teachersandpupilsfromwhichitisdrawn).Thevisualanalysisintheformofbarchartsandlinegraphspresentedalongsidethetextserveadescriptivepurpose,tofacilitatethenarrativeontheresults.Alargersetofdescriptivestatistics,includingmeanestimates,standarderrorsandnumberofobservationsforeachvariableisincludedinAnnexK.Attheendofthissectionwealsopresentaregressionmodelthatfocusesoncorrelationsbetweenourmainlearningoutcomevariablesandtheirmaininfluencingfactors.Thiscorrelationanalysisallowsustomakemorestatisticallyrobustclaimsregardingtherelationshipbetweentheselectedexplanatoryandoutcomevariables,asisexplainedinmoredetailintherelevantparagraph.ThesebaselinefindingscanbeusedtoshedlightontherobustnessofGEP3’sToCassumptions,withafocusontheDACcriteriaofthe‘relevance’oftheprogramme’sdesign.Inparticular,thedescriptiveandregressionresultspresentedinthissectioncanhelptoinvestigatewhetherassumptionsregardingteachers’knowledgeandpractices,schoolenvironmentandpupils’characteristicsareconfirmedbythedata.However,thisstilldoesnotamounttoacausalanalysisofimpact,whichwillbecarriedoutinthenextstagesoftheevaluationthroughtheproposedRCTapproach.
3.4.1 Teachingcontext
Teachingcontextincludestheschoolleadershipandmanagementoftheschool,IQSintegration,schoolinfrastructureandresources,andthegendersensitivityoftheschoolingenvironment.
3.4.1.1 Schoollocationandsize
Thegreatmajorityofschoolsinourearlylearningsamplearelocatedinruralareas,withonlyaround14%ofschoolslocatedinurbanareas.TheproportionsaresimilarwhenwelookseparatelyatIQSsandprimaryschools,withthelatterslightlymorerepresentedinruralareas.
Theaverageschoolinthesamplehasaroundeightteachersintotal,thoughinthiscasethereisacleardifferencebetweenthetwotypesofschools,withpublicprimaryschoolsconsiderablylargerthanIQSs.Whenlookingatthesub-sampleofteacherswhoteachintegratedsubjects,theaverage
EvaluationoftheGirl’sEducationProject3
©EDOREN 51
numberacrossschoolsdropstoaroundsixduetothesmallnumber(aroundtwo)ofintegratedteachersinIQSsonaverage.Similarly,theaveragenumberofteachersperschoolacrossthewholeearlylearningsampledropsfurtherwhenlookingatteachersteachingearlygrades.Inthiscase,theaveragenumberisreducedbyasmallernumberofearlygradeteachersinpublicprimaryschoolswhencomparedtothewholeteacherroaster–whereas,inIQSs,practicallyallteachersengagedinintegratedsubjectsarefoundtobeteachingearlygradeclasses.Thegraphbelowhelpssummarisethiscomparison,whichprovidesanintroductoryideaoftheschoolsize.
Figure11: Schoolsizebasedonaveragenumberofteachers,byschooltype
Animportantpointtonoteisthatover40%ofIQSsintheearlylearningsamplereportedlyhaveonlyonesingleteacherofintegratedsubjects,withasimilarproportionofteachersteachingintegratedsubjectsinearlygrades.Theproportionofpublicprimaryschoolsthathaveonlyoneteacherofintegratedsubjects,ontheotherhand,ismuchlower,ataround3%.Giventhefocusoftheinterventiononteacherpeer-to-peerexchangesandcollaboration,thelargenumberofschoolswithonlyoneteacherisofconcern.Inparticular,thisfindingseemstosuggestthatschool-levelpeer-mentoringisnotapplicabletoalargeproportionofIQSs.Sincepeer-to-peerisanimportantassumptionunderpinningtheeffectivenessofthetrainingenvisaged,itisrecommendedthattheprogrammeidentifiesalternativewaysofensuringthatskillsandknowledgeacquiredbyteachersthroughtheearlylearninginterventionaresharedamongstteacherspossiblyfromdifferentbutnearbyschoolswithonlyasinglefacilitator.
Thissectioncontinueswithadescriptionoftheenvironmentinwhichteachingandlearningtakesplace.Thisincludesadiscussionofrelevantinformationatourdisposalonleadershipandmanagement,schoolinfrastructureandexternalmonitoring.ItalsoincludesashortsectiononIQSs’integration.Thesearefactorsthatmayinfluencetheteachingpracticesintheschools.
3.4.1.2 Schoolleadershipandmanagement
Informationwascollectedattheschoollevel,throughtheheadteachersurvey,onarangeoffactorsthatcanbeassociatedwithschoolleadershipandmanagement.Withinourearlylearningsampleofschoolstheoverallpicturethatemergesfromthedescriptiveanalysisseemstoshowarelatively
0 2 4 6 8 10 12
Numberofteachersinearlygrades
Numberofteachersofintegratedsubjects
Numberofteachersintotal
PublicSchools IQSchools AllSchools
EvaluationoftheGirl’sEducationProject3
©EDOREN 52
positivesituationwhenitcomestotakingactiononteacherandpupilattendance.Asshowninthegraphbelow,around70%ofheadteachersreporttakingactiontoimproveteacherattendance,andthisproportionrisestoover90%whenitcomestotakingactiontoimprovepupils’attendance.Lesspositiveistheproportionofheadteacherswhohaveobservedalesson,whichisjustunder50%.Also,thepercentageofheadteacherswhohavenomeetingswithteachersorwhomeetthemlessthanonceamonth,includingindividualorgroupmeetings,isaround50%.
Figure12: Prevalenceofheadteachermanagementactions
Whilstgenderdoesnotseemtobeafactorinthelevelofschoolleadershipandmanagementrecorded,theheadteacherageappearstohavearelationshipwiththis.Asshowninthegraphbelow,olderheadteachersarefoundtobemoreproactivethanyoungeronesonteachermanagement:amongstheadteachersolderthan50,over50%observedalessonandalmost70%tooksomeformofactiontoimproveteacherattendance;inbothcases,theproportionoftheyoungestgroupofteachers(under25yearsofage)observingalessonortakingactionismuchlower,atjustunder20%.Thistrenddoesnotapplytoimprovingpupils’attendance,forwhichpracticallyallteachersacrossthedifferentagegroupsarereportedtobetakingsomeformofaction.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Observedalesson Tookactiontoimproveteacherattendance Tookactiontoimprovepupilattendance
EvaluationoftheGirl’sEducationProject3
©EDOREN 53
Figure13: Actionstakenandagegroups
Cleardifferenceswithregardtoobservingteachersandtakingactiontoimproveteacherattendancearealsonoticeablewhencomparingstatesandschooltypes,withagainnonoticeabledifferencedetectedregardingactiontakentoimprovepupils’attendance.Whilstalmost60%ofheadteachersobservedalessonand75%tookactiontoimproveteacherattendanceinZamfara,theequivalentproportionsinKatsinaare34%and67%.EvenmoreaccentuatedisthedifferencebetweenpublicprimaryschoolsandIQSs:only53%ofheadteachersinIQSstakeactiontoimproveteacherattendance,whichcomparestoalmost90%ofheadteacherstakingactioninpublicprimaryschools.Itseemsthereforethatthewillingnessandabilityofheadteacherstoleadandimproveonteachingintheirschools,whichisanassumptionoftheprogramme’sToC,mightbeanissueinQur’anicschools.ThismayalsobeduetothedifferenttypeofleadershipstructureinIQSs,whereheadteachersarenotalwaysthesamepersonastheschool’sprivateproprietor.Althoughthesizeforthesub-sampleofheadteachersinIQSsistoosmalltoprovidestatisticalrobustness,theindicationthatseemstoemergeisthatwhentheheadteachersarenotalsotheproprietortheyarelesslikelytotakeactiontoimproveteacherattendance90.Itwillbeparticularlyimportantfortheearlylearninginterventiontoensurethattherelevantchainofteachingresponsibilitywithineachschoolisclearlyunderstoodanditsinfluencemaximisedwhenrunningthetraining.
Interestingly,thepersonalcharacteristicsoftheheadteacheralsoseemtoplayaroleintheactiontakentoimprovetheattendanceofteachers.Havingsomeformofqualification,oratleastaSeniorSecondaryCertificateExamination(SSCE)academicdegree,isassociatedwithahigherprobabilityoftakingactiontoimproveteacherattendance.Atthesametime,andevenmoreimportantlyinthecontextofGEP3,headteacherswhohaveattendedsomeformoftrainingarereportedlymorelikelytotakeactiontoimproveteacherattendance:theproportionofnon-trainedteacherswhotakeactionis63%,whilsttheproportionoftrainedteacherswhotakeactionis74%.
90Amongstthe26IQSheadteachersobservedwhowerereportedlyalsoschoolproprietors,18(69%)declaredthattheytookactiontoimproveteacherattendance,whilsttheproportionisapproximately50%forthenon-proprietorheadteachers.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Observedalesson Takeactiontoimproveteacherattendance
Under25 25to34 35to49 Over50
EvaluationoftheGirl’sEducationProject3
©EDOREN 54
3.4.1.3 Externalmonitoring
Aspartoftheearlylearningintervention,regularmonitoringvisitsbymastertrainersandgovernmentstaffareplannedfor.Itisthereforeinterestingtoexamineatbaselinewhethertheschoolsreceivedanymonitoringvisitsfromgovernmentofficialsorotherexternalagentsduringthelastschoolyear.Accordingtoheadteacheraccounts,almost80%ofschoolsreceivedamonitoringvisit.Amongsttheschoolsthatreceivedthevisits,theaveragenumberofvisitperschoolisaroundeight.Visitswereperformedbyarangeofgovernmentagenciesandschoolswerevisitedbymultiplevisitingofficers.Aroundtwo-thirdsofschoolswerevisitedbyanLGEAofficer,withthetwoothermostprevelantcategoriesofvisitorsbeingSUBEBofficersandrepresentativesfromNGOsandotherdonoragencies.
ThereappearstobeacleardifferencebetweenIQSsandpublicprimaryschoolsinthelevelofexternalmonitoringthattheyreceive.Practicallyallpublicprimaryschoolswerefoundtoreceivesomemonitoringvisits,whileonly58%ofIQSsreportedhavingreceived(anytypeof)monitoringvisitduringthelastschoolyear.Moreover,amongstthoseschoolsthathavereceivedavisit,thenumberofvisitsagainvariesconsiderablybetweenpublicprimaryschoolsandIQSs.Thelatterreportedhavingreceivedjustoverthreevisitsonaverage,whilstfortheformerthefigureisover11visits.Over50%oftheIQSsreceivedoneortwomonitoringvisits.ItwillthereforebeimportanttopayparticularattentiontomonitoringpatternsinIQSssoastoensurethatvisitsbyGEP3trainersandotherstakeholdersoccurasfrequentlyasforpublicprimaryschools.
3.4.1.4 IQSintegration
Theearlylearninginterventionwillbeimplementedin80IQSsinKatsinaandZamfara.TheteachingconditionsinIQSsdiffersignificantlyfromthoseinpublicprimaryschools.Asmentionedabove,thenumberofteachersteachingintegratedsubjectsatalllevelsintheIQSsincludedinourearlylearningsampleisjustovertwoonaverage,whichisconsiderably(butnotsurprisingly)lowerthanthenumberinpublicprimaryschools,atjustover10perschool.ThefractionoffemaleteachersamongstthoseteachingintegratedsubjectsismuchlowerinIQSs,whereonly4%ofnon-religioussubjectteachersarewomen(comparedto14%inpublicprimaryschools).
Acrossall120IQSsinourearlylearningsample,wefoundthat72%ofschoolsarestructuredlikeapublicprimaryschool,16%arestructuredintostagesand12%haveadifferentstructure.Astagereferstoagroupinginwhichpupilsaretaughtthepublicprimaryschoolsyllabuscorrespondingtotheirrelevantgrade.AmongstthegroupofIQSsthatarestructuredintostages,thegreatmajorityreporthavingthreestages,whichcorrespondtothreelevels–ascomparedtothepublicprimaryschoolstructure,whichhassixlevels.Thegraphbelowshowsthedistributionofthesetwoschoolstructuretypes(i.e.publicprimaryschoolsstructurevs.otherstagecomposition)inIQSs.
EvaluationoftheGirl’sEducationProject3
©EDOREN 55
Figure14: SchoolstructureinIQSs
WhencomparingIQSsinKatsinaandZamfara,itisnotablethatalargerproportionofschoolsarestructuredlikepublicprimaryschoolsinKatsinathaninZamfara.TheaveragenumberofyearsthatIQSshavebeenintegratedishigherinZamfarathaninKatsina.Whilsttheheadteachercharacteristics,includinggenderandroleasaproprietor,donotseemtohaveaclearassociationwithintegration,itisthelocationoftheschoolthatagainappearstoberelevant.Alargerproportionofschoolsarestructuredlikepublicprimaryschoolsinurbanareaswhencomparedtoruralareas,andschoolsinurbanareashavebeenintegratedforlonger.
3.4.1.5 Schoolinfrastructureandresources
Infocusingonschoolinfrastructuralstatusitisclearthatalmostallschools(95%)areinneedofrepairs,withonlyaslightlylargerproportionofIQSs(97%)inneedofrepairscomparedtopublicprimaryschools(92%).Locationdoesnotseemtomakeanydifference,withschoolsinurbanandruralareasofbothKatsinaandZamfaraequallyinneedofschoolrepairs,althoughsubstantialdifferencescanbedetectedbetweenstatesacrossotherinfrastructuralvariables(seeFigure15).Asexpected,alargerproportionofschoolsinurbanareashaveaccesstowatersourcesandelectricity.RelativelymoreschoolsinZamfarareporthavingaccesstowater,electricityandfunctionaltoiletsforgirls.ThegraphbelowshowshowtheinfrastructurallevelofschoolsinZamfaraappearstobebetteraccordingtothreekeyaspects.
0%
10%
20%
30%
40%
50%
60%
70%
80%
IQschools
Publicschoolstructure Schoolhas2stages Schoolhas3stages Schoolhas1stage
EvaluationoftheGirl’sEducationProject3
©EDOREN 56
Figure15: InfrastructureinKatsinaandZamfara
WhilsttheproportionofIQSsandpublicprimaryschoolsthathaveaccesstoasourceofdrinkingwaterissimilar,thereisacleardifferencewhenitcomestoelectricity,with37%ofIQSshavingelectricity,comparedtoonly9%ofpublicprimaryschools.However,onaverage,itwouldseemthatpublicprimaryschoolsarebetterresourcedthanIQSs.Forexample,publicprimaryschoolswerefoundtohaveconsiderablymoreroomsthanIQSs,aswellasmorefunctioningtoilets.Atthesametime,whilstalmost20%ofpublicprimaryschoolshaveacollectionofbooks,thispercentagedropsto4%amongstIQSs.Finally,over90%ofpublicprimaryschoolshaveaplaygroundorsportsarea,comparedtounder20%ofIQSs.
Schoolconditionsformapartofthebroadlearningenvironmentinwhichpupilsfindthemselves.ThefactthatpublicprimaryschoolsappeartobeonaveragebetterresourcedthanIQSsisafactorthatwillhavetobetakenintoaccountintheimpactestimationanalysisatendline.Specifically,school-relateddifferencesindescriptiveandexplanatoryfactorswillhavetobediscountedfromtheimpactanalysistoisolatetheeffectsoftheearlylearninginitiativeonpupillearning.Atthesametime,theprogrammeimplementationshouldbeawareofthedifferencesbetweenpublicprimaryschoolsandIQSsinregardtothecollectionofbooksintheschools,forinstance,asthesereflectdifferentschoolenvironmentsthataremoreorlessconducivetolearning.
3.4.1.6 Girl-friendliness
BecauseofGEP3’sfocusongirls’educationwehavecollecteddataonsomemeasuresoftenassociatedwiththegirl-friendlinessoftheschoolenvironment.However,itischallengingtotrytoachieveareliablemeasureofthegirl-friendlinessofschools,asdataongender-sensitiveschool-levelfactorsaredifficulttodefineandcollect.Theinformationthatwasobtainedonthisforourearlylearningsamplecannotthereforebeconsideredexhaustive.Overall,30%ofschoolswerefoundtohaveseparatefunctionaltoiletsforgirls.Thisaverageproportionisclearlydrivenbyfacilitiesfoundinpublicprimaryschools,as48%ofthesehaveseparatetoiletscomparedtoonly7%ofIQSs.Oneinterestingaspecttoreportisthatthreeoutofthefiveschoolsinourearlylearningsamplethathaveafemaleheadteacherhaveafunctioningseparatetoiletforgirls.Theproportionismuchlowerfor
0%
10%
20%
30%
40%
50%
60%
Electricity Schoolhasasourceofdrinkingwater Schoolhasseparatefunctionaltoiletforgirls
Katsina Zamfara
EvaluationoftheGirl’sEducationProject3
©EDOREN 57
therestoftheschools,whichhaveamaleheadteacher(29%).Althoughofinterestasanecdotalevidence,theverysmallandunbalancedsampledoesnotallowforanyrobustdescriptiveanalysisofthistrend.
SomedifferencesarealsoobservedwhencomparingKatsinawithZamfara,withschoolsinKatsinashowingahigherratioofgirlstoboysinP1toP3andoverall(around0.8)thanZamfara(around0.6)91.Interestingly,whilsttheaverageratioofgirlstoboysinP1toP3wasrecordedas0.73,thegenderratioreflectingthosepresentonthedayofthesurveyteam’svisittoP2classeswas1.8onaverage.Thismaypointtowardsabiasduetoknowledgeoftheanticipatedpresenceofsurveyorsintheschool.Whilsttheproportionoffemalepupilsisrelativelyhigh,asalreadydiscussedtheproportionoffemaleteachersinoursampledschoolswasfoundtobelowandthisiscorroboratedbythesub-sampleofteachersspecificallyteachingintegratedsubjects,forwhichtheproportionoffemaleteachersisaround10%.
3.4.2 Teachercharacteristics,knowledge,practiceandmotivations
Thissectionreportsonteachercharacteristics,knowledge,practicesandmotivations.Itbeginsbydescribingtheteacherswithinthesettingandthendescribeswhattheyknowandwhattheydointheirlessons.Thebaselinefindingsrelatedtotheknowledge,skillsandpracticesofteachersarereportedon.FindingsrelatedtothefollowingkeyintermediaryoutcomesintheToCarediscussed-teacherknowledgeinliteracyandlanguageintheearlygrades,improvedinstructionskillsinactivelearningandincreasedtimeontask,theuseofHausa-basedteachingandlearningmaterials,andHausa-basedteachingintheearlygrades.Teachermotivationandattitudesarealsodiscussedasthesehaveaninfluenceonthequalityofteaching.
3.4.2.1 Teachercharacteristics
Age,sexandlanguage
Thetypicalteacherteachingintheearlygradesisa36-year-oldmalewhospeaksHausaandEnglish.Morespecifically,thegreatmajorityofteachers(85%)intheearlylearningsamplearemale.Theratiochangesdependingonthetypeofschool:IQSfacilitatorsarenearlyallmale,withonly3%offacilitatorsbeingfemale,whileacrosspublicprimaryschoolsabout20%ofteachersarefemale.Interestingly,therealsoseemstobeadifferencebetweenthetwostates,withalargerproportionoffemaleteachersinKatsinathaninZamfara.92Notsurprisingly,alargerproportionoffemaleteachersarealsofoundinurbanareasascomparedtoruralareas.
Theaverageageofmaleteachersisslightlyhigherthanfemaleteachers,whichisalsodetectablewhenlookingattheagegroupdifferentiation.Whilst13%ofmaleteachersareabove50yearsofage,only1%offemaleteachersbelongtothatolderagegroup.Acrossallteachers,over80%belongstotheagegroupspanningfrom25to50years,withonly6%youngerthan25.WhencomparingIQSsandpublicprimaryschoolsthough,theproportionofyoung(below25)teachersisconsiderablyhigherinIQSs(11%)thaninpublicprimaryschools(4%).Conversely,thereisahigherproportionofolder(above50)teachersinpublicprimaryschools(13%)thaninIQSs(9%).
91TheASCdataconfirmthehigherratioofgirlstoboysinKatsinacomparedtoZamfara.TheratioinP1–3equals0.78inKatsinaand0.54inZamfara.92Thisisinlinewiththe2014–2015ASCdata,whichreportthat23%ofprimaryschoolteachersinKatsinaarefemale,comparedto15%inZamfara.
EvaluationoftheGirl’sEducationProject3
©EDOREN 58
Whenitcomestolanguage,allteachersindicatethattheyspeakHausa,thegreatmajority(85%)alsostatethattheyspeakEnglish,whilstonlyaroundone-fifthofteachersspeakArabic93.Notsurprisingly,arelativelylargerproportionofteachersspeakArabicinIQSsthaninpublicprimaryschools,asArabiccanbeassociatedwiththestudyoftheQur’an.Atthesametime,whilstaround90%ofteachersclaimtospeakEnglishinpublicprimaryschools,thepercentagedropstoaround70%inIQSs.ThereisnosizeabledifferencebetweenteachersinKatsinaandZamfarawhenitcomestolanguagesspoken.
Classestaught
Allsurveyedteachers(excludingheadteachers)wereteachingP1toP3orequivalentearlylevelatthetimeofthebaselinestudy.ThisisafeatureofthesamplingapproachtakenandisnotrepresentativeofteachingresponsibilitiesinGEP3schools.Thefollowinganalysisprovidesdescriptionsofearlyyearsteacherswithinoursample.Aminorityofthem(around30%)alsotaughtP4toP6classes,thusbeinginvolvedinbothupperandlowerlevelsofteaching.ThiswasparticularlyprominentinKatsina,where55%ofteacherstaughtfromP4toP6,comparedtoonly17%inZamfara.Atthesametime,whilst41%ofteachersinpublicprimaryschoolswerefoundtobeteachingatbothupperandlowerlevels,thisproportiondropsto5%inIQSs.
Onaverageacrosspublicprimaryschoolsjustunder50%ofteachersarefoundtobeteachingonlyonenon-religioussubjectandaround20%ofteachersteachmorethantwonon-religioussubjects.ThereisacleardifferencewithIQSs,wheretheproportionofteachersthatonlyteachonenon-religioussubjectisalmost60%andonly10%teachmorethantwonon-religioussubjects.Interestingly,theproportionofteachersteachingmultiplenon-religioussubjectsisconsiderablyhigherinKatsinathaninZamfara,wherealmost60%ofteachersonlyteachonenon-religioussubject,comparedtojustover30%inKatsina.Finally,whenspecificallylookingatIQSs,thedatashowthatamongsttheintegratedsubjects,HausaandMathsaretaughtby50%ofteachers,Englishbyjustover40%,whilstSocialStudiesandBasicScienceareonlytaughtby5%and2%ofIQSteachers,respectively.Thiscomparesnegativelytopublicprimaryschools,where30%ofteachersarefoundtoteachSocialStudiesandaround25percentteachBasicScience.
ItisimportanttonotethatthisinformationisderivedfromtheteacherquestionnairewhichwasadministeredtooursampleofpublicprimaryschoolandIQSteachers.However,informationcollectedfromheadteachersonsubjectstaughtintheirschoolsshowsadifferentpictureattheschoollevel,withover90%ofschoolsreportedlyteachingMathsandHausa,over90%ofschoolsteachingEnglishandjustover50%ofschoolsteachingSocialStudiesandBasicScience.Thedifferencebetweenthetwotypesofschoolsisalsomarkedinthiscase,withmostpublicprimaryschoolsreportedlyteachingeverycoresubjectandonlyaround10%ofIQSsteachingSocialStudiesandBasicScience.TheproportionofIQSsteachingMathsandHausaisover90%andover70%arereportedlyteachingEnglish.ThedivergenttrendbetweenpublicprimaryschoolsandIQSsisconsistentacrossthetwosourcesofinformation.
Teachingexperienceandqualifications
Thegreatmajorityofteachers(includinginthiscaseheadteacherswithteachingresponsibilities)arereportedtohavesomeformofprofessionalqualification.Inparticular,70%ofteachershaveat
93Informationonlanguagewascollectedfromteachersonlybutnotfromheadteacherswhoalsoperformteachingduties.Similarly,informationonteachinginearlygradeswascollectedfromthesamesampleofteachersbutnotteachingheadteachers.
EvaluationoftheGirl’sEducationProject3
©EDOREN 59
leastaGrade2qualificationorequivalent,whilst54%alsopossessaNationalCertificateinEducation(NCE).Whenlookingatthegenderdifferentiation,almost80%offemaleteachershaveanNCE,comparedtoonly50%ofmaleteachers,thelatterformingaround85%oftheearlylearningsample,asmentionedabove.MostoftheNCEholdersarefoundinpublicprimaryschools,with70%ofteachersinpublicprimaryschoolshavinganNCE,comparedtoonly22%ofteachersinIQSs.Overall,thedifferencebetweenschooltypesisslightlyhigherwhenlookingatanytypeofqualification,as86%ofpublicprimaryschoolteachershavesomeformofprofessionalqualification,comparedto36%inIQSs.Furthermore,while64%ofteachersinKatsinahaveanNCE,only48%inZamfarado.AlargerproportionofteachersinurbanareashaveanNCEqualificationthanthoseinruralareas.
Whenitcomestogeneraleducation,over70%ofteachersacrossthetwoschooltypeshavetakentheSSCEorequivalent,around15%havealoweracademicdegree(primaryorJSS)andonly2%haveauniversitybachelor’sdegree(ahigherproportionhaveadiplomacertificate).Thelatterareonlymen,butamongstthesmallsub-sampleoffemaleteachers,over93%haveanSSCEdegree.Theredoesnotappeartobeagreatdifferenceinacademiceducationbetweenthetwotypesofschoolsorthetwostates.Eventheproportionofteacherswithareligiouseducationissimilarataround20%inbothpublicprimaryschoolsandIQSs.
Figure16: Teacherqualificationandeducationbyschooltype
Amoreobservablepatternofdiversificationacrossschoolsandstatesisobservableforteachers’experience.MoreexperiencedteachersarefoundinpublicprimaryschoolsandinZamfara.Thisisevidentfromboththeindicatorscoveringthetotalnumberofyearsofexperienceandthosespecificallyfocusingonexperienceinthesameschoolinwhichteachersweresurveyed.Acrossthewholesample,theaveragenumberofyearsofexperienceis10,with89%ofteacherswithatleasttwoyearsofexperienceinanyschoolandalloftheteacherswithatleasttwoyearsofexperienceintheschoolwheretheyweresurveyed.Thisindicatesagenerallyhighlyexperiencedpoolofteachersinourbaselinesample,withnonoticeabledifferencebetweengenders.TeachersinpublicprimaryschoolsaremoreexperiencedthanthoseinIQSs,withtheaveragenumberofyearsasateacherinanyschool(includingtheschoolwheretheyweresurveyed)beingover11yearsinpublicprimaryschoolsandundereightyearsinIQSs.
0%
10%
20%
30%
40%
50%
60%
70%
80%
TeacherhasaGrade2orequivalent
TeacherhasanNCE HighestqualificationisSSCE
HighestqualificationisBA/BSc
Teacherqualifiedinreligiouseducation
IQschools Publicschools
EvaluationoftheGirl’sEducationProject3
©EDOREN 60
Justover40%ofteacherswerefoundtohaveattendedsomeformoftraininginthelasttwoyears.Theproportionisslightlyhigherformen(43%)thanwomen(37%),forteachersinKatsina(44%)comparedtothoseinZamfara(41%),andishigherinIQSs(46%)thaninpublicprimaryschools(41%).Interestingly,teacherswhohaveundertakentrainingarealsodisproportionatelyrepresentedamongstthoseteacherswhoreceivesomeformofpayment:41%comparedto35%.Interestingly,alargerproportionofteachersaretrainedinruralareas,yetalthoughthisfigureisarelativeoneitmayalsoreflectthedisproportionatelylargesub-sampleofruralschoolsinourearlylearningsample.Whatemergesclearlyfromthedataisthatteacherswithsomeprofessionalqualificationaremorelikelytohavereceivedtraining,asalmosthalfofteacherswithaprofessionalqualificationhaveattendedtraining,comparedtojustover30%ofnon-qualifiedteachers.Whenthetypeoftrainingreceivedamongstthesub-sampleoftrainedteacherswasinvestigatedthemajorityofthetrainingwasreportedtobeoncurriculumsubjectsandliteracyandnumeracy,witharound30%ofteachersreportedlytrainedonschoolleadershipandmanagementandplanning.Onlyaround20%ofteachersweretrainedspecificallyonHausa.Almosthalfofthetraining(47%)wasconductedbyUNICEFaspartofGEP,withtheotherorganisersbeinggovernmentpartnerassociations(e.g.SUBEBsandLGEAs).Theassumptionthattrainingisrelevantandtargetedisacriticaloneandthefactthatonly20%ofteacherstrainedinthepastattendedHausacoursesmaybeanindicationthatthereismoreinterestinotherteachingandmanagement-relatedsubjects.However,itcouldalsoindicatethatHausacourseswerenotoffered.Inanycase,giventhefocusoftheearlylearninginterventiononHausaliteracyteachingandlearningoutcomesitwillbeimportanttoensureastrongfocusontrainingspecificallyonHausaforearlylearningtraining.
3.4.2.2 Whatteachersknowanddo
Thissectionofthereportdescribesteachers’knowledgeandskillsandpractices,ascollectedthroughthreeprimarysources:ateacherassessment,aclassroomobservationandateacherquestionnaire.Teachers’knowledgeandskillsreferstoteachers’proficienciesasperthesubscalesdrawnfromtheteacherassessmentandaqualitativereviewofteachers’responsestoopenendedquestionswithintheassessment.Teacherpracticesreferstothepracticesofteachersintheclassroom,asobservedthroughclassroomobservation.Teacherpracticeswerecategorisedintothreecategories:teachertalk,teacheractionandpupilaction.ThesectionthenreportsonthebaselinevaluesofrelatedintermediaryoutcomesintheearlylearninginterventionToC.
Whatdoteachersknow?
Theteacherknowledgeandskillstestwasdividedintothreesections,collectivelycomprising30items,includingmultiplechoice,shortresponseandlongresponseitems.Insectiononeteacherswereaskedtomarkpupils’responsestoHausaliteracyquestionsandtoindicatethegradelevelatwhichtheanswershouldbeknownbypupils,asdefinedbythecurriculum.InsectiontwoteacherswereaskedtofillinmissinginformationinordertoprepareananswersheetforareadingtestaimedatGrade2pupils.Teacherswereprovidedwithtwonewspaperarticlesandaskedtofillinmissinginformation,includingbasiccomprehensionquestionsandquestionsthatrequiretheinterpretationofwordsandphrases.Sectionthreeaskedteacherstoidentifypoorandgoodpupils’workandtoreviewpupils’workinordertomakejudgementsaboutpupils’writing,includingtheorganisationofthewriting,theuseofgrammar,punctuation,spelling,thepupils’abilitytoself-correctandreflectontheirwritingandthepupils’abilitytoformlettersandusespacesbetweenwords.Teacherswerethenaskedtodescribehowtheymightsupportthepupilsinimprovingtheirwriting.
EvaluationoftheGirl’sEducationProject3
©EDOREN 61
Definingteachercompetence
Thepercentageofteacherswhodemonstrateminimumteachingknowledgeinliteracyandlanguageismadeupofthesixsubscalesdevelopedfromtheteacherassessment.
Thresholdsweredefinedforeachsubscaletodifferentiatebetweenteacherproficiencylevels.Twocut-offpointsweredefinedforeachscaletocreatethreeproficiencylevelsperscale.Theproficiencylevelsincludelowband–noevidenceofskill,middleband–evidenceofrudimentaryskill,andupperband–evidenceofcompetence.
AscanbeseeninFigure17,inthemajorityofareastherewasnoevidencethatteachershavetheknowledgeandskillsrequiredtoteacheffectively.Rudimentarylevelsofskillswereobservedinasmallproportionoftheassessedteachersinidentifyinglowperformers,evidencingjudgementsanddiagnosingwritingskillsandinterpretingwordsandphrases.Lessthan3%ofteacherswerefoundtobecompetentinidentifyinglowperformersandinterpretingwordsandphrases.AgreaterproportionofteacherswereabletodisplayknowledgeandskillsinGrade2Hausaandincomprehensionthaninotherareasofteacherknowledgeandskills.However,inabsoluteterms,theshareofteacherswhowereabletodisplaycompetenceinthisareawasfairlylow,atjust40%.Furthermore,itisimportanttostressthatthelevelsofHausaassessedwerebenchmarkedtoGrade1-and2-levelHausaknowledge.Itemstestedincludedbasicgrammarandtheinitiallettersofeverydayobjectsandanimals.
Thefindingthatover60%ofteacherswereunabletodisplaycompetenceinGrade1-and2-levelHausaishighlysignificantforanearlylearninginterventionthatfocusesonteachinginHausa.ThisisrecognisedintheearlylearningToC:oneoftheassumptionsintheToCisthatteachersareliterateinHausa.Theeducationliteratureindicatesthattwokeydeterminantsofteachers’abilitytoraisepupillearningaretheirbehaviouralskills(teachingstyleandbehaviouralskills)andtheirowncompetence(Dundaretal.2014).Thisimpliesthatteacherswholackcompetenceinearlygrade-levelHausaareunlikelytobeabletoteachthissubjecttopupils.Inlinewiththisfinding,thebaselinefindingshighlightthattheRANAinterventionwillneedtofocusnotjustonpedagogicalandcurriculumknowledge,butalsoonsubjectknowledge,includingofP1andP2-levelHausa.
Anothernotablefindingisthatnoteachersintheearlylearningschoolsarecompetentinevidencingjudgementsanddiagnosingpupils’workorwriting.Thereisaverystrongandlong-standingperspectiveineducationalresearchthatastudentlearnsbestwhenteachingistargetedtowhats/heisreadytolearn.PsychologistLevVygotskyproposedthis90yearsago(knownasthe‘zoneofproximaldevelopment’,Vygotsky1997).Thisconcepthasbeenmainstreamedineducationaltheoryandpracticesincethe1970sanditiswidelyrecognisedthatteachersshouldtargetteachingbasedonreliableevidenceofwhatstudentsknowandarereadytolearn(Griffin2014;Masters2013p.15;AndersonandScamporlino2013;CentreforEducationStatisticsandEvaluation2015).
TheAmericanpsychologistDavidPaulAusubelhasstated:‘IfIhadtoreduceallofeducationalpsychologytojustoneprinciple,Iwouldsaythis:themostimportantsinglefactorinfluencinglearningiswhatthelearneralreadyknows.Ascertainthisandteachhimaccordingly.’(CitedinMasters2013).
Theinabilityofthevastmajorityofteachersinthesampletoidentifylowperformers,evidenceteacherjudgementsconcerningpupilperformanceanddiagnosethenextstepsofteachingpresentsseverechallengestoimprovingpupillearninginKatsinaandZamfara.
EvaluationoftheGirl’sEducationProject3
©EDOREN 62
Figure17: Percentageofteachersachievingwithinthelower,middleandupperbandsofcompetenceacrosstheteacherknowledgeandskillssubscales
Thewritingskillsofteachersarealsoanareathatraisesconcernsregardingtheirabilitytoincreasepupillearningoutcomesatscale.Whilenoteacherswerefoundtobecompetentinwriting,justover2%displayedarudimentarylevelofwritingskills.Inordertoinvestigatetheextenttowhichteachers’knowledgeandskillsareassociated,ananalysiswasundertakenofthecorrelationbetweenthesubscalesdevelopedfromtheteacherknowledgeandskillsassessment.Thetablebelowshowsthecorrelations(Pearson’s)betweenthesubscales.
Table13: Correlationbetweensubscalesofteachers’knowledgeandskills
Syllabu
sKno
wledg
e
Abilityto
iden
tifylowperform
ers
Abilityto
evide
ncejudg
emen
tsand
diagno
se
Teache
rwritingskills
Teache
rHau
sakno
wledg
e
Teache
rcom
preh
ension
skills
Interpretin
gwordsand
phrases
‘Syllabusknowledge’
PearsonCorrelation
1
Sig.(two-tailed)
N 2935
‘Abilitytoidentifylowperformers’
Correlation .051 1
Sig.(two-tailed) .218
Abilitytoidentifylowperformers
Abilitytoevidence
judgementsanddiagnose
Teacherwritingskills
TeacherGrade2Hausa
Knowledge
Teachercomprehension
skills
Interpretingwordsandphrases
UpperBand 2.3 0 0 39.9 33.6 2.0
MiddleBand 6.2 .2 2.1 23.1 15.3 2.9
LowerBand 91.5 99.8 97.9 37.0 51.0 95.2
0102030405060708090
100
Percen
t
EvaluationoftheGirl’sEducationProject3
©EDOREN 63
‘Abilitytoevidencejudgementsanddiagnose’
Correlation .224** .188** 1
Sig.(two-tailed) .000 .000
‘Abilitytobuildonpupilknowledge’
Correlation -.103* .083* .415**
Sig.(two-tailed) .013 .043 .000
‘Teacherwritingskills’Correlation .272** .527** .476** 1
Sig.(two-tailed) .000 .000 .000
‘TeacherHausaknowledge’
Correlation .219** .273** .171** .214** 1
Sig.(two-tailed) .000 .000 .000 .000
‘Teachercomprehensionskills’
Correlation .068** .246** .134** .227** .228** 1
Sig.(two-tailed) .000 .000 .001 .000 .000 1
‘Interpretingwordsandphrases’
Correlation .159** .141** .157** .528** .124** .225**
Sig.(two-tailed) .000 .001 .000 .000 .000 .000
*Correlationissignificantatthe0.05level(two-tailed),and**thecorrelationissignificantatthe0.01level(two-tailed).
Therearefourcorrelationsinthistablethatarenoteworthy(showninred):
• ‘Abilitytobuildonpupilknowledge’with‘Abilitytoevidencejudgementsanddiagnose’(r=0.415)
• ‘Teacherwritingskills’with‘Abilitytoidentifylowperformers’(r=0.527)
• ‘Teacherwritingskills’with‘Abilitytoevidencejudgementsanddiagnose’(r=0.476)
• ‘Interpretingwordsandphrases’with‘Teacherwritingskills’(r=0.528)
Thesecorrelationssuggestthatteachers’writingskillsinparticularmaybethekeydimensioninfluencingteachers’performanceinthetest,andperhapsalsotheperformanceoftheteachersinthesetasksintheirclassrooms.Inotherwords,theliteracylevelsoftheteachersappeartobeakeyissue:iftheselevelsarelowtheylimitperformanceacrossarangeoftheareasteachersneedtobecompetentintoimprovepupillearning.
Qualitativereviewofteacherresponses
Severalthemesemergedfromaqualitativereviewofincorrectteacherresponsestoitemsintheteacherknowledgeandskillsassessment.Examplesofresponsesfromtwosectionsoftheteacherassessmentareprovidedwithinthissectionofthereport,thatrepresenttheemergingthemes.Onesectionoftheteacherassessmentprovidedalistofpupilnameswithinaclass,thegenderofeachchildandthenumberofwordseachchildcanreadperminute.Teacherswereaskedto:(1)describetheachievementofgirlsandboysintheclass;(2)identifywhichpupilsneedthemostassistanceandsupport;and(3)provideexamplesofwaysinwhichateachercouldsupportpupilstoincreasetheirabilitytoreadandunderstandtext.Thestimulusmaterial(listofnamesandreadingproficiency)describedaclasswhereabouthalfofthepupilsscoredzeroandtherewasasmalldifference
EvaluationoftheGirl’sEducationProject3
©EDOREN 64
betweenboys’andgirls’achievement,withboysscoringslightlybetterthangirlsonaverage.Teacherswerealsoprovidedwithexamplesofpupils’writingandwereaskedtodescribewhattheteachershouldfocusontoimprovethepupil’swriting.
Teachersgenerallyperceivedpupileffortasthecauseoflowperformanceandthereforeperceivedincreasingpupil(andtosomeextentteacher)effortasthebestresponsetolowperformance.EarlylearningexampleAprovidesanexampleofateacherresponsethatfocusesonincreasingeffortandpayingattentioninclass,ratherthandisplayingknowledgeregardingtheerrorsinthepupil’sworkandhowateachermightmeaningfullyimprovethepupil’swriting.
EarlylearningexampleA:Studenteffort
Secondly,extremelylowlevelsofteachercompetencewereobserved.Thiswasevidencedinteacherresponsesthatclearlyhighlightedtheinabilityoftherespondingteachertodifferentiatebetweensubjects.InearlylearningexampleBtheteacherproposesusingstonestohelpthepupilscount,inordertoincreasepupils’abilitytoreadandunderstandtext.
EarlylearningexampleB:Differentiatingbetweensubjects
Thirdly,teacherresponsesreflecteddevelopmentprogrammeaimsandobjectivesinsuperficialorunevidencedways.ExamplesoftheseresponsesincludeearlylearningexamplesCandD,inwhichteacherresponsesstatetheimportanceoffocusingongirlsintheclasswithoutanysupportingstatementslinkingthisimportancetotheidentificationofgirlsaslowperformersintheclass.InexampleD,theteacher(incorrectly)statesthattheperformanceofboysandgirlsisequal,butthenstatesthatgirlsneedmoresupportandassistancefromteachers.
EarlylearningexampleC:Assistinggirls
EvaluationoftheGirl’sEducationProject3
©EDOREN 65
EarlylearningexampleD:Assistinggirls
Whilesometeachersrespondedwith‘politicallycorrect’statementsregardingassistinggirlsintheclassroom,otherresponsespointedtogenderbiases:perceivinggirlsasnaturallylesssuitedtothedemandsofeducationthanboys,ormoresuitedtootherroleswithinsociety.EarlylearningexampleEreflectsoneteacher’sperceptionregardingtheinherentlackofcapacityofgirls.
EarlylearningexampleE:Girls’educationandGenderroles
Thesethemeswerereflectedacrosstheearlylearningteacherassessmentresponses.Collectively,theysuggestthatteachersdonotknowhowtodrivepupillearning,whichincludesteachersbeingunabletoascertainwhichactivitiesarebestsuitedtoimprovereading.Attimesthisincludes
EvaluationoftheGirl’sEducationProject3
©EDOREN 66
proposingactivitiestoincreasenumeracy,ratherthanreadingskills.Thefocusoneffortwithinthiscontextislikelyareflectionofthelowlevelsofcompetenceamongstteachers–ifteacherslackboththeknowledgeandskillstoimprovepupillearningtheyhavefewexplanationsforlowperformanceandfewavailableoptionstoimprovepupillearning,beyond‘tryingharder’.
Responsesstatingthatteachersshouldfocusongirlsintheclasswerenotsupportedbyjustificationsforsuchafocus,didnotreflecttheneedsofgirlsintheclassroomandinsomeinstanceswereunderpinnedwithresponsesregardingthelimitedcapacityorsocietalrolesofgirls.Thisindicatesthatmanyteacherslackeitherthewilland/ortheknowledgeandskillstoimprovegirls’learningintheclassroom.
Overall,thesefindingsstarklyhighlighttheverylowlevelsofteachers’knowledgeandskillsinmostofthedomainsthatarerelevanttoearlygradeteaching.Ofparticularsignificancearethefindingsthat60%ofteachersarethemselvesnotfullyproficientinGrade1and2levelHausa,andthatanoverwhelmingmajorityareunabletoassesspupils’levelsoflearning,whichiscriticaltoeffectiveteaching.ThishasafewimplicationsforGEP3.First,ithighlightssomeofthekeyissuesthattheearlylearninginterventionwouldneedtoaddressthroughanappropriatecombinationofteachertrainingandcarefullytailoredteachingandlearningmaterials.Second,itprovidesanindicationofthescaleofthechallengeconfrontingtheprogramme,whichcouldfeedintodesigndecisionsrelatedtothefrequencyoftrainingandthelevelatwhichitispitched.Finally,itsuggeststhatbaselinelevelsofteacherknowledgeandskillscouldunderminetheprogramme’simpact,andthatthismeritsregularmonitoringbytheprogrammeimplementationteam.
Whatdoteachersdointheirlessons?
Thepracticesofteachersintheclassroomwerecategorised,onthebasisoftheclassroomobservations,intothreegroups:teachertalk,teacheractionandpupilaction.
Analysisoftheclassroomobservationdataonteachertalkindicatesthatteachersmakesgreateruseofrote-basedapproachesthanofpupil-centredapproaches.Ofthedifferenttypesofteachertalkthatwereobserved,threeareconsideredtobepupil-centred:askingorrespondingtoanopenquestion,assistingingroupwork,andusingachild’snameinclass.AshighlightedinFigure18,onlyaminorityofteachersengagedinthesetypesofteachertalk.In10%ofclassroomsteacherswerenotpresentatsomestageduringtheongoingclassroomobservation.Thisdoesnotincludeteacherswhofinishedtheclassearly.
EvaluationoftheGirl’sEducationProject3
©EDOREN 67
Figure18: Percentageofobservedteachersengagingindifferentkindsofteachertalk
Teachers’actionsdidincludetheuseofsomepupil-centredapproaches.Inparticular,mostteachersmovedamongstpupils.However,veryfewusedatextbookorothermaterials.
Figure19: Percentageofobservedteachersengagingindifferenttypesofteacheraction
Pupilactioninclassroomsisalsoausefulindicatorofthepedogogicalapproachbeingundertakenbytheteacher.Highlevelsofnon-roteactivitiyareusuallyanindicationthatatleastsomepupil-centredteachingandlearningintakingplace.Thefollowingtypesofpupilactionfallintothiscategory:groupdiscussion,grouporworkinpairs,respondingtoanopenquestion,askingtheteacheraquestion,usingatextbook,readingaloud,anddoingindividualwork.Thisisincontrasttochanting,listening,andrespondingtoclosedquestions.AsshowninFigure20,theuseofrote-basedapproacheswas
100.0
68.1
19.5
45.3 42.6
60.8
10.00.0
20.0
40.0
60.0
80.0
100.0
Instructs Remindstheclasstopayattention
Assistsgroupdiscussion
Usesachild'sname
Asksorrespondstoanopenquestion
Asksorrespondstoaclosedquestion
Teacherisnotpresent
98.7
11.7
72.0
90.2
16.6 19.1
0
10
20
30
40
50
60
70
80
90
100
Writesorreadsfromtheblackboard
Givesdictation Demonstrateshowtodosomething(includingusingtheblackboard)
Movesamongthepupils
Usesatextbook Usesmaterials
EvaluationoftheGirl’sEducationProject3
©EDOREN 68
moreprevalentthantheuseofnon-roteactivities.Inparticular,groupdiscussion,groupworkandinstancesofchildrenaskingtheteacheraquestionwereobservedinaverysmallminorityofclassrooms.
Figure20: Percentageofclassroomswherepupilsengagedindifferenttypesofpupilaction
3.4.2.3 Teachers’knowledge,skillsandpracticesaspertheToC
ThemaincausalassumptionunderlyingtheToCfortheRANAinterventionisthatliteracylearningoutcomes,particularlyintheHausamothertongue,willimproveinearlygradesifteachingpracticesimprovethroughtheuseofimprovedteachingandlearningmaterialsandthepresenceofmoreknowledgeableandskilledteachers.AsrepresentedintheToCdiagraminSection4.1.3,thefirstcausalchainstatesthatthetrainingandmentoringofleadteachersandheadteacherswillleadtoanincreaseinteachers’knowledgeinliteracyandlanguageacquisitionintheearlygradesandanimprovementinteachers’instructionskills,includingtheuseofgender-sensitiveteachingpractices.Thiswill,inturn,leadtoanimprovementinHausa-basedliteracyteachingintheearlygrades.ThesecondcausalchainstatesthatthedistributionoftheRANALiteracyPackwillleadtoanincreaseduseofHausa-basedteachingandlearningmaterials,whichwillalsoimprovetheHausa-basedliteracyteachingintheearlygrades.Thissubsectionreportsontheintermediaryoutcomesforthesetwocausalchains.
Teacherknowledgeinliteracyandlanguageacquisitionintheearlygrades
AsdiscussedinSection3.2.9,teachersdrawonthreetypesofknowledgewithinclassroompractice.Subjectknowledgereferstoknowledgeoftheessentialquestionsofthesubject,thenetworksofconcepts,theoreticalframeworkandmethodsofinquiry.Pedagogicalknowledgereferstoknowledgeofthelearnersinthesetting,knowledgeofhowtoprovidetheconditionsthatenablepupilstounderstandcontent,andtheselectionoflearningandassessmentmaterials.Curriculumknowledgereferstoknowledgeaboutwhatshouldbetaughttoagroupofstudents,knowledgeofthenationalsyllabus,understandingoftheschoolandgrade-levelplanningdocumentsandknowledgeofthecontentofexaminations.
56.7
19.0 20.2
43.2
61.1
8.3 2.8
60.2
38.2
60.2
89.4
0102030405060708090
100
EvaluationoftheGirl’sEducationProject3
©EDOREN 69
Basedonthismodel,threecompositeindiceswerecreatedfromthedata–oneforeachoftheseknowledgetypes.Thesubjectknowledgeofteachersismeasuredthroughacompositeindexofteachers’writingskills,teachers’Hausaskills,teachers’comprehensionskillsandteachers’abilitytointerpretwordsandphrases.Thepedagogicalknowledgeofteachersismeasuredthroughacompositeindexofteachers’abilitytoidentifylowperformersandevidencejudgementsanddiagnoselearninggaps.Thecurriculumknowledgeofteachersismeasuredthroughthesyllabusknowledgesubscaledevelopedfromtheteacherknowledgeandskillsassessment.Theabsolutevaluesofteachers’scoresontheseindicesarenotinherentlyinformative,butratherprovidebaselinevaluestoassessimprovementsovertime.However,differencesinscoresbetweengroupsareofinterestatbaseline.Theshareofteacherswhoscoredzeroontheseindicesisalsoworthnoting.
Levelsofpedagogicalknowledgeatbaselinewereextremelylow,withover90%ofteachersscoringzeroontherelevantcompositeindex(seeFigure21).Thisindicatesthatthevastmajorityofteacherswereunabletodemonstrateproficiencyinknowledgeofthelearnersinthesetting,knowledgeofhowtoprovidetheconditionsthatenablepupilstounderstand,andintheselectionofappropriatelearningandteachingmaterials(pedagogicalknowledge).Therewereveryfewdifferencesinlevelsofpedagogicalknowledgeacrossdifferentgroupsofteachers.Teacherswithaprofessionaleducationqualificationdemonstratedslightlyhigherlevelsofpedagogicalknowledgethanthosewithoutone,althoughthedifferencewassmall.Therewerenoclearpatternsbyschooltypeorstate,andteacherswhohadpreviouslyattendedGEP3/UNICEFtrainingdemonstratedsimilarlevelsofknowledgetothosewhohadnot.
Levelsofsubjectknowledgewerealsolow,withsomevariationacrossgroupsofteachers.Overall,justoverone-fifthofteacherswereunabletodemonstrateanyproficiencyintheessentialquestions,conceptsandmethodsofenquiryinliteracyandlanguage(subjectknowledge).TeachersinpublicprimaryschoolsdemonstratedslightlybettersubjectknowledgethanthoseinIQSs.TeachersinKatsinadidslightlybetterthanthoseinZamfara,andteacherswithaprofessionaleducationqualificationperformedslightlybetterthanthosewithoutone.TherewerenocleardifferencesinthesubjectknowledgeofteachersbystateorbyattendanceatGEP/UNICEFtraining.
EvaluationoftheGirl’sEducationProject3
©EDOREN 70
Figure21: Percentageofteachersscoringzeroinpedagogical,curriculumandsubjectknowledge
Curriculumknowledgelevelswere,similarly,lowacrossallgroups,withalargeminorityofteachers(43%)unabletodemonstrateproficiencyinknowingwhatshouldbetaughttoagroupofstudents.Teacherswithaprofessionaleducationqualificationdemonstratedslightlyhigherlevelsofcurriculumknowledgethanthosewithoutone,andteachersinZamfarademonstratedslightlyhigherlevelsofcurriculumknowledgethanteachersinKatsina.Therewerenocleardifferencesbyschooltype.
Improvedinstructionskillsinactivelearningandincreasedtimeontask
AspertheRANAToCitisproposedthatanincreaseinteachers’knowledgeinliteracyandlanguageacquisitionintheearlygradesandanimprovementinteachers’instructionskills,includingtheuseofgender-sensitiveteachingpractices,willleadtoanimprovementinHausa-basedliteracyteachingintheearlygrades.
Beyondmeasuringtheproportionofgirlsandboyswithintheclassroomsetting,themeasurementofgendersensitivityisextremelycomplexandoftencannotmeetreliabilitycriteria.Forexample,withinthefacilitatorquestionnaireseveralitemsweredevelopedtomeasureattitudestowardsgirls.Acrossallitemsextremecomplianceeffectswereobserved,significantlycallingintoquestionthevalidityandreliabilityofthemeasure.Therefore,gender-sensitiveteachingcouldnotbemeasuredthroughclassroomobservations.Inlinewiththisconclusion,alternativeapproachestoassessinggender-sensitiveteachingwillbeexploredatmidline.Forthepurposesoftheevaluation,theRANAinterventions’impactongender-sensitiveteachingcanbeassesseddespitethelackofbaselinedataonthisvariableastheRCTwillallowustocomparemidlinedatabetweentreatmentandcontrolschools.BecauseofbudgetaryconsiderationsqualitativeresearchhasnotbeenincludedintheELevaluation.However,thedifficultyofcompilingarobustmeasureofgender-sensitiveteachingfrombasedonquantitativedatacollectionmethodshashighlightedtheneedtoconsiderincludingqualitativemethods.Thiswillrequirefurtherdiscussion.
0 10 20 30 40 50 60 70 80 90 100
Percentofteachersscoringzero
PercentofteachersscoringzeroPedagogicalknowledge 91.5Curriculumknowledge 43.4Subjectknowledge 22.2
EvaluationoftheGirl’sEducationProject3
©EDOREN 71
AcompositeindexwasdevelopedinordertomeasurechangesintheToCintermediaryoutcome–teachers’instructionskillsinearlygrades–frombaselinetomidlinetoendline.Scoresonthecompositeindexarebasedontheextentofpupil-centredlearningactivitiesobservedintheclassroom,observationsoftheteacherlinkingthelessontopreviouslearningandlearningobjectives,andtimeontaskinclass.
1. Themeasureoftheextentofpupil-centredlearningisbasedonwhetherteacherswereobservedengaginginthefollowingpractices:assistingingroupdiscussion,usingachild’sname,askingorrespondingtoanopenquestion,movingamongthepupils,usingavailablematerials,engagingingroupdiscussion,facilitatinggroupworkorworkinpairs,facilitatingopenpupilquestions,facilitatingpupilreadinginclassandrespondingtoopenquestions.
2. Observationsoftheteacherlinkingthelessontopreviouslearningandlearningobjectivesincorporatedobservationsoftheteachertalkingaboutthepreviouslesson,outliningtheobjectivesoftheobservedlesson,revisitingtheobjectivesofthelessonattheendofthesessionandsummarisingthelesson.
3. Timeontaskwasmeasuredbythepercentageoftimethepupilswereengagedduringlessonobservations.Theclassroomobservationtoolrecordedarangeofpupilbehavioursatthree-minuteintervals.Ifpupilswereengagedinanyactivityrelatedtolearning,thiswascategorisedastimeontask.
Overall,teachers’performanceonthecompositeindexwaslowacrossallgroups.Publicprimaryschoolteachersreceivedslightlyhigherscoresinoverallteacherpracticesandtherewerenocleardifferencesbyschooltypeorstate.Therewerealsonoobservablepatternsconnectingteacherpracticesandteachermotivationorperceivedteacherefficacy.Thisindicatesthatteachers’perceivedself-efficacyiseithernotrepresentativeoftheiractualcompetenciesorthelevelsofcompetencyinteacherpracticesaresolowthatdifferencescannotbedetected.ThisisaninterestingfindinginlightoftheToCassumptionthatteachermotivationinfluencesthetranslationofteacherknowledgeandskillsintoteachingpractices.Itwillbepossibletotestatendlinewhetherimprovementsinteachers’pedagogicalcompetencyandteachingpracticesduetotheearlylearninginterventionwillalsochangetherelationshipwiththeirmotivation.Endlinefindingsonpotentialchangesinthisrelationshipfromthebaselinewillthusbeofparticularinterest.
Overallscoresinpupil-centredteachingwerelow,withslightlyhigherlevelsofpupil-centredteachingobservedinKatsina,inpublicprimaryschoolsandforfemaleteachers.Oneobservationofinterestisthatwhileonaverageslightlyhigherlevelsofpupil-centredteachingwereobservedasteachers’experiencelevelsincreased,thelevelsofpupil-centredteachingdipslightlyataround10yearsofexperienceandthendivergesignificantly,withbothhigherandlowerlevelsofpupil-centredteachingoccurringafteraround20yearsofteachingexperience.
EvaluationoftheGirl’sEducationProject3
©EDOREN 72
Figure22: Meanpupil-centredteachingscorebyyearsofexperienceasateacher
Timeontaskwasmeasuredbythepercentageoftimethatpupilswereengagedduringlessonobservations.Thiswasclassifiedasthetotaltimeduringwhichpupilswereperforminganyactionrelatedtolearning(forthedifferenttypesofactionsthatwererecorded,seeFigure20).Theclassroomobservationtoolrecordedpupilactionsatthree-minuteintervalsduringthelesson.Thesedatawereusedtoestimatetimeontask.Onaverage,pupilswerefoundtobeengagedfor96%oflessontime.Thevastmajorityofclassrooms(roughly85%)werefoundtobeontaskforover90%ofthelesson(seeFigure23).However,thesefindingsshouldbeinterpretedcarefullyasitislikelythatthepresenceofobserversintheclassroomincreasedthepercentageofon-tasktimeinlessons.
Figure23: Percentageoftimeontask
0
5
10
15
20
25
0 5 10 15 20 25 30 35
MeanPu
pil-C
entred
TeachingSc
ore
Years'ExperienceasaTeacher
Mean
Linear(Mean)
.2 1.0 1.2 3.19.7
84.9
0
10
20
30
40
50
60
70
80
90
100
Percentageofclassrooms
Lessthan50 50-59.9 60 - 69.9 70-79.9 80-89.9 90-100
EvaluationoftheGirl’sEducationProject3
©EDOREN 73
UseofHausa-basedteachingandlearningmaterials
ThesecondcausalchainintheToCstatesthatthedistributionoftheRANALiteracyPackwillleadtoanincreaseduseofHausa-basedteachingandlearningmaterials,whichwillalsoimproveHausa-basedliteracyteachingintheearlygrades.However,aspertheprogrammeassumptionsoutlinedinSection3.1.3,increasesinHausa-basedteachingandlearningarebasedontheassumptionthatthematerialsarealignedwiththecompetencylevelsofteachers.Therefore,itisproposedthattheknowledgeofteachers,theinstructionalskillsofteachersandthepresenceofappropriateHausa-basedteachingandlearningmaterialswillcollectivelyincreaseHausa-basedteachingintheearlygrades.
InordertoexploretheavailabilityanduseofHausa-basedteachingandlearningmaterials,datawerecollectedduringlessonobservationsontheHausamaterialsavailableforboththesubjecttaughtinthelessonobservedandothersubjects,andtheuseofHausamaterialswithinthelesson.MaterialsinHausawereobservedbeingusedin2.4%oftheobservedlessons.MaterialsinHausawereavailableforthesubjectobserved,butwerenotusedinafurther1%ofobservedlessons.Inabsoluteterms,Hausaresourceswereavailablein72ofthe2,235lessonsobserved,andwereusedin53ofthose72lessons.Interestingly,almost18%ofteachersindicatedthattheyhavetheHausamaterialstheyneedtodotheirjobsintheteacherquestionnaire.Thisindicatesthatapossible14%ofteacherseitherdonotseeaneedforHausamaterialsorhaveaccesstomaterialsthatwerenotobservedorusedduringlessonobservations.
ThekeyindicationthatemergesfromthesebaselinefindingsisthataccesstoanduseofHausamaterialiscompletelyinadequate.ThedistributionofapackageofteachingandlearningmaterialsaspartofRANAisthereforehighlyrelevantintermsofprovidingaccesstosuchmaterials.InlightofthelowHausaliteracylevelsoftheteachersitwillbeimportantforthematerialstobewelladaptedtothelowteachercompetencylevels,andtoensurethatpeer/supervisorysupportisregularlyavailabletofacilitatetheiruse.
Hausa-basedteachinginearlygrades
Section0discussesteachers’knowledgeinliteracyandlanguageacquisitionintheearlygradesandteachers’instructionskillsaspertheToC.TheToCproposesthatincreasesinteachers’knowledgeandinstructionskillsandincreasesintheuseofHausa-basedteachingandlearningmaterialswillleadtoanimprovementinHausa-basedliteracyteachingintheearlygrades.
AcompositeindexwasdevelopedinordertomeasurechangesinHausa-basedteachingintheearlygrades.ThemeasuresincludedinthecompositeindexweredevisedusingthepercentageoftimetheteacherspokeHausainclassandteachers’Hausaskills.Typically,teachersinIQSsscoredhigheronthiscompositeindexthanteacherswithinprimaryschools,teachersinKatsinascoredhigherthanthoseinZamfara,andteacherswhodonotspeakEnglishandthosewhodonotspeakArabicscoredhigherthanteacherswhodospeaktheseotherlanguages.
InallclassroomstheuseofHausawasobservedonatleastoneoccasionandinapproximatelyhalfoftheclassesasecondlanguagewasusedinadditiontoHausa.
EvaluationoftheGirl’sEducationProject3
©EDOREN 74
Figure24: PercentageofclassroomsusingEnglish,HausaandArabic
ThefrequencyofHausausewasdocumentedduringclassroomobservationsandameasureofthepercentageoftimetheteacherspokeHausawascalculated.TherewerenolessonobservationswhereHausawasusedmorethan60%ofthetime.Inaboutthree-quartersofobservedlessons,Hausawasspokenbytheteacherbetween30%and60%ofthetime.
Figure25: FrequencyofHausauseinclass
TeacherswhohavebeenpreviouslytrainedinteachinginHausatypicallyspokeHausainclassforaslightlyhigherproportionofclasstime,asdidteachersinKatsina.MaleteachersalsospokeHausamorefrequently.Nodifferencewasobservedbetweenteacherswhodoanddonotholdaprofessionaleducationqualification.Teachersover50typicallyspokeHausamoreinobservedlessonsthantheiryoungercolleagues(upto10%more)andteachersinIQSstypicallyspokeHausa
0
10
20
30
40
50
60
70
80
90
100
English Hausa Arabic
IQS PS
.0
5.0
10.0
15.0
20.0
25.0
30.0
ShareoftimethatHausawasusedduringlesson
Shareoflesson
sobserved
Lessthan10
10-19.99
20-29.99
30-39.99
40-49.99
50-59.99
60andover
EvaluationoftheGirl’sEducationProject3
©EDOREN 75
forapproximately5%moreoftheobservedlesson.OtherlanguagesspokenbytheteacherinclassroomobservationsincludedArabicandEnglish.
3.4.2.4 Teachermotivationandattendance
Teachermotivation
Teachermotivationisanimportantaspecttoconsiderasitissupposedlyrelatedtoteachers’willingnessandabilitytoeffectivelyimpartknowledgetopupils.Thisisalsowhyourcompositeindexofmotivationisincludedinourregressionmodelthatlooksintostatisticallysignificantcorrelationsbetweenlearningandrelevantexplanatoryvariables.MotivationisthereforeseeninouranalysisasapotentialinfluencingfactorthatcanhelpexplainhowpupilsachievetherecordedlevelsofliteracyinHausaandEnglish.Duetosamplesizelimitations,andthealreadydiscussedskeweddistributionofteacherknowledgeandpedagogyscales,itisnotfeasibletoundertakethesametypeofanalysisoftheassociationandcorrelationbetweenteachermotivationandpupilknowledge.However,theinclusionofbothsetsofinformationinourregressionmodelallowsustocontrolforanydetectedsimultaneouseffectsofknowledgeandmotivationonlearning.
Teachermotivationisinvestigatedthroughtheuseofarangeofmotivationscaledscores,whichformthebasisfortheconstructionofacompositemotivationindex.Thesubscalescoverthefollowingareas:
• perceivedteacherefficacy;• interestin,andenjoymentof,teaching;• effortputinto,andperceivedimportanceof,teaching;• pressureandwork-relatedtension;and• teacher-to-teacherinteraction.
Theanalysisoftheindicationsemergingfromacomparisonofthedifferentmotivationsubscalesshowsthat,onaverage,thesurveyedteachers(wehave474observationswithdataavailableforthemotivationanalysis)scorehighestoneffortandlowestonperceivedteacherefficacy.Atthesametime,interestinandenjoymentofteachingisobservedtoberelativelyhighwhencomparedtopressureandwork-relatedtension.Thisseemstosuggestthatoverallteachersconsidertheirroletobeimportantandenjoyworkingasteachers,whilstbeingconsciousofthefacttheyarelimitedintheirabilitytocontributetopupils’learning.Itisalsointerestingtonotethattheteacher-to-teacherinteractionscoreseemstoindicatearelativelyhighlevelofcollaborationamongstteachers,whichcouldhelpthedevelopmentofspillovereffectswithinschoolsofanyteacher-specificintervention,includingteachertraining.
Thechartbelowpresentsavisualanalysisofthedifferencesacrossmotivationsubscales.
EvaluationoftheGirl’sEducationProject3
©EDOREN 76
Figure26: Comparisonofmotivationsubscales
Theoverallteachermotivationindexwasconstructedasacompositemeasureofallteachermotivationsubscales,withequalweightingforallsubscales.Whilsttheaveragemotivationindexvalueacrossthewholeteachersampleisnotparticularlyinformative,itisanalyticallyinterestingtolookatthedifferenceinmotivationscoresacrosscategoriesofinterestforouranalysis.Whenitcomestogender,itappearsthatfemaleteachersareslightlymoremotivatedthantheirmalecounterparts,althoughoursub-sampleoffemaleteachersisquitesmall,asalreadydiscussed.Intermsofage,thereseemstobelittledifferenceinmotivationacrossagegroups,althougholderteachers(i.e.over50)tendtobeslightlylessmotivatedthanyoungerteachers.
Figure27: Motivationbyagegroups
1.99
3.57
3.73
2.19
3.47
0 0.5 1 1.5 2 2.5 3 3.5 4
Perceivedteacherefficacy
Interestandenjoymentofteaching
Effortandimportance
Pressureandworkrelatedtension
Teacher-to-teacherinteraction
2.91
2.92
2.93
2.94
2.95
2.96
2.97
2.98
2.99
3
3.01
TeachermotivationcompositeindexUnder25 25to34 35to49 Over50
EvaluationoftheGirl’sEducationProject3
©EDOREN 77
Generallyspeaking,themotivationindexshowsarelativelyhomogeneoussituationacrosssomeofthekeycategoriesofinterestfortheanalysis.Urbanandruralmotivationisverysimilar,withteachersinurbanschoolsonlyslightlymoremotivatedthantheirruralcounterparts.NodifferencesareobservedbetweenteachersinKatsinaandZamfaraorteachersinIQSsorpublicprimaryschools.
Ascouldprobablybeexpected,receivingasalaryhelpsmotivation,thoughagainthedifferencewithteacherswhodonotreceiveanypaymentisverysmallandthestandarderroroftheestimatesdonotallowforanystatisticallysignificantinterpretationofdivergingpatterns.Evenwhenfurtherinvestigatingthetypeofpaymentnotrendinregardtointerestseemstoemerge,asteacherswhoreceivetheirsalaryalwaysinfullandontimearefoundtohaveonlymarginallyhighermotivationthantherestoftheteachers.Similarly,havingaqualificationorahigheracademicdegreeisassociatedwithonlyslightlyhigherestimatesofoverallmotivation,butagainwithinastatisticalmarginoferror.
Interestingly,havingattendedtrainingdoesnotseemtohaveanydiscernibleeffectonteachers’motivation.Asalreadydiscussedabove,oneoftheintermediaryoutcomeassumptionsoftheearlylearninginterventionisthatateacherwouldbemotivatedtoimprovetheirknowledgeandskills,thusattendingandmakingthemostoutofthetrainingopportunities.However,thelackofadescriptiveassociationbetweenprevioustrainingandmotivationseemstosuggestthatthiscannotbetakenforgranted.Theendlinewilloffertheopportunitytoassesswhetherastrongercorrelationbetweentrainingandmotivationwillemergeoncetheearlylearninginterventionhasbeenimplemented.
Teacherattendance
Ourbaselinedataonabsenteeismshowsthat60%ofteachersindicatedhavingbeenabsentatleastonceinthelastthreemonths.Thegreatmajorityofteachersidentifytheirownillnessortheillnessofafamilymemberasthemainreasonfortheirabsence,withsocialorreligiousobligationsandweatherconditionsalsoemergingasdeterminingfactors.Theestimatedaveragenumberofdaystheywereabsentinthesameperiodofthreemonthsisjustunderfivedays.Gender-specificanalysisshowsthatmaleteacherswereabsentforslightlylongerthanfemaleteachers,with4.9daysofabsencecomparedto4.6days94,withthesesmalldifferencespointingtowardsanegligiblegenderinequality.Itisinterestingtonotethatwhilstyoungerteachersweremorelikelytobeabsentatleastonceduringthepreviousthreemonths,olderteachers(over50yearsofage)wereabsentforlonger,justover6.2days.
InZamfarateacherswerebothmorelikelytobeabsent(67%comparedto57%)andtendedtobeabsentforlonger(5.8dayscomparedto3.5)thaninKatsina.Atthesametime,absenteeismisfoundtobemoreprevalentinIQSsthaninprimaryschools;inthiscase,thedifferenceintheaveragenumberofdaysateacherwasabsentforisparticularlynoticeable,withIQSteachersabsentforapproximatelyeightdaysonaverage,comparedtojustoverthreedaysinprimaryschools.However,attendancedataonIQSsneedtointerpretedwithcautionastheIQSteachingscheduleisnotasstructuredasinprimaryschools,whichleadstoamoreflexibleteacherattendanceinIQSsandcouldpartiallyexplainthedifferenceintheabsenteeismrates.Giventhefactthatteachersattendingschoolandclassesisindeedanassumptionunderlyingthesuccessfulimplementationoftheearlylearninginterventionoutputs,thedetecteddifferencebetweenpublicprimaryschoolsandIQSsneedstobetakenintoconsideration.Inparticular,itseemsadvisablefortheimplementerstodevise94Thesefiguresaccountforthefactthatpartoftheteachersamplehasbeenabsentfor‘zerodays’inthelastthreemonths.
EvaluationoftheGirl’sEducationProject3
©EDOREN 78
trainingschedulesthathelptheprogrammetoreachthelargestpossibleaudienceofteachersinIQSs.Atthesametime,itwillbeimportanttoincentivisetrainedteachersandleadteacherstoregularlyattendtheirclassessoastoensurethattheirimprovedknowledgeandteachingpracticesarepassedontopupilsinschool.
Notsurprisingly,whenlookingatpaymentstatus,teacherswhodonotreceivesomeformofsalaryorstipendaremorelikelytobeabsentandtobeabsentforlongerperiods.WhilstthelikelihoodofbeingabsentishigherforteacherswhodonotgetpaidinbothIQSsandpublicprimaryschools,thelongerlengthofabsenteeismforunpaidteachersseemstobedrivenbypublicprimaryschoolteachersasinIQSsbothpaidandunpaidteachersarefoundtobeabsentforsimilarlylongperiods,asdiscussedabove.Absenteeismisalsomoremarkedamongstteacherswithnoprofessionalqualification.However,whenlookingattheassociationbetweenattendingatrainingcourseandteacherattendance,theredoesnotseemtobeanydiscernibletrend.Theproportionofabsentteachersandthelengthoftheiraverageabsenteeismareverysimilarbetweenthetwocategories(whicharealmostequallysplitinourearlylearningsample).
3.4.3 Pupilsinearlygrades
Thisthirdsectiondescribesthepupilswithinthesettingbeingdiscussedandsummarisestheircurrentlevelsoflearning.
3.4.3.1 Pupils’backgroundcharacteristics
Theearlylearningsampleofpupilsthatformsthebasisforourevaluationisequallydividedbetweenmaleandfemalepupils,withtheformeronlyslightlymoreprevalent.Theaveragepupilageisjustovernineyears,withsurveyedIQSpupilsolderonaveragethanpublicprimaryschoolpupils.However,alargenumberofobservationsinourearlylearningsampledidnotreportage:around35%ofoursampleofpupilsdidnotprovideinformationonage.Theagegroupcategorisationreportedinthegraphbelowshowsthatthekeydifferencebetweenthetwotypesofschoolinregardtopupilagedistributioniscentredaroundthelowerandupperendoftheagedistribution.Amongthosewhoreportedage,nopupilsinpublicprimaryschoolswerereportedtobeovertheageof16,comparedtosome5%ofpupilsinIQSswhowereaged16ormore;atthesametime,alargerproportionofveryyoung(belowsevenyearsofage)pupilsisfoundinpublicprimaryschoolswhencomparedtoIQSs.
Thechartbelowshowsthisdifferentdistributionofpupilagegroupsbyschooltype,whendiscountingtheproportionofpupilsinoursampleforwhomageinformationwasnotpresent.
EvaluationoftheGirl’sEducationProject3
©EDOREN 79
Figure28: DistributionofagegroupsinIQSsandpublicprimaryschools
WhilstallpupilswerefoundtospeakHausaathome,practicallynochildspeaksEnglishoranyotherlocallanguageathome.Around98%ofoursamplewasattendingP2oranequivalentlevelatthetimeofthesurvey,whichisinlinewiththesamplingstrategyadoptedfortheearlylearningevaluation.Furtherdetailsonpupilschoolattendancecannotbeinvestigatedrobustlyduetoalargenumberofmissingvaluesandissuesrelatedtohowinformationwasgatheredonthisvariable.95
ThedivisionofthesamplebetweenIQSsandpublicprimaryschoolsisunbalancedtowardsthelatter,with60%attendingpublicprimaryschoolsand40%attendingIQSs.Whilstover50%ofthesampledchildrenwerefoundtobecurrentlyattendinganotherschoolotherthantheoneinwhichtheyweresurveyed,anadditional11%ofpupilswerefoundtoalsohavebeenattendingotherschoolsinthepastwheremaths,readingandwritingweretaught.Onapositivenote,only3%ofpupilsreportedfacingsomedifficultyingettingtoschool.
Whilsttherural/urbandifferentiationshowsaclearimbalance,withover80%ofpupilsattendingschoolsinruralareas,thepupilsampleisalmostequallydividedbetweenthestatesofKatsinaandZamfara,withonlyafewmorepupilsattendingschoolsinZamfara.Adescriptiveanalysisofpupils’socioeconomicconditionsindicatesthatmostpupilsareinthemiddlerangeofthepovertyspectrum.ThelatterisinvestigatedthroughtheuseofaHouseholdWealthIndex(HWI)(assetindex)thatweconstructedfortheearlylearningsample,asdescribedinAnnexJ.Inparticular,whenlookingatthewealthtertiledifferentiation,theaverageacrossthesampleissetaroundthesecondtertile.
Mostpupilbackgroundcharacteristicsareconsistentacrossourmainanalyticalcategoriesofinterest.Thereare,forinstance,nodetectablegenderdifferencesacrossthemainpupilcharacteristics,withboysonlyalittlebitolder(justovernineyearsofage)onaveragethangirls(just
95Ontheonehand,thepresenceofpupilsduringthesurveywascalculatedbyreferencetoP2pupilsandwasbasedonmanualcountingduringlessonobservations.Ontheotherhand,thenumberofpupilsnormallyintheclassesissolelybasedonteacherreporting.Inaddition,thereareveryfewschoolsforwhichbothattendanceandenrolmentrecordswereavailable.
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
IQSsPublicschools
Below7 7-10years 11-15years Above16
EvaluationoftheGirl’sEducationProject3
©EDOREN 80
undernineyears)inthesampleofpupilsthatreportedageinformation.However,amarkeddifferenceisobservedbetweenthesocioeconomicstatusofthesub-samplesofpupilsbelongingtothetwostatesandthetwotypesofschoolwhichformpartoftheearlylearningsample.Specifically,pupilslivinginKatsinaandattendingIQSswerefoundtobepoorerthanthoselivinginZamfaraandattendingprimaryschools.ThegraphbelowshowsthatthelesspoorconditiononaverageofpupilsinZamfaraisdrivenbyadisproportionatelylargerpercentageofpupilsbelongingtothehighest(third)tertileoftheHWI.Theimportanceofpupilwealthstatuswillemergeasanimportantfactoraccountingfortheirlearningperformanceinourregressionanalysis.
Figure29: HWItertilecategorisationbystate
3.4.3.2 Pupillearningoutcomes
Learningoutcomesaretheresultofacomplexandmultivariateschoolsystem,andofteacherandstudentfactors,whichareinfluencedbysocialandeconomicconsiderations.Thewaysinwhichthesefactorsinteractarecomplexandcanneverbefullymappedoraccountedforinananalysis.
Significantly,oneofthemostimportantpredictorsoflearningoutcomesiswhatthepupilsbringwiththemintotheclassroom.Innateability,socialandculturalfactors,aswellashousehold-specificfactors,significantlypredictpupilperformanceinmoststandardisedassessments(Hartas2011,OuthredandBeavis2012,OuthredandBeavis2013,Mayer,1997,DahlandLochner2005,VanderBer2015).
Learningoutcomemeasurement
TheHausaliteracyassessmentisdesignedtotestthesameliteracyknowledgeandskillsastheEnglishliteracyassessment.Itemsarenotmerelytranslated,butratherparallelitemsaredevelopedtotestsimilarconceptswhenappliedtotheHausalanguage.TheEnglishliteracyassessmentcontains13items,witheachitembeingmadeupofseveralsub-items.
Theassessmenttestsarangeofliteracyknowledgeandskillsacrossthepre-literacy,emergingandbasicliteracyranges.Knowledgeandskillsincludeletterrecognition,phonologicalknowledge,
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Katsina Zamfara
Lowtertile Mediumtertile Hightertile
EvaluationoftheGirl’sEducationProject3
©EDOREN 81
printconcepts,oralliteracy,verbalcomprehension,initialsoundsandletters,readinghighfrequencywords,verbalandwrittengrammar,writinghighfrequencywords,readingfluency,copyingandspellinghighfrequencywords.
EnglishliteracyandHausaliteracyassessmentswereconstructedfollowingfivesteps,includingclarifyingconstructs,testtargeting,administration,psychometricanalysis,drawingbenchmarksandsecondarydataanalysis.
InthecaseofnorthernNigeria,pupillearninglevelshavebeenfoundtobelowinpreviousstudies,includingpreviousGEP3learningassessments,ESSPINCSs,andtheTDPbaselinesurvey.Itisimportanttoseektounderstandtheextenttowhichlevelsoflearningarepredictedbypupil-,teacher-andschool-levelfactorsinordertoidentifypolicy-andprogramme-levelinputstoimprovelearningoutcomes.
Figure30: Categoriesoffactorsthatinfluencelearningoutcomes
Figure30summarisesthefactorsthatarethoughttoaffectpupils’learningoutcomes.ThissectionofthereportdescribespupillearningoutcomesanddescribesthelearningachievementofGrade2pupilswithinthesampledpublicprimaryschoolsandIQSsinKatsinaandZamfara.ThefollowingsectionsdescribethelearninglevelsofpupilsinHausaandEnglishliteracyandinvestigatethefactorsassociatedwithpupillearninglevels.Theanalysisaimstoprovideaclearerpictureoftheextenttowhichtheobservedlearningoutcomescanbeexplainedbyvariousfactors(e.g.pupils’background,teachermotivationandschoolresources)andtheextenttowhichcontributingfactorsareamenabletopolicyandprogrammeintervention.
Approachtoanalysingdataonlearningoutcomes
TheanalysisofdataonlearningoutcomespresentedinthisreportuseestimatesofpupilachievementbasedonRaschmodelling.Thisapproachallowsforvalidcomparisonstobedrawnacrosslearningassessmentsadministeredtodifferentgradesandoverdifferentyears.Assuch,ascaledscoreisthemathematicaltransformationofapupil’srawscoresinordertoreportherscoreonacontinuumconsistentlyovertheyears,andacrossdifferentversionsoftheassessment(BangladeshDirectorateofPrimaryEducation,2013;Outhred,2015).Raschanalysisalsoallowsfortestdifficultyandpupilabilitytobereportedindependentlyonthesamescale.Inaddition,ascaledscoreof,say,500willmeanthesameatendlinein2017asitdidatbaseline
EvaluationoftheGirl’sEducationProject3
©EDOREN 82
in2014.
Rawscoresarefrequentlyusedtocalculatepupillearningperformancebyanalysingthepercentageoftestitemsthatthepupilansweredcorrectlyorthepercentageofpupilsansweringoneormoreitemscorrectly.Althoughthisisarguablytheeasiestwayofgradingtests,fromastatisticalpointofviewrawscoreshaveseveraldrawbacks(Outhred,2015).First,whencalculatingapercentagescore,eachassessmentitemisgivenanequalweight,whichimplicitlyassumesthateveryitemisequallyindicativeofthelevelofknowledgeorskillsthatthepupilpossesses.Inreality,thisislikelynottobethecase.Theuseofrawscoresalsolimitstheextenttowhichassessmentresultscanbecomparedovertimeandacrosslocations.Finally,rawscoresconfoundthedifficultyleveloftheassessmentwiththeabilityorknowledgelevelofthepupils.Itisnotpossibletoseparateoutthesetwocomponents,whichjointlydeterminethepercentageofquestionsastudentanswerscorrectly.Bearinginmindtheserelativemeritsofscaledversusrawscores,scaledscoreshavebeenusedfortheanalysisofdataonlearningoutcomesinthisreport.
Hausaliteracylearningoutcomes
ThefindingsoftheHausaliteracyassessmentindicatethatthemajorityofpupilshaveyettoacquiretheknowledgeandskillsinHausaliteracyexpectedbytheNigeriancurriculum.ItisexpectedthatpupilswhoarebeginningP2willhavemasteredtheP1curriculumandwillbereadytoreceivetheP2curriculum.However,theresultsindicatethatonly5.3%ofpupilsareperformingwithintheexpectedrange.Afurther2.6%ofpupilsareabletodemonstrateemergingliteracyskills,while92%ofpupilsareunabletodemonstrateliteracyskillsinHausabeyondpre-literacyskills.
Table14: Hausaproficiencyleveldescriptions
Proficiencyrange Descriptionoftheknowledgeandskillsofpupilsachievingwithinthisrange
Pre-literacyHausaPupilswhoachievedwithinthepre-literacyrangewereabletodemonstratesomeofthefollowingskills:knowledgeofprintconcepts,identifytheinitialletterinhis/hernameandwritetheinitialletterinhis/hername.
EmergingHausaliteracy
Inadditiontotheskillsabove,pupilsachievingwithinthisrangewereabletodemonstrateatleastsomeoftheknowledgeandskillswithintherangeexpectedbytheP1curriculum.Pupilsachievingwithinthisrangewereableto:soundoutlettersounds,spellsomehighfrequencywords,andreadashortpassagewithlimitedaccuracy.
BasicHausaliteracy
Inadditiontotheskillsabove,pupilsachievingwithinthisrangewereabletodemonstrateatleastsomeoftheknowledgeandskillswithintherangeexpectedbytheP2curriculum.Pupilsachievingwithinthisrangewereableto:identifysimilarsounds,readhighfrequencywords,spellhighfrequencywordswithaccuracy,copyasentence,soundoutlettersounds,andreadashortpassage.
Figure31representsthedistributionofperformanceinHausaliteracy.TheyaxisrepresentstheHausascalescorederivedfromthepsychometricanalysisofpupilperformanceintheassessment(formoreinformationseeSection3.2.9.1).Ascanbeseeninthefigure,thepeakofthedistributionfallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy,indicatingthat
EvaluationoftheGirl’sEducationProject3
©EDOREN 83
substantialeffortwouldberequiredtoachievelargegainsinthepercentageofpupilsmovingfrompre-literacytoemergingliteracy.
Figure31: DistributionofHausaliteracyproficiency
Englishliteracylearningoutcomes
ThepurposeoftheEnglishliteracyassessmentwithintheevaluationoftheGEP3earlylearninginterventionistoestablishabaselineofEnglishproficiencyinordertoexploretherelationshipbetweenimprovedmother-tongueinstructionandtransitionintoLanguage2(L2)inthelateryearsofprimaryschool.EnglishliteracyscoreswerelinkedtotheESSPINEnglishmetricthroughlinkitems.ThisenablesthedirectcomparisonofresultswiththeESSPINCSfindingsinsixNigerianstates.
TheresultsoftheEnglishliteracyassessmentindicatethatthevastmajorityofpupilshaveyettoacquireeitheremergingorbasicEnglishliteracyskills.Under1%ofpupilsdemonstratedatleastsomeoftheskillsthatfallwithinthebasicEnglishproficiencyrangeand3.3%ofpupilsdemonstratedatleastsomeoftheskillsthatfallwithintheemergingliteracyrange.However,96%ofpupilscouldnotdemonstrateliteracyskillsinEnglishbeyondpre-literacyskills.TheassessmentfindingsalsoindicatethatpupilproficiencyinHausaaftermorethanayearofschoolingisnotsignificantlyhigherthanpupilproficiencyinEnglish.
0 1000 2000 3000 4000 5000 6000 7000
263.23
344.18
391.83
432.47
468.25
499.18
526.67
551.65
574.30
594.73
612.97
629.03
643.14
655.91
668.05
680.30
693.62
709.37
730.13
759.89
799.60
848.81
934.78
2.6%ofpupilsdemonstratedknowledgeandskillswithintherangeexpectedbytheP1curriculum.
Thislevelofproficiencycanbedescribedasemergingliteracy.
Over92%ofpupilsdemonstratedpre-literacyHausaskills.Childrenachievingwithinthepre-literacyproficiency
rangedonotyethaveemergingliteracyskills.
5.3%ofthepupilsdemonstratedGrade
2Hausaliteracyskills.Thislevelofproficiencycanbedescribedasbasicliteracy.
EvaluationoftheGirl’sEducationProject3
©EDOREN 84
Table15describestheknowledgeandskillsofpupilsachievingwithineachoftheproficiencyranges.TheseproficiencyrangesarealsodirectlycomparabletotheESSPINCSproficiencyranges.
Table15: Englishproficiencyleveldescriptions
Proficiencyrange Descriptionoftheknowledgeandskillsofpupilsachievingwithinthisrange
Pre-literacyEnglish
Pupilswhoachievedwithinthepre-literacyrangewereabletodemonstratesomeofthefollowingskills:knowledgeofprintconcepts,understandingandrespondingverballywithagrammaticallycorrectsentencetoasimplequestionabouttheirage,understandingandrespondingverballywithagrammaticallycorrectsentencetoasimplequestionabouttheirname,andsayingtheinitiallettersoffamiliarobjectandanimalnamesandwords.
EmergingEnglishliteracy
Inadditiontotheskillsabove,pupilsachievingwithinthisrangewereabletodemonstrateatleastsomeoftheknowledgeandskillswithintherangeexpectedbytheP1curriculum.Pupilsachievingwithinthisrangewereableto:verballycomposeashortgrammaticallycorrectsentenceinthecontinuouspresenttenseinresponsetoaquestionaboutapicture,listentoashortpassageandrememberspecificdetailstorespondverballytoaquestion,andtocopywordsthatwereclearlyshapedandcorrectlyorientated,withanunderstandingofspaceandfullstops.
BasicEnglishliteracy
Inadditiontotheskillsabove,pupilsachievingwithinthisrangewereabletodemonstrateatleastsomeoftheknowledgeandskillswithintherangeexpectedbytheP2curriculum.Pupilsachievingwithinthisrangewereableto:usephonicknowledgetosaytheinitialsoundsoffamiliaranimalnames;listentoashortpassageandrememberspecificdetailssoastorespondverballytoaquestion(one-wordanswerswereacceptable;useknowledgeofcommoninflectionsinspellings;displayknowledgeofplurals;writetheanswertoaquestion;usephonicknowledge(andawareness)toreadupperandlowercaseletters;spellsimplehighfrequencywordsaccurately;readhighfrequencywordsandphonicallydecodabletwo-syllableandthree-syllablewordsthatincludecommondiagraphsandadjacentconsonants(e.g.,black)insimplesentences;understandandrespondinwritingwithagrammaticallycorrectsentencetoasimplequestionaboutthepositionofaneverydayitem;listentotwosentencesandrespondverballytoaquestionwithagrammaticallyaccuratesentence;independentlyreadformeaningashorttextwitharangeofsentencestructures,highfrequencywords,andtwo-syllableandthree-syllablewordsthatincludecommondiagraphsandadjacentconsonants;verballycomposeashortgrammaticallycorrectsentenceinthecontinuouspresenttenseinresponsetoaquestionaboutapicture;copywordsthatareclearlyshapedandcorrectlyorientated,withanunderstandingofspaceandfullstops;useappropriateintonationwhenreadingtextswitharangeofsentencestructures,highfrequencywords,andtwo-syllableandthree-syllablewordsthatincludecommondiagraphsandadjacentconsonants;useknowledgeofcommoninflectionsinspellings,includingplurals,towritetheanswertoaquestion;readarangeofsimplesentenceswithhighfrequencywords,phonicallydecodabletwo-syllableandthree-syllablewordsthatincludecommondiagraphsandadjacentconsonants(e.g.,black)independently;rememberspecificdetailsfromashortsimpletextreadtorespondverballytoaquestion;andreadasimplesentenceformeaningandcompleteamissingwordusingthecorrectspelling.
EvaluationoftheGirl’sEducationProject3
©EDOREN 85
Figure32depictsthedistributionofperformanceacrosstheEnglishliteracyscale.TheyaxisrepresentstheEnglishscalescorederivedfromthepsychometricanalysisofpupilperformanceintheassessment(formoreinformationseeSection3.2.9.1).Ascanbeseen,thecuspofthedistributionfallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy,suggestingthatsignificanteffortwouldbeneededtoattainlargeadvancesintheshareofpupilsachievingemergingliteracy.
Figure32: DistributionofEnglishliteracyproficiency
3.4.3.3 Discussion
AnanalysisofpupilresponsestoHausaliteracyitemsandthedifficultyofvariousitemsprovidesaninsightintothelearningtrajectoriesofpupilswithinthecontextinwhichRANAoperates.TheanalysisfindsthatitemsthatrequireknowledgeofphonicsrankasthemostdifficultitemsinbothHausaandEnglish.Correctlysoundingoutlettersandidentifyingsimilarsoundsweremoredifficultforpupilsthanwritingorreadingfullpassages.Thisindicatesthatcurrentlypupilswithinthecontextwhohaveachievedbasicliteracyaredoingsowithoutsignificantexposuretophonicsknowledge.
0 2000 4000 6000 8000 10000 12000
202.77
203.56
305.91
337.10
338.50
359.65
361.28
382.01
407.18
411.74
436.34
453.62
460.94
482.66
500.60
513.99
525.58
534.65
551.84
575.49
599.06
624.90
Over96%ofpupilsdemonstratedpre-literacyEnglishskills.Childrenachievingwithinthepre-literacyproficiency
rangedonotyethaveemergingliteracyskills.
.5%ofthepupilsdemonstratedgradetwoEnglishLiteracyskills.Thislevelof
proficiencycanbedescribedasbasicliteracy..
3.3%ofpupilsdemonstratedknowledgeandskillswithintherangeexpectedbytheP1curriculum.
Thislevelofproficiencycanbedescribedasemergingliteracy.
EvaluationoftheGirl’sEducationProject3
©EDOREN 86
Thereisevidencethatinmanysub-SaharanAfricancontexts,the‘look-and-say’methodisusedmoreregularlythanthephonicsapproach.Adu-Yeboah’sstudyfoundthatteachersandteachereducators‘tendedtogravitatetowardsthe“look-and-say”approach’andthatpedagogywasprimarilytransition-based(p.29).Thestudyconcludedthatwhilethecurriculumoftenmakestheassumptionthatteachersareskilledintheuseofbothlook-and-sayandphonicsinstruction,eventeachereducatorsthemselvesarenotequippedwiththeknowledgeandskillsrequiredtoteachphonics(Adu-Yeboah,p.70).
Asystematicreviewoftheresearchliteratureontheuseofphonicsfoundthatsystematicphonicsteachingisassociatedwithbetterprogressinreadingaccuracy.However,therewasweakandnosignificanteffectforreadingcomprehension(Torgeson,BrooksandHall,2006).Ifreadingistobeunderstoodasbeingabletounderstandandinteractwithwrittentext,readingfluencyisnotanadequateindicatorofreadingandcomprehensionasanindicatorofliteracyachievementemergesasbeingofimportance.Whiletheuseofphonicsinstructionhasadvantages,ifweunderstandliteracyasbeingabletounderstandandinteractwithwrittentext,readingfluencyisabridgetoliteracy,ratherthanameasureofliteracyitself.
Inaddition,itisimportanttokeepinmindthattheresultsofstudiesinvestigatingthelinkbetweenphonologicalawarenessandreadingaccuracyandcomprehensionprimarilycomefromwithineducationsystemsthatpossesshighlyliterateandqualifiedteachers.Astheresultsoftheteacherassessmentsdiscussedabovestarklyhighlight,theseconditionsdonotholdinthecontextinwhichGEP3isbeingimplemented.Thisraisesquestionsabouttheadditionalresourcesofsupportthatwouldneedtobeprovidedinorderforaphonics-basedapproachtobeeffectivelyappliedinthiscontext.
3.4.3.4 Factorsassociatedwithimprovedlearning
WhenlookingatthemeanHausaandEnglishliteracyscaledscoreacrossthewholesampleofpupils,itispossibletonotesomedescriptivetrendsthatareworthmentioningatthisbaselinestageoftheevaluation,soastoassesswhatfactorsareassociatedwithlearningperformanceandhowthiswillchangeovertime.Interestingly,HausaandEnglishshowaverysimilarpatternacrossmostcategoriesofinterest,thoughwithdifferentscalescoremagnitude,thusimplyingacommonliteracytrendinthetwosubjects.
Themaindescriptiveindicationsofinterestarethefollowing:
• Malestudentsarefoundtoachievehigheraveragescoresthanfemalepupils.Interestinglythough,whenlookingatthedifferencebetweenpupilsinIQSsandpublicprimaryschools,itisnotedthatwhilstmalepupilsperformbetterthanfemalepupilsinIQSs,thetrendisnotmaintainedinprimaryschools,withthetwogendersperformingatpracticallythesamelevel(slightlybetterlearningoutcomesforfemalepupils),asshowninthegraphbelow.
EvaluationoftheGirl’sEducationProject3
©EDOREN 87
Figure33: Hausaliteracybygenderandschooltype
• Bywayofapartialexplanationofthegendertrendobservedabove,itisimportanttohighlightagainthatpupilsincludedinoursampleofIQSsareconsiderablyolder(almostfouryearsofdifferenceonaverage)thanpupilsinthesampledpublicprimaryschools.Althoughageisaffectedbythemissingobservationissuediscussedabove,thenumberofpupilsforwhichagedataaremissingissimilarforbothIQSsandprimaryschools.Amongstpupilsforwhichwehaveageinformation(thegreatmajorityofourearlylearningsample),olderchildrenperformbetterthanyoungerones,withchildrenover10achievingconsiderablyhigherscoresthanyoungerpupils.
• PupilsinKatsinaperformslightlybetterthanthoseinZamfara,butthisdifferenceisverysmallandnotstatisticallysignificant,withbothstatesshowingscoresaroundthe500pointmarkforHausaandthe350pointmarkforEnglish.
• Asshowninthegraphbelow,olderchildrenperformbetterthanyoungerones,withthoseover16achievingconsiderablyhigherscoresthanpupilsaged11to15andsevento10.Pupilsundertheageofsevenarefoundtoperformworsethantherest.Consideringthatnopupilisagedover16inoursampledpublicprimaryschoolsandthatthepupilsover16intheIQSsscoreonaverage687intheHausascaledscore,thekeyroleofageisundeniable.Atrendemergeswhenreviewingtheachievementofpupilsagainstageandageandgender.AscanbeseeninFigure34,typicallythosepupilswhoaremovingbeyondpre-literacyskillsareintheirlateteens.Thisindicatesthatachievementispossiblyassociatedwithcognitivedevelopmentmoresothanyearsinschool,asthedataindicatetheaverageyearsofschoolingisnotsubstantiallydifferentbetweentheolderandyoungerchildreninthesample.
0 100 200 300 400 500 600 700
Hausaliteracyscalescoreinpublicschools
HausaliteracyscalescoreinIQschools
Female Male
EvaluationoftheGirl’sEducationProject3
©EDOREN 88
Figure34: Hausameanscalescorebyage(95%confidenceinterval)
• WhendisaggregatingliteracyachievementbyageandgenderatrendemergesacrossbothHausa
andEnglish.Genderdifferencesinperformancearesmallintheyoungeryears,howeveroncegirlsreachthepointinthelifecycleassociatedwithpuberty(around12yearsofage)gendergapsinperformanceincrease.Withinthesample,girlsovertheageof16performedmorepoorlythanboththeiryoungercounterpartsandtheirmalepeers.
Figure35: MeanHausascalescorebyageandgender(95%confidenceinterval)
• Asdiscussedabove,pupilsinIQSsachievedahigherscoreonaveragethanthoseinpublicprimaryschools,whichmaybedrivenbythedifferenceinagesbetweenthetwopupilgroups.Itisworthreportingthisdifference,givenitsmagnitude,aswellasthesignificanceofitscorrelationwithlearninginourregressionanalysis,asdiscussedbelow.
450.0000
500.0000
550.0000
600.0000
650.0000
700.0000
750.0000
800.0000
850.0000
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
300.0000
400.0000
500.0000
600.0000
700.0000
800.0000
900.0000
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Boy Girl
Cutpointbetweenpre-literacyandemergingliteracy
EvaluationoftheGirl’sEducationProject3
©EDOREN 89
• Pupilsattendingschoolslocatedinurbanareasarefoundtoachievehigherscoresthanthoseinruralareas,thoughitisimportanttobearinmindthattheurbansampleisconsiderablysmaller.
• Relativelymorewell-offpupilsperformedbetterthanpoorerones,withaclearincreasingpatterndetectableacrossourconstructedwealthindextertile.Thegraphbelowshowshowthejumpfromthemedium(second)tothehigh(third)tertileisparticularlyaccentuatedforbothHausaandEnglish.Thisisconfirmedbyourrelatedregressionanalysis,whichshowsthat,evenwhencontrollingfortheroleplayedbyotherrelevantinfluencingfactors,thecorrelationbetweenliteracyandsocioeconomicstatusisparticularlysignificantforthetopendofthewealthdistribution.
Figure36: Hausabywealth
3.4.4 Analysisofrelationships
Theconsiderablesizeofthepupilsampleallowsustofurtherinvestigatetherobustnessofsomeofthedescriptiveassociationsdescribedabovebyconstructingaregressionmodelspecification.Thelattercanbehelpfulinordertoinvestigatethestatisticalsignificanceofcorrelationsbetweenouroutcomeindicatorsofinterestatthepupillevel:namelypupils’learningoutcomes(i.e.HausascaledscoresandEnglishscaledscores)andarangeofinfluencingfactorsthatcanhelpexplaintheirlearningperformance.Teacher-levelanalysisislimitedinthisrespectbyarelativesmallsampleanditisthereforenotfeasibletobuildarobustmodelthatfocusesonteachers’correlationsatbaseline.Wehave,however,embeddedinformationonteachersubjectknowledgeandpedagogicalpracticesaswellastheirdegreeofmotivationinourpupilcorrelationmodel.
InBox7wepresentthecategoriesofexplanatoryfactorsthatareincludedinthepupilcorrelationmodel.Theinclusionofthesedifferentexplanatoryfactorsinourregressionspecificationwasinformedbytheanalyticalcategoriesillustratedinourconceptualframeworkoffactorsinfluencinglearningoutcomesandbytheavailabilityandqualityofdatacollectedattheschool,teacherandpupillevel.Thechoiceoffactorsthatwecontrolforinourregressionspecificationsisimportantasitdetermineshowmuchtheeffectofeachindividualvariable(asindicatedbythemagnitudeofits
470
480
490
500
510
520
530
Lowtertile Mediumtertile Hightertile
EvaluationoftheGirl’sEducationProject3
©EDOREN 90
coefficientandthesignificancelevelreflectedbyitsp-value)detectedcanbeconsideredrobusttootherpotentiallyconfoundingfactors.
Forthisreasonwehavedecidedtousetheavailableinformationonageinourmainregressionspecification,giventhecriticalinfluenceofthisdemographiccharacteristiconlearning.Asdiscussedabove,though,agedataarenotavailableforallpupils;therefore,thesamplesizeforourmainregressionis1,613observations.Althoughthisstillrepresentsasizeablesamplefortheanalysis,thereisariskofbiasintheregressionestimatesduetotheexclusionofpupilsforwhichthereisnoinformationonage.Inordertodeterminewhetherthisisinfactanissue,wehaveundertakentwodifferentrobustnesstests.Ontheonehandwehaveimputedtheagegroupdistributionofpupilsforwhichinformationismissingonthebasisofthedistributionofthesamplewithageinformation,separatelyforIQSsandprimaryschools.Ontheotherhand,wehavealsobuiltaregressionspecificationwithoutanyagevariable,butinsteadincludingavariableonwhetherthepupilisaboarderornotinstead.Thisvariablewaschosenaspupilswhoareboardersareolderthannon-boarders.Asshownbytheregressionresultstheindicationemergingfromourmainregressionmodel,whichincludestheoriginalagevariable,isconfirmedbythesetwoalternativespecifications.Hence,thisgivesusconfidencethatagecanbecontrolledforasoneofthekeyfactorsinfluencinglearning.
Theinclusionofpupil-,teacher-andschool-levelfactorsentailsthattheeffectofeachvariableisisolatedfromtheconfoundingfactorsthatwecontrolforatthoseanalyticallevels.Atthesametime,wehavealsobuiltaschool-levelfixedeffectsmodel,whichallowsustofocusontheinfluenceofpupil-levelvariables,whilstcontrollingforallschool-levelconfounders(observableandunobservable).
Box7:Explanatoryfactorsincludedinthepupilscorrelationmodel
• Pupilpersonalcharacteristics,includinggenderandagegroup• Pupilsocioeconomicstatus,asdefinedbytheHWItertilecategorisation• Pupilschoolinformation,includingwhetherpupilsattendpublicprimaryschoolsorIQSs,
whethertheyattendotherschoolsinparallelandwhethertheycanwrite• Locationinformation,includingwhethertheschoolisinKatsinaorZamfaraandwhetheritis
locatedinanurbanorruralarea• Schoolinfrastructure96,includingwhethertheschoolhasseparatetoiletsforgirlsandwhetherit
hasaccesstoasourceofdrinkingwater• Teacheraveragemotivation,asdefinedinouroverallmotivationindexillustratedabove,inthe
schoolattendedbythepupil• Teacheraveragesubjectandpedagogicalknowledge,asdefinedintherelevantindexes
illustratedinthesectionabove,intheschoolattendedbythepupil:• teacherknowledge1:teachersubjectknowledge;• teacherknowledge2:teachersyllabusandcurriculumknowledge;• teacherpedagogy1:teacherpedagogicalknowledge;and
96Othervariablesforschoolcontextinformation,includingforinstanceelectricityorpupil/teacher,teacher/classroom,pupil/classroomandpupilandteachergenderratioswerenotincludedastheywouldhavereducedtheregressionsamplesize.Dataonthefactorsaboveandotherschoolaspectswereinfactcollectedonlyfromalimitednumberofschools.Informationonclasssizewasalsocollectedaspartofthelessonobservationexercisethroughmanualcountingineachschool.Whenincludingthisvariableinthemodelnosignificantcorrelationisdetectedandtheotherexplanatoryvariablesapproximatelymaintaintheirdirection,magnitudeandsignificance.Inanycase,thisalternativeclasssizevariablehasnotbeenincludedinthecorrelationmodelspecificationasitisnotrepresentativeoftheaverageclasssizesinourearlylearningschools.
EvaluationoftheGirl’sEducationProject3
©EDOREN 91
• teacherpedagogy2:mother-tongueteachinginearlygrades.
ThetablesbelowreporttheresultsobtainedwithourmainregressionmodelsonfactorsinfluencingHausascaledscoresandEnglishscaledscores,respectively,inourearlylearningpupilsample.Thetablesreportthecoefficientestimatedforeachexplanatoryvariable,theirassociatedstandarderror,t-statisticsandp-values,whichprovideanindicationofthelevelofsignificanceinthecorrelationbetweeneachexplanatoryvariableandtheoutcomevariableofinterest,eitherHausaorEnglishscaledscores.
Table16: MainfactorsinfluencingHausascaledscores
Variables Coeff. Stderr T-stat P-value
Gender(pupilisfemale) -9.212 6.294 -1.464 0.145
Agedfrom7to10 23.376 6.828 3.423 0.001***
Agedfrom11to15 114.695 8.856 12.951 0.000***
Agedover16 178.553 20.394 8.755 0.000***PupilinsecondHWItertile 13.769 7.181 1.917 0.056*
Pupilinthird(top)HWItertile 37.849 6.713 5.638 0.000***
Publicprimaryschool(notIQS) -57.353 11.026 -5.201 0.000***Schoolhasgirltoilets 8.479 7.486 1.133 0.259
Schoolhaswater 8.971 7.026 1.277 0.203
Pupilattendsotherschool -5.628 6.903 -0.815 0.416Schoolinruralareas -32.710 11.559 -2.830 0.005**
SchoolinKatsina 3.438 6.683 0.514 0.607
Teachermotivation 12.749 16.729 0.762 0.447
Teacherknowledge1 25.998 14.323 1.815 0.071*Teacherknowledge2 2.246 4.721 0.476 0.635
Teacherpedagogy1 -3.103 24.650 -0.126 0.900
Teacherpedagogy2 0.414 0.364 1.137 0.257constant 446.139 57.053 7.820 0.000***
Numberofobservations:1,613
R-squared0.339
Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
Table17: MainfactorsinfluencingEnglishscalescores
VariableNames Coeff. Stderr T-stat P-value
Gender(pupilisfemale) -6.752 5.706 -1.183 0.238
Agedfrom7to10 18.687 6.733 2.775 0.006**
EvaluationoftheGirl’sEducationProject3
©EDOREN 92
Numberofobservations:1,613
R-squared:0.332
Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
Thefirstclearindicationemergingfromtheresultsisthatteacher-andschool-levelfactorsdonotappeartobesignificantlycorrelatedwithliteracylearningoutcomes.Inparticular,noneoftheteachermotivation,knowledgeorpedagogyindexesarefoundtobesignificantlyassociatedwitheitherHausaorEnglishscores.Previousmodeliterationsthatincludedeachindexseparatelyalsoconfirmthatnocorrelationisdetected.Thelackofadetectablecorrelationbetweenteacherknowledgeandskillsandlearningoutcomesseemstobeinlinewithwhatemergedfromthedescriptiveandteacherassessmentanalysespresentedabove.Inparticular,theverylowcompetencylevelsrecordedforteachersinearlylearningschoolscanhelpexplainwhytheircurrentsubjectknowledgeandteachingpracticesarenotinfluencingpupils’learningoutcomes.
Atthesametime,schoolcontextvariables,suchaspresenceofseparatetoiletsforgirlsandpresenceofasourceofdrinkingwaterintheschool,alsodonotshowanysignificantcorrelationwithHausa.Whilstthestatelocationoftheschool,ineitherKatsinaorZamfara,doesnothaveanyassociationwithliteracyoutcomes,thelevelofurbanisationisshowntobecorrelatedwithlearning.PupilsattendingschoolsinruralareasarefoundtoachievelowerHausaandEnglishscoresthanthoseinurbanareas,acorrelationthatissignificantinstatisticaltermsandslightlystrongerforHausathanEnglishintermsofmagnitude.Finally,animportantfindingofourcorrelationanalysis,whichseemstoalsoconfirminthiscasetheindicationsofthedescriptiveassociationsdiscussedintheprevioussections,isthatpupilsinpublicprimaryschoolsappeartoachievelowerliteracyscoresinbothlanguagestestedthanpupilsinIQSs.Thenegativecorrelationbetweenliteracyandpublicprimaryschoolsislarger(sizeofthecoefficient)forHausathanEnglishscores.Whilstthedescriptivestatisticsalsopointedtowardsadifferenceinagebetweenthegroupsofpupils,whichcouldpartially
Agedfrom11to15 82.448 9.185 8.977 0.000***
Agedover16 126.046 14.208 8.871 0.000***PupilinsecondHWItertile 16.665 5.759 2.894 0.004**
Pupilinthird(top)HWItertile 27.566 5.902 4.671 0.000***
Publicprimaryschool(notIQS) -45.609 8.715 -5.234 0.000***
Schoolhasgirltoilets 13.246 6.512 2.034 0.043**Schoolhaswater 7.182 5.873 1.223 0.223
Pupilattendsotherschool 3.371 6.704 0.503 0.616
Schoolinruralareas -28.869 9.372 -3.080 0.002**SchoolinKatsina 4.834 5.710 0.847 0.398
Teachermotivation 21.505 15.426 1.394 0.165
Teacherknowledge1 15.479 12.668 1.222 0.223Teacherknowledge2 -0.142 3.710 -0.038 0.970
Teacherpedagogy1 -18.557 20.066 -0.925 0.356
Teacherpedagogy2 0.192 0.350 0.550 0.583
constant 278.692 53.062 5.252 0.000***
EvaluationoftheGirl’sEducationProject3
©EDOREN 93
explainthisdivergence(forthemajorityforwhichagedataareavailable),theresultsofourregressionanalysisareimportantastheyareobtainedbycontrollingforthepotentialroleplayedbyotherinfluencingfactors,suchasage.Inotherwords,thestatisticallysignificantbetterperformanceofIQSpupilsisconfirmedalsowhendiscountingforthepotentiallyconfoundingeffectofpupils’age.Importantly,thisfindingisconfirmedinbothrobustnesschecksperformedwithalternativeregressionspecificationsthatmakeuseofthewholesamplebydealingwiththemissingageinformation.Theotherclearindicationthatemergesfromourregressionmodelspecificationsisthatpupilindividual-levelexplanatoryfactorsaresignificantlycorrelatedwiththeirlearningoutcomes.Notsurprisingly,agecomesoutassignificantlycorrelatedwithlearning,witholderpupilsdoingbetterthanyoungerones.TheageliteracytrajectoryisprominentforbothHausaandEnglish,aspupilsdoincreasinglybetterastheygetolder.Themagnitudeofthecorrelationisdisproportionatelylargerforthesecondandespeciallythirdagegroup,thusindicatingthattherelationshipbetweengettingolderandachievingbetterliteracyoutcomesisnon-linear.Thisisalsoconfirmedwhenimputingageandusingthewholesampleofpupils97.
Genderdoesnotappeartobesignificantlycorrelatedwithliteracyinourmainmodel.Theotherpupil-levelcharacteristicthatemergesascrucialforliteracyistheirhouseholdsocioeconomicstatus.AsshownbytheHWItertilecategorisation,relativelywealthierpupilsarefoundtoperformbetterthantheirpoorercounterparts.ForbothHausaandEnglish,thedifferenceinlearningperformanceisparticularlyaccentuatedinmagnitudetermswhenmovingfromthemiddletothehighesttertile.Thisseemstoindicatethatwhilstwealthisanimportantfactor,thereexistsaneconomicthresholdbelowwhichimprovementsinsocioeconomicstatusarenotsufficienttoaffectlearning.Atthesametime,giventhegenerallylowlevelofliteracyamongstthesurveyedpupils,itseemsreasonabletopresumethatwealthisnotsufficienttoachievegoodlearningoutcomes.Thelackofcorrelationbetween(verylow)teacherknowledgeandpedagogyandlearningmayhelpexplainthis.Asmentionedbefore,ifteacherknowledgeandpedagogyimproveasaresultoftheGEP3interventionandpupillearninginthetreatmentgroupisalsofoundtobebetterthanthecontrolatmidline,wewouldexpecttoseeanincreasedcorrelationbetweenthesetwofactors.
Theanalysisisextendedfurtherbyusingaclusterfixedeffectsmodel(theschoolbeingourcluster),whichenablesustocontrolforallschool-levelinfluencingfactorsatonce,whichinturnallowsustoisolatetheeffectonouroutcomevariablesofinterest(i.e.HausaandEnglishscaledscores)oftheremainingnon-schoolinfluencingfactors.Theseresultsarepresentedinthetablesbelow(Table18andTable19).
Table18: Hausaschoolfixedeffectsmodel
Variables Coeff. Stderr T-stat P-value
Gender(pupilisfemale) -13.161 6.266 -2.100 0.037**
Agedfrom7to10 15.989 5.891 2.714 0.007**
Agedfrom11to15 83.130 10.829 7.677 0.000***Agedover16 133.901 23.743 5.640 0.000***
PupilinHWItertile -0.598 7.513 -0.080 0.937
97Intheregressionspecificationthatusesboardingasanalternativeexplanatoryvariable,pupilswhoareboarders(olderonaverage)arefoundtoperformbetterthantheothers,especiallyinHausa,forwhichthesignificanceofthecorrelationishigher.
EvaluationoftheGirl’sEducationProject3
©EDOREN 94
PupilinthirdHWItertile 20.247 6.027 3.360 0.001***
Pupilattendsotherschool 3.783 7.179 0.527 0.599
Numberofobservations:1,639
R-squared:0.606
Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
Table19: Englishschoolfixedeffectsmodel
Variables Coeff. Stderr T-stat P-value
Gender(pupilisfemale) -10.366 5.698 -1.819 0.070*Agedfrom7to10 15.998 7.250 2.207 0.028**
Agedfrom11to15 61.565 10.618 5.798 0.000***
Agedover16 96.922 17.944 5.401 0.000***
PupilinHWItertile 9.296 5.917 1.571 0.118PupilinthirdHWItertile 14.186 5.770 2.458 0.015**
Pupilattendsotherschool 11.474 7.783 1.474 0.142
Numberofobservations:1,639
R-squared:0.509
Note:Statisticalsignificanceofmeandifferencesidentifiedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
Themainindicationsonindividual-levelinfluencingfactorsareconfirmedbythefixedeffectsmodel,includingtheimportanceofageandsocioeconomicstatus.Inaddition,whencontrollingforallotherschool-levelfactors,genderemergesasbeingsignificantlycorrelatedwithlearning:malepupilsperformbetterthantheirfemalecounterparts.Thesignificantimportanceofgenderisalsoinlinewiththeresultsofourrobustnesscheckmodel.Thefactthatgenderemergesassignificantlycorrelatedwithliteracywhencontrollingforallpossibleschool-levelcharacteristics,whichincludeteachersaswellastheteachingenvironment,indicatesthatbeingamaleorafemalepupilhasaneffectonlearningthatwaspartiallydisguisedbyotherfactorsaffectingbothgroups.Itisnotpossibletopindownwhatalltheseschoollevelfactorsactuallyare,astheyincludebothobservableandunobservableschoolandteachercharacteristics.Itis,however,reasonabletoassumethatthesewouldincludeaspectsthataffectbothboysandgirls,eitherequallyordifferentially.Hence,thespecificinfluenceofgenderonlearningbecomesmoreapparentwhencontrollingforthishostofconfoundingfactors.Itisimportanttopointoutthatthemagnitudeaswellasthelevelofstatisticalsignificanceofthecorrelationbetweengenderandliteracyareinanycasealsolowinthefixedeffectsmodel,at5%significanceforHausa,andespeciallyforEnglishatonly10%significance.
Whilsttherelevanceofdemographicandhousehold-levelfactorsonpupilperformanceisnotsurprising,thelackofaneffectonlearningofteacherknowledgeandpedagogicalpracticesneedstobecarefullyinterpreted.Inparticular,thismaywellreflectthefactthatcompetencylevelsareverylowacrossallteachers.Itislikelythatastrongercorrelationwouldbedetectedifthereweregreatervariationincompetencylevelsacrossteachers,andifteachers’abilitytoinfluencepupils’learning
EvaluationoftheGirl’sEducationProject3
©EDOREN 95
outcomesweregreater.ThisalsoimpliesthatiftheELinterventiondoesleadtosubstantialimprovementsinteachers’knowledgeandskills,weshouldseeamoresignificantlinkbetweenteachercharacteristicsandlearningoutcomesatendline.
3.4.5 Pre-analysisplanforimpactmeasurementatmidline
ThedescriptiveandcorrelationanalysespresentedabovepresentbaselinebenchmarkvaluesfortheGEP3impactevaluationbyfocusingonthebaselinesituationoftwomainareas:ontheonehand,thebaselineanalysisdrawsasummarypictureofthemainteachers’andpupils’knowledgeandskills,aswellastheirbackgroundcharacteristics;ontheotherhand,theyprovideaninsightintosignificantassociationsandcorrelationsbetweenoutcomeindicatorsandcategoriesofinterest,whichshedlightonthefactorsthatarefoundtoinfluencepupils’learning,includingtheroleofteacher-levelintermediateoutcomes.
Atmidline,therewillbeafirstassessmentoftheimpactofGEP3ontheearlylearninginterventionsampleunderevaluation,whichwillbeinformedbybothsetsofbaselineinformation.QuantitativedatawillbeagaincollectedonHausaandEnglishoutcomes,aswellasteachersubjectknowledgeandpractices.Twosetsofanalysisareplanned:
1. Impactanalysis:quantitativemeasurementoftheimpactoftheGEP3earlylearninginterventiononpupillearningoutcomes(i.e.HausaandEnglishscaledscores),aswellasteachers’knowledgeandskills,bycomparingtheoutcomelevelsoftreatmentobservations(i.e.pupilsandteachers)withtheoutcomelevelofcontrolobservations.Disaggregatedanalysisbygirlsandboyswillbeconducteddependingonthestatisticalpoweravailable,whichinturnwilldependonthesizeofthedisaggregatedsub-samples.
2. Correlationanalysis:analysisoffactorsinfluencinglearningoutcomesaftertheGEP3interventionhastakenplaceandteachers’knowledgeandpracticesaresupposedtohaveimprovedaccordingly.ThisanalysiswillbedirectlycomparabletotheregressionanalysisperformedatbaselinetoassesswhetherthesignificanceofthecorrelationshavechangedasaresultofGEP3.
IftheimpactanalysisshowsthatthereisasignificantdifferencebetweentreatmentandcontrolgroupsattributabletotheGEP3earlylearningintervention,thecorrelationanalysisofinfluencingfactorswillhelpdeterminewhetherthishasalsoaffectedtheroleplayedbyschool-levelfactorsdirectlyinregardtolearning.Ifteachers’knowledgeandpedagogyimprovefromthelowbenchmarklevelsmeasuredatbaseline,itseemsreasonabletoexpectthattheirinfluenceonHausaandEnglishscaledscoreswillalsoincrease.ThiswouldindeedbeinlinewiththeassumptionsbehindtheprogrammeandevaluationToC.Thisdemonstratesthecrucialrelevanceofthebaselineanalysis,withrespecttothemidlineimpactanalysis.
Theeconometricmodelenvisagedformidlinewillalsotakeadvantageoftheadditionalparametersandmodellingoptionsthatwillbeavailableatthatstage.IfthecomparisonbetweentreatmentandcontrolgroupsshowsapositiveandsignificantdifferenceinoutcomeindicatorsattributabletoGEP3itwillbeimportanttoaccountforwhichofthetwogroupspupilsandteachersbelongtoinourregressionspecification,withtheuseofadummyvariable.Thiswillallowustomaintainouroverallsamplesizeintact,whilstdiscountingthedirectinfluenceofinterventionimpactfromtheothercorrelationsinthemodel.Finally,itwillalsobepossibletoexploitthelongitudinalstructureofoursampleinourregressionanalysissoastocontrolfortime-invariantobservedandunobserved
EvaluationoftheGirl’sEducationProject3
©EDOREN 96
factors,andtofurtherisolatetheeffectofeachexplanatoryvariablewiththeoutcomeindicatorsofinterest.Inordertoassesstherelevanceofthisapproach,theresultsofapanelregressionanalysiswillbecomparedtothoseofarepeatedcross-sectionalsetting.Anydifferenceinthesignificanceofthecorrelationscapturedbythetwomodelscanthenbeattributedtotheexistenceoftime-invariantunobservablecharacteristicscontrolledforinthepanelfixedeffectsmodel.Thisrangeofeconometrictechniqueswillhelpenrichourmidlinequantitativeanalysis.
EvaluationoftheGirl’sEducationProject3
©EDOREN 97
4 BaselineoftheIQSSevaluation
4.1 GEP3’sIQSS
4.1.1 Objectivesandexpectedresults
GEP3’sstrategyforintegratedQur’aniceducationfocusesontheimprovementofeducationinIQSs,withtheaimofprovidinganacceptablealternativeformofqualitybasiceducationforgirls.98Thefinaloutcomesexpectedare:improvedlearningoutcomesinbasicliteracyandnumeracy(especiallyforgirls),improvedretentionofgirls,and,toalesserextent,increasedenrolmentofgirls.
Atschool-communitylevel,theIQSSfocuses,inthefirstinstance,onstrengtheningthequalityofteachingand,secondly,onimprovingthelearningenvironment,especiallyforgirls,withincreasedcommunityparticipationthroughCBMCs.Atthesupra-schoollevelGEP3seekstoenhancethecapacityofthelocalgovernmentauthoritytosupporteffectivemanagementandimprovecoordinationanddialoguethatcontributestoplanning,coordinationandpolicy.
4.1.2 Interventionstrategy
GEP3targetsregisteredIQTEsthatimplementanintegratedcurriculum(becomingIQSs)andlargelyoperateascommunity-basedinitiatives,butthatarewillingtobuildlinkswithgovernmentforthepurposesofmonitoringandtechnicalsupport.Atleast40%ofthestudentsenrolledshouldbegirlsfortheIQStobesupportedbyGEP3.
Duringthe2015–2017period200IQSsperstateacrossthesixGEP3LGAsinNiger,BauchiandSokotoarereceivingafullschool-levelsupportpackage.Inallstates,exceptBauchi,theIQSsthatarebeingsupportedduringthisperiodwerealreadytargetedduringGEP3’sinitialyears.99Duringthescale-upphase(2017–2020)theprojectseekstoexpandtoanadditional7,500IQSs,ofwhich4,500willreceiveGEP3financialsupporttoimplementthesupportpackage.
TheIQSSpackageatschoolcommunitylevelconsistsofthefollowing:
1. TrainingandmentoringofIQSfacilitators100
GEP3supportscentre-orcluster-basedtrainingthataimstostrengthenfacilitators’classroompracticesandtoensurefacilitatorsbecomemoreeffectiveteachersofcoresubjects.TwofacilitatorswhoteachcorecurriculumsubjectsaretargetedperIQS.Atthecoreofthetrainingaremonthlycluster-leveltraining/mentoringmeetingsbetweenagroupofIQSfacilitatorsandaTeacherFacilitator,whoactsastrainerandmentor.101DuringthemeetingsatacentrallocationintheLGAfacilitatorsreviewthecurriculumforthefollowingmonth,receiveguidanceonit,watchdemonstrationlessons,andhavethechancetogivefeedback.ExperiencedTeacher
98UNICEF(2015b).AftertheGEP3re-designin2014,thefocusofGEP3supportshiftedfromsupportingtheintegrationofQur’anicschoolstoensuringthatQur’anicschools,oncetheyhaveagreedtointegration,areabletoimplementtheintegratedcurriculumatalevelofqualitysuchthatpupilsactuallylearn.99InBauchi,awholenewlistofIQSshasbeenselected,andthereforetheseIQSsmayreceiveGEP3supportforthefirsttime.100TheapproachtothetrainingandmentoringofIQSfacilitatorsisbasedontheapproachandpedagogymaterialsusedbytheTDP.101TeacherFacilitatorsareselectedfromSAMEs,SUBEBsandLGEAs.
EvaluationoftheGirl’sEducationProject3
©EDOREN 98
DevelopmentTeammembersobservethementoringsessions.Beforethestartofthe1.5-yearcycleofmonthlymeetingstheIQSfacilitatorsattendajointfive-dayinductionworkshop,inwhichtheyareintroducedtobasicpedagogicalconceptsandtheusageoftheteachertrainingmodules.Thetrainingcoversbothpedagogyandsubjectmatter(English,mathematicsandsciences).ThecontentisbasedonthecontentdevelopedbytheTDPforprimaryschoolteachertraining.ThemanualisslightlyadaptedtosuittheIQScontextandistranslatedintoHausa.
AfirstcohortofoneIQSfacilitatorperIQSattendedtheinductionworkshopinAugust/September2015.AsecondfacilitatorfromeachIQSwillstartthetrainingandmentoringcycleintheJuly–September2016quarter.
2. Provisionofapackageofclassroomteachingandlearningmaterials
GEP3doesnotaimtodevelopnewclassroommaterialsforteachersorlearners.102Incoordinationwiththestates,locallyavailableteachingandlearningaidsforlowerprimarygradeswillbeprocuredanddistributedtoeachIQS.Currently,thefollowingmaterialshavebeenidentified:Hausaalphabetchart,Hausaconsonantandvowelflashcards,Hausaalphabettextbook,aHausaalphabethandwritingworkbook,andaHausanumericaltextbook.103
ThematerialsareplannedtobedeliveredtotheIQSsbyApril–May2016.
3. Trainingofheadteachers
AccordingtotheGEP3StrategyPaperonIQSs(UNICEF2015b),headteachertrainingismeanttoimproveheadteachers’centre-basedmanagementandpedagogicalskills.Theactualcontentofthetrainingisstillunderdevelopment.104ItisnotyetclearwhowillbetargetedforthetrainingasnotallIQSshaveaheadteacherposition.GEP3estimatethathalfoftheIQSsmayhaveapersonwhoplaystheroleofheadteacherandwhocouldbetrained.Thetrainingapproachislikelytoconsistofconductingtrainingsessionsofthreedayspertermoveratwo-yearperiod.105
ThetrainingwasoriginallyplannedtostartbetweenAprilandJune2016buttheactualstartdatewilldependonthetimingofthefinalisationofthetrainingmaterials.
4. CapacitybuildingofCBMCs
ThecapacitybuildingapproachforCBMCsfollowsthetrainingapproachforSBMCs,aspromotedbyUBEC,withsupportfromESSPIN.Itconstitutesacombinationofaninitialmulti-daycluster-leveltraining,withfollow-upmentoringvisits,atleastonceaterm,bythetrainerstotheCBMCsoveraperiodofnineto12months.DuringtheinitialtrainingCBMCmembersaretrainedonthreekeysubjects:rolesandresponsibilitiesoftheCBMC,financialmanagementandgender-responsivewholecentredevelopmentplanning.106Acascadetrainingapproachisusedwithnationalmastertrainerstrainingstate-leveltrainingteams.Thestatetrainers—sourced
102 This is an adjustment compared to the GEP3 Strategy Paper on IQSs (UNICEF 2015b), in which the development ofsimplifiedinstructionalmaterialsandlessonplanningguidesforfacilitatorswasplanned.103Morespecifically,itisplannedforeachIQStoreceivefiveHausaalphabetchats,sixHausaconsonantandvowelflashcards,fiveHausaalphabettextbooks,20Hausaalphabethandwritingworkbooks,andfiveHausanumericaltextbooks(CommunicationwithUNICEFd.d.20March2016).104FirstdraftsoftheheadteachertrainingguideandhandbookareexpectedbytheendofMarch2016.TheirdevelopmentisledbyESSPIN(CommunicationwithUNICEFd.d.20March2016).105Thetrainingapproachisstillunderdiscussion(CommunicationwithUNICEFd.d.20March2016).106AccordingtotheGEP3StrategyPaperonSBMCs(UNICEF2015g)therolesandresponsibilitiestrainingwillenablenewmemberstounderstandtheSBMCspolicy,guidelinesontheestablishmentofCBMCsandactivitiesexpectedtobecarriedoutbyCBMCs.Thewholeschooldevelopmenttrainingbuildsthecapacityofmemberstoeffectivelyparticipateinevaluatingthestrengthsandweaknessesoftheirschool,toidentifyimprovementprioritiesandtomobiliseresourcestosupportimplementation.ThefinancialmanagementtraininggivesCBMCsmembersbasicskillstoensureschoolresourcesareeffectivelyusedtoachieveimprovementobjectives.
EvaluationoftheGirl’sEducationProject3
©EDOREN 99
fromSAMEs,SUBEBs,LGAsandCSOs—conductthecluster-leveltrainingandfollow-upmentoringvisits.GovernmentofficialsandCSOstaffaregroupedintopairstoconductthetrainingandthefollow-upmentoring.ThetrainingtargetsfourmembersofeachCBMC:thechairandsecretaryareexpectedtoattend,plustwoothermembersselectedbythecommunityfromtheCBMCmembership.
Theinitialcluster-leveltrainingisscheduledtotakeplaceinJuly–September2016.
5. Provisionofmini-grants
GEP3providesgrantstosupportCBMCstoimplementpriorityactivitiesidentifiedintheWholeCentreDevelopmentPlan(WCDP)thatdirectlycontributetotheenrolmentorretentionofgirlsintheIQS.107CBMCsareeligibleforGEP3grantsafterattendingtheCBMCtraininganddevelopingaWCDP.AgrantcanbeprovidedtoaCBMCatmosttwiceoveratwo-yearperiod.108CBMCgrantproposalwillbejointlyapprovedbythestategovernmentandUNICEFfollowingarecommendationbytheLGA.109110Grantamountsareplannedtobeanequivalentof$1,000perIQS.111
Grantsareplannedtobeprovidedfromthebeginningofthe2016–2017schoolyear.
Atsupra-schoollevelGEP3conductsadvocacywithSAMEsandSUBEBsinregardtopayingfacilitatorsalariesand/orotherincentives,andprovidingtextbookssuppliedbyUBEC.Government-levelsupportisfocusedonenablingtheinvolvementofSAMEandSUBEBstaffatstateandLGAlevelinthetrainingoffacilitatorsandCBMCmembers,CBMCeffectivenessmonitoringandgrantprovision.CBMCeffectivenessmonitoringwillbeconductedtermly(threetimesayear),startinginOctober–December2016,andwillbecarriedoutbyCBMCtrainersfromSAMEsinallIQSs.Inaddition,independentenumeratorsmanagedbystateGEP3teamswillconductmonitoringinasampleofIQSs.112
Table20presentsthetargetresultsthatGEP3aimstoachieveduringthepilotperiodandscale-upperiod.
107AccordingtotheGEP3StrategyPaperonSBMCsasecondaryobjectiveoftheGEP3grantistoencouragetheprovisionofgrantsbygovernmenttosupportimplementationofWSDPs.108Asecondgrantiscontingentuponsatisfactoryimplementationoftheplannedactivitiesandaccountingforfundsreceived(CommunicationwithUNICEFd.d.29March2016).109ThiscanbethemandateofSAMEsand/orSUBEBs.Thisiscurrentlynotclearandislikelytobespecifiedinthegrantguidelines.110Grantproposalsandapprovalswillbebasedonagreedgrantguidelines,whichareunderdevelopment(CommunicationwithUNICEFd.d.20March2016).111Thegrantamountisbeingdiscussedinordertoharmonisetheamountacrossprojects(amongothers,toharmonisewiththeGPEproject).112GEP3stateteamsandTeacherFacilitatorsmayalsoconductlessonobservationsinordertoprovidefeedbacktofacilitatorclustermeetings.Thisisstillunderdiscussion(CommunicationwithUNICEFd.d.20March2016).
EvaluationoftheGirl’sEducationProject3
©EDOREN 100
Table20: GEP3targetresultsforIQSS
2015–2017 2017–2020a
Numberoffacilitatorstrained 600 15,000
Numberoftrainedfacilitatorswhoattendedatleast80%ofmentoringsessions
360 5,400
Numberofheadteacherstrained 300 3,750
NumberofIQSswithfunctioningCBMC 420 3,600
Source:GEP3logframea2017–2020targetsonlyincludescale-upschools,notoriginalpilotschools
InNigerandSokotothetargetedIQSsareregisteredwiththeSAME,whichistheleadagencycoordinatingtheimplementationofthesupport.InBauchi,191GEP3IQSsaremanagedbytheSUBEB.TheremainingnineIQSsfallunderBASAME.
4.1.3 Interventionlogic
Figure37presentstheinterventionlogicofGEP3’ssupporttoIQSs,withafocusatthecommunityandschoollevel.WeusedamoredetailedToCasaframeworkforformulatingtheevaluationquestionsduringtheevaluationdesign(seeGEP3’sEvaluationFramework,2015).TheToCwillbediscussedindetailinSection4.2asitformsanintegralpartofthecontributionanalysisapproachthatisusedfortheevaluationoftheIQSS.
ThemainlogicofGEP3’ssupporttoIQSsisthatgirls’learningcanimproveinIQSswhentheIQScanprovidequalitybasiceducation,whichrequiresthatfacilitatorsteachmoreeffectivelyandthattheschoolenvironmentimprovesandbecomesmoregirl-friendly.EffectiveteachingisexpectedtoimprovebyGEP3supportingfacilitatortrainingandmentoring,headteachertrainingandthedistributionofteachingandlearningmaterials.CBMCtrainingandtheprovisionofmini-grantsareexpectedtocontributetoanimprovementintheschoolenvironment.Atthegovernmentlevel,GEP3’sstateadvocacyandLG(E)Acapacitysupportareassumedtocontributetotheimprovementofmonitoringandsupportsupervisionbygovernmentstaffaswellassustainedgovernmentfinancialsupport,whichisexpectedtocontributetotheintermediaryoutcomespresentedinFigure37.
EvaluationoftheGirl’sEducationProject3
©EDOREN 101
Figure37: InterventionlogicdiagramoftheIQSS
4.1.4 PlannedresearchandM&E
InthissectionwedescriberesearchandM&Eactivitiesthatarecomplementarytotheevaluationandthatcouldbeofuseintheinterpretationoftheevaluationfindings.
• CBMCeffectivenessmonitoring
Fromthelastquarterof2016(October–December)GEP3willsupporttheimplementationoftermlyCBMCeffectivenessmonitoring.ThemonitoringwillbeconductedbyLGAandstateSocialMobilisationOfficers.Inaddition,independentenumeratorsmanagedbytheGEP3stateteamswillcarryoutvalidationdatacollectioninasampleofIQSs.MonitoringwilluseatooldevelopedforSBMCeffectivenessmonitoring.ThetoolcapturesdataonCBMCfunctionality,113theparticipationofwomenandchildren,thepromotionofinclusiveeducationandinformationonenrolmentandaverteddrop-outsduetoCBMCaction.114
• Reviews
IntheperiodApril–June2017GEP3hasplannedareviewofIQSteachercapacitydevelopmentworkinordertoprovidelessonsandevidenceforscale-up.Inthesameperiodanassessmentoftheuseandperformanceofthegrantsisscheduled.
4.2 Methodology
Thissectiondescribesthemethodologythatguidedthedesignandimplementationofthebaselineevaluation,andanalysisofthebaselinedata.Itstartswithapresentationoftheevaluationquestionsandevaluationdesignasaframeworkforthebaselineevaluation’smethodologicalchoices.Subsequently,specificmethodologicalissuesarediscussedwithregardstothequantitativeandqualitativedatacollection.Thesectionendswithadescriptionofethicalandinclusionissues,andmethodologicallimitations.
113Thiscoversinformationoncommunitycommunication,resourcemobilisationandutilisation,WCDPandcommunityparticipation.114SBMCMonitoringToolforSocialMobilisationOfficers,versionOctober2015.
EvaluationoftheGirl’sEducationProject3
©EDOREN 102
4.2.1 Evaluationquestions
TheToCofGEP3’ssupporttoIQSswasusedasaframeworktoformulatetheevaluationquestions.Theevaluationquestionsinterrogateawiderangeofthecause–effectassumptionsunderlyingdifferentstepsintheToCinordertobetterunderstandhowchangemaycomeabout.Theevaluationquestionssubsequentlyguidedtheoverallevaluationdesignandbaselinemethodology.TheevaluationquestionsarepresentedinBox8.
Box8:EvaluationquestionsforevaluationofGEP3’ssupporttoIQSs
1. HowwellhasGEP3teachercapacitydevelopment(trainingandmentoring)contributedtoimprovedteachers’knowledgeandskills,andmoreeffectiveteachingintheclassroom?
2. HowwellhasGEP3teachercapacitydevelopmentcontributedtoanimprovementingender-sensitiveteaching?
3. HowwellhasGEP3headteachercapacitydevelopment(trainingandmentoring)contributedtoimprovedpedagogicalleadershipandschoolmanagement?
4. HowwellhasGEP3CBMCcapacitydevelopmentcontributedtoimprovedschoolmanagementandincreasedmobilisationofresourcesforschoolinvestment?
5. HowwellhaveCBMCsbeenabletoadequatelymanagemini-grantsandinvesttheseresourcesintheimprovementofagirl-friendlyschoolenvironment?
6. Howfarhaveteachingandlearningmaterialssuppliedthroughtheinterventionbeenperceivedbyteachersandheadteachersasappropriateandwelltargeted?Havetheybeenusedformoreeffectiveteaching?
7. Towhatextenthavepupilliteracyandnumeracyskills,especiallyofgirls,improvedinGEP3-supportedIQSs?HowhasGEP3contributedtosuchanimprovement?TowhatextentdoestheIQSSinterventioncontributetoreducingthegapbetweenlearningoutcomesandexpectedlearningoutcomes,asexpressedinthecurriculum?
8. Amongcommunitymembersandleaderswhataretheattitudesto,andwhatisthelevelofacceptanceof,integratingformalsubjectsintoQur’aniceducation?Howhavetheseattitudesandhowhasthislevelofacceptancechangedduringtheintervention?Why?Howhaveattitudeschangedinregardtogirlsreceivingformaleducation?
9. WhatunintendedconsequencesdoesIQSShaveforteachers,headteachers,pupilsandproprietorswithintheIQSs,aswellasforthebroaderschoolcommunity?
Inadditiontotheintervention’sToC,thechoiceoftheabovequestionshasbeenbasedonthefollowingadditionalconsiderationsandcontextualfactors:
• AspartofGEP3’sredesignthefocusofIQSShasshiftedfromimprovingaccesstoeducationtoimprovingthequalityofeducationinIQSs.Theevaluationquestionsthereforeemphasiseteachingandlearningoutcomesaswellastheteachingandlearningenvironment.
• FromanequityperspectiveitisimportanttounderstandiflearningistakingplaceforallandwhetherGEP3iscontributingtoimprovingthelearningoutcomeofthepupilswhoarecurrentlyfallingbelowexpectedlevels,asexpressedinthecurriculum.Hence,theevaluationquestionsincludeafocusonexaminingthelearningoutcomesofthelowestperformingpupils.
EvaluationoftheGirl’sEducationProject3
©EDOREN 103
• TheIQSSconsistsofseveralcomponentsandisimplementedinacontext—theQur’aniceducationsystem—thatisdiverseandnotwellunderstood.115Integrationtakesplacewithinaninstitutional,socialandculturalenvironmentthatisunevenlysupportive,withmultiplestakeholders.ThismakestheIQSShighlycomplex,withmultiplefactorsinfluencingthecourseoftheintervention.GiventhiscomplexitytheevaluationquestionsaimtoachieveanunderstandingofhowIQSScontributestointendedchangeswithintheIQScontext,ratherthantoquantifytheattributableeffectsoftheintervention.
• Theinterventioncomplexityintroducesadegreeofuncertaintyregardingtheoutcomesthatcanbeexpected.TheevaluationquestionsthereforepayattentiontounintendedconsequencesoftheIQSS.
4.2.2 Evaluationdesign
MostoftheevaluationquestionspresentedintheprevioussectionrequireinvestigationofthecausallinkbetweentheIQSSandchangesinoutcomevariables.Weproposetheapplicationofanevaluationapproachthatdrawsontheprinciplesofcontributionanalysis.Thisisapragmatictheory-basedevaluationapproachthatassessescausalpathwayswithintheToC,reportswhethertheintendedchangesoccurredornot,andidentifiesthemaincontributionstosuchchanges(DelahaisandToulemonde,2012;Mayne,2012).Inaddition,weproposelookingatunintendedchangestakingplacewithintheIQSsandtheirbroadercontextduetotheGEP3intervention.Weconsiderthisimportantbecause,unliketherelativelywell-researchedpublicprimaryschoolsystem,littleisknownaboutthesystemdynamicsofIQSs.
TheevaluationoftheGEP3’ssupporttoIQSsdoesnotmakeuseofacomparisongrouptomakecausalinferencebecauseofthenatureoftheintervention,itscontextandtheresourcesavailable.IQSScomprisesadiversesetofsupportactivitiesthattakeplaceinacontextthatisdiverseandnotfullyunderstood.ThiswarrantsastrongevaluationemphasisonhowtheinterventioncontributestochangewithintheIQS,ratherthanwhetherchangeisattributabletotheoverallpackageofIQSS.Furthermore,itwasdifficulttoconstructacomparisongroup.SincetheIQSSisprovidedtoallGEP3pilotIQSsintherespectivetargetedstates,acomparisongroupwouldneedtobeselectedfromIQSsthatarenottargetedbyGEP3.NolistofIQSssimilartoGEP3IQSswasavailablefortheconstructionofacomprehensive,unbiasedsampleframefromwhichacomparisongroupcouldbeselected.Therewasalsolittleguaranteethatavalidcomparisongroupcouldbemaintained.Finally,resourceswerenotavailabletoconstructcomparisongroupsforalloftheinterventionsunderevaluation.Prioritywasgiventotheearlylearningintervention.
Attheheartofcontributionanalysisistheaimofbeingabletomakecrediblecausalclaimsaboutthecontributionaprogrammeismakingtoobservedoutcomesinacomplexenvironmentwherethereareanumberofotherinfluencingfactorsatwork(Mayne,2012).Criticaltocontributionanalysisisthedevelopmentofawellthought-outandcredibleToCthatmakesexplicitbothhowtheintervention(orinterventioncomponent,suchasteachertraining)contributestoagivenchange(thisisoftenreferredtoasthecausalmechanism),aswellastheconditionsandinfluencingfactorsnecessaryforthiscontributiontomaterialise.Contributionanalysisalsoinvolvestheassessmentofalternativeexplanationsfortheobservedchanges.Thecredibilityofthecausalinferencedependsontheextenttowhichalternativeexplanationscanbedisprovedandinfluencingfactorsareaccountedfor.Therefore,theevaluationapproachdoesnotseektoquantifythenetattributableeffectofIQSS,
115Forexample,howisQur’aniceducationinterconnectedwiththeformaleducationsystem?
EvaluationoftheGirl’sEducationProject3
©EDOREN 104
butrathertoassesswhetherandhowtheinterventionismakingadifferenceincombinationwithotherinfluencingfactors.
Thecontributionanalysisapproachfollowsasetofsixmethodologicalsteps,whicharepresentedanddiscussedinBox9inthecontextoftheIQSSintervention.Thesestepsleadtoananalysisbasedontheformulationofacontributionstory:thatis,acompellingcaseaboutthecontributionbeingmadebytheintervention.Thecontributionstoryissystematicallyorganisedintoseveralcontributionclaims,whichassessacausalchaingeneratedbytheinterventioninconjunctionwithselectedinfluencingfactors(referredtoasacausalpackage).AsexplainedinBox9andintherepresentationoftheToCoftheIQSSinSection4.2wehaveidentifiedthreecontributionclaimsthatwillbetheframeworkofbuildingthecontributionstory.
Amixed-methodsapproachcombiningquantitativeandqualitativedatacollectionisappliedtomeasurechangealongtheToC,toassesscausallinkagesandtolookintounintendedoutcomes.QuantitativedatacollectionconsistsofrepresentativesamplesurveysamongGEP3IQSsinthesixGEP3LGAsinbothBauchiandNiger,116involvingtheadministrationofpupilandteachertests,structuredteacherclassroomobservationsandclosed-ended,structuredquestionnairesamongpupils,teachers,headteachersandCBMCmembers.Baseline,midlineandendlinesurveyswillbeconductedinacohortofIQSssampledatbaseline.ThequalitativedatacollectioninvolvesKIIs,focusgroupdiscussions(FGDs),qualitativeclassroomobservationandunstructuredteacherpraxisdiscussionsinpurposivelysampledcasestudyIQSs.ThesamecasestudyIQSswillbevisitedatbaseline,midlineandendline.Thecommunity-/school-levelqualitativeresearchiscomplementedwithandinformedbythefindingsofthestate-levelstakeholderinterviews.
Duringthe2015–2017pilotperiodthefocusoftheanalysisisonGEP3’scontributiontoachievingintermediaryoutcomes,suchaschangesinteacherknowledgeandskills,teachingpracticesandschoolenvironment.Learningoutcomeswillnotbemeasuredatmidlinebecauseofmethodologicalreasonsrelatedtotheuseofacross-sectionalsamplesurvey,whichisrequiredbecausetheevaluationdesigndoesnotincludeacomparisongroup.117Across-sectionalsamplesurveyhastwoimportantconsequencesthatconstrainmeasurementoflearningoutcomesatmidline.First,thiswouldrequiremeasurementinthesameschooltermasbaseline(i.e.October–November2017),whichisnotappropriatetimingfromtheperspectiveofevidenceneeds.Second,weestimatethatwiththegivensamplesize—whichiscircumscribedbytheavailablebudget—onlyaneffectsizeofamagnitudethatisunlikelytooccurin1.5yearswouldbeabletobedetectedwithstatisticalconfidence.Comparedtoapanel,across-sectionalsamplerequiresarelativelylargesamplesizetomeasureagivenMDEsize.Therefore,theeffectonlearningoutcomeswillbethesubjectoftheend-of-projectevaluation,meaningtheevaluationquestionregardingimprovementinlearningoutcomeswillnotbeansweredatmidlinebutatendline.Whileendlinedatacollectionhasnotyetbeenformalised,thereisanagreementthatthiswilltakeplace.StakeholdersindicatedthatevidenceontheeffectivenessofGEP3intermsofimprovingteachingqualityandtheschoolenvironmentwillprovidevaluableinformationforscale-updecision-makingatmidline.Itcanbeassumedthatifthequalityofteachingandschoolenvironmentsimproves,andotherfactorsthatmaynegativelyaffect116ThefullpackageofIQSSisalsoimplementedinSokoto.Duetobudgetconstraintsthesurveycanonlybeadministeredintwostates.NigerandBauchihavebeenpurposivelyselectedbythestakeholders,takingintoaccountbudgetconsiderations.117 A panel approach is not feasible because itwould not be possible to separate out the effect of ageing and years ofschooling from the intervention effect sincewe are not comparing the intervention groupwith a comparison group ofpupils.Thecontributionofsuchalternativeexplanationsfortheimprovementoflearningoutcomeswouldbeimpossibletoassess without a comparison group. In a cross-sectional survey, alternative explanations related to a pupil’s cognitivedevelopmentdonotneedtobeaccountedfor.
EvaluationoftheGirl’sEducationProject3
©EDOREN 105
learningoutcomesdonottakeplace,learningoutcomeswillimprove.Thisassumptionwillbeverifiedbyendline.
Box9: KeystepsincontributionanalysisappliedtotheevaluationofGEP3’sIQSS
1. Setoutthecause–effectissuestobeaddressedTheevaluationquestionscircumscribethecausalissue.ThemainfocuswillbeonthecausallinkbetweenGEP3facilitatorcapacitydevelopment,teachers’knowledgeandpedagogicalskills,effectiveteacherpracticesandlearningoutcomesinthecontextofIQSs.GiventherelativelylargenumberofcausallinksintheToC,thebreadthoftheanalysiswillneedtobebalancedwithitsdepth,andthereforewewillnotbeabletoinvestigateeverycausallinktothesamedegreeofdepth.2. DevelopapostulatedToCAfirstversionoftheToCwasincludedintheGEP3evaluationframeworkandsupportedtheformulationoftheevaluationquestions.AspartofthebaselinetheToCisfurtherdeveloped(seeSection4.2).3. GatherevidenceontheToCThroughamixed-methodsdatacollectionstrategyatdifferentpointsintime(baseline,midlineandendline)wewillcollectdataonchangestakingplaceintheoutcomevariablesidentifiedintheToC,aswellasassessthestrengthsandweaknessesofthelinksintheToC.ViaaquantitativesurveyinasampleofIQSsinBauchiandNigerwequantifychangesalongtheToC,aswellasotherinfluencingfactors.QualitativeresearchinanumberofcasestudyIQSswillprovideinformationtoallowustotriangulatethequantitativedataandassessthecontributionofGEP3supportamongdifferentfactorsresultinginthechangesobserved.Thisquantitativeandqualitativeschool-leveldatawillbecomplementedwithinformationfromamidlineroundofstakeholderinterviewsatstatelevel.Finally,theGEP3monitoringandresearchwillprovideanothersourceofinformation,inparticularallowingustoassesshowtheIQSSwasimplemented.4. AssembleandassessthecontributionstoryThequantitativeandqualitativeevidencewillbeanalysedtostatisticallyidentifythechangesthathavetakenplace,assessthestrengthsandweaknessesofthecausalmechanismsandexploreinfluencingfactors,consideringalternativeexplanations.TheanalysiswilleitherconfirmthepostulatedToCorwillsuggestrevisionswheretherealityappearsnottomatchtheToC.Basedontriangulationofevidencefrommixed-methodsdatacollectionacontribution‘story’willbesetout,whichwillexplainwhyitisreasonabletoassumethattheprogrammeactionshavecontributedtotheobservedoutcomes.Thecontributionstorywillbeorganisedaccordingtothefollowingthreecontributionclaims:
a. GEP3’ssupporttoIQSscontributestomoreeffectiveteachingofformalsubjectsinIQS;b. GEP3’ssupporttoIQSscontributestoanimproved,girl-friendlyschoolenvironmentinIQSs;
andc. moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlyschoolenvironment
contributetoimprovedlearninglevels,particularlyamonggirls.
ThesethreecontributionclaimscoveralltheevaluationquestionsthatrequireinvestigationofthecausallinksbetweentheIQSSandchangesinoutcomevariables.5. SeekadditionalevidencewhereneededTofurtherenhancethecredibilityofthecontributionclaimssetoutinthe‘story’,additionalevidencecanbesoughtout.Credibilitywillbetestedbyexposingtheanalysistopeerreview.Triangulationofdatasourceswillstrengthenthecontributionclaims(whichisthereasonwe
EvaluationoftheGirl’sEducationProject3
©EDOREN 106
ThequantitativeandqualitativebaselinedatacollectiontookplaceinOctober–November2015duringthefirsttermofthe2015–2016schoolyear.Atthattime,thefive-dayinductionworkshopaspartofthetrainingofthefirstIQSfacilitatorcohorthadalreadytakenplace(seeFigure38).Forlogisticalreasonsthebaselinecouldnottakeplaceearlier.Wedonotexpectthistosignificantlyaffectthevalidityofthebaselinedatasincethecluster-levelmonthlymeetingsareconsideredthecoreofthecapacitydevelopmentinterventionandhadnotyetstartedatthetimeofthebaseline.Furthermore,whileteachingknowledgeandskillsmayhavebeenaffectedtoalimitedextent,itishighlyunlikelythatpupillearningoutcomeswouldalreadyhavebeeninfluenced.Thedetailsofthebaselinesurveyimplementationarepresentedinthenextsection.MidlinedatacollectionisplannedinMay–June2017.BythattimeallIQSSactivitieswillhavehadayearofimplementation.EndlinedatacollectionisnotincludedinFigure38becausetheresources,scopeandtimingforthisdatacollectionhaveyettobedecided,althoughitstimingispreliminarilysetfor2019.Asdiscussedabove,thereisanagreementthatendlinedatacollectionwilltakeplace,whichwillbeimportantinordertoassessthecontributionoftheIQSStolearningoutcomes.
Figure38: TimelineforIQSSactivitiesanddatacollectioninGEP3pilotIQSs Schoolyear
2015–2016Schoolyear2016–2017
Schoolyear2017–2018
Sep–Dec Jan–Mar April–Jun
Sep–Dec Jan–Mar Apr–Jun Sep–Dec Jan–Mar
IQSS • Facilitatortraining
• Materials
• Headteachertraining • CBMCtraining
• Mini-grants
Datacollection
• Quant.survey
• Qual.casestudies
Baseline Midline
Thepupiltestsandsurveywereadministeredtoacross-sectionofpupilsatbaselineandwillbeagainatendline.Thismeansthatpupilswillbesimilarlysampledatbothsurveymoments.Testoutcomeswillrepresentthelevelanddistributionoflearningamongpupils,girlsandboys,beforetheIQSSandaftertheIQSS.Differentpupilswillbetestedinbothsurveyrounds(seeaboveformethodologicalreasonsfortheuseofacross-sectionalsamplesurvey),whichcontrastswiththepanelsurveyapproachproposedfortheearlylearninginterventionevaluation.Pupiltestswillnotbeadministeredatmidlinebecauseweestimatethattheminimumchangethatcanstatisticallybedetectedwithconfidencegiventhefeasiblesamplesizeistoolargecomparedtowhatcanbeexpectedbymidline(seeabove).Teacherassessmentwillbeconductedatmidline.Hence,changes
proposearigorousmixed-methodsdatacollection).Wehavenotplannedforadditionalevidencegenerationduringthe2015–2017period,duetobudgetconstraints,butthiscanbereassessedaftermidline,withthepotentialforittobeincludedinthe2017–2020evaluationperiod.6. StrengthenthecontributionstoryThefinalproductofthecontributionanalysiswillbeacontributionstorythatgoesbeyondthedescriptionofchangesobservedandalsomakesanassessmentofthecontributionofGEP3supporttosuchchange,whilealsotakingintoaccountotherinfluencingfactors.
Statestakeholderinterviews
Statestakeholderinterviews
EvaluationoftheGirl’sEducationProject3
©EDOREN 107
inthisimportantintermediaryoutcomewillallowustoassessthedifferencethattheIQSSismakinginthequalityofeducationintheIQSs.
AtbaselineandendlinepupilsarerandomlysampledamongallpupilsinP2gradeoritsequivalentlevelinIQSs,whentheyareinthefirsttermoftheschoolyear.Pupilsneedtohavehadatleastaroundoneyearofnon-religiouseducationinthesampledIQSstobeincludedinthetargetpopulation,inordertoensurethattheyhavebeenexposedtoteachingthatisthesubjectoftheIQSS.
Teacherandheadteachersurveyswilltakeapanelsurveyapproach.Thesameteachersandheadteacherswillbesurveyedatbaselineandmidline.Althoughsomeattritioncanbeexpectedweanticipatethatattritionlevelsatmidlinewillbeacceptable,andwillbemoderatedbythefactthattheteacherandheadteacherdevelopmentsupportwillincentiviseagainstattrition.Teacherattritionismorelikelytobeaproblematendlinein2019.Therefore,across-sectionalsampleofteacherswouldneedtobeconsideredatendline.
TheCBMCquestionnairewillbeadministeredamongaselectionofthreetofourcurrentCBMCmembers,including,atleast,theCBMCchairandmembersthathavebeentrainedthroughGEP3support.TheCBMCchairwillbethemainrespondent.TheinterviewwillbeconductedinagroupsettingbecauseinterviewpilotsdemonstratedthatwithmultipleCBMCmemberspresentvaluablecomplementaryinformationisobtained.WhileatbaselinethesurveyallowsforadegreeofflexibilityregardingwhichCBMCmembersarepresentbesidestheCBMCchair,118atmidlinetrainedmemberswillbeinvited.
4.2.3 Methodologyofquantitativedatacollection
4.2.3.1 Samplingstrategyandsampleofthebaselinesurvey
ThesamplingstrategyforquantitativebaselinedatacollectionoftheIQSSevaluationisalmostequivalenttothestrategyusedforthebaselineoftheearlylearninginterventionevaluation.ThischapterwillonlyoutlinetheelementsofthestrategythataredifferentoruniquetotheIQSSintervention.
Universe
ThestudyuniverseforthebaselineoftheIQSSinterventionevaluationconsistsofallIQSsselectedforGEP3interventioninthe12LGAsinBauchiandNigerwhereIQSSwillbeimplemented.SinceGEP3isalsoimplementingIQSSinSokoto,thestudyuniversedoesnotfullymatchGEP3’sinterventionuniverse.DuetobudgetconstraintsNigerandBauchiwerepurposefullyselectedtobecoveredbytheevaluation.
TheinclusionofschoolsintoGEP3requiresthatanIQSfulfilscertaincriteriasetbytheproject.AllIQSsincludedintheGEP3projectneedtohavestartedintegration,shouldteachbothboysandgirlsandberegisteredwithagovernmentagency.119
118 During the CBMC interview the secretary must be present for the interview to take place because he or she is animportantsourceofinformation.Inaddition,atleastonewomanrepresentativeandtheCBMCtreasurerareinvitedtotheinterview,wheneverpossible.Thevice-chaircanreplacethechairasmainrespondent.119Aminimumof40%ofpupilsaregirls.
EvaluationoftheGirl’sEducationProject3
©EDOREN 108
Thedefinitionoftheuniversealsoextendstoteachersandpupils.ThetargetpopulationofpupilsneedtobeenrolledinaP2equivalentgradeandstudynon-religioussubjects.Becauseofthenon-standardcompositionandorganisationoftheintegratedcurriculuminIQSs,aspecificdefinitionofthepupiluniverseiswarrantedinordertoachieveacoherentlydefinedpopulationuniverse(seeBox4inthemethodologicalchapteroftheearlylearninginterventionevaluation).
Similartothepupils,theuniverseofteachersconsideredaspartoftheIQSSinterventionconsistsofteachersteachingatthedesignatedschoolsandteachingpupilsenrolledinGrades1to3orequivalent(seeBox5inthemethodologicalchapteroftheearlylearninginterventionevaluation).
Samplingframe
ThesamplingframeofschoolsincludesallIQSsincludedintheGEP3projectinthetwoselectedstates.ThesamplingframewasconstructedbasedonGEP3schoollistsmaintainedbyUNICEF.ThetotalsamplingframeforthebaselineoftheIQSSinterventionconsistsof400IQSsin12LGAsintwostates.AlloftheIQSsinthesamplingframewereassumedtobeGEP3compliantandthuseligibleforinclusionintheintervention.
BecausetheevaluationdesignoftheIQSSisbasedonacross-sectionalsamplesurveyofpupils,withoutacontrolgroup,itiskeythatthesampledpupilshavebeenexposedtoapproximatelyoneyearofteachingoftheintegratedcurriculum.Itisoflittleuse,inevaluatingtheintervention,tocomparelearningoutcomesofpupilsthathavenotbeenexposedtotheteachingoftheintegratedcurriculumbeforeandaftertheIQSS.
DuringthefieldworkitbecameapparentthatthequalityofthesamplingframeofIQSswaspoor.TheQur’anicschoolsweremeanttohavestartedintegrationbyteachingnon-religioussubjects.Duemainlytonon-integration(schoolsnotteachingnon-religioussubjects)orrecentintegrationmanyoftheIQSsincludedinthesamplingframewerefoundtobeineligibleforbeingsurveyed.Thisrequiredthesamplingofreplacementsschools.
Samplesizeandsamplingparameters
Thesamplesizeisinfluencedbybudgetandsurveydesignconsiderations(i.e.cross-sectionvs.panelsurvey).Withintheseparametersthesamplesizeisintendedtomaximisethestatisticalpowerofthemeasurementofthechangeinthelearningoutcomeindicatorbetweenbaselineandthefollow-upsurvey(andtominimisetheMDE).
Ineachstate,30IQSsweresampledforstudypurposes,creatingatotalsamplesizeof60IQSsacrossthetwostates.Weexpecttosurveythesamesampleofschoolsatbaselineandmidline,andthelikelihoodofattritionwithintheschoolsampleisexpectedtobelow.Ineachschool,thetargetsamplesizeforpupilsissixgirlsandsixboys,resultinginatotaltargetedsamplesizeof720pupilsacrossthe60IQSs(360girlsand360boys).Thesamplesizesarecalculatedtoallowfordisaggregatedanalysisforgirlsandboysatendline.Giventhecross-sectionalnatureofthepupilsampling,thissamplesizewillremainthesameatendlineasatbaseline.
ThetargetedsamplesizeforteachersistwofacilitatorsperIQS,andtherefore120IQSfacilitatorsintotal.Weexpectthesamplesizeoftheteacherstobeloweratthemidlinesurveyduetopossibleattritionofthepanel.
Table21presentsthesamplesizesforallIQSsandinstrumentsimplementedwithineachIQS.
EvaluationoftheGirl’sEducationProject3
©EDOREN 109
Table21: SampleparametersfortheIQSSintervention
Bauchi Niger
NumberofIQSsperstate 30 30
InstrumentsperIQS Headteacherinterview 1 1
Teacherinterview† 2 2
Teachercompetencytest 2 2
Classroomobservation 2 2
CBMCinterview 1 1
Pupilinterview* 6girls,6boys 6girls,6boys
EnglishliteracyassessmentforP2 6girls,6boys 6girls,6boys
HausaliteracyassessmentforP2 6girls,6boys 6girls,6boys
NumeracyassessmentforP2 6girls,6boys 6girls,6boys†Thesameteacherswillbeinterviewed,testedandobserveddeliveringlessons*Thesamepupilswillbeinterviewedandtested
Thesamplesize,amongotherfactors,willdeterminetheminimalchangeinanoutcomeindicatorthatcanbedetectedwithstatisticalconfidencebetweenbaselineandafollow-upmeasurement.Aswasexplainedearlier,thisisreferredtoastheMDE.InthecaseoftheIQSSevaluationtheMDEisbasedonasingledifferencebetweenfollow-upmeasurementandthecorrespondingbaselinevalues.InBox10wesummarisetheMDEsforthelearningoutcomeindicatorandteachers’knowledgeandskillsindicatorgiventhepupilandteachersamplesizes.ThedetailsofthestatisticalpowerprinciplesusedarepresentedintheGEP3EvaluationFramework(EDOREN,2015).
Box10:EstimationofMDEs
MDEsforpupillearningoutcomesInlinewiththeGEP3logframe,themainlearningoutcomeindicatorisdefinedasthepercentageofgirlsandboysthatachievebasicliteracyandnumeracy.Inadditiontothesamplesize,theMDEisafunctionofthebaselinevalueoftheoutcomeindicator.Wedonotknowthebaselinevaluesofthelearningoutcomeindicatorsatthispoint,but,aswasdiscussedpreviously,learningoutcomesinnorthernNigeriaaregenerallyverylow.However,learningoutcomedatamostlycorrespondtothesituationinpublicprimaryschoolsandfewdataexistaboutlearningoutcomesinIQSs.WethereforeestimatetheMDEsusingplausibleandmaximumscenarios.WeassumeanICCof0.3,asimilarITC,80%power,anda95%confidencelevel.Usingaplausiblescenariothatthebaselinepercentageofpupilsthatachievebasicliteracyornumeracyisaslowas10%,theMDEisestimatedat6.8percentagepointswhengirls’andboys’dataareanalysedtogether,and7.3percentagepointswhenanalysedseparately.ThelargestMDEforagivensamplesizewouldbeproducedwhenthebaselinevalueis50%.Insuchacase,theMDEamountsto11.4percentagepointsforacombinedgirlsandboysanalysisand12.2percentagepointsforaseparateanalysis.Wedonotexpectalargemagnitudeofeffectsinlearningoutcomesbetweenbaselineandamidlinein2017,butconsideritreasonablebetweenbaselineandanendlinein2019,whichiswhymidlinepupiltestingisnotproposedaspartofmidlinedatacollection.Ifalowbaselinevalueisassumed,suchas10%,theMDEisinlinewithan8percentagepointincreasebyendline,astargetedintheGEP3logframe.Bycontrollingforsomecovariatesthatinfluencelearningoutcomes,
EvaluationoftheGirl’sEducationProject3
©EDOREN 110
Samplingdesign
ThemainsamplingmethodfortheIQSswasasingle-stagestratifiedsystematicrandomdraw,whileteachersandpupilsweredrawnthroughatwo-stagestratifiedsystematicsample.ExplicitstrataweredefinedbytheLGA.AnequalnumberofschoolswasdrawnfromeachLGA.Withineachexplicitstratumanimplicitstratificationwasbasedontypeoflocation(urban,rural).Thesystematicrandomsampleusedarandomstartandarealsamplingstep.Implicitstratificationwasachievedbysortingwitheachexplicitstratum.
IneachLGAfiveIQSsweredrawn.Inaddition,oneIQSperLGAwasdrawnasaninitialreplacementIQS,toprovideapoolofIQSsthatcouldbesurveyedincaseanyoftheIQSsinthemainsamplewerefoundtobeineligibleorinaccessibleduringthefieldworkperiod.Therefore,thesamplingsetuphadprovisiontoselectsixschoolsinanygivenLGAinNigerandBauchi.
Thereplacementschoolsweremarkedafterthesamplehadbeendrawn.AuniformlydistributedrandomvariablewasusedtosorttheIQSswithineachLGAandthelastIQSinthissequencewasdesignatedasareplacementIQS.
DuringthefieldworkmanyoftheIQSswerefoundtobeineligible.Thereasonforineligibilitywasmainlynon-orrecentintegrationofschools.Forthispurpose,additionalreplacementschoolshadtobesampledduringthefieldwork.Thedrawingofthereplacementschoolsmostlymirroredthemainsamplingapproach.InsomeLGAsthenumberofeligibleGEP3IQSsbecamequitelimitedduetohighproportionsofineligibility.Inthesecasesalloftheavailablereplacementswerereleasedtothefieldforthesurveyteamtoselectreplacementsinanorderthatcorrespondedtoapre-assignedrandomsequenceofrelease.
Specifically,arandomvariablewasusedtodesignatethereplacementsequenceamongthedesignatedreplacementIQSs.
forexampleageofthepupil,weexpecttoreducethevarianceoftheestimatesaspartoftheanalysis.Thismayallowustodetectsmallereffectsizesforagivenpowerandsamplesize.
MDEforknowledgeandskillsoutcomesTheteacherknowledgeandskillsoutcomeindicatoristhepercentageofteachersdemonstratingminimumknowledgeandteachingskills.The2013TDNAassessedteachers’professionalknowledgeandskillsforIQSteachersinsomeGEP3states.Whiletheresultsvariedfromstatetostate,a10%baselinevalueseemsplausible.Betweenthebaselineandmidlineapanelsurveyapproachwillbeused,whichresultsinahighITC.WeassumeanITCof0.8,anICCof0.3,80%power,anda95%confidencelevel.Wetakeintoaccount25%attritionofsampledteachersbetweenbaselineandmidline.TakingintoaccounttheaboveparametersweestimateanMDEof5.1percentagepointsbetweenbaselineandmidline,assumingabaselinevalueof10%.TheMDEwillbehigherbetweenbaselineandendlineifacross-sectionalsampleisdrawn(assuming0.3ITCandnoattrition)–thatis,8.5percentagepoints.Assuminga50%baselinevaluescenario,whichreturnsthehighestpossibleMDEforagivensamplesize,theMDEforapanelsurveyapproachbetweenbaselineandmidlinewouldbe8.4percentagepoints,and14.2percentagepointswhenselectingacross-sectionalsampleatendline.InlinewiththeGEP3logframe,weanticipatethatIQSSwillhaveaconsiderableeffectonteacherknowledgeandskillssinceteacherswillbethedirectbeneficiariesofintenseteachertrainingandmentoring.TheGEP3logframetargetsa30percentagepointincreaseinthepercentageofteachersdemonstratingminimumteachingcompetencies.
EvaluationoftheGirl’sEducationProject3
©EDOREN 111
AnnexCprovidesmapswithanoverviewofthesampledIQSsinthedifferentLGAsinBauchiandNiger.
WithinIQSs,samplingofpupilsandteacherslargelymirroredtheapproachdescribedintheearlylearninginterventionsectionofthisreport–pleaseseeforreferenceSection3.2.3.4.
ThesamplingofIQSfacilitatorsemployedaslightmodificationduetothemodalityoftheintervention.AtleastonefacilitatorfromeachGEP3IQSinNigerandBauchiwastrainedinSeptember2015.Thesetrainedfacilitators,iffoundatthecentre,wereselectedwiththeprobabilityof1asoneofthetwopotentialrespondentsinthebaselinesurvey.Figure39illustratesthesamplingflow.
TheCBMCquestionnairewasadministeredtothechairpersonorvice-chairpersonoftheCBMC,inthepresenceofuptothreeotherCBMCmembers.ThesecretaryoftheCBMChadtobepresentfortheinterviewtotakeplace.Inaddition,atleastonewomanrepresentativeandtheCBMCtreasurerwasalsotobepresentfortheinterview,wheneverpossible.
Weighting
Samplingweightswereconstructedtoreflectthecomplexsamplingdesign.Theweightswereestimatedforalllevelsofanalysisandreflectthemulti-levelnatureoftheschooldata,withpupilsandteachersnestedwithinschools.Theestimatedweightsarepopulationweights,whichsumuptothetotalpopulationsize,asdefinedbytheuniverseandthesamplingframe.Duetotheveryhighsamplingfractionrate(i.e.thelargeproportionoftheeligibleschoolssampled),finitepopulationcorrectionfactorswerealsoestimatedtoadjustthevarianceestimates.AdetailednoteonthecalculationoftheweightscanbefoundinAnnexD.
EvaluationoftheGirl’sEducationProject3
©EDOREN 112
Figure39: TeachersamplingTalkwiththeHeadTeacher
Public
Askforteacherattendanceregistersforthedayofthe
interviewforteachersteachinginP1-P3
YES
Check:• Itisforthecurrentday• Itcoversteachersofthecorrectgrade• Itcoversteachersteachingnon-religious
subjects
Areanyattendancerecords
available?NO
RequesttheHeadTeachertoidentifyallteachers(includingvolunteers)teachinggrades
P1-P3
Arerecordscomplete?
YES
EnterthenamesofalltheteachersinthelistsornamedbytheheadteacherintoasamplingForm(CAPI)
NO
Identifythemissinggroupsofteachersormissingnamesofindividualteachers(includeanyvolunteerteachers)
IstheschoolpublicorIQS? IQS
AretheclassestaughtingradesP1-P6?
YES
Askforteacherattendanceregistersforthedayofthe
interviewforallteachersteachingP1-P3orequivalentpupils
NO
Areanyattendancerecords
available?
YES
RequesttheHeadTeachertoidentifyallteachers(includingvolunteers)teachingP1-P3or
equivalentpupils
NO
Foreachteacherspecifywhetherteacherteachesreligiousornon-
religioussubjects
CAPIselectsteachersrandomly:• 3teachersinpublicschools• 2teachersinIQS
Inviteselectedteacherfortheinterview
Istheteachereligibleandhasgiven
consent?
Administertheinterview,classroomobservationandtestto
theteacher
EnterthenamesofallselectedteachersonSchoolCompletion
Form
YESNO
Samplethereplacementteachers
Whichinterventiondoesthe
schoolbelongto?
EARLYLEARING
IQSSUPPORT
RequesttheHeadTeachertoidentifytheteacherorfacilitatorthathasbeentrainedbyUNICEF
GEPtrainer.
Howmanyteachersor
facilitatorshavebeentrained?
ONE TWO
EvaluationoftheGirl’sEducationProject3
©EDOREN 113
4.2.3.2 Instrumentdesignandpreparation
NineinstrumentsweredevelopedtocollectquantitativedataateachsampledIQS.Theinstrumentdevelopmentprocesswasthesameastheonedescribedinthemethodologicalsectionoftheearlylearninginterventionevaluation(seeSection3.2.5).TheinstrumentsforthebaselineoftheIQSSevaluationweresimilartotheonesusedforIQSsaspartofthebaselineoftheearlylearninginterventionevaluation,inordertoallowforcomparabilityandtoachieveefficiencygains.However,twoadditionalinstrumentsweredevelopedtocoveradditionalevaluationquestionsandelementsintheToC:apupilnumeracyassessmentandaCBMCquestionnaire.TheinstrumentsarepresentedinTable22.
Table22: Overviewoftheinstruments
Instrument Summary
Pupilquestionnaire
Thepupilquestionnaireprecedesthepupilassessments.Itaimstocapturebackgroundbasicbackgroundinformation,includinggender,numberofyearsatschool,ifthepupilattendsotherschoolsetc.Italsoincludeschecksforpupildisabilityandcollectsdataonpupilhouseholdassets.Pupilsidentifiedashavingadisabilityarenotgivenassessmentitemsthatrequirethatparticularabilityinordertorespondmeaningfully(forexample,apupilwithanidentifiedinabilitytoseeisnotaskedtocompletethereadingitems).
PupilEnglishliteracyassessmentforP2
TheEnglishliteracyassessmentcontains13items,witheachitembeingmadeupofseveralsub-items.Whiletheassessmentistargetedatadifferentproficiencyrange,theEnglishliteracyassessmentislinkedtotheESSPINCSassessmentsandisprojectedontotheEDORENEnglishliteracyscale.Therefore,robustcomparisonsbetweenESSPINlearningoutcomeresultsandGEP3evaluationlearningoutcomeresultscanbeundertaken.Theassessmenttestsarangeofliteracyknowledgeandskillsacrossthepre-literacy,emergingandbasicliteracyranges.Knowledgeandskillsincludeletterrecognition,phonologicalknowledge,printconcepts,oralliteracy,verbalcomprehension,initialsoundsandletters,readinghighfrequencywords,verbalandwrittengrammar,writinghighfrequencywords,readingfluency,andcopyingandspellinghighfrequencywords.
PupilHausaliteracyassessmentforP2
TheHausaliteracyassessmentisdesignedtotestthesameliteracyknowledgeandskillsastheEnglishliteracyassessment.Itemsarenotmerelytranslated,butratherparallelitemsaredevelopedtotestsimilarconceptswhenappliedtotheHausalanguage.
Pupilnumeracyassessment
Thenumeracyassessmentcontains16items,witheachitembeingmadeupofseveralsub-items.Whiletheassessmentistargetedatadifferentproficiencyrange,thenumeracyassessmentislinkedtotheESSPINCSassessmentsandisprojectedontotheEDORENnumeracyscale.Therefore,robustcomparisonsbetweenESSPINlearningoutcomeresultsandGEP3evaluationlearningoutcomeresultscanbeundertaken.Theassessmenttestsarangeofnumeracyknowledgeandskillsacrossthepre-numeracy,emergingandbasicnumeracyranges,includingnumber
EvaluationoftheGirl’sEducationProject3
©EDOREN 114
4.2.3.3 Surveyfieldwork
ThesurveyfieldworkimplementationfortheIQSSbaselinesurveyfollowedthesameprotocolsandproceduresthatwerefollowedintheearlylearninginterventionbaselinesurvey.OnlytopicsspecifictotheIQSSinterventionarediscussedinthissection.
Unitssurveyed
Usingtheeligibilitycriteriamentionedabove,theteamsurveyed576pupilsand96teachersfrom60IQSsinBauchiandNiger(SeeTable23).Thetargetedsamplesizeof60IQSswasachieved,whilethepercentageofsampledpupilsandteachersfromtheseschoolsthatweresurveyedequals80%and80%respectively.Inaround20%oftheIQSsonlyhalforlessofthetargetnumberof12pupilscouldbesurveyedbecauseonlyalimitednumberofpupilswereeligibleforthesurvey.Thebelow-targetachievementofteacherssurveyedwasduetothefactthattheschoolsdidnothavethetargetednumberofteachers.
concepts,additionandsubtraction,measurement,moneysums,multiplication,division,volumeandcapacity.
Teacherquestionnaire
Theteacherquestionnairecollectsdataonteachers’backgroundcharacteristics,absenteeism,trainingandremuneration.Italsoincludesquestionsonmeetingsandsupervisionfromtheheadteacherandamoduleonteacherperceptionsandattitudes,whichwillbeusedtodevelopameasureofteachermotivation.
Teacherknowledgeandskillsassessment
Theteacherknowledgeandskillsassessmentisdividedintothreesections,collectivelycomprising30items,includingmultiplechoice,shortresponseandlongresponseitems.ThisincludesmarkingpupilresponsestoHausaliteracyquestions,indicatinggradelevelsforitemsbasedonthecurriculum,developingananswersheetforareadingtestaimedatGrade2pupils,answeringareadingcomprehensionquestion,andmakingjudgementsaboutpupils’writing,includinggrammar,spellingandadditionalsupportneeded.
Teacherclassroomobservation
Theclassroomobservationrecordsthefrequencywithwhichtheenumeratorobservesspecificteachertalk,teacherlanguageuse,teacheractionandpupilaction.Theenumeratoralsorecordsteacheractionatthestartofthelesson,actionattheendofthelesson,resourcesusedandsubjectobserved.
Headteacherquestionnaire
Theheadteacherquestionnaireincludesseveralsections,includingbackgroundinformationontheschool,basicinformationabouttheschool,schoolmanagementandmonitoring,attendanceandenrolment,attitudestowardsintegrationforIQSsandschoolinfrastructure.
CBMCquestionnaire
TheCBMCquestionnairewasadministeredtothechairmanoftheCBMCand,wherepossiblewithothermembersoftheCBMCpresenttoverifyresponses.Thequestionnaireincludesitemscoveringgeneralinformation,CBMCactivities,schoolmonitoringandimprovement,andCBMCfinancingandfundraising.
EvaluationoftheGirl’sEducationProject3
©EDOREN 115
35%oftheoriginallysampledIQSshadtobereplaced,mainlyduetotheIQSnotbeingconsideredeligibleforthesurveygiventhestudyuniverse.120Replacementsweremostlyduetothecentrenotteachingnon-religioussubjectsyet/anymore,oritbeingrecentlyintegrated.121Asexplainedabove,areplacementprocessinlinewiththesampledesign—avoidingtheintroductionofbiasinthisprocess—wasplannedfor,todealwiththechallengeswiththesampleframe.
Table23: Summaryofsurveyachievement
Bauchi Niger
Schools
Targetsampleschools 30 30Totalschoolsurveyed 30 30
Originallysampledschoolssurveyed 16 23Replacementschoolssurveyed 14 7
Totalschoolscontacted 57 48Contactedschoolthatwerenotsurveyed–reasons Recentlyintegrated 14 1
Non-integrated 8 15 Non-existing 4 1 Nogirls 0 1
Other 1 0Percentageofsampleschoolssurveyed 100% 100%Pupils
Targetsamplepupils 360 360
Percentageoftargetpupilssurveyed 82% 78%
Teachers
Targetsampleteachers 120 120
Percentageoftargetteacherssurveyed 77% 83%
Source:Fieldworkreports
StatusoftrainedfacilitatorsinIQSSintervention
InBauchiandNigerstatesthenamesof70-UNICEFtrainedfacilitators(60fromthemainsampleofschools,withadditional10replacements)from12LGAswerereceived,tobeincludedinthesurvey.Withthisinformation,thesurveyfieldteamwasabletomatchandcompleteinterviewswithi)17facilitatorswhowerepartofthelist,andii)twofacilitatorswhosenamesdidnotmatchwiththelistbutwhoparticipatedinthetrainingunderadifferentname.Intheschoolsthatwereconfirmedtobe
120ForanIQStobeconsideredeligibleorqualifiedforthesurvey,itwasrequiredthat:1)theschoolmusthaveaP2equivalentclass;2)itmusthaveteachersofnon-religioussubjectteachingP1–P3;3)itmusthavebeenintegratedforapproximatelyayear;and4)itmusthavefemalepupils.121ThelengthofintegrationintheIQSswasparticularlyanissueinBatsariLGAinKatsina,whereapproximately80%ofschoolsintegratedin2015.Inthiscasetheeligibilitycriteriaofone-yearintegrationwasappliedmoreflexiblybysurveyingtheIQSthathadactuallystartedintegrationinJanuary2015–hencehaving10monthsofintegration.
EvaluationoftheGirl’sEducationProject3
©EDOREN 116
eligible,insixcasesthefacilitatorswerenotknown,inoneschoolthefacilitatornolongertaughtthere,andintwocasesthelistedpersonswereconfirmedtohaveattendedthetrainingbutwerereportednottobeteaching(seeFigure40).In13eligibleschoolsthefieldteamswerenotabletofindthelistedfacilitators.Therefore,atotalof48sampledfacilitatorswerenotfound.
Figure40: Flowchartofdatacollectionrelatingtofacilitators
2StatesBauchiNiger
12LGAs5schoolsineach
70facilitatorNamesprovided
Fielded?
NO
3schoolsNotattempted
YES
67schoolsfielded
Schooleligible?
YES
41schoolseligible
YES
17interviewed2interviewedunderassumedname_____________________________19* (18%)
Facilitatorsfound?
Knownreasons1Nolongerteaches6personsnotknownatschool2knownbutnotteaching_____________________________9* (9%)
Unknownreasons13Notfoundatschool_____________________________13* (12%)
NO
NO
Schoolsvisited105
(100%)
Schoolseligible19Facilitatornamenotprovided_____________________________19* (18%)
Schoolsineligible19Facilitatornamenotprovided_____________________________19** (18%)
38Replacement
schools
Schoolsineligible26schoolsineligible_____________________________26** (25%)
NO
4.2.4 Methodologyofqualitativedatacollection
Thissectiondiscussesthemethodologyofthequalitativedatacollection,includingtheoverarchingmethodologicalapproachused,howrigourwasachieved,thesamplingstrategyandgeneralisability,structuredandunstructuredmethodologies,andinstrumentdesign.Finally,thefieldworkandanalyticalapproachesaredescribed.
4.2.4.1 Methodologicalapproach
Thequalitativeresearch,similartothequantitativeresearch,aimstocollectdataatthreepointsintime:2015,2017and2019(with2019tobefurtherdetermined).Themethodologyforthequalitativeresearchusesthefollowingthreemaintoolstocollectdata,structurefindingsanddrawconclusionsagainsttheoverallobjectivesofthecontributionanalysis:
1. TheprincipalorganisingstructurefortheevaluationmethodologyistheToCoftheIQSSinterventions,whichisusedtoexaminethecausallinkbetweentheinterventionoutputsandoutcomesinthecontextofIQSs.TheToCwillinformtheevaluationasawhole,butisparticularlyimportantforthequalitativeresearchbecauseitpermitsstrongergeneralisation.
2. TheevaluationmatrixturnstheToCoftheIQSSintoasetofhypotheses,assumptions,andareasofinterest,onwhichtheinstrumentationisbased.
3. DatafromasmallnumberofIQScasesisusedtoassessifandhowtheexpectedchainofeventspositedbytheToCmayoccur,validatetheinfluencingfactorsandassessalternative
EvaluationoftheGirl’sEducationProject3
©EDOREN 117
explanations.AnIQScaseconsistsoftheIQSwithinitsschoolcommunity–hence,includingbothschool-levelactorssuchasproprietor,headteacher,teachers,CBMCandpupils,aswellascommunity-levelactors,suchasparents,communityleadersandgovernmentofficials.
Theaimoftheinitialqualitativebaselineresearchistoexaminethehypothesesandassumptionswithkeystakeholders,andcontextualfactorsthatarelikelytoaffectimplementationandoutcomes,andtoassesstherelevanceoftheplannedactivitiesforbettereducationoutcomes.Asaresult,thequalitativebaseline,togetherwiththequantitativeresearch,examinesthebaselinesituationoftheToC(confirmatorymethodology)andrefines,developsnewhypothesesfor,andenrichesandcontextualises,theToC(exploratorymethodology).ThisprocessallowstheEDORENteamtoprovideanevidence-basedcritiqueoftheToC,identifyingpotentialweaknessesandproposingwaysofstrengtheningtheprogramme’sToC.
4.2.4.2 Rigour
Amajormethodologicalchallengeinqualitativeresearchisthedefinitionandachievementof‘rigour’,particularly,asinthiscase,whentheresearchmethodologyneedstobeopentotheidentificationofnewhypotheses,causesandunexpectedimpact.Therefore,themethodologyshouldcontainanemergentdimensionnotfullyprescribedattheoutset.Moreover,qualitativeresearchisoftenaccusedofbeingopentoresearchbiasoranecdotalimpressions,impossibletoreproduceanddifficulttogeneralise(MaysandPope,1995).However,rigourinqualitativeresearchcanbeachievedinthefollowingways.First,sampling‘isacoredesignissuebecausethepurposefulsamplewilldeterminewhatyoulearnabout’(Patton,2015):throughacarefullychosensamplingstrategy,themostrelevantsamplecanbeapproximated.Inaddition,rigourcanbeensuredthroughtheadequacyofqualitativeinstrumentsthatbestreflecttheToC,fieldworkthatinvolvescarefullytrainedandwell-managedresearchers,andqualityassurancethroughouteachresearchstage.Lastly,analysisbiascanbeminimisedbyrelyingonaconfirmatoryanalysisapproachthatutilisesqualitativedatabasedonasetofpre-existinghypotheses.Together,thesemeasuresensurerigourinqualitativeresearchthrough‘systematicandself-consciousresearchdesign,datacollection,interpretationandcommunication’(MaysandPope,1995).Howthequalitativeresearchdesignaddressestheseissuesisdiscussedintherelevantsectionsbelow.
4.2.4.3 Samplingstrategyandgeneralisability
Aswithmostqualitativeresearch,thechosenapproachtosamplingforthequalitativecomponentoftheIQSSevaluationwasdesignedtogenerateresponsesfromsmallnumbersofindividualsandgroupsthatarerepresentative(thoughnotstatistically)ofgroupsrelevanttoGEP3’sIQSS,andwhichallowsomeidentificationofheterogeneouscontributions.
Specifically,theresearchemployspurposive‘typicalcasesampling’and‘extremecasesampling’.Thisisnotdesignedtoproduceresultsthataregeneralisableinthesamesenseasquantitativedata.Rather,thegeneralisabilityofthequalitativeresearchresultsderivesfromtheextenttowhichtheyareembeddedinaToCthathassomevalidityinawidercontext.ThisformofsamplingallowsexplorationofwhattheIQSSisdoinginatypicalcase,butalsoperformancein‘higherperforming’and‘lowerperforming’cases,whichhelpsexplainthereasonsbehindhigherandlowerlevelsofcontributionacrosscontexts.
Forthepurposesofthisevaluation‘lowerperforming’caseswereinterpretedasschoolsandcommunitieslikelytobemostresistanttochangeinincreasingtheeducationoutcomesforgirls
EvaluationoftheGirl’sEducationProject3
©EDOREN 118
(lowerfemaleenrolment,negativeattitudestowardgirls’educationandlowerlevelsofacceptanceoftheintegrationprocess).‘Higherperforming’caseswereinterpretedasschoolsandcommunitieslikelytobelessresistanttochangeinincreasingtheeducationoutcomesforgirls(higherfemaleenrolment,positiveattitudestowardgirls’educationandhighlevelsofacceptanceoftheintegrationprocess).
Thequalitativestudysampleaimstoreflectthe‘typicalcase’andthe‘extremecase’ofthequantitativedatacollectionsample.ThisinitiallyreliedonaUNICEFlistofIQSsthatwerereportedtomeetcertaineligibilitycriteria,suchashavingasufficientpercentageofgirlsastheirpupilsandteachingnon-religioussubjects.However,asthequantitativedatacollectionprecededthequalitativestudy,itbecameclearthatmanyoftheIQSsincludedinthesamplingframewerefoundtobeineligibleforbeingsurveyed(seeSection4.2.3).Forthatreason,toensuretheeligibilityofallsampleschools,thestudysamplewasrestrictedtothoseschoolsthathadbeenpreviouslyinspectedandwereconfirmedasmeetingUNICEFeligibilitycriteria.
SixschoolswerepurposefullysampledamongpreviouslyinspectedGEP3IQSsinBauchiandNiger.Foreachstate,alowerperforming,typicalperformingandhigherperformingschoolwereidentified.Thesamplesizewasdeterminedbyresourceconstraintsandaneedtoahaveaminimalspreadoftypicalandextremecasesacrossthetwostates.
AchallengetoidentifyingthetypicalandextremeIQScasesisthatIQSsarenotcurrentlyincorporatedintothestates’EMISdatabases.Assuch,itwasimpossibletosampledirectlybasedonIQSperformance.Instead,thesampledesignreliedonatwo-stagedapproach.
First,threeLGAsperstateweresampledineachcasecategoryusingaveragepublicprimaryschoolperformanceongirls’educationineachGEP3LGAasaproximateindicatorforIQSperformanceongirls’education.Basedontheevaluationmatrix,andinlinewiththebestavailableEMISdata,thefollowingindicatorswereselected:
• Girls’access:TheGPI,ascalculatedbytheaverageshareofgirlsversusboysenrolledinschool.122
• Girls’retention:Anapproximatedtransitionrate,ascalculatedbyaschool’sshareofgirls’enrolmentinGrade6(endofprimaryschool)dividedbyitsshareofgirls’enrolmentinGrade1.123
• Girls’educationquality:Acompositeindexofthepupil-to-qualified-teacherrate,shareofqualifiedfemaleteachersandpupil–classroomratio.124
Fromthis,anoverallcompositeindexwasgeneratedthatrankedtheLGAsfromhighesttolowestperforming,basedonanequalweightingacrosseachofthethreeindicators.LGAswerethensampledtobeasclosetothetop,middleandbottomofthelistaspossible.
Secondly,IQSswereselectedwithineachLGAbasedonthefollowingcriteria:
122Thiswasusedintheabsenceofbettervariables,suchasthegender-disaggregatednetenrolmentrate.123Thismeasureisanimperfectapproximationandisaffectedbyshiftingenrolmentsizeovertime;atruetransitionratewouldbecalculatedbasedonaspecificcohort’sprogressionfromGrade1toGrade6.124Inabsenceofdirectmeasuresofpupilperformance(e.g.testscores),thesethreevariablestogetherprovideaweak,butcommonlyusedandindicative,measureofschoolqualityandwealth.
EvaluationoftheGirl’sEducationProject3
©EDOREN 119
• Frominspectionvisits,IQSselectionusedtwoextracharacteristics:variationinyearssinceintegration,andthenumberofnon-religioussubjectfacilitators.Aslonger-runningschoolsandmorefacilitatorswereconsideredtobe‘better’IQSs,theywereselectedforthehigherperformingLGAs(withthereverseusedforselectinglowerperformingIQSs).
• IQSlocationwassampledtoincorporateaspreadofurban,peri-urbanandruralschools,withlessurbanareasgenerallyperceivedtoleadtoalowerqualityIQS.Thesamplingofmoreremoteschoolswasbalancedwiththeaimofcompletingasmanyinstrumentsaspossiblewithinthepossibleinterviewingtimeframe,soastoreturnbacktoeachstatecapitalbeforedark.
• Languagealsoplayedapartinsampling:areaswithadominantminoritylanguage(e.g.NuperatherthanHausa)requiredtwo-waytranslationfromoneteammember–overusecouldleadtofatigueandreducethequalityofinterviewresponses.Assuch,onlyoneoutofthesixIQSswassampledasanon-Hausaspeakingschool.
• Securityfactorsweretakenintoconsideration:oneLGAinBauchiwasswappedatlatenoticeinresponsetorumoursregardingBokoHaramactivity.
Basedontheaboveprocess,twoIQSswereselected(firstandsecondchoice)foreachofthesixLGAsinNigerandBauchi(12intotal).ThecategorisationofthesampledIQSswasdiscussedwiththestateGEP3teamandresponsiblegovernmentofficials,whovalidatedqualitativelythecategorisationoftheIQSsinhigherperforming,typicalandlowerperforming.Forbothstates,theteamsmanagedtovisittwooftheirfirstchoiceIQSs,while,duetounresponsivenessorunavailability,theyhadtovisitonesecondchoiceIQS.Noadditionaladjustmentsweremadetothesample.
AteachIQSparents,pupils(girlsandboys)andCBMCmemberswereinvitedtoparticipateinFGDs.TheselectionwascarriedoutwiththesupportoftheIQSproprietororMallam,whichwasimportantinordertoestablishagoodrapportforthequalitativeresearch,butmayhavebroughtaboutselectionbias.InadditiontotheIQSs,thequalitativeresearchalsoundertookKIIswithLGAIQSofficers.ThesewereselectedtomatchtheLGAsofthesampledschools.
4.2.4.4 Qualitativeinstruments
Thequalitativeresearchprimarilymakesuseoffourresearchtechniquesorinstruments:aqualitativeclassroomobservation(QCO),anunstructuredteacherpracticediscussions(TPDs),KIIsandFGDs.
EvaluationoftheGirl’sEducationProject3
©EDOREN 120
Table24: Instrumentsbyparticipantgroup
InstrumentProp
rietor/Owne
r/M
allams
Headteache
rs
Facilitator/Teacher
Pupils
CBMCmem
bers
Parents
Commun
itylead
er
Localgov
ernm
entIQS
officer
QCO �
TPDs �
KIIs � � � �
FGD � � �
ThetoolsfortheKIIsandFGDsweredevelopedinordertocaptureinformationonthecoreareastoprobe,asoutlinedinthequalitativeevaluationmatrix,whicharederivedfromtheunderlyingassumptionsintheToC.
Thestrengthsofcombiningquantitativeandqualitativemethodsinevaluationcanalsoapplyattheclassroomlevel,intheformofclassroomobservations.Theclassroomobservationsundertakenaspartofthequantitativesurveyprovidedataonthedegreetowhichteacherpracticesarechanging,whiletheuseofQCOscombinedwiththeTPDsaremeanttoanswerthe‘how’and‘why’questions(FasseandKolodner,2000;1993;Merriam,1988;Yin,1984).TheQCOissemi-structuredandwasconductedbyaneducationalist.Theinstrumentlistssomeofthecoreareasinwhichtheprogrammeaimstochangeteacherbehaviourandpedagogyandtheresearchertookdetailednotesundereachsectionduringthecourseoftheclassroomobservation.Aftertheobservationhadtakenplace,theresearcherundertooktheTPDwiththeteacher.Thediscussionfollowedthestructureofthelessonobservationbutotherwiseremainedunstructured.Thediscussionfocusedonprobingthepedagogicalapproachoftheteacherandengagingtheteacherinadiscussionabouthowandwhytheteacheremploysspecificmethodswithintheclassroom.Specificattentionwasgiventounderstandinghowandwhytheteacherimplementsgender-sensitiveclassroompractices.AtmidlinethiswillincludereflectingontheextenttowhichthetrainingprovidedthroughGEP3haschangedtheteacher’spraxis125andwillprovidedataonalternativeexplanations.
UNICEFstatesthat‘“Girlfriendly”schoolsareschoolsthat[girls]wanttoattend…[they]providechildren–girlsinparticular–withasafe,nurturingandgender-sensitivelearningenvironment’.126Parents,pupilsandrepresentativesfromtheCBMCprovidedivergentpointsofviewregardingtheextenttowhichgirlswanttoattendschool,andwhy(orwhynot).Thecasestudiesseektoconstructastrongstoryregardingthelearningenvironmentandtoidentifyfactorsthatareperceivedtocontributetochanges.
DiscussionguideswereusedfortheFGDs,tailoredtowardtheareasthatweretobeprobed,andmakinguseofage-appropriatelanguage.Semi-structuredinterviewguideswereused.Thesewere125Praxisreferstothewaysinwhichindividualsengagewithandreflectonasituation,asboththinkersandactors.Praxisincorporatesvariouscomponents– includingthetheoretical (guidingrules), theproductive (applicationofguidingrules),andthepractical(howtoproduceundertheguidingrules). 126www.unicef.org/mdg/mali_59595.html.
EvaluationoftheGirl’sEducationProject3
©EDOREN 121
organisedaroundthecoreareastoprobe,thusensuringadegreeofstandardisationwhileatthesametimeallowingthenationalandEDORENqualitativeresearchersenoughflexibilitytopickuponinterestingthemesandemergingtopicsandconcerns.
KIsarepeoplewithin-depthknowledgeandunderstandingofparticularsubjects.Inthecaseofthisresearch,theKIsaretheproprietor/owner/MallamsoftheIQS,headteachers,communityleadersandthegovernmentIQSofficerattheLGAlevel.127WhilethechoiceofKIswaslargelystandardised,thedesignwasflexibleenoughtoalsoincorporateotherKIsshouldtheneedhaveariseninthefield.ThiswasparticularlyimportantfortheIQSSasthestructureswithinIQSsarenotsetandstandardisedlikeinpublicprimaryschools.Forexample,theinterviewwiththeLGAIQSofficerwasaddedafterthestate-levelstakeholderinterviewshadshowntheimportancethatstakeholdersattachedtomonitoringandsupervisionoftheeffectivenessoftheIQSS.
4.2.4.5 Fieldwork
Implementationandmanagementofthefieldwork
ThefieldworkwasimplementedbyEDORENbetween03and14November2015.Itwasprecededbyafour-daytrainingperiodforresearchers(seebelow).
ThequalitativeresearcherswererecruitedbyEDOREN,andinitially11memberswereinvitedfortrainingfromapoolofhighlyexperiencedcandidates.Selectionwasmainlybasedoncandidates’CVsandtheirexperiencewithotherqualitativeeducationresearchinnorthernNigeria.128Theselectionprocessalsotookintoconsiderationthegenderbalanceoftheteams,andespeciallytheirlocallanguageproficiency.AllmemberswereproficientinbothspokenandreadHausa,whichwasthemainlanguageusedforinterviews.OtherlocallanguagerequirementswerealsotakenintoconsiderationwhenselectingteammembersasinsomeareasHausaisnotthemainlanguagespoken.IntheselectionofinterviewersspecialattentionwasgiventoArabic,Fulfude,NupeandGbagyilanguageproficiencysincetheselanguagesarespokeninthestudyarea.
Twoteamswereestablishedforthefieldwork,eachteamcoveringonestate.Onewasledbyanexternalconsultant(anexperteducationist),whiletheotherwasjoint-ledbyaneducationistandaqualitativeresearchspecialist.Thesestateteamleadssupervisedallinterviews,carriedouttheQCOs,oversawthedatatranscriptionandensuredqualityassuranceateachstageoftheresearch.
Eachteamleadselectedfourresearchersfromtheoverall11memberstrainedtoholdthepositionsofleadinterviewer,focusgroupfacilitator,note-takerandtranslator.Thisselectionwasdonehalf-waythroughtraining,andwasbasedoneachindividual’sengagementandexpertise,suitabilityforeachofthefourpositions,thegenderbalanceoftheteamsandspecificlanguagerequirements.
Training
Toensurethatallsurveyteamscoulddotheirworkeffectivelyandinaccordancewithhighqualitystandards,allqualitativeresearcherstrainedintensivelyfrom31Octoberto02November2015inAbuja.Theinstrumentswerepilotedattheendofenumeratortraining.
127TheseIQSofficerswereLGArepresentativesfortheSAME.128Qualitativeresearcherswereinvolved,amongothers,indatacollectionandanalysisfortheEDORENThematicResearchonTeacherManagement,andonaGEP3assessmentoftheFTTSS.
EvaluationoftheGirl’sEducationProject3
©EDOREN 122
Thisexercisefirstlyincludedanumberofinteractiveexercisestodeepenresearchers’understandingoftheGEP3IQSintervention,theToCandtheevaluationframework.Next,thetrainingmovedtoprudentprocessesforqualitativeresearch,includingdatacaptureandtranslationprotocols,transcriptionproceduresandresearchethics.
Thefinaltwodayswerespentextensivelyengagingwitheachoftheeightinstruments.Thiswasdoneinorderforresearcherstopracticeessentialinterviewingtechniques,suchasprobing,whileconcurrentlyrefiningtheflowandphrasingofeachoftheinstruments.Thetrainingusedseveraldifferenttrainingapproachestoachievethemaximumtrainingeffect,suchassmallgroupwork,mockinterviewsandgrouppresentationsoffindingsandreflections.
Fieldworkorganisation
Foreachofthetwostates,theresearchteamcomprisedof:
• astateteamleader:ashasbeensaid,thestateteamleadssupervisedallinterviews,carriedouttheQCOs,oversawthedatatranscriptionandensuredqualityassuranceateachstageoftheresearch.Oneoftheteamswasjoint-ledbyaneducationistandaqualitativeresearchspecialist;
• oneLeadInterviewer,responsibleforconductingtheKIIs;• onefocusgroupfacilitator,responsibleforconductingFGDswithCBMCs,pupilsandparents;• onetranslator,responsibleforaidingtheteamlead/sinconductingtheQCOsandTPDs;and• onenote-taker,responsiblefortakingnotesduringFGDsand(timepermitting)KIIs,aswellas
lookingafteraudio-andvideo-recordingequipment.
Atthebeginningofthefieldworkeachmemberwasgivenexplicitguidanceandtrainingontheirparticulartask.Forthetranslator,forinstance,thismeantrepeatedexercisesontwo-waysimultaneoustranslation.Fortheleadinterviewerandfocusgroupfacilitator,thismeantthestateteamleaderofferedguidanceonelementssuchasprobingorcreatingasafespacetoinciteresponses.
Whilemembersprimarilyhadresponsibilityfortheirdesignatedrole,inpracticetheyalsotookonadditionalresponsibilities.Forinstance,duetothelargeamountofinterviewstakingplace,allmembersweremaderesponsibleforapartofthetranscription,whichwasthenqualityreviewedbythedesignatednote-taker.Similarly,whenmultipleinterviewstookplaceatthesametime,othermemberscouldstepintofunctionastranslator.However,allKIIswereperformedbytheleadinterviewerorstateteamlead,whileallFGDswereperformedbytheFGDfacilitatororstateteamlead.
Fieldworkchallenges
Someofthemostpertinentchallengesfacedduringfieldworkarelistedbelow:
• TeachinghoursatIQSs:IQSsusuallydonotfollowtheirtimetablesstrictly.Inaddition,someclassesaretaughtearlyinthemorningandsomelateatnight.Furthermore,mostoftheIQSsoperateforjusttwohours.Thisrequiredteamstomakedetailedenquiresregardingtheday’splan,andtosetoutveryearlyorlate,oftenadministratingtheheadteacher,teacherandCBMCinterviewtoolsbeforethearrivalofthepupils.
EvaluationoftheGirl’sEducationProject3
©EDOREN 123
• Holiday:SomeIQSswerenotinsession,althoughtherewasnomentionofthisintheirtimetables.ThismeantthatteachersandpupilshadtocometoschoolinanarrangedmannerinordertocarryouttheKIIs,FGDsandclassroomobservations.
• Distanceanddifficultterrain:ManyIQSsareremotelylocated,withoutgoodroadaccess,whichrequiredlongtraveltimestoreachthem.Thismadeitmoredifficultfortheteamstocompletealltheinstrumentswithinschoolsandtoreturntotheirbasebeforedarknessfell.Combinedwiththedifficultteachinghours,thismeantattimesthattheclassroomobservationshadtoberescheduledtoanunusualtime,orthattheresearchershadtocomebackforseveraldays,tocompleteallinstruments.
• Pre-appointment:SchedulingappointmentswiththeIQSswithoutaphysicalpresencewasverydifficult,asmanyIQSslieoutsideofthereachofmobilecommunicationnetworks.
4.2.4.6 Qualityassurance
Inordertobestensuretherigourofthequalitativeresearchthestateteamleadsprovidedqualityassuranceateachstageoftheresearch.Thiswasfirstensuredatthetrainingstagebycarefullypreparingeachofthepotentialteammembersonqualitativeresearchmethodsandbyselectingonlythemostappropriateindividualsforeachoftheteamroles.Thetrainingstressedunbiaseddeliveryofinstruments,andemphasisedbuildingteammembers’capacityandunderstandingofthetoolsandstudyasawhole.
Rigourandmitigationofbiasinthequalitativefieldworkwasachievedthroughinvolvementofdifferentindividualsinthefieldteams,allowingteammemberstosupporteachother,discussandquestionfindings.Teamleadswerepresentatalltimesduringfieldwork,tooffersupportandaddresstechnicalqueriesinthefield.TeamleadsalsosatinonthemajorityofinterviewsandFGDs,timepermitting,toprovideadditionalguidanceandprobingwherenecessary.Additionalqualityassurancewasachievedthroughdailyteambriefsanddebriefs.Inthedebriefstheteamwoulddiscusskeyfindingsfromtheday,linkfindingstotheevaluationmatrix(whilstnotingfindingsthatweredivergentfromthematrix)andcomparefieldobservationalnotestohelpsituateanalysis.Thedebriefingsalsoallowedtheteamtoaddresslogisticalandtechnicalqueriesthatmayhavearisenduringtheday.AstheteamreturnedtoeachIQSformorethanonedayitwaspossibletofurtherexploreontheseconddayqueriesthataroseoutofthefindingsfromdayone.Briefingswereheldeachmorningbeforetheteamsetout,afterhavingreviewedfindingsandtranscriptsfromthedaybefore,toidentifyquestionsneedingfurtherexploration,andgivetechnicalpointersforthedayahead.
Qualityassuranceoftranscriptionwasensuredbyallowinganinitialtranscripttobewrittenupbyoneindividualandthenverifiedinpartsbyasecondindividual.129Thisback-translationwasusedtoensurestandardisationoftranslationfromHausatoEnglish.Anydiscrepancieswerediscussedamongstteammembers.Transcriptionandback-translationtookplaceduringthefieldwork,andsoalsoprovidedimmediatelessonsintermsoffacilitatinginterviewingortheneedforadditionalprobing.Suchlessonswerethendiscussedduringthedailybriefinganddebriefingmeetings.Theteamleadalsoreviewedallcompletedtranscripts,togiveguidance(particularlypriortogoingtothenextschool),thusallowingforreflexivityinthefield,andstrengtheningrigour.Teamsalsokeptrecordsoftheiractivities,andmadeadditionalnoteswherepossible,sothattheycouldbelinkedtotranscriptsandanalysis.Uponreturningfromthefieldtheteamleadreviewedtranscripts,compared129Thismeantthatoneresearcherwouldtranslate/transcribeawholetranscript.Asecondresearcherwouldthenlistentopartsoftheaudiofromthattranscriptandtranslate/transcribethatpart.Thetwopartswerethencompared.
EvaluationoftheGirl’sEducationProject3
©EDOREN 124
themtofieldnotesandconsultedwiththeprimarytranslatorincaseswheretherewasanyconfusion,toclarifythattranscriptionwascomplete.Thefactthatthetranscriptionswerecarriedoutverbatimhelpedmitigatebiasatthestageofdatacapturing.
4.2.4.7 Analyticalapproach
Theanalyticalapproachtothequalitativedatausedappliedthematicanalysis.Theselectedprincipalapproach—‘confirmatoryanalysis’—aimstoconfirmasetofpre-existinghypotheses(assessingthecontributionclaim)andgeneratecodesfromthehypothesestobeappliedtothedata.Thisisincontrasttoexploratoryanalysis,whichderiveshypothesesfromthedatacollected(Guestetal.,2012).Atbaseline,exploratoryanalysiswasusedasaninitialanalyticaltechnique,toensurethatthequalitativecomponentisresponsivetonewinformation(challengestothecontributionclaim).AsitwasimportanttotesttheassumptionsoftheToC,exploratoryanalysiswasusedtofirstgroupthedata,afterwhich,confirmatoryanalysiswasusedtovalidatethisinformation.
Appliedthematicanalysisrequiresresearcherstointerpretdataanddoesnotrelyoncountingwordsorphrases,butratheridentifiesanddescribesimplicitandexplicitideasthatareorganisedintothemes.Tostrengthentherigourofthematicanalysisandmitigateresearcherbiastheanalysiswascarriedoutinseveralstages.First,familiarisationwiththedatawasachievedthroughdailydebriefsinthefieldandadayinAbujawithbothstateteams,consistingofpresentationsanddiscussionsoftheinitialfindingsandkeythemes.Second,onerandomlysampledschoolfromeachstatewasselectedandusedforinitialinductivebrush-coding.130ThiswasdoneusingNvivosoftware.Alltranscriptswerecodeddirectlybasedonwhatarosefromthedata,andtheinitialcodingstructurewasdiscussedamongstthethreeteamleadstomitigatesingle-researcherbias.Theremainingtranscriptswerethereaftercodedintothisstructure,whilstallowingfornewcodestoemergeduringtheanalysis.Third,theinitialbrush-codeswereanalysedandsynthesisedintothematicnodes.Attimesthismeantgroupingnodestogether,andattimesitmeantre-codingbasedonthisinitialanalysis.
Thesetofhypothesesinthequalitativeevaluationmatrixprovidedaninitialsetofthemestobeusedtodevelopthethematicnodes.Eachpieceofdatawasconsideredinlightofthecontextfromwhichitcame(forinstance,theknowledgethatthepersoncitedislikelytohaveaboutthesubject,theincentivestheymayhavetorespondinparticularwaysandcorroborationbyotherqualitativesources).Thestrengthofthematicnodeswasfurtherdeterminedbasedonthenumberofreferencesandsourcesmakingupthenode,toensuretriangulationoffindings.Asaverbatimtranscriptionwasmadefindingswerealsoanalysedinlightofwhatandhowquestionshadbeenasked.Researchersthenassessedthebalanceofthesegroupsandwhethertheconclusionssupportedtheinitialhypotheses.FindingswereconsideredbothwithIQSsasseparatecases,andthroughcommonthemesacrossthesixIQSs.Theanalysiswasundertakenbythesameresearcherswhoconductedthefieldwork,toensurethaterrorsofinterpretationwereminimised,andfindingswerediscussedamongsttheteamateachstageofanalysis.
130Inductivebrush-codingreferstodevelopinginitialcodesastheyarisefromthedata,regardlessofwhetheritsitswithintheevaluationmatrix.Thisisdonetoensurethatanalysisisdrawnfromthedatacollected,andnotlimitedorbiasedbasedonwhatwasexpectedtobefoundpriortofieldwork.
EvaluationoftheGirl’sEducationProject3
©EDOREN 125
4.2.5 Ethicsandinclusion
TheethicsproceduresforthequalitativecomponentoftheIQSSevaluationweredescribedinthediscussionoftheethicsapprovalprocessgiveninSection3.2.8.On07October2015anoticeofResearchExemptionwasprovidedtoEDOREN.TheExemptionLetterisprovidedinAnnexE.Inaddition,afterthefinalisedtoolswereprovidedtotheOPMEthicCommittee,ethicalapprovalwasgivenon28September2015(seeAnnexE).
4.2.6 Outcomemeasurementandconstructs
Thesamemeasurementandtoolsforpupillearningoutcomes,teachermotivationandteacherknowledgeandskillsareusedintheIQSSevaluationasareusedintheearlylearninginterventionevaluation.AdescriptionofthesemeasurementsandconstructsisgiveninSection3.2.9.
4.2.7 Limitationsandriskstothemethodology
Differentlimitationsandrisksintheabove-presentedmethodologyneedtobetakenintoaccount.Someofthesecanbeaddressedbycarefulimplementationoftheevaluation,whileothersconstituterealriskstothemethodology.Table25reviewsthelimitationsandrisks.
Table25: LimitationsofandriskstotheevaluationmethodologyfortheIQSS
Limitations/risks Whythisislimitingandwhatwehavedonetoaddressthis
PlausibilityoftheToCanddepthofanalysis
Theevaluationdesigndrawscausalinferencesbyverifyingtheintervention’sToCagainstevidenceandaccountingforalternativeexplanations.ThevalidityofthisdesigndependsonaplausibleToC,theunderstandingofthecausalmechanismsatplay,andtheidentificationandverificationofalternativeinfluencingfactors.WithinthecomplexanddiverseenvironmentofreligiouseducationinnorthernNigeriathisischallenging,particularlywhencarryingouttheevaluationundertemporalandresourceconstraints.TotakeonsuchanambitioustaskwehaveconductedathoroughexplicationoftheToCoftheGEP3IQSS(seesection4.3).Furthermore,weconcentratetheevaluationinthefirstinstanceontheinterrogationofhowGEP3hascontributedtotheimprovementofteachingandlearning,whichallowsamorefocusedanalysistobeundertaken.
Neteffectmeasurement
Whilepre-andpost-measurementofoutcomesduringbaselineandfollow-upsurveysallowsthequantificationofchangesininterventionoutcomes,themethodologydoesnotforeseethemeasurementofthenetattributableeffectofGEP3’sIQSSsincechangeisnotmeasuredinacomparisongroup.Judgementaboutcausalimpactwill,rather,bebasedonareasoned,crediblestoryofhowIQSShascontributedtothechangeinoutcomevariables.
TheresultsofthequantitativesurveywillberepresentativeofthetargetpopulationsintheGEP3pilotschoolsinBauchiandNiger
ThestudyuniverseofthequantitativesurveyconsistsoftheGEP3pilotIQSsinBauchiandNiger.TheseIQSsarenotthemselvesrepresentativeoftheentireQur’anicschoolpopulationintheGEP3states.ThisisbecauseGEP3pilotIQSshavebeenselectedbasedoncertaincharacteristicsandtheGEP3LGAsmaynotfullyrepresentallLGAsinthestate.Itisimportanttobearthisinmindinregardtotheexternalvalidityofthefindings.Dueattentionispaidtounderstandingthecontextandinfluencingfactorsthatcontributetotheoutcomesobserved.ThisallowsustoassesswhetherthefindingscanbereliablygeneralisedtothestateasawholeandtoawidergroupofIQSs.
EvaluationoftheGirl’sEducationProject3
©EDOREN 126
Resourcesforendlinesurveystilltobedecided
Itisplannedtomeasurechangesinlearningoutcomesbycomparingthebaselineandendlinesituation.Atthispointtheevaluationresourcesforthe2017–2020periodhavenotyetbeendetermined.Thisposesariskfortheimplementationoftheendlinesurvey.However,thereisagreementbetweenthemainstakeholdersthatanendlineevaluationofIQSSwilltakeplace.
Compositionofcross-sectionofIQSpupilschangesbetweenbaselineandfollow-upsurveys
TheIQSSmayaffectthecharacteristicsofthepupilpopulationintheIQSsstudied,andthereforebaselineandfollow-upsurveysmayrepresentdifferentpupilpopulationsintermsofunderlyingcharacteristics.Totheextentthatsuchachangingcompositionaffectslearningoutcomes,itmayprovideanalternativeexplanationforchangesinlearningoutcomes.Wewillassessthishypothesis,butitwillnotbepossibletoquantifytheeffctofchangesinthecompositionofthepupilbodyatIQSsonlearningoutcomes.
Attritionofthesample
BothquantitativeandqualitativeresearcharebasedonalongitudinalsampleofIQSs.ItispossiblethatsampledIQSswillcloseorwillrefusetobepartoftheevaluationafterbaseline,whichwouldresultinalossofdata.WehadinitiallyassumedthattheIQSssupportedbyGEP3hadbeenselectedbasedonthedegreeoftheirintegrationandconnectionwiththeformaleducationsystem.ThebaselinedatacollectionhasdemonstratedthatGEP3IQSsarenotnecessarilywellintegratedandintegrationisnotalwaysmaintained.Theriskofattritionisthereforehigherthanwasinitiallyexpected.Nonetheless,non-integratedorrecentlyintegratedGEP3IQSswereexcludedfromthesampleframeduringbaseline.WecanthereforeassumethatthesampledcohortofIQSsarerelativelystableschools.Sincethequantitativesurveyfollowsacross-sectionalapproachthereisnoriskofpupilattrition.Teacherandheadteacherattritionispossibleandthusteacherattritionistakenintoaccountinthesamplesizecalculation.
Surveyresponseachievement
Asdiscussedabove,thesurveyteamsucceededinsurveyingahighpercentageoftheintendedschools,althoughaconsiderablenumberofreplacementshadtobesurveyed.Thepercentageofpupilsandteacherssurveyedwasaround80%,whichislowerthanexpected.Thiswasnotduetoalowresponseratebutratherbecausetheschoolsinvolveddidnothavemoreeligiblepupilsorteachers.Thelowerthanexpectedpupilsamplesizemayaffecttheprecisionofthemeasurementofchangesinlearningoutcomes.However,asthedegreeofintegrationincreasesovertime,theintendedsamplesizeofpupilsismorelikelytobeachievedatendline(giventheuseofacross-sectionalsample).Inthecaseofteachers,whichfollowsapanelapproach,weestimatethatlessprecisionwillnotjeopardiseavalidjudgementofchangesinceitisexpectedthattheincreaseinteachercompetencieswillberelativelylarge(a30%increaseaccordingtotheGEP3logframe).Ingeneral,thesubstantialsmallerschoolsamplesizecomparedtotheearlylearningevaluationlimitstheuseofregressionmodellingtoexaminerelationshipsbetweenvariables,andthereforethequantitativeanalysiswillbebasedmoreondescriptiveanalysis,whichwillbecomplementedwiththequalitativefindings.
Flooreffectsinmeasurement
Highlevelsofmissingdatawereobservedintheteacherassessments.Theamountofmissingdataisamajorconcernbecauseitmayindicatethatmanyoftheteacherswhoparticipatedintheprogrammewerenotabletocarryoutmanyofthetasksaskedofthem.Thesetaskswereintendedtorepresentthecorebasicelementsofwhatteachersshouldknowandbeabletodoinordertoteacheffectivelyataminimumlevel,sotheconcernisthattheteachersmaynotbeveryskilled.Whilethissuggeststhatthetestwas‘toodifficult’,thetestwasdesignedaroundnotionsofwhataminimallycompetentteachershouldbeabletoknowanddo.Itisprobably,therefore,unhelpfultobeoverlyconcernedaboutthetest’sdifficulty.Therealissueisthecompetencyleveloftheteachers.
EvaluationoftheGirl’sEducationProject3
©EDOREN 127
4.3 PresentationoftheupdatedToC
TheinterventionToChasanimportantmethodologicalroleintheevaluationofGEP3’sIQSS.Whilecausalinferenceinthecaseoftheevaluationoftheearlylearninginterventionisderivedfromcounterfactualcomparisonusingacontrolgroup,contributionanalysisinferscausalityfroma
However,theseflooreffectsdolimittherangeofanalysesabletobeundertakenandlimittheevaluationteam’scapacitytodrawempiricallystrongconclusions.
Socialdesirabilitybias
Anumberofitemsregardingattitudestowardsgirls’educationwereincludedintheteacherquestionnaire.Whilethereliabilityoftheresultswasacceptable,theveryhighproportionsofteachersrespondingpositivelytostatementsregardingtheimportanceofgirls’educationrevealedacomplianceeffect.Thedistributionofresponsesacrossthestronglydisagree,disagree,agreeandstronglyagreeLikertscaleindicatesthatteachersareconformingtoperceivedcorrectresponses,creatingasocialdesirabilitybiasinthemeasurement.Giventhesecomplianceeffects,thedataonteachers’attitudestowardsgirls’educationwerenotabletobeusedintheanalysis.Thislimitationhighlightstheimportanceofobservationdataandqualitativemethodstoprobesocialnormsandgenderattitudes,particularlyinanevaluationofaprogrammedescribedasaninterventiontargetedatgirls.
Biasinsamplingofthequantitativesurvey
Aspartofthesamplingprocessbiasesmayoccurwhenunitsareselectedthatdonotformpartofthetargetpopulation,orwhensomeunitsofthetargetpopulationarelesslikelytobeselectedthanothers.Insomeschoolsstakeholderstriedtoinfluencepupilsamplingorschooleligibility.ThiswasparticularlyariskinIQSs,giventheirnon-standardorganisationandthepoorqualityofthesampleframe.Totheextentpossiblethisriskwasmitigatedthroughthedefinitionandimplementationofrigorousandsystematicsamplingproceduresunderclosesupervision.
Inferencebeyondtheselectedqualitativecasestudysitesislimited
Whilethecontributionwillbeexaminedatmultiplelevels,thefindingsofthequalitativeresearchwillreflecttheparticularcommunitiesandschoolsselected.Incontrasttothequantitativedesign,thequalitativeresearchdoesnotaimtofacilitatesample-to-populationinference,butrathertogatherin-depthunderstandingofhowtheIQSSToCunfoldswithinaspecificcontext.Purposefulsamplinghasbeencarefullyconsideredtoensurein-depthlearningaboutthephenomenaofintegrationofformalsubjectsinQur’anicschools.Nevertheless,thereremainsariskthatthelearningwillbeaffectedbythechoiceofcommunitiesandschools.
Giventhenon-representativenatureofthequalitativeselectionofcommunitiesandschoolstheinformationprovidedisindicative
Thequalitativecomponentoftheevaluationoffersnuancedfirst-personaccountsofpeople’sperspectivesandexperiencesoftheactivities,withoutclaimingthattheseaccountsarerepresentativeofothersimilarcommunities’andschools’experiences.However,whenconsideredtogetherwiththerepresentativequantitativeresults,thequalitativefindingsprovideinterestingperspectivesonunderlyingissuesandfactorsthatcandeterminethecontributionofGEP3’sIQSStoimprovinglearningoutcomes,especiallyforgirls.
Thenumberofcasestudysitesislimitedduetoresourceconstraintsfortheevaluation
InordertoassesshowGEP3’sIQSScontributestoimprovedlearningoutcomes,andhowthiscontributionmaydifferacrossheterogeneouscontexts,casestudysitesthatcollectivelyaccountforarangeofcontextsshouldbeincluded.Giventheresourceconstraintsoftheevaluationonlysixsuchsiteshavebeenincluded.Ideally,nineor12casestudysitesacrosstwostateswouldprovideagreaterdegreeofgeneralisabilityasthedatageneratedwouldbebasedonagreaternumberofpossiblecontextualsituations.
EvaluationoftheGirl’sEducationProject3
©EDOREN 128
reasonedToC.Therefore,inthissectionwepresentanin-depthdiscussionoftheToCoftheIQSS,startingfromtheToCincludedintheGEP3EvaluationFrameworkandupdatedbasedoninsightsfromthebaselinedatacollection.SincetheoverallToCresultchainpresentedintheEvaluationFrameworkwaswell-founded,littleupdatinginthisregardhasbeenrequired.Thereviewmostlyfocusesonfurtherexplainingthemechanismandassumptionsunderpinningtheresultchain.
EvaluationoftheGirl’sEducationProject3
©EDOREN 129
Figure41visualisestheToCofGEP3’sIQSS.InthefollowingsubsectionswewilldiscusstheToCaccordingtothreecausalpackagesthatmakeupthisToC.Eachcausalpackageconsistsofcausalfactorsinterlinkedviacausalmechanisms,plustheirunderlyingassumptions.Acausalpackageisassumedtobesufficientforgeneratingkeyintermediaryorfinaloutcomesontheconditionthatthecausalchainandassumptionshold.ForeachcausalpackagerelatedtoGEP3’sIQSStheremaybealternative,non-GEP3relatedexplanationsfortheoutcomeoccurring,whichwillrequireexaminationaspartoftheevaluation.
Eachcausalpackagepresentedbelowcorrespondstothecontributionclaimspresentedinthemethodologicalsection.131
131(Mayne,2012).Thecausalpackageisdefinedasbeingsufficienttocauseanoutcome,whileitsindividualelements,inparticulartheinterventionelements,arenecessarypartsforthepackagetomakeadifferencetotheoutcome.
EvaluationoftheGirl’sEducationProject3
©EDOREN 130
Figure41: DiagramdepictingToCoftheIQSS
4.3.1 GEP3’sIQSScontributestomoreeffectiveteachingofformalsubjectsinIQSs
4.3.1.1. Causallinkagesandmechanisms
GEP3’sstrategyforintegratedQur’aniceducationaimstoprovideaccesstoqualityeducationinIQSs,particularlyforgirls,byimprovingeffective,gender-sensitiveteachingoftheformalsubjectsincludedintheharmonisedintegratedcurriculum.MoreeffectiveteachingisconsideredakeyintermediaryoutcomeoftheIQSS.Weconceptualiseeffectiveteachingasacombinationofthefollowingattributes:1)demonstratedteachingcompetency,drawingonthreetypesofknowledge(subjectknowledge,pedagogicalknowledgeandcurriculumknowledge);2)useofapupil-centredlearningapproach;3)timeontask;4)effectiveuseofteachingandlearningmaterials;and5)noobservablegenderbiasduringteachingpractice.
ThecorecausalpathwaytoachievemoreeffectiveteachingisbyimprovingIQSfacilitators’knowledgeandskillsthroughtrainingandmentoring.Thetrainingismeanttoimprovetheirknowledgeandskillsintermsofthecoresubjectsoftheharmonisedintegratedcurriculum,pedagogy,andtheintegrationprocessinQur’aniceducation.Thetrainingisalsomeanttoimpartabetterunderstandingofgender,equityandgender-sensitiveteachingmethods.Thementoring
EvaluationoftheGirl’sEducationProject3
©EDOREN 131
processisassumedtoplayanimportantroleinfacilitatorseffectivelyacquiringthisknowledgeandskills,andputtingitintopractice,byactivatingthefollowingmechanisms:
• peerinteractionthatencouragesfacilitatorstodiscussacquiredknowledgeandskills,shareexperiencesandprovidefeedbackbasedonacollaborativerelationshipamongpeers;
• school-basedsupportthatallowsfacilitatorstoaccessandrefreshtheirknowledgeandunderstandingatanytimewhilstusingtheprovidedmaterials;and
• continuousface-to-face,practicalsupportbyanexperiencedmentorbasedonalong-term,personalrelationshipthatreinforcesthelearninganddeliveryofnewlyacquiredknowledgeandskillswithconfidence.
Mentoringisalsoassumedtoincreaseteachermotivation.Whileincreasedteachermotivationisnotadirectintendedobjectiveoftheintervention,itisassumedtobeakeycontributingfactortomoreeffectiveteaching.Theunderlyingtheoryisthatpeerinteractionandongoingsupportincreasethefacilitators’perceivedteachingefficacy,self-confidenceandself-esteem,whichresultsinincreasedmotivation,asexpressedin(i)moreinterestandenjoymentintheirwork,(ii)moreeffortbeingmade,(iii)moreimportanceattachedtotheirwork,and(iv)lesspressureandtensionexperiencedinrelationtotheirwork.132Thisinturnsenhancesthelikelihoodoffacilitatorsactuallyapplyingtheirimprovedknowledgeandskills,andspendingincreasedtimeontaskonactivitiesthatbenefitstudentlearning.
ThedistributionofHausateachingandlearningmaterialscontributestomoreeffectiveteachingasmaterialsprovideguidancefortheteacherduringtheteachingprocessandfacilitatetheknowledgetransferbetweenteachersandpupils.Thisrequiresteacherstohavethepedagogicalknowledgeandskillstoeffectivelyusethematerialsduringteachingpractice.Pupilworkbooksandnotebooksalsoallowpupilstopracticeandallowteacherstoassesspupilsinwriting.TheprovisionoflearningmaterialsmayalsoensuretheinclusivenessofintegratedQur’aniceducation.However,itmustbenotedthatpovertyhasbeenidentifiedasoneofthekeybarrierstoaccessingpublicprimaryschoolandtotheextentthattheintegrationprocessforcesparentstoinvestinlearningmaterials,comparedtothesituationwithoutintegration,parents’inabilitytodosomaycausepupilstodropout.
Headteachertrainingseekstosupportamoreeffectiveteachingprocessinseveralways.First,thetrainingisassumedtoincreasepedagogicalleadershipintheIQSbyprovidingtheheadteacherwithknowledgeandskillsintermsofpedagogyandleadership.This,inturn,willprovidethefacilitatorswithschool-based,continuousguidanceinhowtobetterdeliveryinstruction.Second,leadershipcaninspireandmotivatethefacilitator,whichcontributestoimprovedattendanceanddemonstratesitselfinincreasedeffortandenjoymentofwork.Third,thetrainingalsoaimstoimprovetheheadteacher’sschoolmanagementpractices,suchasteacherattendancemonitoringandclassscheduling,whichinturncancontributetoincreasedfacilitatorattendanceandtimeontask.Improvedschoolmanagementcanalsoincreasetheresourcesavailableforfacilitatorremuneration,whichcanaffecttheirattendance,andcansubsequentlycontributetomoreeffectiveteaching.
Animproved,moregirl-friendlyschoolenvironmentandimprovedmonitoringandsupportsupervisionbylocalgovernmenteducationstaffisassumedtocontributetomoreeffectiveteaching.Abetterschoolenvironment,intermsofinfrastructure,classroomconditions,safetyandrelationshipsbetweendifferentactorsintheschool,canimproveteachers’andpupils’comfortand
132ThistheoryisbasedonthetheoryofmotivationusedbytheTDPinNigeria.
EvaluationoftheGirl’sEducationProject3
©EDOREN 132
feelingofwellbeingintheschool.Itcanalsoleadtoincreasedinstructionaltimebycreatingthephysicalconditionsrequiredforteaching(e.g.light).Girl-friendlyschoolfeatures,suchasthepresenceoffemaleteachersorfemaleparticipationinschoolmanagement,cancontributetoanincreaseingender-sensitiveteachingpracticesbycreatingaclimatethatismoresupportiveofgirls.Improvedgovernmentmonitoringandsupportsupervisioncancontributetobetterschoolmanagementandcanmotivatefacilitatorsandproprietorstocontinueprovidingtheintegratedcurriculumasitsignalsgovernmentsupportforintegration.Furthermore,sustainedfinancialgovernmentsupportcansupportfacilitatorremuneration,whichwillcontributetofacilitators’motivationandattendance.
Whilemoreeffectiveteachingisassumedtocontributetoimprovedlearningoutcomes,improvementinlearningoutcomescaninturncontributetoanincreaseinfacilitators’perceivedteachingefficacy,whichisexpectedtostrengthentheirmotivation.Anincreaseingirls’enrolmentandretentionmayalsoaffectteachingpractices:first,itmaycontributepositivelybycreatingamoregirl-friendlyschoolenvironment,whichinturncansupportgender-sensitiveteachingpractices;second,itmayhaveanegativeeffectbyincreasingpupil–classroomandpupil–teacherratios.
4.3.1.2. Assumptions
Assumptionsaredefinedhereasinfluencingconditionsthatmightenablecausallinkagesandmechanismstowork,ormayimpedethoselinkagesandmechanism.KeyassumptionsunderlyingthecausalchainofContributionClaim1arepresentedinFigure42.
Implementationassumptionsarenotincludedinthediagram.Itisassumedthatthetrainingandmentoringareimplementedasplannedandcoverallthecomponentsofthetrainingcurriculuminaqualitymanner,andthattheteachingandlearningmaterialsaredistributed.Theseareimportantassumptions,inparticularwithregardstothementoringsinceGEP3’scapacitydevelopmentapproachforfacilitatorsstronglyemphasisescontinuous,school-basedfollow-upbyexperiencedmentorscomplementedwithpeerinteraction.
EvaluationoftheGirl’sEducationProject3
©EDOREN 133
Figure42: Causalpackageandunderlyingassumptionsof‘moreeffectiveteaching’
4.3.1.3. Alternativeexplanations
Weconsidertwoalternative,non-GEP3-relatedexplanationsthatcouldalsobesufficientfortheintendedchangeinteachingintheIQSstobeobserved:
• Facilitatorsbuildcapacityinotherways.IffacilitatorsintheGEP3IQSsparticipateinothertrainingorcapacitybuildingprocessesduringtheGEP3projectperiod,teachingcouldbecomemoreeffectiveregardlessofGEP3.Suchcapacitybuildingmaytakeplacewhenfacilitatorsarealsoteachersinpublicprimaryschools.Furthermore,ifnewfacilitatorswhohaveahigherteachingcompetencylevelareemployed,teachingmayalsobecomemoreeffective.
• Facilitatorsaremotivatedinotherways.IffacilitatorremunerationorworkingconditionsimprovewithoutanyinfluencebyGEP3interventions,facilitatorsmaybecomemoremotivatedandteachmoreeffectivelyregardlessofGEP3.
4.3.2 GEP3’sIQSScontributestoanimproved,girl-friendlyschoolenvironmentinIQSs
4.3.2.1 Causallinkagesandmechanisms
MainlythroughitssupporttoCBMCsGEP3seekstoimprovetheschoolenvironmentoftheIQSandmakeitmoregirl-friendly.Weconsidertheschoolenvironmenttoencompassthephysicalenvironment,intermsofinfrastructureandclassroomconditions,itssecurityandperceivedsafety,andtheschool’sorganisationandinstitutionalculture,intermsofleadership,organisationalprocesses,relationshipsbetweenschoolstakeholders,andtraditions.Theschoolenvironmentaffectsthelearningexperienceofthepupils,aswellastheteachingexperienceofthefacilitators.
EvaluationoftheGirl’sEducationProject3
©EDOREN 134
GEP3seekstocontributetothegirl-friendlinessoftheschoolenvironmentintheIQS,promotinginvestmentininfrastructureandresourcesthatimprovetheschoolexperienceofgirlsandaschoolorganisationthatsupportsgenderequity.
Thecorecausalpathwaytoimprovingtheschoolenvironmentisbyimprovingschoolmanagementandsupportingthemobilisationofmoreresources.Additionalresourcesthataremobilisedcanbeinvestedincriticalinputsthataddressbarrierstogirls’educationandthatareprioritisedinschooldevelopmentplans.Betterschoolmanagementisassumedtoimprovetheschoolenvironmentbecause,throughthedevelopmentofWCDPs,strengthsandweaknessesintheschoolenvironmentareassessed,prioritiesforimprovementidentified,andresourcemobilisationactionsoutlined.ThroughtheactiveparticipationoftheCBMC—whichismeanttorepresentthedifferentschoolstakeholders—theprioritiesandpreferencesofvariousstakeholdersaretakenintoaccount:inparticularthepreferencesofwomenandgirls.Itisassumedthatwomenandgirls’participationintheschoolmanagementbettersignalsgirls’needsandstrengthensthedemandforgirl-friendlyschoolinputs.Furthermore,betterschoolmanagemententailsmoreadequatefinancialmanagementandrecord-keeping,whichsupportsplanningandtransparency,andhencemoreeffectiveuseofresourcesprovidedbythecommunityorotherexternaldonors.Giventheoftenlow-resourcesettingoftheIQSsresourcemobilisationisanecessaryconditiontotranslatedevelopmentplansintoinvestments.
ThroughafunctionalCBMCcommunityrepresentativesareinvolvedintheIQS.ThisisassumedtoincreasecommunitysupportforintegratedQur’aniceducation,particularlyforgirls,becauseitmakescommunitymembersmoreconsciousoftheintegrationandempowersthemtohaveasayinitsimplementation.Thiscontributestoincreaseddemandforgirls’education.Inaddition,ifthecommunitybecomesmoreinvolvedinthemanagementanddecision-makingoftheIQSthenitisassumedthattheywillbemorelikelytosupportitfinanciallyorwithotherresourcesbecausetherewillbecommunityownership,oraperceptionofownership,overtheIQS.Finally,communitymembersareabletobetterholdschoolleadersandteachersaccountableforthequalityoftheschoolenvironmentandattendancesince,throughtheirinvolvement,theyaremoreawareofwhatoccursintheschool,andagainbecauseoftheirstrongersenseofschoolownership.
Themainmeanstoachievetheaboveintermediaryoutcomesisthetraining,mentoringandmonitoringoftheCBMCmembers.Thetrainingaimstoimprovetheirunderstandingoftheirrolesandresponsibilities,whichcanenableschoolmanagementandresourcemobilisationbecausemanagementandsupervisorytasksandfunctionswillbebetterdefined(e.g.teacherattendancemonitoring,planning,andfundraising).BytrainingCBMCmembersonwholecentredevelopmentplanningtheirskillsinschoolplanningaremeanttoimprove.Theinclusionofgenderinthetrainingcurriculumaimstocreateawarenessoftheissue,whichissubsequentlyassumedtoresultinmoregender-responsivewholecentredevelopmentplanning.TrainingonresourcemobilisationandfinancialmanagementismeanttomakeCBMCmembersmoreabletomobiliseandmanageresources.Theperiodicmentoringvisitscanreinforceandrefreshthelearningfromthetrainingworkshops,providingpracticalfeedbacktailoredtothecontextofeachIQS.Itislesscleartowhatextentthementoringvisitswillre-enforcegendersensitisationtoensurethatCBMCmembersfollowuponandmonitorthegirl-friendlinessoftheschoolenvironment.Monitoringcansignaldeficiencies,basedonwhichactioncanbetaken,andcanincreaseaccountabilitybydirectingattentiontotheachievementofresults.GEP3willsupportgovernmentstafftocarryouttermlyCBMCeffectivenessmonitoring,providingatoolthatismeanttoimproveCBMCs’monitoringcapacity.
BeyondtheroleoftheCBMC,theheadteachertrainingseekstoincreasetheheadteachers’skillsinschoolmanagement,whichisalsoassumedtocontributetomoreeffectiveschoolmanagement.
EvaluationoftheGirl’sEducationProject3
©EDOREN 135
Themini-grantsdirectlycontributetoanincreaseinschoolresourcesand,giventhattheyaretargetedatgirl-friendlyinvestments,canimprovethegirl-friendlinessoftheschoolenvironment.ThegrantscanalsoworkasanincentiveforCBMCmemberstodeveloptheircapacity,asmini-grantsareprovidedontheconditionthatCBMCmembershaveparticipatedinthetraining.Furthermore,themanagementofthegrantcanimproveoverallcommunity-basedschoolmanagementthrougha‘learning-by-doing’capacitybuildingapproachfortheCBMC,whichismeanttomanagetheresources.
Gender-sensitiveclasspracticesandtheincreasedpresenceofgirlsintheIQS,duetoincreasedenrolmentandretention,canstrengthenaschoolclimatethatsignalsanopennesstogirlpupilsandcancreateasocialenvironmentwheregirlsfeelwelcomeamongtheirpeers.
4.3.2.2 Assumptions
Figure43presentstheassumptionsabouttheconditionsthatneedtobeinplaceforIQSSinterventionstoactuallyresultinanimproved,moregirl-friendlyschoolenvironment.
Figure43: Causalpackageandunderlyingassumptionsof‘improved,girl-friendlyschoolenvironment’
EvaluationoftheGirl’sEducationProject3
©EDOREN 136
4.3.2.3 Alternativeexplanations
Weconsiderthefollowingalternative,non-GEP3-relatedexplanationsthatcouldimprovetheschoolenvironmentandmakeitmoregirl-friendly:
• CBMCmembersandschoolleadershipimprovetheircapacitythroughothermeans.SchoolmanagementandresourcemobilisationcanimprovebynewstakeholdersbecominginvolvedintheIQS,regardlessofGEP3support.Forexample,aphilanthropistmaytakeaninterestintheIQSortheIQSmaystartbenefittingfromtheinfluenceorcompetenciesofahighlyeducatedor/andrespectedcommunitymember.ItisnotonlydifficulttoidentifyallsuchinfluencesbutitwillbealsohardtoseparateoutGEP3’sinfluenceinsuchinstancessinceCBMCtrainingpromotessuchinvolvement.SomeCBMCmembersmayalsobemembersoftheSBMCintheneighbouringpublicprimaryschool,andmaybuildtheircapacitythroughtheirparticipationinpublicprimaryschoolmanagement.
4.3.3 Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlylearningenvironmentcontributetoimprovedlearninglevels,particularlyamonggirls
4.3.3.1 Causallinkagesandmechanisms
TheprincipalfinaloutcomethatGEP3aimstoachievewithitsIQSSisforpupilslearningoutcomestoimprove,particularlyforgirls.Moreeffectiveteachingisconsideredacriticalcontributingfactor,whichwillbesupportedbyanimproved,moregirl-friendlyschoolenvironment.Asthesetwofactorsimproveandresultinbetterqualityeducationgirls’retentionisalsoassumedtoincrease.Inaddition,girls’enrolmentisexpectedtoincreasebecauseofanincreaseddemandforgirls’educationasaresultofmorecommunityinvolvementandsupportforIQSs.Girls’retentionandenrolmentarenotfurtherdiscussedbelow,astheyarenotthefocusofthisevaluation.However,itisimportanttonotethatincreasedretentionandenrolmentcaninturninfluenceeffectiveteachingandtheschoolenvironment,aswasdiscussedintheprevioussections.
Internationalstudiesindicatethatwhatteachersknow,whattheydoandhowmuchtheycareaccountsformorevarianceinpupilachievementthananyotherpolicy-amenablevariable(Hattie2003).Severalmechanismsareatplaythatmaycausemoreeffectiveteachingtoleadtoimprovedlearningoutcomesforgirls.First,thecombinationofimprovedsubjectknowledgeandpedagogicalskills,supportedbyfit-for-purposeteachingandlearningmaterials,canenablemorecompetentandequippedfacilitatorstobettersupportpupils’learningachievementintheclassroom.Byteachinginamoregender-sensitivemannertheteacher’sinteractionwiththegirlpupilimproves,whichcanimprovethequalityofthegirl’seducationinparticularandprovideadditionalmotivationforhertolearn.Second,student-centredteachingmethods—acompetencythatGEP3trainingintendstostrengthen—encourageschildren,especiallygirls,toattendschools,whichincreasesthepupil’sactualinstructionaltimeandcan,therefore,increaselearning.Third,effectiveteachingalsoentailsfacilitatorsattendingandteachingwheninschool(asaresultofbetterschoolmanagement).This,again,canincreasetheactualinstructionaltime,andhencepositivelyinfluencelearningoutcomes.
Totheextentthattheschoolenvironmentismoreconducivetogirlslearning,learningachievementisfurtherstrengthened.Again,thecausalmechanismshereareseveral.First,improvedphysicalconditionsunderwhichteachingandlearningtakeplace(e.g.lessovercrowdedclassroomsorteachinginashadedlocation)enhancethedegreeofsupportthatteacherscanprovideandtheattentionthatpupilscangivetolearning.Second,thephysicalconditionsinfluencetheinstructionaltimeasprotectionagainstrain,orlightduringeveningclasses,enableclassesto
EvaluationoftheGirl’sEducationProject3
©EDOREN 137
takeplace.Third,asafe,comfortableandenjoyableschoolenvironmentaffectspupils’experienceofschooling,whichislikelytoleadtomorefrequentattendanceandgreaterengagementintheschoolingenvironment.Totheextentthattheschoolenvironmentismoregirl-friendly,girls’schoolexperienceisparticularlyenhanced.
4.3.3.2 Assumptions
Figure44presentstheassumptionswithregardstotheconditionsthatneedtobeinplaceformoreeffectiveteachingandanimproved,girl-friendlyschoolenvironmenttoactuallyresultinimprovedlearningoutcomes,particularlyforgirls.Wedonotfocushereonassumptionsrelatedtoretentionbutmanyofthepresentedassumptionsarerelevanttoachievingtheretentionofgirls.
Therearemanyfactorsthat,inturn,influencewhethertheconditionshold.Thedifferentassumptionsarealsointerdependent.Pupils–inparticular,girls–actuallyattendingclassisinfluencedbyexternalfactors,suchastheiroutsideschoolactivitiesandresponsibilities,publicprimaryschoolattendance,economicconstraints,schoolaccessibility,andtheattitudesofthepupilsandtheirsocialenvironment.Girlsareassumedtobeinterestedinacquiringaformaleducation,whichwilldependontheiraspirationsandthevaluetheyseeinformaleducation–given,inparticular,theexpectationofearlymarriage,whichisanimportantbarriertogirls’retention.
Figure44: Causalpackageandunderlyingassumptionsof‘improvedlearningoutcomes’
4.3.3.3 Alternativeexplanations
Weconsiderthefollowingalternative,non-GEP3-relatedexplanationsthatcouldalsobesufficienttoexplainimprovedlearningoutcomes:
EvaluationoftheGirl’sEducationProject3
©EDOREN 138
• Improvedaccessandqualityeducationinotherschools.ThebaselinedatademonstratethatIQSsarenottheonlyalternativetoaccessingformaleducation.PupilsareenrolledinbothIQSsandpublicprimaryschools.Theextentthatpupilsattendthetwoschoolsdependsonmanyinteractingfactors,suchasaccessibilityofthepublicprimaryschool,therelativecostofattendingpublicprimaryschoolsandpovertyoftheparents,aswellasculturalbeliefsandsocialnormswithregardstoformaleducationandgirls’accesstothis.Iftheseunderlyingfactorschangepublicprimaryschoolattendancemayincreaseandlearningoutcomesmayimprove,regardlessoftheintegrationofformaleducationintheQur’anicschools.Thequalityofeducationalsoplaysarole.Accesstopublicprimaryschooldoesnotnecessarilyresultinimprovedlearningoutcomes.TotheextentthatthequalityofeducationinthepublicprimaryschoolsthatIQSpupilsareattendingbecomesbetter,theirlearningoutcomescanimprove.Furthermore,inlessremoteareaspupilsmayattendvocationalorothertraining,whichmayprovidesomeformofbasiceducationormotivatepupilstolearn.Finally,inmoreurbanareasformalprivateschoolsmayalsoattractpupils,whichcanaffecttheirlearningoutcomes.
• Changesinthecompositionofthepupilpopulation.Overtime,theIQSmayattractadifferentpupilpopulation:forexample,youngerpupilsstarttheintegratedcurriculumormorepupilswithpreviouseducationexperienceareattractedtotheIQS.TotheextentthattheirlearningoutcomelevelsaredifferentfromthelearninglevelsofthebaselinepopulationchangesintheaveragelearningoutcomeintheIQSmaybeobserved,regardlessofimprovedteachingoranimprovedschoolenvironment.
4.4 Analysisofthedata–ContributionClaim1:GEP3’ssupporttoIQSscontributestomoreeffectiveteachingofformalsubjectsinIQSs
Thenextthreesubsections(Sections4.4–4.6)presentthebaselineanalysisofthequantitativesurveydatagatheredin60IQSsandthequalitativecasestudyfindingsinsixIQSsinNigerandBauchi.TheevidencehasbeenstructuredaroundthethreecontributionclaimspresentedintheIQSSToC.Thesebaselinefindingswillfunctionasthereferencepointtowhichthemidlineandendlinedatawillbecomparedinordertoassesstheimpactcontributionoftheprogramme.
Thequantitativeanalysispresentsadescriptiveanalysisofassociationsofbaselineindicatorsacrosscategoriesofinterest.GiventhatthesamplesizeoftheIQSSsurveyisrelativelysmallanddataaredisaggregatedfurtheracrossvariouscategories(bystate,byownership,etc.)theaveragespresentedhavelargeconfidenceintervals.Hence,itisimportanttoemphasisethattheseassociationsshouldnotbeusedtomakestatisticalclaimsofcausalinference.Rather,theyshouldonlybetreatedasasummaryofthebaselinesample.Thefigurespresentedalongsidethetextserveadescriptivepurpose,tofacilitatethenarrativeregardingtheresults.Alargersetofdescriptivestatistics,includingmeanestimates,standarderrorsandnumberofobservationsforeachvariableonaverageandbystate,isincludedinAnnexM.
Inthissectionwepresentthebaselinedatarelatedtoteachers,teachers’knowledgeandteachingpracticesinIQSs.Inaddition,wediscussothercomponentsoftheIQSSToCthatareassumedtosupportmoreeffectiveteaching:thatis,pedagogicalandschoolleadership,andaccesstoteachingandlearningmaterials.Thequantitativedataanalysisispresentedfirst,followedbytheanalysisofthequalitativecasestudyfindings.
EvaluationoftheGirl’sEducationProject3
©EDOREN 139
4.4.1 Quantitativeanalysis
ThequantitativeanalysisbeginswithadescriptionofthecontextsinwhichtheteachingofformalsubjectstakeplacewithinIQSsandprovidesadescriptionofwhatfacilitatorsknowanddowithintheirlessons.TheanalysisthenturnstoreportingonthequantitativeoutcomesasperthefirstcontributionclaimoftheGEP3IQSS.TheToCfortheIQSSprogrammeindicatesthattheteachingoffacilitatorswillbecomemoreeffectiveasaresultof:(1)theincreasedmotivationoffacilitators;(2)improvedsubjectknowledgeandpedagogicalskillsoffacilitators;(3)improvedskillsingender-sensitiveclasspractices;(4)theavailabilityofteachingandlearningmaterials;and(5)increasedheadteacherpedagogicalleadership.Thissectionoutlinestheevidenceatbaselineoneachoftheseoutcomes.
4.4.1.1 TheIQScontext
Overviewofschoolstructure
IQSstakevariousformsinBauchiandNiger,rangingfromthosethatareorganisedatasinglelevel,whereallthechildrenareinthesame‘class’studyingunderasinglefacilitator,tothosewheretheschoolisstructuredexactlylikeapublicprimaryschool,withchildreninclassescalledPrimary1,Primary2andsoon.InthesampleofIQSsamajorityoftheschoolswereorganisedinamannersimilartopublicprimaryschools.However,therewereotherschoolstructuresaswell.Table26mapsthevariousIQSstructuresencounteredagainsttheircorrespondingprimaryschoolclasses.
Table26: MappingIQSstructurestoprimaryschoolclasses
Type1 Type21 Type3 Type4 Type5
P1 Awal
Class1/Stage1/Aji/Tsangaya
Class1/Tsangaya1/Aji1/Stage1
Class1/Stage1/Grade1P2 Sani
P3 Salisu Class2/Stage2/Grade2P4 -2 Class2/
Tsangaya2/Aji2/Stage2
P5 - -
P6 - -
62% 5% 8% 15% 10%
Notes:1Awal,SaniandSalisuareArabicforfirst,secondandthird,respectively.2‘-‘indicatesthatacorrespondingleveldidnotexistattheIQS.Forexample,Type2schoolsdidnotprovideupperprimaryclasses.
BothType1andType2IQSshaveonelevelcorrespondingtoeachprimaryschoolclass,thedifferencebeingthatType1useIQSsthepublicprimaryschoolclassificationofPrimary1andPrimary2whileType2IQSsusealocalcategorisation.67%ofthesampledschoolshadone‘level’correspondingtoeachprimarygradeclass.ItisworthnotingthatType2schoolsonlycateredtothelowerprimarygrades,anddidnothavelevelscorrespondingtoP4–P6.AninterestingdifferenceisthatinNigeralargershareoftheIQSswereType1orType2,whileinBauchiarelativelylargershareofIQSswereType3.Inaddition,therewerenoType2IQSsinBauchi.Thisdifferencealsoholdsintherural–urbandisaggregation:alargerproportionofruralIQSshadonelevelperpublicprimaryschoolclass(Type1orType2)whileurbanIQSsweremorelikelytobeoneoftheremainingthreetypes.FortheIQSSsamplearelativelylargershareofIQSsinBauchiwereurban,comparedtoNiger.Asaresult,moststate-leveldisaggregationsdiscussedinthissectionandinwhatfollowsalsohold
EvaluationoftheGirl’sEducationProject3
©EDOREN 140
trueforurban–ruraldisaggregations,whereNigerisdrivingtheresultsassociatedwithrurality.Forschooltype,thissuggeststhatruralschools,andschoolsinNiger,weremoreoftenstructuredlikeformalpublicprimaryschools.
Type3schoolsarethosewhereallthechildrenstudyinthesameclass,whichcouldbeduetoashortageofspace,studentsorteachers.Thiswastheleastcommontypeofschoolingstructureencountered.Type4schoolsareschoolswhereeachlevelintheIQScorrespondstothreeprimaryschoollevels.Hencetheschoolhaspupilsattwo‘levels’.ThesearemostcommonlyreferredtoasStage1andStage2,butthisisnotalwaysthecase.Thelastcategory,Type5,includesschoolswhereeachlevelcorrespondstotwoprimaryschoollevels.Throughoutthisreport,wherewerefertoP1–P3‘level’orP2orequivalent‘level’theschoolinquestionmaybeorganisedaccordingtooneofthefivetypesofstructuresdiscussedabove.
Schoolshadanaverageof6.6teachersofbothintegratedandQur’anicsubjects,andthisdidnotvarymarkedlyacrossstates.Ofthese,anaverageof2.78teachersperschoolwereteachersofintegratedsubjects(i.e.facilitators),while2.22facilitatorsperschooltaughtattheP1–P3level.25%ofIQSshadonlyonefacilitatorforgradesP1toP6,while36.7%hadonlytwo.Whenfocusingonlyonearlygrades(P1–P3),aroundone-thirdoftheschoolshadonlyoneteacherofintegratedsubjects.Thishasimplicationsfortheteacher–teacherinteractionassumedaspartoftheIQSSToC(seeSection4.1.3),sinceschool-basedpeerinteractionwillnotbepossibleforoveraquarterofthetrainedfacilitators.
Thereareimportantdifferencesinthenumberandgenderofteachersofintegratedsubjects,acrossstates.Theseteachersarereferredtoasfacilitatorsthroughoutthisreport.Fromhereonwardstheanalysiswillfocusonlyonfacilitatorsthatwereteachingregularclassesofsubjectsintheintegratedcurriculum.Table27showsthatNigerhasmoreteachers,andmorefemaleteachers.However,overalltherearefewfemalefacilitatorsinIQSs.
Table27: FacilitatorsinIQSs,bystate
Variable Bauchi Niger
Totalnumberofteachers1 6.66 6.5
Numberoffacilitatorsatalllevels 2.39 3.18Fractionoffacilitatorsthatarefemale 0.03 0.16
NumberoffacilitatorsP1–P3 1.93 2.5
FractionoffacilitatorsthatarefemaleP1–P3 0 0.07
Note:1Referstobothteachersofreligioussubjectsandintegratedsubjects
SchoolownershipforIQSsisdifferenttoownershipofpublicprimaryschools.IQSscanbeheadedbyaheadteacher,whoisresponsiblefortheday-to-dayfunctioningandmanagementoftheschool.Theschoolowneriscalledtheproprietor,andmaybethesamepersonastheheadteacherormaybeadifferentperson.Inone-thirdoftheIQSstheheadteacherwasalsotheproprietor,whilefortheremainingschoolstheseroleswerefilledbydifferentindividuals.Thedifferentschoolownershipstructuresledtointerestingtrendsinthedata.
Schoolswheretheheadteacherandproprietorroleswereseparatedhadmorefacilitatorsonaverage,andhadmorefemalefacilitators.Theyalsohadbetterinfrastructure,asindicatedbythepresenceofmoreroomsoverall,moreroomsbeingusedforclasses,morefunctioningtoiletsfor
EvaluationoftheGirl’sEducationProject3
©EDOREN 141
pupils,andthegreaterlikelihoodofhavinganelectricityconnection.Thishintstowardsthefactthatschoolswheretheheadteacherandproprietorrolesareseparatedareperhapsbetterestablishedasschools.Thisnotionissupportedbythefactthatsuchschoolshadbeenintegratedforlongerthanschoolswheretheheadteacherandproprietorwerethesameperson.SchoolswheretheserolesareseparatedaremorecommoninNigerthaninBauchi.
Asmentioned,allthefacilitatorssurveyedtaughtintegratedsubjectsattheP1–P3levelatthetimeofthebaselinestudy.Aminorityofthem(around12%)alsotaughtP4toP6classes,thusbeinginvolvedinteachingatbothupperandlowerlevelsoftheschool.ThiswasparticularlyprominentinBauchi.Whenfacilitatorswereaskedtospecifythesubjectstheywerecurrentlyteaching,eachfacilitatorreportedteachinganaverageoftwointegratedsubjects.Inlinewithoursamplingstrategy,amajorityofthesurveyedfacilitatorswereteachersofmathematicsorEnglish,with65%offacilitatorsteachingmathematicsand71%ofthefacilitatorsteachingEnglishinthecurrentterm.Surveyedfacilitatorsalsotaughtotherintegratedsubjects:32%ofthefacilitatorsreportedteachingHausa,24%reportedteachingsocialstudiesand11%reportedteachingbasicscience.Thisreflectsthefactthatallschoolswerenotteachingallfivecoreintegratedsubjects:English,mathematics,Hausa,scienceandsocialstudies.Findingsfromtheschool-levelheadteacherinterviewconfirmthatalittleover10%ofsurveyedschoolswereteachingallfiveintegratedsubjects.ThistrendisbeingdrivenbytheschoolsinNiger,ofwhichanoticeablylargerproportionwereteachingallfiveintegratedsubjectscomparedtoschoolsinBauchi.Althougharound70%ofsurveyedschoolstaughtHausaasasubject,thefocusonEnglishandmathematicsteachingwasclearinthedata,with90%oftheschoolsteachingbothofthesesubjects.Thevariedadoptionoftheintegratedcurriculumdeservesattentionduringimplementation,totheextentthatGEP3seekstoimprovetheeffectiveteachingofallcoresubjects.133Forschoolsthatarenotalreadyteachingmultipleintegratedsubjects,additionalsupportmayberequiredforfacilitators,aswellaspupils,tocopewithadditionalsubjects.
Whenlookingattheentirerangeofintegratedsubjectsbeingtaughtbysampledfacilitators(whichincludesphysicalandhealtheducation,creativearts,andcomputerscience,inadditiontothefivecoreintegratedsubjects),weseethatslightlymorethanathirdoftheIQSfacilitatorswereteachingonlyoneofthesubjects,whileanotherthirdwereteachingtwosubjects(seeFigure45).Therearesharpintra-statedifferences:inNiger,onlyaboutaquarterofthefacilitatorswereteachingasinglesubject,whileabouthalfwereteachingtwosubjects.ThistrendwasreversedforBauchi,whereapproximatelyhalfthefacilitatorstaughtonlyasingleintegratedsubject.ThisisinlinewiththefactthatmoreschoolsinNigertaughtmultiplesubjects,whileschoolsinBauchitaughtonlyoneortwo.
133TheGEP3StrategyPaperforIQSs(UNICEF,2015b)indicatesthatGEP3willsupportthedevelopmentofatrainingpackageandimplementationprogrammetoimprovethecapacityofIQSteachers/facilitatorstobeeffectiveteachersofcoresubjects.Thestrategydoesnotspecifywhetherthisreferstoallcoresubjects.TheTrainingManualforIQTETeachersandFacilitatorscoversallcoresubjects.However,theoutcomesstatedintheStrategyPaperfocusontheimprovementofpupils’literacyandnumeracyskills,whichindicatesthatliteracyandmathematicsmaybethecoresubjectstheeffectiveteachingofwhichtheprojectwantstoemphasise.
EvaluationoftheGirl’sEducationProject3
©EDOREN 142
Figure45: Numberofintegratedsubjectstaughtperfacilitator
FacilitatorsinIQSs
ThetypicalfacilitatorinIQSSschoolsismale,around32yearsofageandspeaksHausa.Femalefacilitators,thoughfewinnumber,areconcentratedinschoolsinNiger.Thereseemstobemobilityacrossschools,asindicatedbytheaveragenumberofyearsasateacher(aroundsevenyears)versustheaverageyearsofteachingatthecurrentschool(around3.5years).Alittleoverhalfthefacilitatorshadbeenatthecurrentschoolformorethantwoyears,whichislikelyrelatedtotherecentintegrationofmanyIQSs.WecollectedinformationonteacherturnoverattheP1toP3levelforasmallsampleoftheschoolsandfoundthatalthoughforamajorityoftheschools(abouttwo-thirds)nofacilitatorsfromP1–P3hadlefttheschoolduringthelastoneyear,inschoolswhereteacherturnoverwaspresenttheturnoverrateswerehigh.Asashareoftotalfacilitatorsattheearlygrades,anaverageof80%offacilitatorslefttheschoolinthepreviousyear.Thisamountstoanaverageof2.1facilitatorsleavingtheIQSduringthepreviousyearinIQSswithapositiveturnoverrate.ThishighlightstheneedtoincorporatetheriskofteacherturnoverintothedesignofteachertraininginitiativesbecauseiftrainedteachersleavetheIQSpupilsmaynotbeabletobenefitfromtheinvestmentinteachertraining.
FacilitatorsinBauchiareolder,withnoticeablymoreyearsofteachingexperiencethanfacilitatorsinNiger.ThenumberofyearsspentatthecurrentschoolisalsogreaterforBauchifacilitators.
AlmostallofthefacilitatorsreportbeingabletospeakHausa,andmostofthemreportbeingabletospeakEnglish.InNiger,Nupeisacommonlanguage,with39%offacilitatorsreportedlyspeakingthelanguage.InBauchi,facilitatorsgenerallydonotspeakathirdlanguage,althoughArabicandFulfuldearespokenbyaminority(21%and13%,respectively).Thiscorrespondsclosely,atthestatelevel,tothelanguagesspokenathomebypupils.
Aroundhalfofthefacilitatorshavesomedegreeofprofessionaleducationalqualification.Offacilitatorswhohadaformofprofessionalqualification,around80%hadanNCE,while15%hadaGrade2certificate.TwiceasmanyteachersinBauchihadsomeformofprofessionalqualificationcomparedtoteachersinNiger.Only20%oftheNigerteachersdeclaredthattheyhadanNCE,comparedto51%oftheBauchiteachers.Hence,thefacilitatortrainingneedstobeadaptedto
35.00%33.00%
21.00%
8.00%
2.00%
0%
5%
10%
15%
20%
25%
30%
35%
40%
1 2 3 4 5
EvaluationoftheGirl’sEducationProject3
©EDOREN 143
teachersthathavenoprofessionaleducationalqualificationinteaching,particularlyinNiger.Intermsofacademicqualifications,asmallshareofthesamplewasonlyeducateduptotheprimarylevel,whileamajorityhadanSSCE.Aroundone-fifthoftheteachersalsohaveareligiouseducationqualificationofsomeform.AconsiderablylargershareoftheBauchiteachershadareligiouseducationqualification,comparedtoteachersinNiger(33%versus11%).
Ofthetotalfacilitatorsinterviewed,around43%hadattendedsomeformoftrainingduringthelasttwoyearswhilebeingemployedatthisschooloranotherschool.Facilitatorsattendedaroundsixdaysoftrainingonaverageduringthelasttwoyears.MorefacilitatorsinNigerattendedtraining,butthereporteddurationoftrainingwaslongerforBauchi.Olderfacilitatorsweremorelikelytohaveattendedatrainingcoursethanyoungerones.Inaddition,facilitatorswithsomedegreeofprofessionalqualificationweretrainedmoreoftenandforlongerthanthosewithout.Thissuggeststhatdifferencesinteacheroutcomesseenacrossprofessionalqualificationcategoriescouldpartlystemfromthefactthatthesearetheteachersthathavebenefiteddisproportionatelymorefromtraining.
Therearealsointerestingdifferencesinthecontentofthefacilitatortraining.Thoughtheactualpercentagesareperhapsnotasinformative,itisinterestingtoseethatfacilitatorsinBauchireportedbeingtrainedinHausateachingandintegrationactivitiessignificantlymoreoftenascomparedtoteachersinNiger.Perhapsthesourceofthetrainingcanexplainsomeofthesedifferences–inBauchifacilitatorsreportedbeingtrainedbySUBEB25%ofthetime,whilethiswasthesourceoftrainingreportedbyonly4%oftrainedfacilitatorsfromNiger.TrainingbyGEPandUNICEFwerereportedat77%and67%inBauchiandNiger,respectively.Surprisingly,therewerenoreportsofanytrainingbySAMEineitherofthestates.
Figure46: Facilitatortrainingcontent,bystate
Schoolandclasssize
Schoolsize,asindicatedbyclassroomsinuse,varies,withanaverageoftworoomsobservedbeingusedforclassesonthedayoftheschoolsurvey.Themeanconcealstherealitythataround20%ofIQSshadnoclassroomstructure.Thisfactisalsosupportedbythequalitativeresearch.Anadditional30%oftheschoolshadonlyoneclassroom.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Trainedoncommunityinvolvement
Trainedonintegrationactivities
Trainedonschoolleadership
Trainedoncurriculumsubjects
TrainedonteachinginHausa
Trainedonliteracyandnumeracy
Trainedonteachingmethods
Bauchi Niger
EvaluationoftheGirl’sEducationProject3
©EDOREN 144
Pupilenrolmentdatawerecollectedinordertodevelopanarrativeregardingthepupil–teacherratioandthepupil–classroomratio.Adetailedanalysisofthesedataisnotpossiblesinceonlyaverysmallshare(about30%)oftheIQSshadsomesortofwrittenenrolmentrecordforthecurrentschoolyear.Thispresentedamajorhindranceinthecollectionofreliabledataonpupilnumbers.Despitethelimiteddataavailableonpupilenrolment,itisclearthatthenumberofpupilsstudyingintegratedsubjectsinIQSsvariesgreatly.Figuresrangefromasfewas20toasmanyas400childrenintheschool,withcross-statevariationbeingaslargeaswithin-statevariation.Ofthese,amajorityofpupilswereenrolledintheearlygrades,i.e.P1–P3level.AlthoughthegenderratiodoesnotseemtopointtowardsadisproportionatelyhighernumberofboysinIQSs,averylimitedsamplesizeforenrolmentdatadoesnotallowustomakestrongerclaimsaboutthegenderratiointheseschools,anditsimplications.Overall,theratioofgirlstoboyssuggeststhatIQSsdonothavemoreboysthangirls.Thisinferenceissupportedbythe(limited)dataonpupilenrolment,andmoresobythemanualcountingofP2pupilspresentonthedayofthevisit.Interestingly,theratioofgirlstoboyswashigherinBauchithaninNiger,andincaseswheretheheadteacherwastheproprietor.Thoughthiscouldpartiallybebiasedduetotheschools’knowledgeofthesurveyandthefocusoftheprogrammeongettinggirlsintoschool,thedifferencesarenoteworthy.Thisalsoindicatestheneedtoinculcaterecord-keepingpracticesinIQSsaspartofschoolmanagementpractices.
Theclassroomobservationdatasuggestameanpupil–teacherratioof45andamedianof40.Therangeisquitevaried,fromfourto183childrenbeingtaughtbyoneteacher.Aqualitativeindicationofthefindingthatthepupil–teacherratioishighinIQSscomesfromthefactthat94%ofsampledfacilitatorsagreedwiththestatement‘Therearetoomanypupilsinmyclassroom’.HenceitappearsthatfacilitatorsinIQSsareteachinginvariedandchallengingcontextsintheIQSSschools.
IQSintegration
IQSsinthesamplehavebeenestablishedforapproximately25yearsonaverage,withBauchischoolshavingbeenestablishedforalmosttwiceasmanyyearsasschoolsinNiger.Theschoolshavebeenintegratedfor2.5yearsonaverage,withschoolsinBauchihavingbeenintegratedforaboutoneyearlessthanschoolsinNiger(twoyearsversusthreeyears,onaverage).
Integration,asmeasuredbyschoolsteachingtheentirepackageoffivecoreintegratedsubjects,isfoundtobegreaterinschoolsinNigerthaninBauchi.Inthelatter,only3%oftheschoolswereteachingallfivecoreintegratedsubjects,whileinNiger,20%oftheschoolsweredoingso.SchoolsinNigerwerealsospendingmorehoursperweekonteachingintegratedsubjects.InBauchi,studentsstudiednon-religioussubjectsfor2.2hoursonaverage,whileinNigeritwas3.8hours–asignificantdifferenceandanindicatorthatschoolsinNigeraremoredeeplyintegrated.Thereisoneoutlier–oneschoolintheNigersamplereportedlyprovides17hoursofintegratedsubjectteachingperweek(allotherschoolsacrossbothstateshadintegratedsubjectteachingfor11hoursorlessperweek).Byremovingthisoutlierfromthesample,theaverageforNigerfallsto3.3.Althoughlower,thisstillsupportsthefactthatschoolsinNigerareteachingmoreintegratedsubjectsforalongerdurationthanschoolsinBauchi.
ItisnotstraightforwardtoassumethatsinceNigerschoolsarespendingmorehoursteachingintegratedsubjectstheymustbeswitchingawayfromreligioussubjects.Thechartbelowshowstherangeofreligioussubjectstaughtinschools,bystate.ForallbutonesubjectalargershareofschoolsinNigerwereteachingeachsubjectmentionedbelow.
EvaluationoftheGirl’sEducationProject3
©EDOREN 145
Figure47: Religioussubjectstaughtinschools,bystate
BauchischoolsseemclosertothetraditionalstructureofIQSs,wherechildrenstayonasboarders.43%oftheschoolsinBauchihadboarders,comparedtoonly27%oftheschoolsinNiger.Inaddition,schoolsinBauchiweremostlyfreeforboarders,whereasaround25%oftheschoolsinNigerchargedboarders.
ThefindingsonthedegreeofintegrationsuggestthatIQSsinNigerarebetterintegrated,asevidencedbyteachingofmoreintegratedsubjects,andmorehoursspentonteachingformalsubjects.Bauchischoolshavebeenintegratedforfeweryears,whichperhapsexplainsthelesserfocusonteachingintegratedsubjects.Thevariationinthedegreeofintegrationmayresultinvariationsintheimprovementofeducationqualitythattheprojectwillachieve.Inschoolswithlimitedhoursspentonteachingformalsubjectsandonlyshortexperienceofteachingintegratedsubjectsthepotentialimpactonlearningmaybeless.However,asmentioned,mostIQSsprovidemathematicsandliteracyclasses,whichcorrespondstotheIQSSinterventionsfocusonnumeracyandliteracylearningoutcomes.
4.4.1.2 Whatfacilitatorsknow
Thissectionofthereportdescribestheknowledge,skillsandpracticesoffacilitators,ascollectedthroughthreeprimarysources:afacilitatorassessment,aclassroomobservationandafacilitatorquestionnaire.Facilitatorknowledgeandskillsreferstoproficienciesasperthesubscalesdrawnfromthefacilitatorassessmentandaqualitativereviewoffacilitatorresponsestoopenendedquestionswithintheassessment.Facilitatorpracticesreferstothepracticeoffacilitatorsintheclassroom,asobservedthroughtheclassroomobservation.Facilitatorpracticeswerecategorisedintothreecategories:teachertalk,teacheractionandpupilaction.
Definingteachercompetence
Thepercentageofteacherswhodemonstrateminimumteachingknowledgeinliteracyandlanguageismadeupofthesixsubscalesdevelopedfromtheteacherassessment.
Thresholdsweredefinedforeachsubscaletodifferentiatebetweenteacherproficiencylevels.
60%
47%
57%
40%
63%
53%
60%
50%
70%
23%
70%
63%
0% 10% 20% 30% 40% 50% 60% 70% 80%
Arabic
Tajweed
Quran
Sira
Hadith
Fiqh
Shareofsc
hools
Niger Bauchi
EvaluationoftheGirl’sEducationProject3
©EDOREN 146
Twocut-offpointsweredefinedforeachscale,tocreatethreeproficiencylevelsperscale.Theproficiencylevelsare:lowband–noevidenceofskill;middleband–evidenceofrudimentaryskill;andupperband–evidenceofcompetence.
Facilitators’knowledgeandskills
Thefacilitatorassessmentwasdividedintothreesections,collectivelycomprising30items,includingmultiplechoice,shortresponseandlongresponseitems.
• InsectiononefacilitatorswereaskedtomarkpupilresponsestoHausaliteracyquestionsandtoindicatethegradelevelatwhichtheanswershouldbeknownbypupils,asdefinedbythecurriculum.
• InsectiontwofacilitatorswereaskedtofillinmissinginformationinordertoprepareananswersheetforareadingtestaimedatGrade2pupils.Facilitatorswereprovidedwithtwonewspaperarticlesandaskedtofillinmissinginformation;thisincludedansweringbasiccomprehensionquestionsandquestionsthatrequiredtheinterpretationofwordsandphrases.
• Sectionthreeaskedfacilitatorstoidentifypoorandgoodpupils’workandtoreviewpupils’workinordertomakejudgementsaboutpupils’writing,includingtheorganisationofthewriting,theuseofgrammar,punctuation,spelling,thepupil’sabilitytoself-correctandreflectontheirwritingandthepupil’sabilitytoformlettersandusespacesbetweenwords.Facilitatorswerethenaskedtodescribehowtheymightsupportthepupilinimprovingtheirwriting.
AscanbeseeninFigure48thevastmajorityoffacilitatorsdidnotdisplayevidenceofcompetenceinthesixdomainscoveredbythefacilitatorassessments.Noneofthefacilitatorsassessedwereabletoshowcompetenceinevidencingjudgementsanddiagnosingpupils’work.Similarly,lessthan1%hadeffectivewritingskills.Lessthan3%offacilitatorswerefoundtobecompetentinidentifyinglowperformers,andininterpretingwordsandphrases.Rudimentarylevelsofskillswereobservedinaverysmallproportionoftheassessedfacilitatorsinthesefourdomains.
Figure48: Percentageoffacilitatorsachievingwithinthelower,middleandupperbandsofcompetenceacrosstheteacherknowledgeandskillssubscales
0%
20%
40%
60%
80%
100%
Abilitytoidentifylowperformers
Abilitytoevidence
judgementsanddiagnose
Teacherwritingskills
TeacherGrade2Hausa
knowledge
Teachercomprehensio
nskills
Interpretingwordsandphrases
Upperband 1.9 0 .9 34.0 25.4 2.8Middleband 6.5 .9 .9 27.2 18.1 4.2Lowerband 91.6 99.1 98.1 38.8 56.5 92.9
EvaluationoftheGirl’sEducationProject3
©EDOREN 147
Thefindingthatnoneofthefacilitatorsassessedwerecompetentinevidencingjudgementsanddiagnosingpupils’workorwritingissignificant.Ashighlightedintheearlylearningevaluationsectionofthisreport,thereisaverystrongtraditionineducationalresearchthatastudentlearnsbestwhenteachingistargetedtowhats/heisreadytolearn.Facilitators’inabilitytoidentifylowperformersandevidencetheirjudgementsconcerningpupilperformancepresentschallengesinregardtoimprovingpupillearning.
AgreaterproportionoffacilitatorswereabletodisplayknowledgeandskillsinGrade2Hausaandincomprehensionthanotherareasoffacilitatorknowledgeandskills.However,itisimportanttostressthatthelevelsofHausaassessedwerebenchmarkedtoGrade1-and2-levelHausaknowledge.Itemstestedincludedbasicgrammarandtheinitiallettersofeverydayobjectsandanimals.Thefindingthatover60%offacilitatorswerethemselvesunabletodisplaycompetenceinGrade2-levelHausaraisesseriousquestionsaboutfacilitators’abilitytoraisepupils’learninglevelsinthisarea.
Inordertoinvestigatetheextenttowhichfacilitators’knowledgeandskillsareassociated,ananalysisofthecorrelationbetweenthesubscalesdevelopedfromthefacilitatorknowledgeandskillsassessmentwasundertaken.Thetablebelowshowsthecorrelations(Pearson’s)betweenthesubscales.
Table28: Correlationsbetweensubscalesoffacilitators’knowledgeandskills
Syllabu
skno
wledge
Abilityto
iden
tifylowperform
ers
Abilityto
evide
ncejudgem
entsand
diagno
se
Teache
rwritingskills
Teache
rHau
sakno
wledge
Teache
rcom
preh
ensio
nskills
Interpretin
gwordsand
phrases
‘Syllabusknowledge’
PearsonCorrelation
1
Sig.(two-tailed)
N 2935
‘Abilitytoidentifylowperformers’
Correlation .051 1
Sig.(two-tailed) .218
‘Abilitytoevidencejudgementsanddiagnose’
Correlation .224** .188** 1
Sig.(two-tailed) .000 .000
‘Abilitytobuildonpupilknowledge’
Correlation -.103* .083* .415**
Sig.(two-tailed) .013 .043 .000
‘Teacherwritingskills’Correlation .272** .527** .476** 1
Sig.(two-tailed) .000 .000 .000
EvaluationoftheGirl’sEducationProject3
©EDOREN 148
‘TeacherHausaknowledge’Correlation .219** .273** .171** .214** 1
Sig.(two-tailed) .000 .000 .000 .000
‘Teachercomprehensionskills’
Correlation .068** .246** .134** .227** .228** 1
Sig.(two-tailed) .000 .000 .001 .000 .000
‘Interpretingwordsandphrases’
Correlation .159** .141** .157** .528** .124** .225**
Sig.(two-tailed) .000 .001 .000 .000 .000 .000
*Correlationissignificantatthe0.05level(two-tailed),and**thecorrelationissignificantatthe0.01level(two-tailed).
Therearefourcorrelationsinthistablethatarenoteworthy(showninred):
• ‘Abilitytobuildonpupilknowledge’with‘Abilitytoevidencejudgementsanddiagnose’(r=0.415)
• ‘Facilitatorwritingskills’with‘Abilitytoidentifylowperformers’(r=0.527)• ‘Facilitatorwritingskills’with‘Abilitytoevidencejudgementsanddiagnose’(r=0.476)• ‘Interpretingwordsandphrases’with‘Facilitatorwritingskills’(r=0.528)
Thesecorrelationssuggestthatfacilitators’writingskillsinparticularmaybethekeydimensioninfluencingfacilitators’performanceinthetest.Inotherwords,theliteracylevelsofthefacilitatorsappeartobeakeyissue.Ifthislevelislow,itlimitsperformanceacrossarangeoftheareasfacilitatorsneedtobecompetentintoimprovepupillearning.
Overall,thesefindingsstarklyhighlightthattherearelargegapsinfacilitators’knowledgeandskillsonalldomainsassociatedwitheffectiveteaching,includingbasicHausaliteracy.AsnotedinSection3.4,effectiveandwell-designedtrainingprogrammescanfillanyknowledgeandskillsgaps,includingthoseinteachers’ownsubjectknowledge.ThechallengeforGEP3willbetoensurethatitsfacilitatortraininginterventionisdesignedandimplementedinamannerthatallowsittoaddresstheseverylargedeficienciesinteachers’competencies.Thereisalsoaneedfortheprogrammetoberealisticaboutwhatcanbeachievedintheshorttimeframecoveredbyitspilotphase.
Qualitativereviewoffacilitatorresponses
Thefacilitatorknowledgeandskillsassessmentincludedaqualitativereviewoffacilitators’responsestoincorrectanswersbypupils.Anitemwithinthefacilitatorknowledgeandskillsassessmentprovidedalistofpupilnameswithinaclass,thegenderofeachchildandthenumberofwordseachchildcouldreadperminute.Facilitatorswereaskedtofirstdescribetheachievementofgirlsandboysintheclass,identifywhichpupilsneededthemostassistanceandsupport,andprovidewaysinwhichafacilitatorcouldsupportpupilstoincreasetheirabilitytoreadandunderstandtext.Withintheitem,abouthalfofthepupilsscoredzeroandtherewasasmalldifferencebetweenboys’andgirls’achievement,withboysscoringslightlybetterthangirls.Facilitatorswerealsoprovidedwithexamplesofpupils’writingandaskedtowritewhatthefacilitatorshouldfocusontoimprovethepupils’handwriting.
Thereviewfoundthatfacilitatorsgenerallyperceivedeffortasthecauseoflowperformanceandheldthatincreasingpupileffortwasthebestresponsetolowperformance.IQSSExamplesAandBprovideexamplesoffacilitatorresponsesthatfocusonincreasingeffortandpayingattentioninclass,ratherthandisplayingknowledgeregardingtheerrorsinthepupil’sworkandhowafacilitator
EvaluationoftheGirl’sEducationProject3
©EDOREN 149
mightmeaningfullyimprovethepupil’swriting.Thefocusoneffortwithinthiscontextisalikelyreflectionofthelowlevelsofcompetenceamongstfacilitators–thefactthattheylackboththeknowledgeandskillstoimprovepupillearningleavesthemwithfewexplanationsforlowperformance.
IQSSExampleA:Facilitatoreffort
SupportforIQSExampleB:Teachereffort
Responsesalsoreflecteddevelopmentprogrammeaimsandobjectivesinshallowways.Responsesincludedstatementsabouttheimportanceoffocusingongirlsintheclass,withoutanysupportingstatementsidentifyinglowperformersintheclassorsuggestinghowgirls(andboys)canbesupportedbythefacilitator.
Someresponsespointedtodeeplyingrainedgenderbiases,withperceptionsthatgirlsarenaturallylesssuitedtothedemandsofeducationthanboys,ormoresuitedtootherroleswithinsociety(seeIQSSExampleC).Inanumberofexamples,facilitatorsstatethatgirlsneedmoreassistancebecausetheyhave‘anaturalweakness’.
IQSSExampleC:Girls’educationandgenderroles
EvaluationoftheGirl’sEducationProject3
©EDOREN 150
4.4.1.3 Whatfacilitatorsdointheirlessons
Thepracticeoffacilitatorsintheclassroomwascategorisedonthebasisoftheclassroomobservationsintothreecategories:teachertalk,teacheractionandpupilaction.Thebaselinefindingsonteachertalkindicatethatteachersaremorelikelytouserote-basedapproachesthanpupil-centredapproaches,althoughthereissomeuseofthelatter.Ofthedifferenttypesofteachertalkthatwereobserved,threeareconsideredpupil-centred:askingorrespondingtoanopenquestion,assistingingroupwork,andusingachild’snameinclass.AsseeninFigure49,overhalfoffacilitatorsusedachild’snameandalmost70%askedorrespondedtoanopenquestion.However,lessthan10%assistedingroupdiscussions.Inabout10%ofclassroomsfacilitatorswerenotpresentatsomepointduringclass.Thisdoesnotincludelessonsthatwereendedearly.
Figure49: Percentageofobservedfacilitatorsengagingindifferentteachertalkactivities
AsnotedinSection3,pupilactioninclassroomsisalsoausefulindicatorofthepedogogicalapproachbeingundertakenbytheteacher.Ofparticularinterestisthelevelofnon-roteactivity,whichisindicativeofthedegreeofpupil-centredteachingthatistakingplace.Thefollowingtypesofpupilactionfallintothiscategory:groupdiscussion,grouporworkinpairs,respondingtoanopen
0
10
20
30
40
50
60
70
80
90
100
Instructs Remindstheclasstopayattention
Assistsgroupdiscussion
Usesachild'sname
Asksorrespondstoanopenquestion
Asksorrespondstoaclosedquestion
Teacherisnotpresent
EvaluationoftheGirl’sEducationProject3
©EDOREN 151
question,askingtheteacheraquestion,usingatextbook,readingaloud,anddoingindividualwork.Thisisincontrasttochanting,listening,andrespondingtoclosedquestions.Figure50highlightsthatwhilefacilitatorswereengaginginsomenon-roteactivities,theuseofmostoftheseapproacheswaslimited.Inparticular,groupdiscussionandinstancesofchildrenaskingtheteacheraquestionwereobservedinlessthan10%ofclassrooms.Groupworkwasalsoveryuncommon.Individualworkandreadingaloudweremorecommonandinstancesofpupilsrespondingtoopenquestionswereobservedinamajorityofclassrooms.
Figure50: Percentageofobservedclassroomswherepupilsengagedinvarioustypesofpupilaction
Patternsoflanguageusevariedacrossthetwostates.InBauchi,themajorityoffacilitators(72%)wereobservedusingjustonelanguageduringthelessonobservation.Afurther24%wereobservedusingtwolanguages(seeFigure51).InNiger,25%offacilitatorsusedjustonelanguageduringthelesson,while60%usedtwolanguages.Asmallminorityoffacilitatorsinbothstatesusedthreeormorelanguagesduringalesson.Teachersinurbanschoolswereslightlymorelikelytousejustonelanguageduringthelesson(54%)thanthoseinruralschools(46%).ThemajorityoffacilitatorsinbothstateswereobservedusingHausaatleastonceduringthelesson(Figure52).InBauchi,allfacilitatorsusedHausaatleastonce,and24%usedEnglish.AsmallminorityusedArabic(2%)oranotherlanguage(6%).InNiger,72%usedHausa,59%usedEnglish,and48%usedanotherlanguage.Theclassroomobservationtooldidnotrecordwhattheseotherlanguageswere,althoughitislikelythatNupefeaturesheavilyheregiventhatitisspokenbyanotableshareoffacilitatorsandpupils.
0102030405060708090
100
EvaluationoftheGirl’sEducationProject3
©EDOREN 152
Figure51: Numberoflanguagesusedbythefacilitatorduringonelesson
Figure52: PercentageoffacilitatorsusingHausa,Arabic,Englishandotherlanguagesinclass
4.4.1.4 Facilitators’knowledge,skillsandpracticesaspertheToC
AsdiscussedinSection3.2.9,teachersdrawonthreetypesofknowledgewithinclassroompractice.Thebaselinedatahavebeenusedtocreatecompositeindicesforeachoftheseknowledgetypes.
• Subjectknowledgereferstoknowingtheessentialquestionsofthesubject,thenetworksofconcepts,theoreticalframeworkandmethodsofinquiry.Thisismeasuredthroughacompositeindexofteachers’writingskills,Hausaskills,comprehensionskillsandtheirabilitytointerpretwordsandphrases.
0%
10%
20%
30%
40%
50%
60%
70%
80%
Teacherusedonelanguageduringlesson
Teacherusedtwolanguagesduringlesson
Teacherusedthreelanguagesduringlesson
Teacherusedfourormorelanguagesduringlesson
Bauchi Niger
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Hausa Arabic English Other
Bauchi Niger
EvaluationoftheGirl’sEducationProject3
©EDOREN 153
• Pedagogicalknowledgereferstoknowledgeoflearnersinthesetting,knowledgeofhowtoprovidetheconditionsthatenablepupilstounderstandandtheselectionoflearningandassessmentmaterials.Thisismeasuredusingacompositeindexthatdrawsonteachers’abilitytoidentifylowperformersandevidencejudgementsanddiagnoselearninggaps.
• Curriculumknowledgereferstoknowingwhatshouldbetaughttoagroupofstudents,knowledgeofthenationalsyllabus,understandingoftheschool-andgrade-levelplanningdocumentsandknowledgeofthecontentofexaminations.Thisismeasuredthroughthesyllabusknowledgesubscaledevelopedfromtheteacherknowledgeandskillsassessment.
Theabsolutevaluesofthecompositeindices’scoresarenotinherentlyinformative,butratherprovidebaselinevaluesforcomparisonatmidlineandendline.However,differencesinscoresbetweengroupsareofinterestatbaselinetoinformtheprogramme.Theshareoffacilitatorswhoscoredzeroontheseindicesisalsoworthnoting.
Withregardstosubjectknowledge,justoverone-fourthoffacilitatorsscoredzeroonthecompositeindex.FacilitatorsinBauchihadslightlyhigherlevelsofsubjectknowledgethanthoseinNiger.Thosefacilitatorstrainedonintegrationactivitiesalsohadslightlyhighersubjectknowledge,asdidthosewithaprofessionaleducationqualification.Whilenoclearpatternsemergedoverthefacilitatoragespectrum,facilitatorsovertheageof50hadslightlylowerscoresinsubjectknowledge.Interestingly,asfacilitators’perceivedself-efficacyincreased,theirsubjectknowledgedecreased.Thismayindicatethatasfacilitatorknowledgeincreases,facilitators’self-perceptionsbecomemorerealistic,oritmaysimplyindicatethatfacilitatorsarenotparticularlygoodjudgesoftheirownknowledgeandskills.Conversely,facilitators’subjectknowledgeispositivelycorrelatedwithfacilitators’interestin,andenjoymentof,teaching.
Facilitatorpedagogicalknowledgelevelswereextremelylowacrossallfacilitatorgroups.Overall,91%offacilitatorsscoredzeroonthecompositeindex.FacilitatorsinBauchihadslightlyhigherlevelsofpedagogicalknowledgethanthoseinNiger.Therewerenoclearpatternsasregardspedagogicalknowledgeandfacilitatorcharacteristics,mostlikelyduetotheverylowscoresontheindexacrossallgroups.Aswithsubjectknowledge,therelationshipwithperceivedself-efficacyindicatesthatfacilitatorsmaynotbeverygoodjudgesoftheirownknowledgeandskillsasmanyfacilitatorswhoreportedhigherlevelsofperceivedteacherefficacywereunabletodemonstrateanypedagogicalknowledgeatall(scoringzeroforthismeasure).Facilitatorswhoreportedhigherlevelsofteacher-to-teacherinteractiondisplayedslightlyhigherlevelsofpedagogicalknowledge.However,allobserveddifferencesweresmallduetothelowlevelsofpedagogicalknowledgeobservedacrossthegroups.
Justover40%offacilitatorsscoredzeroontheindexforcurriculumknowledge.Therewerenodifferencesinfacilitatorcurriculumknowledgebystate.Therewerealsofewdifferencesinknowledgebyfacilitatorcharacteristics.
EvaluationoftheGirl’sEducationProject3
©EDOREN 154
Figure53: Percentageoffacilitatorsscoringzeroinpedagogical,curriculumandsubjectknowledge
Referringtothethreetypesofknowledgethatteachersdrawonintheclassroom:thevastmajorityoffacilitatorswereunabletodemonstrateproficiencyinknowingthelearnersinthesetting;theywereunabletodemonstrateknowledgeofhowtoprovidetheconditionsthatenablepupilstounderstand;andtheywereunabletodemonstrateproficiencyintheselectionofappropriatelearningandteachingmaterials(pedagogicalknowledge).Alargeminorityoffacilitators(about41%)wereunabletodemonstrateanyproficiencyinknowingwhatshouldbetaughttoagroupofstudents(curriculumknowledge),andaboutaquarterofthefacilitatorswereunabletodemonstrateanyproficiencyintheessentialquestions,conceptsandmethodsofenquiryinliteracyandlanguage(subjectknowledge).
AcompositeindexwasalsodevelopedtomeasurechangesintheToCintermediaryoutcome:improvedteacherpracticesandgendersensitivity.Theindexisbasedonthedepthofpupil-centredlearningactivitiesobservedintheclassroom,observationsofthefacilitatorlinkingthelessontopreviouslearningandlearningobjectives,andtimeontaskinclass.
Themeasureofthedepthofpupil-centredlearningwasbasedonclassroomobservationsrelatedtofacilitatorandpupilbehaviour.Inparticular,itwasbasedonwhetherornotfacilitatorsassistedgroupdiscussion,usedachild’sname,askedorrespondedtoanopenquestion,movedamongpupils,usedavailablematerials,engagedingroupdiscussion,orfacilitatedgrouporworkinpairs;andonwhetherpupilsrespondedtoanopenquestion,askedthefacilitatoraquestion,usedatextbook,readaloud,orcarriedoutindividualwork.Inlinewiththedataontheseconstituentvariables(seeFigure48andFigure49),facilitators’performanceonthisindexwasrelativelypoor,withmostfacilitatorsscoringinthebottomhalfoftherangeofpossiblevalues(seeFigure54).
0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0
Facilitatorsscoringzero(%)
Facilitatorsscoringzero(%)Curriculumknowledge 41.4Pedagogicalknowledge 90.6Subjectknowledge 26.1
EvaluationoftheGirl’sEducationProject3
©EDOREN 155
Figure54: Depthofpupil-centredteaching,bystate
Facilitators’scoresonthesecondcomponentoftheindex(linkingthelessontopreviouslearningandlearningobjectives)incorporatedobservationsofthefacilitatortalkingaboutthepreviouslesson,outliningtheobjectivesoftheobservedlesson,revisitingtheobjectivesofthelessonattheendofthesessionandsummarisingthelesson.Roughly20%offacilitatorsdidnotcarryoutanyoftheseactivitiesatthestartorendofthelesson,while8%carriedoutallfour.FacilitatorsfromBauchiperformedbetteronthiscomponentoftheindexthanthosefromNiger.
Timeontaskwasmeasuredbythepercentageoftimethatpupilswereengagedduringlessonobservations.Thiswasclassifiedasthetotaltimeduringwhichpupilswereperforminganyactionrelatedtolearning(forthedifferenttypesofactionsthatwererecordedseeFigure50).Theclassroomobservationtoolrecordedpupilactionsatthree-minuteintervalsduringthelesson.Thesedatawereusedtoestimatetimeontask.Withinthemajorityoflessonsobserved(64.5%),childrenspent100%ofthelessononactivitiesrelatedtolearning(seeFigure55).However,thesefindingsshouldbeinterpretedcarefullyasitislikelythatthepresenceofobserversintheclassroomincreasedthepercentageofon-tasktimeinlessons.Theaveragedurationofthelessonsobservedwas24minutes,althoughthisvariedacrossthetwostates(seeFigure56).
0%
10%
20%
30%
40%
50%
60%
0 0-5 5-10 10-15 15-20 above20
Bauchi Niger
EvaluationoftheGirl’sEducationProject3
©EDOREN 156
Figure55: Percentageoftimeontaskinobservedlessons
Figure56: Lengthoflessonsobserved,bystate
ThequantitativecomponentoftheIQSSevaluationalsosoughttoassesstheprevalenceofgender-sensitivepractices.Beyondmeasuringtheproportionofgirlsandboyswithintheclassroomsetting,themeasurementofgendersensitivityisextremelycomplexandoftencannotmeetreliabilitycriteria.Withinthefacilitatorquestionnaireseveralitemsweredeveloptomeasureattitudestowardsgirls.Acrossallitemsextremecomplianceeffectswereobserved,significantlycallingintoquestionthevalidityandreliabilityofthemeasure.Therefore,changesinthegendersensitivityoffacilitatorclassroompracticesarebestdetectedthroughthequalitativeresearch.
Overall,scoresonthecompositeindexofeffectiveteacherpracticewerelowacrossallgroupsoffacilitators.Facilitatorswhometregularly(onceaweekormore)withtheheadteacherdemonstratedlowerlevelsofeffectiveteacherpracticesthanthosewhodidnot.Facilitatorsin
.0
20.0
40.0
60.0
80.0
100.0
Percentofclassrooms
Percen
toftim
eon
task
Lessthan50%
50-59.9
60-69.9
70-79.9
80-89.9
90-99
100percent
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Lessthan12minutes 12-20minutes 21-29minutes 30-36minutes
Bauchi Niger
EvaluationoftheGirl’sEducationProject3
©EDOREN 157
BauchidemonstratedslightlyhigherlevelsofeffectiveteacherpracticethanthoseinNiger.Therewerenoclearpatternsbyfacilitatorcharacteristics,exceptthatfacilitatorswhowereover50yearsoldperformedworsethantheiryoungercolleagues.
4.4.1.5 Facilitatormotivationandattendance
Facilitatormotivationwasinvestigatedthroughtheuseofarangeofmotivationscaledscores,whichformacompositemotivationindex.Thesubscalescoverthefollowingareas:
• perceivedteacherefficacy;• interestin,andenjoymentof,teaching;• effortputinto,andperceivedimportanceof,teaching;• pressureandwork-relatedtension;and• teacher-to-teacherinteraction.
Acomparisonofthedifferentmotivationsubscalesrevealsthat,onaverage,thesurveyedfacilitatorsscorehighestoneffortandlowestonperceivedteacherefficacy(Figure57).Motivationsubscalesforinterestin,andenjoymentof,teachingandeffortputinto,andperceivedimportanceof,teachingarequitehigh,whereassubscalesmeasuringpressureandwork-relatedtensionandperceivedefficacyappearquitelow.Thisseemstosuggestthatfacilitatorsconsidertheirroletobeimportantandthattheyenjoyworkingasteachers.However,thereseemstobeanindicationthatthefacilitatorsfeeltheirabilitytoimpactpupiloutcomesissomewhatlimited.Itisalsointerestingtonotethattheteacher-to-teacherinteractionscoreseemstoindicatearelativelyhighlevelofcollaborationamongstfacilitators.Thiscouldhelpstrengthentheeffectsoffacilitator-specificinterventionsattheschoollevel.Forexample,iffacilitatorssharetheirlearningfromtrainingwithoneanother,theeffectoffacilitatortrainingwillbemultipliedatthatparticularschool.However,itisworthnotingthattheteacher-to-teacherinteractionheremayimplyinteractionwithteachersofreligioussubjectsaswell,aswehavealreadyestablishedthat25%ofIQSsonlyhaveonefacilitator.
Thegraphbelowpresentsavisualanalysisofthedifferencesacrossmotivationsubscales.
Figure57: MotivationsubscalesforIQSfacilitators
1.9
3.64
3.8
1.98
3.5
0 0.5 1 1.5 2 2.5 3 3.5 4
Perceivedteacherefficacy
Interestandenjoymentofteaching
Effortandimportance
Pressureandwork-relatedtension
Teacher-to-teacherinteraction
EvaluationoftheGirl’sEducationProject3
©EDOREN 158
Theoverallteachermotivationindexwasconstructedasacompositemeasureofallteachermotivationsubscales,withequalweightingforallsubscales.Whilsttheaveragemotivationindexvalueacrossthewholefacilitatorsampleisnotparticularlyinformative,itisanalyticallyinterestingtolookatthedifferenceinmotivationscoresacrosscategoriesofinterestforouranalysis.Variablessuchasageandprofessionalqualificationarenotassociatedwithdifferencesinmotivationindexscores.Ruralfacilitatorsareonlyslightlymoremotivatedthantheirurbancounterparts.Animportantcorrelationisthatfacilitatorsthatmeetwithheadteachersindividuallyaremoremotivatedthanthosethatdonot,andthatgroupmeetingswithheadteachersdonotseemtobeanimportantdeterminingfactorofmotivation.
Facilitatorsthatreceiveasalaryorstipendfortheirservicesandfacilitatorsthathaveattendedatrainingsessioninthelasttwoyearsappeartobemoremotivated,asindicatedbyahighervalueforthecompositemotivateindex.Thisisnotentirelysurprising.Itis,however,interestingtonotethatfacilitatorsinBauchiweremoremotivatedthanfacilitatorsinNiger.ThisisnoteworthysincemorefacilitatorsinNigerarepaidthanarepaidinBauchi,andfacilitatorsinNigeraremorelikelytobetrained.Thissuggeststhatthereareotherconfoundingvariablesassociatedwithmotivationbeyondremunerationandtraining.
Facilitatorremuneration
FacilitatorsinNigeraremorelikelytobepaid.Inaddition,facilitatorsinschoolswheretheheadteacherandproprietorareseparatearemorelikelytobepaidteachers.Thisisinlinewiththeideathatinplaceswherethesetworolesareseparate,schoolsareoperatingmoreformally.Thoughremunerationisaninterestingavenuetoexplore,practically,averysmallshareofthefacilitatorsinthesamplewerepaideitherasalaryorastipend:only3%ofthesampleinBauchiand33%ofthesampleinNiger.Hencetheclaimsthatcanbemadebasedonthesedataarequitelimited.
Facilitatorattendanceandtransfers
Around75%ofthefacilitatorswereabsentatleastonceduringthelastthreemonths,withanaveragedurationofabsenteeismofapproximatelysevendays.FacilitatorsinBauchiwerefarmorelikelytobeabsent,andforlonger,thanfacilitatorsinNiger.Absenteeismrateswerelowestforteachersintheyoungestagebracketandhighestamongthe25–34agebracket.
Thegreatmajorityoffacilitatorsidentifiedtransportissues,theirownillnessortheillnessofafamilymemberasthemainreasonfortheabsence.Socialorreligiousobligationssuchasweddingsandfuneralsalsoemergedasdeterminingfactors.Theheadteacherdatarevealalmostidenticalreportedreasonsforteacherabsenteeism.Theaveragenumberofdaysheadteacherswereabsentinthesameperiodofthreemonthsisrelativelylow,ataboutsixdays.
Therewerenomarkeddifferencesintheabsenteeismratesforfacilitatorsbypaymentstatus.However,facilitatorsthatwereunpaidwereabsentforlongerthanthosethatwerepaid.Facilitatorsthatreceivednotrainingweremorelikelytobeabsent,andtobeabsentforlonger,thantrainedfacilitators.Aswehavediscussedabove,trainedfacilitatorswerealsomoremotivated.
Aroundaquarterofthesampledheadteacherswhodidnottakeanyactiontoimproveteacherattendancereportedthatteacherabsenteeismwasnotaproblemintheirschool.Whenexploredfurther,thedatarevealthatmostoftheseresponsescomefromtheBauchisample,where37%oftheheadteacherssaidthatteacherattendancewasnotaproblemintheirschools.Incontrast,only10%oftheheadteachersinNigermadesimilarclaims.ThisfindingiscontradictorytothefindingthatfacilitatorsinBauchiaremorelikelytobeabsent,andareabsentformoredays,thanteachers
EvaluationoftheGirl’sEducationProject3
©EDOREN 159
inNiger.ThismaybeexplainedbytheIQSsinBauchibeingmoretraditionalandmoreinformallystructured,andthereforefacilitatorattendancebeingmoreflexiblyorganisedandflexibleattendancebeingmoreaccepted.ThissuggeststhatadditionalschoolmanagementandleadershipsupportmaybeneededforIQSstransitioningfrominformal,flexiblemodelsofoperatingtoaschoolstructurewithmoreformalmonitoringandmanagement.
Record-keepingintheIQSSschoolswasquitelimitedandaround40%ofthesamplehadnoformofwrittenrecords.Thisaffectedseveralvariablesofanalysis,includingfacilitatorturnover.Awrittenteacherledgerorattendancerecordwasinexistenceinonly12%ofthesampledschools.Hencethefindingsbasedonsuchdataneedtoapproachedcriticallysincethereareincentivestomisreportthisinformationattheschoollevel.Throughthelimitedrecalldataavailable,facilitatorturnoverisestimatedtobearound20%asashareoftotalfacilitatorsinschool.Ratesofturnoverwerehigherinruralareasandininstanceswheretheheadteacherwastheproprietor.
4.4.1.6 Availabilityofteachingandlearningmaterials
Theavailabilityofteachingandlearningmaterialswaslimited–77%oftheclassroomsobservedhadnosuchresourcesavailable.Themostcommonlyavailableresourceswerethosemadebyhandbyfacilitators,followedbytextbooks(see
EvaluationoftheGirl’sEducationProject3
©EDOREN 160
Figure58).In78%ofobservedlessons,allavailableresourceswereusedduringthelesson.Thefactthatthisdidnotapplytoallofthelessonsobservedmaysimplysuggestthatsomeoftheresourcesavailablewerenotappropriateforthatparticularlesson.However,anotherpotentialinterpretationisthatfacilitatorsarenotalwaysawareofhowtomakethebestpossibleuseofavailableresources,andmayneedtoreceivesometraininginthisarea.Overall,thefindingssuggestthatthedistributionofteachingandlearningresourcescouldfillanimportantgap,providedthatthesearecarefullytailoredtoprevailinglevelsofknowledgeandskillsamongstfacilitatorsandpupils.
EvaluationoftheGirl’sEducationProject3
©EDOREN 161
Figure58: Availabilityanduseofresourcesinobservedlessons
4.4.1.7 Pedagogicalleadership
Attheschoollevel,informationwascollectedonheadteacherleadershipandmanagementthroughtheheadteachersurveyinstrument.Oneofthequestionsofinterestwastounderstandwhoattheschoollevelfulfilstheheadteacherrole:i.e.whoisresponsibleforday-to-daymanagementandsupervisionofteachers.Ourpre-pilotingexperiencesuggestedthatthispersonisnotalwaysreferredtoasthe‘headteacher’.ThedataconfirmtheideathatalthoughinmostIQSstheheadteacheriscalledassuch,insomecasestheheadteacherandproprietorarethesameperson,inwhichcasethepersonfulfillingtheheadteacherrolemaybereferredtoastheproprietor.Inone-thirdoftheIQSSsample,theheadteacherwasalsotheproprietor,andthiswasmorecommonforschoolsinBauchi,asalreadydiscussed.
Headteacherswerealmostalwaysmaleandweregenerallyolderthanteachers,withanaverageageof40years.However,thereareimportantdifferencesbetweentheaverageheadteachersinBauchiversusthoseinNiger.TheheadteachersintheBauchisampleareabout10yearsolder,onaverage,thanheadteachersintheNigersample.AroundhalfoftheNigersampleofheadteachersfallwithintheagebracketof25–34years,whiletheBauchisamplewasconcentratedinthe35–49yearsagebracket.HeadteachersinBauchihad10moreyearsofworkexperienceatthecurrentschoolcomparedtoheadteachersinNiger(ameanoffiveyears,versus15),inadditiontohavingmoreyearsofteachingexperience.ThoughheadteachersacrossbothstateshadsimilarlikelihoodsofhavinganNCE,headteachersinNigerhadhigheracademicqualifications,asindicatedbymorethanhalfofthemhavinganSSCEorhigher,comparedtoonly25%oftheBauchiheadteachers.Inaddition,agreatershareofBauchiheadteachershadareligiouseducation(over55%,comparedtolessthan20%inNiger).ItisnoteworthythatasmallershareofheadteachershadanNCE(23%)comparedtofacilitators(35%).ThisdifferenceisparticularlylargeinBauchi,where51%offacilitatorshadanNCE,comparedto27%ofheadteachers.Thismayaffecthowfacilitatorsperceivethecapabilityofheadteachersinregardtotakingonapedagogicalleadershiprole.
Incaseswheretheheadteacherdoubledastheproprietorthemeanagewashigher.Infact,45%oftheheadteachersthatwerealsoproprietorsoftheirschoolwereovertheageof50,whilethiswas
0.0
5.0
10.0
15.0
20.0
25.0
Video Audio
Toolsorobjectsfromthelocal
environment
Handmade MaterialsinHausa
Poster,chartorpictures Textbook
Resourcesused 0.0 0 3.8 19.9 1.0 1.9 8.8
Presentbutnotused 0.0 0 4.8 4.7 .9 1.9 1.9
Percen
tofo
bservedlesson
s
EvaluationoftheGirl’sEducationProject3
©EDOREN 162
thecaseforonly15%oftheheadteachersthatwerenotproprietors.Suchheadteacherswerealsomoreexperiencedasheadteachersandteachers,andtheyhadloweracademicqualifications;asmallerproportionhadanNCE.Theschoolsoftheseheadteacherswerealsoestablishedearlier,buthadfeweryearsofintegration.Thesetrendsparallelthetrendsfoundacrossthetwostates–whichisunsurprisingsinceBauchihadmoreheadteachersthatwerealsoproprietorsthanNiger.ThisisinlinewithotherevidencediscussedsofarwhichsuggeststhatBauchischoolsareclosertothetraditionalQur’anicschoolstructure,andlessintegrated,thanschoolsinNiger.
AlargershareofheadteachersinNiger(50%)hadreceivedsomesortoftraininginthelasttwoyearsthanheadteachersinBauchi(37%).TheheadteachersinBauchithatweretrained,however,weretrainedformoredays.Thecontentofthistraining,asreportedbytheheadteachers,alsovariedgreatly.ItappearsthattheheadteachersinBauchiwerereceivingmoretrainingonintegration,schoolmanagementandleadership,andgeneralteachingpracticesthanheadteachersinNiger.ThoughbothstatesreportedtrainingbyGEPandUNICEF,alargershareofheadteachersinBauchireportedtrainingconductedbySUBEBsandtheLGEAs,whichsuggestsgreaterinteractionwithgovernmentbodies.ThisisalsosupportedbythefindingthatagreatershareofschoolsinBauchireportedmonitoringvisitsfromagovernmentbodyduringthelastschoolyear(44%,versus33%).
Figure59: Contentofheadteachertraining,bystate
Headteachersthathadsomesortofprofessionaleducationalqualificationweremorelikelytobetrainedthanthosewithout,andthisdifferenceislargerthantheBauchi/Nigerdifference.SimilartrendsappearwhendisaggregatingthesamplebetweenheadteachersthathaveanSSCEandthosethatdonot.ThisdifferenceispartlydrivenbythefactthatmorequalifiedheadteachersarefoundintheNigersample.However,thiscouldalsoindicatethatinthepast,betterqualifiedheadteachersweremorelikelytohavebeenselectedfortraining.
Thesurveyinstrumentsmeasuredpedagogicalleadershipthrougharangeofvariables,suchasheadteacherlessonobservations,headteachermeetingswithteachersingroupsorindividually,andactionstakentoimproveteacherandpupilattendance.Throughoutthesepedagogicalleadershipvariables,thereisastrongassociationbetweenheadteachersbeingtrainedandimprovedleadershipandmanagementpractices.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Communityinvolvement
Integrationactivities
Developinginstructionalmaterials
Managementandplanning
Schoolleadership
Extracurriculars Curriculumsubjects
TeachinginHausa
Literacyandnumeracy
Teachingmethods
Bauchi Niger
EvaluationoftheGirl’sEducationProject3
©EDOREN 163
In40%ofthecases,theheadteacherobservedlessonsduringthelastterm.Thecross-statedifferencesuggeststhatagreatershareofheadteachersinNigerareobservingteacherlessons.Thisisvalidatedbythedatafromtheteacherquestionnaire:over80%oftheteachersinNigerreportedtheheadteacherobservingtheirlesson,whilethiswasonly65%forBauchi.However,noneoftheheadteachersinNigerkeptanywrittenrecordsoftheirobservations.Therearenostrongassociationsbetweenlessonobservationsandtheseparationoftheheadteacherfromtheproprietor,orbyagecategories.Trainedheadteachers,however,weremorelikelytocarryoutlessonobservations,andweremorelikelytohaveobservationrecords.
Inaddition,63%oftheheadteachersreportedtakingactiontoimproveteacherattendance,whilethiswas93%forpupilattendance.Therewerenonoticeabledifferencesacrossstatesorownershipstatusofschools(i.e.whethertheschoolisownedbytheheadteacherornot).Aroundaquarteroftheheadteachersreportedthattheydidnotneedtotakeanyactionsinceteacherabsencewasnotaproblemintheirschool.ThiswasgreaterforheadteachersfromBauchi,wherejustover35%oftheheadteachersreportedthis,comparedtoonly10%oftheheadteachersinNiger.Agreatershareoftrainedheadteachersreportedtakingactiontoimproveattendanceofteachersandpupilsintheirschools.Actionstakentoimproveteacherattendanceincludeddiscussingreasonsforabsencewithteachersandencouragingteacherstoattendschool.
Anotherindicatorofschoolleadershipmeasuredbythesurveywashowoftenheadteachersmetwithteachers,individuallyorinagroup,todiscussteachingandperformance.Thisincludedbothformalandinformalsessionsheldduringthelastcompleteterm(ApriluntilJuly,atthetimeofthesurvey).Onlyabout13%ofheadteachersdidnotmeetwithteachersatallduringthelastterm.ThiswasalotmorecommoninschoolsinBauchithaninNiger.Inaddition,individualmeetingswerelesscommoncomparedtogroupmeetings,acrossbothstates.Thechartbelowsuggeststhattrainedheadteacherswerefarlesslikelytonevermeetwithteachers,andweremeetingmorefrequentlywithteachers,thanuntrainedheadteachers.Similartrendsappearforgroupmeetingswithteacherswhendisaggregatedbyheadteachertrainingstatus.
Figure60: Frequencyofheadteachermeetingswithteachers,bytrainingstatusofheadteacher
3%
16%
3%
41%
38%
21%
8%
8%
37%
25%
0% 5% 10% 15% 20% 25% 30% 35% 40% 45%
Meetsteachersonceaweekormore
Meetsteachersmorethanonceamonth
Meetsteachersonceamonth
Meetsteacherslessthanonceamonth
Nevermeetsteachers
Training Notraining
EvaluationoftheGirl’sEducationProject3
©EDOREN 164
HeadteachersinNigermetmorefrequentlywithteachersindividually(andingroups)comparedtotheheadteachersinBauchi,asshownbelow.Thoughinbothcasesaround30%oftheheadteachersreportingnevermeetingwithteachers,headteachersinNigermetmoreoften.
Figure61: Frequencyofheadteachermeetingswithteachers,bystate
Thecontentofmeetingswithteacherswaslargelyconcernedwithpupilabsenteeismandprovidingsupporttoindividualstudent’sneeds,inadditiontoteachingpracticesandpedagogy.TheonlynoticeabledifferenceacrossstatesisthatinNigerteachingpracticesandpedagogyarediscussedfarmorethaninBauchi,whileinBauchithefocusisontheneedsofindividualstudents.
Sofar,thedatasuggestthattrainedheadteachersarefulfillingrolesofschoolleadershipandmanagementbetter,andthatheadteachersinNigerareperformingbetterthanheadteachersinBauchiacrossvariouscategories.However,intermsofformalrecord-keeping,suchasmaintainingupdatedattendanceandpupilenrolmentregisters,andwrittenlessonobservationrecords,headteachersinBauchiareperformingbetter.Thisdifferencemaybeassociatedwiththevariedcontentofheadteachertrainingacrossthetwostates,orwithgreatergovernmentoversightinBauchi.ThoughBauchiisperformingslightlybetterinthisregard,therearestillgainstobemadeacrossallIQSssincetheoveralllevelofrecord-keepingisquitelow.ThisfurtherhighlightstheneedforheadteachersinIQSstobetrainedin,andsupportedon,schoolmanagementandmonitoringactivities.
3%
10%
7%
47%
33%
19%
15%
4%
30%
31%
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%
Onceaweekormore
Morethanonceamonth
Onceamonth
Lessthanonceamonth
Never
Niger Bauchi
EvaluationoftheGirl’sEducationProject3
©EDOREN 165
Figure62: Availabilityofschoolrecords,bystate
4.4.2 Conclusionanddiscussion
ThediscussionsofarrevealsthatintegrationactiviteshavebeenimplementedinschoolsacrossBauchiandNigertovaryingdegrees.Moreoever,thelandscapeofIQSsiscomplex,andnouniformmodelseemstobepresent.Varyingschoolstructures,disparitiesinownership,aswellasdifferencesinthenumberandhoursofintegratedsubjectstaughtperschool,suggestthattheoutcomesofpupilsattendingtheseschoolsarelikelytodiffergreatlyfromoneanother.Onaverage,however,schoolsinNigerseemtomoreintegrated,withgreaterheadteacherpedagogicalleadershipandtraining,thanschoolsinBauchi.
Levelsoffacilitatorknowledgewereextremelylow,particularlyinregardtopedogogicalknowledge.Effectiveclassroompracticescoreswerealsolowforallfacilitatorgroups.Therewereveryfewresourcesavailabletobeusedbyfacilitators,withhand-maderesourcesbeingusedmorethananyotherresourcetype.Theanalysisalsofindsthatwhereresourcesareavailable,theyaregenerallyused.
Evidencesuggeststhatincreasingtheknowledgeandskillsoftheteacherwithintheclassroomandexposingchildrentoreadingmaterialsearlyinlifearetwoofthestrongestpolicy-amenablepredictorsoflearningachievement.Sowhilethoselowbaselinelevelsareconcerningandlimittheanalysistheyalsoprovideopportunitiesfortheintervention.However,expectationsshouldremainrealisticastherearealsosignificantbarrierstosubstantiallyraisingtheknowledgeandskillsoffacilitatorsatscale,asverylowlevelsofknowledgeandskillsareobservedatbaseline.
4.4.3 Qualitativecasestudyanalysis
ToassessContributionClaim1,‘GEP3’sIQSScontributestomoreeffectiveteachingoftheintegratedcurriculuminIQSs’,thequalitativestudyaimedtoexplorethequestion‘willfacilitatortrainingandmentoring,accesstofit-for-purposeteachingandlearningmaterials,andimprovedheadteacherpedagogicalleadershipandsupportleadtomoreeffective,gender-sensitiveteachingoftheintegratedcurriculum?”
0%5%
10%15%20%25%30%35%40%45%50%
AttendancerecordexistsforP2forcurrent
year
Updatedattendance
recordexistsforP2forcurrent
year
EnrolmentrecordexistsforP2forcurrent
year
Writtenrecordsofmeetings
kept
Teacherattendancerecordexists
TimetableprovidedforP2
Nowrittenrecordsprovided
Bauchi Niger
EvaluationoftheGirl’sEducationProject3
©EDOREN 166
Thesectionsbelowdiscussthequalitativefindingsatbaselinerelatingtothecausalpackagethatcontributestoeffectiveteachingofformalsubjects.TheanalysisdrawsondatafromthreeschoolsinBauchiState(lowerperforming,typical,higherperforming)andthreeschoolsinNigerstate(lowerperforming,typical,higherperforming).
4.4.3.1 Perceivedbenefitsandchallengesoftheintegratedcurriculum
Akeyassumptionbehindcontributionclaim1-thatfacilitatortrainingandmentoring,accesstofit-for-purposeteachingandlearningmaterials,andimprovedheadteacherpedagogicalleadershipandsupportwillleadtomoreeffectiveteachingofformalsubjects-isthatIQSstakeholders(proprietors,teachers,parents,pupilsandothercommunitymembers)willbesupportiveoftheintroductionoftheintegratedcurriculumintheQur’anicschool.Thiswilldependonthebenefitsandchallengesthatthestakeholdersperceiveinregardtointegration.Ifstakeholdersdonotacceptintegration,thiscouldposeabarrierinregardtotrainedfacilitatorsturningtheirimprovedknowledgeandskillsintoimprovedpractices.
Thesixschoolssampledforthequalitativestudyoperateindiversecontexts.Asaresult,theydisplaydifferentlevelsofawarenessof,andattitudestowards,formaleducation.ItisworthnotingthatschoolswillhaveacceptedintegrationandafocusupongirlsasaprerequisiteforanyGEP3support.Asaresult,GEP3IQSstendtodisplaypositiveattitudestowardsintegrationandgirls’education,eitherintrinsicallyorduetoresponsebias.
Theintegratedcurriculumappearstohavelargelygainedacceptanceinthecasestudycommunities.Whilststakeholdersacknowledgedtheinitialchallengesinwinningoverthecommunity,withsomeparentswithdrawingtheirchildrenfromtheschool,theintegratedQur’aniceducationnowseemstobeanintegralpartoftheschoolsinwhichitissituated.InBauchi—whereintegrationhasbeenmorerecent—itsembeddednesswithincommunitiesseemsmorefragile.IntheBauchitypicalIQS,afacilitatorreferredtosomemembersofthecommunitywhostilldonotsupportformaleducation,andmocktheIQSfacilitatorsasaresult.Overall,however,respondentshighlightedthegradualacceptanceofthevalueof‘Westerneducation’,describingthisasdueto‘enlightenment’and‘havingunderstoodtheworldinwhichwelive’.
‘Previously,manypeoplewerelookingatitasaformofdiversion[fromtheirtraditionalways],butgraduallyparentsthemselvesbecameawareofthesystemandstartedsendingintheirchildrentotheschool,particularlywhentheystartedrealisingthatpracticallychildrenaregettingeducatedbothways–theQur’anicandwestern.”(Mallam/headteacher,NigerhigherperformingIQS)
Thisshowsarecognitionthatbothreligiousandnon-religiouseducationareimportant,andthatneithertypeofeducationissufficientinisolation.Respondentsdescribedthetwotypesofeducationascomplementingeachother,enablingchildrentosucceedinallaspectsoflife:‘traditional’and‘now’,aswellasintheafterlife.AstheMallamfromtheBauchitypicalIQSexplained:‘Thereisonechildren’sIslamicbookcalled“NuruDalala”.ItteachesthatamanisrequiredtohaveIslamicandformaleducation.Withoutthetwo,itwillbelikeabirdflyingononewing,whichisimpossible.Soweneedtheintegrationofbothsothateverythinginlifecanbeachieved.’
Twoseparatecasestudyschoolsandvariousrespondentsusedtheexampleofthebirdwithonewingtoexemplifytheperceivedvalueofintegration.Somerespondentsdescribedthevalueof‘Western’educationasopeningupopportunitiesforemploymentandexperience.Inisolation,however,itwassuggestedthatthiseducationcanchangeachildsothattheybecometoomuchlike
EvaluationoftheGirl’sEducationProject3
©EDOREN 167
the‘Englishman’andlosetheirculturalrootsandvalues.Theseresponsespointtoatendencytoviewformaleducationas‘Western’education,whichresultsinongoinghesitationamongsomecommunitymemberstowardstheintroductionofformaleducation.Languageplaysanimportantrolehere:‘Boko’isusedinterchangeablytomeanWesterneducationandalsotorefertoformaleducationwhenneeded.134
Communitiesperceivethattheacceptanceofformaleducationhasincreased.Thisisowingtoacombinationof‘outsiderscomingandexplaining’(intheformofsensitisationcampaigns)andpeopleseeingthepositiveoutcomesofreceivingformaleducation.Seeingthatpeoplehaveendedupinhigher-statuspositionswhilstremaining‘devoutMuslims’isarguablyimportant.ThemainreasongivenfortheimprovementinacceptanceistheawarenessandunderstandingthatformaleducationcancomplementIslamicteachingswithoutde-valuingmoralandtraditionalaspects.Additionally,theperceptionisthattheneedformoreformalskills—suchascounting(forthemarketandotherincome-generatingactivities)andreadingletters(forprivacy,andthereforenotrequiringthehelpofneighbours)—isincreasing.
‘Iamverymuchcomfortablewiththeintegrationprocessinvolvingbothboysandgirlsbecauseteachingtheformalsubjectsisveryimportant.Forinstance,theexampleIcitedaboutbanking.Ifyouarenoteducatedintheformalsubjects,youhavetocallanotherpersontowriteforyouwhowillknowallyoursecretsandprivacybutifyouareeducated,youwilldoalltheseforyourself.Ifyouareaskedtofillinaparticularform,anditisnoticedthatyoudonotknowhowtofillit,youbecomeembarrassed.Ifyouareeducated,[thereare]nosuchchallengesonyourpart.”(Mallam/headteacher,NigerlowerperformingIQS)
Respondentsdescribededucatingachildinformalsubjectsasbenefittingbothafamilyandacommunityasawhole.Educatingachildensuresthatparentsandothersareabletoreceivehelpfromtheirchildreninreadinglettersandinothertasksrequiringcertainlevelsofformaleducation.Moreover,childrencantakeupemploymentinsectorssuchasteachingorinhospitals,thushelpingthecommunitytodevelop.
Keystakeholdersinallsixcasesstressedtheimportanceofgradualintegration.ItisessentialtobeconsiderateofthefearsthatparentshaveoftheIQSsresemblingpublicprimaryschoolstoocloselytherefore,negativelyaffectingreligiousteaching.Therearestillchallengesforintegration135andthoughKIs,includingparents,refertothebenefitsofthecombinationofthetwo,themajorityofstudyIQSsstillplacegreateremphasisonreligiouseducation.
‘Iftheschoolisrelocated,theparentswillcomplainsincetheyareyettounderstandtheintegratedsystem.Theywillfeeltheschoolhasbeenconvertedtoaformalschoolinsteadofthereligiousschoolthatitusedtobe;andthatwillleadtowithdrawalofchildren.Nowweareintegratingtheformaleducationgraduallytoavoidthat.’(CBMC,Bauchitypicalperformingschool)
However,casestudycommunitiesdonotseeintegratingtheformalcurriculumintoQur’anicschoolsastheonlywaytocombinereligiousandformaleducation.Inseveralcases,childrenareattendingbothprimaryschool(inthemornings)andtheIQS(intheafternoon):‘likenowthe
134‘Boko’isaHausawordmeaningboth‘education’and‘secular’.ItisalsousedtodescribetheRomanscript,andcanalsobetranslatedinto‘trick’or‘manoeuver’(HippocrenePracticalDictionary:Hausa-EnglishEnglish-Hausa).Itispossiblethat,unlessanothertermcanbeusedtopopularlydescribeformaleducation(withanarrativeofmodern,northernNigerianeducationatitsheart),theongoingtensionarounda‘loss’ofcultureandreligionwillremainachallenge.135Exploredthroughoutthequalitativeanalysis.
EvaluationoftheGirl’sEducationProject3
©EDOREN 168
majorityofpupilsherestillattendtheconventionalprimaryschoolinthecommunity,theygototheprimaryschoolandstillcometotheIslamiyya’(CBMC,BauchihigherperformingIQS).136ThisseemslessthecaseinIQSsthatareremotelylocated.InthecaseofoneofthemoreremoteruralIQSsinBauchi,communitieshadnoaccesstoaformalprimaryschoolatallandpupilsinterviewedindicatedthatfewattendedpublicprimaryschool.137ItwillbeimportantatmidlinetoexplorefurthertheperceivedbenefitsofintegratedQur’aniceducationinacontextinwhichpublicprimaryschoolingisreadilyavailable.
4.4.3.2 Facilitatorandheadteacherknowledge,skillsandteachingpractice
TheGEP3ToCarguesthattrainingandmentoringoffacilitatorswillcontributetotheirimprovedsubjectknowledgeandpedagogicalskills.Anunderlyingassumptionisthatthecapacitydevelopmentprocesstargetstheneedsofthefacilitatorsandheadteachers.Therefore,inthissection,weprovideaqualitativebaselineassessmentofcurrentknowledge,skillandpractice,whichinfluencetrainingneeds.
Facilitators’andheadteachers’backgroundandqualifications
TheleveloffacilitatorqualificationsvariessignificantlyacrossthesixIQScasestudies.InthehigherperformingIQSsinbothstates,facilitatorshavetheNCE;whereasinthelowerperformingIQSsfacilitatorshaveonlyjust,orhavenearly,completedsecondaryschool.InboththehigherperformingIQScases,thefacilitatorsarealsoteachinginpublicprimaryschools.Inallcases,facilitatorsarefromthelocalcommunity.
Thequalificationandteachingexperienceofheadteachersdifferssignificantlyacrossschools,andtheextenttowhichheadteachershavesufficientsubjectandpedagogicalknowledgetotakeontheroleappearstobedependentonwhethertheMallamhaspriorqualificationsornot.ThelevelofeducationamongstheadteachersrangesfromnoformaleducationtoholdingtheNCE138.InoneBauchischool,theheadteacher,whowasalsotheIQSproprietor,wasauniversitygraduate.Overall,thereisalackofclaritywithinthemajorityofIQSsregardingtherolesandresponsibilitiesofeachactor.SeveralrespondentsinitiallystatedthattheIQShasthreeseparaterolesofproprietor,Mallamandheadteacher,allheldbydifferentpeople.However,furtherprobingshowedthatthereisoftenonepersonwhoplaysallthreeroles.Insomeschools,theproprietoroftheschool—whoisalsooftentheMallamoftheQur’anicschool—appearstotakeontheroleoftheheadteacherastheIslamiyya/Tsangayaintegrates.Wherethereisaseparateproprietoroftheschool,theheadteacheroftheIQStendstobetheMallamoftheQur’anicschool.Thesub-section‘Schoolmanagement–actors,rolesandresponsibilities’willfurtherexplorethechallengesaroundsystematicallyidentifyingappropriateheadteacherstotakeonpedagogicalleadership.
Facilitatorsandheadteacherswhohavehigherlevelsofqualificationsfeelmoreconfidentteachingtheintegratedcurriculum.ThiswasnotrestrictedtofacilitatorswiththeNCE,butfacilitatorswhohadalevelofformaleducationbeyondsecondaryschool(suchasnon-teachingdegrees).FacilitatorswiththeNCEalsotendtoteachinpublicprimaryschools.IncaseswherefacilitatorsdonothavetheNCE,thereseemstobeacleargapinteacherknowledge,intheareaofsubjectandcurriculumknowledge—withteachersbeingunawareofwhatisrequiredtobetaught.
136Seefurtherdiscussionaroundpublicprimaryschools/IQSsandgirls’enrolment/retentioninthediscussionaroundContributionClaim3.137ItisimportanttonotethatduetosecurityconsiderationsthequalitativeresearchteamonlyvisitedrelativelyaccessibleIQSs.Hence,thestudywillnotbeabletorevealthedynamicsinremoteareas.138TheNCEisanA-Levelgradecourseintendedforsecondaryschoolleaversandtheirequivalents.
EvaluationoftheGirl’sEducationProject3
©EDOREN 169
Whilstfacilitatorsreportthattheyhavehadsometrainingonmethodsforteaching,orthattheyusetextbookstoplanlessons,theycanonlyteachwhattheythemselvesknow.
Facilitators’subjectknowledgeinthetypicalandlowerperformingIQSsisweak.Facilitatorsareunabletoexplainwhichsubjectsitisimportanttoteach,orwhytheyteachwhattheyteach.Moreover,duringtheQCOsitwasclearthatfacilitatorsthemselves,attimes,didnothavebasicliteracyskills,beyondrecitingthealphabet.Manyfacilitatorswereteachingatthelevelthattheywerethemselvestaught.Inthosecaseswheretheirowneducationwaslimited,theywereonlyabletopassonbasicreadingandnumeracyskillstotheirstudents.Facilitatorswhoalsoteachinpublicprimaryschools(mostlyfoundinthetwohigherperformingcases)oftenhavetheirsubjectspecialisations,whichtheythenalsoteachintheIQS.Intheotherfourcases,facilitatorseitherhavenotreceivedfurthereducationorhavenon-teachingdegrees,suchasaccountingorplumbing.ManyTeacherFacilitatorsarethusteachingsubjectswithouteitheradvancedsubjectknowledge,oranypedagogicaltrainingonimpartingthatknowledgetostudents.
‘Well,themajorthingsoffocusthatweteachthemareissueslikespellingsofbasicthingswhichtheycaneasilylearntodoeveniftheydidn’tseethebasicthingspractically.Iftheyknowhowtospellthethingsandtheyseethempracticallylaterthatwouldgivethemanideaofthemeaningofthatthing.’(Niger,lowerperformingIQS)
However,communitymembersoftenperceivefacilitators’subjectknowledgetobehigh.AsGloveretal.(1990)discuss,perceptioncanbedependentonthereferenceframeofarespondent.Thatis,arespondent’sperceptionofthesubjectknowledgeofteacherswillbedependentontheirownlevelofknowledge.ParentsintheIQScommunitiesoftenlackanyformaleducationandthoughtheyperceiveteacherstoteachwelltheyexpresshavingdifficultiesfollowingattimes:‘theyareteachingthemverywell.Youknowsometimes,itisdifficulttounderstand’(ParentsFGD,BauchihigherperformingIQS).Evenincaseswherefacilitatorsthemselvesdonothavesubjectspecialisationbeyondsecondaryschool,respondentsbelieveteachershaveasubjectfocus.TheperceptionofknowledgeisthushighlyinfluencedbytherelativelackofformaleducationwithintheIQScommunities.
Pedagogicalskillsandteacherpractices
InmanyoftheIQSsvisited(withtheexceptionoftheNigerhigherperformingIQS),whilepedagogywasnotstrongamongthefacilitators,facilitatorsdisplayedsomeunderstandingoftheimportanceoflessonplanningandstructure,withattemptsatdeliveringlessonsinsomekindofsystematicmanner.Forthemostpart,facilitatorswhowereabletoexplainhowandwhytheypreparedorexecutedalessoninacertainwaydidsoonasurfacelevel.Whilesomefacilitatorsidentifiedpedagogyasimportantforeffectiveteaching,therewaslessunderstandingofwhatitentailed:‘ifyoushowanythingwithstudents,ordoanyteachingthatthechildrencan’tunderstand,youmustteachverywell.Sothatifyouareteaching,thestudentswillunderstandtheteaching.’(Headteacher,NigertypicalIQS)
Facilitators’abilitytomakeuseofeffectivemethodsisrelatedtothedepthofsubjectknowledgethattheyhave139.IntheQCO,facilitatorsdemonstratedsomeknowledgeofeffectiveteachingmethods,suchasstatingthelessonobjective,askingquestionsaboutandreviewingthepreviouslesson,andusingparticipatorymethods,suchasgroupwork.DuringtheTPDs,facilitatorsalsoreferredtotheimportanceofchild-centredteachingandofactivelyincludingchildreninthelesson.However,whilstfacilitatorsdisplayedawarenessofteachingandpedagogicalmethods,inmostcases139AdediwuraandTayo(2007).
EvaluationoftheGirl’sEducationProject3
©EDOREN 170
facilitatorsthemselvesdidnotseemtohavesufficientsubjectknowledge.Itseemsthatsomefacilitatorshaveknowledgeofpedagogicaltechniquesandmethodswithoutadeepenedknowledgeofsubjectmatter.Thus,theyoftenlacktheabilitytoteacheffectively.
‘Weweretaughthowtopreparelessonplansandhowtouseflashcards.Backthen,thewayweusedflashcardisquitedifferentfromhowweuseitnow.Backthen,weholdflashcardlikethis(forlonger)tothepupilsbutnowweweretaughtthatfromthemeaningofflash,wearesupposedtoallowthechildrentohavejustaglanceandwecoverthecard.Andweweretaughtthatmostoftheknowledgeofourlessonshouldcomefromthepupils,theyshouldsaymorewhileasateacherIammorelikeaguardian.’(TPD,BauchilowerperformingIQS)
Themostcommonlyusedteachingmethodsarerepetition,recitalandaskingtheclassquestions.Thoughvarioustechniquesareused,lessonstructuringseemsadhocandverybrief,withfacilitatorsmainlyemployingthemethodstheyhavebeentoldtouseorwhattheyhaveexperiencedorobservedinpublicprimaryschools.Itseemsthatteachersdonotalwayshaveadeeperunderstandingofthevalueofcertainmethods,andarenotsurewhentouseaparticularmethod.OneofthefacilitatorsintheBauchihigherperformingIQSexplainedwhyhethoughtrepetitionwasanimportantteachingmethod:‘Iwantthemtounderstandbetter.IfIdonotrepeatitseverally,theywillnotlearnandifIrepeatjustonceortwice,mostofthemwillforgetitbytomorrowbutnowithasbecomepartofthem,likeasong.Sometimesyoufindthechildrensinging“Monday”andsoon.’AnotherobservationfromtheQCOswasthefacilitators’inabilitytosetmeaningfulclassworkandtomovearoundtheclassroominordertocheckandmarkpupils’progress.InallthreeschoolsinBauchi—eventhehigherperformingone—thefacilitatorsreferredtopupilsneedingtowriteintheirbooks,andmadeoverturesregardingcorrectingclasswork.Nevertheless,inpracticethisbehaviourwasnominal:itwasoftendonerightattheendoftheclass,withnorealtimegiventoexplainingcorrections.Asnotallstudentshadnotebooks,thepracticewasalsoinequitable.
ItisclearthatthelevelofeffectiveteachingvariessignificantlyacrossIQSs.Amongstthesixcasestudyschools,theNigerhigherperformingIQSstoodout.ThemajorityoffacilitatorshadanNCEandalsotaughtatapublicprimaryschool.Facilitatorsdisplayedahigherlevelofsubjectmatterknowledge,andconfidenceinusingpedagogicaltechniquesandeffectiveteachingmethods.IntheotherIQSs,facilitatorswhohavereceivedanyformoftrainingdodisplayacertainlevelofknowledgeregardingteachingmethods,butthisdoesnotseemtobecomplementedwithsufficientsubjectknowledge,whichaffectstheeffectivenessofthesemethods.Ingeneralthough,inallschoolssampled(andparticularlyinfiveofthesix)thereisacriticalneedforongoingteachertrainingandmentoringthatstrengthenbothpedagogicalskillsaswellassubjectknowledge.
Effectiveteachingoftheintegratedcurriculumwillalsodependontheinstructionaltimeallottedtoteachingofformalsubjects.Instructionaltimevariesbetweenlower,typicalandhigherperformingIQSs.Observedlessonslastedbetween10and45minutes,withthemostcommontimespentteachingbeing10–20minutes.Classesobservedincludedmaths,computerscience,English,andHausa.Thestudymayhaveinfluencedtheobserveddurationsinceclasstimeswerescheduledinordertoallowforobservation.However,pupils,facilitatorsandotherstakeholdersreportedthatlessonsnormallylastaround20minutes,andthatabouttwosubjectsaretaughtperday.Thisdidvarybetweencases,withhigherperformingIQSsreportingthatclassesnormallylastaboutanhour.ThequalitativedataarenotclearonhowfrequentlyallIQSsteachformalsubjects.ThoughsomeIQSsreferredtoformalsubjectsbeingtaughtfivetimesaweek,othersspokeoftwotothreetimesaweek.Still,apartfromintheNigerhigherperformingIQS(teachingfivetimesperweek),therewereinconsistencieswithinthecasesthemselvesregardinghowoftenanIQSteachesformalsubjects.
EvaluationoftheGirl’sEducationProject3
©EDOREN 171
Thereare,however,someindicationsthatclassesdonottakeplaceregularly,owingtofacilitatorabsenteeismandweatherconcerns.
Facilitatortrainingandsatisfactionwithacquiredskills
AstherehavebeenpriorroundsofinterventionsunderGEP,thequalitativedatacapturesomeperceptionsregardingGEP3’straining,andsatisfactionwithacquiredskills–evenatbaseline.TheIQScasesallreporthavingreceivedsomeformoftraining,althoughthebaselinedataarenotfullyabletocapturetheextentofthis.Moreover,itisnotalwayspossibletodistinguishGEPtrainingfromotherformsofsupportandtraining–somethingthatwillbeimportanttoconsideratmidline.
FacilitatorsinallIQScasesexpressedaninterestinreceivingtraining.GEPtrainingalreadyreceivedhas,accordingtorespondents,focusedonsensitisationandtheimportanceofformaleducationthatisinclusiveofgirls.InsomecasesitseemedthatfacilitatorshadreceivedtrainingaspartoftheCBMC,ratherthanreceivingfacilitator-specifictraining.Somefacilitatorsalsoreferredtohavingreceivedtrainingonhowtouseteachingandlearningmaterials,andtheimportanceofmother-tongueinstruction.Perceptionsaroundtrainingarethatyouaretaught‘tools’withwhichtoteachandvaluesrelatingtotheimportanceofinclusiveinstructionandformaleducation.
‘Iwastaughtonteachingaidsandhowtorelatebetterwiththepupilsbyputtingonasmilingfaceandalsoplayingwiththem,andalsonotusecanetobeatthem.’(TPD,BauchilowerperformingIQS)
Facilitators,however,refertoneedingtraininginformaleducation–tofurthertheirownknowledgeinordertobeabletoteach.Asdiscussedabove,theabilityoffacilitatorstousetheskillsacquiredintrainingeffectivelywillvarydependingontheirinitialknowledgebase.Iffacilitatorsdonothavesufficientsubjectknowledgetomakeuseoftheteachingmethodsthiswilllikelylimittheextenttowhichtrainingincreasestheeffectivenessofteaching,asteachersdonotknowwhenandwhydifferentteachingmethodologiesareappropriate.Thisisakintolearninghowtouseallthetoolsinatoolbox,withoutknowingforwhatpurpose,andwhen,touseeachone.
Itisimportantthattrainingiscontextuallydrivenandtargetedtothelevelofknowledgeoffacilitatorsandheadteachers.TherangeofqualificationsandknowledgeofformaleducationacrossthesixcasesislikelytohaveimplicationsforseveralGEP3outputs.ItfurtherhasimplicationsfortheabilityofaheadteachertotakeonapedagogicalleadershippositionwithintheIQS,asdiscussedbelow.
Inschoolswithmorethantwofacilitators,oneneedstoconsiderhowknowledgecanbedisseminatedandembeddedbeyondthetwofacilitatorstargetedforGEP3training.IntheBauchitypicalschool,forexample,theMallam/proprietorhadhimselfattendedthefacilitatortrainingandhadbeensharinghisknowledgewiththetwofacilitatorsunderhisremit.However,thelikelihoodofthishappeningconsistentlycannotbereliedon,asitisnotoftentheMallam/proprietorwhoattendsthetraining.IntheBauchihigherperformingschool,theinterviewswiththefacilitatorsdemonstratedthat,whileonefacilitatorhadattendedGEP3training,theotherwasnotevenawareofit.Notonlymaythisdampentheimpactofthetraining,itmayalsocauseresentment.Furthermore,socialrelationships(e.g.betweenyoungerandolderteachers)mayaffecthowmuchtransferofknowledgeactuallytakesplace.
EvaluationoftheGirl’sEducationProject3
©EDOREN 172
4.4.3.3 Gender-sensitiveteaching
GEP3seekstoimprovefacilitators’skillsingender-sensitiveclasspracticesinorderforteachingtobecomemoreeffective,particularlyforgirls.Thisassumesthatfacilitatorsareamenabletohelpingtoimprovegirls’education.Facilitators’perceptionsof,andattitudestowards,theeducationofgirlsandboyswilllikelyinfluencethis.
Perceptionsof,andattitudestowards,girls’andboys’education
Facilitatorsexpresspositiveattitudestowardsgirls’educationinthesixcaseIQSs.However,therelativehigherimportancetheyattachtoboys’comparedtogirls’educationpointstopersistingbiases.Asbrieflymentionedabove,therehavebeensensitisationcampaignsandtrainingaboutgirls’education140.Still,subtletiesemergefromthevariousdiscussionsinformingthequalitativecases.Forexample,althoughallfacilitatorsperceivegirls’educationasimportant,somefacilitatorsthinkthatgirlsarelessablethanboysare,andthatgirlsdonotrequireasmucheducationasboys.Thus,oneneedstoconsiderpositiveattitudestowardsgirls’educationasrelativetoattitudestowardsboys’educationandabilities.
Inallbutonecase(NigerhigherperformingIQS)facilitatorsperceivedboystobemoreintelligentthangirls.Somerespondentsconsideredthisdifferencetobesomethingthatcame‘fromGod’(Niger,typicalIQS).Furtherprobingonwhatwasmeantby‘intelligence’wasneeded,asrespondentssometimesusedthewordinterchangeablywith‘performance’.Itisnotalwaysthecasethatfacilitatorsconsidergirlsasinnatelyless‘able’thanboys.Somefacilitatorsareawareofthegenderedcontextinwhichtheyteach—bothinstitutionallyandculturally.
‘Thegirlsarenotasbrilliantastheboys.
Whydoyouthinktheboysareabletoperformbetter?
Because,backtheninthiscommunity,girlsarenotusuallyenrolledinschool.Youfindoutthatifparenthasfivegirlsandaboy,theywillgiveoutthatboytobeenrolledinschoolandleaveoutthegirls…theboyshavestayedlonger,thegirlsjoinedrecently.’
(TPD,BauchihigherperformingIQS)
InpupilFGDs,boyssaidthatformalsubjectsaremoreimportantforboysthantheyareforgirls.Ratherthansayinggirlswerenotasabletolearnformalsubjects,thereasonprovidedwasthatboysaremorelikelytoneedtheskillstaughtinformalsubjects,suchasbusiness,whereasgirlsaretobemarried(seefurtherdiscussiononearlymarriageandenrolmentinSection3.2).
Thoughgirlsthemselvessaiditwasimportantforthemtolearnformalsubjects,therewasadifferenceinaspirationsbetweenboysandgirlsinsomeschools.Aspirationscanbeagoodindicationofattitudesinacommunitysincetheyshowthegeneralframeofreferencepossessedbygirls.Inallschools,boysaspiredtobecomedoctors,teachersandbusinesspersons,ortojointhemilitary.Manygirls,whenasked,tendedtoshrugtheirshouldersandsaytheywantedtoget
140Researchbiasmayhaveaffectedresponsessincetheresearchteamwasassociatedwiththe‘GirlsEducationProject’.Assuch,oneneedstoconsidertheseexpressionsofpositiveattitudescarefully.Thelackofpriorstudiesandexperiencesfromthecasestudyschoolsalsomeantthattheresearchteamexercisedahighlevelofinitialcautioninregardtonotoffendingrespondents,whichpreventedmoredirectpersonalprobingaroundgenderattitudes insomecases. ‘True’attitudesandperceptionsofgirlsarethusdifficulttopindownatthisstage.Formidline,questioningmethodscanbedevisedbasedonthegreaterfamiliaritywenowhave,inordertoprobemoredeeply,whilststillbeingcautiousofsensitivities.
EvaluationoftheGirl’sEducationProject3
©EDOREN 173
married.Interestingly,thiswasnothomogenousacrossIQSs,butappearedtobelinkedtothelevelofschoolingavailable,andhowlongintegrationhadbeeninplace.Forexample,intheBauchilowerperformingschool(withnoformalschoolclosebyandonlyonefacilitator)girls’aspirationswerelowest.IntheNigertypicalIQS,thecommunityhasmadeitcompulsoryforallchildrentocompletesecondaryschoolbeforemarriage.Here,girlswantedtobecometeachers,doctors,businesswomenandmechanics.Thisisnottosaythatgirlswillbeabletocontinueontotheirpreferredprofessions.Inseveralcaseschools,somegirlsspokeoftheirdesiretogoontobecomeprofessionals,suchasmidwivesanddoctors,butwerealsoawarethatearlymarriagewilllikelytakethemoutofschoolandwillthusbeabarriertotherealisationoftheseaspirations.
Inclusiveinstruction
Theresearchpointedtovariousformsofgenderbiasinclassroomorganisationandinstruction.Insomeclassrooms,girlssatatthebackoftheclassroomwhilstboyssatatthefront,whileinothersgirlsandboyssatondifferentsides.Reasonsforthisseparationincluded‘Islamicteachings’,andassertionsthatthissegregationisessentialinorderforparentstoevenconsidersendingtheirchildrentotheIQS.
‘BecauseinIslam,itisnotgoodforthegirlstobeinfrontandtheboystoseetheirbacks.SothatistheshariaoftheIslamandthatiswhywedoit.’(TPD,BauchitypicalIQS)
Thisindicatesthatregardlessoftheacceptanceofanintegratedcurriculumwithintheschool,traditionalQur’anicschoolingpracticeswithinthegender-biasedclassroomarrangementprevailinthemajorityofthesampleschools,andattitudesinthisareaappeartoremainquiteentrenched.Inthoseschoolswherethegirlswereattheback,theirinclusionwithintheclasswasclearlymoredifficulttomaintain,althoughforthosegirlswhoweredeterminedtoparticipateitwasnotadeterrent.Theobservationsalsoindicatedthatboysseatedatthefrontweremoreeasilyabletoengage,andautomaticallyreceivedmoreofthefacilitator’sattention.Thereisanunderstandingamongstsomefacilitatorsthatsittingatthefrontoftheclassroomispreferential.Asonefacilitatorstated:‘evenintheformalschool…theyoungonesinfrontandtheolderonesatthebackbecauseoftheeaseofcopyingnotes.Ifyoumixthemup,boththeyoungonesandtheolderones,theyoungonesatthebackwillbeunabletoseethewritingsontheboard.’(TPD,BauchihigherperformingIQS)
Whilefacilitatorsseemtodemonstrategendersensitivityintheirinteractionwithpupils,genderedexpectationsaboutclassroombehaviourstillneedsfurtherattention.DuringtheQCOs,facilitatorswouldactivelyaskbothboysandgirlsquestions.ThepupilFGDsconfirmedtheseobservationsandbothboysandgirlssaidthatfacilitatorsallowanyonetoanswerquestions.However,respondentsoftenperceivegirlstobeshyerandquieterthanboysare,describinggirlsas‘lessinterested’or‘lessmotivated’.Thoughfacilitatorsreportthattheyalwaysactivelytrytoengagethesestudents,somegirlssaidthatteachershaveneveraskedthemaquestion,andthatgirlshaveneveransweredaquestioninclass.InsomeIQSsboyssaidthatgirlsneveranswerquestionsorpayattentioninclass,astheyareonlyinterestedinmarriage.InoneoftheseIQSs,girls,however,reportedthattheMallamgetsangryiftheyspeaktoomuchinclassandthatthefemalefacilitatorpreferstheboystoanswer.
Somefacilitatorsclaimthattheyfocusmoreongirlsinschoolsincegirlswillbethereforashortertimeand,assuch,needtolearnmoreinlesstime.Interestingly,thefactthatgirlsarelesslikelytocontinueontofurthereducation,orevencompletesecondaryschool,hasvariousoutcomesasregardsinclusiveinstruction.
EvaluationoftheGirl’sEducationProject3
©EDOREN 174
‘Fornow,wewantthegirlstolearnmorebecausetheydon’tgotoschoolforlong.Theyoftengetmarriedandsometimesrelocateafterthemarriagebut,fortheboys,theyarealwayswithusandcanfurthertheirstudiesanytime.Butforagirl,onceshehasgrownshestartsthinkingofgettingmarriedsotheboysstaylongerinschoolandsomeofthem(boys)wanttobecometeacherstoo,thatiswhyweareconcentratingonthegirls.’(TPD,BauchitypicalIQS)
Somegirlshavesimilarperceptions:
‘Thegirlsshowmoreinterestbecauseherethegirlisnotallowedtolearnmuch.Andwehavebeentoldtoacquireknowledgeforourownsakeandthesakeofourchildrenbecausesomeday,youmightendupteachingothersoratleastteachingyourownchildren.’(GirlsFGD,BauchitypicalIQS)
Similarly,differingperceptionsoftheabilitiesofboysandgirlsaffectthenatureofinstruction.However,insomecases,thisdoesnotleadtoformsofinstructionthatdisfavourgirls,butrathertoeffortstocombatperceiveddifferencesinperformancelevels.
‘Butifyourememberduringthegroupworkactivitywecombinedboththeboysandthegirls.Itisnotlikewefindthegirlsdisgusting,orwebelittlethem.But,fromourownunderstanding,theboyslearnfasterthanthegirlssothatiswhyduringgroupactivitywetrytomixthemupinordertoshareideas.ButifIgroupthemaccordingtotheirgender,andIgivethemworktodo,therewouldbechallengesandthechallengeisthattheboysperformbettersothatiswhywemixthemtoshareideas?’(TPD,BauchilowerperformingIQS)
Therearealsosomegenderedelementstopupilassessment.Somefacilitatorsdescribepittingboysandgirlsagainsteachotherthroughperformancecompetitions,bygroupingpupils:whichevergroupwins‘getspampered’–‘thiswillnaturallymaketheothergroupmoreserious’(CBMC,Bauchihigherperformingcase).
WithintheQCOstheseoftenconflictedperspectivesregardinggirls’educationcamethroughinthewayfacilitatorswouldtrytoincludegirlspracticallyaspartoftheclass.InthetwoBauchischoolswheregirlssatattheback,thefacilitatorswouldfluctuatebetweenfocusingontheboyswhowereclosertothem,tothensuddenlymakinganefforttoengagewiththegirlswhowereattheback.IntheBauchitypicalschoolandtheNigerhigherperformingschoolgirlsweremorelikelytoraisetheirhandsandensuretheirownengagement(therebyalsobeingmorelikelytobecalledtotheboardtocompleteasumetc.),whileinthelowerperformingschoolthegirlswerelargelysilent.
Somefacilitatorstriedtoredressanygenderimbalancesinparticipationbycallingonlyongirlstoanswercertainquestions.However,itisimportanttoconsiderthattheresearchteamobservingtheclassmayhaveinfluencedthisbehaviour.Itisworthconsideringthepitfallsofsegregatingboysandgirlsasregardstheirresponsestoquestionsinclass(e.g.whenafacilitatorasks‘onlytheboys’tostandupandanswer,andthen‘onlythegirls’tostandupandanswer),asthiscanservetoentrenchdifference.
Overall,thequalitativeresearchfoundthatgenderbiasesaffectclassroomorganisationandpracticesduringtheteachingoftheintegratedcurriculum,despitethedisplayofgender-sensitivetechniques,duetoentrenchedgenderedattitudesandexpectationsaboutthebenefitsofeducationforgirls,andculturalnormsabouthowgirlsandboysrelate.Itis,therefore,importanttodelivergender-sensitivetechniquesintheclassroomwiththesupportofincreasinglytransformedgender-sensitiveattitudesamongstthefacilitatorsthemselves.
EvaluationoftheGirl’sEducationProject3
©EDOREN 175
4.4.3.4 Facilitatormotivation
WhilefacilitatormotivationisnotadirectintendedobjectiveofGEP3’sIQSS,weassumeitisanimportantcontributingfactortomoreeffectiveteaching.Weconsidermotivationtodemonstrateitselfin,amongotherthings,theeffortthefacilitatorsmakeintermsofattendance,punctualityandtimededicatedtothework.Wefirstdiscusstheseaspectsandthecontextinwhichtheyoccur.Next,wepresentfindingsonsomeimportantfactorsaffectingmotivation,suchasremuneration,jobsatisfactionandsocialstatusattachedtoafacilitatorrole.
Ingeneral,thequalitativefindingsfromthesixIQSsindicatethatissuesofremuneration,socialstatusandcommunityrolesandrelationsshapefacilitators’motivation.TheIQScontextprovidesadifferentsettingforfacilitatormotivationascomparedtopublicprimaryschools.Sincefacilitatorsareoftenfromthecommunitiesthemselvestheyareoftenaskedbythetraditionalleader,ortheMallam,totakeuptheroleofschoolfacilitators.
Attendance,punctualityandinstructionaltime
Communitystakeholders(includingpupils)perceivefacilitatorstobededicatedandpunctual.Thequalitativestudyexploredreasonsforlossofinstructionaltime,andwhilstrespondentsbroughtuplackoffacilitatorsasakeyreasonforinsufficientinstructionaltimefacilitatorabsenteeismwasnotseenasakeyreason,butratherthelackoffacilitatorsinIQSs.
‘Theteachersarealwayspunctual,youseethatourteacherwiththeredcap,therewasatimehewashavingeyeproblembutthatdidnotstophimfromcomingtoteachus,heworeglassesandcame.Heisalwaysaroundtoteachmaths.HewaseventakentothehospitalinBauchitogettreatment.’(BoysFGD,BauchitypicalIQS)
FewoftheIQScasesseemtohavesettimetables,withformalsubjectstaughtasandwhenafacilitatorisavailable.Whilstrespondentsmentionedthatformalteachingtakesplaceforcertainperiodsoftime,forexamplefortwohourstwiceaweek,inpracticeteachingappearstohappeninamoreadhocway,withaclassfor20minutes‘betweenthehoursof4and6pm’.Thetimetableforformalteachingalsoappearslessstructuredthanthatforreligiousteaching.Thismakesitdifficulttogetauniformunderstandingofhowandwhenformalsubjectsaretaught,whichmakesitchallengingtomonitorformalsubjectclasses.
‘TheQur’aneverydayintheeveningaround4pm,exceptThursdays,whiletheformalsubjectsareusuallyonWednesdaysandSaturdays.Intheeventthattheformalteacherisunabletocome,wecontinuewiththeQur’an.TheformalteacherteachesfortwohoursandteachesEnglish,writing,reading,Hausaandmathematics.Heteachesoneortwosubjectsperday.’(Mallam,BauchilowerperformingIQS)
TheFGDswithpupilscalledattentiontotheroleoftheclassmonitor,whoteachesyoungerstudentsifthefacilitatorisnotpresent.ThiswascommoninseveraloftheIQSs,wherepupilssaidthattheyoften‘runtheclasses’.DuringtheQCOintheNigertypicalIQS,thefacilitatorstartedtheEnglishclassbyhavingchildrensingthealphabetonebyone.Hethenlefttheclassroomonandoffandsatoutsideorinthedoorwayoftheclassroom.Assuch,facilitatorsmaybepresentandstartaclassontime,butitisnotnecessarilythecasethattheyarepresentintheclassroomtoensureeffectiveinstructionaltime.
Out-of-schoolactivitiesandtheweatherfurtherdisruptinstructionaltime.ManychildrenattendingIQSshaveresponsibilitieshelpingonthefarmorathomewithincome-generating
EvaluationoftheGirl’sEducationProject3
©EDOREN 176
activities,suchashawking(seefurtherdiscussioninthesection‘Qualitativecasestudyanalysis’inrelationtoContributionClaim3).Thisadverselyaffectsinstructionaltime,asparentsarelesslikelytosendtheirchildrentoschoolifittakesuptoomuchofthechildren’stime.Asfarmworkisseasonal,theattendanceofpupilsissignificantlyworseduringharvest.141Similarly,incontextswheretheIQSislocatedinafarmingcommunity,thefacilitatorsarealsoinvolvedinagriculturalworkandthereweresomeindicationsthatfacilitatorswouldbelesslikelyto‘runschool’duringthistime.Additionally,manypupilsattendpublicprimaryschool,thusrestrictingIQSinstructionaltimetoafternoonsandevenings,whenQur’anicteachingsalsooccur.142
Someofthequalitativecaseschoolsdonothaveaphysicalschoolstructure,butteachoutside.Thismeansthatinstructionaltimeseverelydecreasesduringtherainyseason:‘itisonlyduringrainyseasonthatweareafraidtocomebecausewehavenoshadeandthewindsarestrong.Wedonotevencomeregularly’(BoysFGD,BauchitypicalIQS).Additionally,thehotsunintheabsenceofaroof(asintheBauchitypicalIQS)meansthatthepresenceofshadecouldalsodeterminethetiminganddurationofaclass.Evenincaseswherethereisaphysicalschoolstructure,floodingduringtherainyseasoncanpreventschoolsfromopeningorchildrenfromattending,asIQSsareoftendifficulttoaccess.Thequalitativeresearchalsoexploredsecurityasareasonforlossofinstructionaltime.Securityconcernswerelesspressingthaninitiallyassumed,althoughthismaybeduetotheinfluenceofsecurityconsiderationsinsampling.
Remuneration,jobsatisfactionandbeingvaluedbyothers
ThemajorityofstakeholdersinIQScommunitiesreportthatfacilitatorsarenotpaidfortheirwork,apartfromtokensgatheredbythecommunity.Facilitatorsreceiveanamountconsideredinsufficienttoliveon.
‘Infact,I/youcancallthemvoluntarybutbecausetheyareonlygivensomepaltryassistancecomparedtotheirneeds.Despitethefactthatmajorityofthemarewomentheyalsohavetheirownuniquedailyneedsforsurvival,whatwegivethemcannotbeequatedtoasalary,honestly.Itisjustasmallallowanceandtheyarenotworkingelsewhere.’(CBMCFGD,NigerhigherperformingIQS)
IQSstakeholdersacknowledgethatthereisaninherentinconsistency,withfacilitatorsexpectedtodeliverlessonsatthestandardofteachers,whilefendingforthemselvesfinancially.Thereappearstobeanadditionalgenderedcomponenttothisroleconflict,143wherebymalefacilitatorsarerequiredtoprovideforafamily,aswellastohaveacertainsocialstandinginordertomarry.Thus,manyfacilitatorsareinvolvedinotherincome-generatingactivities,suchasfarmingorteachinginthepublicprimaryschool,orareengagedinothersocialdevelopmentprogrammingactivities.ThishasimplicationsforinstructionaltimeandtheamountofeffortputintoteachingattheIQS.Asanexample,thequalitativeteamfoundthattryingtoarrangetheQCOandTPDaccordingtotheclassschedulefortheBauchilowerperformingschoolwasdifficult,asthe(sole)facilitatorinthatschoolwasengagedinanotheragency-sponsoredvaccinationdrivejustoutsideofthecommunity.Itis
141Theagriculturalcycleandfarmworkaffectbothboysandgirls.Whileboysaremorelikelytobeengagedinfarmlabouractivitiesatplantingandharvesttime,suchaslandpreparationandthreshing,girlsandwomenareoftenresponsibleforoff-farmandpost-harvestactivitiesrelatedtotheyieldsproduced,suchascropprocessing,storage,andfoodpreservationandprocessing.142TheonlyschoolwhereitwasnotpossibletoattendbothIQSandprimaryschoolwasintheBauchilowerperforming,wheretheresimplywasnoprimaryschoolavailablenearby.Laterinthereportwediscussissuesaroundpovertyandresidualresistancetoformalschoolingasabarriertoattendingpublicprimaryschool.143GetzelsandGuba(1957).
EvaluationoftheGirl’sEducationProject3
©EDOREN 177
importanttoconsiderthecaptureofthefeweducatedindividualswithinacommunitybyotherprogrammeswithinthewiderissueofinsufficientfacilitatorremuneration.
Bynotpayingfacilitators,theexpectationsplacedonafacilitator,andthedegreetowhichacommunitycanholdfacilitatorstoaccount,mightdecrease.Asisevidentfromthequalitativefindings,thisisapressingconcerninthecommunities:
‘Theteachersarenotpaidsalarysoweacceptwhatevertheyofferratherthantoallowthechildrentobeleftlikethatwithoutlearning.’(ParentsFGD,BauchitypicalIQS)
‘Whenteachersarenotpaidgoodsalaryitmakesthemnonchalantandthatdiscouragesthechild.’(ParentsFGD,BauchitypicalIQS)
Yet,remunerationclearlyisnotthemainmotivationforfacilitatorsteachingatanIQS.InallsixIQSs,facilitatorsweremembersofthelocalcommunity.Stakeholdersperceivedthistobethereasonwhytheyagreetoteachvoluntarily.ItisalsoworthnotingthatseveralofthefacilitatorshadpreviouslybeenpupilsundertheQur’anictutelageoftheMallamorhispredecessor(thiswasthecaseinfiveofthesixIQSs),possiblycreatingcontextsinwhichasenseofresponsibilitytothecommunityorevendutytoarespectedformerteacherareatplay.IntheBauchitypicalschool,thefacilitatorreferredtobeingcalledbyhisteacher(theMallam)tocomeandteachinhisschoolinorderto‘helpteachouryoungerones’oncehehadcompletedhishighschooldiploma.
FinancialconstraintsmakeitdifficultforIQSstoattractqualifiedfacilitators.InmanyoftheIQScommunitiesthemajorityofpeoplelackaformaleducationbeyondsecondaryschool.Facilitatorsarethus,‘childrentaughtbythiscommunity’,andinmanycasestheirsubjectknowledgeislow(asdiscussedabove).
‘Youknowgettingateacherfromoutsidethevillagethatisfarfromtheroadisdifficult.Wemustgetmoneyforfuelorbicycles.Wedonothavemoney.Sowegivelittlemoneytotrytogetsomeoneandtheyappreciatebecausetheyarealsoteachingtheiryoungerones.’(CBMCFGD,NigertypicalIQS)
Nevertheless,inspiteofthelackofremunerationfacilitatorsinIQSsreportfeelingintrinsicallymotivatedbywhattheydo:‘educationisimportantparticularlyforthegirlchildbecauseitwillhelpthemgreatlyinlife.ThatiswhyIwanttogiveitmybestasacontributiontoseetheyprogressinlife.’(Femalefacilitator,NigerhigherperformingIQS).Facilitatorscitetheirmotivationascomingfromwantingtohelpchildrenandtohelpthecommunitytodevelop,beinginterestedinteachingandthebeliefthatarewardwillcomefrom‘God’.
‘WegetourrewardfromthecommunityandfromGodinheaven.Thecommunityrespectandvalueus;whereverwegoboththeparentsandthepupilsrespectusandarehappytoseeus.’(Facilitator,BauchitypicalIQS)
Therearecontrastingaccountsoftheperceivedsocialstatusofbeingafacilitator,andthislinkstoissuesofremuneration.Thoughcommunitymembersrefertotheimmenseimportanceoftheteachingprofessionandhowtheywillberewardedinheavenforthissacrifice,facilitatorsreportnotfeelingvaluedduetotheirlowpay,withexamplesgivenofhavingbeendeniedagirl’shandinmarriage.Thefinancialchallengesmeanthatfacilitatorsstruggleto‘moveupalevel’inlife,ascomparedtotheirpeers:‘IhavetaughtforalmostfouryearsbutoneofmyfriendswhomIschooledtogetherwithinAzareisnowworkingwiththeairport.Heownsahouseandacar.Asforme,only
EvaluationoftheGirl’sEducationProject3
©EDOREN 178
thatmotorcyclethatyouareseeingoutside.Thedifferenceisclear.[IfIwas]notateacherIwouldhavepassedthislevel.’(TPDBauchihigherperformingIQS)
Thesamefacilitatorspokeoftherewardforateacherasbeing‘onlyinheaven’,aresponsethat—whencoupledwithhisexperienceofhavingalowstatusandbelow-parpayincomparisontopeers—indicatesapotentialchallengetothecurrentgoodwillandmotivationoffacilitatorswhobelieveintrinsicallyintheeducationalpurposeoftheirwork.Genderedexpectationsofmenwithinthesocietyasbreadwinnersplaysacriticalrolehere(asexemplifiedbyfacilitators’inabilitytosecureamarriage),andsuchpressuresinapredominantlymaleprofessioncannotbeunderestimatedwhenitcomesnotonlytothemotivationoffacilitators,butalsotoretentionwithintheprofession.Thereportentitled‘BeingaManinNigeria:PerceptionsandRealities’(2015)identifiesthesocialnormthatviewsmenas‘solebreadwinners’inNigeriansocietyasengenderingfeelingsoffailurewhenitisnotpossibletoliveuptothoseexpectations144.Meanwhile,Kelleheretal.(2011)identifiedconsistentlyinadequateteacherremunerationinmanycountriesas‘oneofthecorereasonswhymanymeneitherchoosenottoentertheprofessionorendupleavingitinthelongrun”145.
TheGEP3ToCassumesthatmotivationwillincreasewherefacilitatorsareabletoaccessallthetechnicalproficiencies,guidance,support,andmaterialsneededinordertobeagoodteacher.However,suchmotivation,anditstranslationintoteachingeffort,isinfluencedbytheverybasicandpracticalunderpinningofanIQSfacilitator’seconomicneedsandresponsibilities,andbyhowthesemayconflictwiththewell-intentionedcommitmenttoteachingtheintegratedcurriculumintheabsenceofadequateremunerationandthestatusthatthisismorelikelytobringwithit.Therefore,adequateremunerationisanimportantcontextualfactorthatislikelytoinfluencetheimpactpathwaytomoreeffectiveteaching.
4.4.3.5 Pedagogicalleadership
OneofthekeyintendedoutcomesofGEP3isenhancedschoolleadershipandmanagement.Theprogramme’sToCnotesthat,throughcapacitydevelopment,headteacherswillbeabletoleadschoolsandfacilitatorsmoreeffectively.Thissectionwillfocusprimarilyonpedagogicalleadership,whilstContributionClaim2discussesschoolmanagementmorebroadly.
Schoolleadershippositions(proprietor,Mallam,headteacherandCBMCmembership)arenotnecessarilyseparateroleswithintheIQScontext.AsdiscussedinthesectiononContributionClaim2,thereisalackofclaritywithinmostcommunitiesregardingwhohaswhatrole,andwhatthatroleentails.Whenasked,somerespondentsconsideredthateachIQShaseachmanagementposition.However,whenprobedfurtherthemostcommonscenarioseemstobeonepersontakingonallthreerolesofproprietor,Mallamandheadteacher.Inmanycases,thispersonalsoholdsaprominentpositionontheCBMC,suchaschairpersonorsecretary.Consequently,whiletherolesmayallexistwithinasingleIQS,thisdoesnotnecessarilymeanthattherearedifferentpeopleholdingeachrole.WhileapersontakingupseveralrolesinanIQSdoesnotnecessarilyaffectGEP3’sschoolleadershipandmanagement,lackofclarityabouttheserolescanmakeitdifficulttoidentifywhoseleadershipcapacitiestobuild.Differentactorsmaybeinvolvedintheimplementationofsimilarroles,andthemostqualifiedpersonmaynotleadontherole,forexample,ofpedagogicalleader(seebelow).
144VoicesforChange(2015)BeingaManinNigeria:PerceptionsandRealities,DFID.145Kelleheretal.(2011)WomenandtheTeachingProfession:ExploringtheFeminisationDebate,UNESCOandtheCommonwealthSecretariat.
EvaluationoftheGirl’sEducationProject3
©EDOREN 179
TheremaybesomevarianceinassigningrolesandresponsibilitieswhenitcomestodifferenttypesofIQS(bothpriortoandafterintegrating).AsAntoninis(2014)outlines,thereare,broadly,twotypesofreligiouseducationinstitutionsinnorthernNigeria:Qur’anicreligiouseducationinstitutionsandIslamiyyareligiouseducationinstitutions.Qur’anicinstitutionsencompassbothmakarantaallo(‘schooloftheslate’)andTsangaya.146Inbothinstitutions,theMallam(teacher)isinchargeoftheschool,andtimetablesareadhoc,ontheinitiativeoftheMallam.147Assuch,childrenareattimesabletoattendpublicprimaryschoolaswell.148TheTsangayaelementmeansthattheinstitutionhasamobileelement,inwhichtheMallammoveswithhispupils‘inthebeliefthatanitinerantlifeisessentialforthemtofullyconcentrateontheirstudy’.149SincetheMallaminTsangayasis‘incharge’ofthelocationoftheschool,beitinhishouseoronthemove,heislikelytoautomaticallybetheowneroftheschool.IslamiyyainstitutionsgobeyondmemorisingtheQur’anandincludeavarietyofIslamicsubjects,150suchasArabic,Islamichistoryetc.Islamiyyastakeamorestructured,stationaryformthanMakarantaallosandTsangayas,andtendtohavesetinstructionaltimes.151MallamsdonotnecessarilyownIslamiyyas.
MobilityoftheTsangayaschoolsislikelytointerruptteachingoftheintegratedcurriculumandmayaffectthegenderinclusivenessoftheschool.InoneTsangayaIQSinBauchi,theheadteacher(Mallam)saidthattheywerenotmobileanymore,duetothesecuritysituationinBauchi,whiletheboysintheFGDreferredtohavingbeen‘onthemove’andmadeclearthatfurthermobilitywasplannedforthefuture.TheMallamwouldthenmoveonlywithboys.Itisunclearatthisstagewhethertheformalsubjects’facilitatorwouldcontinuetoteachinthecommunityintheabsenceoftheMallamandtheboys.However,thereisahighlikelihoodthatthiswouldinterrupttheteachingoftheintegratedcurriculum,andthatgirlsmaybecomedisconnectedfromtheschool–thereforeaffectingtheinclusiveinstructionofthegirls.
ThecasestudyIQSswerereferredtoasTsangayainBauchiandIslamiyyainNiger,althoughallschoolsappearedtodisplaycharacteristicsclosertoIslamiyya.ThequalitativestudyfoundTsangayasreferredtoasschoolswithamobileelement,incontrasttothestationaryIslamiyyathatfollowamoreformalstructureconcerningtimetablesandteachingapproach.However,inpractice,therewerecomeinconsistencieswiththisterminology,particularlyinBauchi,wherealltheschoolswerecalledTsangayasbutalldisplayedslightlydifferentcharacteristics.Furthermore,inconversationrespondentsfrequentlyusedTsangayasinterchangeablywithMakarantaAllo.AllthreeBauchiTsangayasappearedtohave,onaspectrum,similarcharacteristicstoIslamiyyas,althoughnotalloftheBauchiIQScaseshadphysicalinfrastructure.Thehigherperformingschoolhadasolidphysicalstructure,moregirlsattendingschoolthanboys,andalargenumberofQur’anicteacherscoveringawiderangeofIslamicsubjects.OftheothertwoBauchiIQSs,onlyinthecaseofthelowerperformingschoolwastheresomeindicationthatitwasstillmobile,althoughthiswasinconclusive.InNigerstate,allthreeschoolswereIslamiyyas.Theschoolshadseparateformalstructuresinallbutonecase(NigerhigherperformingIQS),wheretheschoolwaslocatedinthehouseoftheproprietor.
RolesandresponsibilitiesvarysignificantlyfromoneIQStoanother.Thecommunitiesconsiderthepeoplewhofoundedtheschool,andwhoownandrunit,tobetheproprietors.IncaseswherephilanthropistsstartedtheIQS,orwherethecommunitydidsothroughdonations,theperceptionisthatschoolsare‘owned’bythecommunityanditisthuslessclearwhowouldbeactingas
146Antoninis(2014).147Ibid.148Ibid.149Antoninis(2014:83).150Ibid.151Ibid.
EvaluationoftheGirl’sEducationProject3
©EDOREN 180
proprietor.WhentheschoolhastransitionedfromaQur’anicreligiousinstitutionintobecominganIQS,theMallamislikelytobetheactingproprietor.TheMallamisinchargeofreligiouseducation.InallthesixIQSvisitedtheMallamisalsotheheadteacher,although,asinthecaseofNigertypicalIQS,he/shemaynotbethehighestreligiousauthorityinthecommunity.Inseveralcases,theMallam/headteacheralsoholdsaprominentpositionontheCBMC,suchaschairperson.Additionally,eachcommunityhasacommunityleader,which,inallsixcases,wasseparatefromtheabove-mentionedroles.Still,thecommunityleaderisnormallyabletoinfluenceschoolmanagementandisoftenpartoftheCBMC.
Theheadteacherroleismainlyconsidereda‘schoolleader’role.Asdiscussedabove,thequalificationsandteachingexperiencethattheMallamhassignificantlydiffersacrossthesixcases,rangingfromhavingreceivednoformaleducationtohavinganNCE.Theabilityofheadteacherstoprovidepedagogicalsupervisionandmentoring,andtheplausibilityofthis,isthusquestionableincaseswhereheadteachersdonothavepriorsubjectknowledgeorteachingexperience.
‘AllinallIspentabout18yearsinsearchofknowledgebutthroughQur’aniceducation,noformaleducationatall.But,Iwastaughta,b,c,dbyafriend,andnowIcanreadinHausa.IcanevenreadHausanewspaper.ButIhavenotgonetoschoolintermsofformal.’(Headteacher/Mallam,BauchilowerperformingIQS)
However,mostheadteachersinthequalitativestudyappearedconfidentintheirabilities.Assenioreducatorswithintheirestablishmentstheyviewedthemselvesashavingtheabilitytoinstructtheirfacilitators.ThenotableexceptiontothiswastheBauchilowerperformingcase.Headteachersfrequentlyconsultwithfacilitators,andgiveadviceonhowtoteachandinteractwithpupils.Advicerangesfromadviceonhowchildrenshouldsitintheclassroom,tohowtouseparticipatorymethods,dependingonthecapabilityoftheheadteacher.Pedagogicalleadershipseemstobesharedamongstmanystakeholders,withCBMCmembersguidingheadteachers,headteachersadvisingfacilitators,andproprietorsandcommunityleadersweighingin.Itwilllikelybeachallengetopinpointwhototrain,astheassignedheadteachersareoftennotthemostqualifiedintermsofformaleducation.152
‘He[theMallam]givesmeadvicebecausesometimeswhenIamteachinghecomesinandsitsdownjustthewayyoudidtoday,ifheseesanythingwrongthatconcernsthechildren,afterthelessonhewillaskandsaywhyareyourthingsdonedifferentlyfromours(Arabicstudies),doitthiswayandthenhewillgivemeadvice.Youseethisproprietor[Mallam]hasformaleducationtoo.Hedidnotgettofinishbuttheywerethefirsttoattendformalschoolinthiscommunity.Hecompletedhisprimaryschool,gottosecondaryschool,whichwastheonlyschool[backthen],buthedroppedoutofschoolinGrade4.Heisalwayshappywithwhatwedo,sometimesifmyattentionisneededelsewhereorurgently,hetakesoverthelesson.LikeifIamteachingalphabetsorsimplesentenceshewilljusttakearulerorstickandcontinuesthelesson.’(TDP,BauchihigherperformingIQS).
Asdiscussedabove,theappointmentofMallamsasheadteachersisnotnecessarilybasedonabilityandqualificationsbuthastodowithsocialstatuswithinthecommunityandperceptionsaroundleadershipmoregenerally.Inordertodiscussthevariousrolesandresponsibilitiesfurther,weneedtoclarifywhateachmanagementroleentails,andthustoidentifyneedsforcapacity
152Thehigherperformingcase inNigersignificantlydiffers fromtheother IQSs,aspreviouslydiscussed(seeannex).ThisIQShasaproprietorwithanNCE,andshehasclearpedagogicalleadership.ThereisaseparateMallam/headteacherwhodoesnothaveanNCE,andseveralfacilitatorswithNCEs–theheadteacheristhustheleastqualified;however,theremaybeagenderedcomponenttothis.
EvaluationoftheGirl’sEducationProject3
©EDOREN 181
developmentandtraining.Theability,perceivedcredibility,andpowertoenacttheseresponsibilitieswouldthusvarydependingonwhethertheheadteacherisresponsibleformonitoringwhetherteachinghappens,orwhethertheheadteacherisresponsibleforthedeliveryofqualityteaching.Pedagogicalleadershiprequiresknowledgearoundformaleducationandmethodsofinstruction,whilenormativeleadershipalignsmorecloselywithideasofvision,valuesandmoralleadership.153WithintheIQScontext,itmaynotbeplausibletoassignthesameindividualtheseresponsibilities.Mallamswillhaveahighsocialstandingandcredibilityintermsof‘whatisrightandwrong’morally,andthusholdanimportantpositionintermsofinfluencingthecommunityconcerningintegrationandgirls’education.Assuch,Mallamsmaybeassignedpositionsbasedonperceivedpoliticalandsocialvalue,ratherthanteachingknowledge:‘ThisschoolhasbeenaQur’anicschoolfor10yearsnow,butwhenwedecidedtointegrateformalsubjectswedecidedtoallowthesameMallamtocontinuetoteachthechildreninsteadofbringingsomeoneelsetoteach.’(CBMCFGD,NigertypicalIQS)
ForGEP3,thevariancebetweenthecapacityoftheheadteacher/Mallamintermsofconfidenceandqualificationsraisesthequestionofwhotoinviteforpedagogicalleadershiptraining.Itislikelythattheprogrammewillneedtotaketimetoaddressthisonanindividual(IQS)basis.Theapplicationofcertaincriteriaandcontextualissues–suchasplacingafacilitatorabovehischildhoodMallamwhenitcomestopedagogicalleadership–needscarefulconsideration.SimplyaskingtheIQStonominateandsendsomeonebasedonthetitleofheadteacher/Mallammaynotyieldpositiveresultswithoutprovidinganexplanationoftheexpectationsofthisroleintermsoftechnicalcapacity,andnotjustseniority/status.
Itmayalsobethatthereisnoclear‘candidate’forheadteacher/pedagogicalleader,asseeninsomeoftheIQScasesvisited,wherenooneinthecommunityhadanyformaleducationbeyondsecondaryschool.ThelowerperformingschoolinBauchiinparticularexemplifiesthis.InthisIQS,theMallamclearlydeferredtotheIQSfacilitatorinmatterspertainingtoformallearning,believinghimselfunabletoadvisehisfacilitator(achildhoodfriend,ofasimilarage,andwithwhomhehadmoreofapeerrelationship),becausehehadnotgonethroughformalschooling,whereasthefacilitatorhad.ItwasclearthattheIQSfacilitatorwasverymuchguidinganddirecting,himself.Astheonlyfacilitator,hecouldnotbeidentifiedasaheadteachertoprovidepedagogicalleadershiptoothers.Incontrast,theBauchitypicalcaseshowedtheclearestevidenceofasuccessfulpedagogicalrelationship,wheretheMallamhadhighereducationalqualificationsthanbothhisfacilitators(hewasauniversitygraduate)andwasalreadytrainingthetwoIQSfacilitatorsinpedagogicaltechniquesfromtheGEP3training.
Consequently,thecomplexrelationshipsandthechallengesinidentifyingclearheadteacher/facilitatorrelationshipsmayhaveimplicationsfortheGEP3ToCpathwaythroughwhichformalcurriculumpedagogicalleadershipandmentoringwilltakeplace.
4.4.3.6 Availabilityofteachingandlearningmaterials
GEP3supporttoIQSwillincludethedistributionofteachingandlearningmaterials,meanttocontributetoteachingthatismoreeffective.Below,wefirstdiscussthebaselineobservationoftheavailabilityanduseofteachingandlearningmaterialsinthecasestudyIQSs.Next,wediscusstheperceptionsofIQSrespondentsabouttheappropriatenessofthematerials.
153NSCL(2003).
EvaluationoftheGirl’sEducationProject3
©EDOREN 182
Availabilityanduseofteachingandlearningmaterials
ThecurrentavailabilityanduseofteachingmaterialsvariessignificantlybetweenthesixcasestudyIQSs.TheNigerhigherperformingIQShasaccesstoawidearrayofmaterials,includingexercisebooks,textbooks(forallsubjectstaught),variousteachingaids(flashcards,countingblocks)andevenfunctioningcomputers.OtherIQSshaveonetextbookfromwhichthefacilitatorsextractclassexercisesandrepeatlessons.InotherIQSstherearenotextbooksavailablebutfacilitatorslookthroughbooksinthepublicprimaryschooltoplanslessons.OneIQSmadeuseofstones,sticksandbottletopstocountinordertoallowchildrentovisualisemaths.AllIQSsvisitedhadablackboard;insomecasesthiswastheonlymeansofenablingchildrentoreadandwrite.
DuringQCOsitbecameclearthatmanychildrenlackbasicmaterials,suchasnotebooksandpens.Thisrequiresthefacilitatorstoassesshomeworkverbally.
‘Ithappensalot.Youfindapupilcomingtotheclasswithoutpensorbooks,soweusuallytellthemtotelltheirparentstobuythempensandbooks,andiftheycontinuetocometoschoolwithoutwritingmaterialswejustleavethem,thereisnothingwecando.’(TPD,BauchihigherperformingIQS)
AssomeIQSsvisitedhavenoabilitytoprovidepupilswithlearningmaterials,thisresponsibilityfallsontheparents.Thiscouldhaveanegativeimpactoninclusiveinstruction.Severalrespondentsconsidertheinabilityofparentstopurchaselearningmaterialsasareasonwhychildrenwillnotattendschool.
Perceptionsofteachingmaterialsfortheintegratedcurriculum
SomeIQSshavereceivedacurriculumforteachingHausa,andsomeflashcardsandotherteachingaids.AmongstthoseIQSsthathavereceivedsomematerials,respondentsperceivematerialstobeappropriateandcapableofaidinglearningamongpupils.Itwaschallengingtoprobethisissueofteachingandlearningmaterials,asatthisstagefewIQSshaveanymaterialsandassuchthequestionwasinterpretedas‘whatdowewant’.Hence,respondentsmainlyciteditemsneededbytheschool,rangingfromkettlesandmats,toexercisebooksandpencils.
Respondentsreferredtochallengeswithregardtotheuseofmother-tongueinstructioninthecommunitiessampled.Somerespondentspointedtothelackofmother-tonguematerials–inparticular,teachingandlearningmaterialsinHausaandtheHausacurriculum.However,inotherIQSs,Hausawasnotthemothertongueforall.Forexample,intheNigertypicalcase,thecommunitymainlyspokeNupe,withbarelyanyHausaspoken.InanotherIQS,althoughthemajorityofstudyparticipantsspokeHausa,therewerereferencestopupilswhodidnot.However,itcannotbeclaimedthatthequalitativecasestudiesarerepresentativenessofthewiderpupilpopulation.
‘It[training]wasgood,butourproblemhere,thosepeoplearoundheredon’tunderstandHausa.WeunderstandEnglish154andNupethatisourarea.IhaveonlyreceivedcurriculumforHausa.’(Headteacher/Mallam,NigertypicalIQS)
‘ThereasonwhywelikeHausa,seenowyouaskedthisquestionwithHausaandshedoesnotunderstandtheHausaright?ThatiswhywewanttolearnitverywellbeforetheystarttoteachusEnglish.’(FGDgirls,BauchilowerperformingIQS)
154Theheadteacher,whospokealevelofPidginEnglishandstruggledtocommunicatewiththeresearchteaminEnglish,saidthis,andthewidercommunitydidnotspeakmuchEnglish.
EvaluationoftheGirl’sEducationProject3
©EDOREN 183
‘WealsowishtheteachersuseMiya,theindigenouslanguage,toteachtheMiyamaswhodonotunderstandHausaandEnglishtopickup.’(ParentsFGD,BauchitypicalIQS)
Atpresent,thecompleteabsenceofqualityliteracyandnumeracylearningmaterialsinsomeschoolspresentsaclearbaseline.Theextenttowhichprovisionoffit-for-purposeteachingandlearningmaterialsisabletocontributetoeffectiveteachingisagainhighlycontextdependent,astheneedsofIQSsvarysignificantly.Moreover,teachingandlearningmaterialsneedtobefit-for-purpose,notonlyinrelationtocontent,butalsoinrelationtotheskillleveloffacilitators–assomefacilitatorsonlyhavebasicliteracyandnumeracyskills.
4.5 Analysisofthedata–ContributionClaim2:GEP3’sIQSScontributestoanimproved,girl-friendlylearningenvironment155
Inthissectionwepresentthebaselinedataonschoolmanagement,resourcemobilisation,andcommunitysupportforintegration.Inaddition,wediscussothercomponentsoftheIQSSToCthatareassumedtosupportmoreeffectiveteaching:thatis,thedegreeofgovernmentsupportandthegeneralschoolconditions.Thequantitativedataanalysisispresentedfirst,followedbytheanalysisofthequalitativecasestudyfindings.
4.5.1 Quantitativedata
4.5.1.1 Schoolmanagement
Roleoftheproprietor
Ofthe60schoolsintheIQSSsample,over90%hadaproprietor.ThissuggeststhatinmostofthecasestheIQSwasinfactownedbyanindividual,ratherthanthecommunity.Intheremainingcases,theIQSwascommunityowned,exceptinonecasewhereitwasownedbythegovernment.Almostalloftheproprietorsweremale.FemaleproprietorsonlyexistedintheNigersub-sample,whichisinlinewiththetrendsinfacilitators’genderobservedacrossthetwostates.
Inone-thirdofthesampledschoolstheheadteacherwasalsotheproprietor.Insuchschools,inadditiontoowningtheschool,theproprietorwastheoneresponsibleforday-to-dayschoolmanagement.ThisoperatingstructurewasmorecommonforschoolsinBauchithaninNiger.Asalreadydiscussed,schoolsinBauchiwereoperatinginawaythatwasclosertothe‘traditional’modelofanIQS.Thisfindingfurthersupportsthatnotion.
TheestablishmentoftheCBMC
CBMCswereonlyavailableforinterviewin52outofthe60IQSSschoolssurveyed.Insteadof30schoolsineachstatehavingaCBMC,28schoolsinNigerand24schoolsinBauchihadaCBMC.TheremainingschoolshadnotformedaCBMCyet.
ThereareinterestingdifferencesbetweenschoolsthathaveaCBMCandthosethatdonot.AlthoughthesamplesizeforschoolswithoutaCBMCisquitesmall,whichsuggeststhatweshouldbecarefulaboutextrapolatingthisevidence,thissectionpresentsabriefsummaryofthedifferences.SchoolswithaCBMCareapproximatelytwoyearsolder,morelikelytoteachallfiveintegratedsubjects,andspendmorehoursperweekteachingtheintegratedcurriculum.Suchschoolsarealsomorelikelyto
155Thisrelatestolinks4,5,6,7and10intheToC.
EvaluationoftheGirl’sEducationProject3
©EDOREN 184
haveseparatedtheroleofheadteacherandproprietor.ThissuggeststhatCBMCsweremoreprevalentinschoolsthathad‘deeper’integration,andnotnecessarilyjustmoreyearsofintegration.
InsomecasestheCBMCwasformedbeforetheyearinwhichtheQur’anicschoolintegrated,andthuscouldhavefunctionedasoneofthedriversofintegrationactivitiesattheschoollevel,whileinothercasestheCBMCwasformedafterintegration,possiblyasanoutcomeofbeingintegrated.Intheoverallsample,theCBMCwasformedbeforeintegrationin37%ofthecases.AlmosthalfoftheschoolsinNigerhadaCBMCbeforeintegration,whilethiswastrueforonlyaquarteroftheschoolsinBauchi.AlookattheyearssinceCBMCformationsuggeststhatinapproximately70%ofthecaseswheretheCBMCwasformedbeforeintegration,theCBMCwasformedtheyearbeforeintegration.
Onaverage,schoolshadhadaCBMConlywithinthelastthreeyears.SchoolsinNigerandschoolswheretheheadteacherwasnottheproprietor,hadhadaCBMCforlongerthanschoolsinBauchiandschoolswheretheheadteacherwasalsotheproprietor.
CBMCmembership
Onaverage,CBMCshad14members,withaquarterofthembeingfemale.Therepresentationofchildren,definedasanyoneundertheageof15years,wasverylow,asonly17%ofCBMCshadachildmember(andagirlchildmember).Asashareoftotalmembers,only2%ofalltheCBMCmemberswerechildren.ChildmembershipwashigherinBauchithaninNiger.InCBMCswithachildmember,therewereapproximatelytwochildmembersonthecommittee.Overall,however,therepresentationofchildrenwasquitesmallinabsolutenumbers,whichposesachallengeinregardhowtovoicetheconcernsof,andtorepresent,children–especiallygirls–inthemanagementoftheschool.
87%ofallCBMCshadatleastonefemalemember.BauchiCBMCsnotonlyhadmoremembers,buthadagreatershareoffemalemembers,byalmost10percentagepoints.Thisisinterestingtonote,giventhatfemalerepresentationinteachingandschoolownershipisextremelylimitedinBauchi.Thissuggeststhatalthoughtherearebarrierstofemalesworkingformallyinschools,femaleinvolvementwithintheschoolsinBauchiisnotnecessarilylesserascomparedtoNiger.However,membershipdoesnotimplyparticipation,sincethefemalememberattendanceduringthelastCBMCmeetingwasmuchhigherinNiger(48%)thaninBauchi(26%).Itisalsoencouragingtonotethatin80%ofthecases,theCBMCshadawrittenrecordofCBMCmembership,whichwasusedasevidencewhengatheringinformationonmembership.AgreatershareofCBMCsinBauchimaintainedtheseformalrecordsthaninNiger.
InabouthalfoftheCBMCs,theCBMCwaschairedbytheproprietororheadteacher.Intheremainingcases,theCBMCwaschairedbyaparent,religiousleaderorcommunitymember,wherethesewereseparatefromtheheadteacherandproprietor.SuchCBMCsweremorecommoninNigerthaninBauchi.HeadteachersandproprietorswereusuallymembersofCBMCs,eveniftheywerenotthechairs.Inabout80%ofthecasestheheadteacherortheproprietorwerepartoftheCBMC.OthermembersoftheCBMC,andtheirparticipationintheCBMC,aredepictedinFigure63below.
EvaluationoftheGirl’sEducationProject3
©EDOREN 185
Figure63: AveragenumberofmembersinCBMCs
ThissuggeststhatthereisconsiderablerepresentationofparentsandcommunitymemberswithintheCBMC,asintended.Inaddition,communitymembersfillingthechairorvice-chairpositionwithintheCBMCscouldsuggestthattheyhavepowerwithintheschoolsetting.ThenumbersindicatethatthereweremoreparentsintheCBMCchairandvice-chairroleinNigerthaninBauchi.
CBMCmeetingsandtrainings
Amajority(90%)oftheCBMCsreportedmeetingduringthelastyear,withanaverageoffourmeetingsperyear.Two-thirdsoftheCBMCswereabletoprovidesomesortofevidenceoftheirmeetings.Thisevidencerevealsthattheattendanceratewas41%ofthemembersatthelastCBMCmeeting.InschoolswheretheCBMCwasestablishedafterintegration,CBMCsweremeetingmoreregularly,andhadgreaterattendanceandgreaterfemaleattendanceatmeetings.BauchiCBMCsalsoappeartobemeetingalmosttwicethestipulatedamountofthreemeetingsayear.InNiger,thoughthemeetingswerelessfrequentcomparedtoBauchi,theattendanceratewashigher(46%inNiger,versus36%inBauchi).
Onaverage,approximately60%ofCBMCshadreceivedsomesortoftrainingduringthelasttwoyears.Similarlytotheprevalenceofteacherandheadteachertraining,agreatershareoftheCBMCsinNigerreceivedsomesortoftrainingcomparedtotheCBMCsinBauchi.InNiger,70%oftheCBMCshadreceivedtrainingoverthepasttwoyears,whileinBauchithiswasonlyslightlyabove50%.Ineachstatethetraininglastedanaverageoffourdays,thoughonaveragethenumberoftrainingsessionsattendedbyCBMCmembersinNigerwasslightlyhigherthanthenumberoftrainingsessionsattendedbyCBMCmembersinBauchi.AsreportedbytheCBMCchair,thetrainingcontent,asummaryofwhichispresentedinFigure64below,didnotvarygreatlyacrossstates.
0
2
4
6
8
10
12
14
16
TotalmembersofCBMC
TotalfemalesinCBMC
TotalchildreninCBMC
NumberofteachersinCBMC
NumberofparentsinCBMC
Numberofoldorcurrentpupilsin
CBMC
Numberofreligiousleadersin
CBMC
EvaluationoftheGirl’sEducationProject3
©EDOREN 186
Figure64: ContentofCBMCtraining
MostoftheCBMCmembersreportedbeingtrainedonCBMCresponsibilities,wholeschooldevelopmentplanningandcommunitysensitisationaboutgirls’education.ThemostcommonsourceoftrainingdeliveredinboththestateswasGEPandUNICEF,ataround65%.InbothstatestherewerealsoreportsofCBMCsbeingtrainedbyLGEAsandSUBEBs,thoughthiswasgreaterforBauchithanNiger.AverysmallshareinNigerreportedtrainingbySAME.
4.5.1.2 Schoolmanagementpractice
AkeyroleoftheCBMCistodeveloptheWCDP,andtomonitorprogressagainstit.ThiswasalsoindicatedasbeingakeyareaonwhichCBMCsweretrained(Figure64).Overall,onlyabout20%oftheCBMCshadcompletedaWCDP,andanother20%wereintheprocessofdevelopingaWCDP.OftheCBMCsthathadacompleteplanavailableorwereintheprocessofdevelopingaplanonlytwo-thirdshadawrittenformoftheplanavailable,andone-thirdcouldprovidesomekindofwrittenevidencetodemonstrateCBMCinvolvementindevelopingtheWCDP.Intheremaining60%ofCBMCsnoWCDPexisted.Differencesbetweenstatesseemminor,whileproportionallymoreIQSslocatedinruralareashadacompletedWCDPcomparedtoIQSslocatedinurbanareas.However,smallsamplesizesleadustocautionagainstanyhardconclusionsaboutsuchdisaggregatedanalysis.ThemainactionpointmentionedintheWCDPwasthebuildingofnewinfrastructure.
TheevidenceonfinancialmanagementbyCBMCmembersislimited.WeattemptedtoverifyanyclaimsmadebytheCBMCmembersbyaskingforsupportingevidence.Only27%oftheCBMCsreportedusingacashoraccountsbook,ofwhichlessthanhalfhadanupdatedcashbookavailable.QuitealotofCBMCs(about70%ofthesample)usedabankaccounttomanagefunds.Inmostofthesecasestheaccountwasinthenameoftheschool,followedbyitbeinginthenameofaCBMCmember(inapproximately27%ofthecases).Inlessthan3%ofthecasestheaccountwasintheheadteacher’sname.Only40%oftheCBMCsindicatedthattheykeptrecordsofbankdepositsandwithdrawals,ofwhichonlyhalfcouldshowanyrecordsfromthelastyear.Theuseofacashbookorbankaccountwasmorecommoninurbanareas,perhapsduetoeasieraccesstofinancialservices.IncaseswheretheCBMCwaschairedbyacommunitymembertheCBMCsweretwiceaslikelytouseabankaccounttostorefunds,ascomparedtoCBMCsthatwerechairedbytheproprietororheadteacher.ThisevidencesuggeststhatCBMCsneedfurthertrainingandsupporttoimprovetheir
0% 10% 20% 30% 40% 50% 60% 70% 80%
CBMCresponsibilities
Enrolmentdrives
Communitysensitisationaboutgirls'education
Wholecentredevelopment planning
Communitymobilisationforschoolplanning
Resourcemobilisation
Financialmanagement/accounts
Communitysensitisationaboutboys'education
Lessonobservationsandteachingmethods
Schoolenvironment
Schoolintegrationtechniques
EvaluationoftheGirl’sEducationProject3
©EDOREN 187
financialmanagementpractices,whichisunsurprisingsinceonlyaboutaquarteroftrainedCBMCsreportedbeingtrainedonfinancialmanagement(Figure64).
CBMCmemberswerequiteextensivelyinvolvedinschoolmonitoringactivities.AlmostallCBMCchairsindicatedhavingvisitedtheschooltomonitoritsperformanceduringthelastterm,withtheaveragenumberofreportedvisitsbeingseven.Thefrequencyofmonitoringishigherwhenthechairistheheadteacher,butingenerallargedeviationsarenotobservedwhenthechairisacommunitymember.Furthermore,over80%ofCBMCsreportedlymonitoredpupilandteacherattendanceandtookactiontoimprovepupilandteacherattendanceduringthelastterm.Thisholdstruewhendisaggregatingbystate,trainingstatus,andbytheCBMCbeingchairedbyacommunitymember.Inaddition,oftheCBMCsthathadaWCDPorwereintheprocessofdevelopingone85%reportedmonitoringactivitiesagainsttheWCDP.ItisimportanttopointoutthatinalmostallthecasesthevisitsoftheCBMCmemberstoschoolswerenotrecordedinthevisitors’bookoranyotherlog.Hence,thesevisitscouldnotbeverifiedbyevidence.
TherewasgreatervariationinthemonitoringactivitiesofCBMCmembersregardingqualityofteaching.ItappearsthattrainedCBMCmembersmonitorteachingqualitymoreoftenincaseswheretheCBMCchairistheheadteacherorproprietorthanwhenthechairisacommunitymember.In92%ofthecasesinwhichtheheadteacherorproprietorwasthechairorco-chairteachingqualitywasmonitored.Thiswasonlyhappening82%ofthetimeincaseswheretheCBMCwaschairedbyacommunitymember.MonitoringofteacherqualitywasmoreprevalentinNigercomparedtoBauchi,despiteBauchihavingmoreCBMCschairedbytheheadteacherorproprietor–perhapsbecausemoretraininghadtakenplaceinNiger.
TheCBMCmembersreportedmonitoringteacherattendancebyvisitingtheschoolandclassesduringlessons.Actionstakentoimproveteacherattendanceincludeddiscussingreasonsforabsenteeismwithteachersandmotivatingthemtobepresentregularly.Forpupilattendance,CBMCmembersreporteddiscussingreasonsforabsenteeismwiththepupilsandparents,andencouragingparentstosendtheirchildrentoschool.Itisinterestingtonotethatsecurity-andtransport-relatedissueswerenotmentionedatallduringanyofthediscussionsaroundabsenteeism.ActionstakenbyCBMCmemberstoaddresspupildropoutlargelyentailedahighlevelofinvolvementwiththecommunity,asindicatedbythedirectcommunicationwithparentsandmeetingswithcommunitymembers.
EvaluationoftheGirl’sEducationProject3
©EDOREN 188
Figure65: ActiontakenbyCBMCstoaddresspupildropout,bystate
4.5.1.3 Resourcemobilisation
CBMCsaremeanttocontributetothemobilisationofresourcesforschoolimprovementandtheimplementationofschoolactivities.Thedatashowthat65%oftheCBMCsreportedmakingeffortstomobilisecashtosupportschoolimprovementduringthelastschoolyear,ofwhichover80%successfullymobilisedfunds.InruralIQSsproportionallymoreCBMCstriedtomobilisecashcomparedtourbanIQSs.IQSschairedbycommunitymemberswerelesslikelytomakeeffortstomobilisefunds,butequallylikelytosuccessfullymobilisefundsamongtheCBMCsthatmadeaneffort.ThereseemtobelittledifferencesinsuccessfulcashmobilisationdependingonwhoistheCBMCchair.CBMCsinBauchiappearsomewhatmoresuccessfulinmobilisingcashthenthoseinNiger,thoughthesamplesizeforthisisquitesmall.AlloftheCBMCsthatweresuccessfulinmobilisingcashreportedmobilisingcashfromthecommunity,withaverysmallsharereferringtocontributionsfromthegovernment,NGOsandotherexternalagencies.
TheamountmobilisedbyCBMCsduringthelastyearvariedgreatlyacrossschools.Theabsenceofwrittenrecordsforthismoduleofthesurveymadeitchallengingtocollectcomplete,accurateinformationattheschoollevel.50%oftheCBMCsthatwereabletomobilisefundsraisedlessthanNGN16,500duringtheyear.TherestraisedbetweenNGN20,000andNGN50,000.Therearetwooutliersinthisdataset,wheretheCBMCraisedNGN151,000andNGN285,000duringthelastschoolyear.ThemedianamountraisedwasNGN20,000.Thoughresourcemobilisationfromthecommunityisproductive,whencomparedagainstthemagnitudeofthefinancialcontributionofschoolgrants(seebelow)itisclearthatthefundsmobilisedbytheCBMCsarenotasignificantpooloffundingforschoolimprovementactivities.Thissuggeststhatthereisstillroomtoaugmentcommunityinvolvementwithgovernmentsupport.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Communitysensitisationbygoing
doortodoor
CBMCorganisedmeetingswith
communitymembers
Directcommunicationwithparents
Pupilattendancemonitoring
Pupilcounselling
Bauchi Niger
EvaluationoftheGirl’sEducationProject3
©EDOREN 189
Figure66: AmountmobilisedbyCBMCs
Inadditiontoresourcesraisedthroughactivefundraising,around20%oftheCBMCsalsoreceivedfundsfromvarioussources.ThemostcommonsourcesoffundsreceivedwereUNICEF,SAMEorcommunitymembers.ThequestionnairealsoaskedCBMCmembersdirectlyaboutthereceiptofaGEPorUNICEFschoolgrant.Thefigurebelowshowsthatabout40%oftheCBMCsinBauchireportedhavingreceivedaGEPschoolgrantduringthelastschoolyear,comparedtojustunder30%oftheCBMCsinNiger.TheamountsthatwerereportedlyreceivedbyCBMCsweremostlyNGN150,000orNGN120,000;theformerwasmostlyrecordedinNiger,whilethelatterwasrecordedinBauchi.ThemaximumamountreportedwasNGN285,000,whiletheminimumamountwasNGN6,500.Thisincludesfundsreceivedfromallsources,thoughthelargestsharestemsfromUNICEForGEPgrants.
Figure67: ReceiptofGEPschoolgrant,bystate
0
50000
100000
150000
200000
250000
300000Am
ount(N
aira)
42%
28%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Bauchi Niger
EvaluationoftheGirl’sEducationProject3
©EDOREN 190
Overall,CBMCsinBauchiraisedmorefundingthanCBMCsinNiger.TheshareoffundsraisedfromthecommunityasashareoftotalfundsraisedwasalsohigherinBauchi.ThissuggeststhattheBauchiCBMCsweremoresuccessfulinmobilisingresourcesfromthecommunity.However,CBMCsinNigerhadaccesstomorefundingoverallthanCBMCsinBauchi,afteraccountingforfundsreceived.Secondly,almost30%oftheCBMCshadaccesstonofundsduringthelastyear,eitherduetonoeffortsbeingmadebytheCBMCtoraisefundsorduetounsuccessfulefforts,coupledwithnofundsreceivedfromanysource.ThissuggeststhatasevereresourcegapexistsinalargeshareofIQSs.Finally,around40%oftheCBMCsalsomobilisednon-cashresources,whichcameexclusivelyfromthecommunity,andweremorefrequentlyutilisedinBauchithaninNiger.
4.5.1.4 Communityinvolvementandsupportfortheschool
TheevidencediscussedsofarsuggeststhatthereiscommunitysupportforIQSs.ThisisexhibitedbycommunitymembersbeingpartoftheCBMCandchairingtheCBMC,aswellascommunitymembersdonatingmoneyandotherresourcesforschoolimprovementactivitiesthroughtheCBMC.Furthermore,CBMCmembersarereportedlyextensivelyinvolvedinschoolmonitoringactivities.
TheCBMCsseemtoactivelyreachouttothecommunity.Over80%statethattheyhaveraisedawarenessaboutthevalueofnon-religiouseducationinthecommunityduringthelastschoolyear,60%ofwhichreportedlyorganisedmeetingswithcommunitymembers,39%haddirectcommunicationwithparentsand26%organisedmeetingswithreligiousandtraditionalleaders.Aroundthree-quartersofCBMCsindicatedthattheyhaveraisedawarenessaboutthevalueofnon-religiouseducationspecificallyforgirlsinthecommunity.
CommunityfinancialsupportwasgreatestinschoolswheretheCBMCwaschairedbyacommunitymemberwhowasnottheheadteacherorproprietorattheparticularschool.SuchCBMCsmobilisedagreateramountfromthecommunity(anaverageofNGN61,000,comparedtoNGN21,000),mobilisedalargershareoftotalfundsfromthecommunity(100%versus97%),andwererecipientsofmorefunds(averageofNGN154,000,versusNGN88,000).Theywerealsomorelikelytohaveaccesstofundsduringthelastyear,andweremorelikelytohaveutilisednon-cashresourcesraisedfromthecommunity.
However,morecommunityinvolvementwithintheCBMCisnotalwayscorrelatedwithimprovedactivitiesattheschoollevel.Forexample,whentheproprietororheadteacheristhechairthereseemstobeahigherlikelihoodoftheCBMCmeeting,aswellasahigherattendancerateatthemeetings.Inaddition,althoughCBMCsareatleastaslikelytomonitorteacherandpupilattendanceiftheyarechairedbyacommunitymember,CBMCschairedbytheheadteacherorproprietoraremorelikelytotakeactiontoimprovepupilandteacherattendance.Whenaskedaboutthereasonforinactivity,CBMCchairslargelyreportedthatteacherandpupilattendanceisnotaproblematthisschool.AlthoughtheCBMCschairedbyacommunitymemberdonotappeartofaceinstitutionalchallengesinbeinginvolvedwithschoolmanagementactivities(asindicatedbythefrequencyofmonitoringactivities)theydoappearlessequippedtotranslatemonitoringintoaction.BuildingcapacityinthisregardshouldbeapotentialfeatureofCBMCtrainingundertheprogramme.
4.5.1.5 Schoolenvironment
Schoolenvironment
Theschoolenvironmentplaysanimportantroleinthelearningexperienceofthepupils,aswellastheteachingexperienceofthefacilitators.Weassesstheschoolenvironmentforchildrenalongthe
EvaluationoftheGirl’sEducationProject3
©EDOREN 191
linesof‘soft’infrastructure,whichincludesgroupsforpupilstointeractwithoneanother,alibrary,andaplayground,aswellas‘hard’infrastructure,whichreferstophysicalinfrastructure.
TheIQSsinoursamplearealmostuniversallydeficientintermsofsoftinfrastructure.Only3%oftheschoolshadalibrary,definedas‘acollectionofbooksotherthancoursebooks’,withnosharpcross-statedifferences.Onlyaboutone-thirdoftheschoolshadaplayground.Whereschoolshadaplayground,girlsalmostalwayshadaccesstoit.InNigerandinurbanareasplaygroundswerelesscommon.Only3%oftheschools,whichamountstoonlytwoschoolsinthesample,hadgroupswherechildrencouldcometogetherandsharetheirconcerns.Thesegroupstooktheformofamothergroupandateacher–studentassociation.GiventhelowmembershipofchildreninCBMCsthisraisesagainthequestionofhowchildren’svoicescanberepresentedindecisionstoimprovetheschoolenvironment.
Intermsofphysicalinfrastructure,schoolswerealsoinapoorcondition.Accordingtothesurveydata90%oftheschoolsare‘inneedofmajorrepairs’,withalloftheschoolsinBauchiinneedofrepairsand80%oftheschoolsinNiger.Only40%oftheschoolshadanelectricityconnection,withmoreschoolsinNigerhavingaconnectionthanschoolsinBauchi,despiteBauchihavingahigherrepresentationofurbanschoolswhereelectricityconnectionismorelikelytobeavailable.Theshareofschoolsconnectedtothegridthathadelectricityonthedayofthesurveywasevenlower,at30%.Accesstodrinkingwaterwaslimitedaswell,withonly30%ofschoolshavingsomesortofwatersource.Oftheschoolsthathadawatersource,around90%hadwateravailablefromthesourceonthedayofthevisit.Acrossbothstates,therewasanaverageoftworoomsbeingusedforclassesonthedayofvisit.However,around20%oftheIQSshadnoclassroomstructureandanadditional30%oftheschoolsonlyhadoneclassroom.TheschoolsoperatingwithoutanyformalclassroominfrastructurewereconcentratedinBauchi,whereasalargeshareofschoolsinNiger(about50%)onlyhadoneclassroom.
Lessthan25%oftheschoolshadafunctioningtoiletforpupils,whileonly8%oftheschoolshadaseparatefunctioningtoiletforgirls.However,alltheschoolsthathadmorethanonefunctioningtoiletforpupilshadaseparatetoiletallocatedforfemalepupils,whichsuggeststhatwhenthereisinfrastructureavailable,schoolmanagementisawareofthisneed.22%oftheschoolshadmadesecurityarrangements,suchasafenceorboundarywall.AlargershareoftheschoolsinNiger(30%)hadafenceorothersecurityarrangementscomparedtoonly10%oftheschoolsinBauchi.
Thissuggeststhattheinfrastructuresituationisquitepoorinboththestates,andchildrendonothaveaccesstobasicinfrastructuresuchasfunctionaltoiletsanddrinkingwaterattheschool.TheinfrastructurerepairneedsofschoolsareshowninFigure68.ThissuggeststhatthetwostateshavesimilarneedsbutthatmoreIQSsinBauchiareinneedofrepairs.WhileIQSslocatedinurbanareashavebetteraccesstoelectricityandwaterandmoreIQSshavefunctioningtoilets,theinfrastructureisgenerallyingreaterneedofrepair.InIQSswheretheproprietoristheheadteacher,theinfrastructuresituationisrelativelyworseoff.Theneedforschoolrepairswasalsoexpressedbyfacilitators.Inthefacilitatorinterviews,over50%ofthefacilitatorsagreedwiththestatementthat‘Itistoodifficulttoteachinthisschoolbecausethebuildingisinapoorcondition’.Giventhatabout30%ofCBMCsreportedhavingnoaccesstofundsinthelastschoolyear,thelikelihoodofrapidinfrastructureimprovementsislimitedunlessexternalsupportisprovided.Schoolsalsoneedadditionaltrainingontheimportanceofsoftinfrastructure.ThesecouldbeareasthattheGEP3interventioncouldfocusonattheschoollevelastheschoolenvironment(bothsoftandhardinfrastructure)influencespupillearning.
EvaluationoftheGirl’sEducationProject3
©EDOREN 192
Figure68: Infrastructurerepairneeds,bystate
Investmentactivities
CBMCsspentanaverageof80%ofthetotalfundsraisedduringthelastschoolyearonschoolimprovementactivities.UtilisationoffundswashigheriftheheadteacherorproprietorwasthechairoftheCBMCversusifacommunitymemberheldthisposition.FundsutilisationwasalsohigherinBauchithanNiger,thoughthetotalvalueoffundsavailabletoschoolsinBauchiwaslower.Unsurprisingly,schoolinfrastructureinBauchischoolswaspoorerascomparedtoschoolsinNiger,asalltheschoolsinBauchiweremarkedasbeinginneedofmajorrepairs.Overall,thephysicalconditionofschoolsinNigerwasnotsignificantlysuperior.Thiscouldbeduetothefactthattheactualamountoffundsavailableattheschoollevelisnotsubstantialandisdividedacrosstoomanyinvestmentcategories.Thefigurebelowsuggeststhatthismaybeaplausibleexplanation.ThetypeofinvestmentsthatweremostfrequentlyreportedbytheCBMCareinfrastructureworks,suchastheconstructionofnewbuildings,renovationofthebuildingorconstructionofatoiletandwaterfacilities.Secondinorderistheprocurementofteachingandlearningmaterials.Itisnoteworthythatdespitefacilitatorsreceivinglittleremunerationandthefactthatoftenonlyafewfacilitatorsareavailable,fundsarerarelyinvestedinfacilitatorrecruitmentorremuneration.
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Windows
Rooforceiling
Classroomwalls
Fenceorboundarywall
Floor
Furniture
Watersource
Toilets
Niger Bauchi
EvaluationoftheGirl’sEducationProject3
©EDOREN 193
Figure69: Investmentcategories,bystate
Girl-friendlinessoftheenvironment
Thediscussionoftheschoolenvironmentsofarpresentsstrong,consistentevidencethattheenvironmentinIQSsisnotchild-friendlyorgirl-friendly.Schoolshaveasevereshortageofhygienefacilitiesforpupils,andespeciallyforgirls.Inaddition,bothgirlsandboysarebarelyrepresentedintheCBMC.TherepresentationofchildandgirlchildmembersintheCBMCwasnegligible,asonly17%ofCBMCshadachildmember(andagirlchildmember),andchildrenmadeuponlyaround2%oftotalmembership.87%ofCBMCshadatleastonefemalemember.AlthoughtheattendancerateatthelastCBMCmeetingoffemalememberswasslightlylowerthantheoverallattendancerateofCBMCmembers(35%forfemalemembers,versus41%overall),forNigerthefemalememberattendanceratewasonaparwiththeoverallattendancerate.ForBauchi,thefemaleattendanceratewasmuchlowerthanthestateaverage.ThissuggeststhatthereisvariedparticipationoffemalemembersinCBMCfunctionsacrossthetwostates.
Therearealmostnoschoolswithinteractivegroupsforpupilstodiscusstheirproblemsandexperiences.ThisindicatesthatifGEP3wantstopilotG4GgroupsinBauchiorNigertosupportgirls’retention,groupsforpupilswillneedtobeestablished.Inaddition,thereisrelativelylowfemalerepresentationintermsofteachers,headteachers,andproprietors,thoughtheabilityoftheinterventiontoinfluencethelattermaybelimited.
Despitealloftheabove,itappearsthatgirlsarenolesslikelytobeattendinganIQScomparedtoboys.Thiswasindicatedduringthemanualcountingofpupilspresentonthedayofthevisit,aswellasduringthelessonobservationsandenrolmentregisters,thoughpoorenrolmentandattendancedatamakesitdifficulttoobtainthesecomparisonsacrossallthesampledschools.Inaddition,thefactthataround90%oftheCBMCshadatleastonefemalememberandfemaleattendanceratesatCBMCmeetingswerehigherthantheoverallaveragesuggeststhatculturalperceptionsaboutfemaleinvolvementintheseactivitiesareperhapsnotextremelyrestrictive,andthisholdstruemoresoinNigerthaninBauchi.HencethereiscapacityforgainstobemadeandtoincreasetheparticipationandrolesofwomeninthesebodiesandanyothergroupsthatareformedattheschoollevelundertheGEP3.
0%5%
10%15%20%25%30%35%40%
Bauchi Niger
EvaluationoftheGirl’sEducationProject3
©EDOREN 194
4.5.1.6 Governmentmonitoring
BauchiIQSsseemtohavegreaterinteractionwithvariousgovernmentbodies.TwiceasmanyheadteachersinBauchireportedtrainingconductedbySUBEBandtheLGEAthanheadteachersfromNiger.ThisholdstruefortrainingreceivedbyCBMCmembersaswell.Moreover,agreatershareofschoolsinBauchi(44%)reportedmonitoringvisitsduringthelastterm,comparedtoschoolsinNiger,whereonly33%oftheschoolsreportedmonitoringvisits.ThisdoesnotseemtobeassociatedwiththefactthatBauchicountsmoreurbanIQSsbecauseurbanIQSsinthesampleareaslikelytohavereceivedamonitoringvisitasruralIQSs.ThetotalnumberofmonitoringvisitswasalsogreaterinBauchithanNiger,withanaverageof4.2visitslastyear,comparedto3.1visitsinNiger.Thesourcesofmonitoringvisitswerecomparableacrossstates,withLGEAandSUBEBofficialsmonitoringschoolsinbothstatesandSAMEonlyvisitingschoolsinNiger.
Figure70: Sourcesofmonitoringvisitstoschools,bystate
4.5.1.7 Conclusionanddiscussion
TheevidencesuggeststhatexposuretoGEP’seffortsofimprovingschoolmanagementandgirl-friendlinessisvariedacrosstheIQSswithinandacrossstates.Thebaselinelevelandconditionofschoolinfrastructuresuggeststhereisroomforimprovementinthewaytheseschoolsattempttoattractgirlsandretainthem.Lackofresourcesappearstobeakeycorrelateofthis.However,itisencouragingtonotethatfinancialmanagementisformalisedtoalargeextentintheseschools,withCBMCsusingabankaccountinthenameoftheschooltomanageresources.Additionaltargetedtrainingonfinancialmanagementandresourceutilisationwillservetoimprovetheschoolmanagementaswellasthestateofgirl-friendlinessintheseschools.
4.5.2 Qualitativecasestudyanalysis
Thequalitativecasestudiesexplorethequestion:‘Willimprovedschoolmanagement,increasedcommunityinvolvement,andadditionalresourcescontributetoanimprovedlearningexperienceforchildren,particularlyforgirls?’ThisanalysisaidsinthediscussionofContributionClaim2oftheGEP3ToC:GEP3’sIQSScontributestoanimproved,girl-friendlylearningenvironment.Thus,the
0%
10%
20%
30%
40%
50%
60%
70%
80%
LGEA SAME SUBEB MinistryofEducation
UBEC TrainersfromSBMC/CBMC
training
NGOs,donors,otherexternal
agencies
Don'tknow
Bauchi Niger
EvaluationoftheGirl’sEducationProject3
©EDOREN 195
qualitativestudyexploresthemesaroundleadership,managementandresponsibility,aswellasperceptionsaroundwhatconstitutesagirl-friendlyschoolenvironment.ThroughFGDsandKIIs,thediscussionsalsoconsideredsocialrelationswithinthecommunities,bothbetweenschoolmanagementstakeholders,andbetweenpupils(inparticulargirls)andschoolleadership,toanalysetheextenttowhichschoolplanningprocessesactivelyconsidergirls’voices.
4.5.2.1 Schoolmanagement–actors,rolesandresponsibilities
OneofthekeyoutputsofGEP3isenhancedschoolleadershipandmanagement.Thequalitativefindingsshowthatvariousstakeholdersareinvolvedintheintegrationprocess.CommunityKIsidentifycommunityleaders,proprietors,imams,headteachers,teachers,parentsandtheCBMCasbeingessentialforsuccessfulintegration.Asdiscussedinthesection‘Schoolleadership:Roleofcommunityleadersandstakeholdersinmanagementonpedagogicalleadership’,itisimportanttogainabetterunderstandingofhowschoolleadersandmanagementunderstandandenacttheirrolesandresponsibilities.
Schoolleadership:Roleofcommunityleadersandstakeholdersinmanagement
IntheIQScontext,thedelegationofresponsibilitiesforintegrationandrunningtheschoolisnotclear.Instead,stakeholdersperceiveseveralactors–whoalsoperceivethemselves–tobepartofthesuccessfulrunningoftheschool,inmanycasesmeaningthatmanagementbecomescollective.Thisinturncreatessomeconfusionaroundwhoisinchargeofwhat,whoseapprovalisneededand,assuch,‘whohasthelastword’whenitcomestodecision-making.Asinthediscussionaroundpedagogicalleadership,ingrainedsocialrelationsinthecommunitiesattimesmeansthatthemostqualifiedpersonisnotalwaystheonewhoisresponsiblefordecision-makingandtasks.
Schoolleadershipdoesnotonlyconsistofthoseactivelyinvolvedwiththeschool,butincludesstakeholderssuchascommunityleadersandimams.TheembeddednessofIQSswithinpre-existingcommunitystructuresmeansthatthesestakeholdersoftenhaveinfluenceovertheschool’svision,andholdrespectedleadershippositionsintermsofguidingcommunityacceptanceandthesensitisationprocessonissuessuchasintegration.Assuch,theattitudesofthesekeyfigureshaveahighdegreeofinfluenceovercommunities’acceptanceofintegration.InallsixIQSsvisited,communityleadersshowedgeneralacceptanceofintegration,andwereinmanycasesactivelyinvolvedinschoolmanagementthroughpositionsontheCBMC.
‘They[theCBMC]usuallyconsultmeonissuesconcerningtheschoolandtheeducationofthechildren.Theyareawarethatoncealeaderacceptssomething,hecanconvinceothermembersofthecommunityandtheideawillhavewideracceptability.’(Communityleader,BauchilowerperformingIQS).
InNigerlowerperformingIQS,thequalitativeteamwasunabletointerviewthecommunityleaderduetointernalpoliticalrivalriesbetweentheMallam(theheadteacher)andthecommunityleader.TheMallamhadrecentlycomeoutontopinadisagreement,andthusaskedtheteamnottointerviewthecommunityleaderasthismightnegativelyaffecttheintegrationprocess.Itwasuncleartowhatextentthiswouldhavebeenthecase;however,itgavesomeinsightintotheimportanceofunderstandinginternalpowerrelationsforsuccessfulinterventions.ItwasclearthatinthiscasetheMallamhadahighersocialstandingamongstcommunitymembersandassuchhadbeenabletoproceedwithintegrationregardlessofsupport(orlackthereof)fromthecommunityleader.However,inotherIQScontextstheattitudesofcommunityleadersappeartohavesignificantimplicationsforthesuccessoftheintegrationprocess.
EvaluationoftheGirl’sEducationProject3
©EDOREN 196
Respondentsmainlyunderstandtheroleoftheheadteacherinschoolmanagementasaleadershipresponsibilitythatinvolvesadvocatingforintegrationandgirls’education.Headteacherscarryoutsomebasicfunctionstofulfiltheseresponsibilities,throughspeakingtothecommunity156andencouragingparentstosendtheirchildrentoschool.Inthismanner,teacherssaytheyreporttotheheadteacherwhentheyhaveissueswithpupilattendance,andtheheadteachercommunicatesthistotherestofthecommunity.
‘Onissueslikepupilsabsenteeismorlatecomingwecomplaintohimandhegiveshisfullsupporttoaddressthematterbyannouncinginthemosqueduringprayers,orhesendsamessagetothecommunityleadertoaddresstheparentsandletthemknowthattheteachersarecomplainingaboutpupilsattendanceandbythatmeansweseechanges.’(TPD,BauchihigherperformingIQS)
‘Everysocietyhasitscultureandtradition,sothiscommunityhasaculturethatdoesnottoleratetheintegrationofformaleducationwiththeQuranic.Butatthemomentthereareseriesofawarenesscampaignbythevillageheadandmyselfviathemosqueandweareenjoyingtotalacceptancenow.’(Mallam,BauchitypicalIQS).
HeadteachersdoholdauthorityinmanyIQSsandarethemainpeopletowhomteachersreport.ThestrengthoftherolerelatestothefactthattheheadteacherisoftenalsotheMallam,andattimesalsotheproprietor,andthusholdsahighstatuspositionthatisnotnecessarilyassociatedwith,orlimitedto,‘headteacher’.Therespectandtrustgiventothisinstitutionwithinthecommunity–someofwhichappeartohavebeenestablishedover100yearsago–seemstobeacriticalsuccessfactorintheintegrationprocess,withthepositionoftheMallam(oftenahereditaryrole)beinghighlyimportant.
Headteacherstendtohaveprimaryresponsibilityforpupildiscipline,withteachersreportinganymisbehaviourtotheMallam(headteacher)forhimto‘sanction’pupils.Theheadteacherisalsoresponsibleforencouragingfacilitatorstocometoschool,andisoftenthepersonwhohasinitiallyaskedamemberofthecommunitytotakeontheteacher/facilitatorrole.
Insomecases,157themajorityofthefacilitatorswerealsoformerpupilsoftheMallam.Thus,theheadteacherhassomeinfluenceovermonitoringfacilitatorattendanceandmanagingfacilitators.Asdiscussed,pedagogicalleadership,thelevelofqualificationsandfacilitatorexperienceamongstheadteachersvariessignificantlyacrossIQSs.Assuch,thelevelofleadershiponacademicachievementandfacilitatorguidance(fromapedagogicalstandpointforsecularlearning)isgenerallyweak.Still,amoreinformalprocessofmentoringappearstobetakingplaceinsomeIQSs158.InviewofsomeoftherelationshipsthatMallams/headteachershavewithfacilitatorswhoareex-pupils,theirpositionasrespectedQur’anicteachersmeansthatfacilitatorsconsidertheiroverallguidancetobeimportantandworthlisteningto.
InsomeIQSstheheadteacheristhemainpersonresponsiblefordeliveryoftheformalcurriculum.Thus,theheadteacheriseffectivelysupposedtomonitorthemselves,intermsofattendanceandperformance.Thesub-sectionbelowonCBMCs’capacitydevelopmentfurtherdiscussesthelackofclearstructurearoundmonitoring.
156ThisisincaseswheretheheadteacheristheMallam.Understandingsof‘headteacherresponsibilities’arethuscollapsedwiththoseoftheMallam.157BauchihigherperformingandtypicalIQS,NigerlowerperformingandtypicalIQS.158ObservedinBauchihighandtypicalperformingcases,andNigerlowperformingcase.
EvaluationoftheGirl’sEducationProject3
©EDOREN 197
Whiletheresponsibilitiesofkeycommunityfiguresareattimessymbolic,theyoftendoholdpositionsontheCBMC,andthusactivelyformpartoftheschoolmanagement.TrainingforCBMCsmayhavelessimpactinIQSswherecommunityleadersarepartoftheCBMCbuthavenotbeentrainedbyGEP3.Aspreviouslydiscussed,individualswithexperienceofformaleducationand/orschoolmanagementmaynotalwaysholdpositionsofimportance.TheheadteacherinNigerlowperformingIQS,forexample,takesontheresponsibilitiesofpedagogicalleadershipandchairstheCBMC,inspiteofnothavinganyformaleducation.Furthermore,religiousleadershipappearstoconferhighersocialstandingthanformaleducationalexperience.Thismaylimittheoverallcontributionofsecularheadteachertrainingtoimprovedschoolmanagement.TheimportanceofexistingsocialstructuresandnormswithintheIQScommunitiesmeanthatGEP3willneedtoconsiderhowtoensurethatallrelevantstakeholdersareincluded,andfortrainingtoespeciallytargetthosewhoholddecision-makingauthoritywithincommunities.
CBMCs:Characteristicsandfunctionality
AlltheIQSsvisitedhaveanestablishedCBMC.MostschoolscreatedCBMCsbecauseoftheintegrationprocess.DuringtrainingstakeholdersinthecommunitywereinstructedtostartaCBMCtosupportintegrationandgirls’education.IntheNigerlowerperformingIQS,CBMCmemberssaythattheyhavenotreceivedtraining.Instead,thecurrentCBMCsecretary,whohaschildreninthelocalpublicprimaryschoolandhadseenhowtheSBMCwasworkingthere,initiatedtheCBMC.Assuch,therewasaspillovereffectfromtheimplementationandsupportofthepublicprimarySBMCtotheIQSinthisparticularcommunity.
‘ThereasonhowtheCBMCcameaboutwasaftertheschoolwasbuiltitwasbasicallyQur’anicschool.Thentherewasnothinglikecommittee,itwasjusttheproprietorthatwasinchargeoftheschool.Wecalledontheproprietoranddiscussedwithhim,whichwashowtheCBMCcameabout.Youknowineverythingthereisneedforhavinganassociation,inschoolifthereisassociationandchallengesariseswewillknowhowtogoaboutit,tosolveitbutifthereisnoassociationthingsmightnotmovesmoothly.SoforthereweappointedtheChairman,Secretary,Treasurer,etc.Weare12membersinCBMC.ThereasonwhyweestablishedthisCBMC,firstly,istosolveanychallengesthatariseintheschool.ThisisthefirstreasonwhyweestablishedtheCBMC;itwasn’texistingintheinitialstage.’(CBMC,NigerlowerperformingIQS)
However,thefunctioningoftheCBMCvariessignificantlyacrossschools.InNigerlowerperformingandtypicalIQSs,thoughtheCBMCmeetsweeklyitisunclearwhatthepurposeofthesemeetingsis.Overall,theobjectivesandroleoftheCBMCarevague(‘encouragetheintegrationprocesses’and‘planthefutureofthisschool’),especiallyintermsofplanningtowardsachievingtheseobjectives.Still,thisprovidesastartingpointforcapacitybuilding,withCBMCs’attitudestowardintegrationappearingtobemostlypositive.
CBMCmemberstendtobeamixtureofkeystakeholderswithinthecommunityandparents.Insomecases,facilitatorsalsositontheCBMC.InallIQSsincludedinthequalitativestudytheMallamholdsakeypositionintheCBMC,suchaschairperson.ThisisimportantsincetheMallamisalsooftentheproprietorand,assuch,isactivelyinvolvedinbothresourcemobilisationandmonitoring.ThereisthusaquestionaroundpotentialconflictsofinterestwithintheIQSstructure,and,furthermore,theextenttowhichmonitoringcanbeeffectivewhenrolesandresponsibilitiestendtobelargelyoverlapping.Forexample,thepresenceoftheMallam/headteacher/proprietorwithinaCBMCcouldbeproblematicincaseswherethatindividualisquitedominant,aswasobservedintheBauchitypicalIQSFGD.Decision-making–suchasregardingtheuseoffunds–couldeasilybe
EvaluationoftheGirl’sEducationProject3
©EDOREN 198
heavilyinfluencedbysuchanindividual.Accountabilityandtransparencycanthereforebecomeissues.
ThemaincriteriaforbeingamemberoftheCBMCis‘forthepersontoberesponsibleand(ableto)adviseandimpactpositivelywhentheneedarises’(CBMC,BauchihigherperformingIQS).Thisdoesnotmeanthatonehastoholdapositionofpowerwithinthecommunity,onlythatoneiswillingtomaketheeffortandconsiderthevaluesoftheCBMC.OnedoesnothavetohavechildrenintheIQS,butoneneedstounderstandtheimportanceofeducationandagreewiththeprogramme(GEP/GEP3).
‘Itsometimeshappenthatforyoutobeamember,itmustbeseenthatyouarecapableandhavebeenputtingineffortstowardsimprovingtheprogramme.Itcouldbethatyourlovefortheprogrammeisobvious.Thatway,youcanbeco-optedintoit.’(CBMC,BauchilowerperformingIQS)
CBMCsmostlyconsiderwomentobeanimportantresourceoftheCBMCastheyhavethepotentialtoconvincemotherstosendtheirchildrentoschoolandtoadvocatetheobjectivesoftheCBMCtowomeninthecommunities.NotalloftheCBMCsvisitedhavefemalemembers,andinthosethatdothemembershipappearstobehighlydisproportionate,withoneortwomembersoutof12–17beingfemaleinthelowerperformingandtypicalperformingIQSsinBauchi,andinthelowerperformingschoolinNiger159.WhilstinthehigherperformingIQSsinbothstateswomencurrentlyholdaroundathirdofmembershippositions,theNigertypicalIQSdoesnothaveanyfemalemembersatpresent.Assuch,therepresentationofwomenonCBMCsvariessignificantlyacrossIQSs.CBMCmembersdescribedchallengesingettingwomentojointheCBMC.Forexample,inNigerlowerperformingIQSwomenaregenerallynotactivelyapartofthepublicsphereoractivitiesinthecommunityaftertheygetmarried.Thus,thecommitteehadrecentlylostafemalememberasshehadgottenmarried.ThismeansthatthereiscurrentlyonlyonefemalememberontheCBMC,andsheisthefemalefacilitatoroftheschool.Theyarenotsurehowtoconvinceanotherwomantojoinoncethisfacilitatorgetsmarried.Thus,genderrolesandresponsibilitiesoutsideofschoolmanagementaffecttheextenttowhichwomenformanactivepartoftheCBMC.Thisisinspiteofrecognisingfemalemembersasbeingimportantinordertoachievetheobjectivesoftheintegration.ThismayprovetobeachallengefortheadvocacyaspectoftheCBMC,anditwillbeimportanttoconsiderhowtoreachwomenincommunitieswheretherearenowomenbeingtrainedbyGEP(eitherthroughfacilitatortrainingorthroughbeingmembersoftheCBMC).
ThetypeofIQSmayfurtheraffectthefunctionalityoftheCBMC.InallthequalitativecasestheIQSsarenotmobile,andappeartobeactivelyengrainedinthecommunity.CBMCsconsiderthisanimportantaspectofwhytheyareabletoenacttheirresponsibilities,sincecommunitymembersperceivethemtoberepresentativeofthecommunityinwhichtheyaresituated.WhereanIQSismobile,thismayaffecttheextenttowhichaCBMCcanbeofuse.ItisalsoimportanttonotethatifanIQSismobile,girls’learningwilllikelyceasecompletely(asgirlsdonottravelwiththeMallam)unlessitisarrangedthattheIQSfacilitatorremainsbehindandcontinuesteachingthegirlsandremainingboyswhodidnotmove.Inthatinstance,theIQSfacilitatorwouldlikelybeonher/hisownwithouttheheadteacher/Mallam.TheMallamnotbeingpresentcouldweakentheCBMC’seffectiveness.
159ItwasdifficulttoestablishtheexactmembershipandgenderdistributionofCBMCsasschoolsapartfromNigerhighperformingIQSdidnotkeepamembersregistry.
EvaluationoftheGirl’sEducationProject3
©EDOREN 199
CBMC:Perceptionsofmaintasks,responsibilitiesandcapacityconstraints
InthesixIQScases,CBMCmembersgenerallyunderstandtheirroleandresponsibilities,buthaveweakcapacitytoenactthese.ThequalitativeresearchfoundthatCBMCsseetheirrolesasbroad–coveringawiderangeofresponsibilities,fromresourcemobilisationtoadvocacyoftheimportanceofeducation.CBMCsmainlyseetheirroleasinvolvingidentifyingwhychildrenarenotenrolledandlearning,andto‘solve’theseproblems.However,CBMCsarguethattheylackthefundstoseethesesolutionsthroughandstate‘wecandobetter’.Moreover,CBMCsappearwillingtotakeonnewskillsandtasks.SeveraltimesduringFGDs,CBMCmemberswouldsay‘wewilldothismorenowsinceyouhaveaskedusit’(NigerlowerperformingIQs),inresponseto,forexample,questionsonwhethertheymeetwithparents.CBMCsandotherstakeholdersperceivethemainresponsibilitiesoftheCBMCtobe:
• increasingawarenessoftheimportanceofeducation;• ensuringattendanceofpupilsandteachersthroughproblem-solvingandmonitoring;• resolvingproblemstogetherwithcommunitystakeholders;and• improvingschoolinfrastructure.
AmainresponsibilityoftheCBMCisincreasingtheawarenessoftheimportanceofeducationforbothboysandgirls.SincetheCBMCsvisitedaspartofthequalitativestudyallhadkeycommunitystakeholdersasmembers,inmanyregardstheyoperateas‘minivillagecouncils’.CBMCsthusdiscussallproblemsrelatedtoeducation(includingreligiouseducation).Sincecommunityandreligiousleadersarekeyincallingmeetingswiththecommunityandinadvocatingfortheintegratedcurriculumandgirls’education,theinvolvementofthesefiguresandtheiracceptanceofproposedchangesisessentialfortheCBMCtobeabletofulfilitsrole.
‘Wevisitanddiscusswiththem,tellingthemthatifweallowthechildrentogetenrolledittranslatedintosomethingmeaningforthewholecommunityinthefuture,tellingthemthatthebenefitsmaynotbeimmediatebutperhapsinthefuture.Assuch,theyshouldhelpthechildrenbyallowingthemtobeenrolledsothattheycanreapboththeworldlybenefitsandthebenefitsinthehereafter.’(CBMCNiger,lowerperformingIQS)
CBMCsgenerallyfeelthattheyaredoingagoodjob,butbelievetheycoulddobetteriftheyfacedfewercapacityconstraints.Respondentsseelackoffundstoimproveinfrastructureandtopayfacilitatorsasthemainchallenges.Theperceptionislessofparentsbeingunwillingtocontributefunds,butmorethatparentshavelimitedfundstogive.ThefactthatoneofthekeyreasonswhyparentssendtheirchildrentoIQSsappearstobeduetopovertyandaninabilitytoaffordtosendchildrentothepublicprimaryschoolsupportsthis.ThesefindingssuggestthatresourcemobilisationbeyondtheGEP3grantsmaybevolatileandvariable,especiallyincaseswhereCBMCsrelyonphilanthropy/richcommunitymembersinthecity,orevenpoliticalphilanthropy.
CBMCsreportbeingawareofthechallengesconcerningnotpayingfacilitators,sincethismayaffectbothfacilitatorandpupilattendance.Theyarealsoawareofthedifficultyofmobilisingresourcesfromparents.However,theylackthecapacitytosolvethesechallenges.BeingeitherprivatelyrunbyaproprietororbythecommunitytheseIQSsrelymoreonresourcemobilisationthanpublicprimaryschools160.Byrelyingoncontributionsfromcommunitymemberswhomaynothavefundstogive,thereisariskthattheworkofCBMCswilldiminishas‘externalresources’runout.
160Haq and Islam (2005).
EvaluationoftheGirl’sEducationProject3
©EDOREN 200
‘Concerningraisingoffund,weusetotellthechildrentobring100Nairawhenevertheyresumefromsallahbreak.Butjustfew,about10pupilswereabletopaysinceweresumedfromthelastsallahbreakwhichwasseveralmonthsago.’(CBMC,BauchitypicalIQS)
IntheNigerhigherperformingIQStheCBMCasksparentstocontributeinanywaytheycan:‘Wehavemadecontributions.Eachofuswasrequestedtobringeitherwood,nailsorzinc,whichwedid.(Girls’FGD).Thesenon-monetarycontributions,whilstimportant,arenotsufficientasregardspayingfacilitators,whichrespondentsperceivetobethemainfinancialissue:
‘Ourmajorproblemismoney;wedonothavesufficientfundstosupporttheschoolthewaywewantto.Thereisevennomoneytogivetheteachersinordertomotivatethem.Theyjustvolunteer.’(CBMC,BauchihigherperformingIQS)
CBMCsfurtherconsiderpromotingfacilitatorattendanceandperformancetobepartoftheirresponsibilities.However,asCBMCmembersmonitorfacilitatorperformancewithouthavingpriorteachingexperienceandlackofguidance,monitoringoftenbecomessimplysymbolic.AsoneCBMCmemberstated:‘Wealsocarryoutvisits(schoolvisits),althoughwehavenolessongoalsatthattime’(CBMC,BauchitypicalIQS).Nevertheless,stakeholdersconsidermonitoringbyCBMCstobeuseful,perceivingthatitmotivatespupilsandparentsbydemonstratingthatcommunityleadersandkeystakeholderstakeeducationseriously.
CBMCsinallschoolsbelievethattheywouldbebetterabletomonitorfacilitators,andholdthemtoaccount,iftheywereabletopaythemasalary.ThiswouldalsoenabletheIQStoattractqualifiedfacilitators.Thisisparticularlychallengingwhentherearenotenoughqualifiedfacilitatorswithinthecommunity161,creatingtheneedfortheCBMCtoincentivisepeoplefromoutsidethecommunitytoteachattheIQS,oratleasttoreimbursetheirtravelcosts.
‘Wealsotalkaboutteachers.Wewantmoreteachersbroughtwhocanteachformalsubject.AsfortheArabic,thereisnoproblem.Fortheformalsubjectteacheritisnotconsistent.Sometimeshewillcomeandsomeothertime,hewillnotcome.Ifwehavepeoplewhocandevotetheirtimetous,itwillguaranteebetterquality.’(CBMC,BauchilowerperformingIQS)
Thus,CBMCsallstatethatoneoftheirmaintasksisraisingfundsforthefacilitators.However,thereisanimplicitconflicthereasCBMCsacknowledgethatparentshavelimitedcapacitytocontributefunds.CBMCsseethisassomethingthatsupportfrom‘externals’mustsolve.
However,CBMCsdonotnecessarilyinvestfundsexclusivelyforformalcurriculumfacilitators.InBauchihigherperformingIQS,formalfacilitatorscomefromthepublicprimaryschoolandare,assuch,paidsalariesfortheirregulargovernmentdayjobs,althoughtheystillworkvoluntarilyintheIQS.TheCBMChencemobilisesresourcesinordertopaytheArabicteachersbutnottheformalfacilitators:
‘Youknowthereare10teachersforIslamicstudiesinthisschoolandwedonatesomemoneyduringtheCBMCmeetingtosupporttheseteachers…becausetheydon’thavegovernmentjobs.’(TPD,BauchihigherperformingIQS)
AnotherkeyresponsibilityofCBMCsisdealingwithpupilabsenteeism.CBMCsperceivetheissueofpupilabsenteeismtobemultifaceted,althoughtheyseeitasrootedinparentsnotunderstandingtheimportanceofformaleducation.CBMCsallperceivetheirresponsibilitiesasincludingexplaining
161By‘qualified’ismeantsomeonewhocanreadandwrite,orsomeonewhohassomeyearsofformaleducation.
EvaluationoftheGirl’sEducationProject3
©EDOREN 201
thepurposeofformaleducationtoparents,andtakingthetimetositdownwithparentstomakesuretheyunderstandthis.CBMCsinmanyoftheIQSsvisitedhencegofromhousetohousetocontactparents.IncaseswhereaCBMChasfemalememberstheywillgoandspeaktomothers.
‘They[theCBMC]arereallyhelpingusinthattheyencourageourparentstoalwayssendustoschoolandnotsendusonerrandsthatcouldpreventusfromcomingtoschool.’(Girls,BauchihigherperformingIQS)
MethodsusedbyCBMCstoensurethatchildrenattendschoolalsoincludewardheadsmobilisingpupilstocometoschool(inNigerhigherperformingIQS),youthmonitoringgroupsbeingformedtopromoteattendance(BauchihigherperformingIQS),andCBMCmembersspeakingtoparentsofchildrenwhoareoftenabsent(NigerlowerperformingIQS).InNigertypicalIQS,theCBMChasdecidedtomakecomingtoschoolcompulsory.Whenparentsdonotcomply,theCBMChastheauthoritytophysicallyfighttheparentsandbringchildrentoschool:‘Itiscompulsorytocomply.Suchparents,wetalktothemandwecanevenforceonsuchparentsbecausewefightthem.’(CBMC,NigertypicalIQS)
However,respondentsperceivepupilabsenteeismtobeaconsequenceofpoverty.Parentsoftenrequirechildrentoworkorlookafteryoungersiblings,andthustheybearanopportunitycostifthesendtheirchildrentoschool.Tosolvethis,theNigertypicalIQSseestheschoolmuchasacommunitydaycarecentre,whereallchildrengo,sothatparentscanstillgotothefarm.Thismaynegativelyaffectlearning,aschildrenofallages(includingbabiesandtoddlers)arepresent:‘Whenschoolishappeningallthechildrenofthecommunityareinschool.’(CBMC,NigertypicalIQS)
Inordertomotivatepupils,CBMCsinbothhigherperformingIQSs‘testpupils’andprovidegiftsforthosewhoperformbest.Rewardingpupilsisseenasawayofencouragingparentstosendtheirchildrentoschool.TheNigerlowerperformingIQSaskschildrentocomeuponstageduringcommunitygatheringsandrecitethe‘a,b,c’,withtheideathatthiswillmakeparentsproudandwillmotivatethemtosendtheirchildrentoschool.InNigerlowerperformingIQSamainissueidentifiedwasthelocationoftheIQS.Pupilshadtocrossaroadwithheavytraffictoaccesstheschool,whichpreventedparentsfromlettingtheirchildrenattendtheIQS.MembersoftheCBMCthereforestandandstopthetraffictohelppupilscrossbeforeandafterschool,inordertoincreaseattendance.Inmostcases,CBMCsthusactas‘problemsolvers’,identifyingreasonsforpupilabsenteeismandsolvingthem.
Lastly,CBMCsperceivethemselvesasbeingresponsibleforschoolinfrastructureandimprovingthelearningenvironment162.AllIQSshavemadeeffortstoimprovethephysicalschoolenvironment,eitherthroughconstructingaschoolorthroughpurchasingmats,kettlesetc.However,insomecasestherearequestionsaboutwhetherCBMCsusefundsinthebestpossiblemanner.IntheBauchitypicalperformingIQS,theCBMCusedresourcestopurchaseatruck-loadofsandtoleveltheground,inordertocombatrainfallerosionthathadcreatedslopes,leadingtopupilssittinguncomfortably.However,thesandhadonceagaineroded,resultinginareturnoftheunevenslope.TheCBMChadusednofundstoerectaroofofsomesort,leavingpupilsunprotectedfromtherainandthesun.ThisraisesquestionsregardingwhatinformsCBMCs’decisionsregardingspendingfunds,andthusraisesquestionabouttheirdecision-makingcapacity.
162SeefurtherdiscussioninSection2.2.2ongenderconsiderationsinschoolmanagement.
EvaluationoftheGirl’sEducationProject3
©EDOREN 202
CBMCs:Capacitydevelopment
Asdiscussedabove,CBMCmembersarewillingandabletotakeonnewskills,knowledgeandwaysofworking.However,thesituationsCBMCsfaceinIQSs,intermsofthesocioeconomiccontextandlevelofqualifications(aswellasnavigatingengrainedsocialnorms),meansthattheyfacearangeofpracticalchallengeswhenitcomestomobilisingresourceseffectively.Thesecontextualconstraintsmayneedtobeconsideredinordertotargetcapacitybuildinginresourcemobilisation.EvaluatingtheefficiencyandeffectivenessofaCBMCthusneedstotakeintoaccountthecontextwithinwhichitoperates.Inthissense,CBMCscannotbeeasilycomparedwithSBMCsinpublicprimaryschools.Moreover,CBMCstendtobehighlyembeddedintheschoolstructure:theydealwithaspectsofreligiouseducationandgeneralcommunitywellbeing,inadditiontoaspectsofintegration.Becauseofthisbroaderremit,CBMCswilllikelybeengagednotonlyindiscussingandmanagingIQSmatters,butalsoablendofbothQuranicandsecularlearning,aswellascommunityissuesexternaltoeducationmatters.
InsomeIQSsvisited,CBMCshadreceivedtrainingongendersensitisationandtheimportanceofformaleducation.SomeCBMCs(forexampleNigertypicalIQS)referredtohavingreceivedtrainingonhowtospendallocatedfunds.ThequalitativedatashowthatCBMCsimplementactivitiesbasedonwhattrainingcontenttheyhavereceived,butthelackofresourcesandclarityaroundrolesandresponsibilitiesofvariousstakeholderslimitstheenactmentoftasksandresponsibilities.Asthequotebelowexemplifies,theperceptionsaroundtrainingreceivedwerethatifCBMCsstartedimplementingintegrationactivities,supportwaslikelytocomefurtherdowntheline.
‘TheytaughtusalotIcanrememberonlyfew.FirsttheytoldustheimportanceofformaleducationinourQur‘anicstudyandtobeintunewiththeworld.Ifthemelodychanges,thedanceshouldalsochange.WetheMallamsshouldalsogiveoursupportbyallowingchildrentogotoschool,andweagreed.Theysaiditisverylikelyattheendthegovernmentwillgivetheirsupport.SowedecidedtofindteacherstoteachtheseformalsubjectssincewearenoteducatedthoughIamnotamongtheuneducatedbecauseIameducatedbutsinceIfoundmyselfteachingtheQur’anicknowledge,IaccepteditbutIteachHausaalso.IgotanEnglishteacherbutIhadwishedthegovernmentcouldtaketheresponsibility.ThenIgotMallamAnas,amathematicsteacher,sinceIdonotnowknowhowtocalculate…Theysaidtheparentsshouldsupport…Theydosupportaccordingtotheirstrengthbuttheschooldoesnotreceiveanyfunds,itjust10Nairainaweek.Theyalsosaidthechildrenshouldbecaredforandnottobebeaten,nowyoudon’tseemyteacherscarrycane.’(CBMC(Mallam),BauchitypicalIQS)
CBMCcapacitywillremainacriticalissueforGEP3.TheextenttowhichGEP3planstofocusonfurtherCBMCcapacitybuildingbeyondwhathasalreadyoccurredcouldbecriticalifthisaspectoftheToCistofollowitsdesiredcausalpath.CBMCsappeartoneedfurthertraining,goingbeyondtheirdutiesinregardtosensitisationanddevelopingmoretheircapacitiesinmonitoringandensuringtheactivesupportofgirls’educationinIQSs.Thiscanincludetrainingonhowtoengagewiththefacilitatorsinregardtotheirattitudesandteachingbehaviours,andagreatersenseofaccountabilityonthepartoftheCBMCtoensureagirl-friendlylearningenvironment.ThechallengestheLGEAfaceinmonitoringalloftheIQSalsomeansthatfollow-throughonCBMCeffectivenessisdifficult,witheachIQSoperatinginwhatappeartobelargelyunstructuredandcontextuallydrivencircumstances.
EvaluationoftheGirl’sEducationProject3
©EDOREN 203
4.5.2.2 Attitudesandinvolvementofparents
Thequalitativecasestudiesfoundparentstobesupportiveofintegrationandtheassociatedeffortsbyschoolleadersandmanagement.TheperceptionsarethattheprovisionoftrainingtoCBMCmembersandmini-grantstotheIQSscontributestoincreasedcommunityinvolvementinandsupportfortheIQSs.ItisworthnotingthattherewasaselectionbiasintheparentsspokentosincetheMallamwouldgatheragroupofparentstotakepartintheFGD.163Moreover,insomeIQSsthequalitativeteamspoketonomothers,asitwouldhavebeeninappropriatewithintheculturalcontext.Incontrast,intheNigerhigherperformingIQStheteamonlyspoketomotherssincefatherswereawayonmarketdayduringthequalitativevisit.Fromthevariousdiscussionswithschoolleadersandmanagement,aswellaspupils,itisclearthatIQSsrelyoncommunityinvolvementandsupportinordertofunction.Asdiscussed,theIQSs’embeddednesswithinsocialstructuresmeansthatsupportfromtheMallamandotherreligiousauthorities,aswellasfromcommunityleaders,seemsvitalforsuccessfulintegration.Theactivesupportoftheserespectedindividuals,inturn,helpstremendouslyingarneringsupportfromparents.
‘Iamtheowneroftheschoolanditbearsmyname,butIdon’tclaimownership.Iamastakeholderandthecommunityleader.CBMCandothermembersofthecommunityareallstakeholdersthatensureasmoothrunningoftheschool.’(Mallam,BauchihigherperformingIQS)
‘Wemeettheteacherstotellthemourworries,astheyalsotellustheirsandtogetherwerubmindsforsolution.’(Parents,BauchitypicalIQS)
CommunicationbetweenparentsandtheIQSseemstofunctionwell.IncaseswheretheCBMCorheadteacherhaveissueswithpupilabsenteeismorresourcestheycallparentstomeetingstoeitherdiscusstheimportanceofeducationorfortheCBMCtorequestfunds.InseveralIQSsparentsreporthavingbeeninvolvedindecidinghowtospendgrantsreceived.Assuch,thereseemstobeahighlevelofcommunicationbetweenschoolsandcommunities.Thiscommunicationhappensthroughpre-arrangedmeetingsbetweenCBMCsandparents,throughmeetingsatthemosquebetweenheadteachersandparents,aswellasthroughone-to-onediscussionsbetweenheadteachersorCBMCsandparents.Parentsreporthavingnoproblemsinspeakingtoheadteachers.Thisappearstoberelatedtothefactthattheheadteacherisoftenareligiousauthority.Itappearscommonforparentstoaskaboutchildren’sschoolingafterprayerhasfinished.Theimportanceofthemosqueasameetingpointmeansthatcommunitymembersfrequentlyinteractwitheachother.Thiscreatesopportunitiesforparentstoengagewithschoolleadersandmanagement.
Moreover,CBMCsactivelyinvolveparentsinordertodealwithpupilabsenteeism.ParentsinallIQSsreportthattheCBMCand/orheadteacherhasaskedthemtomonitorwhethertheirchildrengotoschoolbyaskingchildrenwhattheylearnwhentheygethome,andbywalkingchildrentoschooland/orcheckinginwithfacilitatorstoseethatchildrenattendschool.ParentsinallFGDsreportregularlyaskingfacilitatorsortheMallamiftheirchildrenattend,andattimesaskingtheirchildrenwhattheyhavelearntinschool.Moreover,intheBauchihigherperformingIQStheCBMChasstarteda‘vigilantegroup’toensurethatpupilscanwalksafelytoschoolandtomonitorattendance.
163Atmidline,itwillbeimportanttostrategisehowbesttoavoidthisselectionbias.ThiscouldpotentiallybedonebyapproachingtheCBMCmorebroadlyinordertoseekparticipantsfortheparentFGD–thiswouldtakethedecision-makingoutofthehandsofoneindividual.However,thismaystillbesubjecttoselectionbias,so,ifpossible,analternativecouldbetogatherthewholecommunityaspartofanintroductorysession,andaskforvolunteerstoraisetheirhandsfromthere.IfIQSshaveintroducedanattendancelistatthetime,itmayalsobepossibletorandomlysamplebothpupilsandparentsthroughtheselists.
EvaluationoftheGirl’sEducationProject3
©EDOREN 204
Pupilabsenteeismmightbeanindirectindicatorofcommunitysupportforschools,andCBMCsandfacilitatorsreportthemainissueinthisregardbeingparentsnotunderstandingthevalueofeducation.However,theIQScontextiscomplex.ThoughchildrenmaynotregularlyattendformalclassesintheIQS,thisdoesnotmeantheydonotattendpublicprimaryschools.Moreover,theperceptionisthatsocioeconomicreasons(asdiscussed)arethemainreasonwhychildrendonotattendschool,andpupilabsenteeismdoesnotnecessarilymeanthatparentsdonotsupporttheIQS.
‘Wedoaskthem(children)questionsonceinawhileaboutwhattheyarelearningintheschool,accordingtoourlevelofunderstanding.Thereisasmallgirlthatweobserveddoingwellinmaths.Sometimesweevenchecktheirbooks.Asparents,weindividuallysendandmakesurethatthekidscometoschooltolearn.Wemeetanddiscusswiththeteachersandthenadviseappropriatelyaboutthewayforwardasfarastheschoolisconcerned.’(Parents,BauchilowerperformingIQS)
However,notallparentsareequallyinvolvedintheactivitiesoftheIQS.Someparentsactivelyengageinschoolactivities,andformpartoftheCBMC,whilst‘somedon’tcare’(Parents,NigerlowerperformingIQS).Althoughparentsmaybeinvolvedindecisions,andactivelyengagewithdiscussions,thatdoesnotnecessarilymeanthattheyhavedecision-makingpower.SincetheMallamoftenhashighstatus,throughbeingtheproprietor,theheadteacher,aCBMCmemberandareligiousauthority,hisdecision-makingauthoritytendstobestrong:‘Ourparticipationislimited.Youknowhedecidesbecauseheistheimamandiswellrespected.’(Parents,NigerlowerperformingIQS).Furthermore,oneschoolinBauchi(lowerperforming)haspupilswhoattendfromoutsidethecommunity,whocomefromasfarawayasNigerStateandstayforyears.Itisnotclearwhoisresponsiblefortheattendance,andformonitoringthelearning,ofthosepupilswhoseparentsarenotpresentinthecommunity.
Moreover,theinvolvementofmenandwomendifferssignificantlydependingontheIQScontext.AsdiscussedinSection0,mencurrentlymakeupthemajorityofCBMCmembershipinallschoolsvisited.Incaseswherewomenarepresenttheytendtohavefewerformalpositions,andonlyinonecase(NigerhigherperformingIQS)didawomanholdanamedposition(treasurer).IntheNigerlowerperformingandtypicalIQS,onlymenattendmeetingswhentheIQScallsparentstomeet:‘No,they(women)don’tcometothemeetings;weinthevillagearestrictwhenitcomestothis.Wekeepthemathome.’(Niger,lowerperformingIQS).Consequently,inthiscasewomenareonlyinvolvedthroughtheactiveengagementoftheonefemaleCBMCmemberbut,asdiscussedinSection0,thismightnolongerbepossibleoncethewomenmarry.Inmostcases,womenarethusreliantontheinformationtheyreceivethroughtheirhusbandsandothermeninthefamily.
InNigerhigherperformingIQSwomenaremoreinvolved,andevenmakeupasignificantnumberofCBMCmembers.Theproprietoroftheschoolisfemale,andisperceivedtobean‘eldersister,ormother’tothecommunity.SincetheIQSisinherhouse,itisindeedeasierformotherstobeactivelyinvolvedintheIQS.Moreover,thisIQSalsorunsformaleducationclassesformarriedwomenandthuswomenarepresentintheschool,ascomparedtootherIQSswherewomenaremoreconfinedtotheprivatesphereofthehousehold.
‘Actually,itisthisgoodthingshestarted.Thatmadeustoknowheranditisbecauseofthetrustthathasbeenestablishedbetweenusandthetrainingourchildrengetthatwehavegivenherourchildrentotrainandeducate.Ourchildrencomeheretobetaughtandtheyaretaughtgoodbehaviours.Weareveryhappy,wetrusteachotherandwehavebeenlivingpeacefully.Wemothersseeherasaneldersister,someasamotherandwehavebecomelikeonefamilynow.’(Niger,higherperformingIQS)
EvaluationoftheGirl’sEducationProject3
©EDOREN 205
4.5.2.3 Perceivedcommunitysupport
AcrossallsixIQSvisitedparentsandkeycommunityfiguresreportprovidingsomeformofsupporttotheschool.Thequalitativeresearchfoundthattheperceptionsaroundsupportarenotlimitedtofinancialcontributions,butencompassmoralsupportandtrust.
‘Thewidercommunityistrying.Thepeopleappreciatemyeffortsandcontributionstowardstheeducationoftheirkids,byprayingformeandpattingmeontheback.Ialsoappreciatetheircontributionsandconcerntowardstheschool.’(Mallam,Bauchihigh-performingIQS)
ParentsinallcasesreportprovidingsomesortoffinancialsupporttotheIQS,mainlythroughcontributingtowardspayingfacilitators‘atoken’.Theyareawareofnotbeingabletocontributeasmuchasrequired,butattributethistopovertyratherthantoalackofwillingness.Parentshighlightthattheysupporttheschoolsinotherways,eitherthroughmonitoringtheattendanceofchildren,providingadvice,orthroughcarryingoutcommunityworksthatbenefittheschool.
4.5.2.4 LGEAsupportandmonitoring
Governmentauthoritiesand‘externals’areperceivedtobethemainactorsresponsibleforprovidingfurthersupporttotheIQS.
‘Yes,they(theLGEA)cametoseeus;theymetevenwiththecommunityleader.Thatiswhywecounselledtheteacherstheothertime,forthemtobepatient.Thevisitorswenttothedistrictheadandreiteratedthesamedetails.Andemphasisedthatperhapsinthefuturewhoeverisinvolvedmaybeemployedtobepaidfully.Thatisthewayitis…’(CBMC,NigerlowerperformingIQS)
RespondentsreportthatLGEAofficialsor‘government’rarelyornevervisittheIQS(apartfrominNigerhigherperformingIQS).TheLGEAofficersinterviewedaspartofthequalitativeanalysisconfirmedthis.Theyperceivethechallengeofattemptingtovisit30schoolsandmoreonaregularbasistobeunmanageable,giventhedistancesconcerned,thelackoftransportationinmanycases,andtheirworkloads.Furthermore,morefundsaresaidtohavebeenpromisedfromtheLGEAbutnotreceived.TheCBMCinNigerlowerperformingIQSsaystheyhaveneverreceivedsupport.BycontrasttherespectiveLGEAofficersaidthatallschoolshavereceivedgrants.TheCBMCinthiscasedoesnothaveabankaccount.Generally,communitiesviewsupportfromthegovernmentnegatively,consideringgovernmentactorstobeunreliable164.Still,theviewisthatthegovernmentshouldbeprovidingfacilitators:ifanIQSdoesallthatisaskedofit,thenthisshouldberecognisedthroughfinancialsupportorsupportoffacilitators.Iftheperceptionisthatnomajorsupportistocome,thismayhaveimplicationsforcommunityattitudestowardsintegration.
LGEAofficers’engagementwithcommunities–whenitoccurs–appearstofollowasimilarformula.Theheadteacher/Mallam/proprietoristheprimaryentrypointforengagementwiththeIQS,theCBMC,andthecommunityatlarge.OfficersinBauchispokeofreceivingmonthlyreportsfromtheIQS.Itappearsthattheseoccurthroughactualvisitstotheschoolbytheofficerhimself,whowouldthencompileareportbasedondiscussionsandmeetingswithstakeholdersinthecommunity.Giventhechallengesfacedinreachingalltheschoolsonaregularbasis,thismakestheregularityofthesereportsandtheircontributiontoefficientlymonitoringandsupportingtheIQSasignificant
164NigerhigherperformingIQSisanexceptionhere,reportinghavingseveralvisitsfromtheLGAandhaving‘governmentmembers’aspartoftheirCBMC.
EvaluationoftheGirl’sEducationProject3
©EDOREN 206
challenge.TheeffectivenessofwhattakesplacewhenLGEAofficialsmakethesevisitsisalsoaconcern,asarticulatedhere:
‘Undermyrole,thechallengesIhavefoundisthoseschoolsthatwereselectedarehardtoreach.Wheneverthereisanimportantissue,wewanttodiscuss,wehavetopreparetime,butIcannotreachthemontime.Thentohavecommunicationwiththem–theydonothavenetworkservice–Ihavetotrek,ortakevehiclefrommyLGAdowntothecommunity.ThenwheneverIorganiseameetingwiththecommunitymemberstodiscussanimportantissueabouteducation,mostoftheparentsorcommunitymembersdon’tliketoattend,thinking[that]thegovernmentisgivingsomething,andwedon’twanttogiveitout,[but]theyalwayswanttogetsomethingwhenevertheyarecalledtoameeting.’(BauchiLGEAofficer,typicalcase
LGEAofficersalsoseetheirroleasprovidingsupervisionandadviceonteachingandmanagementpractices,andofferinginstructionandcorrectionbasedontheirobservations:
‘Youknow,whenwevisit,andwefindoutthattheyarenotcarryingouttheactivitiessuccessfully,wewilladvisethem,orstillre-trainthem.Maybeinthemethodofteachingtheyarenotgoingstraightforward,ortheyareleavingsometacticsormethods,eitherinevaluation,eitherinmethodology…iftheyleavesome[thingout],wewillremindthem.Wewilltellthemthatisthewaytheyshoulddoit”.(LGEAofficer,Bauchitypicalcase)
4.5.2.5 Schoolexperiencefor(girls)children
ThequalitativeresearchinthesixIQSsacrosssixdistrictsoffersinsightsintosomeoftheexperiencesofgirlchildren,andhighlightssomeoftheconstraintsthatgirlsmayfaceinattendinganIQS.ThisisbynomeansrepresentativeofthechallengesandexperiencesofallgirlsattendingIQSsintheGEP3states,butprovideshintsastoareaswheregenderconsiderationsareimportantforprogramming.
TheFGDswithpupils(boysandgirlsseparately)askedchildrentodescribetheirtypicaldayinschool,andtodiscussthewaysinwhichtheyinteractedwiththeirclassmatesandfacilitators.Bothboysandgirlssaythattheenvironmentisfriendlyforallchildren,andthatsincetheyteachthemtorelatewelltoeachothertherearefewproblems.
However,duringFGDsandinQCOs,girls(inallIQSsapartfromtheNigerhigherperformingIQS)weresignificantlyquieterthanboyswere.Girlsoftenwhisperedanswersorhidbehindtheirhijabs.Whilstboyswouldimmediatelyhighlighttheirknowledge,girlswouldgive‘yesandno’answersandwouldtakealotlongertowarmupandsharetheirperspectives.
Intheclassroomsvisited,girlssitattheback,orboysandgirlssitonseparatesidesoftheclassroom.Neitherboysnorgirlsperceivethistobeanissuesince‘theMallamhastoldusthisisthewayhewantsustosit’(Girls,NigerlowerperformingIQS).Whenasked,girlssaytheywouldliketositinthefrontbutthatthisis‘notthewayitisdone’.IntheBauchitypicalIQS,theschoolislocatedoutsideandchildrensitonaslope,withchildrenfurtherawayfromtheblackboardsittingatlowerslevelsoftheslope.Inthiscase,thepracticeofseatinggirlsatthebackoftheclassdirectlyunderminestheirabilitytoseetheblackboard:‘Wherethegirlssit,waterhaserodedtheplace’(TPD).
Pupilsbroughtuptheseissuesofshynessandseatingarrangementsasperceiveddifferencesbetweenboysandgirls,especiallyintheFGDswithboys.Boysinterpretedgirls’shynessasasignthatboysaremoreintelligentthangirlsare–becauseboysanswerquestionsinschoolmoreoften
EvaluationoftheGirl’sEducationProject3
©EDOREN 207
thangirlsdo.Boysperceivegirlstobedistractedand‘notinterestedinschool,butonlythinkofmarriage’(Boys,NigerlowerperformingIQS).
“Betweentheboysandthegirlswhoexpressesmorefreedominclass?
Theboysarefreer.
Idonotknowhowtheyfeel.
Why?
Becausetheyareshy
Theyarenotsupposedtobehaveliketheboysdo.’
(Boys,BauchihigherperformingIQS)
Insomeofthequalitativecases,girlsperceivethefacilitatortobemoreconsiderateofboysandtogetangryifgirlsanswertoomanyquestionsinclass(NigerlowerperformingIQS).Theperceptionthatgirlsare‘shyer’maydirectlyrelatetotheattitudesoffacilitatorstowardsboysandgirls.IntheNigerhigherperformingIQSgirlswereasactiveandvocalinclassasboyswere,andconsideredthemselvesintheFGDtobemoreintelligentthanboys.IntheotherIQSs,girlswouldquietlygiggleinresponseandboyswouldanswerclearlythattheyweredefinitelymoreintelligent.
ThecaseoftheNigerhigherperformingIQShighlightstheeffectofthepresenceofastrongfemalerolemodel.Inthiscase,girlssaidthatofcoursegirlsweremoreintelligentbecause‘lookatMallama’(Niger,higherperformingIQS).ThisIQShasafemaleproprietorwhoisastrongcharacterinthecommunityandwhorunstheschoolandmanagesfacilitators.Sheisalsothemostqualifiedintermsofformaleducation,andspeaksfluentEnglish.Thisclearlycreatedasenseofconfidence,notonlyamongstthegirls,butalsoamongstmothers,believingthattheirdaughterscouldgoontostudyandbecome‘likeMallama’(Parents,NigerhigherperformingIQS).Thepresenceofastrongfemalerolemodelthusseemsimportant.
However,thefactthatanIQShasfemalefacilitatorsdoesnotnecessarilymeanthatgirlswilldisplayahigherlevelofconfidence.ThetwoIQSsvisitedinNiger(lowerandhigherperforming),whichhadfemalefacilitators,showedthemostevidentdifferences,bothinfacilitators’attitudestowardsgirlsandgirls’agencyintheclassroom.Thus,itmaybemoreimportantthatthefacilitatorhaspositiveattitudestowardsgirlsandthatsheisalsoempoweredwithintheschool,thanthatthefacilitatorisfemale.
AllIQSsvisitedusesomecorporalpunishment.Pupilsperceive‘flogging’asthemainthingtheydislikeaboutschoolandthatdetersthemfromattendingschool.ThelevelofopennessaroundcorporalpunishmentvariessignificantlybetweenIQScases.Insomeschools,facilitatorsandstakeholdersreportneverusingcorporalpunishment,whilstchildrensaytheyarefloggedorbeatenwhentheymisbehaveoraretooloud(NigerlowerperformingIQS).Inothercases,facilitatorsareopenabouttheuseofcorporalpunishmentandconsideritanappropriatewayofdiscipliningpupils.Intheseschools,CBMCssaidthatoneoftheirresponsibilitiesistospeaktofacilitatorsaboutbetterwaysofdiscipliningchildren(Bauchi,typicalIQS).IntheBauchitypicalIQS,theMallam/headteacherreportedthattheuseofthecanenolongertookplace.However,themathsfacilitatoropenlyspokeofusingit.Thisindicatesthatwhiletheunderstandingattheleadershiplevelisthere,thepracticehasyettobecurbed.Thereasonsforthislackoftranslationintopracticeareunclear.Inthehigher
EvaluationoftheGirl’sEducationProject3
©EDOREN 208
performingIQS,facilitatorssaytheydonotusecorporalpunishmentunlesstheyreallyhaveto:‘Ihatedoingit,beatingthemisthelastresortIfallbackto’(TPD,NigerhigherperformingIQS).Still,somepupilsspeakofbeatingsiftheyspeakHausainschoolwhentheyshouldbespeakingEnglish,oriftheymaketoomuchnoise.Interestingly,allpupilsconsiderpublicprimaryschoolsaslessofasafespace:‘Nobodycanesusherebutattheformalschooltheycaneusandthatdiscourageusfromattendinglessonsthere’(BoysFGD,BauchihigherperformingIQS).Itwillbeinterestingtoexplorefurtheratmidlinealternativemeansofdiscipliningchildren.Severalrespondentsmentionlearningintrainingthattheyarenottobeatchildrensinceitwilldeterthemfromattendingschool.Thisappearstohavebeenactedupontosomeextent.
InmostIQSsvisited,girlslackthespacetoplay.Theperceptionsarethatboyshavemorefreedomtoplay,thoughgirlsexpressawishtoplaythewayboysdo.Oneaspectrelatedtothisissecurity.SomeschoolsinBauchicomplainaboutthelackofafencearoundtheIQS,andissuesofboysdisturbinganddistractinggirlsfromreading(CBMC,BauchihigherperformingIQS).Thisissuefurtherhasthepotentialtoaffectinstructionaltime.Thoughrespondentssaygirlsendschoolatthesametimeasboys,theperceptionisthatitisdangerousforgirlstoattendlatesessions,andthisisresultinginlessinstructionaltime.
Bothboysandgirlsreportbeinguncomfortableinschool,andsomeIQSslackaccesstofacilitiessuchasrunningwater,toiletsandshade.InbothhigherperformingIQSstherearebathroomfacilitiesavailableforbothboysandgirls.However,thesearenotconsideredsufficientandpupilssaythatboysoftengointhebushwhilstgirls‘gobackhometoeasethemselves’(Boys,BauchihigherperformingIQS).InBauchilowerperformingandtypicalIQSsrespondentssaythatgirlscanusethefacilitiesintheMallam’shouse.However,therearecontrastingaccountsofwhetherornotthistakesplaceinpractice.
‘Thechallengesthatboysandgirlsfaceinthisschoolisthatduringlessonsachildmaywanttoeasehimselfbutyouknowforboysitiseasierforthemtofindwheretoeasethemselves,butforagirlshecannoteaseherselfanywhere.Sheneedsagoodplacewhichwedon’thave.’(TPD,BauchitypicalIQS)
InNigerlowerperformingandtypicalIQSsgirlsdonothaveanybathroomfacilitiesbutreturnhometoeasethemselves.Thereisarecognitionofthisasanissue,sincewhengirlsgohometheyrarelycomebacktoschool,leadingtogirlsmissinglargepartsofclasses.
4.5.2.6 Genderconsiderationsinschoolmanagement
‘Sogirlsshouldhaveagoodenvironmentwheretheycaneasethemselvesandalsoagoodplacetoseat.’(TPD,BauchitypicalIQS)
Thereisevidencetosuggestthatlargedifferencesinschools’cultureshaveaninfluenceongirls’educationalperformanceandtheiragencyindemandinggenderequality165.Ensuringgenderresponsivenessacrosscross-schoolsystems,structuresandprocessesseekstoaddressinstitutionalculturesinamannerthatwillpromotegirls’education.Thisincludesnotonlyteachingandthecurriculum,butalsocommunication,managementstructures,andpremises.166Untherhalteretal.(2014)identifyseparatetoiletfacilitiesforboysandgirls,andeffortstochangetheinstitutional
165Untherhalteretal.(2014).166Atthill,CandJha,J(2009)TheGenderResponsiveSchool:AnActionGuide,CommonwealthSecretariat.
EvaluationoftheGirl’sEducationProject3
©EDOREN 209
culturethroughadvocacyandcapacitybuilding,tobeimportantmeansofcreatingagirl-friendlyschool.
Onecanthusgroupaspectsofgirl-friendlylearningenvironmentsintotwobroadcategories:institutionalandinfrastructural.Institutionalcultures–acceptedtraditionsandbehaviourswithinaschoolcontext–arecriticalinthetransferofgendernormsthatcaneitherbenefitornegativelyaffectgirls’education.Infrastructuralrealitiesrefertotherecognitionofgirls’specificneeds(includingbiologicalneed),andinparticularissuesrelatedtosafetyandacceptedsocialnorms.
IntheIQSsvisited,schoolleaderswerewillingtoreflectonandconsidergenderequityinschoolplanning.CBMCs,asmentioned,saythatoneoftheirmainresponsibilitiesistoincreaseawarenessoftheimportanceofformalschoolingforbothboysandgirls.InallIQSsinthequalitativesample,CBMCsaytheyactivelyworktosensitiseparentsandtoconvincethemtosendtheirchildrentoschool.SeveralCBMCsconsidermotherstobethemainopponentsofthischange.Theynotetheneedtobepatient,andtoallowthischangetooccur–whenmotherswillrealisethevalueofeducationforallchildren.Theroleofmothersasabarriertogirls’educationcriticallyrelatestotheeconomicrealitiesoftheirincome-generatingandpovertyalleviationactivities.
‘ThemainwayCBMCisassistinginthewomenaspectisthatwehavesomechallengeswithsomeparents.Somechildrenmaydesiretocometoschool,buttheirmotherswillhinderthembysendingthemonvariouserrands.Whatwedoisgofromhousetohousetosensitiseourfellowwomenontheimportanceofeducation.’(CBMC,BauchilowerperformingIQS)
SomeCBMCs,suchastheCBMCforBauchihigherperformingIQS,alsohighlighttheimportanceoffemalerolemodels.TheideahereisthatifwomenareencouragedtobecomemembersoftheCBMC,andgirlsseetheirmothersplayinganactiveroleontheCBMC,thiswillencouragethemtostayinschool.Inspiteofengrainedgenderdifferencesinallthecommunitiesvisited,respondentsreportthatchangeshaveoccurred:‘Letmetellyou,seethesewomen?Beforeweonlygreetcasually,butlookatthemnow.Amongus.Interactingwithus.Thiskindofthingwasnotpossiblebefore.’(CBMC,BauchihigherperformingIQS)
Thereisanawarenessthatinstitutionalculturesmayaffectgirls’learningoutcomes:forexample,thetraditionofhavinggirlssitatthebackoftheclass.IndiscussionwithfacilitatorsandCBMCssomeparticipantsreportedthatseatingarrangementsinclassmaybeunfavourableforgirls.However,thereisafearofmixingboysandgirlsintheclassroomforseveralreasons.OneisthatitissimplynotwhatIslamicteachingssay.Secondly,itisfearedthattherewillbeinappropriaterelationsbetweenboysandgirlsiftheyareseatedtogether.Ifgirlssitatthefrontoftheclassroomthiswouldfurtherallowboystowatchthemfrombehind.
‘WhyIfeeltherewouldbeproblemisbecauseboysandgirlsrelationship.Aboywilltouchagirljusttolookforhertrouble,sothatiswhyitisgoodtoseparatetheboysfromthegirlswithdifferentseatingspaceandwithouttouchingeachother.Soeveniftheteacherisnotaroundtheysitinthatposition.Everygirlwithherspaceandeveryboywithhisspace.’(CBMC,BauchitypicalIQS)
IntheNigertypicalIQS,theschoolhasrespondedtothisissuebyconstructingtworoomsinthenewschoolbuilding,oneforgirlsandoneforboys.Byseparatingboysandgirlsinthismanner,girlswillbeabletositupfront,closertotheboard,whilststillnotmixingwiththeboys.Theperceptionisthatthiswouldsolvetheissue.However,thismayhaveseveralimplications.Forexample,thefacilitatormaynotgiveasmuchattentionorefforttoteachingthegirls.Still,itmayhaveapositiveimpact,sincegirlsmightperceiveaseparateclassroomtobeasaferspaceforlearning.Thiswillneed
EvaluationoftheGirl’sEducationProject3
©EDOREN 210
furtherexplorationatmidline–ifthecommunitydoesinfactcompletethebuildingandimplementthenewsegregatedteaching.
CBMCsappeartohaveconsideredtheattendanceandretentionofgirlswhenallocatingresources.AllIQSsreportthattheyarewillingtoinvestcommunityresourcesinschoolinginfrastructurethatbenefitsgirls.CBMCsperceiveakeychallengeforgirls’attendancetobelackofuniforms.SeveraloftheIQSsinNigereitherprovide(NigerhigherperformingIQS)orhavestartedtoprovide(NigertypicalandlowerperformingIQSs)schooluniforms(Hijabs)togirls.However,inNigertypicalIQS,thefundsonlysustainedabout15pupilsreceivinguniforms,andthustheCBMCdecidedthatitwasbettertoallowchildrentowearanythingtoschool,sothatthechildrenwhodidnotreceiveuniformswouldnotfeelbad.Girlsagreethatuniformswouldmakethemfeelencouragedtocometoschool.
Otherperceivedneedsforcreatingagirl-friendlyenvironmentarebuildingtoiletsandprovidingwaterinschool–recognisingthatthesefacilities(inparticular,toilets)aremoreessentialforgirlsthanforboys.SomeIQSshavealsoinvestedinkettles(‘teapots’)167.Girlsreportkettles/’teapots’asbeingsomethingtheyneedinordertogotothetoilet:‘ifoneaskspermissionfromtheteacher,ifshehasnothingwithher,shewillhavetogolikethat’(Girls,BauchihigherperformingIQS)–meaningthatshewillhavetogowithoutawatercontainer.However,thoughIQSsmayallocateresourcestoinvestinfacilitiesandmeansthatcouldimprovethelearningenvironmentforgirlsitwillbeessentialatmidlinetoexploretheextenttowhichtheseareactivelyreservedforandusedbygirls.Forexample,communitiesalsousekettles/’teapots’inablutionspriortoprayer,sothereisapotentialthatboyswillhavepriorityusageofkettles/’teapots’andthatnonewillbeavailableforgirlstouse.
Givensafetyconcerns,distancetoschoolisparticularlyimportantforgirls.Somecommunitiesreportchallengeswithboyscomingtotheschoolandharassingthegirls.Inonecommunity,theCBMChasstartedavigilantegroupofparentsthatmonitortheschoolandevenwalkgirlshome.Still,thequalitativeteamobservedboysaskingafacilitatortodeliveramessagetotheschoolproprietor:‘nomatterhowmuchvigilantetheMallambringstoguidetheschoolwewillstillseethosegirls’(Bauchi,higherperformingIQS).Thisunderlinesthechallengesofensuringsafetyforgirls.
4.6 Analysisofthedata-ContributionClaim3:Moreeffectiveteachingofformalsubjectsandanimproved,girl-friendlyschoolenvironmentcontributetoimprovedlearninglevels,particularlyamonggirls
ThissectionfocusesonthebaselinedataofpupilslearningoutcomesinIQSs,andfactorsthatinfluencelearningoutcomes.Wewillpayparticularattentiontolearningoutcomesforgirls.Thequalitativefindingsalsodiscussfactorsthatinfluenceretention,asecondaryoutcomeintheToC.168Thequantitativedataanalysisispresentedfirst,followedbytheanalysisofthequalitativecasestudyfindings.
167Whatisreferedtoas‘kettles’,isabasic‘teapot’.Thesearefilledwithwaterforgirlstousethemwhentheygotothebathroom.Theyarealsousedforablutionpriortoprayer.168TheenrolmentdataavailableintheIQSwerenotofsufficientqualitytoquantitativelyreportonretention.
EvaluationoftheGirl’sEducationProject3
©EDOREN 211
4.6.1 Quantitativeanalysis
4.6.1.1 PupilsinIQSs
NumberofpupilsinIQSs
DatacollectiononschoolenrolmentsuggeststhatthegenderratioinIQSsisquitebalanced.BothgirlsandboysseemtobeattendingIQSs,andthisholdstrueacrossbothstates.Giventhelownumberofschoolsthathadformalenrolmentrecords(only30%),thesedatacouldonlybecollectedforasmallshareofthesample,hencethisfindingcannotbetakenasrepresentativeofthesampleasawhole.Rather,itshouldbeviewedasadescriptionofthestateofsomeschoolsandnotbeusedtogeneralisefindings.Thelackofformalrecord-keeping,aswellasthefluidorganisationandtimingofclasses,provedtobearealchallengeinregardtoobtainingformalinformationonschooloperationswithinIQSs.Thepupilgenderratio,i.e.theratioofenrolledgirlstoboys,wasfoundtobeslightlyhigherthanonefortheschoolsasawhole,andinearlygradeenrolments.ThissuggeststhatthereareatleastasmanygirlsenrolledinIQSsasthereareboys.Thisholdsacrossbothstates,withfiguresbeingslightlymoreinfavourofgirlsinBauchi.However,giventhesamplesizeissuesinthisindicator,wecannotreportabsolutevaluesorsignificancevalues.
Toaddressthedatalimitationimposedbyalackofenrolmentrecords,wealsocollecteddataonpupilattendanceonthedayofthevisit,throughamanualcountingexerciseofallthepupilsstudyingintegratedsubjectsinP2ortheequivalentlevelonthedayofthevisit.Thesedataexistforalmostalltheschools,andshowthatjustasmanygirlsasboyswereattendingintegratedsubjectclassesinIQSsonthedayofthevisit.Thisisaninterestingfinding,whichsupportstargetingIQSsasameansofeducatinggirls.However,thereisthecontinuedriskthattheattendanceonthedayofthesurveyisnotrepresentativeofwhathappensattheschoolonadailybasissincetherewasanincentiveforschoolstomisrepresentthisinformation.Effortsweremadetomitigatethisrisk,butitisnotpossibletoaddressthiscompletely.ItisalsointerestingtonotethatthegenderratiowasslightlyhigherinBauchiandincaseswheretheproprietordoubledastheheadteacher,aswellaswheretheheadteacherwastrained.
Thenumberofpupilsregisteredforintegratedsubjectclassesrangefromasfewas20toasmanyas400childrenatalllevelsintheschool.Ofthis,themajorityareattheearlygradelevel,i.e.P1–P3level.Acrossstates,aslightlyhighershareofNigerschoolshadenrolmentabove100pupilscomparedtoschoolsinBauchi.
Itwasnotpossibletocalculatetheteacher–pupilratioforschoolsasawholegiventhelackofqualitydata.Instead,othersourcesofevidencehavebeenusedtoestimatethis.Anindicationofclasssizecomesfromthedataonlessonobservations.Theclassroomobservationdatasuggestsameanpupilteacherratioof45andamedianof40.Thisrangeisalsoquitevaried,withfromfourto147childrenbeingtaughtbyoneteacher,withnonoticeabledifferencesinthepupil–teacherratioacrossstates.ThismeansthatclasssizeisincrediblyvariedacrossIQSsinbothstates.TheattendanceofboysandgirlsinthelessonsobservedalsosuggeststhattherearealmostequalnumbersofboysandgirlsstudyingintegratedsubjectsinIQSs,withslightlymoregirlsthanboysinP2lessonsobservedonthedayofsurvey.
Pupilcharacteristics
Atotalof576pupilsweresurveyed,ofwhich296werefromBauchiand280werefromNiger.Thesamplingstrategytargetedanequalproportionofgirlsandboys.Thiswasfairlywellachieved:48%
EvaluationoftheGirl’sEducationProject3
©EDOREN 212
ofpupilssurveyedweregirls.ThereissomevariationbetweenBauchiandNiger,with44%femalepupilsintheformerand52%inthelatter.AlmostallpupilssurveyedwereenrolledinP2oranequivalentlevel.
TheaverageageofaP2-levelpupilinanIQSis8.3years.ChildreninBauchiwereolder,onaverage,comparedtochildreninNiger,byhalfayear,withameanageof8.5years.Itisimportanttopointoutthattheseareself-reportedagesofthechildren,anddonotexistfortheentiresampleas32%ofthechildrendidnotknowtheirage.FortheremainingchildrenthereportedagehasbeenplottedseparatelyforboysandgirlsinFigure71below.Thisshowsthepercentageofboys(andgirls)thatareofaparticularage,asashareoftotalboys(andgirls)inthesample.Thisindicatesthatthepatternofparticipationintheearlyyearsissomewhatdifferentforboysandgirls.BoysinP2equivalentareyoungerthangirls,onaverage.Thissuggestsearlierentryintoschoolingforboysovergirls.Thiscouldalsosuggestthatgirlsarebeingheldbackmoreoften.Thisinformationcannotbeconfirmedusingrepeaterdata,however,duetoalackofformalrecordsattheschoollevel.
Figure71: Ageofpupilsbygender
Amajorityofthepupils(75%)speakHausaathome,followedbyNupe(22%)andFulfude(2%).NoneofthepupilsreportedspeakingEnglishathome.Thecross-statedifferencesbylanguagespokenathomearedepictedinthefigurebelow.AlloftheNupespeakersareconcentratedinNiger,whileFulfudeandKanuriarespokenbypupilsinBauchi.
0%
5%
10%
15%
20%
25%
30%
35%
4 5 6 7 8 9 10 11 12 13 14
Percen
tageofp
upils
Girls Boys
EvaluationoftheGirl’sEducationProject3
©EDOREN 213
Figure72: Languagesspokenathome,bystate
Itisinterestingtocomparethistothelanguagesspokenbythefacilitatorsandthelanguageinwhichteachingtakesplaceatparticularschools.Accordingtotheinterviewswithfacilitators,thefacilitatorsinBauchiandNigerspeakalloftheabove-mentionedlanguages.Forexample,39%ofthefacilitatorsinNigerreportedbeingabletospeakNupe,while13%offacilitatorsinBauchireportedbeingabletospeakFulfulde.Thusitappearsthatatthestatelevel,pupillanguagecorrespondswellwiththelanguagethatfacilitatorsreportbeingabletospeak.
Thesurveyfindingsindicatethatteachingwithinschoolsistakingplaceinavarietyoflanguages.However,teachingappearsnotalwaystobetakingplaceinthemothertongueofthepupils:inparticularwhenitconcernsminoritylanguages,suchasNupe,FulfuldeandKanuri.Accordingtotheinformationcollectedduringheadteacherinterviews,oneofthreelanguagesisusedtoteachmathematicsinschoolsattheP1–P3level.Figure73showsthatalthoughNupeisusedinschoolsinNiger,theshareofNupelanguageuseislessthantheshareofpupilsthatspeakNupe.Whencomparingthelanguagepupilsreportedspeakingathomewiththelanguageofinstructionreportedbytheheadteacherduringinterviewwefindthatinabout82%ofthecasespupilswerebeingtaughtintheirmothertongue,andthiswasexclusivelyeitherHausaorNupe.Intherestofthecases,therewasamismatchbetweenthepupils’mothertongueandthelanguageofinstruction.ThissuggeststhatchildrenthatspeakNupesufferfromthegreatestlinguisticdisadvantageattheschoollevel,intermsoflanguageofinstructionandthenumberofchildrenaffected.Inaddition,nativespeakersofotherminoritylanguages,suchasFulfudeandKanuri,alsosufferadisadvantage,thoughthenumberofchildrenaffectedissmallerinabsoluteterms.
3%
93%
3% 0% 1%1%
54%
0%
43%
2%
Fulfulde Hausa Kanuri Nupe Others
Bauchi Niger
EvaluationoftheGirl’sEducationProject3
©EDOREN 214
Figure73: Languageofinstruction,bystate
Almostallofthepupilsinthesamplecouldhear,see,speakandwrite.Onlyonepupilwasvisuallyimpaired,andfourwereunabletowrite.Allfiveoftheseweremalepupils.Thiscouldperhapsindicatethatparentsarenotsendingdisabledchildrentoschool,thoughthereisnowaytoverifythisindication.
Alookathouseholdassetownership,asreportedbychildren,suggeststhatchildreninIQSsacrossthetwostatescomefromhouseholdsthathavebasicassets,likemats,chairs,bedsormattresses,morefrequentlythanfromhouseholdswithluxuryassets,likecomputers,cameras,airconditioningandTV.Thisisnotsurprisinggiventhatpovertyisprevalentintheseareas.Cross-statedifferencessuggestthatchildreninBauchicomefromhousesthathaveslightlybetterassetownershipthanchildreninNiger,asdepictedbygreaterownershipofbasicassetsaswellasluxuryassets.However,theshareofthepopulationthatownsluxuryassetsissolowoverallthatthisdifferenceisunlikelytotranslateintosubstantialdifferencesacrossthetwosamples.Forexample,only8%oftheBauchipupilsamplereportedowningeitheracomputerorcamera,whilethisis4%and5%,respectively,inNiger.
Pupils’schoolexperience
PupilsintheIQSsamplewereaskediftheycurrentlyattendedanotherschool.Itissurprisingtoseethat40%ofthechildrenreportedcurrentlyattendinganotherschoolinadditiontotheschoolatwhichtheyweresurveyed.Thisisnotrestrictedtoschoolsthatteachintegratedsubjects.Thepercentageofpupilsattendinganotherschooldoesnotchangebypupilgender.Thecross-statedifferenceissignificant,however:72%ofpupilsinBauchireportedcurrentlyattendinganotherschool,comparedtoonly13%ofpupilsinNiger.InthecaseofBauchithisotherschoolmostlyreferredtoapublicprimaryschool,whileinNigerthiswasonlythecaseforhalfthepupilsattendinganotherschool–hencetheotherschoolcouldalsorefertoanotherreligiousschools,avocationaltrainingschooloranyotherprivateschool.Thecross-statedifferenceispartydrivenbytheurban–ruraldifference:about70%ofthestudentsinurbanareaswereattendinganotherschool,comparedtoaround35%ofthestudentsinaruralarea.Thefactthatpupilsareattendingotherschools,particularlyinBauchi,isimportanttonotebecauseitmayaffectthetimethattheyhaveavailabletoattendtheintegratedcurriculumclasses,anditindicatesthatasubstantialshareofIQSpupils
50%
100%
0%
67%
66%
17%
MathematicsistaughtinEnglish
MathematicsistaughtinHausa
MathematicsistaughtinNupe
PercentageofschoolsNiger Bauchi
EvaluationoftheGirl’sEducationProject3
©EDOREN 215
haveaccesstoformaleducationthroughachannelotherthantheGEP-supportedIQSs.Thisislikelytohaveimplicationsfortheimprovementsinlearningattributabletotheprogramme,aspupillearningofintegratedsubjectswillalsobeinfluencedbytheirattendanceatotherschoolsthatteachintegratedsubjectsduringthecourseoftheevaluation.
Onlyaverysmallpercentageofpupilsreporteddifficultiesingettingtoschool.Lessthan4%indicatedthattheyexperiencedifficultiesingettingtoschool,andthisdidnotvaryacrossstateorpupilgender.Ofthosewhoreporteddifficulty,themajorityreportedtrafficanddangerincrossingroadsasthemaindifficulty.Longdistancestogettoschoolandhazardousterrainthroughwhichtotravelweretheonlyotherdifficultiesmentioned,bylessthan20%ofthesamplethatreportedadifficulty.ThissuggeststhattheIQSsareaccessibleandwelllocatedinthecommunities.
Asmallshare(only6%)ofthesampledpupilswereboardersattheIQSs.Asexpected,boysmadeupalargershareofthisthangirls.12%oftheboysandonly1%ofthegirlswereboardingstudents.AgreatershareofpupilsinBauchi(10%)wereboarderscomparedtochildreninNiger(3%).Thisisinlinewiththeschool-leveldata,whichsuggestthatalargershareofschoolsinBauchiwereboardingschoolsandwereclosertothetraditionalIQSoperationalstructure.
4.6.1.2 Studentlearningoutcomes
Learningoutcomemeasurement
TheHausaliteracyassessmentisdesignedtotestthesameliteracyknowledgeandskillsastheEnglishliteracyassessment.Itemsarenotmerelytranslated,butratherparallelitemsaredevelopedtotestsimilarconceptswhenappliedtotheHausalanguage.TheEnglishliteracyassessmentcontains13items,witheachitembeingmadeupofseveralsub-items.Theassessmenttestsarangeofliteracyknowledgeandskillsacrossthepre-literacy,emergingandbasicliteracyranges.Knowledgeandskillsincludedletterrecognition,phonologicalknowledge,printconcepts,oralliteracy,verbalcomprehension,initialsoundsandletters,readinghighfrequencywords,verbalandwrittengrammar,writinghighfrequencywords,readingfluency,andcopyingandspellinghighfrequencywords.EnglishliteracyandHausaliteracyassessmentswereconstructedfollowingfivesteps,includingclarifyingconstructs,testtargeting,administration,psychometricanalysis,drawingbenchmarksandsecondarydataanalysis.
Asstatedearlierinthisreport,learningoutcomesaretheresultofacomplexandmultivariateschoolsystem,andteacherandstudentfactors,whichareinfluencedbysocialandeconomicconsiderations.Thewaysinwhichthesefactorsinteractarecomplexandcanneverbefullymappedoraccountedforinananalysis.Therefore,studentlearningoutcomesneedtobeinterpretedwithcare.Forexample,oneofthemostimportantpredictorsoflearningoutcomesiswhatthepupilsbringwiththemintotheclassroom.Intellectual,socialandculturalfactors,aswellashousehold-specificfactors,significantlypredictpupilperformanceinmoststandardisedassessments(Hartas2011,OuthredandBeavis2012,OuthredandBeavis2013,Mayer,1997,DahlandLochner2005,VanderBer2015).Therefore,whilecomparisonsoflearningoutcomesbetweenstudentsfromdifferentschoolingsystemscananswerimportantequityquestions,cautionneedstobeexercisedwhendrawingcausalinferences.
ThissectionofthereportdescribesthelearningachievementofP2pupilswithinthesampledIQSsinBauchiandNiger.Thefollowingsectionsdescribekeyteacher-andschool-levelindicatorsonwhich
EvaluationoftheGirl’sEducationProject3
©EDOREN 216
datawerecollected,andinvestigatetheextenttowhichfactorsbelongingtothesedifferentanalyticalcategoriesareassociatedwiththeobservedlearningoutcomes.
Hausaliteracylearningoutcomes
ThefindingsoftheHausaliteracyassessmentsuggestthatthemajorityofpupilshaveyettoacquirebasicliteracyinHausa.Only2.5%ofthepupilsinNigerandBauchiwereabletocompletesomeofthetaskswithinthebasicliteracyrange.Afurther1.8%ofpupilswereabletodemonstrateemergingliteracyskills,whilethevastmajorityofpupilswerefoundtohaveonlypre-literacyHausaskills(pleaserefertoTable14inSection3.4.3fordescriptionsofeachoftheHausaproficiencyranges).
Figure74representsthedistributionoftheperformanceinHausaliteracy.TheaxisrepresentstheHausascalescorederivedfromthepsychometricanalysisofpupilperformanceontheassessment.Ascanbeseeninthefigure,thepeakofthedistributionfallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy,indicatingthatsubstantialeffortwouldberequiredtoattainasignificantincreaseintheshareofpupilsmovingfrompre-literacytoemergingliteracy.
Figure74: DistributionofHausaliteracyproficiency
Englishliteracylearningoutcomes
TheresultsoftheEnglishliteracyassessmentsuggestthatpupilproficiencyinEnglishafterayearofschoolingisslightlyhigherthanpupilproficiencyinHausa,although,again,alargemajorityofpupilshaveyettoacquireanyknowledgeorskillsbeyondpre-literacy.Ofthepupilsinthesample,3.4%wereabletodemonstratesomeoftheknowledgeandskillsthatfallwithintheemerging
0 500 1000 1500 2000 2500
263.23344.18391.83432.47468.25499.18526.67551.65574.30594.73612.97629.03643.14655.91668.05680.30693.62709.37730.13759.89799.60848.81
2.5%ofthepupilsdemonstratedGrade2Hausaliteracyskills.Thislevelof
proficiencycanbedescribedasbasicliteracy.
1.8%ofpupilsdemonstratedknowledgeandskillswithintherangeexpectedbytheP1curriculum.
Thislevelofproficiencycanbedescribedasemergingliteracy.
Over95.7%ofpupilsdemonstratedpre-literacyHausaskills.Childrenachievingwithinthepre-literacyproficiency
rangedonotyethaveemergingliteracyskills.
EvaluationoftheGirl’sEducationProject3
©EDOREN 217
literacyrangeand2.4%wereabletodemonstratesomeoftheskillsthatfallwithinthebasicliteracyrange.Theremaining94.1%wereonlyabletodisplaypre-literacyskills(pleaserefertoTable15inSection3.4.3fordescriptionsofeachoftheEnglishproficiencyranges).
ThefigurebelowdepictsthedistributionofperformanceacrosstheEnglishliteracyscale.TheaxisrepresentstheEnglishscalescorederivedfromthepsychometricanalysisofpupilperformanceontheassessment.AswithpupilperformanceinHausa,thecuspofthedistributionfallswellbelowthecut-offpointbetweenpre-literacyandemergingliteracy,suggestingthatsignificanteffortwouldbeneededtoachievelargeadvancesintheshareofpupilsachievingemergingorbasicliteracy.
Figure75: Englishliteracyproficiencydistribution
Numeracylearningoutcomes
Pupils’performanceonthenumeracyassessmentwasconsiderablybetterthanthatontheHausaandEnglishliteracyassessments.Intotal10.6%ofpupilsdemonstratedP2-levelnumeracyskills.Afurther68.9%demonstratedknowledgeandskillswithintherangeexpectedbytheP1curriculum.Theremaining20.5%hadnumeracyskillsexpectedatpre-schoollevel.Table29describestheknowledgeandskillsofpupilsachievingwithineachoftheproficiencyranges.TheseproficiencyrangesarealsodirectlycomparabletotheESSPINCSproficiencyranges.
0 500 1000 1500 2000 2500 3000202.78
203.56
305.91
337.10
359.37
381.54
407.18
411.74
432.60
453.62
482.66
495.07
502.02
516.29
534.65
550.54
563.77
587.17
611.49
657.78
Over94.1%ofpupilsdemonstratedpre-literacyEnglishskills.Childrenachievingwithinthepre-literacyproficiency
rangedonotyethaveemergingliteracyskills.
2.4%ofthepupilsdemonstratedGrade2EnglishLiteracyskills.Thislevelof
proficiencycanbedescribedasbasicliteracy.
3.4%ofpupilsdemonstratedknowledgeandskillswithintherangedescribedasemergingliteracy.
EvaluationoftheGirl’sEducationProject3
©EDOREN 218
Table29: Numeracyproficiencyleveldescriptions
Proficiencyrange Descriptionoftheknowledgeandskillsofpupilsachievingwithinthisrange
Pre-numeracy
Childrenachievingatpre-schoollevelareabletodemonstrateknowledgeandskillinatleastsomeofthetasksthatareconsideredtobewithintherangeofpre-numeracyproficiency.Theseskillsincludebeingabletocomparethelengthoftwostraightlines,usenon-standardunitsofmeasuretocomparethecapacityofthreecontainers,andcountto10.
Emergingnumeracy
ChildrenachievingatemergingnumeracylevelareabletodemonstrateknowledgeandskillsinatleastsomeofthetasksthatareconsideredtobewithintherangeofGrade1proficiency.Theseskillsincludebeingabletorecogniseandcompleteasequenceofthreetwo-digitnumbersthataremultiplesoffive;subtractaone-digitnumberfromatwo-digitnumbercrossingthe10boundary,involvingmoney;subtractaone-digitnumberfromatwo-digitnumberfromoneto19;readananalogueclocktothehour;usenumberknowledgetocompleteasequenceofthreenumbersbelow10;understandunitfractions(1/3)andusethistofindfractionsofacommon2Dshape;understandunitfractions(¼)andusethistofindfractionsofacommon2Dshape;addtwothree-digitnumbers;usenon-standardunitsofmeasuretomeasurelength;chooseastrategytoaddathree-digitnumberandatwo-digitnumbercrossingthe10boundary,involvingmoney;comparethelengthoftwostraightlines;readananalogueclocktothehour;addtwothree-digitnumbersverticallythataremultiplesoffive,involvingmoney;chooseastrategytoaddathree-digitnumberandatwo-digitnumbercrossingthe10boundary,involvingmoney;recallthetwotimestable;subtractaone-digitnumberfromatwo-digitnumberfromoneto19;understandunitfractions(1/4)andusethistofindthefractionsofacircleandasquare;andaddtwotwo-digitnumbersthataremultiplesoffive.
Basicnumeracy
ChildrenachievingatGrade2(basicnumeracy)areabletodemonstrateknowledgeandskillsinatleastsomeofthetasksthatareconsideredtobewithintherangeofGrade2proficiency.Theseskillsincludebeingabletousenon-standardunitsofmeasuretocomparethecapacityofthreecontainers;subtractatwo-digitnumberfromathree-digitnumber(bothmultiplesoffive)crossingthe10sboundary,involvingmoney;readananalogueclocktothehourandhalfhour;recogniseandcompleteasequenceofthreenumbersthataremultiplesof50andlessthan200,andcompleteasequenceofthreetwo-digitnumbersthataremultiplesoffive;subtractatwo-digitnumberfromatwo-digitnumber;addtwothree-digitnumbersthataremultiplesoffive,involvingmoney;usestandardunitsofmeasuretomeasurethelengthofasmallobject(cm);subtractaone-digitnumberfromatwo-digitnumberfromoneto19crossingthe10boundary;namecommon2Dshapes;addtwothree-digitnumberscrossingthe10boundary;subtracttwothree-digitnumbers;addandsubtractlengthordetermineareasize;chooseastrategytoaddathree-digitnumberandatwo-digitnumbercrossingthe10sboundary,involvingmoney;extendcountingpast800andcountin10s;addtwothree-digitnumberscrossingthe10sboundary;extendcountingpast1,000andcountin100s;andidentifyandcountthefacesoffamiliar3Dshapes.
Ascanbeseeninthefigurebelow,thecuspofthedistributionfornumeracyachievementfallsjustabovethecut-offpointbetweenpre-numeracyandemergingnumeracy.Thisindicatesthatlargegainsinthepercentageofpupilsfallingwithintheemergingnumeracyrangecouldbeachievedas
EvaluationoftheGirl’sEducationProject3
©EDOREN 219
manychildrenareonthecuspofachievingemergingnumeracy.Gainsinthepercentageofpupilsachievingbasicnumeracywillbemorechallengingandwillrequiremoreeffort.
Figure76: Distributionofnumeracyproficiency
Quantitativedataonassociatedfactors
Theanalysisoffactorsassociatedwithachievementhighlightsthattheverylowlevelsoflearningintheinterventionschoolsarelargelyconsistentacrossgroups,althoughboysdoperformslightlybetterthangirls.AscanbeseeninTable30,girlsareover-representedinthepre-literacyandpre-numeracyproficiencyrangesandunder-representedinthebasicliteracyandnumeracyproficiencyranges.
Table30: Learningoutcomesbygender
0 200 400 600 800 1000 1200 1400 1600 1800 2000
185.51
322.29
397.32
452.08
490.81
513.70
529.90
543.92
557.76
572.69
589.46
608.00
628.10
650.58
677.49
715.34
Hausa English Numeracy
Girls Boys Girls Boys Girls Boys
Pre-literacy/numeracyskills 97.5% 94.0% 96.2% 92.2% 22.4% 18.5%
Emergingskills 0.4% 3.0% 2.0% 4.8% 70.7% 67.5%
Basicskills 2.0% 2.9% 1.8% 3.1% 7.0% 14.0%
20.5%ofpupilsdemonstratedpre-numeracyskills.Childrenachievingwithinthepre-numeracyproficiency
rangedonotyethaveemergingnumeracyskills.
68.9%ofpupilsdemonstratedknowledgeandskillswithintherangeexpectedbytheP1curriculum.
Thislevelofproficiencycanbedescribedasemergingnumeracy.
10.6%ofthepupilsdemonstratedP2numeracyskills.Thislevelofproficiencycanbedescribedasbasicnumeracy.
.
EvaluationoftheGirl’sEducationProject3
©EDOREN 220
Figure77showsthatallagegroupstypicallyachievewithintheHausapre-literacyproficiencyrange,andthattherearenosubstantialdifferencesbetweentheachievementofboysandgirlswithineachagelevel,withtheexceptionoffourandfive-year-olds.
Figure77: MeanHausascalescorebyageandgender(95%confidenceinterval)
Incontrast,inthecaseofEnglishliteracy(Figure78)andnumeracy(Figure79)theperformancegapbetweenboysandgirlswidensatthestageinthelifecycletypicallyassociatedwithpuberty(around12yearsofage).
Figure78: MeanEnglishscalescorebyageandgender(95%confidenceinterval)
0
100
200
300
400
500
600
700
4 5 6 7 8 9 10 11 12 13 14
Hausa
Boys Girls
0
50
100
150
200
250
300
350
400
450
500
4 5 6 7 8 9 10 11 12 13 14
Boys Girls
Cut-offpointbetweenpre-literacyandemergingliteracy
Cut-offpointbetweenpre-literacyandemergingliteracy
EvaluationoftheGirl’sEducationProject3
©EDOREN 221
Figure79: Meannumeracyscalescorebyageandgender(95%confidenceinterval)
Acrossthethreeassessments,pupilsinBauchiperformedslightlybetterthanpupilsinNiger,asrepresentedinFigure80,Figure81andFigure82.ThismaybeexplainedbythelargerlanguagediversityinNigerorthefactthatalargershareofpupilsinBauchiattendpublicprimaryschoolsatthesametimeasattendingtheIQS.
Figure80: MeanEnglishscalescores,bystate
0
100
200
300
400
500
600
4 5 6 7 8 9 10 11 12 13 14
Boys Girls
200
250
300
350
400
450
500
Bauchimean Nigermean
Cut-offpointbetweenpre-numeracyandemergingnumeracy
Cut-offpointbetweenemergingnumeracyandbasicnumeracy
EvaluationoftheGirl’sEducationProject3
©EDOREN 222
Figure81: MeanHausascalescores,bystate
Figure82: Meannumeracyscalescores,bystate
Overall,thebaselinefindingsindicatethatpupils’learningoutcomesinGEP3IQSsareverylow.Thismeansthatthereistremendousscopeofimprovementbutalsohighlightsthatinterventionswillneedtobecarefullytailoredsothattheyareappropriatelytargetedtoexistinglearninglevels.Furthermore,asnotedabove,inbothHausaandEnglish,thebulkofpupilshavepre-literacyskillsandmanyarealongwayfromthecut-offpointforemergingliteracy.Thissuggeststhatachievinglargegainsintheshareofpupilswhohaveemergingliteracyskillsorhigherislikelytorequireconsiderableeffort.
GirlsandboysappeartobeattendingIQSsequally.Atanyparticularlevelgirlsareslightlyolderthanboys,whichsuggeststhattheyareenteringschoollater.Pupilsalsocomefromvariedbackgrounds,intermsoflanguagespokenathomeandassetownership,andthisiscorrelatedwithpupillearningoutcomes.
200
250
300
350
400
450
500
550
600
650
Bauchimean Nigermean
300
350
400
450
500
550
Bauchimean Nigermean
EvaluationoftheGirl’sEducationProject3
©EDOREN 223
4.6.2 Qualitativecasestudyanalysis
ContributionClaim3bringstogethertheprevioustwoclaimsaroundeffectiveteachingandagirl-friendlyschoolenvironmenttoconsiderhowthesemaycontributetoimprovedlearningoutcomes,particularlyforgirls.Thequalitativecasestudiesprovidemorein-depthinsightintofactorsthatinfluenceeffectiveteachingandanimprovedschoolenvironmentresultinginimprovedlearning.Wefirstexaminepupil-andschool-levelcharacteristicsthatmayaffectpupils’learning.Next,wediscusskeybarrierstogirls’schoolattendanceandretention.
4.6.2.1 Pupilandschool-levelcharacteristics
Pupils’attitudesandaspirations
Pupil’saspirationsandmotivationarelikelytoaffecttheextenttowhichtheyattendschoolandputeffortintolearning.This,inturn,islikelytoaffecttheextenttowhicheffectiveteachingandanimproved,girl-friendlyschoolenvironmentdocontributetobetterlearningoutcomesforgirls.
Thereappearstobeadifferenceinhowboysandgirlsviewthevalueofeducationfortheirfuturelife.Inallschools,pupilsconsiderformalsubjectsimportantforbothboysandgirls.Still,inmanyIQScommunities,stakeholders(includingboys)perceivegirlsasbeingtoofocusedonmarriage,andtheybelievethatthisaffectstheirattendanceandinterestinschool.
‘Thechallengesaretheeagernesstomarry.Themomentagirlturns12allshecanthinkofismarriage,youmostlyfindhere.Thosegirlsyousaw169areallpreparingtogetmarried.Soourmainchallengehereisearlymarriage.’(TPD,BauchihigherperformingIQS)
Pupilsthemselvesrecognisethatinmanycasesgirlswillgetmarriedatanearlyageandthatthiswillinfluencewhatgirlscanchoosetodowiththeireducation.Somegirlsperceiveformaleducationasimportantinordertoteachtheirchildren,andtobeabletodobasiccalculationsandreadletters.Inmostcases,girlsdoaspiretobecomeprofessionallyactive,wishingtobecometeachersorbusinesswomen(sellingriceetc.).However,thisisnotaconsistentfindingacrosstheIQSs.IntheNigerlowerperformingIQS,manygirlswishtobemarriedwhenfinishingschool,whereasboysaspiretoworkinthegovernment.Incontrasttoperceptionsofgirlsasonlybeinginterestedinmarriage,somegirlsclearlyindicatefrustrationattheideaofleavingschoolearlierthanboys,inordertomarry.
ThemanagementandfacilitatorsinIQSsperceivegirls’interestinformalsubjectstohaveincreased.Inthepastgirlsoftenleftschoolwhentheformalsubjectsstarted.However,throughtheactivitiesofCBMCsandfacilitators(suchasencouraginggirlswithgiftsandbiscuits),girlsarenowmorelikelytoattendclasses.
Pupils’participationineducationbeyondIQS
PupilsoftenattendbothanIQSaswellasapublicprimaryschool.Therefore,thereislikelytobesomespillovereffectofinterventionsatlocalprimaryschoolsonoutcomesattheIQS.SeveralofthecasestudyIQSshavenearbypublicprimaryschools.SomegirlsattendboththeIQSandthepublicprimaryschool,attendingpublicprimaryschoolinthemorningandtheIQSintheafternoonorevening.Inthiscase,thequalityofteachingatthepublicprimaryschoolhasadirectbearingongirls’learningoutcomes.Therearealsolikelytobesomeindirectspillovereffectsforgirlswhodo169ReferstothegirlsparticipatingintheFGD.
EvaluationoftheGirl’sEducationProject3
©EDOREN 224
notattendpublicprimaryschools.Inparticular,someIQSfacilitatorsalsoteachatthepublicprimaryschool(asintheBauchihigherperformingIQS),orareinfluencedandpeer-assistedpedagogicallybyfriendswhoteachattheprimaryschool(aswiththeBauchitypicalIQS).ThiscreatesalikelihoodofsomespilloverfrominterventionsattheprimaryschooltooutcomesattheIQS.ItalsomeansthatIQSsthathaveapublicprimaryschoolnearbyarelikelytobeatanadvantagecomparedtothosethatdonot.
ThemainreasonforpupilsattendingonlytheIQSappearstobealackoffunds(seebelow),althoughparticipantsstatethatsomegirlsdonottoattendpublicprimaryschoolduetoparentsnot‘valuingeducationforgirls’.Incaseswhereparentsmaynotwishtosendgirlstothepublicprimaryschool(eitherforculturaloreconomicreasons),theIQSoffersanopportunityforgirlstobeexposedtoformaleducation.However,thereissomeriskthat,followingtheintroductionofsecularteachingattheIQS,parentsmayrefrainfromsendingtheirdaughterstopublicprimaryschoolsandsendthemtotheIQSonly,inordertosavemoney.
Thebroaderpointhereisthattheevidencepointstotwodiscreteissuesthatunderminegirls’accesstoschooling:thecostofschoolingandsomepersistingresistanceamongstparentstosecularschoolingforgirls.Thesetwoissuesinteracttounderminegirls’accesstoschooling.ItwillbeimportantforGEP3tounderstandtheinteractionbetweenthesetwoissues,andtoengagewithbothintandem.
InseveralcasespupilsalsoattendvocationaltrainingoutsidetheIQS/publicprimaryschool.InBauchitypicalIQS,participantsrefertoan‘empowermentcentre’forgirlsandwomen,wheretheylearnsewingandsoap-making.Thereareafewreferencestothiscentrealsoteachingliteracy.Thesetypesofalternativelearninginterventionsandsourcesofsupportmightaffectlearningoutcomespositively,bysupplementingchildren’sknowledge.
TypesofIQS
ThecasestudyIQSsseemtoresideinafixedlocation,althoughsomeuncertaintyremainsaboutthemobilityofoneIQS.TheIQSsvisitedstatedthattheywerenolongermobile.InNiger,allschoolswereestablishedIslamiyyasandwerefixedinonelocation.InBauchi,schoolswerecalledTsangayasbuttwoatleastseemedtobeoperatingasIslamiyyas.ThelowerperformingIQSstillhadcharacteristicsofaTsangaya,albeitbeingintransitiontowardsmoreIslamiyyafeatures.Inthatschoolatleast,thereweresomeaccountsbypupilsthattheboysmightbemoving–althoughitisunclearwhetherthisisthecase.Reasonsgivenfornotmovingincludedthesecuritysituationinthestate.Assuch,itisunclearwhethertheTsangayamaymoveagainifthesecuritysituationstabilised.
IfanIQSwastomovethiscouldhaveadirectimpactonlearningoutcomes.InthecaseofboysmovingwiththeMallamforseveralmonthsthismayresultinnotreceivinganyformaleducationduringthistime–unlesstheMallamwasalsoteachingformalsubjectsanddecidedtodosoduringthemobility.Forgirls,sincetheywouldnotbemoving,itwoulddependonwhethertheteachingofformalsubjectscontinuedintheabsenceoftheMallamandmalepupils.AsthenomadiccultureofcertainTsangayasmayhavesignificantimplicationsforlearningoutcomesandtheretentionof,inparticular,girls,itisimportanttoexplorethisfurtheratmidline.AspartoftheimplementationofGEP3goingforward,itmaybenecessarytoconsiderhowthisinformationcanbeverifiedandtrackedaspartofthemonitoringprocessattheLGAlevel.
EvaluationoftheGirl’sEducationProject3
©EDOREN 225
4.6.2.2 Barrierstoattendanceandretention(includinggendereddifferences)
Attitudestowardsgirls’education
AttendanceandretentionofgirlsinintegratedQur’aniceducationremainsachallenge.AcrosstheIQSsstakeholdersreferredto‘allchildrenattendingschool’170whilstsimultaneouslystatingthatsomeparentspulledchildren(inparticulargirls)outofschoolwhentheintegratedsubjectswereintroduced.Similarly,notallgirlsattendformalclasses:someleaveafterreligioussubjectshavefinished(Bauchi,higherperformingIQS).
AllIQSsvisitedfacechallengeswithregardtoretaininggirlsinschool,duetoearlymarriageorotherculturalconstraints(seebelow).Theyalsofacechallengeswithregardtotheattendanceofbothboysandgirlsbecauseofout-of-schoolresponsibilities(suchaschoresorwork).Otherchallengesaretherelativelackofunderstandingoftheimportanceofformalsubjects,andconcernsabouttheinfrastructureandfacilitiesatIQSs(discussedinSection4.4.1.2).Theformerpossiblyrelatestotheperceptionthatthesesubjectsarenotrelevanttothelivesofpeopleinthecommunities.Forboys,thisrelatestoadearthofjobopportunitiesthatrequireaformaleducation.Forgirls,thisrelatestothefactthatwomentypicallydonotenterformalworkoutsideofthehomeaftermarriageowingtoculturalpracticesofseclusionandtherealitiesoftheirreproductiveresponsibilities.
Therearealsopersistingbarriersrelatedspecificallytoattitudestowardsgirls’education.Alackofawarenessoftheimportanceofeducation,inparticularforgirls,waswidelycitedasareasonforcontinuedchallengesinregardtoattendance.CBMCsinallIQScasesspokeofthisissue:‘Youknowthisisavillage,peopledon’tvalueformaleducationforgirls.Theybelievesincethey(mothers)werenoteducated,whyshouldtheirchildren(girls)beeducated?’(CBMC,BauchitypicalIQS).Inaddition,whilstparticipantsconsiderededucatingthe‘girlchild’asimportant,theperceptionisthatitismoreimportantinmostcasestoeducatethe‘boychild’.
‘Wepreferthatthemanhasmoreknowledgethanthegirlbecausebyhernatureagirlissupposedtoacquireknowledgefromtheman.Everydayoftheman,evenreligionspokeonit,ifthewomanseeksoneknowledgethenthemanseekstwoknowledge.Themanshouldteachhiswifeoneandherknowledgeisincreased.Thenthemangoestoseekfurtherknowledgeandteachesthewoman.Thatispartofthelaw.Yousee,knowledgebestowsononeresponsibilities.Ifthewomanismorelearnedthantheman[is]thenthereisaproblem.Evenamongsiblings,thefemaleknowsmorethantheman,thereisproblem.’(Parents,BauchihigherperformingIQS)
Thereweresomedifferencesinattitudesbetweenruralandurbanareas.InBauchi,thehighertolowerperformingcasesfollowedtheperi-urbantoremoteruraldivideclosely.Inthelowerperforming(rural)case,anabsenceofwomenattheparents’FGDpossiblyindicatedmothers’lackofengagementwiththeIQSprocess.Thisinturncorrelateswithwomen’slowereducationalandliteracylevelsinruralcommunitiesoverall.
However,stakeholdersperceivedattitudestowardsgirls’educationtobechanging(seeSection4.4.1.2).Whilstpreviouslytherewouldbesignificantchallengesingettinggirlstoattendschool,acceptanceofintegrationhasbecomemorewidespreadinallcases.Moreover,stakeholdersagreedthatmorecould,andneededtobe,donetoincreaseawarenessoftheimportanceofeducation.Asparentshavenotyetexperiencedthepositiveconsequencesofallowingtheirchildrentoattend
170WhileKIsperceiveparticipationinIQSstobehigh,thismayattimesbeduetochildrenattendingreligiousclasses,anddoesnotnecessarilyindicateattendanceduringformaleducation.
EvaluationoftheGirl’sEducationProject3
©EDOREN 226
school,thereisstillawaytogo.CBMCssawthemselvesassettinganexamplebysendingtheirchildrentostudy,evenbeyondsecondarylevel,toencourageotherparentstodothesame.Overall,parentsandcommunitiesdoappeartobewillingtoreflectonandreconsidertheroles,responsibilitiesandcapabilitiesofgirls,andattitudestowardsgirls’educationappeartobeslowlychanging.
Capacityconstraints
EventhoughCBMCactivitiesandwidercontextualchangesappeartobepositivelyinfluencingattitudestowardseducation,includingeducatinggirls,changesinattitudesdonotnecessarilyleadtochangesinbehaviour.Thequalitativeresearchindicatesthatfinancialcapacityconstraintshaveacriticalbearingonattendanceandretention.
‘Wedidnothavetheopportunityofformaleducationproperlyandourknowledgeofreligionthewayitshouldbeimpacted.Wetakethesetwoasveryimportantbutthebarriersarethatwehaveplentychildrenandthereisnohelpfromanywhere.Forinstance,ifonehas10–15childrenintheschoolandeachchildweretopay#50,itwillbedifficultfortheparenttomeetup.Thesamethinggoesforbothprimaryandsecondary.Theresultisthatyouwillsendsometoschoolandleavesomebehindaccordingtoyourfinancialstrength.Atthesametime,wewantofallofthemtogo.Ihavesoninprivatesecondaryschoolandthebooksaresoexpensive.Thesearethekindofthingsthatcouldhindereducation.Wewantthegovernmenttostepinandhelp.’(Parents,BauchihigherperformingIQS)
Respondentscitepovertyasakeyreasonwhyparentsdonotsendtheirchildrentoschool(publicprimaryorIQS).Costsofeducationplayasignificantroleindeterminingwhetherparentssendtheirchildrentopublicprimaryschools.IQSsvisiteddonotchargeashigh(ifany)schoolfeesaspublicprimaryschoolsdo,andinmostcasesdonotrequireuniformseither.Schoolfeesareoftennon-compulsoryorareawayfortheCBMCtomobiliseresourcestofeedbackintotheschoolandtoofferteachersasymbolic‘tokenofgratitude’.However,eveninIQSsthathavelowfeesornofees,povertyremainsabarriertoaccessforsomechildren,owingtotheopportunitycostofschooling.Thislinkstotherolethatchildren(andparticularlygirls)playinsupportingthefamily’slivelihoodactivities.
SchoolmanagementinallsixIQSsbelievefeesdiscourageparentsfromsendingtheirchildrentoschool.Feesmayaffectboysandgirlsdifferently.
‘Becauseofpovertyaroundhereitcanbeaseriousissue.Itcandiscouragetheparentsandwewillendupwithlowattendance.IwillpreferbothboysandgirlstogotoschoolbutincasethereissuchasituationIthinktheboywillbesenttoschoolbecausetheboyisinapositiontoassistothers.’(Mallam,BauchilowerperformingIQS)
Parentsappeartobemorelikelytoprioritiseboysinregardtogoingtoschool,althoughthereweresomereferencestoparentspreferringtosendgirlstoschoolsincetheyaremorelikelytostaybehindandhelpthecommunityifeducated.Inordertoencourageparentstosendalltheirchildrentoschool,someIQSschargeparentsasetamountregardlessofhowmanychildrentheysend.Additionally,schoolfeescanaffectattendance,aschildrenfeelreluctanttocometoschoolduetoembarrassmentatnotbeingabletopaythefee(Parents,BauchitypicalIQS).
CBMCandschoolleadersinallsixcasestudyschoolsfeeltheycannotinsistonpayment,sinceitwillresultinparentssendingthechildrenaway.Thisincludesaskingparentstopayforschooluniformsorrequiringchildrentobringpens.Respondentsbelievethattheresourceconstraintsof
EvaluationoftheGirl’sEducationProject3
©EDOREN 227
thecommunitylimitthecapacityofschoolmanagementtoinvestinIQSs,whichinturnmayaffectthequalityofeducationprovidedandthecapacitytoretaingirls.
‘Wewouldwanttoaddmoresubjectsbutthechallengewehaveistheteachersthatteachthechildren,wecan’taffordtopaythem.Initiallythechildrenpaid10Nairaeveryonceinaweek.Wenoticedthatmostchildrendonotcometoschoolagainbecausetheirparentscouldnotaffordit.Sowestopped.’(CBMC,NigerlowerperformingIQS)
Theissueoffees,therefore,alongwiththewideropportunitycostofschoolingrelatedtochildwork,indicatesthatimprovedteachingandanimprovedschoolenvironmentwilllikelynotbeenoughtoaddressthesemoredeep-rootedchallenges.Evenifthequalityofeducationimproves(andaqualitysupplyisthereforeprovided),theinabilityofparentstoreconcilethepossiblecostimplicationscouldremainprohibitive,unlessthevalueofeducationfromaninstrumentalistperspective(whatitcantangiblyleadto)isalsoeffectivelycommunicated,withdemonstrableresultsthatcanoffsetthosecosts.171
Ontheothersideofthisargument,assumingthatretentionispartlyaffectedbyperceptionsaroundthevalueandqualityofeducation,itwillbeimportantforschoolstobeabletoteacheffectivelywhenthedemandforeducationincreasesincommunities.DuringQCOs,researchersobservedovercrowdedclassroomsinsomeIQSs,withsomepupilssatoutsidetheclassroom.CBMC’slimitedabilitytomobilisesufficientresourcescouldhaveanegativeimpactonthequalityofteaching,andthuspotentiallyretention.Yet,someCBMCsfeelthatparentsdonotprioritiseeducationwhenallocatingresourcesandthatasattitudeschange,sowillspending.
‘Fornow,itisbecausetheyhavenotreallyenjoyededucationwhycostcanaffectthem.Whentheybegintoseethebenefits,theywillratherspendmoneytheyuseinbuyingsemira172inpreparationfortheirmarriageonsendingthem[theirchildren]toschool.Becausetheplateswillbebrokeneventually,buttheknowledgewillalwaysbethereandwilldiewiththeowner.’(CBMC,BauchilowerperformingIQS)
Outsideschoolactivitiesandresponsibilities
Thequalitativeresearchexploredhowboysandgirlsspendtheirtimeoutsideofschool.IntheNigercases,theteamspoketoboysandgirlsestimatedtobebetweentheagesofsevenand13.InBauchiStatetheywerebetweentheagesofeightand15years.AcrossthesixcasestudyIQSs,boysandgirlswereallresponsibleforincome-generatingactivities.Childrenwouldengageinincome-generatingactivities,suchasfarmingorlookingaftercattle(boys),orhawkingandsellingproduceinthemarket(mainlygirls).Additionally,parentsconsiderchildrenhelpingwithresponsibilitiesinthehousehold(likelookingafteryoungersiblingsorfetchingwater)tobeanecessarypartofincome-generatingactivities.Girlsgenerallyhadmoreresponsibilitiesforhouseholdchores.
Nocleardifferencesemergedfromthequalitativedatainregardtothetimingofpupils’dailyroutines.PupilsinallsixIQSsreferredtowakingupandeitherstudyingArabicorcompletingchores,
171Convincingparentsthateducationoffersavaluethatfitswiththeirneedsisstillonlyonepartoftheprocess.Iftheyareunabletopracticallyapplylearninginamannerthatleadstothoseincurredcostsbeingmitigatedorovercome,oriftheopportunitiesarenotthereinthefirstplace(likejobs),thevaluewillnotbeseen.Forexample,ifmothersaretoldthatkeepingtheirdaughtersinschool(andawayfromhawking)tolearnnumeracyandliteracywillmakearealdifferencetotheirdaughtersbeingabletoruntheirownbusinessesmoreprofitably,theywouldneedtoseethatdemonstratedquitequicklyinordertobeconvinced,otherwisetheywillperceivethattheyarejustincurringlossesbyremovingthemfromtheirincome-generatingroles.172Traditionalplates.
EvaluationoftheGirl’sEducationProject3
©EDOREN 228
afterwhichtheywouldattendeitherpublicprimaryschoolorwork.Intheafternoonandevening,childrenreportedeitherworkingorattendingtheIQS.Thereafter,pupilsdohomeworkinsomeIQSs(NigerhigherperformingIQS,NigertypicalIQS)andvariouschores.Homeworkappearstobedominatedbyreligiousstudies.ChildreninallIQSsreportedhavingsometimetoplay,althoughnotasmuchastheywouldwant,whereasparentsconsiderchildrenplayingtoomuchtohaveanegativeeffectonattendanceandpupil’smotivationtolearn.
Out-of-schoolresponsibilitiesaffectchildren’sschoolattendance.Schoolstakeholdersattributethistotheperceivedlackofcapacityofparentstoprioritisechildren’seducation,whichtheyassociatewithalackofunderstandingofthevalueofeducation.173Parents,however,seeitasafinancialandpracticalnecessity.
‘Thereismajorchallengeforthegirlsbecausemostoftheirparentsmakethingsforsale,likebeanscake,soyabeancake,riceandsoon,andtheygivethegirlstogosell.Thisaffectsthechildren’sattendancebecauseeithertheycometoschoolverylate,almostintheclosinghours,ortheymissschool.’(TPD,BauchihigherperformingIQS)
InthreeoftheIQSsCBMCshaveactivelyinstigatedthatpupilshavetodropallchoresandworktheminuteitistimeforschool(NigertypicalIQS,NigerhigherperformingIQS,andBauchihigherperformingIQS).Thismeansthatparentsarehesitantaboutextendingtheinstructionaltime,since‘theyneedustobehomefordomesticjob’(Boys,BauchihigherperformingIQS).Accordingtoonestakeholder,possiblesolutionsaretoprovideparentswithfarmimplements,suchasploughsorfertilisers,toaidparentswhilsttheirchildrenareinschool(Mallam,NigerlowerperformingIQS).Aslongasparentsrequirechildrentoassistinincome-generatingactivities,directlyorindirectly,thisislikelytohaveanegativeeffectonattendanceandenrolment.Asparents’attitudestowardsformaleducationbecomemorefavourable,itwillbecomemoreimportanttoaddressbarriersrelatedtotheopportunitycostofschooling.
Respondentsperceivethatmothersarelesspositiveaboutgirls’educationthanfathers.Participantsstatethatwomenattimeshindergirlsfromattendingschool.Thismighthavebeenthoughttobeacaseofisolatedexperienceifitwasonlyreferencedonce;however,fiveoutofsixIQSsmentionedthisoccurring.
‘Therearechallenges.Youknowyouwomenareclosertothegirlchild.Theysaythatknowledgeforachildliesatthefootofthemother.Wemenarehardlyathome.Yousendyourdaughtertoschoolandgoouttofindfoodforthefamily.Thenthemotherwillnottalkherintodoingotherthingsinsteadofgoingtoschool.Wearefacingsuchchallenges.’(Parents,BauchihigherperformingIQS)
Pupilssharetheviewof(mainlymale)stakeholdersthatmothersoftenrequiregirlstostayathome.Theexplanationgivenwasthatwithgirlshelpingthemothersandboyshelpingthefathers,girlsattendingschoolmeansthemothercannotgenerateasmuchincomeasneeded.Itappearsthatthemotherisakeyproviderinsustainingtheday-to-daylifeofthehousehold.Moreover,sincemanymothersthemselveshavenotattendedschool,itappearsthatthereislessunderstandingamongstmothersinsomeIQSsofwhygirlswouldbenefitfromformaleducation.
CBMCsinallsixschoolsconsidersensitisingwomenaskeyinordertoincreasetheenrolmentandattendanceofgirls.However,asdiscussed,thisconstraintisunlikelytoberemovedthroughsensitisationalone,aswomenrequiregirlstoengageinincome-generatingactivities(suchas
173Thesewereperceptionsdrawnfromacombinationofteachers’TPDs,andMallamandLGEAofficerKIIs.
EvaluationoftheGirl’sEducationProject3
©EDOREN 229
hawking)ortotakecareofhouseholdchoressoamothercan,forexample,gotothefarm.InNigerlowerperformingIQS,forexample,womenrarelyleftthehouse.Girlswereresponsibleforactuallygoingoutandsellingwhatthewomenproduced,and,assuch,mothersaredirectlydependentongirlstobringinanymoney.
ThereasonsforthisareheavilyrootedintheroleofwomenaseconomicagentswithinnorthernNigeriancommunities,andaskeyincomegeneratorsforthefamily.Therunningofhome-basedbusinessesbywomeninnorthernNigeriaisatraditionalpartofthegenderedeconomiccontext,andtheuseofchildrenandyouthtopurchaseinputsandtomarketproductshasalwaysbeenacriticalpartoftheenterpriseinfrastructure.174Unmarrieddaughtersareattheheartofthissystem–moresothanboys,whowillhaveotherdutiesonthefarminruralareas.Thepracticeofaurenkuli,orseclusion,bymanyMuslimwomenwhenmarried,exacerbatesthisrelianceonunmarrieddaughters,aswomenmustmanagetheirentirebusinessinfrastructureremotely.175
ThegenderedpoliticaleconomyofthisrealityisonethattheToCofGEP3’sIQSSdoesnotaddresswhenitcomestoensuringthatgirlsareabletonotonlyaccesstheIQS,butalsotoremainfullyengagedconsistentlyandlongenoughforlearningtooccur.Successfullyaddressingthischallengewillrequireinterventionsrelatedtowomen’spovertyalleviationandalternativeincome-generatingopportunities.Assuch,itwillneedtogobeyondactivitieswithintheeducationsector.
TheextenttowhichCBMCadvocacyandimprovedschoolmanagementalone,andwithoutamoreholisticapproach,canraisegirls’participationinschoolingisquestionable.WithCBMCmembersalsobeingapartofthecommunity’seconomicreality,awillingnesstotakeonthechallengetogirlseducationthatagenderedpoliticaleconomyposes,maybelimitedwithoutfurthersupportinthisarea,despitethepresenceofgoodintentions.Atmidlineitwillbeimportanttointerrogatethisareafurther,tounderstandhowtoreconcilethetensionbetweengirls’successfulretentionandcompletionofschoolingandwomen’sincomegenerationandpovertyalleviationactivitiesinnorthernNigeria.Thisisalsoacyclicalissue,asmanyofthegirlsinquestionwillalsobetransitioningtowomanhood(andneweconomicroles)duetotheanotherkeyfactorthatinfluencesgirls’accesstoschooling–earlymarriage.
Earlymarriage
Oneofthemainreasonsstatedforlowenrolmentandretentionofgirlsisearlymarriage.ThislinkstothefactthatIQSsarelargelymulti-gradelearningenvironments,withclassesoftencateringtoawiderangeofages.IntheBauchicontext,whereintegrationhasbeenamorerecentprocess,thepresenceofolderchildrenreflectsthenumbersofchildrenwhowereout-of-school(formalschooling)untiltheIQSmadesecularlearningavailable.
Whilsttheacquisitionofformaleducation,includingforgirls,maybegainingacceptanceinthesixIQScases,earlymarriageisstillprevalent.Thisisparticularlyproblematicforgirlsovertheageof10whoareacquiringabasiceducationforthefirsttime.CommunitiesandLGEAofficersconsistentlyrefertothisasthekeychallengeforgirls’educationandlearningoutcomes.Yet,withcommunitiesconsideringitaspartoftheirIslamicfaith,theviewisthatitisalmost‘inevitable’.
174Tayloretal.(2014)‘EconomicOpportunitiesandObstaclesforWomenandGirlsinNorthernNigeria’,DFID.175 Zakaria,Y(2001)‘EntrepreneursatHome:SecludedMuslimWomenandHiddenEconomicActivitiesinNorthernNigeria’,NordicJournalofAfricanStudies,10(1):107-123(2001).
EvaluationoftheGirl’sEducationProject3
©EDOREN 230
Thereisnosetageforagirltogetmarried,ratherthisisdecidedbyreferenceto‘maturity’.176Girlsmaythusreceiveanynumberofyearsofschoolingbeforetheymarry,atagesrangingfrom12to20.Themostcommonlyreferredtoageformarriageis15.Dependingonthe‘ageofmaturity’,somegirlsintheGEP3communitiesmaynothavereceivedanyeducationbythetimecommunitiesdeemthemreadyformarriage:‘Shewasalreadygrownevenbeforetheschoolstarted’(Girls,BauchilowerperformingIQS).Ageofmaturityreferstothegirl’sphysicaldevelopment.IntheBauchilowerperformingIQS,theMallam/headteacherreferredtothefactthatwhenagirlhadphysicallydeveloped,itbecameimportantforhertobemarried.However,agirlwhowasstillundevelopedbeyond15couldcontinueherschooling.
Asmanygirls,especiallyinBauchi,appeartobearound12andarestillinP3,itisplausiblethatearlymarriagewillaffectretention,evenatthestageofprimaryeducation.Withsomecommunities,andinparticulargirls,nothavinghadaccesstoformaleducationpriortointegrationtheremaybegirlsstartingeducationatanolderage.ThequestionofhowthisrelatestoGEP3willneedfurtherexplorationatmidline.
Thereareoftenprovisionsmadeforgirlstocontinueeducationaftermarriage.However,thisisdependentonthewillingnessofthehusbandtoallowhiswifetoattendschool.Participantsstatethatpartiesincreasinglyagreeuponthisbeforemarriage,butsometimesthehusbandchangeshismindaftermarriage.Thoughparentsmaysaythattheirattitudestowardsgirls’educationarepositive,andthattheywantthemto‘gofar’andstudytoahighlevel,thereisanunderstandingthatonceagirlismarriedsheisnolongertheparents’responsibility.Asaresult,changesinparentalattitudeswillnotnecessarilytranslateintoimprovedretentiononcegirlsarebeyondtheageofmarriage.
‘AccordingtothereligionofIslam,agirlchildshouldmarryearly,butyouaskherhusbandtosponsorhercontinuededucation.Ihaveadaughterintheuniversity.Shegotmarriedlastyearandsheisstillcontinuingwithhereducation.Wewillnotdiscouragethegirlsiftheydesiretogotoschool,butwhentheygettotheirhusband’shousetheymaycontinueandtheresponsibilityisnolongeryours.’(Parents,BauchihigherperformingIQS)
Thus,thoughanattitudechangeonthepartofparentsisimportantintheyearsbeforemarriage(andinnegotiatingamarriage),changingtheattitudesofthemenwhoarelikelytomarrygirlsinthecommunityislikelytobeequallyessential.ThoughGEP3indirectlydoesthisthroughcurrentpupils,andparentsofboys,theremaybeaneedtoexplorefurthertheextenttowhichboys/mennotattachedorconnectedtotheIQSareinfluencedbytheintervention,andwhetherthiswillimpactongirls’continuededucationaftermarriage.
ThreeofthesixcasestudyIQSsreferredtoprovidingsomeformofeducationforwomenthatincludedformalsubjects.Integrationinitiativesthusstretchbeyondthedirecttargetgroupsofgirlstoalsoincludeincreasedawarenessoftheimportanceofformaleducationforwomen.However,inmostcasesitappearsthatwomenonlycontinuewithreligiousstudies.Whetherawomancontinuesformalschoolaftermarriageappearsdependentontheattitudesofherhusband-to-be.
‘Someofthemwhoearnsalariesandwhoalsoknowthattheirwivesareresponsibleallowthemtocontinuewiththeireducation.Butsomewillevenpromisethegirlbeforemarriagethatoncetheygetmarried,shewillcontinuewithhereducationonlytogobackonthepromiseoncethegirlbecomeshiswife.Somanymarriageshavebrokenupbecauseofthis.’(Girls,BauchitypicalIQS)
176Thisisareferencetoagirl’sbiologicalandphysicalmaturity,indicatingthepointatwhichherbodystartstodevelop.
EvaluationoftheGirl’sEducationProject3
©EDOREN 231
Girlscontinuingtoaccessschoolaftermarriagewillalsobedependentontheflexibilityoftheirnewhome-basedresponsibilitiesaswivesandmothers,andtherestrictionsimposedwherethepracticeofseclusionapplies.TheimportanceattachedtoreligiouslearningsuggeststhatgirlsandyoungwomenareencouragedtocontinuetheirQuraniceducationaftermarriage,andtheirmobilityforthisreasonisnotconstrained.Thismayofferawayformarriedwomentoalsocontinuetheirsecularlearning.
ThoughchangingnormsandpracticesaroundearlymarriageisnotadirectobjectiveofGEP3,thequalitativeresearchindicatesthatsuchchangeswillberequiredinordertofacilitatecontinuededucationofgirls.Whilesomestakeholdersseemtobeawareofthechallengeofearlymarriageforgirls’education,thisdoesnotnecessarilychangeattitudesorpractices.Forexample,theMallaminNigerlowerperformingIQSsaidheconsidersearlymarriageamainchallenge,andthathewouldliketoseegirlsstudytothelevelofuniversity.Atthesametime,itappearshehadmarriedhisowndaughtersatyoungages.AsimilarcontradictioncamefromtheBauchilowerperformingcase.FGDswithboysfurthermirrorthis.Someboysarehappytolettheirfuturewivescontinueeducationsince‘Iwantaneducatedwife’(Niger,typicalIQS),whilstothersconsideritsomethingthatisnotdone:‘Iwillnotallowmywife,weinthevillagewedon’tallowourwivestogotoschoolbecauseofsuspicionofinfidelity.’(Bauchi,lowerperformingIQS)
Additionally,changingnormsandpracticesintheIQScontextishighlylimitedbypressuresbothfromwithinthedirectcommunityandpotentiallyfromawiderreligiouscommunity.Parentsfacepressurestomarrytheirdaughterssince‘shewillsoonbecomeworthlessifsheisnotmarriedoff’(Girls,BauchitypicalIQS)
‘Somepeoplewillevenkeepmalicewithyourfatherifherefusestolistentothem.Insomecases,itiseventhegrandparentswhowagewaragainsttheirsonstomarrytheirdaughtersoffearlyandthenthefatherswillsuccumbtopressureandaskthedaughtertobringhomeamanshewantstomarryorahusbandwillbechosenforher.Thatisonemajorchallengewegirlsfacehere.Wearenotallowedtogotoschoolforaslongaswewant.’(Girls,BauchitypicalIQS)
Aslongasgirlsaredependentonmarryingforsocialsecurityandtomakealivingthisislikelytoimpactontheabilityagirlhastostudyfurther.Itisalsolikelytoaffectgirls’aspirationsinschoolsincetherewillbelimitedoptionsavailableforgirlstopursue.InNigerhigherperformingIQStheproprietorisfemaleandactivelystressedtheimportanceofformallyeducatinggirlsevenbeforetheGEP3integration:
‘Nowwehaverealisedthateducationisveryimportant.Beforewemarriedoutourdaughtersatatenderage,butnowwehaveseenthelight.Godwilling,weshallcontinuetoensurethatourchildrengeteducated.Itwasourfaulttohaveallowedourchildrentoplayinthepast,butnowthatwedonotallowourgirlstopayattentiontoboys,theyarenolongerinterestedinmarriage.Allchildrenareveryinterestedineducation.’(Parents,NigerhigherperformingIQS)
EvaluationoftheGirl’sEducationProject3
©EDOREN 232
Bibliography
Adediwura,A.andTayo,B.(2007).‘Perceptionofteachers’knowledgeattitudeandteachingskillsaspredictorofacademicperformanceinNigeriansecondaryschools’.EducationalResearchandReview,2,pp.165–171.
Anderson,M.andScamporlino,R.(2013)‘TheMasterofTeachingattheUniversityofMelbourne:aclinicalmodelforpre-serviceteacherpreparation’.InternationalSchoolsJournal,32(2),pp.33–42.
Arnold,C.(2011)CashTransfers,LiteratureReview.UKDepartmentforInternationalDevelopment.
Baird,S.,McIntosh,C.andŐzler,B.(2011)‘CashorCondition?EvidencefromaCashTransferExperiment’.TheQuarterlyJournalofEconomics,126(4),pp.1709–1753.
Benhassine,N.,Devoto,F.,Duflo,E.,Dupas,P.andPouliquen,V.(2014)‘TurningaShoveintoaNudge?A“LabeledCashTransfer”forEducation’.NBERWorkingPaper,No.19227.
Burns,D.,andWorsley,S.(2015)NavigatingComplexityinInternationalDevelopment.Rugby,UK:PracticalActionPublishing.
Cert,Y.(2010)‘TeacherEfficacyScale:TheStudyofValidityandReliabilityandPreserviceClassroomTeachers’Self’.JournalofTheoryandPracticeinEducation,6(1),pp.68–85.
CESE.(2015)Whatworksbest:evidence-basedpracticedtohelpimproveNSWstudentperformance.CentreforEducationalStatisticsandEvaluation(CESE).Availableathttp://www.cese.nsw.gov.au/images/stories/PDF/what_works_best.pdf,Accessedon5June2015.
Cogill,J.(2008)‘Primaryteachers’interactivewhiteboardpracticeacrossoneyear:changesinpedagogyandinfluencingfactors’.EdDthesisKing’sCollegeUniversityofLondon.Availableatwww.juliecogill.com.
Coladarci,T.(1992)‘Teachers’SenseofEfficacyandCommitmenttoTeaching’.TheJournalofExperimentalEducation,60(4),pp.323–337.
De,S.,Pettersson,G.,Morris,R.andCameron,S.(2015)TeacherDevelopmentProgramme(TDP),ImpactEvaluationofOutput1:In-ServiceTraining,FinalBaselineTechnicalReport,VolumeIResultsandDiscussion.Oxford,UK:EducationData,Research&EvaluationinNigeria(EDOREN)
Delahais,T.andToulemonde,J.(2012)‘Applyingcontributionanalysis:Lessonsfromfiveyearsofpractice’.Evaluation,18(3),pp.281–293.
DFID(2011)DFIDEthicsPrinciplesforresearchandevaluation.UKDepartmentforInternationalDevelopment.
Dundar,H.,Béteille,T.,Riboud,M.andDeolalikar,A.(2014)StudentLearninginSouthAsia:Challenges,Opportunities,andPolicyPriorities.DirectionsinDevelopment.Washington,DC:WorldBank.
EDOREN(2015)EvaluationFrameworkandPlan,UNICEF’sGirls’EducationProjectPhase3(GEP3).Oxford,UK:EducationData,Research&EvaluationinNigeria(EDOREN).
Getzels,J.W.andGuba,E.G.(1957)‘Socialbehaviourandtheadministrativeprocess’.SchoolReview,65,pp.423–441.
Glover,J.A.andRobert,H.B.(1990)Educationalpsychology:principlesandapplications.HarperColonsPublisher.
EvaluationoftheGirl’sEducationProject3
©EDOREN 233
Graham,A.,Powell,M.,Taylor,N.,Anderson,D.andFitzgerald,R.(2013)EthicalResearchInvolvingChildren.Florence:UNICEFOfficeofResearch–Innocenti.
Griffin,P.(2014)AssessmentforTeaching.CambridgeUniversityPress.
Guajardo,J.(2011)TeacherMotivation:TheoreticalFramework,SituationAnalysisofSavetheChildrenCountryOffices,andRecommendedStrategies.SavetheChildren.Availableathttp://www.oxfamnovib.nl/Redactie/Downloads/English/SPEF/281-24%20Teacher%20Motivation%20Report.pdf.
Henson,R.(2001)TeacherEfficacyResearchKeynoteAddress–TeacherSelf-Efficacy:SubstantiveImplicationsandMeasurementDilemmas.UniversityofNorthTexas.Availableathttp://www.uky.edu/~eushe2/Pajares/EREkeynote.PDF.
Humphreys,S.andCrawfurd,L.(2014)ReviewoftheLiteratureonBasicEducationinNigeria:IssuesofAccess,Quality,OutcomesandEquity.Oxford,UK:EducationData,Research&EvaluationinNigeria(EDOREN).
Jones,H.,Jones,N.,Shaxson,L.andWalker,D.(2013)Knowledge,policyandpowerininternationaldevelopment:apracticalframeworkforimprovingpolicy.ODIBackgroundNote.
Johnson,D.andHsieh,P.J.(2014)Thecontextofschooling,professionalcompetenciesandlearningoutcomesinnorthernNigeria.DraftSummaryReports.UNICEFGEP3.
Kauffman,D.(2011)‘ValidationoftheMotivationtoTeachScale’.H.U.JournalofEducation,40,pp.279–290.
Kelleher,Fetal.(2011)WomenandtheTeachingProfession:ExploringtheFeminisationDebate.UNESCOandtheCommonwealthSecretariat.
Leach,J.(2002)‘TheCurriculumKnowledgeofTeachers:anAnalyticReviewofaLargeScale,NationalElectronicConferenceEnvironment’.CurriculumJournal,3(1),pp.87–120.
Lynch,KandGood,H.(2011)‘Girlfriendly/childfriendly’schoolsprovideabrighterfutureinMali.UNICEF:Mali.Availableathttp://www.unicef.org/mdg/mali_59595.html.
Masters,G.N.(2013)Reformingeducationalassessment:Imperatives,principlesandchallenges.AustralianCouncilforEducationalResearch(ACER).Availableathttp://research.acer.edu.au/cgi/viewcontent.cgi?article=1021&context=aer,Accessedon21July2015.
Mayne,J.(2012)‘Contributionanalysis:Comingofage?’Evaluation,18(3),pp.270–280.
Mays,N.andPope,C.(1995)‘QualitativeResearch:RigourandQualitativeResearch’.BMJ.311(6997),pp.109–112.
McGrane(2014)UnpublishedreportonESSPINitemfunctioning.OPM,Oxford.
Merriam,S.B.(1988).Casestudyresearchineducation:Aqualitativeapproach.SanFrancisco:JosseyBass.
OECD.(2014)EducationataGlance2014:OECDIndicators.OECDPublishing.
OECDDevelopmentAssistanceCommittee(2010)DACGuidelinesandReferenceSeries:QualityStandardsforDevelopmentEvaluation.OECD.
Patton,M.Q.(2015)QualitativeResearchandEvaluationMethods.SagePublications.
Pedhazur,E.J.andSchmelkin,L.P.(1991).Measurement,DesignandAnalysis:AnIntegratedApproach.Hillsdale,NewJersey,LawrenceErlbaumAssociates.
RANA(2016)RANAQuarterlyReportSeptember–December2015.
EvaluationoftheGirl’sEducationProject3
©EDOREN 234
SparcandSavi(2015)StateoftheStates–PoliticalEconomy,ProspectsforReformandFiscalConditions(UnpublishedPowerPointPresentation,13July2015).
Suhr,DandShay,M.(2009)GuidelinesforReliability,ConfirmatoryandExploratoryFactorAnalysis.UniversityofNorthernColorado.Availableathttp://www.wuss.org/proceedings09/09WUSSProceedings/papers/anl/ANL-SuhrShay.pdf.
Tayloretal.(2014)EconomicOpportunitiesandObstaclesforWomenandGirlsinNorthernNigeria.DFID.
Tschannen-Moran,M.andWoolfolk-Hoy,A.(2001)‘Teachers’SenseofEfficacyScale’inTeacherefficacy:Capturinganelusiveconstruct.TeachingandTeacherEducation,17,783-805.Availableathttp://u.osu.edu/hoy.17/files/2014/09/TSES-+-scoring-zted8m.pdf.
UNICEF(2015a)UnpublishedStrategyPaper.GEP3TheoryofChange.GEP3.
UNICEF(2015b)UnpublishedStrategyPaper.Girls’EducationProject3,StrategyforIntegratedQur’anicSchools.GEP3.
UNICEF(2015c)UnpublishedStrategyPaper.Girls’EducationProject3,StrategyforEducationManagementInformationSystem(EMIS)CapacityBuilding.GEP3.
UNICEF(2015d)UnpublishedStrategyPaper.Girls’EducationProject3,GEP3ValueforMoneyStrategy2014.GEP3.
UNICEF(2015e)Girls’EducationprojectPhase3,OperationalPlan.RevisedJanuary2015.
UNICEF(2015f)UnpublishedStrategyPaper.Girls’EducationProject3,StrategyPaperonEarlyLearning(literacy&numeracy)RANA.GEP3.
UNICEF(2015g)UnpublishedStrategyPaper.Girls’EducationProject3,StrategyforBuildingtheCapacityofSchool/CentreBasedManagementCommittees.GEP3.
USAID(2014)NigeriaReadingandAccessResearchActivity:Resultsofthe2014HausaandEnglishEarlyGradeReadingAssessments(EGRAs)inGovernmentPrimarySchoolsandIQTECentersinJigawa,Kaduna,KanoandKatsina.RTIInternational.
VoicesforChange(2015)BeingaManinNigeria:PerceptionsandRealities.UKAid.
Yin,R.(1984)Casestudyresearch:Designandmethods.BeverlyHills,CA:SagePublishing.
YuandWatkins(2011)Attitudesandmotivationinsecondlanguageacquisition:AstudyofinternationalstudentsinChinafromaCulturalPerspective.UniversityofHongKong
Zakaria,Y.(2001)EntrepreneursatHome:SecludedMuslimWomenandHiddenEconomicActivitiesinNorthernNigeria.NordicJournalofAfricanStudies.10(1):107–123.
EvaluationoftheGirl’sEducationProject3
©EDOREN 235
AnnexA GEP3TheoryofChangeDiagram
Source:UNICEF(2015a)
EvaluationoftheGirl’sEducationProject3
©EDOREN 236
AnnexB BaselineTheoryofChangeassessment
B.1 Focusofkeyinformantinterviewsperstate
Katsina Zamfara Sokoto Niger Bauchi
1. Cashtransfers � � �
2. Enrolmentdrives � � �
3. GirlsforGirls �
4. Earlylearning(EL)intervention � �
5. IQSsupport(TT,HTT,CBMC) � � �
6. SbTD(TTor/andHTT) � � �
7. SBMCsupport � �
8. EMIS � �
9. HilWA � �
10. GESC/advocacy � � � � �
Rationalforfocus‒ ELpilotstate‒ CAPPNGOactiveingirls
spaces
‒ ELpilotstate‒ StrongEMIStrackrecord‒ StakeholdersforEnrolmDr,
SBMCandHiLWAaresimilar
‒ CTpilotstate‒ IQSsupportstate‒ StrongEMIStrackrecord‒ StakeholdersforEnrolmDr,
andSBMCaresimilar
‒ CTpilotstate‒ IQSsupportstate‒ Poor13/14EMIStrackrecord‒ StakeholdersIQSfacilitator
trainingandHTtrainingaresimilar
‒ IQSsupportstate‒ Teachertrainingplanned‒ ActiveHiLWA
EvaluationoftheGirl’sEducationProject3
©EDOREN 237
B.2 OverviewofkeyinformantsinterviewedforthebaselineTheoryofChange
assessment
National
PersonalinterviewsatnationallevelinAbujatookplacebetween3-8August2015.
Organisation Person Position
FederalMinistryofEducation
1. AgwaomaJoyceEke HeadofGenderUnit
2. H.T.Abdu MemberofGenderUnit
3. N.T.Kpalobi MemberofGenderUnit
4. E.E.Nnorom MemberofGenderUnit
5. O.P.Oshiba MemberofGenderUnit
WorldBank 6. OlatundeAdetoyeseAdekola Educationspecialist
UniversityofGuelph 7. OlabanjiAkinola PdDStudent,researcher
TDP 8. LilianBreakell InterimDeputyProgrammeManager
ESSPIN 9. JohnKay Leadspecialist–educationquality
RTI/RARA 10. R.DrakeWarrick ChiefofParty
SPARC 11. MarkWalker Nationalprogrammemanager
Katsina
PersonalinterviewsinKatsinatookplaceon10-11August2015.TheinterviewwiththeSPARCstateprogrammemanagerwasbyphoned.d.16September2015.
Organisation Person Position
StateMinistryofEducation
1. SagirIbrahimMuhammad PermanentSecretary
2. KabirLawalRuma DPRSSMoE
3. BintaAbdulmuminu Deskofficer
4. HalimaLawal DirectorofSchools
SUBEB5. AbuKasimJibia SecretarySUBEB
6. AliyuUmarImam DPRS
7. LawalMuhammedK/Seruri GEP3Deskofficer
SAME8. AhmedAhmedDoki ActingExecutiveDirector
9. AdamaSuleBakor DirectorWomenEducationFocalperson
CCTPIU,OfficeofSpecialAdviserChildDevelopment
10. MaharazuAbubakarSadiq PIUCoordinator
11. ShehuSadiq MISOfficer
CollegeofEducationDutsinama12. KabirMagaji GEP3FocalPerson
13. AdoIsmaelFuntua HeadofdepartmentEarlyChildEducation
CommunityActionforPopulationParticipation(CAPP) 14. FatimaYusufSaddiqh ProgrammeOfficer
SPARC 15. MohammedOlaniyi Stateprogrammemanager
TDPKatsinastateprogramme 16. HussainaHassan Stateteamleader
EvaluationoftheGirl’sEducationProject3
©EDOREN 238
17. MarkToscanini Deputystateteamleader
GEP3stateteam18. MaduguIbrahimGobir StateProjectCoordinator
19. SimonOsilamaIzuagie M&EConsultant
20. ShehuSuleimanAbdullahi Output2lead
UNICEFFieldOfficeKatsina 21. ShehuMohammed EducationOfficer
Zamfara
PersonalinterviewsinGusau,Zamfara,tookplaceon13-14August2015.TheinterviewwiththeSPARCstateprogrammemanagerwasbyphoned.d.16September2015.
Organisation Person Position
StateMinistryofEducation
1. SaniMande DirectorofHigherEducation
2. IbrahimM.Mailalle DPRS
3. MaheBello EMISOfficerSMoE
4. AbdurahmanAbdurazuk EMISOfficerSMoE
SUBEB
5. UsmanAbdullahi DirectorofSpecialProgrammes
6. AminuMusaKanoma DirectorofSocialMobilization
7. MaryamYahayaShantali DPRS
8. KasimuMu’azu EMISOfficerSUBEB
9. IbrahimMuhammadGusau EMISOfficerSUBEB
CollegeofEducationMaru 10. Dr.LawalSaadu FocalpersonGEP3,Lecturer
SBMC 11. AbubakarMohammadDogo SBMCStatechairman
HiLWA 12. MaryamIbrahim MemberofStateHilWA
FOMWAN 13. HalimaMohammedAmira(Leader)BunguduLGA,PermSec.MinistryofTourismZamfara
SPARC 14. HadjaraShibkau Stateprogrammemanager
CSACEFA 15. ShehuAdamuDinawa StatecoordinatorCSACEFA
GEP3stateteam16. Dr.IbrahimYabo StateProjectCoordinator
17. Dr.SalawSahid M&Eofficer
18. ZaraS.Abubakar Output2lead
EvaluationoftheGirl’sEducationProject3
©EDOREN 239
Sokoto
PersonalinterviewsinSokototookplaceon17-18August2015.
Organisation Person Position
StateMinistryofEducation
1. Alh.(DR.)M.B.Abubakar PermanentSecretary
2. MuhammadAttahiruAhmad DPRS
3. MamudaGaladima OperationsofficerCTP
4. MuhammadShamsuddeenSambo
EMISofficerCTP
5. AbdulkadirMalami Planning,M&EofficerCTP
SUBEB
6. IbrahimHaliruDingyadi Secretary
7. UmaruA.Boyi IQTEdeskofficer
8. JunaiduUmarJabo DirectorSocialMobilization
SAME9. AhmedBelloGidadawa Permanentsecretary
10. AbubakarMuhammadAlkammu
ProgrammeDirector
SBMC 11. AminuMusaYabo Statesecretary
CSACEFA 12. KabeerAliyu DeputyNationalModerator
CouncilofUlama 13. HamzaAlhajiSuleiman Stateassistantsecretary
UNICEFFieldOfficerSokoto14. TukurLabbo-Yabo Educationofficer
15. SafiyaTahirAbdullahi Educationspecialist
GEP3stateteam16. YahayaAliyuMaiyama Stateprojectcoordinator
17. HadizaI.Ibrahim Output1lead
Niger
PersonalinterviewsinMinna,Niger,tookplaceon20and24August2015.TheinterviewwiththeSPARCstateprogrammemanagerwasbyphoned.d.21September2015.
Organisation Person Position
StateMinistryofEducation1. HajiyaFatimaAbdullahi ActingPermanentsecretary
2. GarbaA.Yahaya DPRS
3. GimbaShuaibu Chiefstatistician
SUBEB
4. AyubaUsmanKatajo Permanentsecretary
5. AlhajiMahmoudSaniBabanna
DirectorofSchoolServices
6. ChadoIsahDokogi IQTEdeskofficer
PIUCashtransferprogramme/EducationResourceCentre
7. SuleAgboola Coordinator
8. RamatuK.Umar PlanningofficerSMoE
9. AminatAbdullahi MISofficerSMoE
10. AdamuIdrisArzika TrainingofficerSUBEB
11. SaniMuhammedKusharki OperationsofficerSUBEB
EvaluationoftheGirl’sEducationProject3
©EDOREN 240
12. MohammedShuaibuKagara AccountantCTPSUBEB
SAME13. YunanaGagare Executivedirector
14. UmarBerryWalo Chiefaccountant
15. AbubakarAdamu GEP3focalperson
CollegeofEducationMinna
16. Prof.FarukR.Haruma Provost
17. Dr.RuthGaladimaDeputyprovostAcademics&coordinatorGirlChildEducation
18. Dr.MohammedIsahTsado Deputyprovost
19. IsahAbdullahiAloma GEP3deskofficer
AmeenLiteracyFoundation 20. HajiyaAminaKereAhmed Founder
LifeRehabFoundation
21. FatimaBintaFarouk Coordinator
22. Dr.Mrs.LadiShambo
23. YhuzaA.Abdullahi
24. EmmanuelAmuta
25. KabiruAhmed
SPARC 26. GraceOkuchuwi Stateprogrammemanager
UNICEFFieldOfficeKaduna 27. ElizabethObaba Educationofficer
GEP3stateteam
28. IbrahimU.Nagwamatse Stateprojectcoordinator
29. SakaAdebayoIbrahim M&Eofficer
30. HauwaAbdulkadir Output1lead
31. YusufKoboAngo Output2lead
Bauchi
PersonalinterviewsinBauchitookplaceon25-27August2015.
Organisation Person Position
Stateministryofeducation
1. NasiruMuhammadYalwa PermanentSecretary
2. DanAzumiZakariTilde DPRS
3. UmarSani GEP3focalperson
4. BintaSambo DirectorSchoolServices
5. LouisB.Musa DirectorPolicy
SUBEB
6. SaboBappayoAhmed PermanentSecretary
7. AbdullazeezSaibo StateAlmajiriofficer
8. MuhammadBello IQEDeskOfficer
9. IsmailH.Umar DirectorSocialmobilization
CollegeofEducationAzare10. BoseOdeyemi
HeadofDepartmentEarlyChildhoodCareEducation(HeadTeacherDeskOfficer)
11. ShehuMohammedSeniorLecturerSchoolofScience(DeskOfficerSbTD)
BASAME12. HauwaAbubakarBalewa ExecutiveSecretary
13. HaliluUsmanRishi DirectorLiteracy/IQS
14. AsabeLawa GEP3focalperson
EvaluationoftheGirl’sEducationProject3
©EDOREN 241
15. AbubakarMaishanu DeskOfficerIQS
RARA16. SadiqIlelah Researchcoordinator
17. EmenguAugustus M&Eadvisor
HiLWA
18. JummeiS.Abudcar ChairHiLWA
19. TabittaaSabo DeputychairHiLWA
20. ZainabA.Gidado MemberHiLWA
21. VictoriaYusuf SecretaryHiLWA
22. SalamatuA.Lasan MemberHiLWA
23. TabithaC.Shawulu MemberHiLWA
RahamaWomenDevelopmentProgramme 24. MiriamIilya Executivedirector
FahimtaWomenandYouthDevelopmentInitiative 25. MaryamGarba Executivedirector
UNICEFFieldOfficeBauchi 26. MuntakeM.Mukhtar Educationofficer
GEP3stateteam27. SalehMuhammed Stateprojectcoordinator
28. BimbiTahir Output1lead
EvaluationoftheGirl’sEducationProject3
©EDOREN 242
AnnexC Mapsofsampledschools
Katsina–KankiaLGA
Katsina–BatsariLGA
EvaluationoftheGirl’sEducationProject3
©EDOREN 243
Katsina–RimiLGA
Zamfara–BunguduLGA
EvaluationoftheGirl’sEducationProject3
©EDOREN 244
Zamfara–KauraNamodaLGA
Zamfara–TsafeLGA
EvaluationoftheGirl’sEducationProject3
©EDOREN 245
Bauchi
Niger
EvaluationoftheGirl’sEducationProject3
©EDOREN 246
AnnexD CalculationofWeightsandFinitePopulationCorrection
FactorsforGEP3School,TeacherandPupilData
D.1 CalculationofGEP3Weights
TheweightingproceduresfortheGEP3datareflecttheoveralldesignandimplementationofthesample,describedinaprevioussectionofthisreport.ThecombinedsamplingframeconsistsofalltheeligiblepublicandintegratedQur’anicschools(IQS)includedintheGEP3projectinfourstatesinnorthernNigeria:Zamfara,Katsina,NigerandBauchi.TheGEP3projectoperatesin6selectedLGAsineachstate.InZamfaraandKatsina3LGAsoutofthe6availablehavebeenselectedtobeincludedinthiscurrentstudy.TheselectionoftheLGAswaspurposiveandleadbyUNICEF.Thestratumcorrespondstostate/LGAandtype(publicschoolorIQS)intheframeforeachdomain,andtheprimarysamplingunits(PSUs)aretheschools.
TheGEP3baselinesurveyisdesignedtoevaluatetwoseparateinterventionsunderaoverallGEP3project:earlylearningintervention(RANA)andIQSsupport.Inthecaseoftheearlylearninginterventiontheschoolswererandomisedintotreatmentandcontrolgroupsaspartoftheevaluation.Sincethesampletreatmentandcontrolschoolsareeachrepresentativeoftheframeforthecorrespondingdomain,theweightswerecalculatedseparatelyforeach.
Inordertomaintaintheeffectivesamplesize,anysampleschoolforwhichdatacouldnotbecollected(becauseofeligibilityorresponseproblems)wasreplaced.Itwasnecessarytoselectmorethanonerandomsetofreserveschoolsforreplacementinsomestrata.Eachsetofreplacementschoolswasselectedrandomlyfromtheremainingschoolsthathadyetnotbeenselectedinthecorrespondingstratum.Sinceallschoolswithinthestratumhadthesameprobabilityofbeingselectedeitherintheoriginalsampleorinoneofthesetsofreplacements,ultimatelyeachschoolinthefinalcompletedsampleforeachstratumhadthesameprobabilityofselectionandcorrespondingweight,regardlesswhetheritwasanoriginalsampleschoolorfromoneofthesetsofreplacements.
Whensomeofthesampleschoolswerecontactedduringthesamplingimplementationitwasfoundthattheydidnotmeetthecriteriaforeligibility.ThiswasespeciallyaproblemforIQS,sincemanyschoolswerenotimplementingtheteachingprogramsasmandated.Asaresult,thecorrespondingframeforeachstratumwasreduced.Alloftheschoolsintheoriginalandreplacementsamplesthatwerecontactedwereclassifiedaseligibleorineligible.However,theeligibilityoftheremaining(non-contacted)schoolsintheframeisunknown.Inordertoestimatehowmanyschoolsintheframeforeachstratumwereeligible,itwasnecessarytocalculatetheproportionofeligibleschoolsamongthosewithinformation,andmultiplythetotalnumberofschoolsintheframeforthatstratumbythecorrespondingproportion.
Withineachstratum,thesampleschoolswereselectedwithequalprobability.Thereforetheweightfortheschoolsineachstratumwouldbecalculatedasfollows:
,
where:
= weightforthesampleschoolsinstratum(state/LGA,type)h
= estimatednumberofeligibleschoolsintheframeforstratumh,calculatedasdescribedpreviously
h
hSh s
SW^
=
ShW^
hS
EvaluationoftheGirl’sEducationProject3
©EDOREN 247
= numberofsampleschoolsinstratumhwithcompletedata
Thisweightistheinverseofthecorrespondingprobabilityofselectionoftheschoolswithineachstratum.Separateweightsarecalculatedforthetreatmentandcontrolschools.Thenumeratoroftheweight(estimatednumberofeligibleschoolsinthestratum)isthesameforthetreatmentandcontrolschools,sincetheyareselectedfromthesameframe,butthenumberofsampleschoolsinthedenominatorisbasedonthenumberofsampleschoolsinthecorrespondingtreatmentorcontrolgroup.Actually,sincethenumberofsampletreatmentandcontrolschoolsinastratumisgenerallythesame,bothtypesofschoolshavethesameweight,buteachgrouprepresentstheframeseparately.
TheheadteacherandtheCBMCineachsampleschoolwouldhavethesameweightasthecorrespondingschool.
Withineachsampleschoolasampleofteachersandpupilswasselectedforthecorrespondinginterviewsandtests,sotherearetwostagesofselection.Thefirststageprobabilityisthesameasthatforthecorrespondingschool,andthesecondstageprobabilityisequaltothenumberofselectedteachersorpupilswithsurveydatadividedbythetotalnumberofteachersorpupilsintheschool.Thereforethecorrespondingweightsarecalculatedastheinverseoftheprobabilitiesasfollows:
,
where:
= weightforthesampleteachersinthei-thsampleschoolofstratumh
= totalnumberofteachersinthei-thsampleschoolofstratumh
= numberofsampleteachersinterviewed/testedinthei-thsampleschoolof
stratumh
,
where:
= weightforthesamplepupilsinthei-thsampleschoolofstratumh
= totalnumberofpupilsinthei-thsampleschoolofstratumh
= numberofsamplepupilstestedinthei-thsampleschoolofstratumh
Inordertoascertainthedenominatorforestimationofwithinschoolselectionprobabilitiesanumberofsourcesneededtobecombined.Theprimarysourceofinformationonthetotalnumberofpupilsandteachersateachschoolwerethefieldworkmonitoringformsfilledoutbysupervisorsateachschool.Duetologisticchallenges(acartransportingsomeoftheformsgotstolen)othersourceshadtobeusedtocomplementthemissinginformation.Thisalsoallowedtobenchmarkthenumbersreportedonthe
hs
hi
hiShThi tTWW ×=
ThiW
hiT
hit
hi
hiShPhi p
PWW ×=
PhiW
hiP
hip
EvaluationoftheGirl’sEducationProject3
©EDOREN 248
monitoringforms.OthersourcesincludetheteacherselectionCAPIquestionnaireforthenumberofteachers,informationextractedfromunstructuredsupervisornotes,andsurveyresponsestoanumberofquestionswithintheheadteacherinterview177.Thefinaltotalnumberofeligiblepupilsandteachersineachschoolwasreachedbycomparingdifferentdatasources.Fieldworkmonitoringdatawasassumedtobethemostcrediblesourceofinformationandothersourceswereonlyusedifthefieldworkmonitoringdatawasmissingordeemnotcredible.Thecredibilityofthedatawasascertainedthroughbenchmarkingofdifferentsources.Decisiononthefinalnumberofeligiblerespondentswasreachedonschoolbyschoolbasis.
Theweightsforschools,teachersandpupilsspecifiedherearefullweightsthatexpandthedatatotheframelevel,andalsocorrectlyweighteachstratumincalculatingdomain-levelestimates.Theywillresultinthesamevaluesforestimatesofallrelativeindicators(suchasmeans,proportionsandotherratios)asnormalisedweights.Ifanyparticularanalysisrequiresnormalisedweights,theanalystcanstandardisetheweightsbydividingthembytheaverageweightforfullsampleincludedinthatparticularanalysis.
D.2 FinitePopulationCorrectionFactorsfortheGEP3Analysis
IncalculatingthesamplingerrorandconfidenceintervalsforestimatesofdifferentindicatorsfromtheGEP3data,itisveryimportantthatthevarianceestimatortakeintoaccountthefinitepopulationcorrection(fpc)factorsforeachstageofselection,giventhesmallframeandrelativelyhighsamplingrates.Basicallythefpcfactorisequalto1minusthesamplingrateforthecorrespondingstageofselection.Itreducesthecomponentofvariancefromeachsamplingstagebythecorrespondingproportion.Inthecaseofthefirststageofselectionthefpciscalculatedatthestratumlevelsincetheprobabilitiesforallschoolsarethesamewithinastratum.Forthesecondstageofselectionthefpcfactoriscalculatedattheschoollevel,sincethesecondstageprobabilitiesforteachersandpupilsvarybyschool.
FortheSVYcommandinStata,toapplythefpcfactorsinthevarianceestimationitisnecessarytoincludeinthedatafilethefirststageprobabilityforsampleschoolsinthecorrespondingstratumandthesecondstageprobabilityforthesampleteachersandpupilsinthecorrespondingschool.TheStatasoftwarethenusestheseprobabilitiestocalculatethefirstandsecondstagefpcfactors.
Basedonthediscussionoftheprobabilitiesandweightsabove,theprobabilityoftheschoolsforthefirststagefpcwouldbecalculatedasfollows:
,
where:
= firststageprobabilityforthesampleschoolsinstratumh
Inthecaseofthesecondstagefpcfactorsforthesampleteachersandpupils,thecorrespondingprobabilitieswouldbecalculatedasfollows:
177Thevariablesconsideredforauxiliaryinformationwere:totalnumberofteachersteachingnon-religioussubjectsingradesP1-P3,totalnumberofpupilsenrolledinP2,totalnumberofpupilsinP2presentonthedayonthesurvey(headcountperformedinclass),totalnumberofpupilsattendingP2asreportedbytheteacherduringtheheadcount.
Shhi
hh WS
sp 1^1 ==
hp1
hi
hiThi T
tp =2
EvaluationoftheGirl’sEducationProject3
©EDOREN 249
,
where:
=secondstageprobabilityforthesampleteachersinthei-thsampleschoolofstratumh
=secondstageprobabilityforthesamplepupilsinthei-thsampleschoolofstratumh
hi
hiPhi P
pp =2
Thip2
Phip2
EvaluationoftheGirl’sEducationProject3
©EDOREN 250
AnnexE EthicalReview
EvaluationoftheGirl’sEducationProject3
©EDOREN 251
EvaluationoftheGirl’sEducationProject3
©EDOREN 252
AnnexF TeachermotivationsubscalesforGEP3
F.1 Distributionofthefiveteachermotivationsubscales
Thegraphsbelowshowsthedistributionofscoresoneachofthefivesubscales178179.Theaxesarethesameacrosseachofthefivegraphstomakethemeasiertocompare.Themean,standarddeviationandnumberofcasesarealsoshownwitheachgraph.
Threeofthefivesubscalesarestronglyskewed:
1. Interestandenjoyment,theskewheremaybeduetoacomplianceeffect
2. Effortandimportance,againwiththeskewpossiblyduetocomplianceeffects
3. Teacher-to-teacherinteraction.
178Thesubscales‘Importanceofteachingeffortvpupilbackground’and‘Importanceofteachingeffortvschoolinfrastructureandprofessionalsupport’makeupageneric‘perceivedefficacy’subscale.179The‘Perceivedteacherefficacy’andthe‘Pressureandworkrelatedtension’subscalesusedatathatwerereversecodedsoallgraphscanberead,fromtheperspectiveoftheGEP3,withvaluestotherightofthehorizontalscalebeingseenasabetteroutcomethanthosetotheleft.Forexample,theitem‘Iteachtoomanyclasses’inthePressureortensionsubscalewasreversecodedsothataresponseof‘Stronglyagree’ineffectbecomesequivalentto‘stronglydisagree’ontheotherscales.
EvaluationoftheGirl’sEducationProject3
©EDOREN 253
EvaluationoftheGirl’sEducationProject3
©EDOREN 254
AnnexG Randomisationchecksoftreatmentandcontrolgroup
G.1 SchoolLevelBalance
Table31: AllHeadTeachers
VariableNameTreatme
ntN
Treatme
ntMean
Control
N
Control
MeanTotalN
Total
MeanP-value
HeadTeacherisfemale 120 0 120 0 240 0.02 0.653
Hastheheadteacherbeenabsentinthelastterm? 113 0.6 112 0.7 225 0.61 0.114
NumberofdaystheHeadTeacherwasabsentinthelastterm?
63 7.7 74 8 137 7.88 0.911
Shareofschoolsthathaveseparatefunctioningtoiletforgirls
120 0.3 119 0.2 239 0.26 0.583
Numberofyearssinceintegration 60 1.8 59 2 119 1.92 0.652
TotalnumberofboysenrolledinintegratedsubjectsfromP1-P6
57 173.8 58 201.1 115 187.56 0.522
TotalnumberofgirlsenrolledinintegratedsubjectsfromP1-P6
57 121.4 58 145.2 115 133.41 0.487
Ratiooftotalnumberofchildrenstudyingintegratedsubjectstoteachers
54 42.4 60 42.2 114 42.33 0.978
RatiooftotalnumberofchildrenstudyingintegratedsubjectstoteachersinP1-P3
57 39.6 58 42.2 115 40.94 0.694
Ratioofgirlstoboysenrolledintotal 54 0.7 57 0.7 111 0.71 0.347
Headteachersatinonanylessonfortheentireduration
113 0.5 112 0.5 225 0.46 0.846
SchoolRepairs 120 0.9** 119 1 239 0.95 0.047
RatioofgirlstoboysinZamfarainP1-P3 25 0.6 30 0.6 55 0.64 0.841
RatioofgirlstoboysinKatsinainP1toP3 29 0.8 27 0.9 56 0.81 0.354
Ratiooftotalpupilsenrolledtototalroomsusedforteachingontheday
51 49.9 56 77.7 107 64.43 0.182
Fractionoffemaleteachersofintegratedsubjects 120 0.1 120 0.1 240 0.09 0.172
OverallF-test,F-stat 1.875 P-Value: 0.1722
Note:Statisticalsignificanceofmeandifferencesidentifedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
EvaluationoftheGirl’sEducationProject3
©EDOREN 255
Table32: IQSHeadTeachers
VariableNameTreatme
ntIQSN
Treatme
ntIQS
Mean
Control
IQSN
Control
IQS
Mean
Total
IQSN
Total
IQS
Mean
P-value
HeadTeacherisfemale 60 0 60 0 120 0
Hastheheadteacherbeenabsentinthelastterm? 60 0.6 58 0.7 118 0.61 0.175
NumberofdaystheHeadTeacherwasabsentinthelastterm?
33 5.9 39 7.4 72 6.71 0.412
Shareofschoolsthathaveseparatefunctioningtoiletforgirls
60 0.1 60 0.1 120 0.06 0.7
Numberofyearssinceintegration 60 1.8 59 2 119 1.92 0.652
TotalnumberofboysenrolledinintegratedsubjectsfromP1-P6
14 36.9 12 27.9 26 32.73 0.23
TotalnumberofgirlsenrolledinintegratedsubjectsfromP1-P6
14 24.9 12 28 26 26.35 0.756
Ratiooftotalnumberofchildrenstudyingintegratedsubjectstoteachers
13 42.2 15 23.5 28 32.16 0.142
RatiooftotalnumberofchildrenstudyingintegratedsubjectstoteachersinP1-P3
14 46 12 27.8 26 37.6 0.156
Ratioofgirlstoboysenrolledintotal 13 0.6 12 0.9 25 0.73 0.147
Headteachersatinonanylessonfortheentireduration
60 0.4 58 0.4 118 0.44 0.872
SchoolRepairs 60 0.9** 60 1 120 0.97 0.042
RatioofgirlstoboysinZamfarainP1-P3 3 0.5 5 0.9 8 0.74 0.133
RatioofgirlstoboysinKatsinainP1toP3 10 0.6 7 0.9 17 0.72 0.381
Ratiooftotalpupilsenrolledtototalroomsusedforteachingontheday
10 43.6 13 45.6 23 44.71 0.916
Fractionoffemaleteachersofintegratedsubjects 60 0.1** 60 0 120 0.04 0.017
OverallF-test,F-stat 5.8102 P-Value: 0.0175
Note:Statisticalsignificanceofmeandifferencesidentifedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
EvaluationoftheGirl’sEducationProject3
©EDOREN 256
Table33: PublicSchoolHeadTeachers
VariableName
Treatme
ntPublic
Schools
N
Treatme
ntPublic
Schools
Mean
Control
Public
Schools
N
Control
Public
Schools
Mean
Total
Public
Schools
N
Total
Public
Schools
Mean
P-value
HeadTeacherisfemale 60 0.1 60 0 120 0.04 0.651
Hastheheadteacherbeenabsentinthelastterm? 53 0.6 54 0.6 107 0.61 0.389
NumberofdaystheHeadTeacherwasabsentinthelastterm?
30 9.8 35 8.7 65 9.17 0.811
Shareofschoolsthathaveseparatefunctioningtoiletforgirls
60 0.5 59 0.4 119 0.46 0.644
TotalnumberofboysenrolledinintegratedsubjectsfromP1-P6
43 218.4 46 246.2 89 232.79 0.586
TotalnumberofgirlsenrolledinintegratedsubjectsfromP1-P6
43 152.8 46 175.8 89 164.69 0.583
Ratiooftotalnumberofchildrenstudyingintegratedsubjectstoteachers
41 42.5 45 48.5 86 45.64 0.526
RatiooftotalnumberofchildrenstudyingintegratedsubjectstoteachersinP1-P3
43 37.5 46 46 89 41.92 0.262
Ratioofgirlstoboysenrolledintotal 41 0.7 45 0.7 86 0.7 0.931
Headteachersatinonanylessonfortheentireduration
53 0.5 54 0.5 107 0.48 0.92
SchoolRepairs 60 0.9 59 0.9 119 0.92 0.313
RatioofgirlstoboysinZamfarainP1-P3 22 0.7 25 0.6 47 0.62 0.129
RatioofgirlstoboysinKatsinainP1toP3 19 0.8 20 0.9 39 0.85 0.887
Ratiooftotalpupilsenrolledtototalroomsusedforteachingontheday
41 51.4 43 87.4 84 69.84 0.175
Fractionoffemaleteachersofintegratedsubjects 60 0.1 60 0.1 120 0.14 0.855
OverallF-test,F-stat 0.0333 P-Value: 0.8554
Note:Statisticalsignificanceofmeandifferencesidentifedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
EvaluationoftheGirl’sEducationProject3
©EDOREN 257
Table34: AllTeachers
VariableNameTreatme
ntN
Treatme
ntMean
Control
N
Control
MeanTotalN
Total
MeanP-value
Teacherisfemale 236 0.1 241 0.1 477 0.12 0.613
DoestheteacherhaveanSSCE? 236 0.7 241 0.7 477 0.72 0.515
DoestheteacherhaveanNCE? 236 0.6 241 0.5 477 0.53 0.33
DoestheteacherhaveaGrade2Certificate? 236 0.1 241 0.1 477 0.08 0.344
Hastheteacherbeenabsentinthelastterm? 221 0.7 229 0.7 450 0.66 0.862
Numberofdaystheteacherwasabsentinthelastterm 221 6.1 228 5 449 5.52 0.341
DoestheteacherspeakHausa? 177 1 179 1 356 1
Hastheteacherattendedtraininginthelast2years? 233 0.4 240 0.5 473 0.46 0.238
Istheteacherabletoidentifylowperformers? 231 0.4 232 0.5 463 0.49 0.43
Istheteacherabletogiveevidenceforjudgementsanddiagnose?
231 0.1 232 0.1 463 0.07 0.556
Doestheteacherhavewritingskills? 231 0.2 232 0.2 463 0.17 0.578
DoestheteacherhaveHausaknowledge? 231 2.9 232 2.8 463 2.87 0.763
Doestheteacherhavecomprehensionskills? 231 2.1* 232 2.3 463 2.21 0.091
Istheteacherabletointerpretwordsandphrases?
231 0.2 232 0.1 463 0.16 0.652
OverallF-test,F-stat 0.2037 P-Value: 0.6519
Note:Statisticalsignificanceofmeandifferencesidentifedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
Table35: IQSTeachers
VariableNameTreatme
ntIQSN
Treatme
ntIQS
Mean
Control
IQSN
Control
IQS
Mean
Total
IQSN
Total
IQS
Mean
P-value
Teacherisfemale 88 0 90 0 178 0.02 0.306
DoestheteacherhaveanSSCE? 88 0.7 90 0.7 178 0.67 0.674
DoestheteacherhaveanNCE? 88 0.3 90 0.2 178 0.22 0.428
DoestheteacherhaveaGrade2Certificate? 88 0.1 90 0.1 178 0.07 0.413
EvaluationoftheGirl’sEducationProject3
©EDOREN 258
Hastheteacherbeenabsentinthelastterm? 85 0.8 88 0.8 173 0.76 0.959
Numberofdaystheteacherwasabsentinthelastterm 85 8.7 87 7.4 172 8.04 0.524
DoestheteacherspeakHausa? 59 1 64 1 123 1
Hastheteacherattendedtraininginthelast2years? 87 0.5 90 0.5 177 0.51 0.333
Istheteacherabletoidentifylowperformers? 85 0.3 88 0.6 173 0.45 0.103
Istheteacherabletogiveevidenceforjudgementsanddiagnose?
85 0 88 0 173 0.02 0.534
Doestheteacherhavewritingskills? 85 0.1 88 0.2 173 0.15 0.312
DoestheteacherhaveHausaknowledge? 85 2.9 88 2.8 173 2.84 0.84
Doestheteacherhavecomprehensionskills? 85 1.6 88 1.9 173 1.76 0.261
Istheteacherabletointerpretwordsandphrases?
85 0.1 88 0.1 173 0.1 0.211
OverallF-test,F-stat 1.5788 P-Value: 0.2106
Note:Statisticalsignificanceofmeandifferencesidentifedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
Table36: PublicSchoolsTeachers
VariableName
Treatme
nt
Public
Schools
N
Treatme
nt
Public
Schools
Mean
Control
Public
Schools
N
Control
Public
Schools
Mean
Total
Public
Schools
N
Total
Public
Schools
Mean
P-value
Teacherisfemale 148 0.2 151 0.2 299 0.18 0.817
DoestheteacherhaveanSSCE? 148 0.7 151 0.8 299 0.74 0.619
DoestheteacherhaveanNCE? 148 0.7 151 0.7 299 0.71 0.437
DoestheteacherhaveaGrade2Certificate? 148 0.1 151 0.1 299 0.08 0.567
Hastheteacherbeenabsentinthelastterm? 136 0.6 141 0.6 277 0.59 0.802
Numberofdaystheteacherwasabsentinthelastterm 136 4.4 141 3.5 277 3.95 0.479
DoestheteacherspeakHausa? 118 1 115 1 233 1
Hastheteacherattendedtraininginthelast2years? 146 0.4 150 0.4 296 0.43 0.461
Istheteacherableto 146 0.5 144 0.5 290 0.52 0.901
EvaluationoftheGirl’sEducationProject3
©EDOREN 259
identifylowperformers?
Istheteacherabletogiveevidenceforjudgementsanddiagnose?
146 0.1 144 0.1 290 0.1 0.418
Doestheteacherhavewritingskills? 146 0.2 144 0.2 290 0.19 0.935
DoestheteacherhaveHausaknowledge? 146 2.9 144 2.9 290 2.88 0.828
Doestheteacherhavecomprehensionskills? 146 2.3 144 2.6 290 2.48 0.173
Istheteacherabletointerpretwordsandphrases?
146 0.2 144 0.2 290 0.19 0.278
OverallF-test,F-stat 1.1813 P-Value: 0.278
Note:Statisticalsignificanceofmeandifferencesidentifedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
G.2 PupilLevelBalance
Table37: AllPupils
VariableNameTreatme
ntN
Treatme
ntMean
Control
N
Control
MeanTotalN
Total
MeanP-value
TotalscoreinEnglishAssessment 1323 354.374 1326 356.946 2649 355.662 0.429
TotalscoreinHausaAssessment 1322 507.341 1327 509.568 2649 508.457 0.6
SpeaksHausaathome 1318 0.991 1305 0.994 2623 0.992 0.381
Femalepupil 1324 0.457 1327 0.444 2651 0.45 0.498
Ageofpupil 843 9.61* 834 9.345 1677 9.478 0.077
Hasachair 1290 0.9*** 1288 0.897 2578 0.875 0.001
Hasamotorcycle 1291 0.673 1290 0.656 2581 0.664 0.352
Hasacar 1290 0.176 1288 0.193 2578 0.185 0.256
HasaTV 1290 0.375 1289 0.386 2579 0.38 0.587
Hasacomputer 1286 0.05 1288 0.049 2574 0.049 0.92
Hasacamera 1289 0.052 1287 0.041 2576 0.047 0.194
Hasamobilephone 1291 0.9** 1289 0.951 2580 0.941 0.025
Hascattle 1291 0.556 1288 0.53 2579 0.543 0.187
Hasagoat 1291 0.865 1288 0.866 2579 0.865 0.973
Hasahorse,donkeyormule 1290 0.178 1289 0.164 2579 0.171 0.352
Hasasheep 1291 0.7** 1289 0.684 2580 0.705 0.018
Hasachicken 1291 0.92 1290 0.91 2581 0.915 0.356
OverallF-test,F-stat 0.8539 P-Value: 0.3555
Note:Statisticalsignificanceofmeandifferencesidentifedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
EvaluationoftheGirl’sEducationProject3
©EDOREN 260
Table38: IQSPupils
VariableNameTreatme
ntIQSN
Treatme
ntIQS
Mean
Control
IQSN
Control
IQS
Mean
Total
IQSN
Total
IQS
Mean
P-value
TotalScoreinEnglishAssessment 618 390.115 644 393.839 1262 392.015 0.467
TotalscoreinHausaAssessment 618 560.557 644 565.208 1262 562.931 0.492
SpeaksHausaathome 617 0.987*** 643 1 1260 0.994 0.05
Sexofpupil 619 0.417 644 0.402 1263 0.409 0.598
Ageofpupil 409 11.675** 428 11.273 837 11.47 0.034
Hasachair 586 0.882*** 607 0.931 1193 0.907 0.004
Hasamotorcycle 586 0.683 607 0.664 1193 0.673 0.492
Hasacar 586 0.184 607 0.155 1193 0.169 0.176
HasaTV 586 0.416* 606 0.37 1192 0.393 0.099
Hasacomputer 586 0.07** 606 0.038 1192 0.054 0.015
Hasacamera 585 0.067*** 607 0.03 1192 0.048 0.003
Hasamobilephone 586 0.945 607 0.952 1193 0.949 0.593
Hascattle 586 0.584 606 0.561 1192 0.572 0.432
Hasagoat 586 0.882 607 0.883 1193 0.883 0.967
Hasahorse,donkeyormule 586 0.23 606 0.224 1192 0.227 0.807
Hasasheep 586 0.783 607 0.751 1193 0.767 0.191
Hasachicken 586 0.935 607 0.931 1193 0.933 0.764
OverallF-test,F-stat 0.09 P-Value: 0.7643
Note:Statisticalsignificanceofmeandifferencesidentifedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
Table39: PublicSchoolPupils
VariableName
Treatme
nt
Public
Schools
N
Treatme
nt
Public
Schools
Mean
Control
Public
Schools
N
Control
Public
Schools
Mean
Total
Public
Schools
N
Total
Public
Schools
Mean
P-value
TotalscoreinEnglishAssessment 705 323.044 682 322.109 1387 322.584 0.771
TotalscoreinHausaAssessment 704 460.625 683 457.106 1387 458.892 0.332
SpeaksHausaathome 701 0.994 662 0.988 1363 0.991 0.213
Sexofpupil 705 0.492 683 0.483 1388 0.488 0.737
Ageofpupil 434 7.7*** 406 7.313 840 7.494 0.005
Hasachair 704 0.8* 681 0.866 1385 0.848 0.066
Hasamotorcycle 705 0.665 683 0.649 1388 0.657 0.514
Hasacar 704 0.2*** 681 0.228 1385 0.198 0.006
HasaTV 704 0.3** 683 0.4 1387 0.37 0.023
EvaluationoftheGirl’sEducationProject3
©EDOREN 261
Hasacomputer 700 0** 682 0.059 1382 0.046 0.022
Hasacamera 704 0.04 680 0.051 1384 0.046 0.298
Hasamobilephone 705 0.9** 682 0.95 1387 0.934 0.015
Hascattle 705 0.533 682 0.503 1387 0.518 0.258
Hasagoat 705 0.851 681 0.85 1386 0.851 0.965
Hasahorse,donkeyormule 704 0.135 683 0.111 1387 0.123 0.18
Hasasheep 705 0.7** 682 0.625 1387 0.652 0.032
Hasachicken 705 0.908 683 0.892 1388 0.9 0.317
OverallF-test,F-stat 1.0007 P-Value: 0.3173
Note:Statisticalsignificanceofmeandifferencesidentifedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
Table40: AllFemalePupils
VariableName
Treatme
ntGirls
N
Treatme
ntGirls
Mean
Control
GirlsN
Control
Girls
Mean
Total
GirlsN
Total
Girls
Mean
P-value
TotalscoreinEnglishAssessment 605 339.4** 589 351.064 1194 345.169 0.011
TotalscoreinHausaAssessment 605 490.917 589 498.089 1194 494.455 0.213
SpeaksHausaathome 601 0.988 580 0.991 1181 0.99 0.604
Sexofpupil 605 1 589 1 1194 1
Ageofpupil 375 9.176 367 9.155 742 9.166 0.923
Hasachair 604 0.9** 587 0.906 1191 0.887 0.035
Hasamotorcycle 605 0.674 588 0.651 1193 0.663 0.401
Hasacar 605 0.177 586 0.212 1191 0.194 0.13
HasaTV 604 0.4* 588 0.425 1192 0.399 0.072
Hasacomputer 603 0.043 586 0.055 1189 0.049 0.359
Hasacamera 603 0.033 586 0.038 1189 0.035 0.683
Hasamobilephone 605 0.932 588 0.947 1193 0.94 0.275
Hascattle 605 0.529 586 0.483 1191 0.506 0.113
Hasagoat 605 0.853 588 0.84 1193 0.847 0.542
Hasahorse,donkeyormule 604 0.142 588 0.133 1192 0.138 0.626
Hasasheep 605 0.701 588 0.667 1193 0.684 0.205
Hasachicken 605 0.909 588 0.896 1193 0.903 0.455
OverallF-test,F-stat 0.5578 P-Value: 0.4553
Note:Statisticalsignificanceofmeandifferencesidentifedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
EvaluationoftheGirl’sEducationProject3
©EDOREN 262
Table41: AllMalePupils
VariableName
Treatme
ntBoys
N
Treatme
ntBoys
Mean
Control
BoysN
Control
Boys
Mean
Total
BoysN
Total
Boys
Mean
P-value
TotalscoreinEnglishAssessment 718 366.967 737 361.647 1455 364.272 0.24
TotalscoreinHausaAssessment 717 521.2 738 518.73 1455 519.947 0.684
SpeaksHausaathome 717 0.993 725 0.996 1442 0.994 0.47
Sexofpupil 719 0 738 0 1457 0
Ageofpupil 468 10** 467 9.495 935 9.726 0.025
Hasachair 686 0.8** 701 0.889 1387 0.866 0.012
Hasamotorcycle 686 0.672 702 0.66 1388 0.666 0.623
Hasacar 685 0.175 702 0.178 1387 0.177 0.888
HasaTV 686 0.376 701 0.352 1387 0.364 0.359
Hasacomputer 683 0.056 702 0.044 1385 0.05 0.327
Hasacamera 686 0.1* 701 0.044 1387 0.056 0.05
Hasamobilephone 686 0.9** 701 0.954 1387 0.942 0.041
Hascattle 686 0.58 702 0.57 1388 0.575 0.696
Hasagoat 686 0.876 700 0.887 1386 0.882 0.525
Hasahorse,donkeyormule 686 0.21 701 0.191 1387 0.2 0.384
Hasasheep 686 0.7** 701 0.699 1387 0.724 0.036
Hasachicken 686 0.93 702 0.922 1388 0.926 0.552
OverallF-test,F-stat 0.3544 P-Value: 0.5517
Note:Statisticalsignificanceofmeandifferencesidentifedas*atthe10%,**atthe5%and***atthe1%level.Thisisbasedonthep-valuemeasuresreportedinthetable.
EvaluationoftheGirl’sEducationProject3
©EDOREN 263
AnnexH QualitativeEvaluationMatrixforSupportforIQS
Contribution Hypotheses Assumptions(outputtooutcomelevel) Coreareastoprobe(includingchangeovertimeforeach)Sourceof
information
Improvedlearningoutcomes(literacyandnumeracy),especiallyforgirls
Teachertrainingandmentoring,accesstofit-for-purposeteachingandlearningmaterialsandimprovedheadteacherpedagogicalleadershipandsupportwillleadtomoreeffective,gender-sensitiveteachingoftheintegratedcurriculum.
• Teachersandotherstakeholderswill,throughtheIQSactivities,bemorewillingandabletotakeonnewskills,rolesorwaysofworking.
• High-qualitymentoringcanimproveteachingquality.• Teacherpedagogiesandattitudesregardingtheroles,
responsibilitiesandcapabilitiesofgirlsandboyscanchangeasaresultoftrainingandmentoringingendersensitivity.
• Fit-for-purposeteaching/learningmaterialscanincreasequalityofteaching.
• Teachermotivationcanbeimprovedthroughteachermentoring,improvedheadteacherpedagogicalleadershipandsupport.
• Teachersandpupilswillbepresent,andsufficienttimewillbeallottedtotheteachingofformalsubjects.
• Theharmonisedcurriculumallotmentofeighthoursfortheteachingoftheintegratedcurriculumwillbeadheredto.
• Trainingandmentoringwillbewelltargeted.• ThereissufficienttimeforGEP3todemonstrateresults
withinthepilotperiod.
• Howthementoringandtrainingisimplemented.• Perceivedbenefitsandchallengesofworkingasateacher
inIQS.• HowtheorganisationoftheIQS,thetypeofpupilsit
attractsanditsrelationshipwiththecommunityinfluencestheeffectiveteachingoftheintegratedcurriculum.
• Perceivedbenefitsandchallengesoftheintegratedcurriculumandteachingandlearningmaterialssuppliedthroughtheintervention.
• Satisfactionwithacquiredskillsandskillsthatcanbeimproved.
• Willingnessandabilitytousenewskills.• Teacherperceptionsandattitudestowardgirlsandboys.• Waysinwhichtraininghasimpactedonteacherpraxis.• Extenttowhichgirlsareabletospeakupaboutissuesand
challengestheyface.• Schoolleaders’perceptionsandattitudestowardgirlsand
boys.• Perceivedchangesbeyondlearning,teachingandschool
leadershipthattheIQSsupporthascontributedto.• TheplausibilityofGEP3interventionsdemonstrating
resultswithinthepilotperiod.
• KIIswithmallams
• KIIswithheadteachers
• FGDswithparents
• FGDswithCBMCmembers
• FGDswithpupils
• QCOwithteacher
• TPDwithteacher
Improvedretentionofgirls
Improvedschoolplanning,mobilisedcommunityandGEP3resources,andimprovingheadteachersplanningwillimprovetheschoolexperienceforgirlsandtheywillbemotivatedtostayinschoolandwillbeexposedtoopportunitiestolearn.
• CBMCmembersarewillingandabletotakeonnewskills,knowledgeandwaysofworking.
• Schoolleadersarewillingtoreflectonandconsideredgenderequityissuesinschoolplanning.
• CBMCsarewillingtoinvestcommunityandGEP3resourcesinschoolinginfrastructurethatbenefitsgirls.
• Governmentandcommunitieshavesufficientcapacitytoretaingirlsandsustainquality.
• Communitymembersarewillingtoreflectonandconsidertheroles,responsibilitiesandcapabilitiesofgirls.
• CBMCpresenceandestablishmenthistory.• Securityandsafetysituationinthelocation.• Proprietorviewsoftheintegratedcurriculum.• Proprietor,communityandCBMCperceptionsand
attitudestowardgirlsandboys.• CBMCrepresentationofcommunityandinfluencing
power.
• KIIswithmallams
• KIIswithheadteachers
• KIIwithcommunityleaders
• FGDswithparents
• FGDswithCBMCmembers
• FGDswithpupils
Increasedenrolmentofgirls
CBMCsensitisation,animprovementinteachingqualityandagirl-friendlylearningenvironmentwillleadtoanincreaseddemandforgirls’education.
• Parentsandcommunitiesarewillingtoreflectonandconsidertheroles,responsibilitiesandcapabilitiesofgirls.
• Parentsandcommunitieshavesufficienteconomicresourcestoenrolmoregirls.
• CBMCsensitisationiswelltargetedandofhighquality.
• Parental,communityandCBMCperceptionsandattitudestowardgirlsandboys.
• Effectivenessofthecashtransferinalleviatingfinancialbarriers.
• Communityandparentalperceptionsoftheintegratedcurriculum.
• KIIswithmallams
• KIIswithheadteachers
• KIIswithcommunityleaders
• FGDswithparents
• FGDswithCBMCmembers
• FGDswithpupils
EvaluationoftheGirl’sEducationProject3
©EDOREN 264
AnnexI Evaluationgovernance,managementandindependence
TheevaluationoftheGEP3isdesignedandimplementedbyEDOREN,aDFID-fundedproject,andthereforehasclearaccountabilitytoDFIDthroughprojectreporting.TheevaluationistobegovernedbyasteeringgroupcomposedofDFIDstaff,aGEP3representative,andanEDORENrepresentative.Theindependenceoftheevaluationisassuredthroughtransparencyandrigorouspeerreview.
I.1 Governanceandmanagement
ThisevaluationframeworkandtheevaluationthatitsetsoutaremanagedandimplementedbyEDOREN,aDFID-fundedprojectmanagedbyOPM,andspecificallyEDORENworkstream1.EDORENworkstream1isresponsibleforthedesignoftheevaluationassetoutinthisdocumentandtheimplementationoftheEDOREN-ledevaluationactivities.TheEDOREN-ledevaluationactivitiesdrawextensivelyoninformationanddatagatheredbyOPM’sAbujaOfficeaspartofamilestonecontractundertheEDORENcontract.EDOREN-ledqualitativedatacollectionandanalysishasbeenundertakenbyanOPMQualitativeLeadwithateamofnationalqualitativeresearchersundertheEDORENcontract.TheOPMAbujabaselinesurveyteamreportregularlytotheEDORENworkstream1GEP3evaluationteaminordertoensurethatthebaselinemeetstherequirementsoftheevaluationframework.EDORENteammembersandtheOPMAbujaFieldworkManagerreporttotheGEP3EvaluationProjectManageronallaspectsoftheworkthatrelatetoevaluationresources,clientliaisonandtimelines.EDORENteammembersreporttotheGEP3TeamLeaderonalltechnicalcomponentsofthework.TheOPMAbujaFieldworkManagerreportstotheEDORENSurveyLeadonalltechnicalcomponentsofthework.
TheEDORENFieldworkManagerliaiseddirectlywithstateandLGAgovernmentrepresentativesregardingGEP3evaluationsurveyactivities.LiaisonwithstateandLGAgovernmentofficeswasundertakenpriortoanyschoolvisitsforpre-piloting,pilotingandsurveydatacollection.Afterundertakingschoolvisits,theGEP3FieldworkManagerreportedbacktogovernmentofficestodetailthedata-collectionactivities.
Figure83: GEP3evaluationgovernancearrangements
TheevaluationisgovernedbyDFID,whichentailsthattheevaluationteamreportstotheDFIDNigeriaeducationteamonprogresstowardevaluationobjectives.ThistakesplacethroughregularEDOREN
EvaluationoftheGirl’sEducationProject3
©EDOREN 265
quarterlywrittenreportingtoDFIDandsix-weeklyverbalprojectmanagementupdates.ThisprocessmeansthattheEDORENGEP3evaluationteamprovidesfortnightlyupdatesdirectlytotheDFIDEducatioAdvisor.
Inaddition,itwasrecommendedintheEvaluationFrameworkthatanEvaluationSteeringCommitteebeformalised,composedofDFIDNigeriaeducationandresultsadvisers,GEP3andEDORENstaff–agroupofpeoplethatalreadymeetsregularly.Thiscommitteecouldformallymeeteverysixmonthstodiscussevaluationprogress,andwouldberesponsibleforpeerreviewandQA(inadditiontoEDOREN’sinternalQAprocesses).
TheissueofgovernmentrepresentationontheGEP3EvaluationSteeringCommitteewasdiscussedbyEDOREN,UNICEFandDFID,180withUNICEFnotingthatgovernmentparticipationisimportantforownershipandtake-upofresults.EDORENfullyagreedandindicatedthatitwouldwelcometheparticipationofagovernmentrepresentativebutwouldseektohaveDFIDandUNICEFtaketheleadinapproachingandliaisingwithgovernmenttosetupthisparticipation.DFIDandUNICEFagreedtoliaisewithgovernmentonthisissue.Atpresent,DFIDandUNICEFplantousethebaselineevaluationreporttoengagegovernmentandbegintheprocessofestablishingaGEP3EvaluationSteeringCommitteeasoriginallyenvisaged.
I.2 Stakeholderinvolvementinframeworkdevelopment
TheGEPTeamLeadandtheGEP3ProjectManager/EducationSpecialistengagedwithDFIDNigeria,DFID’sResearchandEvidenceDivision,GEP3management,UNICEF,internationalresearcherswithinOPMandEDOREN,othereducationprojectsandimplementingagenciesinbothface-to-facemeetingsandoverSkypeonmultipleoccasionsthroughouttheevaluationdesignprocess.EngaginggovernmentthroughouttheevaluationdesignwascomplexinthecaseofGEP3astheEvaluationFrameworkwasputonholdduringtheredesignoftheGEP3programme.Therefore,theevaluationteamplacedaspecificfocusonengagingwithgovernmentduringthestate-levelstakeholderengagementactivityintheearlystagesoftheevaluation.
AftertheGEP3programmedesignwasfinalised,theimplementingpartner,DFIDandtheEDORENGEP3Evaluationteamspent10daysinMarch2015completingtheevaluationmatrix,byfinalisingtheevaluationquestions,assigningresponsibilitiesandresourcesagainstevaluationactivities,andfirmingupthetimingofvariousactivities.
TheGEP3evaluationteamhasdocumentedthediscussionsundertakenthroughoutthisengagementandhaskepthighlydetailedminutesofmeetingswithkeystakeholders.Thishasenabledtheteamtoreviewtheviews,prioritiesandconcernsofstakeholders,indetail,whiledevelopingtheevaluationframework.Atpointswheretrade-offsorcompromiseswerenecessaryduetoscopeorresources,theGEP3EvaluationTeamLeadandProjectManagercommunicatedtheoptionsandtrade-offstokeystakeholdersinordertoensureallpartieswereinvolvedinthedecision-makingprocesses.
DFIDeducationspecialistsandtheprogrammeimplementerswereprovidedwithafinaldraftoftheevaluationframeworkpriortofinalisationsoastoensurealldecisionsandagreementsarereflectedinthefinalproduct.
1802July2015EDORENGEP3Evaluationmeetingminutes,circulated8July2015.
EvaluationoftheGirl’sEducationProject3
©EDOREN 266
AnnexJ RobustnessChecksonAgeforMainRegressionModel
Table42: Hausa-MainModelwithAgeImputedforPupilswithMissingAgeData
Variables Coeff. StdErr T-Stat Pvalue
Gender(pupilisfemale) -11.980 4.791 -2.500 0.013**
Agedfrom7to10 20.709 5.418 3.823 0.000***
Agedfrom11to15 60.352 6.835 8.830 0.000***
Agedover16 110.609 21.360 5.178 0.000***
PupilinsecondHWItertile 9.662 5.174 1.867 0.063*
Pupilinthird(top)HWItertile 33.167 5.815 5.704 0.000***
Publicschool(notIQS) -73.045 9.629 -7.586 0.000***
Schoolhasgirltoilets 4.140 7.389 0.560 0.576
Schoolhaswater 9.821 7.219 1.360 0.175
Pupilattendsotherschool -1.819 5.956 -0.305 0.760
Schoolinruralareas -35.313 11.824 -2.987 0.003**
SchoolinKatsina 10.923 6.917 1.579 0.116
Teachermotivation 10.614 18.391 0.577 0.564
Teacherknowledge1 22.352 16.166 1.383 0.168
Teacherknowledge2 2.065 5.076 0.407 0.685
Teacherpedagogy1 -20.369 23.863 -0.854 0.394
Teacherpedagogy2 0.206 0.363 0.567 0.571
constant 475.843 58.768 8.097 0.000***Numberofobservations:2,538R-squared=0.280
Table43: English-MainModelwithAgeImputedforPupilswithMissingAgeData
Variables Coeff. StdErr T-Stat P-Value
Gender(pupilisfemale) -9.170 4.154 -2.208 0.028**
Agedfrom7to10 7.771 5.020 1.548 0.123
Agedfrom11to15 39.531 6.544 6.041 0.000***
Agedover16 65.032 13.021 4.994 0.000***
PupilinsecondHWItertile 13.196 4.378 3.014 0.003**
Pupilinthird(top)HWItertile 24.565 4.665 5.265 0.000***
Publicschool(notIQS) -56.480 8.107 -6.966 0.000***
Schoolhasgirltoilets 9.899 5.999 1.650 0.100
Schoolhaswater 9.997 5.356 1.866 0.063*
Pupilattendsotherschool 4.288 5.781 0.742 0.459
Schoolinruralareas -35.278 9.084 -3.884 0.000***
SchoolinKatsina 8.077 5.381 1.501 0.135
Teachermotivation 18.153 15.030 1.208 0.228
EvaluationoftheGirl’sEducationProject3
©EDOREN 267
Teacherknowledge1 19.939 13.245 1.505 0.134
Teacherknowledge2 -0.172 3.563 -0.048 0.961
Teacherpedagogy1 -25.700 17.271 -1.488 0.138
Teacherpedagogy2 0.291 0.320 0.908 0.365
constant 302.603 50.886 5.947 0.000***Numberofobservations:2,538R-squared=0.226
Table44: Hausa-Fixed-EffectsModelwithAgeImputedforPupilswithMissingAgeData
Variables Coeff. StdErr T-Stat Pvalue
Gender(pupilisfemale) -9.959 4.492 -2.217 0.028**
Agedfrom7to10 12.423 4.899 2.536 0.012**
Agedfrom11to15 40.786 6.077 6.712 0.000***
Agedover16 77.086 21.974 3.508 0.001***
PupilinHWItertile -0.805 4.868 -0.165 0.869
PupilinthirdHWItertile 20.616 4.551 4.530 0.000***
Pupilattendsotherschool 4.316 4.868 0.887 0.376Numberofobservations:2,580R-squared=0.502
Table45: English-Fixed-EffectsModelwithAgeImputedforPupilswithMissingAgeData
Variables Coeff. StdErr T-Stat Pvalue
Gender(pupilisfemale) -9.858 3.939 -2.502 0.013**
Agedfrom7to10 3.465 4.960 0.699 0.485
Agedfrom11to15 26.774 6.466 4.141 0.000***
Agedover16 42.177 13.284 3.175 0.002**
PupilinHWItertile 7.630 4.275 1.785 0.076*
PupilinthirdHWItertile 17.053 3.985 4.279 0.000***
Pupilattendsotherschool 8.633 5.009 1.723 0.086*Numberofobservations:2,580R-squared=0.421
ExclusionofAgeVariable
Table46: Hausa-MainModelwithBoardingVariableandNoAgeVariable
Variables Coeff. StdErr T-Stat Pvalue
Gender(pupilisfemale) -10.529 4.681 -2.249 0.025**
Pupilisaboarder 28.227 12.449 2.267 0.024**
PupilinsecondHWItertile 10.091 5.016 2.012 0.045**
Pupilinthird(top)HWItertile 33.216 5.498 6.042 0.000***
Publicschool(notIQS) -85.004 10.847 -7.837 0.000***
Schoolhasgirltoilets 1.701 8.077 0.211 0.833
EvaluationoftheGirl’sEducationProject3
©EDOREN 268
Schoolhaswater 11.559 7.874 1.468 0.143
Pupilattendsotherschool -2.310 6.238 -0.370 0.711
Schoolinruralareas -34.169 12.822 -2.665 0.008**
SchoolinKatsina 12.939 7.442 1.739 0.083*
Teachermotivation 16.851 20.010 0.842 0.401
Teacherknowledge1 26.633 17.606 1.513 0.132
Teacherknowledge2 2.821 5.304 0.532 0.595
Teacherpedagogy1 -25.077 26.668 -0.940 0.348
Teacherpedagogy2 0.194 0.388 0.499 0.618
constant 488.359 64.009 7.630 0.000***Numberofobservations:2,538R-squared:0.232
Table47: English-MainModelwithBoardingVariableandNoAgeVariable
Variables Coeff. StdErr T-Stat Pvalue
Gender(pupilisfemale) -8.922 4.176 -2.136 0.034**
Pupilisaboarder 15.718 9.213 1.706 0.089*
PupilinsecondHWItertile 13.515 4.180 3.233 0.001***
Pupilinthird(top)HWItertile 24.613 4.642 5.303 0.000***
Publicschool(notIQS) -65.381 8.801 -7.429 0.000***
Schoolhasgirltoilets 8.065 6.489 1.243 0.215
Schoolhaswater 10.959 5.863 1.869 0.063*
Pupilattendsotherschool 3.564 5.839 0.610 0.542
Schoolinruralareas -34.187 9.692 -3.527 0.001***
SchoolinKatsina 9.195 5.866 1.568 0.118
Teachermotivation 21.703 16.128 1.346 0.180
Teacherknowledge1 22.302 14.233 1.567 0.119
Teacherknowledge2 0.349 3.706 0.094 0.925
Teacherpedagogy1 -29.324 19.202 -1.527 0.128
Teacherpedagogy2 0.289 0.337 0.857 0.392
constant 310.617 54.694 5.679 0.000***Numberofobservations:2,538R-squared:0.190
Table48: Hausa-Fixed-EffectsModelwithBoardingVariableandNoAgeVariable
Variables Coeff. StdErr T-Stat Pvalue
Gender(pupilisfemale) -9.041 4.442 -2.035 0.043**
Pupilisaboarder 21.837 11.471 1.904 0.058*
PupilinHWItertile -1.228 4.840 -0.254 0.800
PupilinthirdHWItertile 20.030 4.437 4.514 0.000***
Pupilattendsotherschool 4.041 4.799 0.842 0.401Numberofobservations:2,580
EvaluationoftheGirl’sEducationProject3
©EDOREN 269
R-squared:0.482
Table49: English-Fixed-EffectsModelwithBoardingVariableandNoAgeVariable
Variables Coeff. StdErr T-Stat Pvalue
Gender(pupilisfemale) -9.226 3.967 -2.326 0.021**
Pupilisaboarder 17.875 7.862 2.274 0.024**
PupilinHWItertile 7.400 4.198 1.763 0.079*
PupilinthirdHWItertile 16.649 4.102 4.059 0.000***
Pupilattendsotherschool 8.328 4.853 1.716 0.088*Numberofobservations:2,580R-squared:0.407
EvaluationoftheGirl’sEducationProject3
©EDOREN 270
AnnexK GEP3AssetIndex
TheassetsincludedintheGEP3PupilQuestionnairetoapproximatethehouseholdwealthwereacombinationoftheassetsavailableintheHNLSSandtheMICS.TheHNLSSincludesarangeofassets(bed,chair,car,radio,TV,etc.)butdoesnotincludeanylivestockvariables.AlthoughtheMICShassomelivestockvariables(cattle,sheep,goats,etc.)ithasfewerassetsoverall.Sinceapriori,wewereunsureofwhetherHNLSSassetsorMICSassetswoulddoabetterjobofdifferentiatingoursampleaccordingtohouseholdwealth,weincludedbothsetsofvariablesinthepupilquestionnaire.
Atthetimeofdesigningthequestionnaire,themodelinmindtodeveloptheassetindexincludedthefollowingassets(allfromHNLSS).ThisdecisionwasmadebasedontheabilityofthissetofassetstodifferentiatetheHNLSSsamplefortheGEP3statesintoquantiles.Insteadofassigningeachassetanequalweightingintheindex,PrincipleComponentAnalysiswasusedtoderivecoefficients(orrelativeweights)foreachassetinthecompositeindex181.Table50givesthecoefficientsderivedfromtheHNLSSdatawhileTable51givesanindicationoftheinitialmodel’sabilitytodifferentiatethesampleintomeaningfulquantiles.
Table50: CoefficientsoftheVariablesincludedintheassetindex,usingPCAonHNLSSdata
Variable Level HNLSScoefficient
Car 0 -0.03928 1 0.567445Fridge 0 -0.0619
1 0.511007
Radio 0 -0.11876 1 0.052846Bed 0 -0.11163 1 0.03721TV 0 -0.12148 1 0.370352Motorcycle 0 -0.06149 1 0.121296Chairs 0 -0.069 1 0.28338Mat 0 -0.02556 1 0.004833Sewingmachine 0 -0.02329
1 0.345258
Airconditioning 0 -0.01403 1 0.75289Fan 0 -0.11178
1 0.39432 Generator 0 -0.04403
1 0.54349
Washingmachine 0 -0.00099
1 0.474309
181Recommendedbymany,including:Filmer,D.,&Pritchett,L.H.(2001),McKenzie,DavidJ.(2005),Vyass,Seema,andLilaniKumaranayake(2006).
EvaluationoftheGirl’sEducationProject3
©EDOREN 271
Stove 0 -0.07671 1 0.385413Gascooker 0 -0.00486 1 0.730374Camera 0 -0.0033 1 0.6756Computer 0 -0.01056 1 0.793258Mobilephone 0 -0.16452 1 0.26767Bicycle 0 0.002975 1 -0.00737
Theabilityofthecompositeindextodividethesamplewasverifiedbytheproportionofthepopulationthatfellwithineachx-tile:
Table51: X-tiledivisionswiththeassetindex
Population SubdivisionwithHNLSS
Decile Quintile Quartile Tertile
1stDivision 11.78 20.31 25.27 40.89
2ndDivision 8.53 20.58 24.97 26.72
3rdDivision 10.31 19.16 24.95 32.39
4thDivision 10.27 20.11 24.81
5thDivision 9.35 19.85
6thDivision 9.81
7thDivision 10.07
8thDivision 10.04
9thDivision 9.97
10thDivision 9.88
AftertheGEP3datacollectionwascomplete,theintendedplanwastoapplytheHNLSSderivedcoefficientsonthedatacollectedtodevelopacompositeassetindexafterverifyingtherobustnessoftheproposedindexbyderivingcoefficientsusingPCAontheGEP3data.PCA,usingthecompletesetofassetsdiscussedabovecouldnotbecarriedoutontheGEPdatasincegascookersandwashingmachineswerenotcommoninthesample.
Table52: Ownershipofgascookerandwashingmachine
Variable Mean Std.Deviation TotalNAssetownedby
(n)
Gascooker .0236011 .1518316 2,627 62
Washingmachine .0125619 .1113947 2,627 33
Asshownabove,only1.25%ofthesampleownedawashingmachinewhileonly2.4%ofthesampleownedagascooker.Asaresult,thesevariableswerenotsuitableforinclusionintheGEP3assetindex.
EvaluationoftheGirl’sEducationProject3
©EDOREN 272
Were-ranthemodelbyexcludingtheabovevariablesi.e.washingmachineandgascooker,fromtheinitiallistofvariables(Model1),andbyexcludingtheabovevariablesbutaddingthelivestockvariables(Model2).ThePCAcoefficientsforeachmodelarepresentedbelow(Table53)alongwiththeabilityofthemodeltodifferentiatethesamplebyhouseholdwealth(Table54).
Table53: CoefficientsoftheVariablesincludedintheassetindex,usingPCAonGEP3data
Variable Variablevalue Model1coefficients Model2coefficients
Car 0 -0.085882 -0.079132Car 1 0.3937 0.362757Fridge 0 -0.066786 -0.060538Fridge 1 0.479059 0.43424Radio 0 -0.303788 -0.298376Radio 1 0.06644 0.065257Bed 0 -0.460478 -0.453385Bed 1 0.014353 0.014132Tv 0 -0.184467 -0.167311Tv 1 0.30777 0.279146Motorcycle 0 -0.258315 -0.258875Motorcycle 1 0.133362 0.133651Chairs 0 -0.288097 -0.283432Chairs 1 0.040449 0.039794Mat 0 -0.250035 -0.292829Mat 1 0.002247 0.002632Sewing_Machine 0 -0.142277 -0.135761Sewing_Machine 1 0.246866 0.23556Ac 0 -0.006957 -0.006557Ac 1 0.468972 0.44199Fan 0 -0.141947 -0.127169Fan 1 0.375635 0.336527Generator 0 -0.12895 -0.123251Generator 1 0.371828 0.355396Stove 0 -0.153832 -0.145716Stove 1 0.274951 0.260447Camera 0 -0.024753 -0.023854Camera 1 0.52441 0.505365Computer 0 -0.026476 -0.024326Computer 1 0.510992 0.469496Mobile 0 -0.47705 -0.476349Mobile 1 0.030202 0.030157Bicycle 0 -0.224093 -0.231451Bicycle 1 0.115498 0.119291Goat 0 -0.165671Goat 1 0.025922Cattle_Cows_Bulls 0 -0.125839Cattle_Cows_Bulls 1 0.107044Horse_Donkey_Mule 0 -0.049963Horse_Donkey_Mule 1 0.241427
EvaluationoftheGirl’sEducationProject3
©EDOREN 273
Sheep 0 -0.179079Sheep 1 0.075002Chicken 0 -0.221941Chicken 1 0.020529
Thougheachofthemodelssuccessfullydividedthesampleintotertiles(with33%ofthepopulationineachband),thedivisionintoquartiles,quintilesanddecilesvaried.
Table54: DivisionIntoTertiles,ByModel
Population Model1 Model2
1stDivision 33.61 33.35
2ndDivision 33.08 33.35
3rdDivision 33.31 33.31
Table55: DivisionIntoQuartiles,ByModel
Population Model1 Model2
1stDivision 25.39 25.01
2ndDivision 24.70 26.38
3rdDivision 25.01 23.64
4thDivision 24.90 24.97
Table56: DivisionIntoQuintiles,ByModel
Population Model1 Model2
1stDivision 20.02 20.02
2ndDivision 24.25 20.21
3rdDivision 16.18 20.40
4thDivision 20.18 19.38
5thDivision 19.38 19.98
Table57: DivisionIntoDeciles,ByModel
Population Model1 Model2
1stDivision 10.01 10.01
2ndDivision 10.01 10.01
3rdDivision 11.42 10.01
4thDivision 12.83 10.20
5thDivision 5.82 11.15
6thDivision 10.35 9.25
7thDivision 10.05 9.36
8thDivision 10.13 10.01
EvaluationoftheGirl’sEducationProject3
©EDOREN 274
9thDivision 9.40 10.09
10thDivision 9.97 9.90
Model1andModel2assignsimilarweightstocommonassetsincludedinthemodelwhichsuggeststhatModel1isarobustspecificationsinceityieldssimilarcoefficientswithasmallersetofvariables.Model2faresbetterindividingupthesampleintogroups–i.e.itsdivisionsholdforquintilesaswellasdeciles.Model1,ontheotherhand,successfullydividesthesampleintotertilesandquartiles.
AnadditionalrobustnesscheckforModel1istoderivethisnewmodelusingtheHNLSSdataforGEPstates,aswasdoneinitially.ThiscannotbedoneforModel2sinceithaslivestockvariableswhicharenotincludedintheHNLSSdataset.BycalculatingtheHNLSScoefficientsaccordingtoModel1,weseethatthiscompositeindexstillsplitsupthesampleintoapproximatelyequivalentquartilesandquintiles.
Table58: X-tiledivisionswiththeassetindex(Model1)onHNLSSdata
PopulationSubdivisionwithHNLSS
Decile Quintile Quartile Tertile
1stDivision 12.01 20.09 25.49 42.73
2ndDivision 8.08 22.64 26.22 23.95
3rdDivision 10.80 18.84 23.38 33.31
4thDivision 11.85 18.59 24.91
5thDivision 8.97 19.83
6thDivision 9.87
7thDivision 8.46
8thDivision 10.12
9thDivision 9.84
10thDivision 10.00
ByapplyingtheHNLSScoefficientsfromthenewmodel(excludinggascookerandwashingmachine),wefindthattheHNLSScoefficientswhenappliedtotheGEPdatastilldifferentiatethesampleintoequivalentquantilesandquartiles.Thoughtheabsolutevaluesoftheasset‘score’varybasedonthecoefficientsused,theabilitytosplitupthepopulationholds.
EvaluationoftheGirl’sEducationProject3
©EDOREN 275
Table59: ComparisonofModel1usingGEPandHNLSScoefficients
PopulationModel1
UsingGEPcoefficients UsingHNLSScoefficients
Min Max Mean Min Max Mean
1stDivision -3.486999 .702507 -1.001735 -1.083544 -.65499 -.8612033
2ndDivision -2.450946 1.155099 -.0800134 -.649942 -.371386 -.5463656
3rdDivision -2.225513 1.661305 .2626109 -.364134 .399303 -.0181731
4thDivision -1.6906 2.165238 .8327104 .401963 5.160044 1.389426*differentdatasourcesusedtoderivecoefficients,appliedtoGEP3dataset ThefinalchoiceofthecoefficientsusedtodeveloptheGEP3assetindexwasafunctionofthevariationinassetownershipintheactualdata,robustnesscheckonthemodelusingtheHNLSSdata,andtheabilityofthemodeltodifferentiatethesampleintogroupsbasedonassetownership.
Lastly,theassetindexwasonlyusedinthebaselineanalysisforthepupilsintheEarlyLearningevaluationsample.TheindexwasnotconstructedforpupilsintheIQSsupportsampleduetoasmallsampleofthesampledpupilsowning‘luxury’assetscombinedwithlimitedvariationinassetownership(Table60).Principalcomponentanalysisimputesascoreforeachobservationusingthevariationacrossthesample.However,inthecaseofIQSsupportpupils,ownershipofassetssuchascamera,computerandairconditionerwereperfectpredictorsofownershipofotherassetssuchasbed,matandmattress.Duetothis,principalcomponentanalysiswasnotpossibleforthissampleusingtherobustmodeldevelopedfortheEarlyLearningsample.HenceithasnotbeenreportedorusedintheanalysisofIQSsupportschoolsandassociatedpupiloutcomes.
Table60: OwnershipofassetsinIQSsupportsample
Variable Mean Std.Deviation TotalNAssetownedby
(n)
Computer .0664336 .2492564 572 38
Airconditioner .0313589 .1744377 574 18
Camera .0596491 .2370436 570 34
EvaluationoftheGirl’sEducationProject3
©EDOREN 276
AnnexL EarlyLearningDescriptiveStatistics
Table61: SchoolLevelDescriptiveStatistics
VariableLabel TotalMean Meanbystate
Katsina Zamfara
HeadTeacherPersonalCharacteristics
Respondentisfemaleefef
Mean 0.02 0.03 0.01
StandardError 0.009 0.016 0.009
N 240 120 120
Age
Mean 42.16 38.05 46.34
StandardError 0.619 0.908 0.84
N 236 119 117
HeadTeacherProfessionalCharacteristics
NumberofyearsinHeadTeacherroleincurrentschool
Mean 6.43 4.19 8.72
StandardError 0.477 0.563 0.775
N 233 118 115
Everworkedasateacher?
Mean 0.96 0.89 1
StandardError 0.036 0.113 0
N 28 9 19
Numberofyearsofteachingexperience
Mean 15.46 10.19 20.81
StandardError 0.606 0.807 0.907
N 234 118 116
DoestheHeadTeacherhaveanNCE?
Mean 0.49 0.53 0.44
StandardError 0.024 0.039 0.027
N 240 120 120
HeadTeacherhasSSCEorhigherdegree
Mean 0.58 0.7 0.47
StandardError 0.031 0.043 0.044
N 232 115 117
HeadTeacherTrainings
HeadTeacherhasattendedtrainingorworkshop
Mean 0.63 0.61 0.66
StandardError 0.031 0.045 0.043
N 237 118 119
MaximumtrainingsandworkshopsattendedbytheHeadTeacher
Mean 1.54 1.4 1.67
StandardError 0.067 0.067 0.114
N 150 72 78
Numberofdaystrained
Mean 9.54 7.11 11.78
StandardError 1.328 0.971 2.392
N 150 72 78
IstheHeadTeacher Mean 0.31 0.21 0.4
EvaluationoftheGirl’sEducationProject3
©EDOREN 277
trainedonmanagementandplanning?
StandardError 0.037 0.047 0.056
N 150 72 78
IstheHeadTeachertrainedonteachinginHausa?
Mean 0.2 0.18 0.22
StandardError 0.032 0.044 0.046
N 150 72 78
IstheHeadTeachertrainedonteachingmethods?
Mean 0.63 0.72 0.55
StandardError 0.038 0.054 0.053
N 150 72 78
IstheHeadTeachertrainedoncommunityinvolvement?
Mean 0.25 0.08 0.41
StandardError 0.031 0.033 0.052
N 150 72 78
IstheHeadTeachertrainedonintegrationactivities?
Mean 0.22 0.19 0.25
StandardError 0.028 0.041 0.039
N 150 72 78
IstheHeadTeachertrainedondevelopinginstructionalmaterials?
Mean 0.09 0.04 0.13
StandardError 0.023 0.023 0.039
N 150 72 78
IstheHeadTeachertrainedonschoolleadership?
Mean 0.32 0.19 0.45
StandardError 0.037 0.047 0.057
N 150 72 78
IstheHeadTeachertrainedonextra-curricular?
Mean 0.08 0.07 0.09
StandardError 0.022 0.03 0.032
N 150 72 78
IstheHeadTeachertrainedoncurriculumsubjects?
Mean 0.3 0.29 0.31
StandardError 0.38 0.055 0.052
N 150 72 78
IstheHeadTeachertrainedonliteracyandnumeracy?
Mean 0.27 0.29 0.26
StandardError 0.037 0.055 0.05
N 150 72 78
HeadTeacherabsenteeism
HowmanydaysistheHeadTeacherabsent?
Mean 7.9 5.96 10.05
StandardError 1.146 0.878 2.211
N 137 72 65
HeadTeacherSchoolLeadership
Observeanylessonfortheentireduration
Mean 0.46 0.34 0.57
StandardError 0.032 0.044 0.046
N 225 112 113
Totalnumberofwritten Mean 0.2 0 0.31
EvaluationoftheGirl’sEducationProject3
©EDOREN 278
observationsavailable StandardError 0.1 0 0.158
N 103 38 65
TakeactiontoimproveHeadTeacherattendance
Mean 0.71 0.67 0.75
StandardError 0.026 0.037 0.038
N 225 112 113
Takeactiontoimprovepupilattendance
Mean 0.97 0.99 0.96
StandardError 0.011 0.009 0.02
N 225 112 113
Receiveanymonitoringvisits
Mean 0.78 0.77 0.79
StandardError 0.023 0.032 0.034
N 240 120 120
Numberofmonitoringvisitsperschool
Mean 8.27 8.47 8.07
StandardError 0.503 0.704 0.719
N 186 92 94
Note:ForeachvariablethetableaboveincludesMeanvalue,StandardErrorandNumberofobservations.
VariableLabel TotalMean Meanbystate
Katsina Zamfara
SchoolStructure
Yearssinceschoolestablishment
Mean 36.34 32.86 39.58
StandardError 1.339 1.673 2.062
N 191 92 99
PublicIsthereaPublicPrimarySchoolnearby?
Mean 0.89 0.85 0.93
StandardError 0.029 0.047 0.033
N 120 60 60
Subjectstaughtinschool-English
Mean 0.87 0.81 0.92
StandardError 0.018 0.03 0.021
N 240 120 120
Subjectstaughtinschool-Mathematics
Mean 0.97 0.95 0.98
StandardError 0.011 0.02 0.012
N 240 120 120
Subjecttaughtinschool-SocialStudies/CivicEducation
Mean 0.57 0.53 0.61
StandardError 0.016 0.016 0.028
N 240 120 120
Subjectstaughtinschool-GeneralScience/BasicScience
Mean 0.54 0.54 0.54
StandardError 0.018 0.019 0.029
N 240 120 120
Subjectstaughtinschool- Mean 0.95 0.93 0.97
EvaluationoftheGirl’sEducationProject3
©EDOREN 279
Hausa StandardError 0.014 0.024 0.015
N 240 120 120
Subjectstaughtinschool-Arabic
Mean 0.65 0.47 0.83
StandardError 0.023 0.03 0.033
N 240 120 120
Schoolteachesall5coreintegratedsubjects
Mean 0.07 0.03 0.1
StandardError 0.023 0.023 0.039
N 120 60 60
Shareofintegratedteachersthatarepaid
Mean 0.82 0.83 0.81
StandardError 0.032 0.045 0.046
N 116 60 60
IntegratedteachingperweekatP2level(hours)
Mean 9.41 10.14 8.72
StandardError 0.227 0.335 0.307
N 211 102 109
DoesthisschoolhaveboardersinP2?
Mean 0.82 0.78 0.85
StandardError 0.035 0.052 0.046
N 120 60 60
TeacherRoster
Numberofteachersofintegratedsubjects-AllLevels
Mean 6.43 4.61 8.24
StandardError 0.469 0.496 0.795
N 240 120 120
Ratiooffemalestototalintegratedsubjectteachers
Mean 0.09 0.09 0.09
StandardError 0.011 0.017 0.014
N 240 120 120
Numberofteachersofintegratedsubjects-P1-P3
Mean 4.31 3.41 5.22
StandardError 0.273 0.311 0.45
N 240 120 120
Numberoffemaleteachersofintegratedsubjects-P1-P3
Mean 1.18 0.92 1.44
StandardError 0.204 0.187 0.358
N 159 78 81
Ratiooffemalestototalintegratedsubjectteachers-P1-P6
Mean 0.12 0.13 0.11
StandardError 0.016 0.023 0.021
N 159 78 81
Teacherturnoverasashareoftotalteachers
Mean 0.21 0.25 0.18
StandardError 0.026 0.033 0.04
N 240 120 120
SchoolInfrastructure
SchoolRepairsMean 0.95 0.95 0.94
StandardError 0.014 0.02 0.02
EvaluationoftheGirl’sEducationProject3
©EDOREN 280
N 239 120 119
Schoolneedsclassrooms
Mean 0.77 0.63 0.82
StandardError 0 0 0
N 40 11 29
Electricity
Mean 0.23 0.18 0.28
StandardError 0.026 0.032 0.04
N 239 120 119
Electricitytoday
Mean 0.05 0.04 0.06
StandardError 0.014 0.017 0.021
N 239 120 119
Schoolhasasourceofdrinkingwater
Mean 0.44 0.36 0.51
StandardError 0.032 0.044 0.046
N 240 120 120
Availabilityofwaterfromsourcetoday
Mean 0.7 0.67 0.72
StandardError 0.045 0.069 0.059
N 104 43 61
Numberofrooms
Mean 5.93 5.78 6.07
StandardError 0.307 0.477 0.385
N 239 120 119
Numberofroomsusedforclassestoday
Mean 3.73 3.46 4
StandardError 0.188 0.255 0.277
N 239 120 119
Numberoffunctioningtoilets
Mean 1.7 2.03 1.36
StandardError 0.194 0.341 0.181
N 239 120 119
Schoolhasfunctionaltoiletsforgirls
Mean 0.26 0.23 0.3
StandardError 0.025 0.034 0.038
N 239 120 119
Fenceorboundarywall
Mean 0.12 0.09 0.15
StandardError 0.021 0.025 0.033
N 239 120 119
Collectionofbooks
Mean 0.11 0.14 0.09
StandardError 0.02 0.03 0.027
N 239 120 119
Playgroundorsportsarea
Mean 0.55 0.53 0.58
StandardError 0.021 0.023 0.036
N 239 120 119
GirlFriendliness
RatioofgirlstoboysinP1-P3
Mean 0.73 0.82 0.64
StandardError 0.03 0.055 0.025
EvaluationoftheGirl’sEducationProject3
©EDOREN 281
N 111 56 55
Ratioofgirlstoboysintotal
Mean 0.71 0.81 0.6
StandardError 0.031 0.055 0.028
N 111 56 55
RatioofgirlstoboyspresentonthedayofthevisitinP2
Mean 1.8 1.63 1.96
StandardError 0.106 0.161 0.139
N 220 107 113
Ratiooffemalestototalintegratedsubjectteachers
Mean 0.09 0.09 0.09
StandardError 0.011 0.017 0.014
N 240 120 120
Ratiooffemalestototalintegratedsubjectteachers-P1-P3
Mean 0.12 0.13 0.11
StandardError 0.016 0.023 0.021
N 159 78 81
Schoolhasseparatefunctionaltoiletsforgirls
Mean 0.26 0.23 0.3
StandardError 0.025 0.034 0.038
N 239 120 119
Committeesorgroupswithinschool
Mean 0.35 0.2 0.5
StandardError 0.028 0.036 0.043
N 240 120 120
SchoolRecords
AttendancerecordexistsforP2forcurrentyear
Mean 0.78 0.84 0.71
StandardError 0.025 0.031 0.038
N 240 120 120
UpdatedattendancerecordexistsforP2forcurrentyear
Mean 0.25 0.25 0.24
StandardError 0.027 0.039 0.036
N 240 120 120
Updatedattendancerecordforlastyearavailable
Mean 0.24 0.23 0.25
StandardError 0.026 0.038 0.035
N 240 120 120
EnrolmentrecordexistsforP2currentyear
Mean 0.48 0.47 0.5
StandardError 0.026 0.038 0.037
N 240 120 120
Updatedenrolmentrecordforcurrentyearavailable
Mean 0.5 0.51 0.49
StandardError 0.028 0.042 0.036
N 240 120 120
Updatedenrolmentrecordforlastyearavailable
Mean 0.49 0.48 0.5
StandardError 0.025 0.035 0.037
N 240 120 120
EvaluationoftheGirl’sEducationProject3
©EDOREN 282
Table62: TeacherLevelDescriptiveStatistics
Writtenrecordsofmeetingskept
Mean 0.25 2.3 0.28
StandardError 0.028 0.042 0.038
N 200 93 107
Numberofmeetingrecordsavailable
Mean 1.84 2.34 1.48
StandardError 0 0 0
N 50 21 29
Teacherattendancerecordexists
Mean 0.6 0.51 0.69
StandardError 0.017 0.017 0.029
N 240 120 120
Updatedteacherattendancerecordexists
Mean 0.48 0.44 0.52
StandardError 0.02 0.025 0.032
N 240 120 120
Nowrittenrecordsprovided
Mean 0.81 0.85 0.77
StandardError 0.022 0.028 0.033
N 240 120 120
TimetableprovidedforP2orequivalent
Mean 0.49 0.45 0.53
StandardError 0.02 0.026 0.031
N 240 120 120
Note:ForeachvariablethetableaboveincludesMeanvalue,StandardErrorandNumberofobservations.
VariableLabel TotalMean
Meanbystate
Katsina Zamfara
TeacherPersonalCharacteristics
Respondentisfemale
Mean 0.15 0.17 0.13
StandardError 0.023 0.03 0.033
N 477 221 256
Age
Mean 36.08 35.08 36.74
StandardError 0.432 0.625 0.593
N 471 219 252
SpeaksYoruba
Mean 0.02 0.02 0.03
StandardError 0.01 0.011 0.013
N 356 139 217
SpeaksHausa?
Mean 1 1 1
StandardError 0 0 0
N 356 139 217
SpeaksEnglish?Mean 0.85 0.86 0.85
StandardError 0.022 0.039 0.027
EvaluationoftheGirl’sEducationProject3
©EDOREN 283
N 356 139 217
SpeaksArabic?
Mean 0.22 0.21 0.23
StandardError 0.024 0.04 0.031
N 356 139 217
ProfessionalCharacteristics
TeacheriscurrentlyteachingP1-P3equivalent
Mean 1 1 1
StandardError 0 0 0
N 357 140 217
TeacheriscurrentlyteachingP3-P6orequivalentlevel
Mean 0.3 0.55 0.17
StandardError 0.029 0.055 0.032
N 357 140 217
Teacheriscurrentlyteachingatupperandlowerlevels
Mean 0.3 0.55 0.17
StandardError 0.029 0.055 0.032
N 357 140 217
DoestheteacherhaveanNCE?
Mean 0.49 0.64 0.48
StandardError 0.024 0.035 0.03
N 240 221 256
Doestheteacherhaveprofessionaleducationalqualifications?
Mean 0.7 0.82 0.62
StandardError 0.021 0.025 0.03
N 477 221 256
DoestheteacherhaveGrade2orequivalentqualification?
Mean 0.07 0.05 0.09
StandardError 0.014 0.017 0.02
N 477 221 256
DoestheteacherhaveanNCE?
Mean 0.54 0.64 0.48
StandardError 0.023 0.035 0.03
N 477 221 256
Highestacademicqualificationoftheteacherisprimaryeducation
Mean 0.1 0.09 0.11
StandardError 0.014 0.019 0.019
N 477 221 256
HighestacademicqualificationoftheteacherisJSS
Mean 0.04 0.05 0.03
StandardError 0.012 0.014 0.018
N 477 221 256
HighestacademicqualificationoftheteacherisSSCE
Mean 0.72 0.75 0.7
StandardError 0.23 0.032 0.032
N 477 221 256
HighestacademicqualificationoftheteacherisOND
Mean 0.08 0.05 0.09
StandardError 0.014 0.017 0.021
N 477 221 256
HighestacademicqualificationisBA/BSc
Mean 0.02 0.02 0.03
StandardError 0.008 0.012 0.012
EvaluationoftheGirl’sEducationProject3
©EDOREN 284
N 477 221 256
Doestheteacherhavereligiouseducationqualification?
Mean 0.19 0.16 0.21
StandardError 0.021 0.026 0.031
N 477 221 256
Totalexperienceasateacherinthisschooloranyotherschool
Mean 10.03 8.63 10.96
StandardError 0.385 0.581 0.515
N 476 221 255
Totalyearsasateacherincurrentschool
Mean 6.2 6.63 5.96
StandardError 0.324 0.577 0.389
N 406 167 239
Teacherhasatleast2yearsofteachingexperienceinanyschool
Mean 0.89 0.81 0.95
StandardError 0.014 0.026 0.015
N 477 221 256
Teacherhasatleast2yearsofteachingexperienceinthecurrentschool
Mean 0.71 0.75 0.69
StandardError 0.026 0.036 0.035
N 406 167 239
TeacherTrainings
Hastheteacherattendedtrainingduringlast2years?
Mean 0.42 0.44 0.41
StandardError 0.025 0.032 0.036
N 473 218 255
TotalnumberoftrainingsattendedbytheTeacher
Mean 1.31 1.25 1.35
StandardError 0.047 0.05 0.072
N 216 101 115
Numberofdaystrained
Mean 8.9 8.61 9.11
StandardError 1.014 1.281 1.472
N 216 101 115
HastheteacherattendedtrainingorganisedbySUBEB?
Mean 0.17 0.16 0.17
StandardError 0.031 0.04 0.044
N 216 101 115
HastheteacherattendedtrainingorganisedbyGEP/UNICEF?
Mean 0.47 0.41 0.51
StandardError 0.044 0.058 0.062
N 216 101 115
HastheteacherattendedtrainingorganisedbyLGEA
Mean 0.09 0.11 0.08
StandardError 0.022 0.036 0.027
N 216 101 115
IstheteachertrainedonteachinginHausa?
Mean 0.19 0.12 0.24
StandardError 0.026 0.03 0.039
N 216 101 115
Istheteachertrainedon Mean 0.77 0.77 0.77
EvaluationoftheGirl’sEducationProject3
©EDOREN 285
teachingmethods? StandardError 0.035 0.047 0.05
N 216 101 115
Istheteachertrainedoncommunityinvolvement?
Mean 0.11 0.08 0.13
StandardError 0.021 0.028 0.03
N 216 101 115
Istheteachertrainedonintegrationactivities?
Mean 0.11 0.11 0.12
StandardError 0.019 0.028 0.026
N 216 101 115
Istheteachertrainedondevelopinginstructionalmaterials?
Mean 0.11 0.09 0.12
StandardError 0.024 0.035 0.032
N 216 101 115
Istheteachertrainedonmanagementandplanning?
Mean 0.15 0.15 0.16
StandardError 0.027 0.037 0.038
N 216 101 115
Istheteachertrainedonschoolleadership?
Mean 0.15 0.14 0.17
StandardError 0.027 0.033 0.039
N 216 101 115
Istheteachertrainedonextra-curricular?
Mean 0.07 0.08 0.07
StandardError 0.019 0.025 0.028
N 216 101 115
Istheteachertrainedoncurriculumsubjects?
Mean 0.21 0.31 0.14
StandardError 0.029 0.049 0.035
N 216 101 115
Istheteachertrainedonliteracyandnumeracy?
Mean 0.34 0.24 0.4
StandardError 0.04 0.052 0.057
N 216 101 115
Teacherabsenteeism
Teacherisabsentatleastonceduringlast3months
Mean 0.63 0.57 0.67
StandardError 0.025 0.039 0.032
N 450 209 241
Numberofdaysteacherisabsentduringthelast3months
Mean 7.73 6.21 8.58
StandardError 0.719 1.041 0.966
N 294 127 167
TeacherMotivation
Teachermotivationcompositeindex
Mean N/A 2.98 2.99
StandardError N/A 0.022 0.02
N N/A 220 254
TeacherKnowledge
EvaluationoftheGirl’sEducationProject3
©EDOREN 286
Table63: PupilLevelDescriptiveStatistics
VariableLabel TotalMean Meanbystate
Katsina Zamfara
PupilCharacteristics
Femalepupils
Mean 0.48 0.48 0.47
StandardError 0.008 0.013 0.009
N 2651 1294 1357
Age
Mean 9.14 9.3 8.96
StandardError 0.147 0.224 0.181
N 1677 871 806
PupilspeaksEnglishathome
Mean 0 0 0
StandardError 0.001 0 0.002
N 2623 1279 1344
PupilspeaksHausaathome
Mean 0.99 0.99 0.99
StandardError 0.003 0.004 0.003
N 2623 1279 1344
CurrentlyattendingP2orequivalentlevel
Mean 0.98 0.97 0.99
StandardError 0.008 0.014 0.01
N 2651 1294 1357
Attendingotherschoolbesidesthisone
Mean 0.54 0.46 0.61
StandardError 0.024 0.037 0.028
Teachersubjectknowledgecompositeindex
Mean 0.49 0.51 0.48
StandardError 0.022 0.034 0.028
N 463 211 252
Teacherpedagogicalknowledgecompositeindex
Mean 0.05 0.06 0.05
StandardError 0.012 0.013 0.019
N 463 211 252
Mother-Tongueteaching(Hausa)inearlygradesindex
Mean 38.33 39.77 37.4
StandardError 0.815 0.938 1.164
N 459 209 250
Hausateaching/learningmaterialscompositeindex
Mean 0.02 0.03 0.02
StandardError 0.005 0.008 0.006
N 469 217 252
Syllabus/Curriculumknowledge
Mean 0.89 0.77 0.97
StandardError 0.056 0.086 0.072
N 463 211 252
Note:ForeachvariablethetableaboveincludesMeanvalue,StandardErrorandNumberofobservations.
EvaluationoftheGirl’sEducationProject3
©EDOREN 287
N 2651 1294 1357
Pupilisaboarder
Mean 0.09 0.09 0.1
StandardError 0.012 0.015 0.018
N 2651 1294 1357
Facesdifficultyingettingtoschool
Mean 0.03 0.03 0.02
StandardError 0.006 0.01 0.006
N 2317 1119 1198
PupilAssessmentScaledScores
Hausaliteracyscaledscore
Mean 504.03 506.67 501.47
StandardError 4.373 6.785 5.565
N 2649 1293 1356
Englishliteracyscaledscore
Mean 352.95 353.27 352.64
StandardError 3.534 5.527 4.45
N 2649 1293 1356
Note:ForeachvariablethetableaboveincludesMeanvalue,StandardErrorandNumberofobservations.
EvaluationoftheGirl’sEducationProject3
©EDOREN 288
AnnexM IQSchoolSupport(IQSS)DescriptiveStatistics
Table64: SchoolLevelDescriptiveStatistics
VariableLabel TotalMean Meanbystate
Bauchi Niger
HeadTeacherPersonalCharacteristics
Respondentisfemale
Mean 0.03 0 0.07
StandardError 0.024 0 0.048
N 60 30 30
Age
Mean 40 44.07 35.63
StandardError 1.225 1.73 1.733
N 56 29 27
HeadTeacherProfessionalCharacteristics
NumberofyearsinHeadTeacherroleincurrentschool
Mean 9.61 14.48 4.8
StandardError 1.25 2.35 0.939
N 60 29 30
Everworkedasateacher?
Mean 0.89 1 0.76
StandardError 0 0 0
N 9 5 4
Numberofyearsofteachingexperience
Mean 13.58 17.41 9.62
StandardError 1.048 1.566 1.386
N 59 30 29
DoestheHeadTeacherhaveanNCE?
Mean 0.23 0.27 0.2
StandardError 0.053 0.079 0.07
N 60 30 30
HeadTeacherhasSSCEorhigherdegree
Mean 0.4 0.23 0.57
StandardError 0.061 0.082 0.091
N 60 30 30
HeadTeacherhasprofessionaleducation
Mean 0.33 0.4 0.27
StandardError 0.056 0.088 0.07
N 60 30 30
HeadTeacherhasreligiouseducation
Mean 0.38 0.56 0.2
StandardError 0.06 0.091 0.078
N 60 30 30
HeadTeacherTrainings
HeadTeacherhasattendedtrainingorworkshop
Mean 0.43 0.37 0.5
StandardError 0.067 0.088 0.1
N 60 30 30
EvaluationoftheGirl’sEducationProject3
©EDOREN 289
MaximumtrainingsandworkshopsattendedbytheHeadTeacher
Mean 1.34 1.45 1.26
StandardError 0 0 0
N 26 11 15
Numberofdaystrained
Mean 4.72 5.28 4.31
StandardError 0 0 0
N 26 11 15
IstheHeadTeachertrainedonmanagementandplanning?
Mean 0.27 0.36 0.2
StandardError 0 0 0
N 26 11 15
IstheHeadTeachertrainedonteachinginHausa?
Mean 0.27 0.46 0.13
StandardError 0 0 0
N 26 11 15
IstheHeadTeachertrainedonteachingmethods?
Mean 0.85 0.82 0.87
StandardError 0 0 0
N 26 11 15
IstheHeadTeachertrainedoncommunityinvolvement?
Mean 0.16 0.18 0.14
StandardError 0 0 0
N 26 11 15
IstheHeadTeachertrainedonintegrationactivities?
Mean 0.31 0.36 0.27
StandardError 0 0 0
N 26 11 15
IstheHeadTeachertrainedondevelopinginstructionalmaterials?
Mean 0.04 0 0.07
StandardError 0 0 0
N 26 11 15
IstheHeadTeachertrainedonschoolleadership?
Mean 0.31 0.63 0.07
StandardError 0 0 0
N 26 11 15
IstheHeadTeachertrainedonextra-curricular?
Mean 0.04 0 0.07
StandardError 0 0 0
N 26 11 15
IstheHeadTeachertrainedoncurriculumsubjects?
Mean 0.11 0.09 0.13
StandardError 0 0 0
N 26 11 15
IstheHeadTeachertrainedonliteracyandnumeracy?
Mean 0.31 0.36 0.26
StandardError 0 0 0
N 26 11 15
HeadTeacherabsenteeism
EvaluationoftheGirl’sEducationProject3
©EDOREN 290
HowmanydaysistheHeadTeacherabsent?
Mean 5.87 5.88 5.86
StandardError 0.825 0.663 1.791
N 47 28 19
HeadTeacherSchoolLeadership
Observeanylessonfortheentireduration
Mean 0.42 0.3 0.56
StandardError 0.065 0.078 0.105
N 57 30 27
Totalnumberofwrittenobservationsavailable
Mean 0.04 0.11 0
StandardError 0 0 0
N 24 9 15
TakeactiontoimproveHeadTeacherattendance
Mean 0.61 0.63 0.59
StandardError 0.066 0.085 0.101
N 57 30 27
Takeactiontoimprovepupilattendance
Mean 0.91 0.93 0.89
StandardError 0.037 0.04 0.064
N 57 30 27
Receiveanymonitoringvisits
Mean 0.38 0.44 0.33
StandardError 0.062 0.091 0.085
N 60 30 30
Numberofmonitoringvisitsperschool
Mean 3.75 4.24 3.1
StandardError 0 0 0
N 23 13 10
Whocameformonitoringvisits-LGEAOfficer?
Mean 0.7 0.69 0.7
StandardError 0 0 0
N 23 13 10
Whocameformonitoringvisits-SAMEOfficer?
Mean 0.09 0 0.2
StandardError 0 0 0
N 23 13 10
Whocameformonitoringvisits-SUBEBOfficer?
Mean 0.13 0.08 0.2
StandardError 0 0 0
N 23 13 10
Whocameformonitoringvisits-MinistryofEducationOfficer?
Mean 0.04 0.08 0
StandardError 0 0 0
N 23 13 10
Whocameformonitoringvisits-UBECOfficer?
Mean 0.04 0 0.1
StandardError 0 0 0
N 23 13 10
EvaluationoftheGirl’sEducationProject3
©EDOREN 291
Whocameformonitoringvisits-TrainersfromSBMC/CBMCtraining?
Mean 0.04 0 0.1
StandardError 0 0 0
N 23 13 10
Whocameformonitoringvisits-RepresentativesfromNGOs,donorsorotherexternals?
Mean 0.17 0.23 0.1
StandardError 0 0 0
N 23 13 10
Whocameformonitoringvisits-Others(Specify)
Mean 0.26 0.38 0.1
StandardError 0 0 0
N 23 13 10
Whocameformonitoringvisits-Don’tknow?
Mean 0.13 0.15 0.1
StandardError 0 0 0
N 23 13 10
Whywasactionnottaken?–Teacherattendanceisnotaproblem
Mean 0.23 0.37 0.1
StandardError 0.05 0.085 0.053
N 60 30 30
Reasonsforteacher’sabsence-Transport
Mean 0.05 0.1 0
StandardError 0.029 0.058 0
N 60 30 30
ReasonsforTeacher’sAbsence-OwnIllness
Mean 0.47 0.6 0.33
StandardError 0.061 0.085 0.088
N 60 30 30
Reasonsforteacher’sabsence-Illnessoffamilymembers
Mean 0.25 0.27 0.23
StandardError 0.056 0.081 0.078
N 60 30 30
Reasonsforteacher’sabsence-Lateornon-paymentofsalary
Mean 0.15 0.07 0.23
StandardError 0.046 0.047 0.078
N 60 30 30
Reasonsforteacher’sabsence-salarycollection
Mean 0.02 0 0.03
StandardError 0.016 0 0.032
N 60 30 30
Socialorreligiousobligations
Mean 0.17 0.17 0.17
StandardError 0.051 0.17 0.071
N 60 30 30
Note:ForeachvariablethetableaboveincludesMeanvalue,StandardErrorandNumberofobservations.
EvaluationoftheGirl’sEducationProject3
©EDOREN 292
VariableLabel TotalMean Meanbystate
Bauchi Niger
SchoolStructure
Yearssinceschoolestablishment
Mean 26.65 34.81 18.15
StandardError 2.437 3.555 3.321
N 53 27 26
PublicPrimarySchool?
Mean 0.98 0.97 1
StandardError 0.017 0.033 0
N 60 30 30
Subjectstaughtinschool-English
Mean 0.92 0.97 0.87
StandardError 0.035 0.034 0.062
N 60 30 30
Subjectstaughtinschool-Mathematics
Mean 0.92 0.97 0.87
StandardError 0.035 0.034 0.062
N 60 30 30
Subjecttaughtinschool-SocialStudies/CivicEducation
Mean 0.23 0.17 0.3
StandardError 0.055 0.071 0.085
N 60 30 30
Subjectstaughtinschool-GeneralScience/BasicScience
Mean 0.2 0.2 0.2
StandardError 0.05 0.071 0.07
N60
30 30
Subjectstaughtinschool-Hausa
Mean 0.73 0.7 0.77
StandardError 0.057 0.081 0.078
N 60 30 30
Subjectstaughtinschool-Arabic
Mean 0.6 0.6 0.6
StandardError 0.066 0.097 0.088
N 60 30 30
Schoolteachesall5coreintegratedsubjects
Mean 0.12 0.03 0.2
StandardError 0.039 0.034 0.07
N 60 30 30
Shareofintegratedteachersthatarepaid
Mean 0.18 0.03 0.33
StandardError 0.047 0.033 0.088
N 60 30 30
IntegratedteachingperweekatP2level(hours)
Mean 2.98 2.2 3.81
StandardError 0.378 0.34 0.693
N 58 30 30
DoesthisschoolhaveboardersinP2?
Mean 0.35 0.43 0.27
StandardError 0.062 0.091 0.082
EvaluationoftheGirl’sEducationProject3
©EDOREN 293
N 60 30 30
TeacherRoster
Numberofteachersofintegratedsubjects-AllLevels
Mean 2.78 2.39 3.18
StandardError 0.23 0.334 0.318
N 60 30 30
IQSonlyhasoneteacherofintegratedsubjects
Mean 0.25 0.33 0.16
StandardError 0.054 0.088 0.061
N 60 30 30
Ratiooffemalestototalintegratedsubjectteachers
Mean 0.9 0.03 0.16
StandardError 0.027 0.02 0.05
N 60 30 30
Numberofteachersofintegratedsubjects-P1-P3
Mean 2.22 1.93 2.5
StandardError 0.138 0.182 0.207
N 60 30 30
Numberoffemaleteachersofintegratedsubjects-P1-P3
Mean 0.13 0 0.27
StandardError 0 0 0
N 23 12 11
Ratiooffemalestototalintegratedsubjectteachers-P1-P6
Mean 0.03 0 0.07
StandardError 0 0 0
N 23 12 11
Teacherturnoverasashareoftotalteachers
Mean 0.23 0.24 0.22
StandardError 0.064 0.079 0.1
N 60 30 30
SchoolInfrastructure
SchoolRepairs
Mean 0.9 1 0.8
StandardError 0.033 0 0.067
N 60 30 30
Schoolneedsclassrooms
Mean 0.79 0.7 1
StandardError 0 0 0
N 14 10 4
Electricity
Mean 0.4 0.3 0.5
StandardError 0.058 0.079 0.085
N 60 30 30
Electricitytoday
Mean 0.28 0.3 0.27
StandardError 0.056 0.079 0.078
N 60 30 30
Schoolhasasourceofdrinkingwater
Mean 0.3 0.3 0.3
StandardError 0.056 0.091 0.064
N 60 30 30
EvaluationoftheGirl’sEducationProject3
©EDOREN 294
Availabilityofwaterfromsourcetoday
Mean 0.89 0.89 0.89
StandardError 0 0 0
N 18 9 9
Numberofrooms
Mean 2.67 2.73 2.61
StandardError 0.264 0.407 0.335
N 59 30 29
Numberofroomsusedforclassestoday
Mean 2.08 2.1 2.05
StandardError 0.259 0.402 0.325
N 60 30 30
Numberoffunctioningtoilets
Mean 0.35 0.3 0.4
StandardError 0.097 0.115 0.156
N 60 30 30
Schoolhasfunctionaltoiletsforgirls
Mean 0.08 0.1 0.07
StandardError 0.035 0.052 0.047
N 60 30 30
Fenceorboundarywall
Mean 0.22 0.13 0.3
StandardError 0.053 0.067 0.081
N 60 30 30
Collectionofbooks
Mean 0.03 0.03 0.03
StandardError 0.023 0.033 0.032
N 60 30 30
Playgroundorsportsarea
Mean 0.36 0.53 0.2
StandardError 0.059 0.097 0.066
N 60 30 30
GirlFriendliness
RatioofgirlstoboysinP1-P3
Mean 1.22 1.55 0.85
StandardError 0 0 0
N 19 10 9
Ratioofgirlstoboysintotal
Mean 1.31 1.64 0.95
StandardError 0 0 0
N 19 10 9
RatioofgirlstoboyspresentonthedayofthevisitinP2
Mean 1.15 1.21 1.09
StandardError 0.093 0.15 0.11
N 58 29 29
Ratiooffemalestototalintegratedsubjectteachers
Mean 0.09 0.03 0.16
StandardError 0.027 0.02 0.05
N 60 30 30
Ratiooffemalestototalintegratedsubject
Mean 0.03 0 0.07
StandardError 0 0 0
EvaluationoftheGirl’sEducationProject3
©EDOREN 295
teachers-P1-P3 N 23 12 11
Schoolhasseparatefunctionaltoiletsforgirls
Mean 0.08 0.1 0.07
StandardError 0.035 0.052 0.047
N 60 30 30
Committeesorgroupswithinschool
Mean 0.03 0 0.06
StandardError 0.02 0 0.039
N 60 30 30
SchoolRecords
AttendancerecordexistsforP2forcurrentyear
Mean 0.33 0.43 0.23
StandardError 0.054 0.085 0.066
N 60 30 30
UpdatedattendancerecordexistsforP2forcurrentyear
Mean 0.07 0.1 0.03
StandardError 0.031 0.053 0.032
N 60 30 30
Updatedattendancerecordforlastyearavailable
Mean 0.05 0.07 0.03
StandardError 0.029 0.047 0.034
N 60 30 30
EnrolmentrecordexistsforP2currentyear
Mean 0.32 0.33 0.3
StandardError 0.055 0.075 0.081
N 60 30 30
Updatedenrolmentrecordforcurrentyearavailable
Mean 0.32 0.33 0.3
StandardError 0.055 0.078 0.078
N 60 30 30
Updatedenrolmentrecordforlastyearavailable
Mean 0.17 0.23 0.1
StandardError 0.049 0.078 0.057
N 60 30 30
Writtenrecordsofmeetingskept
Mean 0.06 0.08 0.04
StandardError 0.035 0.056 0.041
N 49 25 24
Numberofmeetingrecordsavailable
Mean 1.33 2 0
StandardError 0 0 0
N 3 2 1
Teacherattendancerecordexists
Mean 0.12 0.1 0.13
StandardError 0.042 0.058 0.062
N 60 30 30
Updatedteacherattendancerecordexists
Mean 0.08 0.07 0.1
StandardError 0.035 0.047 0.052
N 60 30 30
EvaluationoftheGirl’sEducationProject3
©EDOREN 296
Nowrittenrecordsprovided
Mean 0.38 0.4 0.37
StandardError 0.06 0.088 0.081
N 60 30 30
TimetableprovidedforP2orequivalent
Mean 0.22 0.23 0.2
StandardError 0.047 0.067 0.066
N 60 30 30
Note:ForeachvariablethetableaboveincludesMeanvalue,StandardErrorandNumberofobservations.
Table65: TeacherLevelDescriptiveStatistics
VariableLabel TotalMean Meanbystate
Bauchi Niger
TeacherPersonalCharacteristics
Respondentisfemale
Mean 0.09 0.02 0.16
StandardError 0.037 0.02 0.067
N 96 46 50
Age
Mean 31.62 33.64 29.87
StandardError 0.865 1.344 1.114
N 95 45 50
SpeaksYoruba?
Mean 0.02 0 0.03
StandardError 0.018 0 0.034
N 76 38 38
SpeaksNupe?
Mean 0.2 0 0.39
StandardError 0.033 0 0.057
N 76 38 38
SpeaksIgbo?
Mean 0.01 0.03 0
StandardError 0.012 0.026 0
N 76 38 38
SpeaksHausa?
Mean 0.96 1 0.93
StandardError 0.024 0 0.044
N 76 38 38
SpeaksFulfude?
Mean 0.07 0.13 0.02
StandardError 0.027 0.049 0.024
N 76 38 38
SpeaksEnglish?
Mean 0.9 0.9 0.91
StandardError 0.032 0.047 0.042
N 76 38 38
SpeaksEbira?
Mean 0.02 0 0.03
StandardError 0.018 0 0.034
N 76 38 38
EvaluationoftheGirl’sEducationProject3
©EDOREN 297
SpeaksArabic?
Mean 0.17 0.21 0.13
StandardError 0.048 0.079 0.055
N 76 38 38
ProfessionalCharacteristics
TeacheriscurrentlyteachingP1-P3equivalent
Mean 1 1 1
StandardError 0 0 0
N 76 38 38
TeacheriscurrentlyteachingP3-P6orequivalentlevel
Mean 0.12 0.15 0.09
StandardError 0.043 0.069 0.054
N 76 38 38
Teacheriscurrentlyteachingatupperandlowerlevels
Mean 0.12 0.15 0.09
StandardError 0.043 0.069 0.054
N 76 38 38
DoestheteacherhaveanNCE?
Mean 0.35 0.51 0.2
StandardError 0.042 0.07 0.052
N 96 46 50
SubjectsTaught-Mathematics
Mean 0.65 0.65 0.65
StandardError 0.042 0.064 0.056
N 76 38 38
SubjectsTaught-English
Mean 0.71 0.69 0.74
StandardError 0.049 0.064 0.073
N 76 38 38
SubjectsTaught-Hausa
Mean 0.32 0.36 0.28
StandardError 0.051 0.057 0.082
N 76 38 38
Doestheteacherhaveprofessionaleducationalqualifications?
Mean 0.45 0.59 0.33
StandardError 0.043 0.068 0.056
N 96 46 50
DoestheteacherhaveGrade2orequivalentqualification?
Mean 0.08 0.06 0.09
StandardError 0.033 0.044 0.049
N 96 46 50
Highestacademicqualificationoftheteacherisprimaryeducation
Mean 0.12 0.15 0.09
StandardError 0.032 0.048 0.044
N 96 46 50
HighestacademicqualificationoftheteacherisJSS
Mean 0.04 0.05 0.04
StandardError 0.02 0.036 0.027
N 96 46 50
EvaluationoftheGirl’sEducationProject3
©EDOREN 298
HighestacademicqualificationoftheteacherisSSCE
Mean 0.66 0.61 0.7
StandardError 0.051 0.081 0.064
N 96 46 50
HighestacademicqualificationoftheteacherisOND
Mean 0.16 0.15 0.16
StandardError 0.039 0.053 0.057
N 96 46 50
HighestacademicqualificationisBA/BSc
Mean 0.02 0.04 0
StandardError 0.019 0.039 0
N 96 46 50
Doestheteacherhavereligiouseducationqualification?
Mean 0.21 0.33 0.11
StandardError 0.046 0.08 0.048
N 96 46 50
Totalexperienceasateacherinthisschooloranyotherschool
Mean 6.81 8.79 5.09
StandardError 0.748 1.136 0.953
N 95 45 50
Totalyearsasateacherincurrentschool
Mean 3.39 4.42 2.33
StandardError 0.435 0.738 0.404
N 79 41 38
Teacherhasatleast2yearsofteachingexperienceinanyschool
Mean 0.82 0.87 0.77
StandardError 0.042 0.048 0.068
N 96 46 50
Teacherhasatleast2yearsofteachingexperienceinthecurrentschool
Mean 0.63 0.7 0.56
StandardError 0.064 0.077 0.1
N 79 41 38
TeacherTrainings
Hastheteacherattendedtrainingduringlast2years?
Mean 0.43 0.36 0.49
StandardError 0.046 0.06 0.068
N 96 46 50
TotalnumberoftrainingsattendedbytheTeacher
Mean 1.29 1.3 1.29
StandardError 0 0 0
N 40 16 24
Numberofdaystrained
Mean 6.46 7.63 5.73
StandardError 0 0 0
N 40 16 24
HastheteacherattendedtrainingorganisedbySUBEB?
Mean 0.11 0.24 0.04
StandardError 0 0 0
N 40 16 24
Hastheteacherattendedtrainingorganisedby
Mean 0.71 0.77 0.67
StandardError 0 0 0
EvaluationoftheGirl’sEducationProject3
©EDOREN 299
GEP/UNICEF? N 40 16 24
HastheteacherattendedtrainingorganisedbyLGEA
Mean 0.19 0.18 0.2
StandardError 0 0 0
N 40 16 24
IstheteachertrainedonteachinginHausa?
Mean 0.25 0.53 0.07
StandardError 0 0 0
N 40 16 24
Istheteachertrainedonteachingmethods?
Mean 0.78 0.77 0.78
StandardError 0 0 0
N 40 16 24
Istheteachertrainedoncommunityinvolvement?
Mean 0.16 0.12 0.18
StandardError 0 0 0
N 40 16 24
Istheteachertrainedonintegrationactivities?
Mean 0.38 0.59 0.25
StandardError 0 0 0
N 40 16 24
Istheteachertrainedondevelopinginstructionalmaterials?
Mean 0.12 0 0.2
StandardError 0 0 0
N 40 16 24
Istheteachertrainedonmanagementandplanning?
Mean 0.05 0.12 0
StandardError 0 0 0
N 40 16 24
Istheteachertrainedonschoolleadership?
Mean 0.18 0.47 0
StandardError 0 0 0
N 40 16 24
Istheteachertrainedonextra-curricular?
Mean 0.04 0 0.7
StandardError 0 0 0
N 40 16 24
Istheteachertrainedoncurriculumsubjects?
Mean 0.24 0.18 0.27
StandardError 0 0 0
N 40 16 24
Istheteachertrainedonliteracyandnumeracy?
Mean 0.18 0.18 0.18
StandardError 0 0 0
N 40 16 24
Teacherabsenteeism
Teacherisabsentatleastonceduringlast3months
Mean 0.76 0.9 0.64
StandardError 0.058 0.046 0.096
EvaluationoftheGirl’sEducationProject3
©EDOREN 300
N 93 45 48
Numberofdaysteacherisabsentduringthelast3months
Mean 6.77 7.91 5.33
StandardError 0.655 0.993 0.924
N 72 40 32
TeacherMotivation
Teachermotivationcompositeindex
Mean N/A 3 2.93
StandardError N/A 0.043 0.05
N N/A 46 50
TeacherKnowledge
Teachersubjectknowledgecompositeindex
Mean 0.44 0.54 0.35
StandardError 0.041 0.074 0.036
N 96 46 50
Teacherpedagogicalknowledgecompositeindex
Mean 0.06 0.09 0.03
StandardError 0.021 0.035 0.026
N 96 46 50
Mother-Tongueteaching(Hausa)inearlygradesindex
Mean 31.2 41.2 22.32
StandardError 1.569 1.497 2.339
N 96 46 50
Hausateaching/learningmaterialscompositeindex
Mean 0.03 0.01 0.04
StandardError 0.012 0.01 0.021
N 96 46 50
Syllabus/Curriculumknowledge
Mean 0.81 0.87 0.76
StandardError 0.107 0.167 0.134
N 96 46 50
Note:ForeachvariablethetableaboveincludesMeanvalue,StandardErrorandNumberofobservations.
Table66: PupilLevelDescriptiveStatistics
VariableLabel TotalMean Meanbystate
Bauchi Niger
PupilCharacteristics
Femalepupils
Mean 0.48 0.44 0.52
StandardError 0.027 0.037 0.038
N 575 296 279
Age
Mean 8.33 8.52 8
StandardError 0.173 0.228 0.262
N 387 260 127
PupilspeaksEnglishathome
Mean 0 0 0
StandardError 0.001 0.002 0
EvaluationoftheGirl’sEducationProject3
©EDOREN 301
N 573 295 278
PupilspeaksFulfuldeathome
Mean 0.02 0.03 0.01
StandardError 0.007 0.013 0.009
N 573 295 278
PupilspeaksHausaathome
Mean 0.73 0.93 0.54
StandardError 0.045 0.036 0.063
N 573 295 278
PupilspeaksKanuriathome
Mean 0.01 0.03 0
StandardError 0.013 0.027 0
N 573 295 278
PupilspeaksNupeathome
Mean 0.22 0 0.43
StandardError 0.044 0 0.066
N 573 295 278
PeoplespeaksOtherathome
Mean 0.01 0.01 0.02
StandardError 0.007 0.007 0.011
N 573 295 278
CurrentlyattendingP2orequivalentlevel
Mean 0.89 0.96 0.81
StandardError 0.032 0.018 0.061
N 574 296 278
Attendingotherschoolbesidesthisone
Mean 0.41 0.72 0.13
StandardError 0.05 0.071 0.038
N 574 296 278
Typeofotherschoolbeingattended
Mean 0.88 0.94 0.55
StandardError 0 0 0
N 256 215 41
Pupilisaboarder
Mean 0.06 0.1 0.03
StandardError 0.023 0.044 0.014
N 574 296 278
Facesdifficultyingettingtoschool
Mean 0.03 0.03 0.02
StandardError 0.009 0.013 0.013
N 539 273 266
PupilAssessmentScaledScores
Hausaliteracyscaledscore
Mean 487.61 518.3 458.4
StandardError 8.256 10.625 10.877
N 575 296 279
Englishliteracyscaledscore
Mean 362.46 371.97 353.38
StandardError 7.161 10.812 9.49
N 574 296 278
PupilAssetOwnership
EvaluationoftheGirl’sEducationProject3
©EDOREN 302
Hasacar
Mean 0.24 0.29 0.19
StandardError 0.029 0.038 0.039
N 574 296 278
Hasafridge
Mean 0.25 0.25 0.25
StandardError 0.034 0.04 0.055
N 574 296 278
Hasaradio
Mean 0.78 0.89 0.68
StandardError 0.042 0.027 0.06
N 574 296 278
Hasamattress/bed
Mean 0.94 0.99 0.88
StandardError 0.029 0.004 0.05
N 574 296 278
HasaTV
Mean 0.57 0.53 0.61
StandardError 0.054 0.067 0.083
N 574 296 278
Hasamotorcycle
Mean 0.74 0.72 0.75
StandardError 0.024 0.03 0.04
N 574 296 278
Hasachair
Mean 0.87 0.89 0.85
StandardError 0.032 0.027 0.056
N 574 296 278
Hasamat
Mean 0.98 1 0.96
StandardError 0.01 0.002 0.018
N 574 296 278
Hasasewingmachine
Mean 0.37 0.42 0.32
StandardError 0.034 0.051 0.039
N 574 296 278
Hasanairconditioner
Mean 0.03 0.02 0.03
StandardError 0.008 0.011 0.012
N 574 296 278
Hasafan
Mean 0.5 0.4 0.6
StandardError 0.052 0.059 0.078
N 574 296 278
Hasagenerator
Mean 0.38 0.32 0.44
StandardError 0.034 0.036 0.056
N 572 296 276
Hasastove
Mean 0.39 0.41 0.38
StandardError 0.037 0.057 0.047
N 573 296 278
EvaluationoftheGirl’sEducationProject3
©EDOREN 303
Hasacamera
Mean 0.06 0.08 0.05
StandardError 0.012 0.016 0.016
N 570 296 274
Hasacomputer
Mean 0.06 0.08 0.04
StandardError 0.012 0.022 0.012
N 572 296 276
Hasamobilephone
Mean 0.93 0.96 0.91
StandardError 0.02 0.011 0.034
N 574 296 278
Hasabicycle
Mean 0.62 0.63 0.6
StandardError 0.034 0.045 0.05
N 574 296 278
Hasagoat
Mean 0.77 0.81 0.73
StandardError 0.033 0.021 0.058
N 574 296 278
Hasacattle
Mean 0.47 0.51 0.43
StandardError 0.029 0.039 0.046
N 570 296 274
Hasahorse,donkeyormule
Mean 0.05 0.06 0.05
StandardError 0.014 0.024 0.014
N 574 296 278
Hasasheep
Mean 0.5 0.63 0.37
StandardError 0.041 0.034 0.061
N 571 296 275
Hasachicken
Mean 0.87 0.88 0.87
StandardError 0.028 0.03 0.049
N 574 296 278
Hasapig
Mean 0.01 0 0.02
StandardError 0.007 0.005 0.013
N 573 296 277
Note:ForeachvariablethetableaboveincludesMeanvalue,StandardErrorandNumberofobservations.
EvaluationoftheGirl’sEducationProject3
©EDOREN 304
AnnexN CBMCIQSSDescriptiveStatistics
VariableLabel TotalMean Meanbystate
Bauchi Niger
CBMCMembership
TotalMembership
Mean 14.24 15.83 12.88
StandardError 0.645 0.606 1.08
N 52 24 28
Totalfemales
Mean 3.65 4.79 2.66
StandardError 0.339 0.459 0.491
N 52 24 28
Femalesasashareoftotalmembers
Mean 0.24 0.29 0.19
StandardError 0.021 0.028 0.031
N 52 24 28
Totalchildren
Mean 0.32 0.62 0.07
StandardError 0.091 0.181 0.069
N 52 24 28
Childrenasashareoftotalmembers
Mean 0.02 0.04 0
StandardError 0.005 0.011 0.004
N 52 24 28
YearssinceCBMCestablishment
Mean 2.99 2.25 3.63
StandardError 0.417 0.607 0.575
N 52 24 28
Yearssinceintegration
Mean 2.44 2 2.82
StandardError 0.291 0.31 0.471
N 52 24 28
CBMCformedbeforeschoolintegration
Mean 0.37 0.25 0.47
StandardError 0.067 0.092 0.098
N 52 24 28
HeadTeacherinCBMC
Mean 0.65 0.67 0.64
StandardError 0.069 0.101 0.095
N 52 24 28
Chairperson-Headteacher
Mean 0.23 0.29 0.17
StandardError 0.04 0.072 0.042
N 52 24 28
Chairperson-ProprietorMean 0.32 0.37 0.28
StandardError 0.064 0.098 0.084
EvaluationoftheGirl’sEducationProject3
©EDOREN 305
N 52 24 28
ProprietorinCBMC
Mean 0.6 0.58 0.61
StandardError 0.073 0.11 0.098
N 52 24 28
NumberofteachersinCBMC
Mean 2.9 3.32 2.55
StandardError 0.325 0.437 0.474
N 52 24 28
NumberofparentsinCBMC
Mean 9.6 9.76 9.47
StandardError 0.714 0.733 1.168
N 52 24 28
NumberofoldorcurrentpupilsinCBMC
Mean 3.03 3.08 2.99
StandardError 0.373 0.533 0.521
N 52 24 28
NumberofreligiousleadersinCBMC
Mean 0.83 1.04 0.64
StandardError 0.131 0.192 0.18
N 52 24 28
CBMCMeetings
CBMCMeetingbetween2014-2015
Mean 0.9 0.88 0.93
StandardError 0.038 0.064 0.045
N 52 24 28
CBMCMeetinginthecurrentyear
Mean 0.65 0.79 0.53
StandardError 0.068 0.087 0.101
N 52 24 28
Numberofmeetingsheldbetween2014-2015
Mean 4.4 6.19 2.95
StandardError 0.37 0.799 0.178
N 47 21 26
Attendancerateatlastmeeting
Mean 0.41 0.36 0.46
StandardError 0 0 0
N 30 16 14
Attendancerateoffemalesatlastmeeting
Mean 0.35 0.26 0.48
StandardError 0 0 0
N 26 15 11
CBMCtrainings
CBMCmembersattendedtraining
Mean 0.63 0.54 0.71
StandardError 0.062 0.103 0.082
N 52 24 28
TrainingbyLGEA
Mean 0.16 0.23 0.11
StandardError 0 0 0
N 32 13 19
EvaluationoftheGirl’sEducationProject3
©EDOREN 306
TrainingbyGEP/UNICEF
Mean 0.66 0.62 0.68
StandardError 0 0 0
N 32 13 19
TrainingbySIP/TSP
Mean 0.03 0.08 0
StandardError 0 0 0
N 32 13 19
TrainingbySUBEB
Mean 0.06 0.08 0.05
StandardError 0 0 0
N 32 13 19
TrainingbySAME
Mean 0.03 0 0.05
StandardError 0 0 0
N 32 13 19
Totaltrainings
Mean 1.34 1.23 1.42
StandardError 0 0 0
N 32 13 19
TotalnumberofdaysofCBMCtrainings
Mean 4.03 4 4.06
StandardError 0 0 0
N 31 12 19
CBMCMonitoringandPlanningActivities
CompleteWCDPexists
Mean 0.19 0.21 0.18
StandardError 0.053 0.078 0.072
N 52 24 28
WrittenformofWCDPisavailable
Mean 0.67 0.78 0.59
StandardError 0 0 0
N 21 9 12
Numberofvisittomonitorprogress
Mean 7.25 8.2 6.43
StandardError 0.906 1.625 0.943
N 52 24 28
CBMCmonitoringofpupils’attendance
Mean 0.85 0.88 0.82
StandardError 0.047 0.065 0.067
N 52 24 28
CBMCstepstoincreasepupils’attendance
Mean 0.85 0.83 0.86
StandardError 0.052 0.076 0.071
N 52 24 28
CBMCmonitoringofteacher’sattendance
Mean 0.82 0.84 0.81
StandardError 0.057 0.078 0.081
N 44 18 26
CBMCtookactiontoimproveTattendance
Mean 0.94 1 0.89
StandardError 0 0 0
EvaluationoftheGirl’sEducationProject3
©EDOREN 307
N 33 15 18
CBMCqualityofteachingmonitoring
Mean 0.85 0.75 0.93
StandardError 0.051 0.093 0.051
N 52 24 28
CBMCResources
CBMCefforttomobilizecash
Mean 0.65 0.63 0.68
StandardError 0.066 0.104 0.084
N 52 24 28
CBMCeffortatcashmobilization
Mean 0.82 1 0.68
StandardError 0 0 0
N 34 15 19
Communitymembers(includingCBMCmembers,localbusinessesandindividuals)
Mean 1 1 1
StandardError 0 0 0
N 28 15 13
GovernmentBodies
Mean 0.07 0 0.15
StandardError 0 0 0
N 28 15 13
NGO,donorsorotherexternalagencies
Mean 0.07 0.07 0.08
StandardError 0 0 0
N 28 15 13
AmounttheCBMCmobilizefromcommunity
Mean 39171.9 31210.9 53255.68
StandardError 0 0 0
N 22 14 8
Totalfundsraisedlastyear
Mean 39474.2 31210.2 54092.8
StandardError 0 0 0
N 22 14 8
Othermonetarysupportwithoutfundraising
Mean 0.21 0.25 0.18
StandardError 0.058 0.093 0.072
N 52 24 28
CBMCreceivedfundingfromUNICEF
Mean 0.73 0.84 0.6
StandardError 0 0 0
N 11 6 5
Receivedschoolgrant
Mean 0.35 0.42 0.28
StandardError 0.066 0.092 0.094
N 52 24 28
Amountofschoolgrant
Mean 124892.3 105911.4 148190
StandardError 0 0 0
N 20 11 9
EvaluationoftheGirl’sEducationProject3
©EDOREN 308
TotalfundingavailabletoCBMC
Mean 112258.5 94258.66 135938.1
StandardError 0 0 0
N 30 17 13
Shareoftotalfundsinvested
Mean 85.68 90.93 78.19
StandardError 0 0 0
N 29 17 12
Utilizationofanynon-cashresource
Mean 0.42 0.46 0.39
StandardError 0.069 0.1 0.094
N 52 24 28
CBMCFinancialManagement
Useofcashbook
Mean 0.27 0.38 0.18
StandardError 0.062 0.099 0.073
N 52 24 28
Cashflowisrecordedincashbook
Mean 0.63 0.57 0.75
StandardError 0 0 0
N 11 7 4
Useabankaccounttostorethefundsreceived
Mean 0.69 0.75 0.65
StandardError 0.064 0.092 0.083
N 52 24 28
Accountinschool’sname
Mean 0.69 0.89 0.5
StandardError 0.079 0.078 0.119
N 36 18 18
AccountisinCBMC’sname
Mean 0.28 0.06 0.5
StandardError 0.074 0.055 0.119
N 36 18 18
Evidenceofbanktransactionavailable
Mean 0.53 0.63 0.43
StandardError 0.144 0.186 0.186
N 15 8 7
CBMCCommunitySupport
Communitymembers(includingCBMCmembers,localbusinesses,andindividuals)
Mean 1 1 1
StandardError 0 0 0
N28
15 13
AmounttheCBMCmobilizefromcommunity
Mean 39171.9 31210.19 53255.68
StandardError 13846.3 10168.79 32507.92
N 22 14 8
Proportionoftotalfundsraisedfromcommunity
Mean 98.41 100 95.81
StandardError 1.585 0 3.838
EvaluationoftheGirl’sEducationProject3
©EDOREN 309
N 21 13 8
CBMCmobilisedfundsfromcommunity
Mean 0.27 0.16 0.4
StandardError 0 0 0
N 11 6 5
CBMCRecordKeeping
Cashflowisrecordedincashbook
Mean 0.63 0.57 0.75
StandardError 0 0 0
N 11 7 4
Evidenceofbanktransactionsavailable
Mean 0.53 0.63 0.43
StandardError 0.144 0.186 0.186
N 15 8 7
EvidenceofCBMCmeetingsinthelastschoolyear
Mean 0.66 0.53 0.76
StandardError 0.088 0.126 0.109
N 32 15 17
WrittenformofWCDPisavailable
Mean 0.67 0.78 0.59
StandardError 0 0 0
N 21 9 12
Note:ForeachvariablethetableaboveincludesMeanvalue,StandardErrorandNumberofobservations.
Cover Photo Credit: © Global Plus News
EDOREN is a consortium of leading organisations in international development and education: Oxford Policy Management (OPM), and the Institute of Development Studies (IDS) at the University of Sussex, and is supported by UK Aid. EDOREN cannot be held responsible for errors or any consequences arising from the use of information contained in this report. Any views and opinions expressed do not necessarily reflect those of OPM, IDS and EDOREN or any other contributing organisation.