evaluation of global partnership for education programme ...4.2.3 strengthening sector policy...

Evaluation of GlobalPartnership for EducationProgramme in South SudanFinal report

6 April 2018

D5bC:\Users\VAN63802\Documents\1- ACTIVE PROJECTS\UNICEF GPE -SOUTHSUDAN\Deliverables\Final report\Final to submit\180406 GPE Endline finalreport_CLEAN.docxCambridge Education

Cambridge Education22 Station RoadCambridge CB1 2JDUnited Kingdom

T +44 (0)1223 463500F +44 (0)1223 461007camb-ed.com

Evaluation of GlobalPartnership for EducationProgramme in South SudanFinal report

6 April 2018

Mott MacDonald Limited trading asCambridge Education. Registered inEngland and Wales no. 1243967.Registered office: Mott MacDonald House,8-10 Sydenham Road, Croydon CR0 2EE,United Kingdom

Cambridge Education | Evaluation of Global Partnership for Education Programme in South SudanFinal report

Information class: Standard

This document is issued for the party which commissioned it and for specific purposes connected with the above-captioned project only. It should not be relied upon by any other party or used for any other purpose.

We accept no responsibility for the consequences of this document being relied upon by any other party, or beingused for any other purpose, or containing any error or omission which is due to an error or omission in data suppliedto us by other parties.

This document contains confidential information and proprietary intellectual property. It should not be shown to otherparties without consent from us and from the party which commissioned it.

This Re por t has be en p rep are d solely for use by t he p arty w hich c om mission ed it (the 'Client') i n co nnecti on wit h the cap tione d p roject . It s hould not be used for any oth er p urp ose. N o p erso n ot her tha n th e Client or any party who has expr essly a gre ed t er ms of relia nce wit h us (the 'Recipie nt(s )') m ay r ely on the cont ent, info rma tion or any view s exp ress ed in the R epo rt. This R epo rt is co nfide ntial and c ont ains p rop riet ary in tellect ual p rop erty and we ac cept no duty of ca re, resp onsibility or li ability t o any oth er recipi ent o f this R epo rt. N o re pre sent ation , wa rran ty o r un dert aking , exp ress or i mplie d, is made an d no res ponsi bility or liability is acce pted by us to any p arty oth er t han the Cli ent or a ny Reci pient (s), as t o the accu racy or c om plete ness of th e info rm ation cont aine d in t his Rep ort. Fo r t he av oida nce o f do ubt t his Re port do es no t in any way pu rpo rt to includ e a ny leg al, ins ura nce or fin ancial advic e or opini on.


Contents

List of abbreviations 1Executive summary 3

1 Introduction 81.1 Introduction 81.2 Background 9

2 Methodology 112.1 Survey Design and Instruments 112.2 Sample strategy 152.3 Data collection approaches 152.4 Outline of data analysis procedure 162.5 Research limitations 17

3 Relevance 193.1 What is the relevance of the GPEP interventions in relation to the

GESP/priorities of the South Sudan education sector? 193.2 What other relevant issues in the Education Sector could have been

addressed by South Sudan GPEP? 203.3 To what extent were GPEP interventions directly relevant to promoting

equitable access to quality education and improving children's performance? 213.4 To what extent was it necessary to modify the design or implementation of

GPEP to accommodate changes affecting the country (e.g. conflict, famine,and drought?) 23

4 Effectiveness and impact 264.1 What outcomes did GPEP achieve? 26

4.1.1 Proportion of students who, by the end of three grades of primaryschooling, demonstrate that they can read and understand themeaning of grade level text (outcome indicator A1) 27

4.1.2 Perception index for Teachers and Head Teachers on SchoolLeadership (outcome indicator A2.1) 29

4.1.3 Perception index for School Inspection and Supervision (outcomeindicator A2.2) 29

4.2 What worked and what didn’t in GPEP Component A: National SystemsStrengthening 294.2.1 Strengthening literacy and numeracy learning in primary schools 294.2.2 Strengthening primary school leadership, school inspection and

supportive supervision 304.2.3 Strengthening sector policy development, strategic planning,

monitoring and mainstreaming gender responsiveness 32


4.3 What worked and what didn’t in GPEP Component B: Community and School-based service delivery 324.3.1 Providing learning support services to primary schools 324.3.2 Support to School Management Committees (SMCs) 334.3.3 Improving physical infrastructure to enhance learning 344.3.4 Support to out-of-school children (OOSC) and young people 34

4.4 What worked and what didn’t in GPEP component C: Learning and SharingLessons? 344.4.1 Research: tracking the GPEP 34

4.5 Teacher training and teacher performance 354.6 Has GPEP achieved leverage? 354.7 Has GPEP achieved scale? 36

5 Gender and inclusion 375.1 Girls’ enrolment 375.2 Female representation in school management 375.3 Promoting gender equity in schools 385.4 Gender mainstreaming at a national level 395.5 Inclusion 39

6 School construction 416.1 To what extent and in what ways did school construction achieve desired

outcomes? 416.1.1 Increasing access 416.1.2 Improving quality education and learning 42

6.2 Was the school construction model approach successful? 426.2.1 Site and school design 426.2.2 Site and school selection 456.2.3 Cost of Model Schools 466.2.4 Community ownership 486.2.5 National systems strengthening 49

7 Sustainability 517.1 Community support and buy-in 517.2 Ownership 527.3 Continuity of activities absent funding 527.4 Catalysing additional funding 537.5 Learning and sharing of information 53

8 Collaboration and coherence 558.1 Education sector collaboration 558.2 Governance mechanisms 558.3 Staff turnover 568.4 Coherence 56


9 Lessons learnt and recommendations 579.1 Lessons learnt 579.2 Recommendations 57

Annex A: List of interviewees 60

Annex B: Evaluation Framework with references to report sections 62

Annex C: Sampling strategy 65

Annex D: Leadership and School Inspection and Supervision indices 67

Annex E: Sample of GPEP supported schools selected for evaluation 72

Annex F: Recommendations for future school construction in South Sudan 73

TablesTable 1: Overview of GPEP evaluation research instruments 11Table 2: Pass rates of GPEP students on comparable EGRA subtasks, baseline vs. endline 28Table 3: Pass rates of GPEP students on comparable EGMA subtasks, baseline vs.endline 28

FiguresFigure 1: Responses from PTA and SMC members showing what they would invest in inorder to improve the quality of education 21Figure 2: Responses from Head Teachers and teachers showing what they perceive as themost important intervention to improve access to education 22Figure 3: Responses from Head Teachers and teachers showing what they perceive as themost important intervention to improve children's performance 22Figure 4 Average percentage of females in PTA and SMC at baseline and endline 38Figure 5 Responses from students showing total number of concerns with environmentalaspects of their school 43Figure 6 Responses from students showing total number of environmental concerns perschool 44Figure 7: Responses from PTA and SMC focus groups showing who they consider asresponsible for small school repairs 48Figure 8: Responses from PTA and SMC focus groups showing who they consider asresponsible for large repairs 49

Cambridge Education | Evaluation of Global Partnership for Education Programme in South Sudan 1Final report

List of abbreviations

AET Africa Educational TrustAFOD Action for DevelopmentCE Coordinating EntityCEC County Education CenterCFS Child friendly schoolsCO Country OfficeDFID Department for International DevelopmentECD Early Childhood DevelopmentEDoG Education Development GroupEDOG Education Donor GroupEEP Education in Emergency ProgrammeEES (former) Eastern Equatoria stateEGMA Early Grade Mathematics AssessmentEGR/M Early Grade Reading/MathematicsEGRA Early Grade Reading AssessmentEMIS Education Management Information SystemEPRP Emergency Preparedness and Response PlanningESPIG Education Sector Plan Implementation GrantEU European UnionFGD Focus Group DiscussionGEC Girls’ Education ChallengeGESP General Education Sector PlanGESS Girls Education South SudanGPE Global Partnership for EducationGPEP Global Partnership for Education ProgrammeHT Head teacherHTQ Head teacher questionnaireIMED Improved Management and Education DeliveryJSR Joint Sector ReviewLKS (former) Lakes stateLEG Local Education GroupM&E Monitoring and EvaluationME Managing EntityMoGEI Ministry of General Education and InstructionNBEG (former) Northern Bahr el Ghazal stateNGO Non-governmental organisationOECD-DAC

Organisation for Economic Co-operation and Development's DevelopmentAssistance Committee

OOSC Out of School ChildrenPEG Partners for Education


POC Protection of CivilianPSLP Primary School Leadership ProgrammePTA Parent Teacher AssociationQA Quality AssuranceQ&A Question and answerRFP Request for ProposalSC Steering CommitteeSIL Summer Institute of LinguisticsSMC School Management CommitteeS/MoE State Ministry of EducationSSP South Sudanese PoundSSSAMS South Sudan Schools’ Attendance Monitoring SystemSSTEP South Sudan Teacher Education ProgrammeT TeacherTTI Teacher Training InstituteTWG Technical Working Group

UNESCO United Nations Educational, Scientific, Cultural Organization

UNICEF United Nations Children’s FundUSAID UNICEF: United Nations Children’s FundUSD United States Agency for International DevelopmentUSD United States DollarWASH Water, Sanitation, and HygieneWES (former) Western Equatoria stateWRP (former) Warrap stateYTTC Yei Teacher Training College


Executive summary

The GPE programme (GPEP) in South Sudan was implemented from December 2013 toNovember 2017, and extended at no-cost until May 2018. The programme was funded both byGPE and USAID. The overall funding envelope was USD 66.1m, of which USD 30.1m was theUSAID contribution from December 2013 to December 2016. UNICEF is the GPEP Grant Agentand worked closely with the Ministry of General Education and Instruction (MoGEI) on GPEPimplementation. GPEP was committed to supporting the implementation of the objectives in theGeneral Education Strategic Plan (GESP) 2012-2017, which were articulated in three principalGPEP components:

a. National Systems Strengthening;b. Community and School-based Education Service Delivery (implemented in five states

only);c. Learning and Sharing Lessons.

This GPEP endline evaluation was undertaken between November 2017-February 2018. Theobjectives were to 1) document the present status of the GPEP with regard to primary areas ofprogramme focus; 2) capture quantitative and qualitative Results data of the status of GPEPimplementation; 3) document achievements, constraints and lessons learnt over theimplementation period and provide solid recommendations to inform future programming andthe next South Sudan GPE programme. The evaluation conducted research in a sample ofGPEP supported schools, carried out key stakeholder interviews and a literature review. Allevaluation tools were developed based on an evaluation framework that included questionsunder standard OECD-DAC evaluation criteria (Relevance, Efficiency, Effectiveness, Impact,Sustainability,) and Collaboration and Coherence.

Over the course of GPEP implementation, protracted conflict and persistent economic crisishave affected education in South Sudan severely, constraining the national budgetary allocationfor education. The conflict has caused mass population displacement internally and acrossborders, the closure of some schools, and widespread food insecurity. In response to thesecrises, donors largely shifted funding to humanitarian support since 2014.

Relevance

The evaluation found that the GPEP did not make fundamental changes to the originalprogramme design and maintained a long-term development focus, despite the drastic changesin the operational environment compared to when the programme was conceptualised. Theevaluation concludes this was positive set against the alternative of fragmentation and short-term, smaller scale programming that characterises an Education in Emergencies (EiE)approach which can be a risk to education sector coherence. The main relevant GPEPachievements include the development of a new national curriculum and providing support toMoGEI for policy and strategy development, notably the development of the GESPII in 2017.Further examples include the English Language Policy Framework, Annual Education SectorReviews and the M&E Strategy which were all considered positive national developments andstrong contributions to the education sector.

Still, the evaluation concludes that GPEP could have reflected more frequently on whether allcore components remained relevant as the crisis context evolved and conflict spread. Inparticular GPEP’s approach to school construction, a sub-component of Component B, lostrelevance. The ‘model school’ approach concentrated investment in 25 newly constructed


schools in five states. The unit cost of school construction significantly increased in the contextof conflict and inflation, from an estimated USD 343,000 in 2013 to an actual average cost ofUSD 643,850 in 2015. This reduced the scope from 40 to 25 schools. Conflict also posed realrisks of damage and destruction to GPEP supported schools, with two schools (Ayii and Deretoin Eastern Equatoria State) closed and reportedly damaged due to conflict at the time of theevaluation. The evaluation strongly recommends a more equitable approach to improvingaccess in the next phase of GPEP.

Effectiveness and impact

The evaluation collected endline data for two GPEP outcome indicators: reading comprehensionand leadership, supervision and inspection indices. The evaluation found the proportion ofstudents who, by the end of three grades of primary schooling, demonstrate that they can readand understand the meaning of grade level text (Outcome indicator A1) decreased from 43% atbaseline to 17% at endline. The Early Grade Reading Assessment (EGRA) endline scores arein line with what other programmes have found in South Sudan. The evaluation suggestsproblems with baseline administration leading to unusually high baseline results (for example100% pass rates in schools in Lakes state) as a possible explanation for the drop in scores. Asnoted at baseline, the use of local languages for instruction, rather than English, at lower levelsof primary may generally affect pupils’ ability to perform well on EGRA in English. Pass rates ofGPEP students on comparable EGMA subtasks increased from baseline to endline.

Reading comprehension in English was chosen as an outcome indicator of the GPEP. Theevaluation found this indicator was not fully representative of the programme. Literacy andNumeracy Kits, developed in five local languages and English, could have strengthened readingcomprehension, however respondents raised concerns about the impact of the kits at schoollevel. The roll out of the materials experienced delays and did not include sufficient training forteachers. Training was provided relatively late in the cycle leaving insufficient time for learningto be put into practice. For example, the Action Research which involved mentoring and teachertraining in the GPEP supported schools, only started in 2017 for a period of seven months.Respondents felt this timeframe was too short to achieve behavioural change in teaching andschool administration practice and improved learning outcomes.

The index score representing teachers’ perceptions of head teachers’ leadership (Outcomeindicator A2.1) increased from 77% at baseline to 85% at endline, narrowly missing the 90%endline target. The index score representing teachers’ perceptions of school inspection andsupervision (Outcome indicator A2.2) also increased, from 75% at baseline to 83% at endline,surpassing the endline target of 80%. Scores were calculated based on teachers’ responses toleadership statements replicated from the baseline survey to facilitate comparison. Thesescores suggest the trainings provided under the Primary School Leadership, School Inspectionand Supportive Supervision Programme (PSLP) had a positive impact. Nine out of ten schoolsincluded in the endline also benefitted from school management and supervision trainingprovided by the Girls’ Education South Sudan (GESS) programme. The positive impact cantherefore not be fully attributed to GPEP.

The PSLP programme was implemented under GPEP Component A by a consortium led bySave the Children. Achievements of the programme included the development of a PrimarySchool Leaders, School Inspectors and Supervisors Strategy and Standards which wereapproved by MoGEI. Both basic and accredited training courses were designed andimplemented through the PSLP, which met its overall target with 8970 school leaders and 696inspectors and supervisors trained. The course accreditation by the University of Juba meansthat the training can easily be replicated in future years, although this would require externalsupport.


Challenges of the PSLP included the selection of participants. Whilst criteria were set it becamechallenging to find participants meeting these. Some respondents said the selection processbecame un-transparent leading to some poorly qualified nominees, some of whom could notengage fully in an English-led course. Insecurity also affected implementation and increasedcosts. For example, the accredited training logistics were changed to charter flights forparticipants to travel to Juba. Respondents further noted that the training duration was short andwould benefit from more follow-up and in-service mentoring. These lessons should beconsidered at design stage of the next phase of GPEP.

The Action Research approach appears to have had a positive impact on school management,increasing the number of GPEP supported schools with functional School Development Plansfrom 32% in June 2017 to 100% in November 2017.

The evaluation found little evidence of impact of GPEP support to County Education Centres(CEC) and Out of School Children (OOSC). The CEC survey responses showed only marginalimprovements compared to the baseline. GPEP support to OOSC was revised several timesand significantly delayed. GPEP finally supported OOSC in 2017 through UNESCO-ledinterventions which met targets, but were not coherent with the overall programme and lackedsustainability.

Gender and inclusion

The evaluation found evidence that GPEP was gender responsive and promoted gender equityat school level. GPEP succeeded in increasing the proportion of girls enrolled in GPEPsupported schools, largely due to an increase in all-girls school enrolment. The evaluation foundthat GPEP accommodated 60.7 displaced students on average, with two schoolsaccommodating 200 displaced students each. This shows that displaced students are a majorissue for the design of education programmes in the current context in South Sudan. The nextphase of GPEP should carefully consider how to meet their education needs.

School construction

The evaluation of the school construction component found that enrolment in GPEP supportedschools increased. GPEP ‘Action Research’ includes data on the number of children enrollingfrom neighbouring schools and the number of children enrolling who were previously out ofschool for Academic Year 2017. The evaluation found evidence that GPEP supported schoolsare attracting children from neighbouring schools, causing overcrowding in some classroomswhich was the most frequently cited concern in the school assessment survey. A commonperception existed amongst stakeholder that GPEP supported schools provide a moreconducive learning environment, but the evaluation did not yet find robust evidence that schoolconstruction has improved quality education and learning. The school assessment responsesshowed that most respondents were satisfied with the school facilities. Concerns around costsand risks of school construction and the model school approach are noted above.

Sustainability

The evaluation considered programme sustainability based on the level of community supportand buy-in, ownership of GPEP, the continuation of activities absent funding and the extent towhich GPEP has catalysed additional external support and strengthened education systems.The current security and economic context clearly jeopardises the sustainability of the gainsmade by any education programme after funding stops. All respondents recognised the need forcontinued external support to build on the achievements of GPEP. With regards to communitysupport and buy-in, this was well managed in the school construction component, but lessintegrated in the design of other components such as the Primary School Leadership


Programme (PLSP). There were mixed responses to questions around programme ownership.Education sector partners perceived GPEP as government-owned, whereas respondents fromthe MoGEI said they would like to be more involved in the next phase, especially arounddecision-making on resource allocations. At school-level, respondents said the GPE programmewas perceived as donor- or NGO-led. Increased visibility of the MoGEI as education serviceprovider at sub-national level should be considered for the next phase of GPEP.

Collaboration and coherence

Education sector planning and collaboration was strengthened through GPEP’s explicitalignment to GESP, the development of GESP II, and a range of other sector strategies andpolicies. The GPEP technical working groups and reference groups set up within the MoGEIwere useful mechanisms for technical oversight, information sharing and decision-makingduring implementation. GPEP played a key role in establishing governance mechanisms forsector coordination, such as a Joint Steering Committee for the ‘big four’ educationprogrammes. After initial momentum, this committee did not continue to meet regularly partiallyfor reasons beyond GPEP’s control such as the early closing of two of the four programmes in2016. However, respondents also said that GPEP could have been more visible and pro-activein pursuing more in-depth exchanges on strategic programme alignment.

On coherence, there is evidence that sub-components of Component A (national systemsstrengthening) were largely coherent with each other. GPEP would have been more coherent ifthe linkages between Component A and Component B were more strongly developed. Achieve-ments that strengthened national systems, such as the curriculum, have not yet been rolled out at scale at school level. The GPEP still appeared somewhat fragmented due to many different(sub) components. Partners contracted to deliver these covered only a set of specific activitiesfor a relatively short period of time. The evaluation therefore recommends a more focuseddesign in the next phase of GPEP. Fewer core components will make the programme more co-herent. This will help to achieve greater visibility and understanding of the programme object-ives and develop a strategy for government ownership at sub-national levels and sustainability.

Recommendations

In addition to, and based on the above findings, the evaluation presents five keyrecommendations to inform future programming in South Sudan:

Recommendation 1: Maintain long-term development focus to avoid fragmentation in theeducation sector which is a risk of short-term emergency programming. Carefully considerwhich activities have greatest chance of success in the protracted crisis context of South Sudan.

Recommendation 2: Focus on fewer core components to make the programme morecoherent, and allow longer timeframes for the implementation of these components.

Recommendation 3: Support the MoGEI to lead sector coordination. To do so effectivelyrequires: 1) clarification of roles and responsibilities of all stakeholders (Grant Agent andCoordinating Agency in particular), and 2) reconsideration of the team size and expertiserequired to manage GPEP.

Recommendation 4: Implementation model should be considered at design stage toachieve more coherence between national-level activities and roll out to schools, and avoid adhoc contracting and short timeframes for implementation. The design of GPE2 should considerhow implementation at state, county, payam and school-level will be managed at scale.


Recommendation 5: Implementation model should strengthen education structures todeliver results in schools. The implementation model needs to be flexible and able to respondto the unpredictable operating environment. This will require the ability to manage a temporaryscale-down or scale-up of activities in response to localised insecurity. The implementationmodel should consider ways in which education officials can be more involved in theimplementation and oversight of activities.


1 Introduction

1.1 IntroductionThe GPE programme (GPEP) in South Sudan was implemented from December 2013-November 2017, and extended at no-cost until May 2018. The programme was funded both byGPE and USAID. The overall funding envelope was USD 66.1m of which USD 30.1m was theUSAID contribution from December 2013 to December 2016. UNICEF is the GPEP Grant Agentand worked closely with the Ministry of General Education and Instruction (MoGEI) on GPEPimplementation.1

The GPEP was designed in 2012 by government and development partners to supportimplementation of the General Education Strategic Plan (GESP) 2012-2017. The GPEP hadthree main objectives:

1. Strengthen national systems that are fundamental to providing equitable access to qualityeducation;

2. Improve school performance, and in the process generate model approaches for improvingquality;

3. Attract additional support to the education sector in South Sudan by demonstratingsustainable successes.

The three objectives were articulated in three principal components of the programme: (A)National Systems Strengthening; (B) Community and School-based Education Service Delivery;and, (C) Learning and Sharing Lessons. Components “A” and “C” were designed to cover all ten(former) states of South Sudan, while component “B” was to cover only five (former) statesnamely Warrap (WRP), Lakes (LKS), Western Equatoria (WES), Eastern Equatoria (EES) andNorthern Bahr El Ghazal (NBEG).2 These components were interdependent and collectivelydesigned to provide a holistic approach to support government priorities and improve pupil’slearning outcomes in South Sudan.

UNICEF, as the Grant Agent, commissioned an independent, external evaluation of the GPEPthat was carried out from November 2017 to February 2018 by a team of five experts ofCambridge Education (part of the Mott MacDonald group).3. The objectives were:

1. To document the present status of the GPEP with regard to primary areas of programmefocus.

2. To capture quantitative and qualitative data which provides a comprehensive picture of thestatus of GPEP implementation against the Results Framework.

3. To document achievements, constraints and lessons learnt over the implementation periodand provide solid recommendations to inform future programming or related interventionsand the next South Sudan GPE programme.

1 GPE Terms of Reference for Grant Agents defines this as an operational role to disburse the GPE transferred funds to theimplementing partners, and provide fiduciary oversight and technical support as appropriate (December 2017)

2 A presidential decree in 2015 divided South Sudan in 28 states, out of the former 10. Subsequent decrees created a total of 32+1states. For the purpose of this report, the names of the former 10 states are used to refer to these geographic areas, consistent withUNICEF’s reporting.

3 The team consisted of Emma van der Meulen (Team Leader and M&E Specialist), Ashley Craft (Statistician), Caroline Jordan(Education Adviser), Francis Babodo (National Education Adviser) and Rene Dierkx (Architect, School Construction Specialist).


The evaluation team made two visits to South Sudan, the first from 20-30 November 2017 and asecond from 23 January-6 February 2018. Data was collected in 10 GPEP supported schoolsby trained enumerators in December 2017, just before the end of the 2017 Academic Year.

This report presents the evaluation findings, associated recommendations and lessons learnt.The evaluation and this report are guided by the following standard Organisation for EconomicCo-operation and Development's Development Assistance Committee (OECD-DAC) criteria asspecified in the UNICEF Request for Proposal document:

● Relevance: The extent to which the objectives of the programme are consistent withrecipients’ requirements, country needs, global priorities and partners’ policies.

● Effectiveness: The extent to which the programme’s objectives were achieved, taking intoaccount their relative importance.

● Efficiency: A measure of how economically resources/inputs (funds, expertise, time,equipment, etc.) are converted into results.

● Impact: Positive and negative changes produced by the intervention, whether directly orindirectly, intended or unintended.

● Sustainability: The continuation of benefits from the intervention after major developmentassistance has ceased.

UNICEF added questions under a sixth category ‘Collaboration and Coherence’ that focused onthe programme’s governance structures, sectoral collaboration and intra- and inter-programmecoherence of GPEPs components. UNICEF also encourage concepts of equity and genderequality to be addressed in evaluations. This report includes a section ‘gender and inclusion’ toassess gender responsiveness and promotion of equal rights of women and girls in theprogramme.

The structure of the report broadly follows the six evaluation categories for ease of reference.Under each of the criteria specific evaluation questions were addressed, as specified in theRFP. Questions on ‘Efficiency’ are addressed within the section on school construction, as mostavailable financial information related to this component. A full list of the evaluation questionsand where they are addressed in this report can be seen in Annex B. In addition to ‘gender andinclusion’, the findings on school construction are also presented as a separate section giventhe significant investment in this component.

1.2 BackgroundGPEP was conceptualised in a context of peace, but implemented in the context of civil war,economic crisis, and other emergencies. The most notable political, economic and securityevents during the GPEP implementation period (2013-2017) were 1) the outbreak of violence inJuba in December 2013, which led to protracted conflict in the Greater Upper Nile states; 2) aneconomic crisis and hyperinflation of the SSP against the USD since mid-2015; 3) Presidentialdecrees creating 28, and now 32 states out of the previous 10; 4) the outbreak of violence inJuba in July 2016 followed by intensifying conflict in Greater Equatoria and other previouslymore stable areas. The conflict has caused mass population displacement and food insecurityacross South Sudan.

These developments not only created an extremely challenging operating context, but alsoaffected education sector planning and coordination as most donor funding shifted tohumanitarian relief. GPEP was designed as a development programme and did not shift focusto Education in Emergencies over the course of implementation. GPEP continued to work withthe Ministry of General Education and Instruction on the implementation of activities that werealigned to the General Education Strategic Plan (GESP, 2012-2017), albeit with some


adaptations to the original plans. As the importance of longer term planning for education inemergencies and protracted crises is increasingly evidenced, evaluating the GPEP experiencein South Sudan presents an opportunity to learn from the implementation of an educationdevelopment programme in a fragile context.


2 Methodology

The endline evaluation was carried out over a period of three months by a team of fiveconsultants. The team worked remotely during a two-week inception phase in November torefine the workplan, finalise the evaluation framework and start up survey design work. Theteam agreed with UNICEF that the inception phase and the evaluation design phase for GPEPcomponent B would run concurrently. Evaluating component B required data collection at schoollevel and there was only a very short window to collect this before the end of the 2017Academic Year in mid-December. These time constraints limited the sampling approach andsample size to what could be collected within the window – see Annex C for full details. Duringthe first month, the team also analysed GPEP documentation and presented this LiteratureReview to UNICEF in December 2017.The evaluation of components A and C did not involveschool-level data collection and were therefore evaluated during the second visit to Juba inJanuary 2018. This involved interviews with key stakeholders in government and the educationsector.

The initial findings of the endline research were presented in Juba on 5 February 2018 as partof a Programme Design Workshop for the next phase of GPE. The presentation was attendedby key MoGEI officials including the Minister of Education, education sector partners, civilsociety representatives and the GPE representative from Washington DC.

2.1 Survey Design and InstrumentsThe endline evaluation administered a total of seven surveys at school and county level. Thedesign of the research instruments was informed by the original RFP which lists specificevaluation questions (see Annex B) and the Realigned Programme Results Framework, inaddition to the Literature Review, baseline materials and clarifications from the GPEP team in-country. The table below presents an overview of the objectives, respondents and data type ofeach instrument.

Table 1: Overview of GPEP evaluation research instruments

Instrument Objective Respondents Sampletype

Data type

EGRA/EGMA To collectcomparativeendline data for theProgrammeResults Frameworkoutcome indicator1

10 Grade 3 pupils(6 boys, 4 girls) ineach of 10 GPEPsupported schools

Stratified /Fixedinterval

Quantitative

TeacherQuestionnaire

To collectcomparableendline data for theProgrammeResults Frameworkoutcome indicator

2 teachers in eachof 10 GPEPsupported schools

Stratified /convenience

Mostlyquantitative


In administering the final evaluation, the conditions and tools used in the baseline werereplicated as closely as possible, while streamlining the baseline material by asking thosequestions that are relevant to the evaluation questions. This means:

● Core EGRA and EGMA literacy and numeracy tests were administered with instructionsgiven in English to Grade 3 (P3) students following the same methodology used at baseline.

● The same questions were used at baseline and endline to measure school leadership,School Inspection and Supervision through interviews with the Head Teacher and otherteachers, with only a few questions removed for redundancy and minor wording changes.

● Focus group discussions (FGD) were conducted with members of the SMC/PTA to collectfeedback on the GPEP support to their school. Discussion topics addressed specific aspectsof the evaluation framework (based on the evaluation questions in the original RFP).

In the inception phase, we had planned to conduct a FGD with Grade 6 pupils as well. However,the learning environment and school construction assessments collected feedback directly from

2, especially onSchool Leadership,School Inspectionand Supervision

Head TeacherQuestionnaire

To collect endlinedata for theProgrammeResults Frameworkoutcome indicator2, especially onSchool Leadership,School Inspectionand Supervision

1 Head Teacher ateach of 10 GPEPsupported schools

Stratified Mostlyquantitative

Learningenvironment andschoolconstructionassessment

To evaluate sub-component B3 andcollect answers toevaluationquestions in theoriginal RFP.

1 male and 1female from each of10 schools


Qualitative

SchoolCommunityFocus GroupDiscussion

To collectqualitativefeedback on theGPEP interventionin a community

4 or moreSMC/PTAmembers/parentsfrom each of 10schools


Qualitative

Questionnaire forSchoolInspectors orSupervisors

To collectqualitativecomparative datafrom inspectors

2-3 inspectors /supervisors perstate


Quantitativeand shortanswer

Questionnaire forofficials at CECsor selected TTIs

To collectqualitativecomparative datafrom officials atCECs

1 official per state Stratified /convenience

Quantitativeand shortanswer


upper primary pupils on many aspects of the school environment. Therefore, the issues to bediscussed in the FGD were sufficiently covered by this instrument and a separate FGD was nolonger considered necessary.

Learning environment and school construction assessment

The School Infrastructure Audit was developed specifically for the endline evaluation. Thisassessment was not included in the baseline. Its methodology is adapted from otherassessments of child-friendly school construction in the region. The assessment covers 23environmental aspects grouped in five basic domains:

1. Aspects of indoor environmental quality include end-user experience with regard to thermalcomfort, auditory comfort, visual comfort, and sensory stimulation.

2. Aspects of social climate include sociality, privacy, crowding, personalization & ownership,aesthetics & appearance, and place-meaning.

3. Aspects of organizational structure include safety, functional flexibility, functional proximity,way-finding and orientation, and structural adaptability, resource sustainability.

4. Aspects of outdoor environmental quality include shaded schoolyard, building configuration,play-and sports, school garden, water conservation, solid waste management

5. Aspects of build quality include structural integrity, disabled-access, craftsmanship,application of safe and healthy building materials

The assessment allows students to reflect on the suitability of their learning environment whileobserving the environment. In each school, at a minimum two pupils (one boy, one girl) wereengaged during the assessment to provide feedback on these aspects, express any concernsand share how these concerns have been addressed. In addition, enumerators took photos ofthe schools from various angles.

EGRA/EGMA learning assessments

EGRA/EGMA learning assessments were adapted from the baseline materials andadministered with instructions in English. The assessments included the following core modules:

● Literacy (EGRA): Letter sounds, made-up words, reading comprehension and fluency (wordsper minute), listening comprehension

● Numeracy (EGRA): Number identification, number ordering, number patterns, Level 1 and 2addition and subtraction

The endline EGRA and EGMA was designed to be closely comparable to the baseline.However, new questions and reading and listening passages of equivalent difficulty weresubstituted, having been checked for appropriateness by the local education advisor, so thatteachers could not teach to the old test.

The endline EGRA and EGMA also included harder pattern recognition and Level 2 additionand subtraction problems, as the baseline report noted that many schools were scoring highly inmaths and there could be ceiling effects to measuring further improvement. Scores on theharder pattern recognition, addition and subtraction sections were reported separately andexcluded from the comparison of baseline and endline results.

Headteacher and teacher questionnaires

The headteacher and teacher questionnaires were revised to ensure that data responded to theendline evaluation questions. Headteacher and teacher questionnaires included sections onrespondent demographics, effectiveness of GPEP interventions, equity of interventions,


community support, teacher training, school resources, leadership and management, andinspection and supervision.

Sections were added to address additional questions in the evaluation framework:

Evaluation question Additional topic area

C8 To what extent, and in what ways didprograms result in measurable improvementsin access, school experience and girlsachievement? What interventions tended tobe most effective?

Which GPEP interventions contributed mostto improving enrolment, school environment,and learning outcomes for boys and girls

A3 To what extent were GPEP interventionsdirectly relevant to promoting equitableaccess to quality education and improvingchildren's performance?

Which GPEP interventions affected girls,children with disabilities, or displacedstudents.

C7 Which outputs and outcomes can beconsidered direct results of UNICEF support?

C9 To what extent did GPEP support/play acatalytic role in mobilizing actions andresources from other partners (includinggovt., NGOs, civil society private sector,external support agencies)?

What other support GPEP supported schoolsreceived

Sections on school resources, leadership and management and inspection and supervisionwere kept from the baseline with only minor changes to facilitate comparison.

Sections on school infrastructure were moved to the school assessment tool.

Questions on headteacher and teacher surveys had similar wording so that information could betriangulated.

School Inspectors/Supervisors questionnaire

The school inspectors and supervisors’ questionnaire was developed to allow simultaneousinterview of 2-3 inspectors and supervisors at the county education office. It included questionson teacher training, community support, school resources, and instruction and supervision,which allowed triangulation of responses between inspectors/supervisors and teachers.

CEC officials’ questionnaire

A short questionnaire for CEC officials was developed to gather information on support toschools, PTA, and SMC, out-of-school girls, improved CEC infrastructure.

PTA/SMC focus groups

An open-ended topic guide was developed for focus groups with parents and communitymembers, ideally from the PTA and SMC. These questions were designed to validate questionsfrom the headteacher interviews and, amongst other topics, understand the process of GPEPschool construction and the extent to which community members were consulted.


Primary 6 focus groups

This research activity was included in the school assessment surveys, where P6-P8 studentswere asked in detail about the infrastructure in their schools.

All survey instruments were submitted to UNICEF for approval in the week of November 27th,2017. Table 1 above provides an overview of the objectives, respondents and types of data thatwas collected.

2.2 Sample strategyIt was necessary to select a sample of schools to visit, because of budget limitations. Theminimum requirement for the endline was to include at least one GPEP supported school fromeach state. The endline included two GPEP supported schools from each (former) state. The listof schools included in the sample is provided in Annex D.

Review of the baseline data showed there was a relatively large difference in learning outcomes(EGRA and EGMA) between GPEP supported schools in each state. Selecting well or poorlyperforming schools in the endline sample was identified as a potential design effect that coulddecrease the power of the survey and thus increase the chance of a Type II error, where thesurvey would find no significant difference at endline when in fact there was one. To account forthis, GPEP supported schools were first categorised into high, medium, and low performingbased on EGRA and EGMA tests at baseline. Then, a high-performing and low-performingschool were selected from each state; where two schools performed similarly, one was chosenat random. This approach minimised the design effect of selecting a single school or group ofschools which are skewed towards high or low performance at endline, while also recognisingschools which performed lower at baseline have a greater scope for improvement at endline.

Selected schools were then assessed on location to ensure they included both town and ruralschools, school structures that were originally permanent, semi-permanent, and TemporaryLearning Spaces (TLS), and schools with both sufficient and insufficient land for expansion. Thefinal school sample included several schools from each category so that lessons could be drawnabout different school types.

The selected schools were shared with UNICEF for feedback on the accessibility of eachlocation. This led to the replacement of several schools with substitutes due to insecurity in thelocations of several selected schools. The final sample size, adjusted for accessibility, includeda mix of schools by location, structure type, and availability of land for expansion.

Please see Annex C for a full explanation of the sampling approach and sample sizecalculations.

2.3 Data collection approachesData collection in the states

Given the limited timeframe for the assignment before the end of the academic year 2017, andlogistical and security challenges related to short-term international consultants’ travel out ofJuba, Forcier Consulting was contracted to collect data from ten selected schools in five(former) states. All research instruments were provided by the endline evaluation team, whoalso trained five research supervisors on all data collection instruments from 23-29 November2017 at Forcier’s offices in Juba. The headteacher and teacher survey, school assessment, andEGRA/EGMA learning assessments were piloted at a primary school in Juba on 26 November2017.


Forcier then:

● Recruited and trained 2-4 qualified enumerators in each state● Coded survey instruments onto electronic devices for data collection and used paper-based

methods to administer EGRA/EGMA and qualitative research instruments● Contacted and coordinated research visits with the state, regional, and county level Ministry

of Education offices and Head Teachers to ensure research permissions were obtained andschools were in session

● Deployed into the field between 2-6 December, recruited and trained enumerators between5-8 December, and gathered data between 8-22 December.

● Explained the purpose of the research and obtained informed consent for all participants.● Collected all data for Component B in the selected schools, including surveys of

headteachers and teachers, CEO officials, inspectors and supervisors, student EGRA/EGMAlearning assessments, focus groups with SMC/PTA, and school construction assessmentsincluding observations, student comments and photographs

● Provided raw, clean data sets for all quantitative data and typed up notes of the qualitativedata by 31st of December.

The raw data was quality assured and analysed by the evaluation team. No major issues wereidentified in data quality or collection that would affect the sample.

Data collection in Juba

Data collected through surveys at sub-national levels was complemented with interviews withkey informants in Juba, that largely focused on Components A and C. This reflected that theseprogramme components worked largely on a national level, as well as logistical and securityconsiderations that prevented the interview team from travelling outside Juba. National leveldata was complemented with data from county, and payam level officials gathered by the fieldteam during November 2017. However, the in-depth interviews held with stakeholders in Jubamay mean that findings emphasised this perspective.

Interviews included officials in the MoGEI, education donors and sector partners and NGOs thathad been contracted by GPEP for the implementation of specific activities. Interview guideswere developed for each type of respondent (government, donor/education sector partner andimplementer) and shared with UNICEF for approval in the week of January 22nd. Interviewguides were pre-tested with each type of respondent and refined before administering them tothe target list of stakeholders.

In-depth semi-structured interviews were conducted with 21 key stakeholders in Juba between23 January-2 February 2018.

2.4 Outline of data analysis procedureResearch instruments were aligned to the evaluation framework to allow analysis of the resultsand to answer each evaluation question set out in the RFP. Similar questions were included ondifferent survey instruments – headteachers survey, teachers survey, PTA/SMC,inspectors/supervisors’ questionnaire – to allow comparison and triangulation of responses.

Quantitative survey data was analysed using Excel and the SPSS statistical software package.Qualitative data from focus groups, school assessments, and short survey responses werealigned to existing closed-ended response categories. Open-ended responses were hand-coded using thematic categories based on all responses received.


For outcome level indicators, where comparisons between baseline and endline were required,only questions which appeared on both baseline and endline were used. For changes inmethodology, an adjusted baseline figure using the endline methodology was provided forcomparison. This applied to the EGRA and EGMA assessments (Outcome A1) and the schoolleadership and inspector and supervisor indices (Outcome A2).

Enrolment data

GPEP school enrolment data for 2017 was taken from the latest progress report (Jan-Jun2017). The GPEP school enrolment data was chosen as the best source of data as it wasgathered from head teachers using a similar method to what was used at baseline. AET’s ActionResearch data provides a different set of enrolment data. The SSSAMS dataset was incompletefor GPEP supported schools at the time of writing the endline report.

Validation workshop in Juba

Initial findings were presented to stakeholders on 5 February 2018 as part of the ProgrammeDesign Workshop for the next phase of GPE. The workshop was attended by key MoGEIofficials, education sector partners and civil society representatives and GPE representativesfrom Washington DC. During a Q&A session after the presentation of the findings, keycomments and questions were noted and these have been considered for the final report.

2.5 Research limitationsIn general, constraints on the time and budget available for the endline meant that a stratifiedsampling approach was used, with data gathered from only 10 of the 25 GPEP supportedschools. This two-stage sampling approach means the representativeness of data for certaininstruments is limited at the programme level. Moreover, the selection criteria for GPEPsupported schools means findings for the GPEP may have limited generalizability for all schoolsin South Sudan.

Learning environment and school construction assessment

The school environment and construction assessment is a relatively detailed tool of 23distinctive environmental aspects distributed over five domains which capture the extent towhich the school buildings and environment is fit-for-purpose (teaching and learning), as itdocuments the perspective from the end-users. However, obtaining rich, purposeful data maybe hindered by the level of understanding of the facilitator administering the assessment, as wellas end-users’ abilities to fully grasp the nuances between the various environmental aspectsand provide a valuable response. Another possible limitation is the time available for introducingthe various aspects of the assessment to the end-users, administering the assessment and (therichness) of the responses being documented, was relatively short as an in-depth assessmentof a school environment and construction may take considerable time. Finally, the number ofrespondents was limited to two (a school girl and a boy) due to the same time-factor whereas ahigher number of respondents would likely give richer, more nuanced responses.

EGRA/EGMA learning assessments

The EGRA/EGMA sample was designed to test whether scores changed significantly frombaseline to endline in selected GPEP supported schools, and to supply outcome level data onthe proportion of students who, by the end of three grades of primary schooling, demonstratethat they can read and understand the meaning of grade level text. At endline the learningassessments employed a two-stage sampling method, and as such results may be biased by aclustering effect as certain schools performed higher on average than others. To mitigate this,


the sample included one higher-performing and one lower-performing school from each state,as determined from baseline EGRA/EGMA scores. At endline schools were not selected norscores weighted by enrolment, which may affect the overall representativeness of the survey.

Headteacher questionnaires

Headteacher questionnaires were administered to each headteacher at 10 selected schools.Factual information from the headteacher, such as whether the school has benefitted fromexternal support, can be seen as representing the entire school, and the selection allowscomparison of selected schools at baseline and endline. However, data from the 10headteachers has a wide margin of error (+/- 24.5%) and may not be representative of allheadteachers at GPEP supported schools.

Teacher questionnaires

Teacher questionnaires were administered to two teachers per school. The small number ofteachers administered and the method of selection (teachers who were available, and did notchoose to opt out) means that data from teachers should be interpreted as illustrative only andnot representative of teachers at the school, state, or programme level.

School Inspectors/Supervisors questionnaire

Questionnaires were administered to an average of four inspectors or supervisors per state,similar to baseline. The information gathered are informative of the issues and perceptions ofschool inspectors and supervisors but may not be representative of all inspectors andsupervisors.

CEC officials’ questionnaire

Questionnaires were administered to one CEC official per state, where offices had between 3-8professional staff. Most questions asked were factual, but reporting could be biased byindividual perspectives and opinions, particularly on the perceived adequacy of office equipmentand materials. It is notable that opinions towards the adequacy of office equipment tended to bepolarized (see Section 4.3.1 for CEC staff perceptions of adequacy of facilities and materials).

PTA/SMC focus groups

Focus groups were conducted with volunteers in the community. Responses may therefore beaffected by self-selection bias, where the most committed or opinionated community membersvolunteered to participate.


3 Relevance

OECD-DAC defines relevance as the extent to which the objectives of the programme areconsistent with recipients’ requirements, country needs, global priorities and partners’ policies.Within the context of GPEP this means relevance of the programme in relation to the GESP,relevance to promoting access to quality education and improving children’s performance, abilityto adapt to meet the changes in the implementation environment and relevant issues in theeducation sector that GPEP omitted and could have addressed. This chapter is structured toanswer each of the four relevance questions from the evaluation framework and conclude onthe overall relevance of GPEP.

3.1 What is the relevance of the GPEP interventions in relation to theGESP/priorities of the South Sudan education sector?GPEP was strongly aligned to the General Education Strategic Plan (GESP). Alignment tocountry priorities was one of the conditions for GPE funding. GESP was the first strategic multi-year plan for education after South Sudan’s independence. Through its explicit alignment, theGPEP contributed to the implementation of GESP. All stakeholders interviewed appreciated thisalignment.

The conceptualisation of GPEP took place in the context of relative peace in South Sudan. TheGPEP prioritised access and systems development and included components to improve thequality of education. In 2012-2013, these were considered to be the right priorities for thesector.

However, two outbreaks of conflict- first in 2013 and then in 2016 caused a change in theimplementing environment which gave reason to re-assess the relevance of the earlierestablished sector priorities. For GPEP this required a re-assessment of the relevance of theapproaches taken to improve access, equity and quality in the education sector.

GPEP remained consistent in its focus on longer term education development, even when manyplayers shifted focus to Education in Emergencies (EiE) in 2014. All stakeholders consideredthis as positive and relevant for the education sector. Fragmentation and short-term, small-scaleprogramming that characterises an EiE approach was considered a risk to the sector. However,when asked about the relevance of specific components within GPEP, some stakeholdersquestioned the relevance of the school construction in the fragile context. Their perceptionswere that the conflict had caused inflated construction costs and GPEP could have respondedby changing the school design, materials and construction method to offer a lower-tech(appropriate-tech) alternative. Respondents deemed a lower-tech construction approach forexample one that rehabilitates existing schools or uses local skills and materials to be of greaterrelevance to the South Sudanese education sector. This is discussed in more detail in Section 6on School Construction.

Based on the literature review and stakeholder interviews, the most relevant achievements ofGPEP, were the following:

National Curriculum Development. The refreshed GPEP documents in 2014 increased focuson the sub-component ‘strengthening literacy and numeracy learning in primary schools’ toinclude curriculum development. Curriculum development is an important element in GESP andmomentum had already begun with DFID funding the South Sudan Curriculum Frameworkwhich was developed and approved at the end of 2013. It was therefore strategic to build upon


this and develop subject overviews and syllabi for all subjects from ECD through to end ofSecondary education during 2014 and 2015. The new curriculum is competency-based.Defining learning outcomes is also an integral part of the curriculum development process andwas pertinent in this overall process. In 2017 unspent funds were re-allocated to textbooksunder this sub-component in support of the National Curriculum Reform. Stakeholder interviewsconfirmed that this was a highly relevant achievement for independent South Sudan. The newcurriculum provides a basis for a more harmonised approach to teaching and learning in thecountry, where multiple curricula have been in use to date.

Strengthening sector policy development, strategic planning and monitoring TheMonitoring and Evaluation strategy, the approval of five national languages as official languagesof instruction, the English Language Policy Framework and the development of literacy andnumeracy assessment tools in the national languages were all mentioned as relevantachievements for the sector that can provide policy and operational guidance in the comingyears.

In addition, GPEP played a key role in the development of the GESPII and the organisation ofGeneral Education Annual Review which both strengthened the education sector.

Strengthening primary school leadership, school inspection and supportive supervision.This was implemented through the Primary School Leadership Programme by a consortium ledby Save the Children. PSLP designed both basic and accredited courses, accredited byUniversity of Juba and endorsed by MoGEI. Accredited courses served to professionalise theschool leadership, supervision and inspection roles and provide a national standardisedapproach to school leadership.

Strengthening Literacy and Numeracy learning. The Curriculum Framework, SubjectOverviews, Syllabi, Learning Outcomes, Literacy and Numeracy Assessments, Literacy andNumeracy Strategy and Literacy and Numeracy kits were all developed, reviewed and approvedby MoGEI. International and national development organisations, programmes and individualswere engaged in the development of numeracy and literacy deliverables. This served tostrengthen the quality assurance and ensure they were designed to meet both contextual andintellectual needs as well as international standards (as stated in acknowledgements of LiteracyKit).

3.2 What other relevant issues in the Education Sector could have beenaddressed by South Sudan GPEP?Nearly all stakeholders interviewed said the lack of qualified teachers was the biggest challengefacing the South Sudan education sector. GPEP could have addressed teacher training beyondthe leadership training that has benefitted head teachers and selected teachers. Somerespondents cited studies that showed teacher training has a bigger impact on learningoutcomes than school construction. Without improving the quality of teaching, the impact ofschool construction on the quality of education remains limited. This is confirmed through theAction Research and endline research results which show limited or no gains in learningoutcomes.

However, it was noted by education sector partners that a large-scale multi-donor programmetargeting teachers was being planned in 2014-2015. In the expectation that this programmewould start imminently, GPEP would not have prioritised teacher training at the time of design.The programme was conceptualised but did not materialise due to political developments.


Other issues that were mentioned included:

● Secondary school construction: GPEP focused on constructing only primary schools,whereas there is a shortage of secondary schools as well. In some counties there is nosecondary school at all.

● Education for children with Special Needs: The chapter on Gender and Inclusion discusseshow GPEP has addressed this within its components, however GPEP did not include acomponent that specifically addressed this and some stakeholders said this would havebeen relevant.

● Pastoralists Education: Several stakeholders mentioned there should be greater targetedsupport for children from pastoralist communities. GPEP included a focus on Out Of SchoolChildren (OOSC) and supported the finalization of the Pastoralist Education Pastoralist(PEP) Strategic Framework. The PEP Strategic Framework was validated on 30 August2017 with participation of senior MoGEI officials, UNICEF GPE Manager and UNESCO. Thissuggests that the implications of the framework are yet to be seen at school and communitylevel and the next phase of GPEP could consider how to support its implementation.

3.3 To what extent were GPEP interventions directly relevant to promotingequitable access to quality education and improving children's performance?Responses of Head Teachers, Teachers and members of SMC/PTAs to selected surveyquestions show their perceptions of the relevance of GPEP interventions in relation to access,quality education and learning improvements.

PTA and SMC members were asked how they would invest resources to improve the quality ofeducation in their communities, and the majority said that teacher training was the highestpriority seconded by school construction in contrast to the point of view of some upstreamrespondents for Stakeholder Interviews (see graph).

Figure 1: Responses from PTA and SMC members showing what they would invest in inorder to improve the quality of education

6

8

3 3 34

0123456789

Constructingclassrooms,

teacher blocks,and otherfacilities

Investing intraining forteachers

Investing inteacher guides

and teachermaterials

Investing inlearning

materials forpupils such astextbooks and

notebooks

Investingschool feedingprogrammes

teacherincentivesN

umbe

r of r

espo

nses

Priority investment for improving quality of education

If you had resources to improve the quality of education in your community,what do you think would be the highest priority for investment?


When Head Teachers and teachers were asked what GPEP intervention was most important forimproving access to education, 80% of head teachers and 50% of teachers chose teachertraining, while 20% of head teachers and 30% of teachers chose school construction. Thisdistribution showed that the preference for teacher training was statistically significant for bothhead teachers (p<.01) and teachers (p<.01).

Figure 2: Responses from Head Teachers and teachers showing what they perceive asthe most important intervention to improve access to education

When asked what GPEP intervention was most important for improving children’s performance,provision of learning materials was the most frequent response given by both head teachersand teachers, followed by teacher training. This distribution showed that preference for learningmaterials and teacher training was statistically significant for teachers only (p<.001).

Figure 3: Responses from Head Teachers and teachers showing what they perceive asthe most important intervention to improve children's performance

These responses are illustrative but suggest that head teachers and teachers may identify theprovision of learning materials and teacher training as more important than school constructionfor improving access and quality of education in their schools. Due to a limited sample size and

810

2

6

02468

1012

HT T

No.

resp

onse

s

Respondent group

What is the most important intervention to promote access?

Sum of Teacher training

Sum of School construction

4

10

2

8

2

00

2

4

6

8

10

12

HT T

No.

resp

onse

s

Respondent group

What is the most important intervention to improve children's performance?

Sum of Learning materials

Sum of Teacher training

Sum of School construction


self-selection, responses from teachers may not represent the opinions of all teachers in theschool or of the GPEP.

It is useful to unpack the relevance of school construction. The intervention of schoolconstruction was deemed to be of relevance. The graphs above show that it featuresconsistently, not as the most relevant but as a relevant intervention in the eyes of HeadTeachers, Teachers, PTAs and SMC. This was also reflected in interviews with developmentpartners and government officials who saw school infrastructure as relevant to improve access.However, the school construction model chosen was not deemed the most relevant for SouthSudan. The reasons for this given by stakeholders include: i) Resources were concentrated intoo few schools and choosing to invest the same resources in more ‘lower-tech’ schools wouldhave been more equitable and relevant given the large need for schooling in South Sudan. ii)The onset of conflict made construction costlier and at this point GPEP should have revised themodel to control costs per school. Revising the number of schools downwards from 40 to 25 inresponse to rising costs meant that fewer children would benefit from the intervention. iii) Theonset of conflict created high risks of damage and/or destruction to the investment made in thenewly constructed schools. The separate section on School Construction will look at this in moredetail (see section 6).

3.4 To what extent was it necessary to modify the design or implementation ofGPEP to accommodate changes affecting the country (e.g. conflict, famine, anddrought?)GPEP’s operational landscape over the last four years has been very dynamic with a range ofchanges in the operational environment directly impacting programme delivery as noted abovein the background section of the introduction. This section considers the impact of conflict,economic crisis and political developments on GPEP programming.

Conflict

The onset of conflict in December 2013 and later in July 2016 caused significant disruption toimplementation of GPEP and subsequent changes to the programme design. GPEP wasdesigned as a development programme; there were no major security risks envisaged, onlyperiodic minor intercommunal conflicts. When conflict erupted for the first time GPEP waslimited in its ability to respond to emergent context needs and adjustments were needed. GPEPgovernance structures facilitated the decision-making on programme adjustments.

The USAID GPE Programme Report notes that a conflict sensitive lens was applied toprogramme interventions, reframing the programme and reconfiguring interventions to bettermeet the emerging circumstances of conflict. For example, application of a conflict sensitiveapproach to clustering resulted in a more equitable and acceptable selection of traininginstitutions rather than the original five exclusive CECs that had created friction that could havecontributed to further tensions and conflict. A similar conflict sensitive process was utilized in theselection of initial five languages for mother tongue literacy and numeracy programming. TheNational Languages Implementation Framework ensured an all-inclusive process that definesthe sequencing of languages as well as their development and preservation.

However, applying conflict-sensitive principles and consensus building takes significant time.This was unforeseen in project planning and therefore led to delays in implementation.

From a programme management perspective, the Risk Profile Monitoring Matrix and BusinessContingency Strategies mitigated conflict related risks. The Risk Profile Monitoring Matrix wasreported against in Progress Reports. In both December 2013 and July 2016 key international


programme staff and consultants from UNICEF and implementing partners were evacuatedfrom South Sudan. Many local staff were required to fill humanitarian roles during these periods.Ethnic and tribal fracturing severely restricted mobility of national staff. This caused a freeze inprogramme implementation, delaying outputs.

UNICEF launched an Education in Emergency Programme (EEP) and tried to dovetailinterventions with GPEP to simultaneously build GPEP sustainability and address immediateEiE needs. The GPE teams met with Education Cluster and shared information through LEGand the Education Donor Group (EDoG), however, the GPE team did not attend EducationCluster meetings. Even though these focus on coordinating Education in Emergencies, with thespread of insecurity to many parts of the country, key informant interviews suggested it wouldhave been relevant for the GPE to connect more with the Cluster. Other sector programmes didupdate Cluster partners on their activities and this led to useful coordination of activities andgenerally more awareness about the programme.

Economy

At the launch of GPEP in 2012 education financing was perceived to be stable and it wasassumed that domestic financing would steadily increase over time. However, GPEPdocuments4 report that the reality since 2013 has been dwindling allocation initially due toausterity measures and then exacerbated by the conflict where more than 50% of the domesticbudget was allocated to security/defence. Furthermore, during this period the South SudanesePound depreciated rapidly against the US Dollar. The Bank of South Sudan imposed limitationson US Dollar transactions. These measures affected cash flows for field trainings, schoolconstruction and procurement, causing delays in implementation. Hyper-inflation meant salariesof teachers and education officials significantly declined and in many cases teachers wentunpaid. This severely impacted teacher motivation and led to teacher absenteeism and anincreasing number of trained teachers leaving the profession for better remunerationopportunities, leaving untrained, less experienced volunteers in teaching positions. In turn thisresulted in inconsistent and limited participation in GPEP activities and a direct and negativeimpact on school effectiveness. GPEP monitored and reported this through the Risk ProfileMonitoring Matrix and continued advocacy with other development partners and donors forcontinued S/MoGEI timely disbursements of teacher salaries.

Political context

GPEP was designed to work with and under the leadership of MoGEI. The political environmentwas, therefore, a key variable within GPEP’s operational environment and impacted directly onprogramme delivery. The meeting minutes show that MoGEI played an active role in the JointSteering Committee and Reference Groups where government priorities were shared. Politicalchanges in mid-2015 surrounding the creation of 28 states and new state leadership causedsome delays on state level activities.

Conclusion

Overall, the evaluation found that the GPEP maintained relevance as an educationdevelopment programme supporting the implementation of the GESP. Stakeholders fromschools, communities, government and development partners mostly agree that GPEPcomponents were relevant in terms of promoting equitable access to quality education andimproving children’s performance. One aspect that lost relevance as conflict spread and theoperating environment changed is school construction. It is not the case that construction itselfwas irrelevant but the construction model implemented by GPEP was deemed to be less

4 USAID GPE Programme Report (2017)


relevant and it was commonly perceived that alternative, low-cost and more localisedconstruction models would have been more relevant for the context of South Sudan.


4 Effectiveness and impact

Effectiveness measures the extent to which the programme’s objectives were achieved, takinginto account their relative importance. This section first presents progress on the GPEPoutcome indicators based on endline data collected as part of this evaluation. Secondly, each ofGPEP’s components is presented separately to understand in what ways the component’sobjectives have been achieved. This section draws on the endline research findings, interviewsand GPEP literature to determine the critical factors for progress or lack thereof. Finally, thesection presents to what extent the GPEP activities are scalable and whether GPEP hasachieved leverage.

4.1 What outcomes did GPEP achieve?The overall objective for GPEP was to increase learning achievement in primary schools andout-of-school learning spaces, in particular in literacy and numeracy. In line with this, fiveprogramme outcome indicators are included in the GPEP realigned programme resultsframework. Achievement against the targets is shown in the table below.

Outcome indicator Baseline Endline target Endline actualA1. Proportion of students who, bythe end of three grades of primaryschooling, demonstrate that theycan read and understand themeaning of grade level text

43% 48% 17%

A2. Perception index for Teachersand Head Teachers on SchoolLeadership, School Inspection andSupervision in the GPEP supportedschool

Leadership Index=83%; Supervision

and InspectionIndex = 70%;

Leadership Index=90%; Supervision

and InspectionIndex = 80%;

Leadership Index=85%;

Supervision andInspection Index

=83%B1. Total Enrolment in GPEPsupported school (Note: 3 All-GirlsSchool enrolment is 2,826 atbaseline)

13,686

(M=7,032;F=6,654)

14,436

16,877

(M= 7,902;F= 8,975)

B2. Number of children thatdropped out in GPEP supportedschools

1,442

(M=682; F=760)

Cumulative Target:1200

Initial Set Target:960

2,461

C1. Models arising from GPEPimplementation disseminated andinfluence Education Planning andManagement approaches identified.

No model. At least two modelsidentified andshowcased

GPEP Model 1:Proof of concept forM&E strategyapproved

GPEP Model 2:School constructionstandards approved

Endline data for the first two indicators (A1 and A2) were collected through EGRA/EGMAassessments and Head teacher, teacher and school supervisors and inspector’s questionnaires.The data for the last three indicators (B1, B2, C1) is taken from the June 2017 GPEP progressreport. Target B2 was not achieved because two schools (Ayii and Dereto) in Eastern Equatoria


closed due to conflict. The GPEP progress report Jan-Dec 2017 states that students from theseschools constitute a significant number of total number of dropouts in GPEP supported schools.

4.1.1 Proportion of students who, by the end of three grades of primary schooling,demonstrate that they can read and understand the meaning of grade level text (outcomeindicator A1)

At endline, 17% of P3 students (at 95% CL 8%-26%)5 scored over 50% (9 or more of 16 points)on the Early Grade Reading Assessment (EGRA) subtask for reading comprehension, and 48%of students (at 95% CL 39%-57%) received zero scores. By comparison, at baseline 43% ofstudents scored over 50%, with 45% receiving zero scores. The result is statistically significant(p <.001, 95% confidence level, margin of error +/- 9%).

● Possible explanations for this include: Endline sample selection. The endline was limited to10 students from each of 10 GPEP supported schools, whereas the baseline targeted 16students across each of 25 GPEP supported schools. Although endline schools wereselected to represent higher and lower achieving schools in each area, and data herecompare only those schools included in both baseline and endline,

● Problems with baseline administration. Baseline figures recorded are high, with 100% passrate in reading comprehension recorded for LKS schools and 91% pass rate in NBEG. Thisis higher than what similar studies have found and suggests that students may have beenselected by the teachers for the assessment exercise. The endline results are in line withwhat other programmes have found in South Sudan.– The South Sudan Teacher Education Programme (SSTEP) Early Grade Reading

Assessment Report assessed 522 P2 students in July 2013 found ‘very poor performanceon nearly all literacy subtests among tested students.’ On average, students answeredonly 2% of reading comprehension questions right, with 97% of students receiving zeroscores.

– BRAC assessed 2415 P1 students in June and July 2015. On average, studentsanswered only 0.82% of reading comprehension questions right, with 99% of studentsreceiving zero scores.

– Endline data aligns with Montrose International, who administered EGRA and EGMA inlate 2015 and early 2016 to a convenience sample of 250 P3 students. On average,students in the Montrose sample got 14% of the reading comprehension questions right,with 45% receiving zero scores.

Possible general explanations of students’ poor performance include:

English was the language used to administer the literacy and numeracy at baseline and endline.The baseline report noted this as a limitation of the survey, and identified that students,particularly those returning to South Sudan, were struggling to adapt to use of English in theclassroom when it was previously Arabic.

As at baseline, the difference between literacy and numeracy scores suggest that significantdifferences in the quality of teaching may persist between literacy and numeracy. The baselinereport noted that all subjects in lower primary including English were taught in local languages,and teachers did not receive English as a Second Language training. Several respondentsnoted that teachers in school struggled with English language proficiency, and this may affecttheir ability to teach the subject.

5 Estimated total population size of P3 students in the 10 schools is 1,350 students. Enrolment figures were not broken down by grade.


Basic literacy outcomes on letter sounds and word sounds (phonics) increased from baseline,whereas comprehension metrics dropped. One respondent provided the potential explanationthat the South Sudanese education system emphasizes rote learning but not critical thinking,and so students may perform poorly on exercises where they are asked to identify orextrapolate information from text. Several respondents noted that teacher training was providedrelatively late in the cycle, and there may have been insufficient time for teachers to put learninginto practice.

Tables of all comparable EGRA and EGMA subtasks at baseline and endline are presentedbelow.

Table 2: Pass rates of GPEP students on comparable EGRA subtasks, baseline vs.endline

EGRA EES LKS NBEG WRP WES GRANDTOTAL

Base End Base End Base End Base End Base End Base End

Familiar sounds 50% 60% 50% 40% 82% 75% 60% 30% 42% 85% 57% 58%

Unfamiliar words 8% 60% 10% 40% 18% 70% 0% 65% 8% 10% 9% 49%

Reading comprehension 4% 20% 100% 5% 91% 35% 40% 25% 31% 0% 45% 17%

Listening comprehension 19% 65% 100% 10% 91% 45% 33% 25% 50% 55% 54% 40%

N resp 26 20 10 20 22 20 15 20 26 20 99 100

Table 3: Pass rates of GPEP students on comparable EGMA subtasks, baseline vs.endline

EGMA EES LKS NBEG WRP WES GRANDTOTAL


Number Identification 85% 100% 91% 90% 77% 95% 88% 100% 70% 80% 80% 93%

Ordering and place value 69% 90% 82% 85% 73% 90% 63% 100% 43% 70% 65% 87%

Addition & subtraction 81% 95% 91% 85% 95% 90% 81% 95% 61% 80% 80% 89%

N resp 26 20 11 20 22 20 10 20 23 20 92 100

These findings were validated by comparing them to three other projects that are known to havecollected EGRA and EGMA data in primary schools in South Sudan. Of these, the data


collected by Montrose International in early 20166 is the closest in terms of grade level (also P3)and timing (administered early 2016), but it is important to note that this was a conveniencesample and is not necessarily representative of all non-GPEP supported schools. In calibratingits research instruments, Montrose found that on average students answered 14% of readingquestions correctly, with 45% of students receiving zero scores.

4.1.2 Perception index for Teachers and Head Teachers on School Leadership(outcome indicator A2.1)

The index score representing teachers’ perceptions of head teachers’ leadership increased frombaseline to endline. At baseline, teachers on average agreed or strongly agreed with anaverage of 77% of leadership statements.7 At endline, teachers on average agreed or stronglyagreed with an average of 85% of leadership statements. The difference in the aggregate indexscore is not statistically significant when the Bonferroni correction is applied. See Annex C forthe full comparison.

Although the perception index registered an overall increase, the number of positive responsesdeclined from 65% to 10% on teachers’ salaries being paid on time in all five former states.Responses on school vision also declined somewhat from 84% to 74% from baseline to endline.

It should be noted that 9 of the 10 schools included in the endline also benefitted from schoolmanagement and supervision training provided by the GESS programme.

4.1.3 Perception index for School Inspection and Supervision (outcome indicatorA2.2)

The index score representing teachers’ perceptions of school inspection and supervision alsoincreased from baseline to endline. At baseline, teachers on average agreed or strongly agreedwith an average of 79% of statements.8 At endline, teachers on average agreed or stronglyagreed with an average of 85% of leadership statements. The difference in the aggregate indexscore is not statistically significant when the Bonferroni correction is applied. See Annex C forthe full comparison.

Although the perception index registered an overall increase, the perception index decreasedfor former Eastern Equatoria state.

CECs overall also agreed less with the statement that School Development Plans had beenenhanced by the recommendations of school inspection and supervision visits from baseline toendline.

4.2 What worked and what didn’t in GPEP Component A: National SystemsStrengthening

4.2.1 Strengthening literacy and numeracy learning in primary schools

The literacy and numeracy sub-component was implemented under the technical support ofMontrose Consultancy in technical collaboration with GESS and Room to Learn. The USAID

6 Montrose International (Sept 2016). South Sudan Early Grade Reading and Mathematics Assessment Report: The development ofteaching and learning materials to ensure improved literacy outcomes among primary school learners and to initiate a system ofperiodic assessments to determine progress against specified benchmarks among early grade learners in South Sudan. Ref:RtLNAT006-Montrose.

7 This figure is reported at 83% in the programme outcome table because a different weighting was used which gave equal weight tosimilar statements on school vision. The adjusted baseline figure using the endline methodology is 77%.

8 Tis figure is reported as 70% in the programme outcome table because a different weighting was used. The adjusted baseline figureusing the endline methodology is 79%.


GPE Programme Report (2017) states that following the conflict in December 2013 it becameimperative to adopt a conflict-sensitive, evidence based process to inform language selection forLiteracy and Numeracy approaches. Language selection optimized mother-tongue instructioncoverage while maintaining inter-ethnic harmony and the principle of ‘Do No Harm’. Based onthis process MoGEI approved the use of five languages (Bari, Dinka, Nuer, Toposa, and Zande)spoken by 65% of the population. Stakeholders from the Ministry reported that they weresatisfied with the languages selected and the process of selection, however there wereindications that this had also caused great delays.

The approach to developing the Literacy and Numeracy (L&N) Strategy, L&N Assessments andL&N Kits was evidence based. L&N Kits were developed for teachers. International and nationaldevelopment partners, organisations and individuals including Summer Institute of Linguistics(SIL) and Yei Teacher Training College (YTTC) were engaged to ensure tools were designed tomeet both contextual and intellectual needs as well as international standards. Whilstrespondents noted that the materials were of good quality, concerns were raised on their abilityto have impact at school level. Insufficient training for teachers in relation to the scope of newknowledge and behaviour change needed to teach with the new Literacy and Numeracy Kitsand no systemic capacity development for State and Payam level officials were perceived asweaknesses of the intervention.

The refreshed GPEP document in 2014 increased focus to include curriculum development. Itwas strategic to build upon momentum of the South Sudan Curriculum Framework and developsubject overviews and syllabi for all subjects from ECD through to end of Secondary education.In 2017 unspent funds were re-allocated to textbooks under this sub-component in support ofthe national curriculum reform. Respondents from the Ministry and the education sectoridentified developing a national curriculum as one of the main achievements of the GPEP.Stakeholders noted that the curriculum was well written, reflected South Sudanese culturalidentity, and was outcome-based. But some noted that there was a gap between developmentand roll-out of the curriculum and textbooks. 80% of teachers and 60% of school inspectors andsupervisors reported that basic literacy and numeracy kits were available at schools. This isreflected in the Action Research, which noted that the new curriculum was not in all schools andwhere it is in school some teachers have difficulty interpreting it. However, some respondentsfrom the Ministry and implementing partners gave evidence that that teacher syllabi andmaterials have helped teachers to prepare lesson plans, teaching notes and improve teachingpractice.

4.2.2 Strengthening primary school leadership, school inspection and supportivesupervision

Primary School Leadership, School Inspection and Supportive Supervision Programme (PSLP)was implemented under this component by a consortium led by Save the Children whichincluded Yei Teacher Training College, Juba University, and Windle Trust. Primary SchoolLeaders, School Inspectors and Supervisors (PSL) Strategy and Standards were developedfollowing a situation analysis and mapping of school leadership, inspection and supervisionacross South Sudan. Standards and Strategy were approved by MoGEI and disseminated toschools. 45% of teachers reported that printed copies of PSL Standards and Strategy wereavailable in their schools. Less than half of teachers in former WES and WRP states reportedthat these were available.

PSLP included both basic and accredited training courses, designed and implemented byGPEP, accredited by University of Juba and endorsed by MoGEI. Accredited courses aimed toprofessionalise the school leadership, supervision and inspection roles and provide a nationalstandardised approach to school leadership. However, whilst the course certification was well


received, several stakeholders reported that candidates used these to apply to other non-teaching jobs with NGOs or the UN.

Training of trainers was conducted at State and national levels in partnership with MoGEI, Stategovernment, development partners and Teacher Training Institutions (TTIs). It aimed to workclosely with government officials to develop a cadre of trainers who can sustain the training afterGPEP. However, some respondents noted during key informant interviews that the governmentofficials who were selected as trainers lacked sufficient experience to be able to deliver highquality, participatory training. It was also noted that whilst a collaborative approach betweendevelopment partners was positive in theory, in some cases poor communication andcollaboration led to confusion in roles and responsibilities when administering the training.

The overall training target was met with 8970 school leaders trained and 696 inspectors andsupervisors trained. Several challenges were reported with trainee enrolment and participation.97 participants dropped out for varying reasons including insecurity and displacement. SomeStates Ministry of Education sent in participants as replacements to those who had attended thefirst phase of the programme but could not make it to Juba in the second phase, however thiscreated inconsistencies and these participants had knowledge gaps from missing the firstphase. Selecting participants that met the selection criteria set by the Ministry of GeneralEducation and Instruction and the University of Juba proved challenging and was notconsistently adhered to. Some respondents felt the process of selection was not fullytransparent. In some instances, participants who met the criteria were Arabic speakers andcould not engage as fully in an English led course - suggesting course delivery in locallanguages should be considered in future. The original participant target of 20% females wasambitious and revised to a target of 13% females. This was met in the school leader training butonly 11% of trained inspectors and supervisors were female. This is due to very few females inthese roles.

Progress reports and stakeholder interviews evidence that delays occurred at several stages ofPSLP; finalising head teacher and inspector standards, producing training material anddelivering training. Protracted contract negotiations contributed to these delays, in some areasproposed training could not take place due to insecurity and training needed to be re-located tosafer locations, the accredited training logistics were changed to charter flights for participantsto travel to Juba. Mass production of the training modules and manuals was delayed due tolimited availability of quality editors, designers and publishers in South Sudan.

Several respondents noted the short duration of the training (two-days for school leaders and 5days for inspectors), training was dictated rather than participatory, and that more follow-up andin-service mentoring needed to be done. They noted that training seemed like it was left to theend of the programme to implement and seemed rushed. The anticipated consequence of this isthat participants will be unable to translate what they have learned into practice. This issupported by the Action Research which found that Head Teachers are not supervisingteaching and learning processes and require capacity building in school management andspecifically supporting the professional conduct of their teachers.

Despite some challenges at output level, at outcome level, leadership training had positivemeasurable results. Government staff and school inspectors both observed better schoolmanagement, clearer roles and responsibilities, engagement with PTA, starting enrolmentcampaigns, improving lesson planning and delivery in classrooms. Teachers in GPEPsupported schools rated their head teachers more positively on the leadership index, withteachers on average agreeing with 85% of items on the index at endline, compared with 77% at


baseline9. In aggregate this difference is not statistically significant when the Bonferronicorrection is applied; however, the increase in teachers attending to their classroom dutiesregularly are significant at p < .05. (See Annex C).

4.2.3 Strengthening sector policy development, strategic planning, monitoring andmainstreaming gender responsiveness

Key achievements in this sub-component include development and approval of EnglishLanguage Policy Framework, Annual Joint Sector Reviews, M&E Strategy developed, approvedand implemented. Ministry and Development Partners confirmed that each of these had beenpositive national developments and GPEP had played a key role in bringing them to fruition.However, key informants were not able to share further information on specific elements ofGPEP success or failure within each of these.

4.3 What worked and what didn’t in GPEP Component B: Community andSchool-based service deliverySchool construction is presented as a separate section given the technical nature and largeinvestment made in this component.

4.3.1 Providing learning support services to primary schools

This Component was designed to support operationalization of the policies and materialsproduced through activities in Component 1. Sub-component 2.1 aimed to provide support toschools (predominantly in literacy and numeracy) through two government structures -Supervisors and County Education Centers (CECs).

Inspectors and Supervisors

At output level, 80% of head teachers reported their school was inspected more than once inthe past year. Both head teachers who did not were from former Warrap state. 100% of headteachers reported receiving inspectors or supervisors from payam level. The frequency of thesevisits varied from once every two weeks to once a term, with once every two weeks being themost common response.

From the inspector and supervisor survey, 85% of inspectors and supervisors reported they hadattended a professional course (not limited to GPEP training). Topics included teachingtechniques, lesson planning, English, maths and other subjects, and supervision. Respondentsidentified leadership, administration and management, instruction and supervision as desiredareas for further training.

CECs

The programme baseline showed that CECs needed capacity building both in terms of humanresources and physical infrastructure. Yei Teacher Training College (YTTC) was subcontractedto support five CECs and two TTIs with office equipment, management capacity building andliteracy and numeracy training for teachers. Respondents surveyed at endline noted someareas of improvement but felt that CECs still have limited functionality, due to low capacity andlimited operational funds.

The number of CEC staff has increased from an average of 10 at baseline to 12 at endline,however ratios of professional to support staff vary widely between CECs. One CEC had 6

9 The baseline score is adjusted because a different weighting was used to calculate the leadership index at endline. The baseline scoreoriginally reported was 78%;


times as many professional staff as support staff, while another had 4 times as many supportstaff as professional staff.

Evidence of improvement of CEC facilities was mixed. On the CEC survey, NBEG and WESstates reported that a majority of infrastructure improvements – administrator and staff offices,filing cabinets, computers, tables and chairs, storeroom, and meeting rooms – were inadequateor very inadequate. All CECs reported that male and female latrines were inadequate. Cleandrinking water was adequate in two of the CECs, EES and WRP. NBEG, WES, and EESresponded that solar power was very adequate, while LKS and WRP responded that they werevery inadequate. A majority of CEC staff stated that resources for in-servicing teachers,resource books, curriculum documents, subject syllabi, and teachers’ guides were inadequateor very inadequate. Overall the five centres were inadequately equipped in terms of the physicalresources perceived necessary for the CECs to undertake their roles effectively. This shows amarginal improvement from baseline - at baseline the situation was worse with overall ‘veryinadequate’.

The CEC survey revealed that one CEC is not functional and does not provide any support toschools. The remaining four have provided training to PTAs, Head teachers and teachers.Information was not provided on the frequency or duration of training. The content of trainingvaried slightly but was in line with building capacity of PTA, Head teachers and teachersrespectively to perform well in their daily roles and responsibilities. 75% of inspectors andsupervisors agreed that schools in their jurisdiction receive consistent support in provision ofteaching and learning resources from the Country Education Centre. However, a majority ofinspectors and supervisors in former WRP state disagreed with this statement.

Implementer interviews indicated that the intervention had been rushed with implementationstarting in May 2017 and ending in December 2017. There was a perception that CEC capacityis extremely weak and requires more sustained support to achieve a level where CECs operateindependently and have the necessary knowledge, skills and behaviours provide quality supportto teachers and schools. Implementers felt that GPEP started seeds of change in some CECs,especially in governance and management but this needs to be built upon.

4.3.2 Support to School Management Committees (SMCs)

GPEP collaborated with MoGEI and implementing agencies (GESS, IMED, Room to Learn) toprovide a unified approach to build capacity of SMCs as the functioning governing body ofprimary schools. Within this GPEP delivered training to SMCs in 25 GPEP supported schoolsusing materials developed by GESS. There was some evidence from implementing partnersthat coordination between partners hadn’t been completely coherent and there was replicationof efforts and lack of cohesion between the different implementing agencies.

A key role of SMCs in South Sudan is to support the Head Teacher to lead the SchoolDevelopment Planning process in an inclusive manner that leads to improved quality education.The June 2017 GPEP progress report found that only 32% of GPEP supported schools reportedfunctional SDPs (missing the target of 60%). Since June 2017, the Action Research referred toSDPs being in place. In the endline survey 100% of Head Teachers and Teachers reported thatthey had a SDP and that it had been used or referred to within the last week. Respondents fromthe Ministry observed that school management techniques are being put into practice – i.e.posters on the wall with qualifications of teachers, SMC/PTA membership, master schedule,school clubs.


4.3.3 Improving physical infrastructure to enhance learning

25 Model Schools were constructed across five States and provided with furniture. 90% ofteachers reported that school furniture was available in their schools, and all teachers reportingthey were available also reported that they were being used. Three-quarters of teachers saidthat desks for pupils and desks for teachers were very adequate or somewhat adequate.Physical infrastructure is a significant component of GPEP, therefore a full analysis of schoolconstruction is provided in Section 6.

4.3.4 Support to out-of-school children (OOSC) and young people

GPEP support to OOSC was revised several times and significantly delayed. The 2012Programme document set an ambitious goal to support 15,000 OOSC by delivering theAlternative Education System (AES) to the selected cohorts through i) needs assessment ofOOSCs in selected counties, ii) training of ALP teachers in literacy and numeracy skills, iii)provision of ALP supplies and learning/teaching materials. In the refreshed document in 2013this was changed to “Support to out-of-school children and young people” and in the 2016 re-alignment it was changed to “locate OOSC in 25 GPEP school catchment areas and provideAES classes in GPEP supported schools for overaged learners”. In 2017 GPEP supportedOOSC through UNESCO who managed four activities: 1) Updating 2013 OOSC Study; 2)Support to Community Girls’ Schools (CGS); 3) Youth Skills Training; 4) IT support and furnitureto AES directorate in MoGEI.

Some development partners felt that the consequence of the revisions and delays was a rushedintervention that met targets but lacked sustainability. Due for completion in April 2018, theOOSC Study will present a national picture of the profiles of OOSC in South Sudan.Implementing partners noted that it is limited by a lack of credible data on OOSC.

GPEP supported 3950 girls and 2201 boys to enrol in Community Girls Schools (CGS) with 213female teachers employed and paid teacher incentives. A presentation at GEAR 2017 notedthat key challenges arose in monitoring and evaluating CGS at scale; maintaining attendancelevel of CGS learners, especially girls; limited resources to provide all needed learningmaterials; insecurity and displacement affecting learning. Recommendations from GEARpresentation included institutionalising CGSs into the public education system and involvecommunities in the management and support of CSGs. Interviews with implementing partnersconfirmed that there is concern over the sustainability of the initiatives and community andgovernment ownership was inadequately considered in the intervention design.

4.4 What worked and what didn’t in GPEP component C: Learning and SharingLessons?

4.4.1 Research: tracking the GPEP

In January 2017, AET was commissioned to undertake 7-month Action Research. Schoolcommittee members, teachers and government officials were supported by mentors to collectevidence of progress towards the targets of GPEP. The mentoring aimed to equip GPEPmembers with skills to use evidence to identify blockages to success and to problem solve toalleviate blockages and achieve results. The endline survey found some evidence of Headteachers and teachers in GPEP-constructed schools receiving mentoring and critical reflectiontools through the action research. There is limited evidence of school committee members,teachers and government officials problem solving to alleviate blockages and improve progressin GPEP.


In addition, GPEP commissioned a baseline assessment and the present endline research todocument progress against results and lessons learnt.

4.5 Teacher training and teacher performanceA common theme emerging from the evidence as inadequately addressed in GPEP, is teacherprofessional development. Respondents from MoGEI, donors, and implementing partners allnoted that most teachers in South Sudan have a poor level of basic education and training.Many respondents identified this as the area of greatest need. They perceive a weak teachingforce as the biggest weakness in the education sector in South Sudan. 45% of teachersrecognised teacher training as the most important intervention for improving access toeducation and children’s learning.

From the GPEP endline survey, 60% of teachers had received no pre-service teacher training.Those who did receive training reported an average of 4 months of training (range 2-9 months).90% of teachers reported receiving in-service training. In-service training was an average of 8days of training per year (range 1-30 days).

Evidence collected in the endline suggests that teachers lack remedial training and basiclanguage skills. This means a short training desired to deliver a new skill or practice is likely tofail because the foundational skills are not in place. Furthermore, some of the new skills (suchas phonics or child centred pedagogy) require a significant shift in behaviour that is unfamiliar toboth the teacher and the learner and takes time to master. For these reasons short, targetedteacher trainings do not work effectively in the current context in South Sudan.

Implicit in the references to ‘more training’ is need for a national approach to strengtheningteacher performance that incorporates continuous professional development, motivation, reward(financial and non-financial) and accountability so that when teachers are trained they want toperform and remain in the profession. Several stakeholders suggested GPEP could support amore comprehensive approach to teacher development in the future, which was out of scope forthe current GPEP.

4.6 Has GPEP achieved leverage?While there is evidence that GPEP supported schools benefitted from other initiatives, there isno evidence that these schools benefitted from other initiatives because they were GPEP sup-ported schools, or that GPEP leveraged any outside funds.

Initiative Source % endline schoolsbenefitting

Teacher incentives EU IMPACT 100%

Cash transfers for girls GESS 100%

Other support NGOs 80%

Capitation grants MoGEI/GESS 70%

Feeding programmes WFP 70%

Additional support from NGOs included learning materials, WaSH, clubs, and teacherincentives. The two schools that did not report other support were both from former WRP state.

In interviews with the PTA/SMC, capitation grants and cash transfers for girls were the initiativesmost frequently mentioned.


4.7 Has GPEP achieved scale?Some Component A initiatives like the national curriculum and teacher and instructor/supervisoraccreditation can be considered to have a national scope and therefore were delivered at scale.

However, Component B initiatives were designed to be delivered at a smaller scale – reduced toModel Schools only in order to provide models for quality service provision that can then beadopted nationwide.

Some Component B initiatives were scaled down from the initial design. The number of GPEPsupported schools was reduced from 40 to 25 due to increased costs of construction asdescribed in Section 6. In 2016 USAID withdrew from the partnership, this impacted deliverytimeframes and meant several interventions (support to OOSC, support to CECs, PSLP,Literacy and Numeracy teacher training) needed to be delivered in reduced time. Someimplementing partners reported that rushed delivery meant that they were not confident GPEPcould demonstrate a quality model. Thus the interventions (support to CECs, Literacy andNumeracy teacher training and PSLP) need more follow up support in schools to ensure theyare working in the pilot locations before they can be replicated elsewhere.

There is potential for taking interventions to scale, but perhaps not all at once. GPE2 may needto consider which initiatives to scale first, taking other donor interest into account. Interviewrespondents suggested that support to girls and children with disabilities, increased mentorshipand follow-up for school leadership training, additional follow-up for instructors and supervisors,extension of GPEP to the five other former states, infrastructure, community girls’ schools, andyouth skills education as areas that would all benefit from scale-up.

6

5

4

3

3

1

3

2

1

1

1

1

1

1

1

0 1 2 3 4 5 6 7

Cash transfers for girls

Capitation Grant

Teachers training

SMC/PTA Trainings

Pupils notes and text books

Teachers guides

School feeding programme

Soap and sanitary pads for girls (plan international,…

Teachers Incentives

Teacher incentives from World Vision

Teaching aids

School construction and furnishing

A mentor recruited by Africa Education trust.

Rural water development provided solar water pump…

School development fund from PTA.

Frequency of response

Res

pons

es

What support did the school receive over the past three years (no promptsgiven)?


5 Gender and inclusion

Documentary review and the endline survey agree that post 2016, GPEP included a genderresponsive approach. The GPEP 2016 realignment document introduced a strategic approachto mainstream gender responsiveness throughout the GPEP. It stated that all componentswould be subjected to a gender lens. This would include actively seeking female participation intraining, production of gender sensitive training materials and approaches as well as gendersensitive provisions in school construction. A new indicator was introduced into the resultsframework to help measure progress in this area: Mainstreaming of Gender responsiveness atschool level piloted in selected schools and states. The June 2017 Progress Report statedgender mainstreaming checklist has been developed and state-level consultative workshops torevise 2015-17 Girls Education Strategy is ongoing.

At baseline gender disparity was identified in pupil enrolment and female representation inschool leadership. This disparity continued to be an issue in the early years of the GPEP, asidentified by the Action Research. This section looks at gender parity in pupil enrolment andschool leadership as well as evidence of gender responsiveness at school and national levels.This evidence provides insight on the extent to which GPEP succeeded in reducing the gendergap and increasing girls’ education.

5.1 Girls’ enrolmentGPEP succeeded in increasing the proportion of girls enrolled in GPEP supported schools. Theproportion of girls enrolled in GPEP supported schools at baseline was 49%. The proportion ofgirls enrolled at endline was 54%. This was largely due to the increase in all-girls schoolenrolment, which saw an additional 818 girls enrol in school in total. It should be noted that theendline result does not demonstrate gender parity because there are fewer boys enrolled.However, given the overall lower proportion of girls in school in South Sudan, this is a positivecontribution towards gender parity at a national level. In addition, GPEP directly supported girls’enrolment through Community Girls’ Schools. There is insufficient data to be able to attributethe gains in girls’ enrolment directly to GPEP. During interviews, several stakeholders attributedincreased enrolment of girls in GPEP supported schools to cash transfers. Given the complexityand interconnected nature of the barriers to learning that girls face, it is unlikely that oneinitiative alone can increase access to school for girls, moreover it is the combination ofinitiatives addressing the gender barriers that serve to improve access. In this respect GPEPhas played a role as will be discussed in the remaining sections.

5.2 Female representation in school managementFemale representation in school management was measured in SMCs, PTAs and PSLP. GPEPset a target to train 13% females in PSLP. The low demographic of females in headteacher,supervisor and inspector roles made this challenging. The target was narrowly missed with 11%inspectors and supervisors trained being female (GPEP Progress Report, June 2017). ThePSLP trainee pool was widened to include senior female teachers. This ensured that femaleswere present and indeed the target was met at 13% females. Stakeholders verified that thefemale participants were actively engaged in the training and that gender responsive materialsand approaches were used, for example using gender based case studies to challengestereotypes. The number of female members in PTAs and SMCs in GPEP supported schoolswas measured and found to have increased from baseline to endline (as shown in the chartbelow). Of those measured, the majority of female members of SMC and PTA had received


training (70% and 75% on average respectively) and most reported that women play an activerole in their SMCs and PTAs.

Figure 4 Average percentage of females in PTA and SMC at baseline and endline

5.3 Promoting gender equity in schoolsThere is evidence of GPEP promoting gender equity in schools in three main areas: 1) Genderresponsive infrastructure; 2) School based interventions to support girls’ education; 3) Teachingand learning materials produced with a gender lens.

It was commonly perceived by both school community and national stakeholders that GPEPschool infrastructure and facilities are inclusive and meet the needs of girls. Examples citedinclude; latrines for girls, fence around the school making it a safe space for girls, WASHfacilities enable girls to attend school during menstruation whilst before they would stay athome.

GPEP subcontracted AFOD to provide school based initiatives that directly address genderbarriers and support girls’ attendance and learning in school. AFOD described theirinterventions to include locally made sanitary pads which both enabled girls to attend school aswell as providing a source of economic empowerment for women in the community. As well as‘talking schools’ whereby some schools received paintings on the school walls addressinggender barriers, for example advising girls’ not to visit teachers houses. However, theseinitiatives were not widely recognised by the school community or national stakeholders duringthe endline survey.

It was widely acknowledged by implementing partners that teaching and learning materials wereproduced with a gender lens. It was reported that consultants producing the materials had anunderstanding of the importance of gender responsiveness and actively sought to use genderresponsive language, photos of girls as well as boys and to challenge stereotypes wherepossible.

3429

37 38

05

10152025303540

PTA SMC

Ave

rage

per

cent

age

of fe

mal

es

Average percentage of females in PTA and SMC at baseline and endline

Baseline Endline


The Action Research also reported that during the observed lessons, there was no evidence ofgender stereotyping. Examples of gender equality included girls and boys accessing textbooksand girls and boys participating in learning and answering questions.

The extent to which gender equality outcomes are driven or supported by stakeholders in GPEPsupported schools is difficult to ascertain. There is evidence of PTA and SMC understandingthat girls’ education is a priority and that girls in the community should be encouraged to attendschool. There is evidence of the school community understanding the specific gender barriersthat both girls and boys face that prevent them from coming to school. There is limited evidenceof the GPEP supported schools challenging the deeply embedded social norms within theeducation system that cause the barriers to girls’ learning once in school.

Whilst there is evidence that GPEP was gender responsive and promoted gender equity at aschool level, this is overshadowed by a common perception amongst interviewed stakeholdersthat ‘GESS does gender’. This means that whilst GPEP was gender sensitive, it hasn’t alwaysbeen highly visible in promoting gender equity. The perception of some stakeholders that ‘GESSdoes gender’ also led some government stakeholders to conclude that GPEP doesn’t need todo anything about gender because GESS covers it. Given the evidence that girls’ learning islower than boys’ and the complex barriers affecting girls’ exist as social norms throughoutsociety– in government, community and school levels, there is need for all educationprogrammes to align and play a highly visible role to challenge gender stereotypes and wherepossible alleviate gender barriers to education.

5.4 Gender mainstreaming at a national levelThe endline evaluation showed GPEP mainstreaming gender at a national level in three areas:1. GPEP supported Girls’ Education Strategy2. Gender responsive curriculum3. Gender factored into the M&E strategy.

It was identified that GPEP had helped bring a gender focus to MoGEI at national level. In orderto build upon these foundations, interviewed stakeholders identified that more needs to be donein the next phase of GPEP to build capacity on gender within MoGEI and ensureimplementation of the new Girls’ Education Strategy.

5.5 InclusionMeeting the needs of the most disadvantaged children is high on State, national andinternational agendas. There was limited evidence of children with disabilities attending GPEPsupported schools. The endline survey found on average, schools have 3.7 children withdisabilities enrolled, (ranging from 1-10). However 70% of Head Teachers reported that GPEPschool infrastructure helped children with disabilities enrol and participate in school. There wasevidence gained throughout the endline survey that GPEP supported children who hadpreviously been excluded from education. This included providing support to Community Girls’Schools and Alternative Education System (AES) as well as conducting an OOSC study toidentify the profiles of children who are ‘left behind’. Some teaching and learning materials wereshared with the POCs. By the very design of the Child Friendly Schools, school infrastructureand facilities are inclusive. Whilst there is evidence that these initiatives were in place there isno evidence gathered to show the impact of this on the most vulnerable.

Stakeholders at national level also raised concerns about groups of children who they felt werethe most marginalised in South Sudan and not had their needs adequately met by GPEP or anyother education programme. This included children with disabilities; children from pastoralist


communities, especially boys; displaced children. All 10 HTs reported receiving displacedstudents over the last year. Schools reported receiving 60.7 displaced students on average, withone school in LKS and one in WES receiving 200 displaced students each. There is need toimprove education for displaced children who are in GPEP supported schools but may havespecific needs that are not being considered and displaced children who are not supportedinside POC sites or through other EIE programmes.

There is an opportunity to build upon the OOSC study which evidences the profiles of the mostmarginalised children in South Sudan and design a second GPEP that specifically addressesthese needs.


6 School construction

School construction is one of the major sub-components of the GPEP. Under component B, theSchool Infrastructure Improvement Programme supported national efforts to deliver quality basiceducation services in line with the GESP. The budget for this sub-component was USD 17.3million. As grant agent, UNICEF has in-built expertise in school construction in South Sudan. Ata global level, UNICEF has a construction unit within the Supply Division in Copenhagen aimedat regulating and monitoring and providing support to country offices on planning, procurementand management processes related to construction, aiming at strengthening local capacities forthe procurement of construction services. UNICEF appointed a Construction Engineer, who wasresponsible for overall project management of the construction component of the GPEP,ensuring effective project management, implementation and oversight of the Works. In addition,the services of independent QA consultancy firms were solicited for, to develop plan-designsand provide full-time site supervision for the Works. By December 2017 GPEP had met theprogramme target by constructing 25 primary schools in 5 States and publishing SchoolConstruction Standards.

This section reviews the school construction components of GPEP. The section is structured inthree parts. The first part looks at GPEP outcomes and the extent to which constructionimproved education access and quality. The second part looks at the Model School approachand considers the attributes determining the success or failure of this output. The third partlooks at sustainability, considering community ownership and alignment to national systemstrengthening. The review will contribute to answering relevant questions in the overallframework. Annex B maps the evaluation questions to where they are answered in this section.

6.1 To what extent and in what ways did school construction achieve desiredoutcomes?

6.1.1 Increasing access

The endline survey revealed a common perception amongst stakeholders that schoolconstruction may increase enrolment in schools. This perception was backed with reasonedarguments. For example, several respondents noted that there is clearly a shortage of learningspaces in South Sudan and GPEP directly addressed this demand by providing new schools,therefore this leads to more children enrolling in school. A second commonly cited reason wasthat more people now see education as attractive because of visibly high quality and inclusivelearning environments.

The perception is that GPEP supported schools alleviate barriers to access especially for themost vulnerable. For example, the fencing makes it safe for children to attend, new buildingsmake the distance to school shorter for some children and the WASH facilities enable girls toattend who would have previously stay at home. Stakeholders at both school and national levelattributed this to increased enrolment in GPEP supported schools. To some extent this issupported by evidence of increased enrolment in GPEP supported schools. As noted in Section4 there is evidence that more children are registering in GPEP supported schools. Butdisaggregated data was not available to analyse whether these children were previously out ofschool or enrolled in another school. There was some evidence in stakeholder interviews bothat school and national level, that children are leaving the neighbouring schools. Thus, the datamay just be showing a shift in pupil populations rather than new learners accessing school.More data would be needed to draw conclusions on this. Where stakeholders described


increase in enrolment and data shows enrolment increases is the extent to which this isattributable to school construction alone is not known and further research would be needed.When asked, many stakeholders explained that it is not GPEP alone that increased enrolment,cash transfers are commonly cited as a source for attracting more children to school.

6.1.2 Improving quality education and learning

The endline survey found a common perception amongst stakeholders that quality schoolconstruction is facilitating quality education and improving learning. This is not yet backed withrobust evidence. GPEP endline survey demonstrates increases in some areas of learning anddecreases in other areas. Where gains in learning are demonstrated there is insufficient data tobe able to attribute it to one component of GPEP. Nonetheless, the stakeholder interviewsprovided rich information on how quality GPEP school construction is being perceived toimprove the quality of teaching and learning. A commonly cited argument is that GPEPsupported schools provide a more conducive learning environment. Previously children werelearning under a tree and were subject to wind and rain. They could not attend school duringpoor weather and learning was often interrupted. Learning inside a building means they canlearn all year round. It is reasoned that increased learning time can lead to increased learningoutcomes. Secondly, the school building and fenced compound now provides a safe space forlearning. It is argued that when children feel safe they are better able to learn. Lastly, severalstakeholders explained that high quality school infrastructure increased motivation amongstboth teachers and pupils, they asserted that increased motivation led to better quality teachingand learning. Whilst international research supports that these factors can contribute toimprovements in learning, further research would need to be conducted in South Sudan toascertain if these factors are true e.g. are teachers more motivated in GPEP supportedschools? Do children feel safer and more able to learn in GPEP supported schools? And doesthis create improvements in learning in this context?

6.2 Was the school construction model approach successful?Throughout the endline survey, stakeholders commonly cited school construction as one of thesuccessful outputs of GPEP. There is agreement that school construction is relevant andneeded in South Sudan. However, the endline survey revealed numerous concerns over theinfrastructure model, which involved investing significant resources in only 25 Model Schools.Many stakeholders gave examples to show that this model was not the most efficient andeffective in the current context of South Sudan. This section looks at three core elements - 1.Site and school design; 2. Site and school planning; 3. Cost of Model Schools, to analyse theextent to which the Model School approach was effective and the attributes that are thedetermining force behind the success and failures.

6.2.1 Site and school design

The child friendly school package consists of 8 classrooms, one administration block (withteacher room, store and head teacher office), separate latrine blocks for girls and boys withdisabled latrine and handwashing facility and a teachers’ block with female and male teacherlatrine, a kitchen with a food store, a school fence, safe water point, school furniture, basic sportequipment and basic levelling and landscaping of the school site.

To evaluate the above package, a physical learning environment and school constructionassessment was conducted in ten schools. Rumbek Girls school in LKS was found to be non-operational because school furniture had not yet been provided and thus responses werecollected from nine schools. The assessment was conducted with two children from eachschool. The tool systematically assessed the school facility across 23 different environmental


aspects grouped in five basic domains: aspects of indoor environmental quality, of socialclimate, of organizational structure, of outdoor environmental quality, and build quality. The toolspecifically aims at post-occupancy end-user appraisal i.e. the children, and how theyexperience their new school in many ways and what they think was done well and where thereis room for improvement. Hence, it evaluates both physical and social school environment.

In viewing images of the various school buildings and schoolyards in Rumbek and Pacong it issignificant that the schoolyards have limited to none undergrowth, vegetation, shrubs norsmaller trees. According to the bills of quantities (preliminaries), all undergrowth and small treeshad to be cleared. This results in a schoolyard and school building with few large trees but formost part not shaded. During the rainy season this could cause a muddy schoolyard and duringthe dry season it will become very dry with dust flowing around which in turn is cause forrespiratory disease and eye infections. Regarding the structures: The overall built quality ofconstruction is acceptable within the context of South Sudan, however, it is noticeable that theexteriors that were selected are on the darker side (dark blue and green), this will be cause foraccumulation of heat of the wall during the morning and afternoons. The roof cover is of a light,reflection colour which is bio-climatically correct. Furthermore, regarding indoor-classroomspace, the main feature is a blackboard in front however, no pin-boards or blackboards on theremaining walls are noticeable nor a small space for storage of teaching materials. There isroom for some additional child friendly elements in the classroom to facilitate both learning andteaching.

The findings showed that the majority of respondents were satisfied with their school facility.Positive comments were relayed and supported in the PTA focus group, such as “it’s of modernstandard”, “it’s a permanent structure”, “it’s the best school in the county”. Across all 23 aspectsand 9 schools a concern was raised 30% of the time. The charts below show the overallfrequency and spread of concerns raised. The first chart shows total number of concerns foreach of the environmental aspects. The second chart shows the number of concerns perschool. These results are analysed in the section below.

Figure 5 Responses from students showing total number of concerns with environmentalaspects of their school

02468

101214

Num

ber o

f con

cern

s ra

ised

Environmental aspects

Total number of concerns per environmental aspect

Total number of concerns


Figure 6 Responses from students showing total number of environmental concerns perschool

The results from the school assessment were triangulated with evidence from PTA focus groupand stakeholder interviews. Across the three sources, three common school design concernswere raised: 1. No school garden; 2. Inadequate private space; 3. Inadequate sports field. Anadditional two concerns were raised in relation to design that, after analysis, reflect not schooldesign but external issues: 1. Overcrowding in classrooms; 2. Security

School food garden: the absence of a school food garden is a missed opportunity sinceinvolvement of children and teachers in the planning and tending to a school garden canenhance resilience, resourcefulness, ownership, pride and practical life-skills. The design of theModel School did not factor in the development of school landscaping and more in particular ofschool gardens (as evidenced in bills of quantities where tree planting nor landscaping isitemised/costed). School food gardens for vegetables, seeds and fruits for use in the newkitchen and to complement children’s diet were not part of the scope of the Model Schoolpackage. In addition, the planting of trees to provide shade to the open schoolyard - whenchildren play and socialise outdoors – and classrooms was also not part of the scope of theproject but instead was left to the community.

Inadequate private space: there is no area factored into the design of the Model School forone-on-one private discussions, between girls or between a girl and a teacher or schoolmentor/counsellor which is important for girls attending a school and as a basic right with regardto child protection. In addition, the planning of (semi-) private, in/outdoor spaces is useful forprivate reading areas or small group learning activities. One could think here of basic, outdoorspaces or pavilion having low (1m maximum) and roofed, like a grass-thatched tukul ordesigning wider verandas up to 3 meters with fixed benches to provide shaded, semi-outdoorareas for reading and/or socialising

Inadequate sports field and playgrounds – several schools reported that they wanted afootball field. This clearly demonstrates importance of participatory planning and co-designingwith children during the conceptual design stage which goes beyond basic consultation or

0

5

10

15

20

25

30

35

Pariang SalamGirls

Pacong Torit 1 ToritModel

Naduru Namaiku Warrapprimary

MaperAgepN

umbe

r of c

once

rns

rais

ed

School

Total number of concerns per school

Total number of concerns


needs assessment. Learning through play and sports is a critical element for the well-roundeddevelopment of children.

Overcrowding in classrooms – was reported in some schools but notably not in all classes,usually just one or two per school. The overcrowding causes high ambient temperatures in theclassrooms and in addition an increase in the classroom noise levels. Furthermore, the inad-equate classroom space forms a hindrance to flexibility in classroom settings for different activit-ies (individual, small project group and large group learning). An overcrowded classroomis not conducive to learning and therefore undermines the gains of the Child Friendly Schoolsapproach. Reasons for overcrowding were due to 100+ children enrolled in a class, this is notjust a design issue, but also an issue of controlling the number of pupils per class. Thissuggests that what needs to be resolved not necessarily by GPEP, but by Government withdevelopment partner support, is how adequate numbers of well-built school infrastructure canbe constructed against an acceptable cost and relatively short time-line so that i) morecommunities are reached, and ii) improved planning monitors and provides school spaces forthe cohorts of children in the school-going age that enter primary school each year.

Security - four schools raised a security concern about poor fencing around the schoolcompound and intrusion of community members. The school fencing was of a high standard,but if the surrounding community wants to enter the schoolyard to fetch water, use of thelatrines or otherwise, they will find a way. This suggests that there is a link with communityownership and that services should be provided to both the school and surrounding communityin tandem so that the community also benefits from the development programme.

6.2.2 Site and school selection

Selection of benefitting States was based on the GPEP document with various factors of criteriato be considered i.e. both equity education indicators and feasibility indicator like probability ofsuccess and access. The selected States for the school improvement eventually came to:

● Eastern Equatoria State,● Western Equatoria State,● Northern Bahr El Ghazal State,● Warrap State and,● Lakes State

Five schools were selected from each State, creating a total of 25 Model Schools to beconstructed. Selection of benefitting schools was based on criteria, guided by the followingindicators:

● Selected education criteria (drop-out rates, primary completion rates, etc.);● Accessibility: Programme must be implementable within three years and in areas where a

good probability of success can be expected● Existing schools: sites must be existing schools under trees, in tents, roof-only learning

spaces, semi-permanent structures, with evidence of children and ongoing learning;● Targeting counties with high number of returnees and/or out-of-school population● Clustering: economies of scale can be made by using a clustering approach where selected

schools are within the same county or neighbouring counties served by an existing CountyEducation Center (CEC);

● Full primary cycle provision: support will be given to schools to upgrade these to the primaryP1 - P8 cycle;


● Provision of teachers: targeted schools to have the required number of teachers.

Some government stakeholders at both State and national levels commended the school siteassessment process. They felt it had been inclusive and they had been adequately involved indecision making. However, other stakeholders gave evidence to the contrary and suggested thesite selection had been unnecessarily political and had led to delays in construction. A commonconcern raised was that the number of schools built was too few and this causes exclusion ofsome communities. There was an expectation that the next round of GPEP should support theother States. Whilst any programme that doesn’t reach national scale is going to create somefeelings of inequality and exclusion, it is pertinent here because there is a strong feeling that thesame funds could have been spread more widely, building more schools, for more communities,at lower cost. This is discussed in more detail in the next section.

6.2.3 Cost of Model Schools

The GPEP document planned initially for 50 schools (2012) with a budget of USD 200,000 perschool for rehabilitation/upgrading with a total of USD 10 million. An element of maintenancetraining was foreseen and estimated at USD 300,000.

In 2013 programme refreshment, the scope of the number of schools was lowered to 40benefitting schools. Costing was undertaken on the status of unit cost for a school constructionpackage i.e. 8 classrooms, administration block, latrines for girls, boys and teachers and a safewater point (hand-pump borehole). Taking into account 40% inflation, the unit cost came to USD343,000. On this basis, it was calculated that 40 schools (23 funded through GPEP and 17funded through USAID) would be constructed.

However, a realistic cost analysis based on the GPEP physical infrastructure implementationplan (Nov 2013) and experience from the DFID programme (2012-2013), resulted in the scopebeing further reduced to 25 schools (200 classrooms). It was further realized that qualityassurance during project implementation was overlooked and not budgeted in despite that witha project of this scope and complexity, quality assurance is essential for cost-efficiency, timelyand quality programme implementation. Hence, costs had to be factored in for quality assuranceservices for design, costing and construction supervision from site supervision to workscertification, granting substantial completion, defects-liability period (warranty period, 12months) and for granting of final completion.10

Due to the fragile and volatile context, contractors factor in a premium for risk of doing business,including inflation, poor weather, impassable roads and site access. Hence, a revised unit costwas established at USD 690,000. Projected was that actual cost of construction were within10% of the estimate. Hence, based on a total construction budget of USD 17,360,000 it wasprojected that 25 primary schools could be constructed with the available budget.

The actual cost breakdown was thus as follows:

● Total cost for 25 schools (direct construction), amounted to USD 10.71 million at an averagecost of USD 535,350 towards actual construction costs;

● Supervision costs for 25 schools was USD 1,336812.56 averaging to USD 53,500 perschool.

● Average cost per borehole per school was USD 20,000● Cost of furniture per school was estimated at USD 35,000

10 UNICEF South Sudan, Improvement to School Physical Infrastructure to Enhance Learning in ten USAID funded Schools 2014-2016Completion Report


Hence, the average cost per school based on actual costs, established through bidding wasconfirmed to be USD 643,850 (USD 535,350 + USD 53, 500+USD 20,000+ USD 35,000).According to the USAID Completion Report (2017), this figure was 93% accurate compared tothe revised projection of USD 690,000 per school.

The endline survey found that common perception amongst stakeholders is that the cost perschool is too high and therefore not replicable in South Sudan’s current context. Governmentstakeholders confirmed that there are insufficient government funds to replicate the model anddonors confirmed that they saw it as a model that was not scalable. This led to a commonperception that GPEP school construction had inadequately addressed the balance of accessand quality. There was an overwhelming preference amongst partners, implementers andgovernment at a national level that funds could have been used to provide more schools tomore children by reducing costs either through rehabilitating existing schools or making use oflocal materials and local human resources as well as community in-kind contributions.

An analysis of the cost drivers of GPEP construction shows that in order to keep the cost ofconstruction under control, one needs to aim at making appropriate design-choices, technologychoices (materialization), and delivery mode choices to find an optimum between these 3 coredrivers of construction cost.

UNICEF South Sudan, Improvement to School Physical Infrastructure to Enhance Learning inten USAID funded Schools 2014-2016 Completion Report explains that South Sudan’s currentcontext of combination of a vulnerable economy, high inflation, and underdeveloped localconstruction industry, lack of local building materials production and lack of local technicalcapacity has the implication that school construction leans heavily on foreign expertise,equipment, materials and skilled labour. This can be adjusted into another direction by having acommunity training and development approach to school construction of schools. This isdiscussed in more detail in the section below.

A more balanced community-approach to school construction which is linked to a robust qualityassurance system would have a lowering effect on the costs of construction while providingvalue for money in the context of South Sudan would therefore aim at:

A choice of design that fulfils pre-conditions and criteria of the design brief against anaffordable cost to the Government of South Sudan; a choice of delivery system that willmaximize the use of local labour and production of local materials for construction, and; achoice of technology which is appropriate for the context of South Sudan and will allow forcommunity-based modes of procurement and school construction in combination with vocationalconstruction-skills training, guided through locally-owned contracting/engineering firms or NGOswho serve as construction programme managers (CPM) per State to provide guidance,technical oversight, quality assurance and daily site supervision regarding the communityconstruction teams as well as have liaison and timely reporting (financial and technical) withUNICEF. The cost drivers are in-line with the construction standards as they fine tune theprocess of how and when to choose a specific design-procurement and construction package.However, the construction standards lean more toward foreign contractor-built (albeit with localSouth Sudan branch) but overlook that the root cause is lack of development of a localconstruction industry and building-cost index, local-materials production-and manufacturingindustry and lack of local skilled carpenters, masons and foreman to carry out constructionprojects to an agreed standard.


6.2.4 Community ownership

The GPEP endline survey asked PTA and SMC members who they think is responsible forsmall and large repairs in school. Their responses are shown in the graphs below. There is aclear division of ownership with daily maintenance and ownership being undertaken at theschool level but oversight and high-level management still viewed as the responsibility of thedonor (in this case UNICEF). The same opinions were also reflected in the HT and Teacherssurvey and in some stakeholder interviews.

Figure 7: Responses from PTA and SMC focus groups showing who they consider asresponsible for small school repairs

0 01

67

1

3

012345678

Num

ber o

f res

pons

es

Person , body or institution responsible for repairs

Who is responsible for small school repairs?


Figure 8: Responses from PTA and SMC focus groups showing who they consider asresponsible for large repairs

At a school management-level this presents a positive picture of school community takingownership for decision making and improving their schools. Teachers, PTA, SMC membersconfirmed that the community had been involved throughout construction process. It could beargued that this has helped the community to feel responsible and accountable for schools.However, there were some missed opportunities that if GPEP had achieved could have servedto further strengthened community ownership. There was no evidence of children taking part ina participatory school planning and co-design process. This is important because it is effectivein obtaining the priorities and needs of children in the schools and it will enhance the ownershipand pride children have in their schools. Furthermore, whilst community were consulted andinvolved in some ways with construction, there was no evidence of a systematic approach toengage the local community in supplying materials or providing skilled/non-skilled labour.Several stakeholders at national and State levels noted that greater effort could have beenmade to build capacity in construction within the community and that encouraging thecommunity to provide in-kind contributions would have strengthened their buy-in. This couldhave alleviated some of the design concerns and ensured that available funds were spent onthe most relevant school facilities.

6.2.5 National systems strengthening

The findings shown in the graphs above suggest that State MoGEI ownership of the schoolsneeds to be more visible. GPEP focus on National System Strengthening could have includedgreater efforts to address this. For example, the Action-Research Report of May 2017, statesthat some schools were completed but the formal handover to the MoGEI and school authorities

2 2

6

0

2

0

2

01234567

Num

ber o

f res

pons

es

Person, body or institution responsible for repairs

Who is responsible for large repairs?


had not taken place11. This gave the impression to PTAs and SMCs that the schools belongedto the donors.

For learning and teaching, the surrounding community and area around the school needs to bepeaceful and conducive to the learning and teaching. Disturbance in the form of civic unrest orprotracted insecurity, the latter which is the case in two schools had as implication that thesebrand-new schools are still closed. Different stakeholders see this as a concern as it is a wasteof resources and a loss in the number of children in school and learning. Some stakeholdersquestioned what could be done at a national level to rectify this.

A commonly cited success of GPEP in relation to national system strengthening is the SchoolConstruction Standards for South Sudan. Expected outcome of GPEP was support provided tothe Physical Infrastructure Department under the Planning Directorate to draft and disseminatethe minimum School Construction Standards to National stakeholders and State Ministries. TheGPEP Realigned Programme Results Matrix indicates by 2014 informal standards were appliedand target by 2016 the Standards to have been disseminated at National and State Level.Standards for school construction in South Sudan were drafted and published by the Ministry ofEducation in a joint effort with UNICEF. The standards are differentiated in three levels ofnecessity i.e. MUST: these are minimal spatial standards for a child friendly school; SHOULD:standards that give guidance on additional school spaces and quality that is encouraged and isin line with best practice, and; MAY, standards that give guidance on school spaces andqualities identified as beneficial if resources permit.

The above is clear and straightforward however regarding technical norms and standards, itgives limited insight in the selection of the core drivers as parameters that drive the cost ofconstruction upwards in a context of an underdeveloped local construction industry (and localbuilding materials production industry and limited local construction capacity).

Recommended therefore is that school construction standards are closely aligned to (potential)opportunities and facilitating local community efforts to provide inputs (in-kind or throughparticipation and labour) in the planning, design, construction and maintenance and repair ofschools.

11 Following substantial completion of construction, there is a joint inspection with QA and contractor and a list is made of all constructionissues that are outstanding and the contractor has to make good. When the certificate of substantial completion is granted to thecontractor after the make-good-period (normally 1-2 weeks). The school is ready for formal handover. Following formal handover theschool can be used and at that point the defects liability period starts (12 months).


7 Sustainability

OECD-DAC defines sustainability as the extent to which benefits from the intervention continueafter major development assistance has ceased. The evaluation framework for GPEP includedten questions on various aspects of sustainability. These considered the extent to whichcommunity support and buy-in contributed to the sustainability of GPEP outputs, the level ofgovernment ownership of GPEP, the extent to which activities will continue when GPEP fundingceases, whether GPEP has catalysed additional external support and the extent to which GPEPhas strengthened national education systems. Knowledge sharing and the extent to which theGPEP experience in South Sudan can inform other programmes is also considered here. Thischapter provides the evaluation findings with regard to these aspects. The information has beenstructured under five sub-headings that reflect key aspects of sustainability covered byquestions from the evaluation framework.

7.1 Community support and buy-inCommunity support and buy-in have been essential to the success of the school constructioncomponent. The focus group discussions with school community members confirmed that thecommunity had been consulted and informed from the start of the project, and that periodicconsultations also took place.

All focus groups said they had been involved at a minimum in the form of consultations, but alsogave other examples of involvement:

● Providing the land for the construction.● Providing security to make sure construction materials were kept safe.● Creating awareness amongst other parents to bring their children to school.● Personal involvement in the construction process and logistics.● Involvement in school governance and monitoring the progress of the school.

These statements are corroborated by the Head Teacher (HT) and Teacher surveys. All 10HTs and 18 teachers agreed that the PTA/SMC was involved in school construction. All 10 HTsand 19 teachers agreed that PTA/SMC felt involved in school affairs.

All 10 HTs reported having a SMC. On average, SMCs are composed of 9.6 males and 5.6females. Frequency of SMC meetings range from 2-3 times a year to more than once a month.All 10 HTs reported having a PTA. On average, PTAs are composed of 7.3 males and 4.4females. PTA mostly meet once every 2-3 months, but some meet more often.

The design of the PSLP did not clearly involve the community. Some respondents said this wasa missed opportunity for sustainability at the local level. SMC/PTA members can holdinspectors, supervisors, head teachers and teachers to account in carrying out their duties in theschool. The focus group discussion with SMC/PTA members showed that members wereunclear about the different roles and responsibilities of inspectors and supervisors.

The Community Girls School model was originally set up and managed by BRAC South Sudan.BRAC scaled down their South Sudan operations in 2016. GPEP stepped in to continue supportfor the Community Girls Schools through GPEP’s OOSC component in 2017. The schools hadstrong community involvement, but respondents said that most communities lacked the meansfor rent and teacher incentives after the GPEP funding ended. It was therefore unclear if these


schools would be operational in 2018. This shows the limits of community ownership in thecurrent context.

One implementing partner said there had been very good involvement of community experts inlocal languages during the development of the local language materials.

7.2 OwnershipThere were mixed responses to the questions around ownership of the programme as a whole.

Most education sector partners said that they perceived GPEP as a government-ownedprogramme. They mentioned close interaction between the Ministry and the GPEP team. Theysaw leadership and responsibility in the Ministry with regards to GPEP. However, respondentsin MoGEI said that ownership in terms of decision-making on resource allocations could havebeen shared more between UNICEF and the Ministry.

With regards to the perception of GPEP activities at the community level, Ministry officialsinterviewed said that more could have been done to communicate MoGEI ownership of GPEPactivities. Implementing partners confirmed that activities subcontracted to NGOs wereperceived as NGO-led projects, not as part of a government-led programme. It was felt it wasimportant for the next phase to give more visibility to the government as an education serviceprovider in the implementation of GPEP on the ground.

The endline survey questions on who is responsible for small and large repairs confirm that theMoGEI or State Ministries of Education are generally not perceived as the responsible institutionfor the provision of education services by the community (see graphs in section on SchoolConstruction).

7.3 Continuity of activities absent fundingA number of GPEP achievements have the potential to benefit the education sector after thecurrent phase of funding ends, for example the curriculum. In addition, the GPEP has builtcapacities of key MoGEI staff and of inspectors and supervisors. Their improved skills willremain beneficial to the education system after GPEP’s current phase ends. Participants in theaccredited school leadership course have a recognised certificate, and the materials and trainedstaff for the course remain with the University of Juba for future use. The M&E strategy for thesector can continue to guide MoGEI’s monitoring strategy, and the literacy and numeracyassessment tools also remain available for future use.

Contextual challenges hinder the sustainability of activities of all education sector programmesin some way, including GPEP activities. The economic crisis has put severe constraints on theMoGEI budget. Currently no funding can be allocated to capital investments, therefore anyschool construction can only continue with external support. The closure of two out of 25constructed schools due to conflict also shows the risks of school construction during conflict.Extensive repairs may be necessary for these schools to reopen once the locations are morepeaceful. Inspectors and Supervisors have not been receiving operational grants in manylocations. They cannot make frequent school visits to practice their newly acquired skills. CECofficials said the centre ceased all activities after GPEP support ended due to lack of funds.Insecurity and displacement means that some supervisors or teachers who have been trainedare no longer in the same location, and some people have been reassigned to new posts in thenewly created states or counties.

The school-level research shows that 6 of 10 Head Teachers and 12 out of 20 teachers believethat the government and other partners will continue to fund all GPEP initiatives once funding


ends. WRP was the state with most concern about funding, with one HT and three teachersbelieving that the initiatives would not be funded after GPEP (HTQ b3, TQ b3). Of thoserespondents who believed initiatives would not be funded or who didn’t know, the initiativesrespondents believed would be scaled down or stopped included:

● Constructing classrooms, teacher blocks, and other facilities (3 HTs and 6 Ts)● Investing in learning materials for pupils such as textbooks and notebooks (1 HT & 2 Ts)● Investing in teacher guides or teacher materials (4 Ts)

Education sector partners recognised that external support to the education sector is crucial tobuild on the achievements of the GPEP. The roll out of the curriculum, including trainingteachers in the new curriculum, requires external support and was considered a priority.

Leaving aside the contextual challenges, all implementing partners of the GPEP said that thetimeline for implementation had been too short to achieve lasting change. There was no time toprovide follow-up support after the training of Inspectors and Supervisors in their locations forexample. Implementers also felt much more time was needed to achieve behavioural changerelated to girls’ education and gender issues. The same was observed regarding the ActionResearch, which involves providing mentoring to the school administration and teachers.Achieving sustained improvements in school management and pedagogy would take at leasttwo years, rather than 7-8 months. This is a key lesson to consider for the design of the nextprogramme.

7.4 Catalysing additional fundingEducation sector partners valued the GPEP as a sector programme that had played a role in thestart of more sector alignment. Ministry officials also said that all sector programmes reinforceeach other. GPEP, GESS and IMPACT work in partnership and benefit and complement eachother. The evaluation did not find evidence that GPEP had catalysed additional funding to theeducation sector, apart from being co-funded by USAID from December 2013 up to December2016. However, catalysing additional funding for education development in South Sudanbecame difficult in a context where donor funding in South Sudan largely shifted focus tohumanitarian relief since 2014.

The schools included in the endline survey sample had all received GESS Cash Transfers in thelast three years, as confirmed by PTA/SMC participants in the FGDs. Most schools werereceiving Capitation Grants and teacher incentives as well. Implementing partners said they hadseen examples of other NGOs providing additional support to the GPEP supported schoolssuch as another borehole, or sanitary pads for girls. However, as stated earlier, it is not clearthat this support was leveraged by GPEP.

7.5 Learning and sharing of informationComponent 3 aims to generate evidence from GPEP implementation and communicate lessonslearned and successful models for replication and scaling.

The GPEP baseline was contracted to the Africa Educational Trust (AET). Due to initial delayscaused by the outbreak of conflict in December 2013, the GPEP baseline study was onlyinitiated in September 2014 and completed in March 2015. Validation meetings were held in thefive GPEP supported States and a meeting held with national MoGEI and partners todisseminate the findings. Baseline findings have been used to populate key indicators in theGPEP Results Matrix and informed the Action Research initiative, also under AET.


In June 2016, GPEP monitoring reports indicated that most of the GPEP baseline indicators interms of quality teaching and learning remained unchanged even after embarking on GPEPimplementation. This implied that learning outcomes were not being achieved satisfactorily andimproving education quality in the 25 GPEP supported Schools remained in jeopardy unlessimmediate action was taken to improve the situation. Consequently, in January 2017, GPEPcommissioned AET to initiate a seven-month action research process. This involved all the keyGPEP stakeholders in the MoGEI from National, State, County, Payam and school levels.

Community members serving in school committees, head teachers and teachers at the schoollevel and policy makers at Payam, County, State and National levels were supported to usedifferent action research methods and tools to collect both qualitative and quantitative data foruse in reflecting upon GPEP targets, identifying the current status and progress towards thetargets, and subsequently solving identified challenges in order to unlock bottlenecks hinderingeffective and efficient implementation of GPEP. The action research produced a mid-termreview that brought out critical issues for action and a final report at the end of 2017. The finalreport is rich in detail with a separate section on challenges in each of the 25 GPEP school.Together with this endline evaluation, this can be used to inform the second phase of GPEPfunding, and other programmes in the sector.

A common observation during stakeholder interviews was that very few respondents had acomprehensive understanding of the entire GPEP. Implementing partners tended to be familiarwith the activities they were contracted to deliver, but not aware of what the programme as awhole comprised of, or worked towards. Education sector partners felt that GPEP could havebeen more proactive and visible in its communication, for example by sharing more progressupdates and other programmatic information.

In conclusion, the current security and economic context clearly jeopardises the sustainability ofthe gains made by any education programme after funding stops. All respondents recognisedthe need for continued external support to build on the achievements of GPEP. The evaluationconsidered programme sustainability in different ways. With regards to community support andbuy-in, this was a clearly well managed in the school construction component, but lessintegrated in the design of other components such as the PLSP. There were mixed responsesto questions around programme ownership. Education sector partners perceived GPEP asgovernment-owned, whereas respondents from the MoGEI said they would like to be moreinvolved in the next phase, especially around decision-making on resource allocations. Atschool-level, respondents said the GPEP was perceived as donor- or NGO-led. Increasedvisibility of the MoGEI as education service provider at sub-national level should be consideredfor the next phase of GPEP.


8 Collaboration and coherence

Collaboration and coherence is not an OECD-DAC criteria for evaluation, but it had beenincluded as a sixth criteria in the RFP. Under this criteria, the evaluation framework includedquestions on the extent to which GPEP enhanced education sector collaboration, intra- andinter-programme coherence, the role of GPEP governance mechanisms and UNICEF andgovernment personnel turnover. This chapter is structured to address each of these fourquestions under separate sub-headings.

8.1 Education sector collaborationGPEP’s presence in the education sector has strengthened sector collaboration. The explicitsectoral approach, closely aligned to the GESP, led to developments of a range of outputs thatbenefitted the sector, as noted above in the section on ‘Relevance’. GPEP also helped todevelop a new sector analysis and plan. This was not an output under the GPEP but arequirement for the next round of GPEP funding to which GPEP contributed funds. Thisconfirms the positive role of GPEP’s presence in South Sudan for education sector planning.

GPEP played a key role in the establishment of a Joint Steering Committee (JSC) for the BigFour programmes12 in the education sector in 2015. The JSC was set up as a more efficientalternative to four separate programme steering committees that would each require MoGEI andeducation sector staff presence. The JSC met several times over the course of GPEP butmeetings did not happen every quarter. After initial momentum in 2015, two of the Big Foursector programmes closed in 2016. During the same year, conflict broke out in Juba in July andthere was a high level of staff turnover in the education donor community. These developmentsaffected the momentum of programme coordination through the JSC.

GPEP collaborated with the Big Four programmes in various ways. For example, GPEP andGESS coordinated their school governance interventions. GPEP agreed with IMED not toduplicate support to MoGEI’s Directorate of Planning and Budgeting and GPEP and Room toLearn collaborated on the development of Literacy and Numeracy materials. Respondents saidJSC meetings were useful for sharing information and building relationships between sectorprogrammes, Respondents suggested that the JSC meetings could have achieved more in-depth discussions on strategic programme alignments. Some respondents said this was due tomany competing priorities and very high workload of staff running the core educationprogrammes. When asked what they would do differently in their engagement with GPEP, allrespondents in the education sector said they would pursue more coordination, but alsoexpected GPEP to be more visible and active in sharing information.

8.2 Governance mechanismsThe technical working groups and reference groups were composed of officials of relevanttechnical departments in MoGEI and in some cases staff of implementing NGO’s. They weregenerally very useful mechanisms for technical oversight, information sharing and decision-making during implementation. The agenda for the monthly meetings was set jointly by MoGEIand GPEP staff. In some instances, other government departments would be invited to theTWG meetings for technical advice, for example the Ministry of Housing on construction

12 The Big Four programmes were, besides GPEP, Girls’ Education South Sudan (DFID), Improved Management for Education Delivery(IMED, EU) and Room to Learn (USAID). IMED and Room to Learn closed in 2016.


standards. These forums did not continue to meet post-implementation of the components theywere set up to oversee.

The GPEP team did not attend Education Cluster meetings, where the Education inEmergencies response is coordinated. With the spread of insecurity to many parts of thecountry, it would have been relevant for the GPEP to connect more with the Cluster. Othersector programmes did update Cluster partners on their activities and this led to usefulcoordination of activities and generally more awareness about the programme.

8.3 Staff turnoverThe evaluation did not find any evidence that staff turnover affected GPEP continuity. Somedelays were noted in the recruitment of a suitable team at the start of the programme, but thefirst GPEP manager remained in post for over two years and there was no gap in theappointment of a replacement. Some members of the initial GPEP team are still in position.Over the course of the last four years, the Minister and Undersecretary in the MOGEI changedonce. The GPEP focal point and other key technical staff in the MoGEI did not change over thecourse of the programme.

The GPEP team was small in comparison to the size and complexity of the programme.Implementing partners said frequent staff travel and leave led to delays in getting approvalsduring implementation. The next GPEP should reconsider the size and composition of the teamto match the size and complexity of the programme and ensure a consistent presence in Juba.A GPEP staff member could also be embedded in the MoGEI to improve coordination.

8.4 CoherenceThe evaluation found subcomponents of component A were largely coherent with each other.For example, training and materials development were coherent with the new curriculum thatwas developed by GPEP. The leadership courses were child-centred and competency-based,modelled on the new curriculum. The Literacy and Numeracy kits and assessment tools weredeveloped in the five national languages. The M&E strategy included the assessment tools toevaluate performance.

The programme would have been more coherent if it had included more linkages betweencomponent A (systems strengthening) and B (community and school-based education delivery).The evaluation found that the roll-out of the curriculum and materials had not yet beenachieved. As a result, the GPEP has had limited impact on the quality of education in schools.Designing feasible roll-out strategies including teacher training in the new curriculum will be akey priority for the next GPEP.

The model school approach could have focused more on training teachers in the new cur-riculum, which could have commenced while construction was ongoing. The action researchcommissioned in 2017 tried to address this and was coherent with the GPEP vision for modelschools. However, it was initiated late in the programme and only lasted seven months.

The implementation of activities targeting OOSC, an objective that was part of the design ofGPEP, were least coherent with the overall programme. The CECs had been expected to play acentral role in reducing the number of OOSC in the areas surrounding the GPEP supportedschools. CEC’s received some hardware support (solar panels, motorbikes) and managementtraining, but did not become the active resource centres that had been envisaged by GPEP atprogramme design. The activities targeting OOSC instead focused on small-scale skills trainingof youth in Juba, and temporary support to CGS. These activities did not reinforce or build onother GPEP activities and were not sustainable.


9 Lessons learnt and recommendations

This section summarises the main lessons learnt from the endline evaluation to inform futureprogramming in South Sudan in general, and the design of the next GPEP.

9.1 Lessons learntGPEP maintained its relevance as an education development programme supporting theimplementation of the GESP, but the GPEP approach to school construction lost relevance inthe context of conflict and economic crisis. GPEP could have reflected more frequently on thecontinued relevance of the design’s core components in particular school construction. Theprogramme would also have been more coherent if component A (systems strengthening) andB (community and school-based education delivery) were more closely joined.

Teacher training should be expanded in terms of numbers of teachers trained, duration of thetraining, and amount of follow-up, refresher courses, and mentorship. Ensure that learningmaterials are distributed to schools, provided in local language of instruction, and teachersreceive adequate training on their use. Evaluation findings suggest that improving quality ofeducation and learning outcomes, requires prioritising support to learning materials and teachertraining over school construction, as head teachers, teachers and other key informants perceivethese to have a greater impact. It should be noted that due to limited sample size and self-selection, responses from teachers may not represent the opinions of all teachers in the schoolor of the GPEP programme.

School construction remains important to increasing access, but the need for schools is so greatthat building 25 schools has not made a big impact. GPEP should consider lower-cost initiativeslike renovating existing schools and continuing support for community-based schools. As theconflict spread, the school construction component and model school approach lost theirrelevance due to rising costs and risks of damage and destruction of the school buildings.

One workable element of the model school approach is using the model school as a focal pointfor Headteacher and Teacher training in the community. The evaluation found some evidencethat this was working well, and that other teachers were observed attending training at GPEPsupported schools. Consideration should be given to strengthening the local networks betweenGPEP and non-GPEP supported schools.

9.2 RecommendationsRecommendation 1: Maintain long-term development focus

GPEP should maintain a long-term development focus. Shifting focus to short-term emergencyprogramming will not benefit the education sector. It creates fragmentation and lacks sustainedimpact. EiE appears to be sufficiently funded at the moment.

However, GPEP should have reflected more frequently during programme implementation onwhether programme adaptations were necessary in response to contextual developments tomaintain relevance. GPEP should also have built more connections with the Education Clusterto create opportunities to reach displaced students with materials or training through partners.

In the protracted crisis context of South Sudan, GPE should carefully consider which corecomponents have greatest chance of success. Investment in human resources through training


and simple learning materials for example can be better adapted to implementation in differentstages of emergency and development contexts across the country than infrastructure.

GPEP had not considered how to provide education for displaced students given the differentcontext at the time of design. The evaluation shows that displaced students are a major issuefor the design of education programmes in the current context in South Sudan. The evaluationfound some evidence that displaced students enrolled in GPEP supported schools whichcontributed to general overcrowding. There is currently no national policy to decide which gradedisplaced students join after missing schooling, although there are some attempts to provideaccelerated programmes to account for one year of missed education. The evaluation found adisconnect between an Education in Emergencies approach to providing basic education andability to integrate students back into the national system. For example, even displaced studentsin POC or refugee camps across the border should be provided the opportunity to sit fornational exams.

Recommendation 2: Focus on fewer core components

The design of GPE2 needs more focus. Fewer core components will make the programme morecoherent. This will help to achieve greater visibility and understanding of the programmeobjectives and develop a strategy for government ownership and sustainability. Despite havinga long-term development focus, the GPEP still appeared fragmented due to many different (sub)components. Partners contracted to deliver these covered only a set of specific activities for arelatively short period of time.

Based on the evaluation findings, supporting the MoGEI to improve teacher performance andproviding equitable, low-cost ways of increasing access seem priorities and should beconsidered as part of these core components

More focus should also allow for longer timeframes for the implementation of activities. This isrecommended to ensure sufficient time for follow up which is needed for behavioural changeand effective training results.

Recommendation 3: Support the MoGEI to lead sector coordination

GPE2 should be more visible and more proactive in sharing information with sector partners,and realise the potential of its role to support MoGEI to lead sector coordination. The roles andresponsibilities of CA and GA need to be clarified for more efficient management and to avoidunnecessary delays.

The GPEP team composition should match the size and complexity of the programme. Evenwhen GPE2 follows this evaluation’s recommendation to focus on fewer core components, alarger team which includes additional areas of expertise may be needed to provide the requiredtechnical and managerial oversight during implementation. GPE2 should consider embeddedstaff in the Ministry which can help to improve sector coordination as well.

GPE should continue to look for complementarity with other sector programmes. A goodopportunity in this regard may be to ensure collaboration on the roll out of the new curriculum.

Recommendation 4: Implementation model should be considered at design stage &Recommendation 5: Implementation model should strengthen education structures todeliver results in schools

GPE2 needs to achieve more coherence between national-level activities and roll out toschools. GPEP design seems not have thought through a workable model to reach schools atscale. As a result, contracting seems to have been ad hoc, timeframes too short and rushed


with no time to follow up after training was completed. Materials’ distribution to schools wasdelayed or not fully completed.

The design of GPE2 therefore needs to consider how implementation at state, county, payamand school-level will be managed at scale. The implementation model needs to be flexible andable to respond to the unpredictable operating environment. This will require the ability tomanage a temporary scale-down or scale-up of activities in response to localised insecurity. Theimplementation model also needs to consider the diversity of South Sudan and build onstrengths of partners with experience working at local levels.

In a protracted crisis context, evidence shows that community ownership and involvement arekey to a conflict-sensitive and more sustainable approach. Community ownership also improveslocal accountability, for example when PTA/SMC members are engaged in school managementand administration. Increased community involvement and ownership could also stimulate moredemand for education, which is particularly needed in more remote areas. The design of theimplementation model should also consider this.

Although GPEP included training for school inspectors and supervisors, it is not clear howeducation officials and state and county levels were systematically involved in theimplementation of GPEP activities in their areas. The GPEP was therefore perceived as NGO-led at school level. The implementation model should consider ways in which education officialscan be more involved in the implementation and oversight of activities. This will also raiseawareness about the programme’s activities and policy developments at state and county level.


Annex A: List of interviewees

Organisation Name

Partners/donors

DFID Shaun Collins

European Commission Kenyi Kilombe

Education Cluster Nicholas Servas

Bazgha Iftikhar

Government

Undersecretary MoGEI Michael Lopuke

Director Planning/Budgeting George Mogga

Director Development Partners Esther Akumu

George Ali (deputy director)

Director School Construction James Odick

M&E Technical Working Group Victor Chol

Acting Director AES Samuel Dem

Director of Basic Education Abdullahi Ali

Implementers

Yei TTC Sarah Amulo

Moses Muto

Save the Children Bester Mulauzi

Daniel McVitalis

Arshad Malik

Juba University Dr. Momo Edward

Dr. Baffoka Michael

Windle Trust David Masua

UNESCO Awol Endris Adem

GESS - Technical Yolanda Ille-Felix

Andabati Omar

GESS – Management Daniel Gesaka

IMPACT Klaas Atsma

AET Jonathan Coxall


Organisation Name

Jackson Okello

AFOD Eriga Jos Perino

Linus Amandu

SIL Tanya Spronk


Annex B: Evaluation Framework withreferences to report sections

Code Evaluation Framework Question Reportsection

Relevance

A1.1 What is the relevance of the GPEP interventions in relation to theGESP/priorities of the South Sudan education sector?

3

A1.2 What other relevant issues in the Education Sector could have beenaddressed by South Sudan GPEP?

3

A1.3To what extent were GPE interventions directly relevant to promotingequitable access to quality education and improving children'sperformance?

3

A1.4To what extent was it necessary to modify the design or implementationof GPEP to accommodated changes affecting the country (e.g. conflict,famine, and drought?)

3

Efficiency

B1.1To the extent relevant information is available, were GPEP activitiesimplemented in an efficient manner, i.e. generating the best possibleoutputs, outcomes, impact at the least possible cost?

6

B2.1If the relevant information is not available, to what extent were attemptsmade to collect and analyze data that would allow for adequatecost/benefit analysis?

6

B2.2 Were there strategies for minimizing costs that offer promising models forfuture programme design and implementation (e.g. school construction)?

6

Effectiveness

C1.1 Which attributes within each of the components is the determining forcebehind success or failure?

4, 6

C1.2 To what extent are gender equality outcomes driven or supported bystakeholders in GPE schools?

5

C1.3Did Infrastructure improvements provide the most effective means ofincreasing access and quality or could resources have been used forother interventions to greater effect?

6

C1.4 To what extent, and in what ways, were GPEP objectives achieved interms of outputs, outcomes, and impact?

4, 5, 6

C1.5 To what extent has the programme succeeded in reducing the gendergap and increasing girls education?

5

C2.1 To what extent has the GPEP been able to create schools that activelypromote gender equality (organization of school activities, teaching,

5


learning materials, etc.)? What are the critical factors for progress or lackthereof?

C2.2 Which outputs and outcomes can be considered direct results of UNICEFsupport?

4, 6

C2.3To what extent, and in what ways did programs result in measurableimprovements in access, school experience and girls achievement?What interventions tended to be most effective?

4

C2.4To what extent did GPE support/play a catalytic role in mobilizing actionsand resources from other partners (including govt., NGOs, civil societyprivate sector, external support agencies)?

4, 7

C3.1 What evidence is there that GPEP activities were scaled up and affectedstrategies aimed at poverty alleviation and overall human development?

4, 7

C3.2 What lessons can be learned for future programme design andimplementation?

9

Impact

D1.1 To what extent is there evidence of GPEP contributing to changes ineducation sector policy at local/state/national level?

4, 5, 6

D1.2Can the GPEP supported schools be regarded as model schoolsdemonstrating case results and attracting more support to the educationsector?

4, 6, 7

Sustainability

E1.1 To what extent was community support and buy-in essential to sustainingGPEP outputs?

6, 7

E1.2 What is the level of ownership of the project outcomes by target groups? 6, 7

E2.1 What is the degree of interaction and alignment between GPEPimplementation and education sector policy management?

3, 4, 5, 6,7, 8

E2.2 To what extent has GPEP attracted additional external support? 7

E2.3 To what extent and in what ways do partners (national/local level) feelownership of the activities?

6, 7

E3.1 To what extent will government and other partners continue activitiesabsent external funding?

4, 6, 7

E3.2 How have national education systems been strengthened forsustainability?

4, 5, 6, 7

E3.3 How firmly has girls' education been incorporated into national andregional agendas?

5

E4.1 How can South Sudan experiences benefit and inform activities andinitiatives in other countries?

7

E4.2

To what extent are GPEP interventions mainstreamed in national policiesand strategies (provide examples) and/or benefited other major programswith external support (e.g. World Bank, bilateral donor agencies, etc.) inthe national and/or regional context?

7

Collaboration and coherence


F1.1 To what extent has the GPEP enhanced collaboration within theeducation sector?

8

F1.2 Have the GPEP components and sub-components been coherent in bothintra- and inter-programme implementation?

8

F1.3What was the role and composition of the Joint Steering Committee,TWGs, reference groups and how did these mechanisms [benefit] theGPEP?

8

F1.4 How did UNICEF staff transfers and government personnel turnoveraffect programme continuity?

8


Annex C: Sampling strategy

The minimum requirement for the endline was to include at least one GPEP school from eachstate. Given the budget and timeframe to conduct fieldwork before schools closed in Decemberthe evaluation team determined that primary data could not be collected from all 25 GPEPsupported schools at endline as had been done at baseline. A stratified random sample wastherefore devised, first selecting two schools from each state, and then selecting a sample ofteachers and students within these schools.

Stratifying the sample reduces the number of field visits needed and so reduces the overall costand time for fieldwork, but can also result in a selection bias if responses are not distributedevenly amongst all schools. This clustering effect occurs when the variance of scores betweenschools is greater than the variance of scores within schools.

One potential limitation of a stratified sample is that it may bias the results due to clustering.One way of offsetting this bias is to include a design effect in the sample size, which meanscompensating for decreasing the number of schools (clusters) by increasing the number ofstudents sampled per school. However, doing this would increase the cost and time ofconducting the sample, an aforementioned constraint to the baseline sample.

Review of the baseline data showed that there was a relatively large difference in learningoutcomes (EGRA and EGMA) between GPEP supported schools in each state. We identifiedselecting well or poorly performing schools in the endline sample as a potential design effectthat could bias the results. To account for this, GPEP supported schools were first categorisedinto high, medium, and low performing based on EGRA and EGMA tests at baseline. Then, ahigh-performing and low-performing school were selected from each state; where two schoolsperformed similarly, one was chosen at random. This approach minimises the design effect ofselecting a single school or group of schools which are skewed towards high or lowperformance at endline, while also recognising schools which performed lower at baseline havea greater scope for improvement at endline.

The selected schools were then assessed from the perspective of location to ensure theyrepresented a right mix of town and rural schools. In addition, the diversity in typology ofbuildings (prior to GPEP construction) and type of land for the new development was assessed.These aspects were all found to be sufficiently diverse within the selected sample.

The selected schools were shared with UNICEF for feedback on the accessibility of eachlocation. This led to the replacement of several schools with substitutes due to insecurity in thelocations of several selected schools. The locations, typology of buildings and type of land ofthe newly selected schools was reassessed, and still found to be sufficiently diverse.

In order to determine the sample size for the study, we asked what endline sample size wouldbe needed to show a minimum detectable effect (MDE) for the two outcome variables for theprogramme, the number of students who could read and understand grade level text and theschool leadership and school supervision and instruction indexes.

For the sample of students, we calculated the endline sample size by assuming a sample sizeof 100 students from comparable schools at baseline and a standard error mean of 0.4 frombaseline data on reading fluency and comprehension. We calculated the minimum samplerequired for a one-tailed independent means test with a power (β) of 0.8 and confidence level(α) of 0.95 using G*Power software was 100 students at endline. The final sample is able to


show that a decrease of over +/- 1.5 points out of a possible 16 points from baseline to endlineis statistically significant.

At baseline, Fisher’s Formula was used to weight the sample according to the school enrolmentpopulation. This resulted in a range of 8 -13 students sampled from each school, with anaverage of 9.7 students per school. At endline, 10 students per school were sampled. This wasdone to strengthen comparison of scores from the group of schools selected at endline with thesame schools at baseline. Fisher’s Formula was not used at endline as it was recognised thatthe sample was not proportionally representative of the full 25 GPEP supported schools, and itwas thought that results could be adjusted weighted by endline enrolment numbers if needed.

For the sample of head teachers and teachers, we evaluated the endline sample size needed,given a sample size of 70 respondents from comparable schools at baseline. We calculated thatfor an average minimum detectable effect of 10% on leadership or school instruction andsupervision measures, the sample required for a one-tailed independent means test with apower (β) of 0.8 and confidence level (α) of 0.95 was 1,700 respondents, which wasunachievable in the current research design. Additionally, in order to measure statisticalsignificance of an index score, we would need to apply a correction to the p values for individualsignificance tests. The Bonferroni correction indicates that the required sample size would needto be even larger to demonstrate overall significance of the difference in index scores.

Instead, given time available to enumerators at the schools, we opted to sample one headteacher and two teachers per school, for a total sample size of 30 respondents at endline.Results from head teacher and teacher surveys should be treated as indicative only andresponses should not be seen as representative of the population of GPEP teachers.


Annex D: Leadership and School Inspection and Supervisionindices


Leadership index (teachers’ perception of head teachers’leadership) EES WES NBEG LKS WRP GRAND

TOTAL


In my school teachers’ salaries are paid on time 67% 0% 89% 0% 80% 25% 50% 0% 40% 25% 65% 10%

In my school teachers attend to their classroom duties regularly 100%100% 56%100%100%100% 50% 75% 40%100% 69% 95%

In my school teachers are regularly in-serviced to improve their skills 0%100% 30%100%100%100% 50% 25% 80% 75% 52% 80%

I have the vision of the school written down and posted in strategicareas, and frequently remind teachers of the school vision in meetings

#41% 88%100% 38% 87% 75%100% 94% 87% 75% 84% 74%

My head teacher frequentlychecks the schemes of work developed byteachers to ensure proper planning for effective curriculumimplementation 59% 75%100%100% 87%100%100%100%100% 75% 89% 90%

My head teacher periodically observes teachers teaching theirrespective lessons to ensure quality and effective learning 65%100%100%100% 90%100%100%100%100%100% 91% 100%

The head teacher is respected by teachers, pupils and parents as anauthority in the school* 71%100%100%100% 97%100% 80%100%100%100% 89% 100%

My head teacher is passionate about the value and importance ofquality education* 71%100% 81%100% 97%100% 75%100% 90%100% 83% 100%

My head teacher has set up various committees in the school eachdealing with specific duties and responsibilities 47%100% 88%100%100% 75%100%100%100%100% 87% 95%

My head teacher frequently evaluates the performance of teachersusing Annual Test Scores of their classes 59%100%100%100% 87% 75% 20%100%100% 75% 73% 90%

The school has a development plan which was developed through aparticipatory process involving teachers, parents and the communitythrough the PTA under the leadership of the head teacher 59% 75%100%100% 87%100%100%100%100%100% 89% 95%


The pupils of this school are well behaved. 29%100% 25% 25% 87%100%100% 75%100%100% 68% 80%

My school has written regulations, rules and policies that governbehaviour of teachers, pupils, and how they relate to one another

#47%100% 98% 83% 98%100%100%100% 93% 83% 87% 93%

Average - all comparable questions 55% 88% 82% 80% 92% 88% 79% 82% 87% 85% 79% 85%

N respondents 17 4 16 4 15 4 10 4 10 4 68 20


School inspection & supervision index (teachers’ perception ofinspectors/supervisors) EES WES NBEG LKS WRP GRAND

TOTAL


School Development Plans have been enhanced by recommendationsof the school inspection and supervision visits. 100%50% 56% 75% 100%50% 100%75% 60% 75% 83% 65%

Learning and teaching in my school have improved due to enhanceduse of information and data collected through classroom observations,registers and term test scores 100%75% 56% 100%100%100%100%100%80% 100% 87% 95%

School inspection visits have helped in identifying areas ofweaknesses in curriculum implementation requiring greater attention 100%75% 56% 100%100%100%100%100%80% 100% 87% 95%

School visits are a surprise* 60% 50% 60% 100%87% 100%100%100%40% 75% 69% 85%

During inspection visits, inspectors ask teachers to provide schemes ofwork, lesson plan books, assessment records, and class registers# 76% 75% 90% 100%78% 75% 85% 100%50% 94% 76% 89%

School supervisors during visits want to observe teaching and learningin progress 78% 75% 80% 100%87% 100%85% 100%1% 100% 66% 95%

During inspection visits, the inspector holds meetings with all teachersafter each visit 41% 75% 100%25% 80% 100%40% 50% 100% 100% 72% 70%

During inspection visits, the inspector gives written comments onobservations made 54% 75% 80% 100%84% 75% 90% 100%100% 75% 82% 85%


Average - all comparable questions 76% 69% 72% 88% 90% 88% 88% 91% 64% 90% 78% 85%

N respondents 17 4 16 4 15 4 10 4 10 4 68 20


Annex E: Sample of GPEP supportedschools selected for evaluation

Primary School Name County

Eastern Equatoria State

1 Torit1 Torit

2 Torit Model Torit

Western Equatoria State

3 Namaiku Nzara

4 Naduru Yambio

Northern Behr El Ghazel State

5 Salam Girls Aweil Centre

6 Panriang Aweil North

Lakes State

7 Pachong Rumbek

8 Rumbek Girls Rumbek East

Warrap State

9 Warrap Tonj North

10 Maper-Agep Gorgrial


Annex F: Recommendations for futureschool construction in South Sudan

This annex provides forward-looking recommendations that can be considered by MoGEI,UNICEF and other partners for a potential future National School Infrastructure Programme.There is a critical need for affordable school infrastructure in South Sudan in order to improveaccess and provide a quality school environment for learning and teaching. Such a programmewill need to provide strategic direction and develop projections for the coming 5,10, and 20years on the numbers for school renovation and the numbers of new schools needed, wherethey are needed, and their related cost. Furthermore, the aim should be at cost-effective schoolconstruction norms and standards while maintaining structural integrity and quality of learningspace as well as a sensible choice of core drivers of the cost of school construction i.e. thechoice of design, the choice of technology, and the choice of delivery.

This novel approach will need to focus on affordability through:

● focusing on resourcefulness and resilience, rather than classrooms alone;● maximizing learning space, in-and outdoors;● stressing participatory processes in school planning, design and construction;● preparation of a range of affordable costed school designs;● promotion of local production of building materials;● enhancing local construction techniques and use of local skilled/non-skilled labour;● training of local community building teams, enhancing skills in (school-) construction;

The above will need to be developed in tandem with i) a procurement modality that allows forcommunity building teams at county level to construct schools; ii) extensive on-site technicalcoaching and monitoring of these community building teams to maintain timely progress andquality of outputs; iii) robust project oversight, reporting and quality assurance to be carried outthrough a joint effort of MoGEI, UNICEF and locally recruited site Engineers and/or locally-owned QA firms, and iv) development of a national building cost index, a building materialscosts database and unit costs per m2 and m3 which is to be updated and published on amonthly basis by the Government of South Sudan to create a level play-field for all stakeholdersinvolved in the construction of schools in the country.

Piloting and scaling of the above outlined approach will largely depend on the political will forchange by the MoGEI and other line ministries. It requires capacity to develop and regularlyupdate the national building cost index, facilitate and support the development of a localconstruction industry (with local building materials production and manufacturing, and vocationaltraining of South Sudanese citizens), and a thorough reflection on an appropriate level of thenational school construction standards and preparation of a range of costed designs focusingon quality of learning space both in-and outdoors as well as enabling local small-to-middle scalebuilders and construction teams to construct schools.

This will require external support from development partners to a) prepare and publish regularlythe building cost index; b) preparation of a range of affordably costed school designs withvarious levels of materialization for the different cultural/climatic contexts of the South Sudan; c)support financially the founding and operation of village polytechnics at county level to train,formally and on-the-job as apprentices young people (girls/boys, young women/men) to become


skilled in construction (and other vocations) to gain employment in the local constructionindustry; d) support financially for a span of 5 years, the technical guidance, coaching andquality assurance provided by UNICEF, other Development Partners and 3rd party QA firms tothese trained young people involved in school construction to ensure that community schoolconstruction programs at county-level are delivered in time and with the agreed level of qualityof works and finishes.

From the side of local communities and local private sector (building firms, quantity surveyors,engineers and architectural firms) this will ask to support the approach of formal/on-the-jobtraining for skilled labour as well as development of a local building materials production andmanufacturing sector towards a local construction sector that will provide opportunities forgainful employment and SME’s (decent work) and in general lower operational costs which inturn will encourage investment in terms of built volume (works contracts).

The following technical approaches are recommended, with references to the evaluationframework included:

Site Planning and Selection: (relevance, A1.4; effectiveness C3.2)

Recommended is that:

1. Site planning would ideally aim at identifying suitable and safe land for developmentof learning spaces both-in-and outdoors with drains naturally and has matureexisting vegetation in order to integrate this with the school and ensure access tothe school throughout the seasons (dry and rainy season).

2. Plan schools around shading trees and develop clusters of shading trees in theschoolyard, both sides of the classroom and administration block in an alternatingpattern, near the kitchen for providing shade to children who eat lunch and nearlatrines to provide shading, privacy.

3. In existing schools: simultaneous enhance the schoolyard for semi-outdoorlearning, school food garden and orchard development and playgrounddevelopment and create an inviting, nurturing transformation of the existingschoolyard to facilitate learning and teaching.

4. In new schools: plan the new school development around the safe water point, ifpossible adjacent to shading trees i.e. drill first a borehole and develop the safewater point in which will become the schoolyard then design and develop further theschoolyard toward a climate-resilient, resourceful, and conducive environment forlearning and teaching.

Site Design: (relevance A1.3; efficiency B1.1; C3.2)


1. Site design should aim at creating at enhancing spatial identity of a school throughthe configuration of indoor space (classrooms), with semi-outdoor / intermediatespace (verandas, decks, corridors) with outdoor space (schoolyard, play spaces,sports field, school garden and orchard);

2. To enable and create multiple opportunities for learning, the classroom rather thanlinear (in a row) could be configured in single/double units in clusters of three singleor double classrooms.

3. Outdoor space and in-between space between the classes creates opportunities tomaximize and utilize shaded outdoor spaces. The use of trellises and canopies incombination with shading trees and vegetation will create bio-climatic comfortoutdoors.


4. In case use of linear classroom blocks is appropriate due to the shape, dimensions,configuration and bio-climatic (sun and prevailing winds conditions), of the schoolsite, then the linearity should ideally be broken up in double or single classroomsblocks which are set back so as to form a linear row which creates additionaloutdoor learning space in-between two sets of double/single classrooms whichcould be shaded through careful tree planting, canopies and trellises, forming awide deck for learning, lunch or leisure.

5. The foregoing points are meant to create a learning environment which allows formultiple opportunities for learning and teaching, reminiscent of the principles ofChild Friendly Schools and 21st Century Schools to enhance quality and learningperformance

Site Landscaping: (relevance A1.3; efficiency B1.1; C3.2)


1. Similar to participation in office and work-space (banks, factories, workshops),planning by end-users, site landscaping is ideally undertaken through participationof the school children by going through a process of design thinking and spatialplanning of thinking, designing and making whereby the children facilitated by acertified designer develops the school landscaping and school garden and orchardplan.

2. Schoolyard transformation through letting children go through a process of designthinking and spatial planning will allow children to express their voices on matters inthe school they take at heart which is one of the core principles of a child friendlyschool;

3. Selected indigenous trees (shading canopy and fruit trees) in combination withnatural materials (reed, bamboo) and outdoor landscaping features (low block orbrick walling) create shaded outdoor spaces for socializing and for learning

4. A design thinking approach to school environment and landscaping will empowerschool children to create ownership and have pride in their achievement.

5. Landscaping can be used as a tool for learning subject matter of nature, climatechange, mathematics, biology and so forth which forms part of creating a learningenvironment that inviting, nurturing and conducive to the learning and teachingtaking place

School Design: (relevance A1.3; B1.1; efficiency B2.1; B2.2; effectiveness C1.3; C3.2)


1. School design is a special field in architectural design and 21st Century Schooldesign in turn is an even more specialized trait in that it aims to translate, visualizeand make tangible principles of creating multiple opportunities for learning; enablingindividual learning, small-and group learning; facilitate participatory processes ofschool environment and school space planning with end-users i.e. children andteachers; create a school environment whereby indoor-and outdoor environmentare integrated and form one integrated environment for learning. Hence,outsourcing school design to a 3rd party architectural practice may assist inmitigating corporate risk for UNICEF but it is recommended that within UNICEFSouth Sudan a design specialist (architect) is appointed to lead the formulation ofthe design brief, discussion with regard to the pre-conditions (CFS, gender,inclusivity, costs, environment, greening, and innovation), and decision-making vis-à-vis a 3rd party design firm.


2. The design specialist will work jointly with the UNICEF construction specialist inproviding technical advice to UNICEF Management, provide technical project-andmanagement oversight, lead in planning and execution of design thinking, visioning,and participatory spatial and environmental planning with children and teachers,lead in conceptual thinking, plan development and advice on programme brief,technical-and pedagogic pre-conditions, spatial/functional analysis as well as carryout regular field monitoring for timely, efficient and quality implementation during allphases of the GPEP programme;

3. School design will aim at design of learning environment rather than conventionaldesign of classroom space and will be maximizing learning space i.e. indoor, semi-outdoor and intermediate space utilized for learning. In practice, core classroomdesign will need to be flexible to different styles of learning (individual, small groupand large group learning). The type and design of school furniture needs to facilitatethe different learning styles and classroom settings.

4. Verandas, decks, corridors, outdoor shaded spaces, walls, floors, windows anddoors will all be utilized for maximizing learning opportunities through sensitive andsensible design (maximize black-and green boards in/outdoors, use of pin-boards,use of children’s displays and home-base for their products

5. In using wide verandas or decks for learning, a group can be split in three groups tofacilitate individual, small group and large learning simultaneous if a teacher has 1-2 teaching assistants or teaching apprentices;

6. The design stage is the crucial stage where the core drivers of the cost ofconstruction are being determined. The client (Government and UNICEF) need tohave in-house capacity as specific design-decisions made by 3rd-party QA firmslead to conventional school design solutions which that do not fully reflect childfriendly school (design) principles and which may also have costly implications

Construction Costs (efficiency B2.1; B2.2; effectiveness C1.3; impact D1.2; sustainabilityE1.1; E1.2)

1. A significant part of school design decision-making which influences theconstruction costs of schools are made by the architect and structural engineer atthe early stage of the sketch plan design of a school. The further elaboration andrefining of full architectural plans, structural plan, preparation of bill of quantities,procurement and contracting and quality assurance will flow from these initialdecisions which define the technology package i.e. choice of materials, standards,and delivery mode that is being selected and applied to develop and construct theschools.

2. Technology packages range from high-tech to intermediate-tech, appropriate/locallyenhanced-tech to low-tech with regard to the choice of materials, standards anddelivery mode;

3. Delivery modes range from: Centralized + contractor-built (international);Centralized + contractor-built (national); Contractor (international) + sub-contractors(national); State level + NGO + sub-contractors (international / national); State level+ NGO + local community builders; Community built and; Design-Built modes.

4. To keep the cost of construction under control, one needs to aim at makingappropriate design-choices, technology choices (materialization), and deliverymode choices to find the optimum balance between the core drivers of constructioncost;


5. In seeking an optimum balance of the three core drivers of construction costs forthe current context of South Sudan (vulnerable economy, high inflation, limitedlocally developed construction and building materials production industry, conflicts),one arrives at the following integration of core drivers hence, recommended is that:

a. Delivery in choosing an option that has the greatest potential of mass-production at low-cost on the local market, using small/medium size firmsmanaged by a large firm (Engineering or Contractor acting as CPM).Alternatively, locally-owned contracting firms can work with independent teamsof local carpenters and masons from the community to build the schools;

b. Technology (choice of materialization) in choosing materials and techniquesfamiliar to the local industry i.e. sand/cement blocks, compressed stabilized soilblocks or fired bricks for walling; corrugated iron sheets for roofing andreinforced concrete structural frame, reinforced hollow-block columns,reinforced brick columns or steel portal frame with in-fill walling

c. Design: correct configuration and/or clustering of classrooms to have extensiveshared walling (cost-saving) and creating multiple opportunities for learning.Considerations of semi-open, pavilion-style classrooms in combination with acentral, reinforced storage facility for learning materials, teaching equipmentand school furniture. Most critical is setting appropriate standards for the SouthSudanese context and economy as these will impact significantly on the choiceof technology package and choice of procurement and school delivery mode.

6. It is furthermore recommended that in the context of South Sudan, UNICEFappoints a cost specialist (quantity surveyor), to control the cost of constructionfrom the on-set of the programme, lead in the discussion on developing costparameters and bills of quantities, regularly monitor and update bills of quantities,lead in developing of costed school prototype plans, lead in certification of works,variations of works, granting of certificates of substantial and final completion andthe preparation of final accounts;

7. It is essential to note that the specialist skills and expertise both a design specialistand a cost specialist bring to the programme cannot be undertaken by aconstruction specialist with an academic background and training in civil/buildingengineering which is not geared towards school design nor cost-control of schoolinfrastructure. Hence, it is furthermore recommended that the design specialist, costspecialist and construction specialist, together form a UNICEF construction unit,comparable to international professional practice.

National School Infrastructure Programme: (efficiency B2.2; impact D1.2; sustainability E1.1,E1.2; E1.3)

The GPEP School Construction Criteria – March 2014, states in assessing the need for learningspaces that “any strategy for the construction of new or renovating of existing schoolinfrastructure should start by at least replacing type 1 schools i.e. open air, roof-only and tentwith permanent structures. This means that almost 9000 of the existing classrooms will need tobe replaced.”

If each of the 9000 classrooms is replaced by P1-P8 (classrooms) then this comes to 1,125primary schools at an average cost of 560K USD (given that the scope will be similar as theGPEP CFS package i.e. include latrines, administration block, kitchen, safe water point andwater harvesting as well), then the total sum for replacement school construction comes to 630


million USD. The Government of South Sudan, Development Partners and International Donorswill need to reflect if the current situation protracts i.e. conflict and a seriously weakenedeconomy with significantly depreciated local currency to international currencies the ModelSchool with its set of standards (design, technology package, and delivery mode) will befinancially sustainable for the years to come and whether an adjustment in the choice of coredrivers of the cost of school construction would deem to be necessary.


MoGEI develops a national school infrastructure and construction programme. The programmewill inform the Government of South Sudan on the status quo (numbers of schools constructedover the last 5 to 10 years (2012-2017), according to typology and materialization), the needs interms of classroom space and learning spaces (indoors and outdoors), classroom density, andinsight into the core drivers of the cost of construction i.e. the costs of various technology andmaterialization packages for school construction, the costs of various modes of procurementand delivery, as well as the cost implications flowing from various decisions taken at the designand scoping stage (the design brief and programme)

The programme, furthermore will inform the Government of South Sudan on 5-and 10-yearprojections, 2018-2023 (2028) for school construction and the numbers per-year earmarked forreplacement-school construction of level1 schools (open-air, under tree, tent, roof-only) and thenumber, scope, cost, mode of delivery and funding sources of school construction projects thatwill be undertaken this coming year 2018 and evolving years.

The programme will also create clear linkages to the development of a local building materialsmanufacturing industry, local building cost index development and local construction skillsdevelopment initiatives to lift the local construction industry (jointly with other Ministries anddonors) from the ground aiming at a significant reduction of the cost of construction, applicationof enhanced local technologies and significant increase of local construction capacity.

Local Building Materials Industry: (efficiency B2.2; sustainability E1.1, E1.2; E1.3)

In South Sudan, a local construction industry with a building materials production andmanufacturing industry is underdeveloped. Initiatives towards developing a local cementindustry in Kajo-Keji in 2015 following feasibility studies and minerals survey, however theseplans were thwarted due to the escalating conflict where after funding by foreign investors wasput on hold.

Nevertheless, planning and development of a local building materials manufacturing industry atdifferent levels - production, supply, use, waste management – both at the Central-and Statelevels and appropriate technologies and application at the village/household level is critical as itplays an important role in the socio-economic development of South Sudan.

Recommended is that 1) surveying country-wide, the deposits of sand, aggregate, murram,limestone, calcite, clay as well as sustainable and renewable (natural) materials such asspecies of bamboos and reeds for the production of treated reed panels, boards, supports andtrusses, and 2) prototyping alternative building materials as an option with regard to theimportation of building materials (cement, steel, and so forth) and, 3) piloting costed classroomprototype plans applying alternative building materials for walling using similar finishes

Local Construction Skills Training: (efficiency B2.2; sustainability E1.1, E1.2; E1.3)

Local capacity will need to be developed among the youth in South Sudan for the manufacturingof building materials, their application in construction of various types of structures and inconstruction technologies in general.


Recommended is the development of State Level Polytechnics and Technical Colleges in largerTowns to train a new cadre of technicians and engineers who are skilled in taking onconstruction projects at State, Town and Village Levels and could form the technical personnelfor the development of locally-owned construction companies in South Sudan

Local Building Cost Index: (efficiency B2.2; sustainability E1.1, E1.2; E1.3)

Contrary to a regularly (monthly) updated building cost index in the East-African Region(Tanzania, Uganda and Kenya), South Sudan does not have a regularly updated and publishedbuilding cost index, in parallel with M2 and M3 costing for various building types andmaterialization types (choice of technology and materials package) and a building materialscosts data-base.


A local building cost index is developed which would enable all stakeholders i.e. Government ofSouth Sudan, Development Partners, CSOs, Architects, Quantity Surveyors, Contractors andthe People of South Sudan to have a level play-field so that cost of construction due to specificcombinations of the core drivers of construction costs (choice of technology package, of deliverymode and of design), could be known on forehand for budgetary and projections purposes.

Local Works Procurement: For the GPEP the mode of procurement selected was InternationalCompetitive Bidding (ITB) instead of the Request for Proposal (RFP) for reasons of a wider fieldof international and national contractors as well as give classified contractors of first category(able to handle multiple schools at a time), and second category (able to handle one school at atime), the opportunity to bid. Further rationale was that as a result of this ‘wider’ tender process,competition would be encouraged and thus bids would be lower than through an RFP tenderprocess. Pre-selection of contractors started with publishing the expression of interest (EOI) on27th March 2015. The South Sudanese Pound traded then at 20 SSD/USD. The EOI ran for 2weeks, closing Friday 10 April 2015. A total of 76 companies submitted proposals. Theproposals were opened and recorded in the presence of the MoGEI and thereafter the bidevaluation began with a joint team from MoGEI and UNICEF.

Quality Assurance: (efficiency B2.1; B2.2; sustainability E1.1, E1.2; E1.3)

A robust quality assurance system for school development and construction is fundamental toachieving value for money within the context of South Sudan.

Recommended therefore is:

The production of a choice of design that fulfills the pre-conditions and criteria of the design briefagainst an affordable cost to the Government of South Sudan; a choice of delivery system thatwill maximize the application of local labor and the production of local materials for construction;a choice of technology which is appropriate for the context of South Sudan and will allow forcommunity-based mode of school construction in combination with locally-owned contracting orengineering firms or NGOs who will serve as construction programme managers (CPM) perState to provide technical guidance to the community construction teams, technical oversight,quality assurance and daily site supervision per State, and liaison to UNICEF, including timelyreporting (financial and technical)

Handover: The Action-Research Report of May 2017, that the GPEP-funded schools were atvarious levels of completion with the supply of resources (school furniture and so forth) notuniformly undertaken. In addition, some schools were completed but the formal handover to theMoGEI and school authorities had not taken place since which gave the impression to PTAs


and SMCs that the schools belonged to the donors. AET’s reports, in addition states that someof the schools had clear defects probably due to poor workmanship or had not been furnished.

Recommended therefore is that:

A joint quality assessment (MoGEI, UNICEF. QA-entity and Contractor), of the works is carriedout in affected schools and a snag list prepared with a strict time-line. In parallel a formalhandover time/work plan in each of the involved States has to be developed and rolled out withlaminated handover certificate prepared for each participating school and signed by the MoGEI,UNICEF and the Donor (GPE / USAID). Furthermore, clarification and documentation has to beprovided to each of the participating schools on the details of the defects liability period (usually12 months) following the formal handover and the change in responsibility (from UNICEF toMoGEI) for the maintenance and repair after the elapse of the defects liability period (warranty,12 months).

School Furniture: school furniture, ideally is being designed, developed and manufacturedprior or during the ongoing school construction works so that by the time of formal handover theschools are handed over in a furnished state for immediate use during the 12 months warrantyperiod (defects liability).

Recommended is that school furniture is designed which is fit for purpose i.e. in various sizesfor age-specific (height) groups; is easy to manufacture at the local level; is easy to repair byindividuals of the school community like teachers and SMC/PTA representatives or anappointed craftsperson; is strong and durable to withstand the daily wear-and tear in classroomsbut ideally is not too heavy for a child to move about or for teachers to re-organize the desksand chairs in another setting to enable small-large group learning or learning on verandas andother (semi-)outdoor learning spaces

School Design and Construction Standards:

The school design and construction standards for South Sudan, drafted and publishedNovember 2016 aim three levels of standards namely: MUST: these are minimal spatialstandards for a child friendly school; SHOULD: standards that give guidance on additionalschool spaces and quality that is encouraged and is in line with best practice, and; MAY,standards that give guidance on school spaces and qualities identified as beneficial if resourcespermit.

The above is clear and straightforward however regarding technical norms and standards, itgives limited insight in the selection of the core drivers as parameters that drive the cost ofconstruction upwards in a context of an underdeveloped local construction industry (and localbuilding materials production industry and limited local construction capacity).

Recommended therefore is that: (efficiency B2.2; impact D1.2; sustainability E1.1, E1.2; E1.3)

School construction standards are closely aligned to (potential) opportunities and facilitatinglocal community efforts to provide inputs (in-kind or through participation and labor) in theplanning, design, construction and maintenance and repair of schools.

In addition, for this, the school construction standards will need to be attuned to communitytraining and development interventions to upgrade the skills of community teams and ofindividuals and small-scale builders (fundis, crafts-persons, masons and carpenters) to haveinputs in the construction of their schools for creating more ownership, for enabling livelihoods


opportunities and creating more capacity of the local construction industry and production ofmaterials.

Furthermore, the school construction standards will need to be aligned with the efforts ofdeveloping a national building cost index, unit costs per M2 and M3, building materials data-base and efforts to develop and build a local construction industry and the production of localbuilding materials (cement-industry, compressed stabilized soil blocks, reinforced-bamboopanels, and so forth).

School Construction vis-à-vis Development Partners and China (efficiency B2.2; impactD1.2; sustainability E1.1, E1.2; E1.3)

Individuals, Charities, Foundations, NGOs, INGOs, Churches, United Nations Organizations,World Bank, African Development Bank, The Government of South Sudan, and bilateral throughother Governments, among which China, Norway among others have over the years contributedto the National School Construction Programme by financing, developing or construction ofschool infrastructure in South Sudan as way to create peace dividend and develop the sector.

An example is the realization of secondary schools through Chinese financing, developmentand construction efforts. Chinese financing for social infrastructure (both vertical and horizontal)will likely go upward the coming years 2018-2023 in order to enable the Government of SouthSudan to reach its goals in the education sector to enhance quality of learning and teachingthrough a more conducive physical school environment. As a strategic response to China’sgrowing influence on the local (construction) industry, MoGEI and UNICEF are recommended toconsider the following:

i. Development of a local building cost index for South Sudan, creating a situation of a levelplay-field for all stakeholders in the sector involved in school construction;

ii. Preparation of a series of costed school designs using specific combinations of the coredrivers of the cost of construction (choice of design, technology and, of delivery);

iii. Pilot a series of affordable classroom designs and let the MoGEI and benefitting schoolcommunities provide feedback;

iv. Adjust school construction norms and standards towards more affordability whilemaintaining quality of physical school environment to enable community building teamswith extensive site coaching, monitoring and quality assurance to plan, develop and buildtheir own schools;

v. Providing training and development to communities to enhance their skills forresourcefulness and resilience;

vi. Raise investments and funding towards establishing educational opportunities (villagepolytechnics and a state level, colleges of technology) in order for local communities toincrease their capacity to take up a role in local economic development;

vii. Increase focus on the enhanced transitional learning spaces – given the local context inSouth Sudan - which can move in phases from transitional to permanent schools;

viii. Have a phased school development approach – in line with the national schoolinfrastructure programme - for the coming decade i.e. for new schools introduce a phasedCFS package of 2 classrooms + safe water point + tree planting + school food gardens perschool per year being constructed in the 6 months’ time-window during the dry season.With the following year 2 more classrooms and girls/ and boys’ latrine blocks could beconstructed; With further evolving years, again 2 more classrooms and an admin-block canbe realized and eventually after 4 years an 8 classroom. full primary cycle school has beenrealized


ix. Stress the integration of the human/social/learning environment with the constructedphysical school environment and the natural environment of the site towards theestablishment of whole, living schools in line with the SDGs on sustainable cities andcommunities and climate change action.


camb-ed.com