annual country portfolio evaluation report...
TRANSCRIPT
GEF/ME/C.46/04 May 1, 2014
GEF Council May 25-27, 2014 Cancun, Mexico.
ANNUAL COUNTRY PORTFOLIO
EVALUATION REPORT 2014
(Prepared by the GEF Independent Evaluation Office)
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Recommended Council Decision The Council, having reviewed document GEF/ME/C.46/04, “Annual Country Portfolio Evaluation Report 2014,” document GEF/ME/C.46/05, “Management Response to the Annual Country Portfolio Evaluation Report 2014” requests the Secretariat:
1) To explore and pursue, where appropriate, the use of established SGP country programmes as service providers to implement community level activities for FSPs and MSPs.
The Council requests the Secretariat and the Agencies:
2) To pay greater attention to national knowledge exchange and promote dissemination of data and information in the relevant national languages.
i
TABLE OF CONTENTS
Executive Summary ............................................................................................................ 2
Introduction ......................................................................................................................... 4
Background ............................................................................................................. 5
Objectives, Scope, Methods, and Limitations ........................................................ 8
Conclusions ....................................................................................................................... 11
Results ................................................................................................................... 11
Relevance .............................................................................................................. 19
Efficiency .............................................................................................................. 21
Recommendations ............................................................................................................. 26
3
EXECUTIVE SUMMARY
This Sixth Annual Country Portfolio Evaluation Report (ACPER) provides a synthesis of the evaluative evidence contained in the Country Portfolio Evaluations (CPEs) and Country Portfolio Studies (CPSs) conducted in the Sub-Saharan Africa region. These include two CPEs, one conducted in Tanzania and one in Eritrea, and one CPS conducted in Sierra Leone. All these evaluations are at different stages of finalization and the reports, once completed, will be provided on the Office’s website at the end of the second quarter of 2014. For the three evaluations, key preliminary findings and areas for recommendations were presented and discussed – and comments were received from GEF stakeholders – at final consultation workshops in each country. At the time of writing this report, the Tanzania report is being finalized, taking into consideration comments received to a draft circulated in February. The Eritrea and Sierra Leone draft reports are being finalized and will be circulated for comments by the end of April.
Support from the Global Environment Facility (GEF) to these countries started in 1992 in Tanzania and Eritrea, and in 1998 in Sierra Leone through participation in a GEF regional project.
Country Type of evaluation
Number of projects included in the evaluation National completed projects
National FSPs & MSPs
SGP Enabling Activities
Regional/ global projects
Eritrea CPE 6 Yes 6 12 8 Sierra Leone CPS 9 Yes 6 16 6 Tanzania CPE 21 Yes 7 53 14
This ACPER focuses on the result and sustainability of GEF support, particularly at the global environmental benefits level; the relevance of the GEF support to the GEF and to the countries; and the efficiency of GEF support. Conclusions
The following conclusions were reached on the results of the GEF support:
(1) GEF support has played an important role in creating the enabling framework necessary to underpin the development of environmental policy and laws in the three countries analyzed in the Sub-Saharan Africa region.
(2) Positive results at completion have been achieved beyond foundational support,
leading in some cases to progress towards impact.
(3) Likelihood of sustainability is mixed, and has been most successful when pursued through fostering institutional and individual capacity development as well as promoting livelihood activities through community-based approaches, e.g. the SGP.
(4) Wider dissemination and uptake of project-derived lessons at national level has been
hampered by language barriers.
On relevance of GEF support the following conclusions should be noted:
4
(5) GEF support has been and remains relevant to the countries’ environmental priorities
as well as to their sustainable development needs, with few exceptions.
(6) Diverse degrees of ownership have been observed in the three portfolios analyzed.
The efficiency of the GEF support was assessed as follows:
(7) Project design factors, particularly over-ambitious objectives, have often caused implementation over-runs.
(8) Monitoring and Evaluation is mixed in the three portfolios, and efforts are deployed to
improve the situation where needed.
(9) Mechanisms for coordination and synergies among GEF Agencies, national institutions, and among GEF and other donor-supported projects and activities have been set up, with mixed results.
Recommendations
The following recommendations emerged from the conclusions formulated here above:
(1) The GEF should explore and pursue, where appropriate, the use of established SGP country programmes as service providers to implement community level activities for FSPs and MSPs.
(2) The GEF should pay greater attention to national knowledge exchange and promote
dissemination of data and information in the relevant national languages.
5
INTRODUCTION 1. The seventh Annual Country Portfolio Evaluations Report (ACPER) provides a synthesis of the main conclusions and recommendations emerged from the evaluative evidence contained in the Country Portfolio Evaluations (CPEs) and Country Portfolio Studies (CPSs) conducted in the Sub-Saharan Africa region. This includes one CPS in Sierra Leone and two CPEs, one in Tanzania and one in Eritrea. 2. Support from the Global Environment Facility (GEF) to these countries started in 1992 in Tanzania and Eritrea, and in 1998 in Sierra Leone. These countries were chosen for country level evaluations based on a selection process and a set of criteria including the size, diversity and maturity of their portfolio of projects.1 As with previous country level evaluations, consultations were held with all major GEF stakeholders, particularly those residing in the countries. Several visits to project sites have also been undertaken. 3. The Sierra Leone CPS was conducted in parallel with the UNDP Assessment of Development Results (ADR) for Sierra Leone (2008-2013). The national consultant conducting the CPS was also responsible for coverage of the UNDP Energy and Environment portfolio within the ADR. As was the case with previous parallel country portfolio evaluations this provided advantages for both the GEF and the UNDP independent evaluation offices, including a broader comparison of issues across sectors in a post-conflict country in the process of building state institutions, lower evaluation burden to the country and cost savings. 4. The evaluative phase of the Tanzania CPE was conducted between December 2012 and September 2013, while the Eritrea CPE was conducted between February and September 2013. The evaluative phase of the Sierra Leone CPS was conducted between September 2013 and February 2014. At the time of writing this report, the draft report of the Tanzania CPE distributed for comments in February 2014, is being finalized, while the draft reports of the Eritrea CPE and Sierra Leone CPS will be distributed for comments in April 2014. 5. The final reports of the GEF Country Portfolio Evaluation: Tanzania (1992–2013), and the GEF Country Portfolio Evaluation: Eritrea (1992–2013) will be completed by the end of the second quarter of 2014. This is also the case for the GEF Country Portfolio Study: Sierra Leone: (1998–2013). However, all evaluations have completed the evaluative phase and had a final stakeholder consultation workshop in the respective countries. The findings and conclusions that emerged from these evaluations, along with key areas of recommendations identified during the final stakeholder consultation workshops, and included in the respective draft reports, have been considered in this report. 6. Three information documents are provided to Council, presenting the conclusions and recommendations of the two CPEs completed during the third quarter of 2013 in the East Asia and Pacific region: the GEF Country Portfolio Evaluation: India (1991-2012), the Joint GEF/Sri Lanka Country Portfolio Evaluation (1991–2012) and the Vanuatu and SPREP Portfolio Evaluation (1991–2012).
1 http://www.thegef.org/gef/sites/thegef.org/files/documents/CPE_final_country_selection_note-0910_0.pdf
6
Background 7. The Office is completing its multiannual cycle of country level evaluations during GEF-5, covering all the GEF geographical regions, including: the Latin America and Caribbean (LAC) region, reported in the ACPER 2012; the Asia and Pacific region, reported in the ACPER 2013; and the Sub-Saharan Africa (SSA) region, reported here. In fall 2013 the Office started country level evaluations in East Europe (ECA) region with the Russia CPE, and in the Mediterranean and North Africa (MENA) region with the Morocco CPE. A last CPE is expected to be launched in Tajikistan (ECA region) during the third quarter 2014. 8. The Sub-Saharan Africa region began participating in the GEF program during the GEF pilot phase in 1991. Since then, the GEF has invested around $1.09 billion with an additional $5.61 billion in co-financing through 548 active or completed national projects in the region. The active national projects represent 40.9% of the total portfolio or $2.74 billion (including GEF amount and co-financing) while the completed projects represent the remaining 59.1% or $3.97 billion. Most of these projects belong to the biodiversity and climate change focal areas (213 and 164 projects respectively), followed by multifocal area projects (69 projects), land degradation (51 projects), Persistent Organic Pollutants (POPs) (46 projects), and international waters (5 projects). There are no projects under Ozone Depleting Substances (ODS). UNDP is the GEF Agency responsible for implementation of 48.5% of the national projects in the region, followed by the World Bank (23.6%), the United Nations Environment Program (UNEP) (19.3%), the United Nations Industrial Development Organization (UNIDO) (4.9%), the International Fund for Agricultural Development (IFAD) (1.8%), the Food and Agriculture Organization (FAO) (0.9%), the GEF Secretariat (0.7%) and the African Development Bank (AfDB) (0.2%). The largest country portfolios are South Africa, Nigeria, Ghana, Madagascar and Uganda, representing 16.2% of national projects and 38% of the total funding committed in the Sub-Saharan Africa region as a whole.2 9. The country portfolios in Eritrea, Tanzania and Sierra Leone include 55 national projects allocated in five of the GEF focal areas (23 in climate change, 19 in biodiversity, 5 multi-focal, 4 in land degradation, and 4 in POPs; there were no projects in international waters or ozone depleting substances). The total GEF financing was $128.3 million, with $793.9 million of co-financing. In biodiversity the national portfolios analyzed totaled approximately $54.7 million in GEF financing and around $128.1 million in co-financing. Financing for climate change in the national portfolios analyzed totals approximately $47.3 million in GEF grants and $293.2 million in co-financing. The GEF has invested approximately $13.4 million with $66.6 million co-financing in multifocal area projects. In land degradation, the GEF has invested around $9.3 million with $46.0 million co-financing. In POPs, GEF financing was equivalent to approximately $3.6 million, with $3.5 million co-financing. UNDP is the main channel for GEF support with 26 projects, followed by the World Bank and UNEP with 8 projects each. 10. The GEF portfolios included in this ACPER are briefly described here below:
2 The Sub-Saharan Africa region comprises of 49 countries: Angola, Benin, Botswana, Burkina Faso, Burundi, Capo Verde, Cameroon, Central African Republic, Chad, Comoros, Congo, Congo DR, Cote d'Ivoire, Djibouti, Equatorial Guinea, Eritrea, Ethiopia, Gabon, Gambia, Ghana, Guinea, Guinea-Bissau, Kenya, Lesotho, Liberia, Madagascar, Malawi, Mali, Mauritania, Mauritius, Mozambique, Namibia, Niger, Nigeria, Rwanda, Sao Tome and Principe, Senegal, Seychelles, Sierra Leone, Somalia, South Africa, South Sudan, Sudan, Swaziland, Tanzania, Togo, Uganda, Zambia, Zimbabwe. Two countries, South Sudan and Somalia, do not have any active or completed projects.
7
i. Eritrea: Participation with the GEF began in 1992, shortly after Eritrean
independence. Since then, Eritrea has been involved in 12 national projects totaling $22.6 million in GEF support and $41.6 in co-financing. The main GEF Agency is UNDP with 6 national projects, followed by the World Bank with 2 projects, and UNEP, UNIDO, FAO and IFAD with 1 project each. The portfolio comprises 4 in biodiversity, 3 projects in climate change, 2 in land degradation, 2 in POPs and 1 multifocal,. Half of the national portfolio is composed of Enabling Activities. Eritrea is also participating in 10 regional and 2 global projects. Two terminal evaluations are available for Eritrea.
ii. Sierra Leone: Since 1996, the GEF has allocated $26.8 million, with about $129.5 million in co-financing, through 14 national projects and the national component of 1 global project3. These projects include 9 in climate change, 3 in biodiversity, 1 in land degradation, 1 in POPs and 1 multifocal. National projects are evenly spread within the GEF project cycle with 6 completed/closed, 5 ongoing and 4 approved or endorsed. UNDP, with 7 projects totaling $10.2 million, has been the main channel for GEF support in Sierra Leone to date, followed by the World Bank and UNIDO (2 projects each, totaling $6.8 and $2.2 million respectively), AfDB and IFAD with one project each ($4.2m and $2.7m, respectively), and finally UNEP with two projects ($0.6m). In addition, Sierra Leone is also a participant country in 12 regional and 4 global projects.
iii. Tanzania: Tanzania’s participation with the GEF began during the first GEF phase in
1992. Up until July 2012, the GEF had allocated $78.8 million through 28 approved national projects (12 biodiversity, 11 climate change, 3 multifocal, 1 POPs and 1 land degradation). These activities involved aggregated co-financing commitments of $366.2 million. 14 of these projects have been completed, 10 are under implementation, and 4 are approved or pending approval. In addition, Tanzania is involved in 39 regional and 14 global projects supported by GEF. 14 terminal evaluations are available within the Tanzania national portfolio.
3 ‘Umbrella Programme for National Communications to the UNFCCC’ (GEF ID 4498).
8
Table 1.1: Portfolio Resource Allocation by Focal Area
Country Type of Evaluation
Biodiversity Climate Change Land Degradation
Multifocal POPs Total
GEF Co-financing
GEF Co-financing
GEF Co-financing
GEF Co-financing
GEF Co-financing
GEF Co-financing
Eritrea CPE 11.31 11.41 2.45 2.95 6.17 23.93 0.20 0.02 2.50 3.24 22.6 41.6 Sierra Leone CPS 7.08 22.18 18.61 106.90 0.50 0.44 0.22 0.02 0.39 - 26.79 129.54 Tanzania CPE 36.31 94.46 26.29 183.31 2.63 21.65 12.95 66.54 0.71 0.21 78.9 366.2
Total 54.69 128.05 47.35 293.16 9.30 46.02 13.36 66.57 3.60 3.45 128.3 793.86
Table 1.2: Projects by GEF Agencies
Country Type of evaluation
WB UNDP UNEP UNIDO FAO UNDP/ WB
IFAD AfDB WB/ IFC
Total
Eritrea CPE 2 6 1 1 1 - 1 - - 12 Sierra Leone CPS 2 7 2 2 - - 1 1 - 15 Tanzania CPE 4 13 5 4 - 1 - - 1 28
Total 8 26 8 7 1 1 2 1 1 55
Table 1.3: Projects by Focal Area
Country Type of evaluation
Biodiversity Climate Change
Land Degradation
Multifocal POPs Total
Eritrea CPE 4 3 2 1 2 12 Sierra Leone CPE 3 9 1 1 1 15 Tanzania CPS 12 11 1 3 1 28
Total 19 23 4 5 4 55
9
Objectives, Scope, Methods, and Limitations 11. Evaluation work in the Sub-Saharan Africa region is being conducted by Office’s staff and consultants with extensive experience with each individual country. The Tanzania and Eritrea CPEs followed country-specific Terms of Reference (TORs) developed from the standard CPE TORs4 and adapted to each country using the information collected and the feedback received during the scoping phase. The Sierra Leone CPS followed the standard CPS TORs.5 In compliance with the standard TORs for these evaluations, the CPEs and CPSs included in this ACPER were all conducted with the following objectives:
• evaluate the effectiveness and results of GEF support in a country, with attention to
the sustainability of achievements at the project level and progress towards impact on global environmental benefits;6
• evaluate the relevance and efficiency of the GEF support in a country from several points of view: national environmental frameworks and decision-making processes; the GEF mandate and the achievement of global environmental benefits; and GEF policies and procedures;7
• provide additional evaluative evidence to other evaluations conducted by the Office; and
• provide feedback and knowledge sharing to (1) the GEF Council in its decision making process to allocate resources and to develop policies and strategies; (2) the country on its participation in, or collaboration with the GEF; and (3) the different agencies and organizations involved in the preparation and implementation of GEF funded projects and activities.
12. The main focus of the CPEs and CPSs included in this ACPER was the projects supported by the GEF at all stages (preparation, implementation and completion) within the national boundaries. The Small Grants Programme (SGP) was assessed against the respective national strategy and not on the basis of each individual SGP grant. Project ideas from either the governments or GEF Agencies included in the respective pipelines were not considered in the analysis. In addition to national projects, the GEF portfolios assessed include a selection of regional and global projects selected according to a set of criteria, including the presence in the country of a project coordination unit and/or project sites; the importance of the project focal area to the country; and the existence of a clear connection to national projects. 13. The stage of each project determined the evaluation focus. For example, completed projects were assessed against the usual three evaluation criteria, namely effectiveness and results (outputs, outcomes and impacts), relevance and efficiency. Ongoing projects were assessed in terms of relevance and efficiency. Projects under preparation, i.e. those with an approved Project Identification Form (PIF) or Project Preparation Grant (PPG), were assessed
4 http://www.thegef.org/gef/CPE%20Standard%20Terms%20of%20Reference 5 http://www.thegef.org/gef/CPS%20Standard%20Terms%20of%20Reference 6 Effectiveness: the extent to which the GEF activity’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Results: in GEF terms, results include direct project outputs, short- to medium-term outcomes, and progress towards longer term impact including global environmental benefits, replication effects, and other local effects. 7 Relevance: the extent to which the activity is suited to local and national environmental priorities and policies and to global environmental benefits to which the GEF is dedicated. Efficiency: the extent to which results have been delivered with the least costly resources possible.
10
primarily in terms of relevance, with some eventual limited assessment of efficiency. The results and sustainability of GEF support, particularly at the global environmental benefits level, were given special attention. Table 1.4 presents the portfolios of projects covered in the CPEs and CPSs included in this ACPER.
Table 1.4 Project Coverage of each Country Portfolio Evaluation and/or Study Country Type of
evaluation Number of projects included in the evaluation National
completed projects
National FSPs & MSPs
SGP Enabling Activities
Regional/ global projects
Eritrea CPE 6 Yes 6 12 8 Sierra Leone CPS 9 Yes 6 16 6 Tanzania CPE 21 Yes 7 53 14
14. The Office’s country level evaluations team continues to strive to update and further develop the set of quantitative and qualitative methods and tools used in these evaluations. Country level evaluations processes, methods and tools can be found on the Office website.8 A new method for analysis of country ownership has been tested during the country level analysis conducted in the framework of the first report of the fifth Overall Performance Study of the GEF (OPS5). A framework was elaborated and used for a meta-analysis of GEF IEO country level evidence, which is based on the five pillars of the Paris Declaration (ownership, alignment, harmonization, managing for results and mutual accountability)9. The Office plans to develop this framework beyond meta-analysis with a view of using it as original evaluative data gathering in future country level evaluations. Other tools are being developed for the country level analysis of private sector and CSO engagement, and gender mainstreaming. 15. As for previous country level evaluations, also for the CPEs and CPSs reported in this ACPER, statistical data and scientific sources were consulted, particularly with regard to national environmental indicators, wherever available. Interviews were conducted with representatives of all GEF stakeholders in the countries concerned, and a conspicuous number of field visits were made. As mentioned, each of the CPEs and CPSs included a national consultation workshop to discuss and receive feedback on the respective key preliminary findings. The quantitative analysis used indicators to assess the efficiency of GEF support using projects as the unit of analysis (e.g., analyzing projects preparation and implementation durations and costs). Progress towards impact was assessed through Review of Outcome to Impact (ROtI) field studies (three studies were conducted in the Tanzania, two in Eritrea and one in Sierra Leone) as well as using the databases pertaining to the Sub-Saharan Africa region extracted from the 90 terminal evaluations of the OPS5 broader adoption and progress to environmental impact analysis. 16. Systematic triangulation of evaluative evidence continues to be applied consistently in country level evaluations conducted by the Office. This included both the Tanzania and Eritrea CPEs. The application of systematic triangulation ensures that the cross-analysis of information results in better understanding of the contributions of the GEF initiatives in the country portfolios analyzed. The Office has conducted an analysis of how effective systematic triangulation as a method has been in achieving is main purpose, i.e. identifying key preliminary evaluation findings at the end of the data gathering and analysis phase, 8 http://www.thegef.org/gef/CPE 9 A full description of the framework and the analysis conducted is described in the 'OPS5 Technical Document 6', available on: http://www.thegef.org/gef/sites/thegef.org/files/EO/TD6_Meta-Evaluation%20on%20Country%20Ownership%20and%20Drivenness.pdf
11
drawing on a total of 9 evaluations in which it has been applied to date. This analysis has shown that for 91% of the initial evaluation questions the method allows for identification of strong findings, contradictory evidence and/or gaps in the analysis, which subsequently leads to ways of conducting further evaluative work and finalize the identification of findings. In the spirit of contributing to the discussion on mixed-methods evaluation an article has been prepared and submitted to the European Evaluation Journal for publication. 17. Joint work with GEF member countries and Agencies continues to be pursued in the conduct of country level evaluations. This ACPER includes one such joint effort, the Sierra Leone CPS. A unique feature of joining efforts with national evaluation expertise is the use of national independent quality assurance / peer review panels, which have continued to support country portfolio evaluations this year both for the Tanzania and Eritrea CPEs. These panels not only provide scientific / technical and methodological support to these evaluations, but also increase their credibility, ownership and potential use in the country, in the sense that they could enhance the credibility and use of the evaluation through follow-up action, especially concerning the recommendations addressed to the countries themselves. 18. GEF country level evaluations face a number of limitations and challenges. The following were at play in the CPEs and CPS summarized in this report:
• Difficulties in defining the portfolio prior to the evaluations continue to be a limitation in country level evaluations as well as in many other evaluations conducted by the Office. To address this limitation country portfolios extracted from PMIS have been carefully cross-checked with GEF Agencies and national stakeholders at the early stages of the evaluation.
• In all the three countries, the GEF is not the only international institution operating in the environment sector. Caution must therefore be exercised in attributing any changes to the interventions of the GEF, while assessments of contribution need to take realistic account of the number and scale of other inputs. This is particularly the case for macro-level changes in the field of environmental policies, strategies and national plans.
• In the three portfolios analyzed the evaluation teams experienced difficulty in obtaining readily available and accurate data on some of GEF’s earlier activities. Furthermore, the quality of evaluative evidence, particularly quantitative trends data, relevant to completed projects is variable, posing challenges to building a comprehensive overview of results and contribution. The teams addressed this limitation through triangulation and cross-checking across sources and key informants.
• The Sierra Leone CPS was postponed several times due to the fact that it had to be conducted in parallel with the UNDP Sierra Leone Assessment of Development Results (ADR), which was postponed due to reasons linked to the need of making the evaluation coinciding with UN country programming activities in the country.
12
CONCLUSIONS Results 19. Results are presented in terms of the aggregate outcomes and impacts of GEF support. Achievements are presented in terms of GEF contribution towards addressing global and national environmental issues as well as national level priorities, including raising awareness and building national institutions and capacities. Reporting on results starts from a concise description of the main findings of a progress to impact analysis in the Sub-Saharan Africa region. This analysis has been conducted using data from the terminal evaluations undertaken in the region, which have been extracted from the terminal evaluations cohort analyzed in OPS5.10 20. In assessing the extent of progress towards impact of GEF support, OPS5 focused on the importance of broader adoption mechanisms taking place during and especially after GEF projects had ended, and the factors that contributed to or hindered this. Specifically, the analysis aimed at assessing in greater detail the status of three research areas at project completion: 1) the extent and scale of broader adoption; 2) the extent and scale of environmental impact, expressed in terms of either environmental stress reduction and/or improved environmental status; and 3) the factors contributing to and hindering progress towards impact, both related to the project and the larger context in which they operate. OPS5 analysis identified five mechanisms for broader adoption to take place:
• Sustaining: where a GEF intervention continues to be implemented without GEF support through clear budget allocations, implementing structures and institutional frameworks defined by the government and/or other project stakeholders. The sustained flow of benefits of the intervention is important to demonstrate the benefits and to provide incentives for adoption by other stakeholders.
• Mainstreaming: Information, lessons, or specific results of the GEF are incorporated into broader stakeholder mandates and initiatives such as laws, policies, regulations, and programs.
• Replication: GEF-supported initiatives are reproduced or adopted at a comparable administrative or ecological scale, often in another geographical area or region.
• Scaling-up: GEF-supported initiatives are implemented at a larger geographical scale, often expanded to include new aspects or concerns that may be political, administrative, or ecological in nature.
• Market change: GEF-supported initiatives catalyze market transformation by influencing the supply of and/or demand for goods and services that contribute to global environmental benefits.
21. OPS5 conducted the analysis described here above on a cohort of 473 terminal evaluations. Out of these, 90 terminal evaluations were from Sub-Saharan Africa. The analysis on this cohort (Table 2.1) indicates that over 54% of the 90 projects successfully implemented most or some broader adoption initiatives, while in the global portfolio 60% of projects achieved the same. Across the Sub-Saharan Africa portfolio, environmental impact was reported – either in the form of environmental stress reduction or environmental status change – in 48% of projects (43), against the global total of 65%. 10 The OPS5 analysis on progress to impact covered the terminal evaluations submitted from 2005 to 2012.
13
Table 2.1: Extent of broader adoption in the Sub-Saharan Africa cohort of available terminal evaluations (n=90)
Extent of broader adoption # %
Most broader adoption initiatives adopted/ implemented 13 14.44% Some broader adoption initiatives adopted/ implemented 36 40.00% Some broader adoption initiated 27 30.00% No significant broader adoption taking place 14 15.56% Grand Total 90 100.00%
22. The most common project-related factors contributing to progress towards impact emerging from the 90 terminal evaluations in the region (Table 2.2) were ‘Good engagement of key stakeholders’ (42.22%) and ‘Good coordination with/ continuity of previous/ current initiatives’ (37.78%), while the most common contextual factors contributing to progress towards impact were ‘Country/Government support’ (56.67%) and ‘Previous/current related initiatives’ (52.22%). The most common project-related factors hindering progress towards impact emerging from the 90 terminal evaluations were ‘Poor project design’ (42.22%) and ‘No activities to sustain momentum’ (28.89%), while the most common contextual factors were ‘Other unfavorable political conditions/events’ (40%), ‘Unfavorable economic conditions/drivers/events’ and ‘Lack of government/country support’ (the latter two at 27.78%). 23. ‘Project-related contributing factors’ is an area where there is some divergence between the Sub-Saharan Africa region and global trends. It appears that ‘Broader adoption processes initiated using project resources’ (23.33% in the region against 39.4% globally) and ‘Highly relevant technology/approach’ (25.56% in region against 36.2% globally) are two less frequent factors in the terminal evaluations of the Sub-Saharan Africa region cohort as compared to the trends and averages of the global cohort. ‘Good coordination with or continuity of previous/current initiatives’ (37.78%) appears to be more frequent in the 90 terminal evaluations in the region as compared with global trends (29.6%). 24. A closer look at the data for the region was taken to identify which factors are more frequeent in more successful versus less successful projects. For the 41 projects that were partially successful and unsuccessful in achieving broader adoption, ‘Poor project design’ (20 projects), ‘No activities to sustain momentum’ (15 projects) and ‘Inappropriate and/or insufficient technology approach’ (11 projects) were the most frequent factors. For the 49 moderately and highly successful projects, the most frequent factors were ‘Good coordination with or continuity of previous/current initiatives’ (23 projects), ‘Good engagement of key stakeholders’ (21 projects) and ‘Highly relevant technology/approach’ (13 projects).
Table 2.2 – Most common factors affecting progress towards impact in SSA (n=90)
Project-related Contributing Factors SSA Global
Good engagement of key stakeholders 42.22% 48.2% Good coordination with/ continuity of previous/ current initiatives 37.78% 29.6% Highly relevant technology/approach 25.56% 36.2% Broader adoption processes initiated using project resources 23.33% 39.3% Follow-up initiatives using GEF resources 6.67% 7.8% Good project design (other than those mentioned above) 5.56% 9.3%
14
Adaptation of project to changing contexts 5.56% 5.5% Previous GEF support 5.56% 5.5% Extended implementation period 1.11% 1.7%
Contextual Contributing Factors
Country/Government support 56.67% 56.7% Previous/current related initiatives (by government, global events, etc.) 52.22% 55.4% Other stakeholder support (e.g. donors, private sector) 38.89% 42.3% Other favorable political conditions/events 18.89% 17.5% "Champions" 6.67% 6.1% Favorable economic conditions/drivers/events 6.67% 9.1% Favorable social conditions/drivers/events 4.44% 4.0% Favorable environmental conditions/drivers/events 2.22% 3.0%
Project-related Hindering Factors Poor project design (other than factors above) 42.22% 37.6% No activities to sustain momentum 28.89% 24.7% Inappropriate/insufficient technology/approach 16.67% 16.1% Poor project management 13.33% 13.7% Insufficient time for implementation 12.22% 10.8% Inability to adapt project to changing context 1.11% 5.9%
Contextual Hindering Factors
Other unfavorable political conditions/events 40.00% 39.5% Unfavorable economic conditions/drivers/events 27.78% 31.1% Lack of government/country support 27.78% 25.8% Lack of other stakeholder support (e.g. donors, private sector) 20.00% 18.2% Unfavorable social conditions/drivers/events 20.00% 14.0% Unfavorable environmental conditions/drivers/events 11.11% 7.2%
Note: Global figures source – OPS5 TD#12: Progress towards Impact 25. This analysis provides further context to – and in some cases confirms – the conclusions and recommendations emerging from the three country level evaluations conducted in the Sub-Saharan Africa region. These are presented in the following paragraphs. Conclusion 1: GEF support has played an important role in creating the enabling framework necessary to underpin the development of environmental policy and laws in the three countries analyzed in the Sub-Saharan Africa region. 26. In recent years, several country level evaluations and ACPERs acknowledged the effectiveness of GEF foundational support to Least Developed Countries (LDCs) and Small Island Developing States (SIDS) in creating an enabling framework for environmental management through Enabling Activities in all GEF Focal Areas. Drawing from available country level evidence to date, the first OPS5 report reiterated the importance and uniqueness of GEF foundational support to LDCs and SIDS. Effectiveness of such support is again confirmed in the three country level evaluations conducted in the Sub-Saharan Africa region, with few exceptions. 27. GEF support has been particularly effective in developing institutional and individual capacity for environmental management in the three countries. In Tanzania GEF support has
15
helped creating the conditions for developing national environmental policies and laws. This has included the development of several national plans and strategies, which were necessary precursors for implementation of Multilateral Environmental Agreements (MEAs). A further example is the Marine and Coastal Environment Management Project (GEF ID 2101), which facilitated important changes to the development of the marine environmental legal framework in Tanzania. 28. GEF started work in Sierra Leone in 1996, with the pipeline entry of the project to support preparation of the First National Communications to the UNFCCC (GEF ID 296). However, because of the disruption caused by the civil war, the project could not become effective and start implementation until the end of the war in 2002. That and other GEF Enabling Activities were therefore implemented after the end of the civil war, between 2001 and 2008. They resulted in the preparation of consolidated national environmental strategies and plans. They also enabled Sierra Leone to meet its obligations to the main international conventions. These plans provided a basis for development of national Medium Size and Full Size Projects (MSPs and FSPs) that comprehensively address environment and natural resource management. A number of such projects have been developed and commenced operation since 2010 with GEF funding. Enabling Activities also contributed to the 2008 amendment to the Environmental Agency Act. 29. Half of the GEF portfolio under review in Eritrea is composed of Enabling Activities. These activities across all focal areas have enabled for priorities to be defined and set to focus on commitments to the various international environmental conventions. Examples include the National Biodiversity Strategy and Action Plan (NBSAP) (GEF ID 137), the National Adaptation Programme for Action (NAPA) (GEF ID 1959) and the National Action Plan (NAP) (GEF ID 1584) for land degradation activities. The Enabling Activities have also enhanced environmental knowledge, understanding and capabilities of personnel of all ranks of the executing ministries and communities. 30. While the GEF portfolio overall has been effective in laying the foundation for future action in Eritrea’s environmental protection and the country did ratify major international environmental agreements and approve the final reports of the NAPA, NAP, and NBSAP, the full implementation of those national reports and strategies is still being pursued. At present, several key environmental related items of legislation remain in “draft” form and have not been promulgated into legally binding acts. While environmental protection has not been majorly hampered by the lack of a full policy framework, key enforcement tools are not in place to abate and deter environmental degradation and allow for further evolutions in sustainable land and coastal management, which would result from, for example, secure land tenure laws. The official endorsement of all elements of national environmental legislation, which has been in draft for almost a decade, will assist better environmental management in the country. Full implementation of policies in place such as national coastal policy identified and drafted through the NBSAP, the forest and wildlife policy and the wildlife conservation and development policy are intended to guide for enhanced conservation of globally significant flora and fauna. Conclusion 2: Positive results at completion have been achieved beyond foundational support, leading in some cases to progress towards impact. 31. Post-completion progress towards impact analysis was conducted on the 7 available terminal evaluations extracted from the OPS5 cohort of 90 terminal evaluations in the region
16
(Table 3.1). All projects were at least partially successful, with 5 moderately successful and 1 highly successful in achieving progress towards environmental impact. The only partially successful project was the Coastal, Marine and Island Biodiversity Project (GEF ID 411) in Eritrea.
Table 3.1: Progress towards Impact in the available terminal evaluations from the SSA region (N=7) GEF ID
Focal Area Agency Modality Project Title Broader
adoption ESR ESC
411 BD UNDP FSP Conservation Management of Eritrea's Coastal, Marine and Island Biodiversity
Partially successful
780 BD UNDP FSP Development of Mnazi Bay Marine Park
Moderately successful
803 BD UNDP MSP Development of Jozani-Chwaka Bay National Park, Zanzibar Island
Moderately successful
1170 BD WB FSP Conservation and Management of the Eastern Arc Mountain Forests
Highly successful
1196 CC UNDP FSP Transformation of the Rural Photovoltaics Market
Moderately successful
1734 BD UNDP MSP Development and Management of the Selous-Niassa Wildlife Corridor
Moderately successful
2151 BD WB MSP Novel Forms of Livestock & Wildlife Integration Adjacent to Protected Areas in Africa
Moderately successful
Note: ESR = Environmental Stress Reduction; ESC = Environmental Status Change.
32. The most common contextual factor contributing to progress towards impact in this cohort is ‘Country/government support’ (6 projects), while the most common contextual hindering factor is ‘Unfavorable economic conditions/drivers/events’ (5 projects). As for the project-related contributing factors, ‘Highly relevant technology/approach’, ‘Broader adoption processes initiated using project resources’, and ‘Good engagement of key stakeholders’, are all equally important and seen in 4 projects. The most common project-related hindering factors are ‘Poor project design’ cited in 3 projects. A certain degree of caution must be exercised with the results of this analysis due to the small cohort of terminal evaluations – 7 out of a total of 17 for the three portfolios analyzed in this APCER. 33. Within these limits, three project-related factors for achieving progress towards impact emerging from this analysis are worth mentioning: enhancing coordination and fostering engagement of key stakeholders; initiating broader adoption processes during the project to foster sustained flow of benefits after completion; and ensuring sound and realistic project design.
34. The importance of country/government support for achieving broader adoption clearly emerges among the context related factors from this analysis. In fact, OPS5 impact analysis clearly indicates that broader adoption mechanisms need to be incorporated into the design of the project. Although country/government support is a context related factor, projects can foster the conditions that are necessary to sustain country/government ownership and support
17
after completion, especially in LDCs. Examples include any activity aiming at fostering inclusiveness and consultation, developing capacity, or providing information that the government considers essential for its own sustainable development/green-growth agenda.
35. The above analysis from OPS5 data is complemented by the findings from the three country portfolios analyzed in this ACPER, which confirm the overall positive outcomes achieved at completion, with few exceptions. At project level, among the 14 completed projects for which a terminal evaluation is available in the Tanzania national portfolio, two (15%) were rated highly satisfactory in terms of outcomes achievement, ten (71%) were rated satisfactory, one (8%) was rated moderately satisfactory and one was rated unsatisfactory. In Eritrea, one of the two available terminal evaluations, for the Conservation Management of Eritrean Coastal, Marine and Island Biodiversity Project (GEF ID 411) rated it satisfactory in terms of outcomes achievement. However, the terminal evaluation review of that same project conducted by the Office downgraded it to marginally unsatisfactory. The other terminal evaluation conducted in Eritrea, on the Wind Energy Applications Project (GEF ID 1136) broadly describes the project as satisfactory. This rating was confirmed by the respective terminal evaluation review conducted by the Office. 36. As far as progress to impact is concerned, ROtI analysis conducted on three completed projects in Tanzania and the two completed in Eritrea was consistent and in some cases upgraded the rating of the terminal evaluation reviews for these projects. In the case of the Jozani-Chwaka Bay National Park, environmental status changes were seen, for example, in the now healthy numbers of Columbus monkey endemic to that area. The market transformation supported by the PV Transformation of the Rural Photovoltaics Market Project (GEF ID 1196) in Tanzania has contributed to a stable regulatory system for solar energy, although the displacement of GHGs due to any installed capacity is not quantified or at scale. Progress to impact is also demonstrated after the Coastal, Marine and Island Biodiversity Project (GEF ID 411), which contributed to mainstreaming scientific information and has also contributed, with other conditions, to thus-far sustained protection of globally significant species of coastal /marine flora and fauna. ROtI analysis conducted on the only completed project in the Sierra Leone CPS, the Sustainable Land Management Project (GEF ID 3510) showed low or negligible progress towards impact. 37. Some examples of broader adoption mechanisms in place in Tanzania and Eritrea are worth mentioning. Mainstreaming in Tanzania has been fostered by means of GEF support to development of a cadre of trained professionals in environmental fields and support for creation of environmental management institutions in the region (e.g. Lake Victoria Basin Commission, Deep Sea Fishing Authority, among others). Some aspects of the Transformation of the Rural Photovoltaics Market Project (GEF ID 1196), such as the installation of solar photovoltaic systems in schools and dispensaries around the project sites of the Development of Jozani-Chwaka Bay National Park Project (GEF ID 803) in Zanzibar were replicated by subsequent SGP grants. The overall project approach was also replicated in three other regions (Mara, Shinyanga and Kagera). In Eritrea, the research unit in the Ministry of Marine Resources established with support from the Coastal Marine Island Biodiversity Project (GEF ID 411) is now mainstreaming research with other institutions, including universities abroad, providing valuable information exchange on, among other things, sea and coastal migratory birds, several species of turtle nesting grounds, species of crabs, coral reefs, sea grass, and dugongs.
18
38. In Eritrea, as observed and reported through stakeholders, as well as by communities engaged in the two Sustainable Land Management (SLM) projects (GEF ID 3362 and GEF ID 2009), besides land degradation measures, the capacity of communities to use drip-irrigation in small farms, building bench terraces and managing nurseries has also been enhanced. GEF’s land degradation portfolio has also included a community component dedicated to improved energy saving stoves. These have been constructed by and largely benefited women and achieved significant replication to amongst Eritrean households in village communities through the SGP program. Overall, the GEF supported projects to combat land degradation have demonstrated solid results through activities such as reforestation while at the same time enhancing farm productivity. 39. Examples of scaling-up include the Development of Jozani-Chwaka Bay National Park Project (GEF ID 803), which promoted a programme to enable sustainable increases in income in communities around the forest reserve. This programme has been considerably expanded from its original scope since project completion. Another example of market change is the reduction in tariffs on solar panels, mentioned earlier, which was supported by the Transformation of the Rural Photovoltaics Market Project (GEF ID 1196), which impacted on reducing market barriers to solar photovoltaic systems installation in rural areas. Conclusion 3: Likelihood of sustainability is mixed, and has been most successful when pursued through fostering institutional and individual capacity development as well as promoting livelihood activities through community-based approaches, i.e. the SGP. 40. Approaches intended to foster sustainability in terms of continued flow of environmental benefits as well as overall results achieved have been applied in GEF projects. Here, the most successful efforts have been those aiming at developing local capacities as well as linking local community benefits to improved environmental management. This was achieved through parallel and in some cases synergistic support in the GEF portfolio, where the FSPs and MSPs as well as the SGP modalities have been used. In some cases, the likelihood of sustainability post-project completion has lowered as compared with during project funding. 41. Specific measures have been taken in Tanzania to provide a basis for sustainability, including capacity and institutional development and the development and implementation of environmental management systems. For example, GEF support to institutional development of the Jozani-Chwaka Forest Reserve through the Development of Jozani-Chwaka Bay National Park Project (GEF ID 803) allowed it to be upgraded to a national park, and a management plan was developed in consultation with local stakeholders. These outcomes have been maintained and in some case expanded. There are now 736 savings and credit groups compared with 47 at project completion, reported to stimulate income generating activities. Conversely, the Eastern Arc Mountain Project’s (GEF ID 1170) gains, at a period of time well after project close, were thought to be under threat due to its failure to implement a sustainable financing strategy to secure funding for its long term objectives. More recently, however, evidence suggests that the sustainability of the fund set-up may be better than anticipated, with the Government of Germany recently providing 2 million Euros for conservation efforts. 42. Another important approach to sustainability has been through efforts to link local community benefits to improved environmental management. For example, in the Eastern Arc Mountain Conservation Project (GEF ID 1176), support for local livelihoods helped to
19
generate support for environmental management. More importantly, SGP grants, which by definition link environmental conservation activities with sustainable livelihoods, have been creatively used in a variety of ways in Tanzania. Many have been implemented in parallel with FSPs and MSPs to provide community stimulus to participate in environmental management by supporting Income Generating Activities (IGAs).
43. In one case the SGP - through the Small Grants Kilimanjaro COMPACT Project – is delivering the community-based components of the ongoing SIP-Reducing Land Degradation on the Highlands of Kilimanjaro’ (GEF ID 3391), supporting community-based organizations for tree-planting, shade-grown coffee and bee-keeping, which are in keeping with both the SGP and the FSPs objectives. Another example is the Development and Management of the Selous-Niassa Wildlife Corridor Project (GEF ID 1734), which obtained support from the SGP to finance local initiatives to promote IGAs in a broad range of villages in the project area. 44. Similarly, the Eastern Arc Mountains Project (GEF ID 1176) obtained SGP funding to promote butterfly farming as an IGA around Amani Nature Reserves, in order to reduce pressure on forest products. The SGP Amani Butterfly Farming Project promoted butterfly farming in the Eastern Usambara Mountains, resulting in conservation of biodiversity, since butterfly farmers and their communities are now conserving natural forests to protect host plants for their butterfly farming. This butterfly activity introduced by the SGP has also been successfully replicated by the GEF FSP implemented near Jozani-Chwaka Bay National Park in Zanzibar (GEF ID 803). Another example is the SGP Installation of Solar Photovoltaic Systems in Schools and Dispensaries around Jozani-Chwaka Bay National in Zanzibar, which extended the activities of the same GEF FSP, resulting in improved access to electricity and lighting through the installation of solar photovoltaic systems in dispensaries and secondary schools. 45. In Eritrea, the Catchments and Landscape Management Project (GEF ID 3362) and the Sustainable Land Management Pilot Project (GEF ID 3364) provide specific examples of linking local livelihood benefits and sustainability. Another example is the widespread dissemination of improved stoves introduced by the SGP, to the extent that these started to become an IGA for women, many of whom are single-headed households. The constant power interruption in all areas of the country connected to the national grid has forced many urban and rural households to build these stoves. Parallel to that, the SGP has provided good opportunities for a number of communities and NGO groups to learn from each other’s experiences and to replicate the results of GEF support. Several communities in and around the project sites are engaged in nature-based conservation activities. 46. A challenge related to the sustainability of results in Eritrea remains the limited capacity at both individual and institutional levels, which strengthening has been targeted through the numerous Enabling Activities (6 out of 12 national projects) as well as dedicated training components of FSPs targeting government staffs and institutions. Another challenge for Eritrea has been to continue supporting and scaling up once funding has ended, despite the government’s efforts to sustain the outcomes achieved. The exit strategies put in place have not adequately addressed the financial, technical and managerial sustainability of projects outcomes. Conclusion 4: Wider dissemination and uptake of project-derived lessons at national level has been hampered by language barriers.
20
47. This conclusion draws uniquely from the Tanzania CPE, but its relevance goes beyond that country and is applicable in several other GEF member countries. The Tanzania CPE highlights that a number of GEF-supported activities, including the Enabling Activities, have targeted the preparation of important documents for dissemination and use nationally and internationally. These also included lessons shared directly with communities and groups at local level, on conservation and restoration know-how, organizational improvement, technology adoption, scaling-up and marketing. While in the earlier GEF projects in Tanzania efforts of translating in the local language were common, stakeholders reported that almost all recent GEF project documents, reports and learning products (hard and soft) in Tanzania are written in English (understood by less than 20% of the population) and are highly technical, resulting in the fact that many project participants and environmental stakeholders can neither understand nor share the content. Relevance 48. Relevance of GEF support was assessed against the country’s national development and environmental agendas, the GEF mandate, and the country’s responsibilities and obligations towards the global conventions. Conclusion 5: GEF support has been and remains relevant to the countries’ environmental priorities as well as to their sustainable development needs, with few exceptions. 49. The three portfolios analyzed confirm strong relevance to national environmental conservation and sustainable development needs and priorities in all the GEF focal areas, with few exceptions in Eritrea. 50. Over the last twenty years, GEF support has played a significant role in helping to raise awareness on the environment as a vital cross-cutting issue for the sustainable development of Tanzania. It has also laid the foundations for the mainstreaming of environmental issues into a range of sectoral policies and plans. Most projects in the GEF Tanzania portfolio are well-aligned to national priority areas as delineated by the government. More specifically, GEF support is contributing to the national sustainable agenda stated in MKUKUTA II and MKUZA II (National Strategy for Growth and Reduction of Poverty II 2010-2015 and Zanzibar Strategy for Growth and Reduction of Poverty 2010-2015) by contributing to activities that have a positive impact on biodiversity conservation, restoration of ecosystems, demonstration of green energy sources, improvement of health care and primary and secondary school education (solar photovoltaic systems), irrigation to increase agricultural productivity and production levels and other income generation activities which are important to communities and national welfare. Similarly, the relevance of the SGP is demonstrated by strong support by the Tanzanian Government, even though its activities are executed by civil society organizations, NGOs and community groups with the assistance of UNDP. Government support was re-confirmed under the Tanzania National Portfolio Formulation Exercise (NPFE), during which the sum of $3.6 million was suggested as an appropriate amount for the SGP out of Tanzania’s allocation through the System for Transparent Allocation of Resources (STAR) of $27.43 million. 51. The GEF portfolio has been relevant to Sierra Leone’s sustainable development agenda and needs. It addresses one of the main pillars of Sierra Leone’s National Development
21
Strategy - Pillar 2 of the country's Agenda for Prosperity (AfP), the third Poverty Reduction Strategy Paper (PSRP) which is on managing natural resources. GEF Enabling Activities have been catalytic and have laid the foundation for follow up activities in biodiversity and climate change making it possible for the country to fulfill its obligations to the UNFCC, UNCBD and UNCCD. GEF support in the area of climate change is highly relevant in allowing the country to address issues on adaptation and mitigation of climate change including development of adaptive agricultural production systems. GEF support in the area of desertification and land management fitted well within local needs due to fact that it addresses one of most pressing constraints in agriculture - the principal livelihood means for rural people, namely soil fertility and land degradation issues. 52. GEF Enabling Activities were relevant to the national sustainable development agenda of Eritrea, as demonstrated by the fact that the NBSAP, the NAPA and the NAP were all officially approved by the government. Besides foundational support, the Wind Energy Applications Project (GEF ID 1136) was relevant to the country’s piloting of renewable energy sources and put in place the policy structure that would allow a renewable energy sector to develop. The two SLM projects (GEF ID 3362 and GEF ID 2009) were developed in 2002, when Eritrea launched its NAP to combat desertification and mitigate the effects of drought. While these GEF supported activities have been relevant to the country’s needs in terms of sustainable land management, Eritrea’s Land Law No. 58 of 1994, empowering communities in land use and related land management aspects, has yet to be approved. 53. An opportunity to further enhance the overall relevance of GEF support exists in the Eritrean renewable energy sector. Through the national communications to the UNFCCC in 2001 and 2012, Eritrea had identified a number of renewable energy sources such as geothermal, wind and solar energy. In Eritrea, 99% of electricity is generated using fossil fuels and even though fuel costs are rising and there is a lack of access to energy, the government has yet to fully develop those alternate energy sources. Except for the Wind Energy Applications Project (GEF ID 1136) there have been no further GEF interventions in this area. The promotion of the development of alternative and renewable energy production in areas with no access to pre-existing grids, both for the household and industrial unit level, would be strongly relevant to the country’s needs, and should be pursued in GEF-6. Conclusion 6: Diverse degrees of ownership have been observed in the three portfolios analyzed. 54. Discussions with key stakeholders found a consistent viewpoint that, under GEF-5, as a result of STAR the governments’ ownership of the portfolio has increased and they have become more empowered in setting priorities and decisions on funding their own environmental priorities, where these overlap with global environment issues. The use of the countries’ own financial and other resources along with donor funding demonstrates ownership in Tanzania and Eritrea. Both have been successful in mobilizing their own resources as well as co-financing from international sources. Sierra Leone showed weaker ownership, except for Enabling Activities. 55. GEF projects in Eritrea have been strategically prioritized by the GEF Operational Focal Point (OFP), taking into account the existing opportunities and constraints, relevance to national agenda and a projects’ objective. Ownership is further demonstrated by the fact that GEF projects in Eritrea originate within national institutions, including the Department of Environment, the Ministry of Agriculture and other government agencies.
22
56. Important support was given by the Tanzanian government for GEF projects (initially mainly in kind) and the measures it has taken to help ensure the sustainability of the results of completed projects, notably by allocating funds to maintain key activities. Whilst many such funds are obtained from international partners, it is also important to note that there has been a substantial increase in the budgeted commitment of national funds dedicated to addressing environmental issues: from Tanzanian Shillings 28.4 billion in 2006/2007 to 151.7 billion in 2009/2010. Though the government’s resources are strained, it does provide funds of its own towards various institutions which contribute to global environmental benefits. 57. The GEF Sierra Leone portfolio has been mainly designed by the GEF Agencies. The government and other stakeholders have committed to activities at various stages of design and implementation, but cannot be said to have led the process, except in the case of the Enabling Activities for the national communications to the UNFCCC. Efficiency 58. Efficiency of GEF support was assessed in terms of time, effort, and financial resources needed to prepare and implement GEF projects; the different roles and responsibilities of the various GEF stakeholders (national, international, and local) and the synergies between projects and these stakeholders. Conclusion 7: Project design factors, particularly over-ambitious objectives, have often caused implementation over-runs. 59. This conclusion confirms the importance of realistic project design highlighted by the progress to impact analysis described earlier in this report. In Sierra Leone, FSPs take much longer to plan and get started and have a greater implementation over-run. In Eritrea, all the 6 FSPs have taken more than 18 months from work-program entry stage to CEO approval. In Tanzania, project document review found that FSPs have consistently over-ambitious objectives, which lead some of them to incur substantial time over-runs. 60. In Sierra Leone, GEF enabling projects and the MSP prepared after the end of the Civil War between 2001 and 2008 have been prepared under the 22 months limit later imposed for GEF 4. However, the FSPs under implementation designed under GEF 4 and 5 have project cycle times that are significantly longer than the GEF guidelines of 22 months under GEF 4 and 18 months under GEF 5. All GEF Agencies have had delays whether with resident or nonresident representation in the country. Delays have been due to the time it takes to collect background information in a situation where there are no centralized data banks on environmental issues thus necessitating field data collection from target communities; the process of identifying and recruiting consultants, often international, due to the limited human resource capacity available in collaborating national institutions; and the extended procedures for project approval in the GEF Agencies. However, there are recent welcome indications that project cycle durations are becoming shorter. 61. Overambitious project design was cause of delay and difficulties in achieving project outcomes in the case of the Sustainable Land Management project (GEF ID 3510), as reported in the ROtI study conducted in the framework of the Sierra Leone CPS. The project was not able to complete more than 3 pilot sites as opposed to the 10 proposed in the project documents. Furthermore, the ROtI analysis indicates that the capacity is already being lost,
23
just 12 months after project end. Possible reasons include limited human resources capacity and key persons no longer being in their positions to follow up on after project activities. The project was also to submit the draft NAP for sustainable land management in Sierra Leone for approval by parliament, but did not get to drafting of the required legislation. 62. In Eritrea, stakeholder views suggested that project formulation processes for preparation of GEF projects are perceived as complex and time consuming. The Conservation management of Eritrea's Coastal, Marine, and Island Biodiversity (GEF ID 411) was supposed to close in 2003, but it closed 5 years later, in 2008. The Wind Energy Applications Project (GEF ID 1136) was supposed to close in 2007, but it closed two years later, in 2009. The Enabling Activities have also been running over their allocated times: 3 out of the 5 projects we delayed, the first by 1 years, and the other two by 2 years. 63. A factor that negatively influenced achievement of results and likelihood of sustainability is the realism of project’s initial design, which in some cases has been overambitious in Eritrea. For example, the Coastal and Marine Island Biodiversity Project (GEF ID 411) originally attempted to cover the whole of the Red Sea coast of Eritrea and its islands. While the scope was later revised to focus on more limited coverage, precious time and resources were initially exhausted. 64. In Tanzania, the project cycle reforms introduced during the GEF-4 cycle will take time to be realized in the performance of GEF projects. If for the 3 closed FSPs none were delayed in their implementation, in the on-going portfolio 3 FSPs have already run over their original intended duration by 1 to 3 years. Of the 5 closed MSPs, 1 was delayed in closing by 1 year and the 1 under implementation is delayed by two years in its expected close. Besides, all 6 completed Enabling Activities were delayed in their close. The one currently under implementation – begun in 2012 with expected duration of 1 year – is already delayed. Prevalent views suggested that even with project formulation support, processes of preparation are still complex and time consuming. Also, national institutions expressed major challenges in both understanding the nature of co-financing and also meeting the requirements, which stakeholders suggested, on occasion, threatens the “grant” nature of GEF funding. Conclusion 8: Monitoring and Evaluation is mixed in the three portfolios, and efforts are deployed to improve the situation where needed. 65. Despite efforts in specific projects, M&E systems in Tanzania are not yet uniformly regarded as an important asset to the design, management and adaptation of projects, and insufficient emphasis is placed on M&E by national partners and project managers. In Eritrea M&E is in place but it is yet to be fully used for adaptive management, although some progress is being made to improve the situation. In Sierra Leone M&E systems are in place and are appropriate thus far. 66. In Tanzania, the Eastern Arc Mountain Forests Project (GEF ID 1170), the M&E system was weak and failed to facilitate adaptive management of the “problem” project. The Development of Jozani-Chwaka Bay National Park Project (GEF ID 803), the interests of multiple financiers resulted in a confusing and “top-heavy” M&E system. Issues in the Development of the Mnazi Bay Marine Park Project (GEF ID 780) included, managers ignoring recommendations from the Mid-term Review, lack of a coherent M&E framework, and an untouched logframe. In other projects, substantial efforts have been made to
24
strengthen both current and future M&E capacity. During the Pangani Basin Water Management Project (GEF ID 2832) an M&E study was undertaken which lead to a series of recommendations for skill building surrounding M&E tasks. The integration of M&E into the project’s work plan is seen as an important achievement. 67. Most GEF projects in Eritrea follow M&E systems and have Project Implementation Reports (PIRs), and two of them underwent Terminal Evaluations (TEs). However, not all these report were available in the GEF PMIS system. Monitoring information was not adequately employed to make on the timely corrections of problematic issues, especially those related to outcome sustainability. A case in point is the Coastal Management and Island Biodiversity Project (GEF ID 411), where supervision (or lack thereof) allowed the over-ambitious project to carry on until close to original project completion date before its parameters were redefined. For the Wind Energy Applications Project (GEF ID 1136) supervision reports simply did not record the inappropriate procurement of technical services and equipment and supplies, not only causing delays but affecting the sustainability of the off-grid wind energy component of the project. 68. The executing institutions for GEF supported projects in Eritrea have made progress by way of establishing mechanisms for M&E. As referred to in Conclusion 9, unfortunately these institutions infrequently come together internally and discuss procedural as well as operational matters related to GEF projects. Within the context of the above, more remains to be desired in terms of putting things into practice as the record of results are not adequately shared and reported on regular basis. Interviews with concerned government officials reveal that a lack of sufficient funds, shortage in transport facilities and limited human capacity are considered as formidable constraints to put effective M&E systems into practice. 69. M&E on the relatively young GEF portfolio in Sierra Leone has played a limited role thus far. Only the Sustainable Land Management project (GEF ID 3510) has been evaluated, all the other projects completed thus far being Enabling Activities, which do not require a terminal evaluation. Otherwise, to date things seem to be in good order as far as M&E systems design, budgeting and implementation is concerned. Agencies manage their projects on the basis of monitoring data, most of which concerns progress against input and output targets, with some consideration of progress towards outcomes. In general, M&E designs for projects are satisfactory; project documents outline a set of objectively verifiable indicators for all its expected outcomes; and baseline information on the status of the indicators at project inception. M&E systems include participatory elements, ensuring that local communities (including project beneficiaries) and partners are involved in the process. In biodiversity projects the GEF Tracking Tool for Biodiversity is being used to measure the achievement of the project objectives. 70. Adequate budgetary provision has been made for M&E in all GEF projects in Sierra Leone, and allocations are within the established guidelines for GEF 4 and 5. The quality of M&E system implementation so far satisfactory. Project coordinators are carrying out their responsibilities for the day-to-day monitoring of implementation progress based on the logframe indicators and project annual work plans and milestones. All GEF Agencies undertake periodic monitoring of implementation progress through quarterly meetings with the project management teams and external supervision missions. Mid-term reviews are undertaken to systematically determine any mid-course corrections needed.
25
Conclusion 9: Mechanisms for coordination and synergies among GEF Agencies, national institutions, and among GEF and other donor-supported projects and activities have been set up, with mixed results. 71. In Tanzania, networks have been developed between GEF projects, national institutions and other donor-supported projects, for example, GEF National Steering Committee members are selected from across the public sector, CSOs, the private sector, academia and other international partners. This gives the potential for good communication and synergy across related activities in the environmental sector, for example, the coordinated successful lobbying of the Government to provide fiscal incentives for solar photovoltaic technology. The meetings, however, of this National Steering Committee are not regular and opportunities for more synergy are lapsed. 72. While mechanisms for networking among GEF Agencies, national institutions, GEF projects and other donor-supported projects and activities exist in Eritrea, they have not been fully effective for better synergies in GEF project programming and implementation. The Eritrean OFP has on several occasions chaired a Steering Committee to guide discussions on portfolio formulation, SGP initiatives, among others, and all GEF supported projects had national steering committees that were formed to guide the project management units as well as to set priorities for project activities. However, the potential for increased synergy and collaborative efforts among the agencies and national institutions involved in programming and implementation could be further realized. Roles and areas of cooperation between the Government and the UNDP, the most predominant Agency in the portfolio, are clearly specified for interactions even beyond the GEF portfolio. In practice, the mechanisms are functioning adequately.
73. However, Eritrean national institutions could be better informed of one another’s related activities and there are few forums to discuss the challenges of sustainable livelihoods, land degradation and biodiversity loss amongst all interested parties. For example, the coordination to steer the implementation of the country’s two SLM projects (GEF ID 3362 and GEF ID 2009) appears to have weaknesses at the national level. More synergies were visible at the sub-national level (i.e. between the line ministries and local administrations), but it appears to be less transparent when it comes to coordination between the various national executing agencies and the OFP. There is a tendency for institutions to move forward with their own agendas with no definite schedule for meetings and contacts between themselves. Improvements were noted with coordination moving steadily in the right direction, with greater awareness and willingness among all those concerned, although cases of ambiguity still remain. Efforts have been underway to achieve a more synergistic approach across the various Eritrean departments involved in delivery of GEF projects. Less evidence is available to assess meaningful synergistic approaches amongst Eritrea national institutions for GEF supported activities. 74. Partnership, collaboration and synergies have been good in Sierra Leone. Most GEF projects have required cross-ministerial collaboration and coordination as climate change, land degradation, and biodiversity are seen as cross-cutting issues. It was common for projects to obtain support across ministries and agencies through a broad participatory process using cross sectoral steering committees and working groups. Projects were implemented by a management team which maintained strong linkages with all relevant stakeholders through committees and workshops, and projects generally utilized complementarities with all necessary actors - University and Ministries, Departments and
26
agencies, and there was some interaction with other donor projects in the same focal area, particularly in biodiversity. However, there were little or no formal linkages with civil society organizations or private sector organizations.
27
RECOMMENDATIONS 75. The findings and conclusions emerging from the CPEs and CPS conducted in the Sub-Saharan Africa region provide the following recommendations: Recommendation 1: The GEF should explore and pursue, where appropriate, the use of established SGP country programmes as service providers to implement community-level activities for FSPs and MSPs. 76. Previous evaluations conducted by the Office, including the OPS5 Technical Document 14 on GEF Engagement with Civil Society Organizations11, have highlighted the high profile of the SGP for its successful work in linking communities to environmental management, particularly through IGAs. This is especially the case in those LDCs where the SGP country programme is well established. 77. In Tanzania, the program has been effective, and many SGP projects have been implemented in parallel, integrated into the overall activities of, and/or synergized with MSPs and FSPs. In some cases, the SGP is formally delivering the community-based component of an FSP. The SGP is also being used in Eritrea to replicate further the activities introduced by the two land degradation FSPs to other regions in the country. A overt and more strategic integration would enable consistent use of accumulated SGP expertise and experience to effectively deliver community level activities for GEF projects while at the same time optimizing the use of GEF resources (cost saving due to the pre-existence of SGP institutional structures, staffs and work procedures in the country). 78. From a global perspective, the SGP represents a highly successful GEF modality, but it is also perceived as having diverted resources away from the development of stand-alone GEF projects, and being often isolated from the overall GEF portfolio of projects in a country. A better integration of well-established SGP country programmes with the overall GEF country portfolio, possibly through an formal mandate to deliver community-level components of those FSP and MSP projects having activities requiring the active participation of local communities, would increase the likelihood of sustainability while at the same time generating cost savings to the GEF as a whole. It is therefore recommended that the SGP – in Tanzania as well as in other GEF member countries, where appropriate – be structured so that it can more formally receive funding from the budgets of larger GEF projects to deliver the community-based activities focusing on sustainable livelihoods. Recommendation 2: The GEF should pay greater attention to national knowledge exchange and promote dissemination of data and information in the relevant national languages. 79. Last year’s ACPER concluded that effective communication and outreach as well as uptake of lessons facilitated broader adoption and that lessons from past interventions are being mainstreamed in the formulation of most recent GEF projects. Dissemination of project produced lessons, information and data are a factor contributing to broader adoption and facilitating progress towards environmental impact.
11 Available on the Office’s web site at: http://www.thegef.org/gef/sites/thegef.org/files/EO/TD14_CSO%20Engagement.pdf
28
80. Building on additional evidence brought forward from the Tanzania CPE, this ACPER calls attention to an issue that often hinders national use of the knowledge produced by GEF projects, the fact that in most cases GEF documentation is in English. Language barriers could be overcome by requiring that project design includes adequate budget allocations for translation of relevant knowledge products and other project documentation into the local language.
29
Annex 1 Main conclusions and recommendations to the GEF Council from the Country Portfolio Evaluations and Studies conducted in the Sub-
Saharan Africa region and included in the Annual Country Portfolio Evaluation Report 201412
Status
Conclusions Recommendations Lessons Results Relevance Efficiency
Com
plet
ed
Tanzania
Overall, GEF-supported projects and Enabling Activities have delivered results in a satisfactory manner.
GEF support has been and remains relevant to the national Sustainable Development and Environmental Management priorities.
Tanzania projects consistently have a greater implementation over-run with few exceptions. National stakeholders perceive the GEF project processes to be time-consuming and complex.
GEF Projects in Tanzania have delivered results in several GEF Focal Areas, both at field level and in the “enabling environment;” including through enhanced institutional capacity at national and local levels. In several instances, projects have made progress to long term impacts.
GEF support in Tanzania has been relevant to a variety of objectives linked to Global Environmental Benefits related to biodiversity, climate change, international waters, land degradation and chemicals focal areas.
There has been no major coordination between national budget procedures and the preparation and funding of GEF project proposals.
The GEF has supported many measures intended to facilitate sustainability of environmental results. This has often been achieved, although not necessarily at the same level as during project funding.
The Government of Tanzania Mainland and the Revolutionary Government of Zanzibar have been supportive of the GEF activities across focal areas, as demonstrated by the use of Government’s own and donor funds to continue environmental protection initiated with GEF support.
12 The Tanzania CPE draft report has been circulated for comments in February 2014. Its conclusions and recommendations are considered as final and included in this table as such. The Eritrea CPE and the Sierra Leone CPS conclusions have been validated at the final stakeholder workshops and therefore can be considered as final. Areas of recommendations and/or lessons were also identified during those workshops. These are included where relevant in this table as such, i.e. not yet in their final form.
30
Com
plet
ed
The GEF has been partially effective in disseminating results and lessons learned from projects: but has over-concentrated on English language documentation and web-based means, neither of which is widely accessible.
Lessons and knowledge from GEF projects should be enhanced amongst GEF and other sustainable development stakeholders, and in particular vis-à-vis translation of knowledge management materials into Swahili, the local language, to ensure the broadest distribution.
Several GEF-supported activities have contributed towards Global Environment Benefits by fostering sustainable livelihood and development approaches.
The GEF SGP Steering Committee should pursue, where appropriate, the systemic use of national SGP programmes to function as service providers for the delivery of community level activities for FSPs and MSPs.
GEF activities have contributed to facilitating broader adoption through a combination of mainstreaming, replication, scaling up and market change.
Mechanisms for more networking among GEF Agencies, national institutions, GEF projects and other donor-supported projects and activities exist; however, they have not been fully effective for better synergies in GEF project programming and implementation.
The GEF Operational Focal Point Office should pursue ways of ensuring greater coordination and synergies among all parties engaged in the GEF portfolio, including synergies for M&E of the portfolio.
There have been weaknesses in the M&E systems of a number of GEF projects, whilst others are reported to have been satisfactory.
The GEF should encourage efforts to engage the GEF Operational Focal Point into M&E related activities.
31
Ong
oing
Sierra Leone
GEF support to Sierra Leone has successfully followed the catalytic path as envisaged in the GEF instruments, from foundation to demonstration to investment in full size projects identified through the foundation activities.
GEF support to Sierra Leone has been relevant to its strategic development plan and priorities, as well as to the country’s efforts to fulfil its obligations under the international agreements to which it is signatory, and contribute to the achievement of Global Environmental Benefits.
All GEF Agencies active in Sierra Leone have experienced problems in keeping projects within their intended time limits.
GEF support in the focal areas of biodiversity, climate change and international waters have helped Sierra Leone to raise the profile of environmental issues, establish national priorities and make a start in addressing critical biodiversity conservation issues that are of global significance and climate change adaptation measure of national importance.
The GEF portfolio has been executed within GEF guidelines as far as distribution of costs is concerned and has successfully leveraged significant co-financing.
GEF support in some focal areas, especially Land Degradation and POPs, has had limited results, and have not succeeded in establishing the intended foundations that would enable the country address critical issues in the two focal areas.
Partnership, collaboration and synergies have been good in the GEF portfolio. However, there are challenges in developing formal linkages with civil society organizations, local government and the private sector.
GEF should ensure that the projects it supports do not have overambitious designs in terms of expected outputs and outcomes, given the size and duration of its interventions and amount of co-financing secured.
32
O
ngoi
ng
Eritrea Overall, GEF supported projects have been effective in producing satisfactory results at the project and national levels.
GEF supported projects were relevant to the Eritrea sustainable national development needs, environmental priorities and national focal area strategies and action plans.
GEF projects have taken much longer to get started and have a greater implementation over-run.
In countries such as Eritrea, with flexible allocations, the GEF should encourage integrated approaches to tackling global environmental issues.
GEF Projects have enhanced institutional and individual capacity at national and local levels.
The Government of Eritrea exhibits a high level of country ownership and commitment to GEF supported projects.
The OFP should work with the Government of Eritrea to promulgate any environmental laws and policies still in draft since they are key enforcement tools.
Several GEF-supported activities have contributed towards environmental benefits by fostering sustainable livelihood and community-based approaches.
The GEF should continue to support the SGP in Eritrea and explore the systemic use of national SGP programmes to function as service providers for the delivery of community level activities for FSPs and MSPs.
GEF completed projects have inadequately addressed sustainability strategies post-completion.
There have been weaknesses in the M&E systems of a number of projects and thus M&E is not significantly contributing to the efficiency and effectiveness of GEF support in Eritrea.
The GEF should encourage efforts to build the GEF Operational Focal Point and national executing agencies capacity in M&E related activities.
Synergies and coordination in programming and implementation among GEF Agencies and Eritrean institutions, as well as among Eritrean institutions themselves are limited.
The OFP should re-instigate regular meetings of the national coordinating committee and undertake a National Portfolio Formulation Exercise for GEF6.