ASIAN DEVELOPMENT BANK PCR:PHI 21040

PROJECT COMPLETION REPORT

ON THE

NONFORMAL EDUCATION PROJECT (Loan 1254-PHI[SF])

IN THE

PHILIPPINES

September 2003

CURRENCY EQUIVALENTS

Currency Unit – peso (P)

At Appraisal At Project Completion 01 August 1993 28 December 2001

P1.00 = $0.036 $0.019 $1.00 = P27.716 P51.480

ABBREVIATIONS A&E – accreditation and equivalency ADB – Asian Development Bank ALS – alternative learning system BMES – benefit monitoring and evaluation system BNFE – Bureau of Nonformal Education CEP – continuing education program CBP – capacity-building program DECS – Department of Education, Culture, and Sports DepEd – Department of Education EA – executing agency FELP – functional education and literacy program LGU – local government unit LSCS – literacy service contracting scheme LSDS – learning support delivery system M&E – monitoring and evaluation MIS – management information system MTR – midterm review NFE – nonformal education NGO – nongovernment organization OSY – out-of-school youth PCR – project completion review PMO – project management office RRP – Report and Recommendation of the President SP – service provider TA – technical assistance

NOTE

In this report, "$" refers to US dollars

CONTENTS

Page

BASIC DATA iii MAP ix I. PROJECT DESCRIPTION 1 II. EVALUATION OF DESIGN AND IMPLEMENTATION 2

A. Relevance of Design and Formulation 2 B. Project Outputs 3 C. Project Costs 7 D. Disbursements 7 E. Project Schedule 7 F. Implementation Arrangements 8 G. Conditions and Covenants 8 H. Related Technical Assistance 9 I. Consultant Recruitment and Procurement 9 J. Performance of Consultants, Contractors, and Suppliers 9 K. Performance of the Borrower and the Executing Agency 10 L. Performance of ADB 10

III. EVALUATION OF PERFORMANCE 10 A. Relevance 10 B. Efficacy in Achievement of Purpose 11 C. Efficiency in Achievement of Outputs and Purpose 12 D. Assessment of Sustainability 12 E. Environmental, Sociocultural, and Other Impacts 13

IV. OVERALL ASSESSMENT AND RECOMMENDATIONS 13 A. Overall Assessment 13 B. Lessons Learned 14 C. Recommendations 14

Appendixes 1. Project Framework 16 2. Summary of Planned and Actual Project Outputs 20 3. Profile of FELP LSCS Service Providers 21 4. Current and Cumulative Enrollment of FELP 22 5. FELP Profile of Learners 23 6. FELP Research Studies/Activities 24 7. CEP Profile of Service Providers for Learning Support Delivery System 25 8. CEP Profile of Learners 26 9. CEP Profile of Examinees 27 10. Planned and Actual Consulting Services 28 11. Planned and Actual Staff Development 29 12. Cost Breakdown by Project Component 30 13. Loan Disbursements 31 14. Project Implementation Schedule 32 15. Project Organization Structure 38 16. Status of Compliance with Loan Covenants 39 17. Technical Assistance Completion Report 45 18. Assessment of Overall Project Performance 47

iii

BASIC DATA

A. Loan Identification 1. Country 2. Loan Number 3. Project Title 4. Borrower 5. Executing Agency 6. Amount of Loan 7. Project Completion Report Number

The Philippines 1254-PHI(SF) Nonformal Education Project The Republic of the Philippines Department of Education, Culture and Sports SDR18.02 million PCR:PHI 704

C. Loan Data 1. Appraisal – Date Started – Date Completed 2. Loan Negotiations – Date Started – Date Completed 3. Date of Board Approval 4. Date of Loan Agreement 5. Date of Loan Effectiveness – In Loan Agreement – Actual – Number of Extensions 6. Closing Date – In Loan Agreement – Actual – Number of Extensions 7. Terms of Loan – Interest Rate – Maturity (number of years) – Grace Period (number of years)

15 Feb 1993 15 Mar 1993 25 Aug 1993 25 Aug 1993 30 Sep 1993 5 Nov 1993 3 Feb 1994 25 Jul 1994 Four 30 Jun 1999 28 Dec 2001 Two 1% per annum 35 10

8. Disbursements

a. Dates Initial Disbursement

29 Nov 1994

Final Disbursement

28 Dec 2001

Time Interval

7 years and 1 month

Effective Date

25 Jul 1994

Original Closing Date

30 Jun 1999

Time Interval

4 years and 11 months

iv b. Amount Disbursed ($)

Last Net Category Original Allocation Amount Revised Amount Amount Undisbursed

SDR US$a Cancelledb Allocationc Availablec Disbursedc Balance Civil Works

Civil Works – Bureau of Nonformal Education 71,000 99,258 50,000 42,481 9,423 9,423 33,058 Civil Works – Department of Education, Culture, and Sports Offices

41,000 57,318 - 54,794 32,179 32,179 22,615

Furniture 40,000 55,920 25,000 27,993 13,152 13,152 14,841 Vehicles and Equipment 683,000 954,834 - 1,116,155 962,068 962,068 154,087 Learning Funds 557,000 778,686 - 11,509,330 10,710,589 10,710,589 798,741 Materials and Other Learning Resources 243,000 339,714 52,133 3,057,951 2,764,536 2,764,536 293,415 Research and Studies 51,000 71,298 183,519 1,524,722 1,433,238 1,433,238 91,484 Training 137,000 191,526 1,389,348 1,479,230 1,568,462 1,568,462 -89,232 Overseas Fellowships 458,000 640,284 - 822,479 597,221 597,221 225,258 Consulting Services 475,000 664,050 1,300,000 448,191 327,398 327,398 120,793 Local Currency Expenditures Learning Funds 6,326,000 8,843,748 - - Materials and Other Learning Resources 2,579,000 3,605,442 - - Research and Studies 1,211,000 1,692,978 - - Training 3,267,000 4,567,266 - - Social Mobilization 158,000 220,884 - 373,226 334,125 334,125 39,101 Consulting Services 945,000 1,321,110 - - Service Charge During Implementation 364,000 508,872 - 480,299 420,432 420,432 59,867 Unallocated 415,000 580,170 - 141,841 - - 141,841 Imprest Account 0 0 - - - -

Total 18,021,000 25,193,358 3,000,000 21,078,692 19,172,823 19,172,823 1,905,869 d

a As of appraisal, SDR rate = $1.398. b Actual amount cancelled may affect the total dollar equivalent becaus e of exchange rate fluctuations between SDR and US$ during project implementation. During

the first partial cancellation of the loan on 19 April 2000, SDR rate = US$1.341 was applied, while the SDR rate during the final cancellation on 28 December 2002 was US$1.269.

c For actual loan allocation, amount cancelled, amount available, and amount disbursed, foreign exchange and local currency costs were combined for learning funds, materials and other learning resources, research and studies, training, and cons ulting services.

d Cancelled at loan closing.

v

9. Local Costs (Financed) - Amount ($ million) 15.2 - Percent of Local Costs 76.8 - Percent of Total Cost 61.3

C. Project Data

1. Project Cost ($'000) Appraisal Estimate Actual

Cost Amount % Amount %

Foreign Exchange Cost 4,944 15.7 4,996 20.1 Local Currency Cost 26,557 84.3 19,800 79.9 Total 31,501 100.0 24,796 100.0

2. Financing Plan ($'000)

Appraisal Estimate Actual Cost Foreign Local Total % Foreign Local Total %

Borrower-Financed 6,301 6,301 20.0 4,060 4,060 16.4

ADB-Financed 4,944 20,256 25,200 80.0 3,960 15,212 19,172 77.3

Government of Norway

1,036 528 1,564 6.3

Total 4,944a 26,557 31,501 100.0 4,996 19,800 24,796 100.0 % 15.7 84.3 100.0 20.1 79.9 100.0

ADB = Asian Development Bank. a Inclusive of taxes and duties, physical and price contingencies, and service charge equivalent to $8,890.

3. Cost Breakdown by Project Component ($'000)

Appraisal Estimate Actuala Component/Category Foreign Local Total Foreign Local Total

A. Base Costs Civil Works 156 522 678 42 204 246 Furniture 56 88 144 13 1 14 Vehicles and Equipment 955 107 1,062 962 112 1,074 Learning Funds 779 7,018 7,797 1,200 9,511 10,711 Materials and Other Learning

Resources 318 2,861 3,179 343 2,422 2,765

Research and Studies 71 1,344 1,415 109 1,324 1,433 Training 191 3,624 3,815 93 1,475 1,568 Overseas Fellowships 640 408 1,048 597 597 Consulting Services 664 1,048 1,712 1,204 687 1,891 Social Mobilization Workshop 23 382 405 13 321 334 Recurrrent Costs 2,591 2,591 3,595 3,595 Taxes and Duties 1,356 1,356 148 148

Subtotal (A) 3,853 21,349 25,202 4,576 19,800 24,376 Contingencies 582 5,207 5,790 Service Charge during Implementation 509 509 420 420

Total Project Cost 4,944 26,557 31,501 4,996 19,800 24,796 a This includes $1.56 million (actual utilization) from the Norway Grant under TA 2036-PHI: Capacity Building in

Nonformal Education for $1.6 million approved on 23 December 1993, after loan approval. The advisory technical assistance financed the capacity-building program component of the Project.

vi

4. Project Schedule

Item Appraisal Estimate Actual

Date of Contract with Consultants

International I Qtr 1994 III Qtr 1994 Domestic I Qtr 1994 III Qtr 1994 Implementation of Functional Education and

IV Qtr 1993 I Qtr 1993 Literacy Program Implementation of Continuing Education Program IV Qtr 1993 III Qtr 1996 Civil Works Contract Upgrading of BNFE /PMO Office

Date of Award II Qtr 1995 I Qtr 1995 Completion of Work IV Qtr 1996 I Qtr 1999 Upgrading of DECS Divisional Offices Date of Award !V Qtr 1993 III Qtr 1995 Completion of Work I Qtr 1998 I Qtr 2000 Equipment, Furniture, and Vehicle Dates National First Procurement I Qtr 1994 I Qtr 1994 Last Procurement IV Qtr 1994 III Qtr 2001 Regional/Provincial First Procurement I Qtr 1994 I Qtr 1994 Last Procurement II Qtr 1998 II Qtr 1999 Staff Development Overseas: Long-Term I Qtr 1994 III Qtr 1995 Overseas: Short-Term I Qtr 1994 III Qtr 1996 Domestic: Long-Term I Qtr 1994 III Qtr 1996 Domestic: Short-Term I Qtr 1994 I Qtr 1995

Other Milestones Partial Cancellation First – $3.0 million 19 Apr 2000 Second – $1.9 million 28 Dec 2001 Loan Closing Date Extension First 31 Dec 2000 Second 30 Jun 2001

BNFE = Bureau of Nonformal Education, DECS = Department of Education, Culture, and Sports, PMO = project management office.

5. Project Performance Report Ratings

Ratings Implementation Period Development

Objectives Implementation

Progress Project Administration Committee (PAC) Notesa From 25 Jul 1994 to 31 Dec 1994 Na AAA

From 01 Jan 1995 to 30 Jun 1995 Na AAA From 01 Jul 1995 to 31 Dec 1995 Na AAA From 01 Jan 1996 to 30 Jun 1996 Na AAA

From 01 Jul 1996 to 31 Dec 1996 Na AAA From 01 Jan 1997 to 30 Jun 1997 Na AAA From 01 Jul 1997 to 31 Dec 1997 Na BAB

na = not applicable.

vii

Ratings Implementation Period Development

Objectives Implementation

Progress Project Performance Report (PPR)

From 01 Jan 1998 to 30 Jun 1998 Satisfactory Satisfactory From 01 Jul 1998 to 31 Dec 1998 Satisfactory Satisfactory From 01 Jan 1999 to 30 Jun 1999 Satisfactory Satisfactory From 01 Jul 1999 to 31 Dec 1999 Satisfactory Satisfactory From 01 Jan 2000 to 30 Jun 2000 Satisfactory Satisfactory From 01 Jul 2000 to 31 Dec 2000 Satisfactory Satisfactory From 01 Jan 2001 to 30 Jun 2001 Satisfactory Satisfactory From 01 Jul 2001 to 28 Dec 2001 Satisfactory Satisfactory a Rating used in PAC Notes: first letter = rating for implementation delay, second letter = rating for cost

overrun, third letter = rating for compliance with covenants (A = Satisfactory, B = Partly Satisfactory, C = Unsatisfactory).

D. Data on Asian Development Bank Missions

Name of Mission

Date

No. of Persons

No. of Person-

Days

Specialization of Membersa

Fact-Finding 15 May–09 Jun 1992 3 66 b, c, l Pre-Appraisalb 15 Feb–15 Mar 1993 7 151 b, c, d, e, f, l Inception - 1 22–23 Nov 1993 2 4 b, o Inception – 2 5–8 Dec 1994 7 16 a, b, f, g, h, m, n Review – 1 13–23 Jun 1995 2 12 a, b Review – 2 6–14 Dec 1995 2 6 a, b Review – 3 9–16 Jul 1996 2 16 b, p Review – 4 4–9 Nov 1996;

19–23 Nov 1996; 29 Nov 1996

2 8 b, f

Midterm Review 1–26 May 1997 3 16 b, p Review – 5 3–6 Mar 1998 3 12 i, l Review – 6 5–6 Mar 1998 1 2 f Review – 7 18–22 May 1998 2 10 b, i Special Loan Administration – 1 8–11 Jan 2000 1 4 d Special Loan Administration – 2 6–8 Sep 2000 1 3 d Special Loan Administration – 3 15–17 Sep 2000 1 3 j Special Loan Administration – 4 13 Mar–19 Apr 2002 3 16 b, d, q Project Completion Reviewc 8 Jul–2 Aug 2002 4 55 b, k, l, p Total 46 400 a a = senior education specialist, b = education specialist, c = programs officer, d = project economist, e =

counsel, f = control officer, g = procurement specialist, h = consulting services specialist, i = senior project implementation specialist, j = senior project specialist, k = senior financial management specialist, l = consultant, m = technical assistant–loan disbursement, n = senior clerk (project administration), o = loan administration clerk, p = assistant project analyst, q = senior general services staff.

b The Mission was upgraded from Pre-Appraisal to Appraisal as per President’s approval of the back-to-office report of the Pre-Appraisal Mission dated 8 June 1993.

c The Mission comprised K. Chowdhury, Education Specialist/Mission Leader; E. Yang, Sr. Financial Management Specialist; G. Mak, Staff Consultant; and Ma. T. Valenzuela, Assistant Project Analyst.

I. PROJECT DESCRIPTION 1. The Philippines has achieved a high literacy rate and universal elementary education ahead of many developing countries in Asia. However, those achievements have been eroded in the past decades to the extent that today, a significant proportion of the workforce of the Philippines is functionally illiterate.1 It was estimated that about 27% of the population over 10 years of age were functionally illiterate and about 2.9 million out-of-school youth (OSY) in the 7-17 years age bracket. The level of functional illiteracy and OSY vary sharply across regions (para. 2). Cohort survival rates in elementary schools were around 69% in 1992. Since there is a link between illiteracy and poverty, there is strong social and economic justification for investing in nonformal education (NFE) to enable the poor to improve their welfare and economic productivity. NFE offers a flexible, cost-effective and viable strategy for poor OSY to improve their educational chances and entry into the labor market. 2. This Project sought to contribute to poverty reduction in the Philippines by (i) raising the literacy and numeracy skills of the uneducated and enhancing their capacity to engage in self-help activities; (ii) expanding access to basic education by supporting the NFE system so as to reach OSY and adults; and (iii) building the capacity of the Department of Education, Culture, and Sports (DECS),2 nongovernment organizations (NGO) and communities in planning, managing, and conducting NFE. 3. The Project targeted rural areas in 24 provinces in nine regions. Provinces were selected on the basis of composite ratings of highest illiteracy rates and lowest elementary school enrollment rates, and high incidences of poverty. The illiteracy rates of these provinces ranged from 30% to 63% in 1989, and their elementary school enrollment rates were 56% to 97% in 1990−1991.3 4. To achieve its objectives, the Project adopted an innovative implementation approach, with emphasis on stakeholders’ participation, decentralization of activities, and interagency collaboration among government and nongovernment stakeholders.4 The Project comprised three interrelated components: (i) the functional education and literacy program (FELP), (ii) the continuing education program (CEP), and (iii) the capacity-building program (CBP). The FELP was to support community-based literacy programs for illiterates and semi-illiterates while the CEP was to establish an alternative learning system (ALS), with an assessment scheme – called accreditation and equivalency (A&E) – testing the competencies of OSY and other candidates to enter the learning programs. The CBP would strengthen the capacity of DECS to implement NFE. NFE was delivered in community/village centers or in service providers’ (SPs’) homes. 5. Integral to the Project was a capacity strengthening program developed under a technical assistance (TA) grant. The TA objectives were to strengthen the institutional and managerial capabilities of the Bureau of Nonformal Education (BNFE) and the DECS/NFE units

1 “Functionally illiterate,” as defined by the US Census Bureau refers to people with less than 6 years’ schooling. 2 The current Department of Education (DepEd) was known as the Department of Education, Culture, and Sports

(DECS) during the project implementation period. The change to “Department of Education” was made effective on 10 August 2001 following a new law known as the Governance of Basic Education Act of 2001.

3 ADB. 1993. Report and Recommendation of the President to the Board of Directors on a Proposed Loan to the Philippines for Nonformal Education. Manila.

4 Interagency collaboration involved all stakeholders, e.g., local communities, nongovernment organizations, local government units, and regional and central DepEd offices.

2

at the regional and provincial (divisional) levels so as to enable them to implement FELP and CEP.

II. EVALUATION OF DESIGN AND IMPLEMENTATION

A. Relevance of Design and Formulation 6. The Project was consistent with the Government’s efforts to reduce poverty through expanded provision of education. In 1989, the report on A Framework for a Philippine Plan of Action on Education for All 1990-2000 was the Government’s response to the international Education for All movement. It advocated a reduction of illiteracy rates and of the gender and regional disparities in literacy by the year 2000. In 1992, the BNFE outlined a plan to provide NFE. The plan’s objectives were to (i) promote functional education for deprived adults, (ii) develop a curriculum framework and an equivalency testing system for the ALS, (iii) shift BNFE’s role from direct program implementation to that of coordination and supervision, and (iv) forge partnerships with NGOs and local government units (LGUs) to integrate NFE in community development. This approach was expected to enhance the relevance and sustainability of NFE activities. The Project design was therefore relevant to the Government’s plans and objectives. 7. During the processing of the Project, Asian Development Bank (ADB) held extensive consultations with the Government and stakeholders at the community level. The Project was part of ADB’s strategy as outlined in the Medium-Term Strategic Framework (1993−1996), which endorsed the implementation of NFE programs as an effective means to reduce illiteracy and poverty. 8. The Project targeted support for two major groups, illiterates/semi-illiterates and school dropouts aged 15 and above, and building the capacity of those involved in providing NFE. Decentralized implementation and interagency collaboration was expected to develop ownership, pool resources and expertise at all levels, and subsequently build the capacity of all stakeholders. Appendix 1 gives the project framework as laid down in the Report and Recommendation of the President. 9. The design was formulated in close collaboration with BNFE. However, due to leadership problems in BNFE implementation progress remained slow until the midterm review (MTR) date. Subsequent strong leadership by BNFE turned the Project around though it was too late to complete all envisaged activities within the project life. On an overall basis, the project design turned out to be too ambitious to be fully implemented in five years.5 Significant adjustments in the components during implementation required adjustment in the output targets of some subcomponents. 10. Implementation delays were also due to a number of other factors. First, the development of a NFE system proved to be highly complex in terms of learner needs assessment, policy and curriculum conceptualization, and learning materials development, delivery, assessment, and accreditation. Second, LGUs and other agencies needed more lead time to develop effective management capacity. Third, while BNFE had highly committed staff, they were inexperienced to deal with a large externally assisted project and had difficulties in keeping project implementation on schedule. Finally, delays due to slow moving Government

5 The weaknesses of the project design are discussed more elaborately in paras. 13-24.

3

processes and systems exacerbated the problems. The project design did not anticipate these difficulties/risks encountered during implementation. 11. The Project was designed 10 years ago when NFE was a new area for ADB. Since then, ADB’s approach to investment in NFE has evolved and based on experience gained it is recognized that NFE is a long-term development agenda and accomplishing and completing a project within five years is not possible as building capacities and ownership takes considerable lead time. B. Project Outputs 12. The physical targets to improve literacy and numeracy skills of illiterates and expand access to elementary and secondary education of OSY and adults had exceeded appraised targets, although the development of learning materials were not completed in time for Project use. Capacity building objectives were largely achieved but the development of systems for monitoring, evaluation and information were only partially completed. Appendix 2 summarizes the planned and actual quantitative outputs of the Project.

1. Improved Literacy and Numeracy Skills

13. Under FELP, the Project supported establishment of the Literacy Service Contracting Scheme (LSCS). Through this scheme, SPs were subcontracted to implement basic literacy programs. Where learners started with minimal or no previous education FELP focused on basic literacy. The Project supported the development, production, and distribution of prototype curricula and training guides, as well as social mobilization activities and in-depth research studies. 14. The Project accomplished the following: (i) the literacy programs served 465,913 learners or 104% of the planned target (Appendix 3 gives a profile of the FELP SPs, Appendix 4 presents the FELP enrollment, and Appendix 5 gives a profile of FELP learners); (ii) the original plan was to cover 120 contact hours for learners spanning 4−6 months. At the MTR, this number was found insufficient and was therefore increased to 200 hours between November 1997 and May 1998. In 1998 the learning period was reduced to 150 hours due to lack of funds; (iii) in addition, the FELP manual of operations and prototype literacy modules and an accompanying facilitator’s guide were developed; (iv) a number of research studies were completed; (v) NGOs, eventually grouped under larger umbrella organizations, had been included in the LSCS; (vi) in line with the recommendations of MTR, advocacy and social mobilization activities were conducted to generate interest in the LSCS; and (vii) procurement and implementation guidelines were prepared (Appendix 6). 15. The most significant achievement of the Project was the establishment of a new working model for decentralized NFE delivery, with SPs comprising mainly NGOs and academic institutions as implementers. This changed the role of the Government from directly providing NFE to setting the overall strategic framework, designing a curriculum and developing learning materials, providing overall guidance and quality assurance, and monitoring and evaluation. 16. The Project experienced considerable implementation delays in the first three years. The MTR in 1997 highlighted three problems. First, the number of learners being reached under the LSCS was far below the anticipated targets. Second, the absence of a FELP framework and an operations manual led to varying interpretations of and deviation from the original design. Third, the participation of NGOs was weak due to inadequate consultation with the local communities

4

and the narrowly interpreted selection criteria that restricted the pool of eligible NGOs. The MTR reached agreement with DECS to reduce the geographic reach and use available resources for developing institutional capabilities more effectively. As a consequence, the Project’s 24 provinces were grouped into two categories, A and B, each comprising 12 provinces. Group A, where LSCS-type activities had already been underway, underwent reorientation and improvement over 1997-1998 to achieve full implementation in 1998-1999. For group B, where no implementation had taken place yet, activities were limited to civil works and procurement, social preparation activities, and literacy mapping. 17. The project scope was expanded to include the National Capital Region (NCR), as well as Cebu City, Lapu-Lapu City, and Mandaue City—all in Region VII—for pilot implementation of the NFE A&E system. Three municipalities were added in Aklan Province and three were added in Maguindanao Province at the request of the latter’s Governor. The MTR and DECS also agreed (i) to develop a literacy-cum-livelihood framework and the FELP operations manual to ensure common understanding among all parties concerned, (ii) to increase consultation with partners to enhance interagency collaboration, (iii) to invite the participation of NGO umbrella organizations6 to speed up implementation, and (iv) to streamline contracting and payment procedures. The Operational Review of 1998 reinforced the MTR recommendations. It also highlighted the importance of placing the literacy programs in the context of the Philippines’ education system, and of using national literacy standards as benchmarks for implementing the literacy programs. 18. During the first three years, delays in the bidding processes slowed literacy materials development. In 1998−1999, the Project developed 29 prototype learning modules, each with an accompanying facilitator’s guide, one general guide, and one workbook for Basic Literacy in Filipino. While the materials were being finalized, BNFE issued a set of draft-version master copies to a limited number of umbrella organizations for trial use. The Project Completion Review (PCR) Mission found that some SPs had to obtain literacy materials from other sources, resulting in differing quality standards of various programs. 19. The Project generally achieved FELP targets. However, the literacy materials, the FELP manual of operations, and national literacy standards were completed after closure of the loan Project. Research studies that had been completed were of limited use because they were too academic and not well coordinated among the different universities that did the research. Literacy mapping, intended to identify the specific needs of target communities produced only summary data on provinces, which were of little use. 20. Had the mentioned weaknesses been addressed during the design phase, the Project may have been delayed further due to the added complexity. A key weakness of the design was the lack of a strategy for sustaining the learning gains though objectives of such a strategy may have been out of reach of the Government’s budget.

2. Expanded Access to Elementary and Secondary Education 21. Under CEP, the Project supported an ALS through a curriculum framework based on essential learning competencies and improved learning materials, assessment and certification. The target groups were primary school leavers (Grade 4), intermediate school leavers, high school dropouts, and FELP graduates. About 130,000 learners were expected to directly benefit from the programs (45,000 from the self-learning program and 85,000 from the radio program),

6 Four large NGO umbrella organizations coordinated the majority of SPs.

5

and 170,000 to directly benefit from the ALS equivalency testing allowing them to achieve a level of schooling equivalent to the formal school system. CEP activities were envisaged to be conducted first in areas covered by FELP to provide FELP graduates with continuing learning opportunities. 22. At completion, the Project had developed and implemented an ALS, better known as the NFE A&E system. The Project (i) supported 71,505 learners and 52,240 examinees in the context of CEP;7 (ii) developed key elements of a NFE A&E system, including a policy framework that relates to the overall education system, a curriculum framework, learning materials, a learning support delivery system (LSDS),8 and an assessment and certification system covering elementary and secondary levels; (iii) subcontracted local groups, umbrella organizations, and DECS-funded implementers to implement LSDS (Appendix 7 provides a profile of SPs for the LSDS; and Appendixes 8 and 9, of learners and examinees of CEP); (iv) conducted advocacy and social mobilization activities to increase public understanding of the NFE A&E system; and (v) completed a number of research studies. The modules developed by the Project are of high quality and are being used after the life of the Project. The NFE A&E has been highly attractive to OSY as an alternative route to educational attainment. While the key elements of the ALS were accomplished late and had limited use during the project period, they are being used in other education programs. 23. The initially slow implementation of CEP was due to the absence of a clear conceptual design and delivery system, limited technical capacity of the contractors and BNFE, as well as insufficient support from the consultants. The MTR identified three problems. First, curriculum development was delayed, and the final output (96 modules) was of poor quality. Second, the assumption that learners had the ability for “self-learning” was over optimistic. There was a need to return to a more guided way of learning. Third, CEP’s mandate in the context of basic education needed adequate clarification. The MTR agreed on the following remedial actions: BNFE would review the CEP mandate and conduct demand surveys of OSY and adults; and instead of covering FELP areas, the scope would be limited to the development of a policy and curriculum framework (including learning materials and equivalency tests) for pilot testing in two urban and two rural sites. 24. Accordingly, the delineation of FELP and CEP was revised. Originally, FELP was to cover both basic and functional literacy while CEP was planned to provide education equivalent to secondary schooling. A 1998 Review Mission reconfigured the scope of FELP and CEP. As a result, FELP was to focus on basic literacy; and CEP was expanded to include both elementary and secondary levels, and was reconceptualized as the NFE A&E system to emphasize its equivalency nature. The revised target numbers for CEP were set at 20,000 learners/examinees for the elementary level and 10,000 for the secondary level (Appendix 2). 25. With regard to learning materials, three universities were contracted in 1995 to develop self-instructional materials. They failed to produce satisfactory materials and their contracts were terminated in early 1997. Under a second initiative the services of the Southeast Asian Ministers of Education Organization (SEAMEO INNOTECH) were contracted from September 1998 to May 1999. The new NFE A&E curriculum framework (para. 22) was to serve as a blueprint for the preparation of these materials. Altogether 152 print-based learning modules and 10 audiotapes were developed under this initiative and have been used since the January 1999 launch of the NFE A&E system. Complete sets were distributed to each SP who was to

7 The targets set in the original project design were 130,000 learners and 170,000 examinees. 8 The LSDS contracted SPs to provide 400 hours of learning support services to groups of NFE A&E learners.

6

reproduce the modules that the learners would need. A quality review of the modules took place from April to August 2000. On the basis of the review, a Phase II NFE A&E Learning Materials Development component was conceptualized and contracted again to SEAMEO INNOTECH. Phase II started in September 2000 and ended in December 2001. It produced 535 print-based modules (including 94 self-instructional modules for the academic bridging program to serve the needs of completers who wish to pursue a college education), 47 audiotapes, and 3 videotapes. These materials were produced too late to be used during the lifetime of the loan Project. In addition, BNFE contracted a university to develop 50 audiotapes for a radio-based program. The audiotapes were completed in October 2001, after the closing of the Project. Although the production of these modules and audio-visual aids came too late to support project activities, they nevertheless represent a positive impact of the Project. BNFE now has a good inventory of up-to-date learning materials in both Filipino and English to service the NFE A&E system. The materials are being widely used after the life of the Project and in non-Project areas funded by LGUs and international aid agencies.

3. Strengthened Institutional Capacity

26. To strengthen institutional capacity, the Project provided (i) consulting services; (ii) staff development in NFE at BNFE and local government levels; and (iii) training workshops for NFE, LGU, and SP staff. BNFE took overall implementation responsibility for this component and was supported by a team of consultants. In addition, a benefit monitoring and evaluation system (BMES) was established, which included a management information system (MIS) and a monitoring and evaluation (M&E) system for BNFE. 27. Before the MTR, CBP achieved its physical targets but faced quality issues because of lack of clear training objectives for staff development. The lack of career management of participants resulted in a loss of some trained DECS personnel due to transfer to other posts. During MTR, agreements were reached with BNFE (i) to reduce the team of consultants to a smaller number of long-term consultants and to re-orient consultant activities to provide technical support at the field level; (ii) to review the requirements of institutional capacity development and staff needs, so as to prioritize the components of training; and (iii) to add baseline information and monitoring of the postprogram status of learners to the MIS. 28. By project completion, CBP generated the following results: (i) the BNFE had acquired experience in managing the Project, using the services of consultants, and delivering NFE programs at local levels; (ii) staff development activities were completed; (iii) training workshops were completed; and (iv) MIS and M&E system were developed. The design of MIS and M&E system was less than satisfactory. Both were not fully operationalized due to their overly sophisticated design, which poorly matched the needs and capacities of BNFE. The less than satisfactory results of MIS and M&E system were primarily due to the frequent turnover of MIS and M&E system consultants in the first few years, lack of cooperation between international and domestic consultants, and insufficient experience of BNFE in monitoring and supervising the work of consultants. Capacity building was therefore only partially successful. Appendices 10 and 11 show the planned and actual consulting services and staff development program. 29. Overall, BNFE’s lower priority vis-à-vis formal education equivalency programs as reflected in the low budgetary provision for NFE before, during, and after the Project affected

7

many aspects of NFE performance. The inadequate funding of NFE in the budget is a severely limiting constraint on NFE development and impacts on the sustainability of NFE initiatives. 9 C. Project Costs 30. Project cost estimated at appraisal was $31.5 million equivalent inclusive of taxes, duties, and contingencies. The total actual project cost amounted to $24.8 million equivalent. The overall decrease in project cost was primarily due to the depreciation of the Philippine peso, from about P27.72 to $1.00 at appraisal to about P51.48 to $1.00 at the close of the Project. As a result, the actual local costs of goods and services anticipated at appraisal were substantially reduced. Inadequate budgetary appropriation resulted in the postponement or cancellation of some scheduled project activities. A summary of the appraisal and actual financing plans and costs of the Project is shown in Basic Data. Details of the appraisal and actual cost by component are shown in Appendix 12. D. Disbursements 31. The first disbursement was made on 29 November 1994 and the final disbursement on 28 December 2001, covering a period of 7 years. Disbursement was very slow during the first 3 years of project implementation (9.75% by MTR in April 1997). During 1998-2001, disbursement sharply increased, spurred by the tapping of large umbrella organizations for FELP LSCS implementation, and the fasttracking of NFE A&E system development and implementation. Appendix 13 shows the disbursement performance. 32. A request for reallocation made in April 2000 was approved by ADB in November 2000. The principal reallocations involved transferring excess loan funds from the categories of training, and learning materials to learning funds to be tapped by FELP LSCS and NFE A&E LSDS. The reallocation increased the total learning funds by $2.9 million to $10.7 million. As the Project was unable to fully utilize the loan proceeds, the Government requested the partial cancellation of the loan amount by $3 million in April 2000. An unutilized balance of $1.9 million remained at the close of the Project that was subsequently cancelled, bringing the total cancellation to $4.9 million or 20.5%. 33. After the closure of the imprest account in June 1999, the disbursement of small project contracts became a serious issue. The Project had to resort to advance payments from government funds, which became a problematic procedure at a time when BNFE lacked adequate budgetary funding. The result was a serious disruption of project implementation.10 E. Project Schedule 34. ADB approved the Project on 30 September 1993. Loan effectiveness was delayed until 25 July 1994. This initial delay caused subsequent delays though the Project achieved most of its physical targets. Appendix 14 summarizes the planned and actual implementation schedule. In view of the initial delays, ADB agreed to the Government’s request to extend loan closing by 18 months (1 July 1999-31 December 2000) to complete planned activities. A second extension

9 At present, the share of NFE is a meager 0.07% of the Philippines education budget, indicating low Government

priority attached to NFE. 10 The problem can be avoided in the future. As is ADB's common practice, the replenishment of the imprest account

stops one year before the loan closing date. However, if the project officer can provide prudential analysis and projection to justify the need of an imprest account, the account can remain open and be replenished until the end of project implementation.

8

of 6 months (1 January-30 June 2001) was granted to allow enhancement of the NFE A&E testing system. Project implementation therefore took 7 years. However, to accommodate the processing of all withdrawal applications, the Project accounts were kept open until 28 December 2001. F. Implementation Arrangements 35. The implementation arrangements were generally consistent with those envisaged at appraisal. The Executing Agency (EA) was DECS and BNFE had overall responsibility for implementation. A project steering committee (PSC) was set up at the national level to oversee project implementation. A project management office (PMO), was attached to BNFE to manage the Project. It was headed by the Director of BNFE in the role of project director, staffed by contractual project staff, and supported by anchor persons drawn from BNFE staff. Under the PMO, NFE staff of DECS in the regions, provinces, and districts managed and coordinated project activities at their corresponding level. The NFE coordinators, who were released from regular duties were assigned fulltime to the Project. Coordinating committees (CCs) were set up at each level for interagency coordination and review of proposals from NFE subproject proponents. Appendix 15 shows the Project organization structure. In December 1994, the ADB Inception Mission found that the PMO was not functioning effectively as a separate unit due to inadequate staffing and poor coordination with BNFE. At the MTR, it was agreed that the PMO would be fully integrated with BNFE. 36. Though adequately formulated, the implementation arrangements were tested to the limit due to: (i) weak institutional capacity at various levels of Government to carry out project activities; (ii) frequent changes in project leadership, with four project directors and four project managers between 1993 and 1996; (iii) most of the NFE supervisors and coordinators had to perform the project responsibilities alongside their normal work in formal education and could not devote full attention to the Project; (iv) although many NFE CCs at the municipal and divisional levels were active, some did not function due to the high turnover of committee members, a lack of understanding of their roles and responsibilities in their capacity as NFE project staff, and insufficient time and funds; (v) conflict of interest at the grassroots level, whereby the same individuals who were NFE project staff submitted subproject proposals in their capacity as representatives of local groups the prevalence of such instances is not known but their existence is a lesson for setting guidelines on participation in the future; and (vi) the original intent of relying directly on numerous grassroots-level NGOs proved unrealistic as they lacked experience. NGO umbrella organizations were therefore included after the MTR. Given their experience they were more effective in program delivery. 37. In sum, lack of capacity among the institutions involved and some organizational issues hampered implementation at the initial stage. Adjusted implementation arrangements after the MTR allowed improved project implementation. G. Conditions and Covenants 38. The status of compliance with major conditions and covenants was generally satisfactory (Appendix 16). Most covenants were fulfilled. The EA complied only partially with the covenant on the hiring of contractual staff of the PMO due to budgetary constraints and the creation of contractual financial positions in regional and divisional offices. To compensate, regular BNFE staff were assigned fulltime to the Project, and regional accountants were designated to act as financial analysts in addition to their regular duties. The EA complied only partially with the

9

covenant on establishing a BMES within six months of the effective date. At project close, the BMES was only partially completed and was not institutionalized. H. Related Technical Assistance 39. The Project included a $1.6 million associated TA grant from the Government of Norway, managed by ADB. The main objective was to support BNFE’s capability to implement the Project’s components and to support DECS/NFE units at regional and divisional levels. The TA made an important contribution to establishing a decentralized delivery approach with local SPs, as well as developing a new model for the delivery of literacy programs through the LSCS. The TA was successful in producing high quality learning modules that are now being used in non-Project sites.11 The TA was also designed to assist DECS in establishing the regional, provincial, and municipal NFE CCs needed, composed of representatives from LGUs, NGOs, and Government line agencies. The TA Completion Report is attached as Appendix 17. I. Consultant Recruitment and Procurement 40. The loan envisaged provision of consulting services for capacity building in the area of computer systems (26 person-months) and finance and accounting (27 person-months) for DECS’ regional and divisional offices. The other consulting inputs were funded by the Norwegian TA as envisaged at appraisal and comprised 34 person-months of international consultants and 186 person-months of domestic consultants. The consultants were recruited following ADB’s Guidelines on the Use of Consultants. 41. The Project included a small civil works and procurement component designed to strengthen the capacity of BNFE and of DECS NFE offices. The component entailed the purchase of furniture, office equipment, and vehicles and the construction of NFE division offices and upgrading of BNFE office facilities. Goods and services were procured following ADB’s Guidelines on Procurement. The civil works contracts were all below $50,000 and were awarded through local competitive bidding. J. Performance of Consultants, Contractors, and Suppliers 42. The consulting services for capacity building under the loan proceeds was underutilized due to BNFE’s inability to implement the assistance. For computer systems training only 12 person-months were utilized and no consultant was recruited for the envisaged finance and accounting training. Similarly, the performance of the consultants provided under the Norwegian TA in the first three years was hindered by BNFE’s limited experience and absorptive capacity in dealing with a large number of consultants and complex project activities. Based on lessons learned, the consulting services’ package was reconfigured and subsequently met the needs of FELP and CEP more effectively.12 The performance of the domestic and international consultants for the MIS and M&E was unsatisfactory due to reasons cited earlier (para. 28). The consultants’ output and performance have been separately evaluated. Given the fact that the

11 A detailed assessment of the contribution to the Project of TA 2036-PHI: Capacity Building for Nonformal

Education Consultants can be found in the Final TA Report of World Education, submitted to ADB and BNFE on 18 December 2001.

12 Under the reconfigured TA, the international consultants’ person-months increased from the planned 34 to 85.98 on completion, and the domestic consultants’ person-months were reduced from the planned 186 to 130.81 on completion. The BNFE with ADB’s endorsement terminated the services of domestic consulting firms and contracted individual domestic consultants for (i) advocacy and mobilization; (ii) NFE A&E system; and (iii) the FELP manual of operation.

10

CBP achieved its major objectives, the overall performance of the consultants is rated satisfactory. 43. Procurement of goods and services proceeded smoothly. ADB’s approval of the inclusion of Lanao Sur I as a project site and the transfer of Sultan Kudarat to Region XI expanded the scope of construction and equipment procurement, and required the provision of additional equipment for DECS Regional Office XI. BNFE was able to procure additional equipment with ADB’s approval by reallocating loan funds. BNFE also generated savings from the civil works by limiting price escalation in the contracts. At ADB’s suggestion, the savings were redistributed to each of the project municipalities to fund the construction of NFE learning centers with the use of indigenous construction materials. The original plan to purchase two vehicles as mobile libraries was given up, as the vehicles would not have been able to cover all the 24 provinces. ADB and BNFE agreed to reallocate these funds to purchase 72 typewriters instead for distribution to district NFE coordinators. K. Performance of the Borrower and the Executing Agency 44. The Project was the EA’s first foreign-funded project in NFE. Effective implementation of the Project required staff continuity and managerial and technical expertise at DECS, which could not be achieved for several years. The appointment of a new director of project/BNFE in 1997 was crucial in providing the required strong leadership. The performance of DECS at the local level depended on the degree of commitment of concerned staff and their relationship with LGUs and community groups. The Government could have more effectively facilitated project implementation by streamlining administrative regulations, providing adequate counterpart funding, and by giving higher budgetary allocations to NFE. In light of the challenges of the Project and BNFE staff’s dedication that compensated for some organizational problems, the performance of the Borrower and EA was satisfactory. L. Performance of ADB 45. The Project was reviewed twice a year during the implementation period. ADB maintained a good relationship with BNFE throughout project implementation. The targets of 130,000 learners in self-learning programs and 170,000 examinees in CEP set by the Project was unrealistic and could not be achieved. The enrollment demand was not adequately estimated at project formulation. ADB provided detailed and timely advice regarding remedial action. Through appropriate adjustment of the project design, revisions of targets, reconfiguration of consulting services’ package, modification of implementation arrangements, and extensions of project duration, ADB effectively addressed all major implementation problems. Overall performance was therefore satisfactory.

III. EVALUATION OF PERFORMANCE A. Relevance 46. The Project was consistent with ADB’s operational strategy and the Government’s goal of poverty reduction through NFE expansion. It reached out to two most needy groups—illiterates/semi-illiterates and school dropouts aged 15 and above—that the formal education system had neglected. The learners of both FELP and NFE A&E programs reported the benefits of their learning in terms of (i) intrinsic rewards, e.g., raising one’s self-esteem; (ii) acquiring skills for enhanced daily life, e.g., literacy and numeracy; (iii) acquiring skills in bringing up and educating children; (iv) learning about civic duties such as voting; (v) better knowledge of

11

hygiene and medication; (vi) learning livelihood skills, e.g., handicraft making; and (vii) the high value of the A&E certificate in helping them find waged employment or reenter formal education. They expressed a strong faith in the value of education for individual advancement and thus desired more NFE. However, interviews with learners and facilitators suggested a lack of livelihood skills in the FELP. 47. Despite the original design weaknesses, efforts were made at the time of the MTR to adjust the design of the Project, revise the output targets, reconfigure consulting services inputs, modify the Project’s implementation arrangements, and extend the Project’s duration. As these steps substantially improved the relevance and implementability of the Project, and considering the Project’s relevance to the Government’s and ADB’s strategies, the Project is rated as relevant. B. Efficacy in Achievement of Purpose 48. Evidence from interviews with learners suggested that FELP reduced illiteracy and learners demanded more NFE. However, the absence of pre- and post-Project literacy rates data in the target areas makes a quantitative assessment of impact difficult. In rural communities with very few reading materials, the new FELP literates may relapse into illiteracy. The Project did not plan for community centers that could have provided the kind of continuity needed to ensure retention of literacy. Urban communities, where printed materials are available, are a more conducive environment for literacy retention. Also, the project impact on livelihood skills would depend on the availability of job opportunities to absorb these skills. Without employment opportunities, the learners’ potential for income generation is limited. NFE as a stand-alone intervention is not necessarily poverty reducing. Despite these caveats, improved nutritional practices and hygiene, and education awareness are measurable benefits of NFE. 49. CEP received highly positive feedback from learners and examinees who saw it as a vehicle to improve their livelihood. NFE A&E raised high expectations among learners in terms of getting support for attending higher levels of education. Its attraction lies in its connection with the formal education system, being an alternative route to school achievement in a society where academic achievement is highly valued. 50. Under CBP, strengthening DECS leadership in NFE by means of setting up coordinating committees at the national and local levels was accomplished. The decentralized interagency approach of the Project is a valid method to promote collaboration and internal coordination. However, the poor quality of vertical and horizontal communication in DECS and other involved agencies affected the efficacy of project implementation and project outputs. The target for improving project M&E was only partially met. Results were achieved too late to yield much benefit during the project lifetime. 51. The Project provided a solid foundation for a better NFE system in the country, e.g., a new working model for NFE delivery, the NFE A&E system, and high-quality self-instruction modules. Given the substantial delays of some critical project deliverables, some of the inputs could not be used during the life of the Project. In view of this and subsequent adjustments and scaling down of the project targets, as well as inadequate quality of research studies and BMES, the Project is rated as less efficacious.

12

C. Efficiency in Achievement of Outputs and Purpose 52. Despite delays, most of the revised quantitative and qualitative targets of the Project were achieved (para. 14). Achievements in FELP included 465,913 learners (104% of the target) in 2,046 communities (118% of the target) and 24 provinces (100% of the target). In terms of cost-effectiveness, the DepEd estimate of the unit cost of sending one student through 1 year of elementary school is P4,461 (or $91 at an exchange rate of $1 = P51). The estimated direct cost of serving a FELP learner for 150 hours over 4-5 months is P990 ($19.40), while the estimated total cost is P1,102 ($21.60). The internal validity of equivalency between 1 year of elementary education and 150 hours of FELP learning has yet to be established before cost-effectiveness can be more definitively estimated. 53. Achievements in CEP included 71,505 learners (159% of the revised target) and 52,240 examinees (31% of the planned target, or 174% of the revised target). The average pass rate between June 1999 and July 2001 was 10%. The low pass rate is partly due to a large number of candidates, including many test takers who had not participated in A&E activities. Of 362 passers reported, 182 (50%) reported having found employment, 56 (15%) reported subsequent enrollment in technical education and 124 (34%) in universities and colleges. These data from the SPs are incomplete, and no firm conclusion on those who received passing grade can be drawn. Regarding cost-effectiveness, DepEd’s estimate shows that the unit cost of sending one student through 1 year of public high school is P5,317 ($104). The estimated direct cost of serving a learner in NFE A&E for 400 hours over 5-7 months is about P2,000 ($39), while the total cost is P2,675 ($52.45). These figures provide a broad orientation. In general, operating costs for NFE would be lower than those for formal education. 54. Staff development and training were completed as planned. Interviews with facilitators and instructional managers revealed that some of them were involved in one or two learning groups only. It would have been more cost-effective if they had served more learning groups or continued to serve in similar activities after project completion. 55. It is too early to assess project efficiency in terms of internal and external efficiency of the NFE system. As noted earlier, the BMES is not operational yet, and in the absence of any data about system efficiency, the Project cannot be assessed. Based on the limited information, the Project is rated less efficient. D. Assessment of Sustainability 56. Sustainability of the Project can be assessed regarding involved institutions, SPs, and learners. At the institutional level, sustainability depends largely on mobilizing local resources and using the procedures and management skills developed by the Project. At project completion, the likelihood of this outcome was not high. Since 1996, besides the national budgetary allocation of 0.07% for BNFE, an annual appropriation of P30 million–P40 million was allocated under the General Appropriations Act to implement LSCS nationwide. Some LGUs have allocated some funds to sustain LSCS programs. The appropriation under the General Appropriations Act for NFE A&E is even less. BNFE reported that LGUs, Government, and community organizations provided NFE classes for a total of 1,139 learners in non-Project sites in Regions I, II, III, X, XI, and CARAGA. The learning materials developed under the Project will continue to provide significant benefits. They are replicated in NFE programs offered by the Government and local NGOs.

13

57. For SPs, sustainability depends also on their degree of commitment. Church-based organizations, with their mission to serve the poor, have shown greater commitment to sustain the programs by pooling new funding sources and contributing voluntary workers. 58. At the learners’ level, the level of sustainability varies. In FELP, livelihood skills such as handicraft making will not be lost. The sustainability of income generating activities depends on whether there is an enabling environment such as markets to absorb the learners’ products. Literacy can be retained in urban settings but less likely in rural areas. The age of the learners also matters. Learning achievements are more sustainable in the case of young learners. The A&E certificate has qualified some learners for reentry into the formal education system, thus creating a sustainable impact at the level of individuals. To enhance sustainability of project benefits future NFE operations should focus on providing reading materials in rural areas, and providing livelihood skills in FELP. Equally important is to adopt a cross-sectoral approach for NFE by including the provision of microfinance and marketing of learners’ products. Children dropouts under 15 should also be included in future NFE projects. In view of the Government’s low budgetary provision, the difficulty of sustaining literacy gains in rural areas, and the less likely continuation of SPs’ programs initiated by the Project, the sustainability of the Project is rated less likely. E. Environmental, Sociocultural, and Other Impacts 59. The environmental impact of the physical infrastructure funded by the Project was negligible. FELP and CEP programs contained some topics on the environment to raise the awareness of beneficiary groups. The Project succeeded to some extent in reducing social inequality as it provided illiterates with social and livelihood skills and, in some cases, increased their earnings. In addition, the Project provided OSY with credentials for a better future, and thus mainstreamed members of a previously marginalized group. 60. The Project sought to narrow gender disparities. It reached out to a sizable group of women (48% of the FELP learners and 57% of the NFE A&E learners). When interviewed, the majority of the female learners reported benefits in learning how to raise families and some reported an increase in earnings. These findings suggest that the Project has contributed to narrow the gender gap in accessing education. 61. The Project had a positive impact on improving the situation of indigenous peoples. With the assistance of an international consultant, FELP prepared a small participatory NFE project for the Ati of Malay in Aklan Province, with a focus on girls and women. Further initiatives would be needed to consolidate learning gains. ADB has funded studies on improving the provision of education to indigenous peoples13 after the closure of the Project. Overall, the environmental, sociocultural, and other impacts of the Project are rated as moderate.

IV. OVERALL ASSESSMENT AND RECOMMENDATIONS

A. Overall Assessment14 62. The Project had a slow and difficult start, but ADB and BNFE were able to diagnose the problems and effectively adjust the design and accelerate implementation. It achieved most of

13 TA 3609-PHI: Studies on the Access of the Poor to Education includes $75,000 for these initiatives. 14 This PCR is part of a sample of about 50% of all PCRs prepared this year that has been independently reviewed

by the Operations Evaluation Department. The review has validated the methodology used and the rating given.

14

its physical targets. The Projects’ qualitative benefits, e.g., a new model for NFE delivery, the NFE A&E system, and high-quality self-instruction modules had only begun to yield results at the end of the project but they have created key building blocks for further NFE development. The Project has put in place a better ALS. Project benefits can become sustainable if there is political commitment and appropriate funding support by the Government. Considering the respective assessments of the Project’s relevance, efficacy, efficiency, sustainability, and other impacts, the overall assessment of the Project is partly successful (Appendix 18). B. Lessons Learned 63. The implementation provided the following important lessons:

(i) Project design was too ambitious and complex to be completed in five years. This resulted in weak institutional arrangements for project implementation. Future intervention should adequately assess the capacity of DECS, BNFE, and NGOs upfront for projects of this nature.

(ii) A weak government budgetary commitment prevented NFE to be fully

institutionalized nationwide and to make the Project sustainable. Without adequate budgetary allocation of the Government, Project impacts cannot become sustainable.

(iii) The inclusion of four NGO umbrella organizations gave them a major role in NFE

delivery and helped speed up implementation. However, the experience gained could not be transferred to other NGOs at grassroot level thus limiting the capacity building impact at NGO level.

(iv) To augment a demand driven approach to delivery of service, the project design

should be based on a more comprehensive analysis of institutional, social, and economic demands. Without employment opportunities, livelihood skills acquired through NFE do not lead to higher income.

(v) Project design should incorporate adequate horizontal and vertical

communication involving many stakeholders at different levels to maximize the quality of project implementation.

(vi) Essential data, e.g. baseline literacy rates, and systematic operational guidelines

for FELP and NFE A&E, should guide implementation and evaluation of NFE projects. The importance of a simple MIS cannot be underestimated.

(vii) Project designs should envisage adjustment to address any shortcomings at the

early stage of implementation, which will make projects more relevant, efficient, and effective.

C. Recommendations

1. Project-related 64. Future Monitoring. The DepEd should continue to monitor post-project activities that are vital to ongoing NFE development, namely, the sustainability of learning benefits, further improvement of the FELP and NFE A&E systems, and improvement and completion of BMES.

15

BNFE should ensure that as part of BMES completion, adequate information should be collected by early 2004 to permit analysis of both internal and external system efficiency. The Government needs to allocate the necessary resources to achieve this objective. 65. Covenants. ADB should ask the EA or the DepEd to complete BMES by end-2003. Completion will affect the assessment of project sustainability. 66. Further Action or Follow-up. The Government could consider the recommendation of its own PCR stating that a follow-up ADB-assisted NFE Project be prioritized to consolidate and build on the gains from the Project and to give attention to illiterates below 15 years old. The fact that functional literacy has broad social benefits and reduces long-term social costs provides a good justification for budgetary allocations for NFE. In the longer term, the development of the NFE A&E could set the foundation for a linkage to higher education levels. NFE as part of the basic education system should become an important pillar of the national education strategy. 67. Timing of PPAR Preparation. It is proposed that the project performance audit report be prepared in mid-2005. 2. General 68. For the preparation and appraisal of NFE projects, the following recommendations are made: (i) the rationale and outputs of project components should be adequately defined to facilitate implementation. Quantitative and qualitative baseline data should be collected for use as a yardstick for achievement of the targets; (ii) the project design should address the problem of literacy and numeracy retention on a long-term basis; (iii) NFE projects should be flexibly designed and allow room for revision during implementation; and (iv) rigorous assessment of the managerial and technical capacity of executing and implementing agencies should be undertaken before any capacity-building program is designed. A simple MIS must be put in use throughout the project. 69. For project implementation the following recommendations are made: (i) staff involved in project implementation should be given clearly defined roles and responsibilities to avoid conflict of interest and duplication; (ii) the Government should try to reduce staff rotation that cause disruption of implementation; (iii) the selection and performance of consultants should be closely monitored; timely termination should be considered to minimize loss to the Project in the case of unsatisfactory performance; and (iv) there should be a mechanism to ensure that knowledge transfer from umbrella NGO organizations to local NGOs takes place to ensure capacity building. Similarly, the dominance of prominent national institutions in conducting research and training activities should, where feasible, be balanced by initiatives to build the capacity of regional and provincial institutions.

16

A

ppendix 1

PROJECT FRAMEWORK

Project Component and Strategy

Coverage/No. Reached Inputs Provided Beneficiary Characteristics

Success Indicators

Functional Education and Literacy Program

• To develop and implement community-based literacy training programs in poor areas, using functional education strategy – i.e. literacy training integrated within a functional livelihood/community training program

• About 540,000 learners/ 22,000 learning groups

• 1,740 communities covered in 24 provinces

• Prototype learning materials, trainers’ guides and teaching aids developed and disseminated widely

• Learning fund provided for (i) adapting materials, (ii) procuring learning supplies, (iii) trainers fees (NGOs, private sector), and (iv) rapid baseline study

• Training of community facilitators and literacy trainers

• Illiterate and functionally illiterate adults (about 17 – 40 years old) out-of-school youth (12 – 17 years old) from the poorest communities — households below 70% of poverty line

• Special targeting for (i) ethnic minority communities, and (ii) households with female heads

• Enrollment and completion rates

• Percentage of those who passed equivalency tests for Levels I and II

• Economic status

- Increased savings and income levels

- Increased membership in cooperatives/self-help groups

• Social impact assessment

- Improved knowledge, aptitude, and practice in basic health, nutrition, and hygiene

- Greater participation in local development activities

- Participation in continuing education programs

Continuing Education Program

• To develop an alternative learning system through the following strategies:

- development of essential learning competencies for nonformal education (primary and secondary equivalency), and

- development of assessment instruments and equivalency testing system for secondary

• Essential learning competencies refined around 6 major content areas

• About 170,000 participants tested and results evaluated

• Pilot groups with about 1,500 instructional managers and 45,000 learners

• Planning and materials development workshops – national and regional levels

• Specialist services

• Field testing and production of testing instruments

• Research and evaluation

• Planning and materials development workshops

• Adults and out-of-school youth from poor households in communities without any elementary school or without a complete elementary school

• Graduate of the functional education and literacy programs (Levels I and II)

• Increased participation in continuing education/self-learning programs

- enrollment rate in self-learning programs

- % success rate of Level III and IV equivalency tests

• Increased savings and income levels

Appendix 1

17

Project Component and Strategy

Coverage/No. Reached Inputs Provided Beneficiary Characteristics

Success Indicators

testing system for secondary equivalency)

• To develop, test, and carry out limited implementation of nonformal self-learning programs for lower secondary equivalency

• To develop, field-test, and implement on a pilot-basis radio and video learning supplements to support Level II and III learners

learners

• 85,000 Level II learners reached by radio program

• Field testing and production of pilot self-learning materials

• Research and evaluation

• Specialist services

• Training of instructional managers

• Planning and materials development workshops

• Field testing and production of radio programs

• Research and evaluation

• Specialist services

• Training of instructional managers

• Grade 4 leavers (Level II)

• Grade 6 leavers (Level III)

• High school dropouts (Level IV)

• Increased membership in cooperatives/self-help groups

Capacity-Building Program

• To strengthen NFE policy formulation and program management capacity

• To improve coordination with local government units, sectoral agencies and NGOs, and mobilize local financial and staff resources

• To strengthen NFE implementation at the community level

• Project steering committee members, NFE coordinating committees

• NFE coordinating committees at national, regional, provincial, and municipal levels

• NFE community facilitators and NGO field workers in target areas

• National strategic planning workshops, social mobilization workshops

• Orientation, planning, and coordination workshops for the NFE coordinating committees

• Social mobilization workshops and dissemination of information, education, and communication materials

• Intensive training workshops in NFE program implementation

• Interagency planning workshops

• Key Government policymakers, NGO/private sector contributors to NFE, and LGUs

• NGOs, LGUs, sectoral government agencies, and institutes of higher education engaged in community-based development programs

• Community-based field workers, sectoral specialists, social workers and local leaders

• Coverage, efficiency, and cost-effectiveness of NFE programs

• Participation rate in coordinative workshops

• % increase in local contributions to NFE programs

• Number of community learning groups formed

• Quality and number of local NFE subprojects submitted

• Number of community learning groups formed

18

Appendix 1

Project Component and Strategy

Coverage/No. Reached Inputs Provided Beneficiary Characteristics

Success Indicators

workshops

• Local short-term staff training program

• Overseas study tours, fellowships

• Special accelerated teacher training programs for ethnic minority groups

• Specialist services

• Equipment and vehicles: office equipment and furniture, utility vehicles, motorcycles and motorized boats, and microcomputers

• To facilitate project management, administration and supervision

• National, regional, provincial, and municipal DECS offices

• Civil works: (i) Bureau of Nonformal Education, and (ii) NFE units in regional and provincial DECS offices

• DECS, NFE staff, NFE coordinating committee, NGOs involved in NFE implementation

• Quality of project management

- Frequency of field supervision

- MIS established and maintained

- Efficient record-keeping and timely reporting

- DECS NFE staff morale, attitudes

Monitoring and Evaluation

• To improve monitoring and evaluation of project implementation and outcome

• Project BMES

• Project documentation and NFE MIS

• Project supervision

• Reporting and MIS

• Research and evaluation studies

- Baseline surveys: literacy mapping and ethnographic studies

- Impact assessment and tracer studies

• Project staff

• DECS

• Effective BMES established

• Regular supervision and monitoring

• Quality of studies and action research

• Effective use of MIS

• Project progress reports

Appendix 1

19

Project Component and Strategy

Coverage/No. Reached Inputs Provided Beneficiary Characteristics

Success Indicators

- Cost-benefit analysis

- Action research BMES = benefit, monitoring, and evaluation system; DECS = Department of Education, Culture, and Sports; LGU = local government unit; MIS = managem ent information system; NFE = nonformal education; NGO = nongovernment organization; OSY = out-of-school youth.

20 Appendix 2

SUMMARY OF PLANNED AND ACTUAL PROJECT OUTPUTS

Component/Items Planned (a)

Actual (b)

b/a Remarks

FELP

No. of learners 448,306 465,913 104% Target number revised to 448,307 at MTR

Literacy materials development

Not specified 29 modules 29 facilitator’s guide 1 exercise booklet 1 general guide

CEP

Development of alternate learning system

Complied with

No. of learners in self-learning programs No. of examinees

45,000 170,000

71,505 52,240

159%

31%

After MTR, CEP re-conceptualized as NFE A&E Revised target of 30,000 examinees during review in 1998, 174% achievement

Dev of learning materials

Not specified

Phase I: 152 print-based modules 10 audiotapes Phase II: 535 print-based modules 47 audiotapes 3 videotapes

Radio & video program No. of learners Materials dev

85,000

Not launched during Project 50 audiotapes

Not completed in time for Project use

CBP

Consulting services (person-months) • International consultants • Domestic consultants

34 186

85.98 184.81

253% 99%

After MTR, TA revised in terms of strategies, team composition and service schedule

Staff development • Long-term studies and

fellowships (person-years) • Short-term study tours and

courses (person-months)

149 152

152.41 151

107%

98%

Training workshops Complied with Project steering committee Complied with PMO Complied with After MTR fully integrated into

BNFE NFE Coordinating Committee Complied with Monitoring & evaluation BMES

NFE MIS Partially completed Partially completed

A&E = accreditation and equivalency; BMES = benefit, monitoring, and evaluation system; BNFE = Bureau of Nonformal Education; CBP = capacity building program; CEP = continuing education program; FELP = functional education and literacy program; MIS = management information system; MTR = midterm review; NFE = nonformal education; PMO = project monitoring office; TA = technical assistance.

Appendix 3 21

PROFILE OF FELP LSCS SERVICE PROVIDERS

Service Providers Learners Served LSCS Contract Type of Service Provider Number % Number % Number Amount (P) University/College 10 6 18,596 4 23 16,393,299 LGU 29 17 26,117 6 29 16,534,085 Peoples Organization/Cooperative 45 27 55,537 12 47 46,338,548 Church-based Organization 13 8 6,912 1 17 7,963,800 Teachers Association 13 8 8,581 2 20 11,059,500 Individual NGO 53 32 60,153 13 34 30,512,855 Umbrella NGO Network 4 2 290,017 62 4 299,791,375 Total 167 100 465,193 100 174 428,593,462

FELP = functional education and literacy program, LGU = local government unit, LSCS = literacy service contracting scheme, NGO = nongovernment organization. Source: Bureau of Nonformal Education Project Completion Report.

22 Appendix 4

CURRENT AND CUMULATIVE ENROLLMENT OF FELP

Region 1995 1996 1997 1998 1999 2000 Total Group A

7,091

16,068

30,950

125,815

25,512

78,602

284,038

Group B na na na 114,531 21,889 45,455 181,875

Total

7,091

16,068

30,950

240,346

47,401

124,057

465,913 na = not applicable.

Notes: Group A includes the project provinces in Regions V, IX, XII, and Autonomous Region of Muslim Mindanao. Group B includes the project provinces in Regions IV, VI, VII, VIII, and Cordillera Autonomous Region.

Appendix 5 23

FELP PROFILE OF LEARNERS

Variables Number of Learners Percentage Gender Male 162,402 34.91 Female 224,392 48.24 Not stated 78,399 16.85 Language Used Tagalog 35,732 7.68 Cebuano/Bisaya 47,808 10.28 Tausug 41,449 8.91 Bicol 22,966 4.94 Hiligaynon 34,383 7.39 Ilokano 40,769 8.76 Tausug 13,798 2.97 Waray 19,957 4.29 Yakan 42,676 9.17 Maguindanaon 36,879 7.93 Other dialects 50,377 10.83 Not stated 78,399 16.85 Age 15-24 years 78,823 16.94 25-29 156,956 33.74 30-49 70,299 15.11 50 and above 45,718 9.83 Not stated 113,397 24.38 Marital Status Single 72,507 15.59 Married 186,829 40.16 Widow/Widower

Separated 14,450 3.11

Not stated 191,407 41.15 Educational Background Unschooled 140,682 30.24 (Literacy Level) Grades 1-4 115,706 24.87 Grades 5-6 385 0.08 Not stated 208,420 44.80 Monthly Income Less than P2,500 142,662 30.67 P2,501-P3,500 143,544 30.86 P3,501-P4,500 13,095 2.81 P4,501-above 6,394 1.37 Not stated 159,498 34.29 FELP = functional education and literacy program. Notes: (i) Totals for each variable vary due to differences in response rates to questions on the enrollment form. (ii) Total number of learners is 465,193 based on the report Profile of Literacy Service Contracting Scheme

Service Providers by Type and Learners Served. Source: Bureau of Nonformal Education Project Completion Report.

24 Appendix 6

Table A6.1: FELP Research Studies/Activities

No. Title

1 Learning from Life: An Ethnographic Study of Functional Literacy in Marginal Philippine Communities

2 Cognitive Consequences of Literacy: Studies on Thinking in Five Filipino Communities

3 Indigenous Learning Systems (Mt. Province)

4 Indigenous learning Systems (Tawi-Tawi)

5 The Algebra of the Weaving Patterns, Music and Kinship System of the Kankaney-ey of Mt. Province

6 Pre-Implementation Rapid Community Assessment and Training at the Community Level

7 Taxonomies of Functional Literacy and Numeracy, Levels I-III

8 Reconstructing Educational Knowledge: Incorporating Community Knowledge in Functional Literacy Programs

9 Preliminary Inventory of Literacy Materials

10 Development and Field-Testing of Exemplar and Instructional Materials for FELP

11 Action Research for the Development of the Continuing Education Program

12 Summative Evaluation and Impact Assessment Study of the Philippines Nonformal Education

FELP = functional education and literacy program. Source: Bureau of Nonformal Education Project Completion Report (2002).

Table A6.2: List of Planned FELP Researches Not Undertaken No. Title

1 Action Research on Indigenous Learning Systems 2 Cost Benefit Analysis of NFE Programs 3 The Special Requirements of Handicapped Groups Study

4 Literacy Mapping and Tracking 5 Evaluation of FELP Curriculum Materials

6 FELP Participants Ethnographies

7 Comparative Case Studies of FELP Proponents 8 Cost Utility of FELP and ALS

ALS = alternative learning system, FELP = functional education and literacy program, NFE = nonformal education. Sources: Report and Recommendation of the President (1993) and Philippines’ Nonformal Education Project

Research and Evaluation Master Plan (1996).

Appendix 7 25

CEP PROFILE OF SERVICE PROVIDERS FOR LEARNING SUPPORT DELIVERY SYSTEM (LSDS)

Service Provider Learners Served LSDS Contract Type of Service Provider

Number % Number % Number Amount ($) 1.

University/College

6

8

6,369

9

11

196,246

2. Local Government Unit 6 8 1,625 2 6 54,682 3. People's Organization/Cooperative 9 12 2,271 3 9 70,126 4. Church-Based Organization 3 4 592 1 3 19,244 5. Teachers Association 1 1 255 0 1 7,579 6. Individual Nongovernment Organization 29 38 13,279 19 33 406,156 7. Umbrella 2 3 38,610 54 7 1,189,328 8. Department of Education-Hired 21 27 8,504 12 22 204,215

Total 77 100 71,505 100 92 2,147,576

A&E = accreditation and equivalency, CEP = continuing education program, NFE = nonformal education. Source: Bureau of Nonformal Education Project Completion Report.

26 Appendix 8

CEP PROFILE OF LEARNERS

Number of Learners Percent Variables

EL SL Total EL SL Total Gender Male 14,539 15,879 30,418 20 22 43

Female 19,918 20,679 40,597 28 29 57

Not Indicated 490 1

Total 71,505 100

Language Used Bicolano 1,112 1,185 2,297 2 2 3

Cebuano 6,490 11,129 17,619 9 16 25

Hiligaynon 1,584 2,450 4,034 2 3 6

Ilocano 1,403 1,315 2,718 2 2 4

Maguindanaon 6,407 1,932 8,339 9 3 12

Tagalog 9,564 14,277 23,841 13 20 33

Tausug 598 371 969 1 1 1

Waray 689 788 1,477 1 1 2

Other dialects 5,001 3,079 8,080 7 4 11

Not Indicated 2,131 3

Total 71,505 100

Age 15-24 years 19,990 23,357 43,347 28 33 61

25-39 9,055 8,145 17,200 13 11 24

40-59 3,419 3,574 6,993 5 5 10

60 and above 1,704 1,388 3,092 2 2 4

Not Indicated 873 1

Total 71,505 100

Marital Status Single 18,702 21,922 40,624 26 31 57

Married 15,051 13,556 28,607 21 19 40

Separated 4 18 22 0 0 0

Widow/Widower 574 852 1,426 1 1 2

Not Indicated 826 1

Total 71,505 100

Educational Background Grades 1-4 11,860 11,860 17 17

Grades 5-6 22,203 22,203 31 31

1st-2d Year High School 20,158 20,158 28 28

3rd-4thYear High School 15,892 15,892 22 22

Not Indicated 1,392 2

Total 71,505 100

Monthly Income Less than P2,500 23,097 22,547 45,644 32 32 64

P2,501-P3,500 5,384 6,654 12,038 8 9 17

P3,501-P4,500 1,391 3,308 4,699 2 5 7

P4,501-up 725 3,664 4,389 1 5 6

Not Indicated 4,735 7

Total 71,505 100

CEP = continuing education program, EL = elementary level, SL = secondary level.

Appendix 9 27

CEP PROFILE OF EXAMINEES

Number of Examinees Percent Variables

EL SL Total EL SL Total

Gender Male 3,192 25,115 28,307 6.1 48.1 54.2

Female 3,354 20,579 23,933 6.4 39.4 45.8

Total 6,546 45,694 52,240 12.5 87.5 100.0

Age 15-24 years 3,991 32,787 36,778 7.6 62.8 70.4

25-39 1,510 8,780 10,290 2.9 16.8 19.7

40-59 736 2,943 3,679 1.4 5.6 7.0

60 and above 98 189 287 0.2 0.4 0.5

(not indicated) 211 995 1,206 0.4 1.9 2.3

Total 6,546 45,694 52,240 12.5 87.5 100.0

Educational Grades 1-4 2,526 759 3,285 4.8 1.5 6.3

Background Grades 5-6 3,363 4,825 8,188 6.4 9.2 15.7

1st-2d Year High School 125 25,438 25,563 0.2 48.7 48.9

3rd-4th Year High School 36 12,983 13,019 0.1 24.9 24.9

(not indicated) 496 1,689 2,185 0.9 3.2 4.2

Total 6,546 45,694 52,240 12.5 87.5 100.0 CEP = continuing education program, EL = elementary level, SL = secondary level.

28 Appendix 10

PLANNED AND ACTUAL CONSULTING SERVICES

Person-Months Types of Services Planned Actual

International Consultants 1. NFE Management, Planning and Policy Specialist and Team Leader 15 54.47 2. NFE Training Plan Specialist 3 1.50 3. NFE Research Planning Specialist 3 0.70 4. NFE M&E/MIS Specialist 6 21.4 5. NFE Examination and Quality Assurance Specialist 3 6.16 6. Educational Media Specialist 4 1.75 Total 34 85.98 Domestic Consultants 1. NFE Management, Planning and Team Building Specialists 42 60.44 2. Functional Literacy Curriculum and Materials Development Specialists 30 9.50 3. NFE Research Coordinator 19 12.50 4. M&E/MIS Specialists 18 25.89 5. Functional Education Training Specialists 30 22.61 6. ALS Examination and Accreditation Specialist 10 7.74 7. Continuing Education Curriculum and Materials Development Specialists 24 16.10 8. Education Media Specialist 13 30.03 Total 186 130.81 ALS = alternative learning system, M&E = monitoring and evaluation, MIS = management information system, NFE = nonformal education. Sources: Report and Recommendation of the President for the Nonformal Education Project and Bureau of

Nonformal Education.

Appendix 11 29

PLANNED AND ACTUAL STAFF DEVELOPMENT

Person-Years Planned Actual

Long-Term Studies and Fellowships Overseas

In-Service Master’s Degree in Nonformal Education 6 6.75 In-Service Certificate Course (6-9 months) 3 3.00 Domestic Master’s Degree in Nonformal Education 10 11.66 Accelerated Teacher Training Program for the

Cultural Minorities

130

131.00 Total 149 152.41 Short-Term Study Tours and Courses

Overseas Study Visits in Nonformal Education Model Countries 20 19 Short-Term Courses/Attachments 30 30 Domestic Short-Term Courses 102 102 Total 152 151 Sources: Report and Recommendation of the President for the Nonformal Education Project and

Bureau of Nonformal Education.

30

A

ppendix 12

COST BREAKDOWN BY PROJECT COMPONENT ($‘000)

Actual Appraisal Estimate

Asian Development Bank

Norway Granta Government Overall Total

Component/ Category

Foreign Local Total Foreign Local Total Foreign Local Total Local Foreign Local Total

A. Base Costs

1. Civil works 156 522 678 42 - 42 204 42 204 246

2. Furniture 56 88 144 13 - 13 1 13 1 143. Vehicles & equipment 955 107 1,062 962 - 962 112 962 112 1,0744. Learning funds 779 7,018 7,797 1,200 9,511 10,711 1,200 9,511 10,7115. Materials & other

learning resources 318 2,861 3,179 343 2,422 2,765 343 2,422 2,765

6. Research and studies 71 1,344 1,415 109 1,324 1,433 109 1,324 1,4337. Training 191 3,624 3,815 93 1,475 1,568 93 1,475 1,5688. Overseas fellowships 640 408 1,048 597 - 597 597 5979. Consulting services 664 1,048 1,712 168 159 327 1,036 528 1,564 1,204 687 1,89110. Social mobilization

workshop 23 382 405 13 321 334 13 321 334

11. Recurrent costs 2,591 2,591 3,595 3,595 3,59512. Taxes and duties 1,356 1,356 148 148 148

Subtotal (A) 3,853 21,349 25,202 3,540 15,212 18,752 1,036 528 1,564 4,060 4,576 19,800 24,376B. Contingencies 582 5,207 5,790

C. Service charge during implementation

509 509 420 420 420 420

Total 4,944 26,556 31,501 3,960 15,212 19,172 1,036 528 1,564 4,060 4,996 19,800 24,796a A grant from Norway amounting to $1.6 million was approved on 23 December 1993 (TA 2036-PHI: Capacity Building in Nonformal Education) after the

Project’s approval on 30 September 1993. The advisory TA financed the capacity-building program of the Project.

Appendix 13 31

LOAN DISBURSEMENTS

($ million)

Year Quarter Actual Disbursement

Actual Cumulative

Disbursement

Percentage of Actual

Disbursement 1994 IV 0.097 0.097 0.5 1995 I

II III IV

0.329 0.213 0.102 0.029

0.426 0.639 0.741 0.770

1.7 1.1 0.5 0.2

1996 I II III IV

0.278 0.311 0.236 0.308

1.048 1.359 1.595 1.903

1.5 1.6 1.2 1.6

1997 I II III IV

0.256 0.226 0.178 0.133

2.159 2.385 2.563 2.696

1.3 1.2 0.9 0.7

1998 I II III IV

0.135 0.194 0.753 0.631

2.831 3.025 3.778 4.409

0.7 1.0 3.9 3.3

1999 I II III IV

1.805 2.060 1.506 1.095

6.214 8.274 9.780

10.875

9.4 10.7 1.9 5.7

2000 I II III IV

0.885 0.916 1.181 2.249

11.760 12.676 13.857 16.106

4.6 4.8 6.2

11.7 2001 I

II III IV

0.850 0.616 0.690 0.909

16.956 17.572 18.262 19.171

4.4 3.2 3.6 4.7

Total 19.173 19.173 100.0 Source: Loan Financial Information System.

32

Appendix 14

PROJECT IMPLEMENTATION SCHEDULE

Activities 1993 1994 1995 1996 1997 1998 1999 2000 2001

Project Start-Up Activities

Appointment/Recruitment of PMO Staff

Literacy Mapping and Ethnographic Studies

Social Mobilization and Orientation Procurement of Equipment, Vehicles,

Furniture for PMO Functional Education and Literacy

Strategic Planning and Policy Review Intensive NFE Team Building and

Orientation Training of NFE Program Managers Interagency Planning and Monitoring

Workshop Training of Community Facilitators

and NFE Trainers Prototype Materials – Survey and

Materials: Collection/Review Prototype Materials

Development/Production Subproject Implementation Literacy Mapping/Ethnographic

Studies Action Research: Indigenous Learning Special Project on Indigenous

Learning

Planned Actual

FELP = functional education and literacy program, NFE = nonformal education, PMO = project management office. Note: Italicized words are new or additional activities.

A

ppendix 14 334

Activities 1993 1994 1995 1996 1997 1998 1999 2000 2001 FELP Learning Services Contracting

Scheme FELP Subproject Impact Studies

FELP Cost-Effectiveness Studies Social Mobilization Workshops, IEC

Dissemination Monitoring, Supervision, and

Evaluation Continuing Education

Consultative Planning Workshops Social Mobilization Workshops/IEC

Dissemination ALS – Development of Curriculum and

Testing System Field Testing of ALS Curricula and

Testing Instruments Production of ALS Curriculum

Guides, Tests ALS Evaluation Study Self-Learning Module, Review, and

Development of Materials Action Research Materials Production Training of Instructional Managers

• Training of Trainors • Training Workshop on the

Authentic Assessment and Certification System

• Assessor’s Training in the Assessment of Presentation Portfolios

Evaluation Study Radio and Video Program Planning

ALS = alternative learning system, IEC = information, education, and communication.

34

A

ppendix 14

Activities 1993 1994 1995 1996 1997 1998 1999 2000 2001 Production of Radio Programs

Training of Instructional Managers a

Pilot Production of Video Materials

Evaluation of Radio and Video Programs b

Consultanciesc

International Consultants

• NFE Management, Planning

& Policy

• NFE Training Plan

• NFE Monitoring and

Evaluation/MIS

• NFE Research Planning and

Evaluation

• NFE Examination and

Quality Assurance

• Educational Media

- TA Coordinator/NFE Specialist

- NFE Community-Based Literacy

MIS = management information system, NFE = nonformal education, TA = technical assistance. a The training of instructional managers for the utilization of audiotapes will be undertaken in future implementation of learning support delivery system. b Evaluation was not undertaken since the audio and videotapes for the radio programs were developed during the later period of the Project. c The presentation of actual versus planned technical assistance is very problematic because of the complete reconfiguration of the technical assistance during

the course of the Project.

Appendix 14

35

Activities 1993 1994 1995 1996 1997 1998 1999 2000 2001 - Management Specialist

- NFE Cultural Specialist

Domestic Consultants

• NFE Management, Planning

and Team Building

• FEFL Curriculum and

Materials Development

• Research Coordinator

• Monitoring and

Evaluation/MIS

• Training Program Development

- Literacy Trainingd

Community organizing/social Communications

- Income – generating skills development

• ALS Examination and Accreditation

• CE Curriculum and Materials

Development

• Educational Video and Radio

Program

CE = continuing education, FEFL = functional education and functional literacy. d This is integrated as subproject implementation for Literacy Service Contracting Scheme (LSCS) under the Functional Education and Literacy Program (FELP).

36

Appendix 14

Activities 1993 1994 1995 1996 1997 1998 1999 2000 2001 NFE FELP

NFE Learning Assessment Specialist

Human Resources Development Specialist

Functional Education Literacy Specialist

Advocacy and Social Mobilization Specialist

Implementation Supporte

Computer Systems Support and Maintenance

• Development of FELP-MIS

• Training of DepEd NFE field officials and SPs on the use of BMES Manual

• Development of NFE A&E M&E System

• Development of the NFE MIS

• Establishment of LAN and Internet Connection

• Training of MIS personnel on the NFE MIS at the local levels

Finance and Accounting

• Training and Workshop of Division Bookkeepers

Staff Development

Overseas: Long-term

BMES = benefit monitoring and evaluation system, DepEd = Department of Education, SP = service providers. e These are not outside of the agreed original scope of the Project. However the BNFE has added its own subcategory Implementation

Support to cover activities in the area of MIS development. This subcategory was not in the original project implementation schedule of the Project, though the activities were subsumed under the other respective subcomponents.

Appendix 14

37

Activities 1993 1994 1995 1996 1997 1998 1999 2000 2001 Overseas: Short-term

Domestic: Long-term

Domestic: Short-term

Procurement of Equipment, Furniture, Vehicles

National

Regional/Provincial

Civil Works

Upgrading of BNFE/PMO Office

Upgrading DECS Divisional Offices

Project Midterm Review

BNFE = Bureau of Nonformal Education, DECS = Department of Education, Culture, and Sports.

38 Appendix 15

PROJECT ORGANIZATION STRUCTURE

DECS Office of the Secretary

Project Steering Committee

Project Management Office (BNFE)

DECS Regional Office Regional Director

NFE Division Chief

DECS Divisional Office Division Superintendent

NFE Supervisor

DECS District Office District Supervisor/Principal

NFE District Coordinator

Regional NFE Coordinating Committee

Provincial NFE Coordinating Committee

Municipal NFE Coordinating Committee

Regional Development

Council

Provincial Planning &

Development Office

Municipal/City Planning &

Development Office

Barangaya Development

Council

NFE Subproject Proponents

Beneficiary Groups

Supervisory Coordinative BNFE = Bureau of Nonformal Education; DECS = Department of Education, Culture, and Sports; NFE = Nonformal

Education. a Community.

Appendix 16 39

STATUS OF COMPLIANCE WITH LOAN COVENANTS

Covenant Reference in

Loan Agreement

Status of Compliance

1. To expedite project implementation through a timely release of funds, the Borrower will establish a project imprest account with the Bangko Sentral ng Pilipinas. The project imprest account will be established, managed, and liquidated in accordance with terms and conditions satisfactory to ADB and consistent with ADB’s Guidelines on Imprest Fund and Statement of Expenditures Procedures dated November 1986, as amended from time to time, and subject to detailed arrangements agreed upon between the Borrower and ADB. The initial amount to be deposited into the project imprest account will not exceed $200,000

LA, Schedule 3, para. 16 (a)

Complied with

2. Maintenance and audit of separate accounts for the ADB-financed components of the Project; unaudited project accounts to be furnished not later than 6 months after the end of each fiscal year; audited accounts and auditors report not later than 9 months after the end of each fiscal year.

LA, Section 4.06 (b)

Complied with

3. Quarterly reports on the carrying out of the Project and on the operation and management of the project facilities to be submitted.

LA, Section 4.07 (b)

Complied with

4. Project Completion Report to be prepared and furnished not later than 3 months after physical completion of the Project.

LA, Section 4.07 (c)

Complied with

5. As the Project Executing Agency, Bureau of Nonformal Education will have overall responsibility for planning, organization, management and implementation of the Project, and will liaise closely with other concerned agencies of the Borrower, local government units, nongovernment organizations (NGOs), and ADB. The Bureau of Nonformal Education will be assisted by the Educational Development Project Implementing Task Force (EDPITAF) of the Department of Education, Culture, and Sports in the financial management, administration, and procurement activities of the project.

LA, Schedule 6, para. 1

Complied with. Seven EDPITAF staff were recruited to the project for their knowledge and expertise in implementing foreign-assisted projects, specifically to assist in the financial management and procurement activities of the project.

6. Prior to the effective date, the Borrower will establish a project steering committee (PSC) at the national level to be responsible for (i) coordinating Project activities among DECS, other concerned line agencies of the Borrower and NGOs involved in the Project, (ii) reviewing and approving policy adjustments, and (iii) reviewing and approving annual project implementation plans

LA, Schedule 6, para. 2

Complied with. The first PSC meeting was held on 11 February 1994. For the duration of project implementation, the PSC held a total of 22 meetings with a frequency of 5 meetings per year.

7. The PSC membership will include the following: (i) the secretary of DECS, as chairperson; (ii) the secretary of the Department of Social Welfare and Development; (iii) the secretary of the Department of Interior and Local Government; (iv) a representative of the institutes of higher education; (v) representative of CODE-NGO; and (vi) the project director referred to in paragraph 4 of this schedule.

LA Schedule 6, para. 3

Complied with

40 Appendix 17

Covenant Reference in

Loan Agreement

Status of Compliance

8. Prior to the effective date, a project management office (PMO) will be established within BNFE to manage the Project. The PMO will be headed by the director of BNFE, who will function as project director. A full-time project manager will be appointed to oversee the day-to-day operations of the Project, and will be assisted by the BNFE division chiefs for Functional Literacy, Continuing Education, and Staff Development, and the executive officers for Monitoring and Evaluation (M&E), and Operational Support (finance, administration and procurement).

LA, Schedule 6, para. 4

Complied with. However, the actual establishment was delayed due to the omission of project funds for personal services of PMO staff in the 1994 General Appropriations Act.

9. The PMO will be staffed jointly by the regular staff of BNFE and contractual staff recruited through EDPITAF. Twenty-eight full-time project staff for PMO will be engaged on a contractual basis to staff the operational support units (Finance and Administration and Physical Facilities) and M&E Unit, and will work alongside the regular staff of BNFE who will be assigned full-time to support PMO. Fifteen of such 28 full-time project staff will be recruited prior to the effective date, pursuant to the provision of Section 6.01 (b) of this Loan Agreement. The remaining project staff positions for PMO that have not been filled prior to the effective date, will be filled by 31 March 1994. To strengthen the central DECS office’s capacity to undertake M&E, and financial management, nine positions related to such tasks will be regularized at the end of the Project, subject to the findings of a review to be carried out jointly by ADB and the Borrower within 6 months of project completion.

LA, Schedule 6, para. 5

Partially complied with. Only 21 staff instead of 28 were authorized by Department of Budget Management (DBM) due to the budgetary constraint of the government. To comply with the agreement, BNFE assigned regular staff to fill up the positions.

10. Under the direction of PMO, DECS’ Nonformal Education (NFE) staff at the regional, divisional (i.e., provincial) and district (i.e., municipal or city) offices in the project area will implement the project activities and coordinate with other concerned agencies of the Borrower and NGOs at those respective levels. During project implementation, NFE supervisors and coordinators in the targeted communities of the project area will be released from all formal school teaching duties and be assigned full-time to the Project.

LA, Schedule 6, para. 6

Complied with; DECS memo issued

11. Additional project staff will be recruited to strengthen the financial management and administration capacity at the regional and divisional DECS offices. Nine full-time financial analysts will be engaged, one for each of the regions included in the project area, to oversee all aspects of the financial management of the project activities in the region. Twenty-four full-time senior bookkeepers will be engaged, one in each of the provinces included in the project area, to assist the divisional DECS offices in establishing and maintaining project accounts and records, and ensuring timely disbursement and liquidation of project funds. Such regional and provincial positions will be filled by 1 January of the start-up year of the Project in the region or province concerned.

LA, Schedule 6, para. 7

Partially complied with. The Project hired only 24 full-time bookkeepers due to budgetary constraints on the part of the Philippine Government. Creation of nine full-time financial analysts positions disapproved by DBM. However, the regional accountants were designated to act as financial analysts and were granted honoraria for the additional workload.

Joint review scheduled from 09 July to 02 August 2002

Appendix 16 41

Covenant Reference in

Loan Agreement

Status of Compliance

The 33 positions will be regularized at the end of the project and absorbed against other positions to be abolished as part of the ongoing DECS plan to streamline the regional and divisional DECS offices, subject to the findings of a review to be carried out jointly by ADB and the Borrower within 6 months of project completion.

12. Within the interagency framework established under the Borrower’s Education for All Action Plan, DECS will establish NFE coordinating committees at the regional, provincial, and municipal levels, comprising representatives from concerned LGUs, NGOs, and line agencies of the Borrower. With the DECS NFE staff providing permanent secretariat support, such NFE coordinating committees will hold regular meetings for coordinated planning, to ensure interagency participation in the training and materials development and social mobilization workshops under the Project, and in the joint monitoring and evaluation of project activities.

LA, Schedule 6, para. 8

Complied with

13. (i) The Borrower, through DECS, will enter with each concerned LGU, into an Implementation Agreement, which will set forth the LGUs commitments to (a) appoint a representative to the concerned NFE coordinating committee referred to in paragraph 8 of this Schedule, and (b) contribute budgetary and staff resources to the implementation of subprojects as well as activities under part B of the Project. A copy of these implementation agreements will be furnished to ADB after signing.

LA, Schedule 6, para. 9

Complied with

(ii) No funds will be disbursed in respect of the activities of the respective LGUs until a relevant implementation agreement between the Borrower and the LGU concerned is in place.

Complied with

14. The organization and implementation of subprojects will be undertaken by organizations with proven track record in undertaking effective community-organizing and community-development programs that are able to make a minimum of 20% equity contribution toward the implementation of the subproject. Such organizations may include (i) Borrower’s agencies or LGUs with field extension capacity, (ii) an accredited NGO with a proven track record for managing community-based projects, or (iii) a State or private college or university with appropriate extension service capacity. Initial screening and identification of the field implementing agencies will be conducted by the NFE coordinating committee at the municipal and provincial levels. All NGOs will be members or affiliates of the Caucus of Development NGO (CODE-NGO) and will be signatories of the NGO Code of Ethics endorsed by CODE-NGO. DECS will be responsible for the final approval of the subproject proponents, pursuant to the +provisions of paragraph 15 of this Schedule.

LA, Schedule 6, para. 10

Complied with

42 Appendix 17

Covenant Reference in

Loan Agreement

Status of Compliance

15. The Borrower will ensure that the accreditation and equivalency be developed under Part B of the Project with the assistance of the National Educational Testing and Research Center and other appropriate educational research institutes.

LA, Schedule 6, para. 11

Complied with

16. The target beneficiaries of continuing education program will be prioritized according to the following categories: (i) graduates of functional education and literacy program, (ii) primary school (Grade 4) leavers, (iii) intermediate school (Grade 6) leavers, and (iv) high school dropouts.

LA Schedule 6, para. 12

Complied with

17. The Borrower will ensure that the activities under Part C of the Project are closely aligned with the recommendations of the Congressional Commission on Education regarding an appropriate institutional arrangement for nonformal education.

LA, Schedule 6, para.13

No longer relevant due to policy change

18. The PMO will be responsible for the administration of fellowships, study tours, and attachment programs provided under Part C of the Project. The PMO will prepare the criteria for the selection of candidates for such fellowships, study tours, and attachment programs; select such candidates according to the procedures of DECS; and make appropriate arrangements, satisfactory to ADB, for carrying out the fellowships, study tours, and attachment programs.

LA, Schedule 6, para. 14

Complied with

19. (i) A subproject will support the identification, organization, and social preparation, under Part A of the Project, of learning groups in one or more communities. Within each province included in the project area, up to 90 communities will be targeted for intensive NFE activity under Part A of the Project on the basis of lowest literacy and school participation rate, and the availability of community-based organizations to undertake community-based NFE programs.

LA, Schedule 6, para. 15

Complied with

(ii) The identification of target groups will be undertaken after literacy mapping and interagency consultations to identify the communities with high illiteracy rates, low participation rates in schools, and high incidence of poverty. The priority target groups are the following: (a) illiterate and functionally illiterate adults and overage out-of-school youth (i.e., school dropouts and stay-outs over 12 years of age who are too old to be admitted back to elementary school) from the target community whose household income level falls below 70% of the poverty line; and (b) functionally literate OSYs and adults in the target communities (sixth grade leavers or graduates of functional literacy classes) who have expressed interest in participating in continuing education programs. In all classes, priority consideration will be given to women from the poorest households, especially those who are heads of the household, and to members of cultural minority groups.

Complied with

Appendix 16 43

Covenant Reference in

Loan Agreement

Status of Compliance

20. (i) Proposed subprojects will initially be reviewed and short-listed by the provincial NFE coordinating committee if the requested level of funding is less than P300,000, and by the regional NFE coordinating committee if such level is P300,000 or more. Proposals will then be submitted for approval, according to the following guidelines:

LA, Schedule 6, para. 16

Partially complied with. While the role of the municipal coordinating committees have been generally limited to review and endorsement of short-listed subproject proposals, the regional and provincial committees have generally been nonfunctional.

(a) Subproject with a cost less than P300,000: to be approved by the divisional DECS office;

(b) Subprojects with a cost between P300,000 and P600,000: to be approved by the regional DECS office; and

Complied with

(c) Subprojects with a cost more than P600,000: to be approved by PMO.

(ii) The subprojects will be selected on the basis of criteria and guidelines as agreed between the Borrower and ADB. Within 6 months of the effective date, ADB and PMO will jointly review the selection criteria and process for subprojects and make necessary adjustments.

Complied with

(iii) The Borrower will provide ADB, for information, with copies of all the short-listed subproject proposals submitted by the provincial NFE coordinating committee.

Complied with

21. A benefit monitoring and evaluation (BME) system will be established under the Project to provide timely information and feedback, both on the quantitative and qualitative aspects of the Project, and to ensure timely adjustments in project inputs and management are made during project implementation. At the commencement of project implementation, PMO will develop and refine the BME system for the Project and, within 6 months of the effective date, and in close consultation with ADB, finalize such a sys tem, including the selection of project performance indicators. The guidelines on the operations of BME will include the following: (i) baseline surveys, prior to project start-up in the targeted communities of the project area; (ii) an NFE management information system to support project management and performance monitoring; (iii) an institutional and process M&E system; and (iv) coordination of outputs from research studies for impact evaluation. The PMO and DECS’ NFE staff will monitor and evaluate project implementation at all levels on a continual basis through quarterly project progress reports. The NFE MIS will be established in PMO, with compatible subsystems installed at regional and provincial levels to provide timely and readily verifiable data for BME purposes. The NFE MIS will also serve as information resource base for disseminating technology and information, and for coordinating and pooling available resources for NFE programs.

LA, Schedule 6, para. 17

Partially complied with. While the project was able to develop the BMES, development was quite delayed and was only finally developed during the middle part of project implementation. Training for the use of the system still needs enhancement.

The development of the manual system for the NFE MIS was also delayed while the computerized NFE MIS was finally developed during the later part of the project and has not been adequately field-tested.

44 Appendix 17

Covenant Reference in

Loan Agreement

Status of Compliance

22. The success of the Project will be measured by the following performance criteria: (i) improvements in simple and functional literacy rates in the targeted communities of the project area; (ii) effectiveness of the subprojects in enhancing the self-help and self-learning capabilities of the participants, as measured by the impact assessment studies to evaluate the effect of FELP on their basic social and economic status; (iii) increase in the participation rate of OSY in alternative learning system programs and their success in attaining secondary equivalency as measured by the ALS testing system; (iv) relevance and quality of locally adapted learning materials developed and produced for FELP, as measured by the demand for the materials; and (v) effectiveness of the institutional and management structure of DECS to support decentralized NFE programs at the local level, including mobilization of resources, as measured by the sustainability of programs initiated under the Project.

LA, Schedule 6, para. 18

Partially complied with.

Appendix 17 45

TECHNICAL ASSISTANCE COMPLETION REPORT

Division: SESS Amount Approved: $1.6 million TA No. and Name

TA 2036-PHI: Capacity Building in Nonformal Education Revised Amount: $1.6 million Executing Agency: Department of Education, Culture and Sports (DECS), now renamed as Department of Education

Source of Funding: Norwegian Government

TA Amount Undisbursed $36,079.50

TA Amount Utilized $1,563,920.50

Date TA Completion Date Original: 31 July 2001 Actual: 31 December 2002

Closing Date Approval 23 December 1993

Signing 01 March 1994

Fielding of Consultants 05 September 1994

Original: 30 June 1999 Actual: 31 March 2002

Description

Loan No. 1254-PHI: Nonformal Education Project (PNFEP) was approved on 30 September 1993, and became effective on 25 July 1994. DECS15 was the Executing Agency. The goal of the PNFEP was to contribute to the poverty reduction program of the Government through nonformal education. The PNFEP included three inter-related components: Functional Education and Literacy Program (FELP), Continuing Education Program (CEP), and Capacity-Building Program (CBP). This advisory technical assistance (TA) was processed in conjunction with the PNFEP.

Objectives and Scope

The main objectives of the TA, as an integral part of the PNFEP, were (i) to strengthen the institutional and managerial capacities of the Bureau of Nonformal Education and DECS/NFE units at the regional and divisional levels to undertake the FELP and CEP; (ii) to support the development and implementation of the FELP, CEP, and CBP components of the PNFEP;16 and (iii) to assist DECS in establishing the NFE Coordinating Committees at the regional, provincial, and municipal levels.

Evaluation of Inputs

The TA was financed by a grant of $1.6 million from the Norwegian Government and managed by ADB. A consortium of consulting firms headed by World Education Inc. was contracted by ADB using the international bidding process. The consortium included two local firms—Philippine Business for Social Progress and the Southeast Asia Consultants Inc. The original TA comprised 34 person-months of international consultants and 186 person-months of domestic consultants over a period of 5 years from 1994 to 1998. At the TA design stage, assessment of the absorptive and implementation capacity of BNFE to deal with the complexity of the TA activities was less than adequate. Due to its lack of experience and management capacity, BNFE was unable to appropriately supervise the work of a large number of consultants during the first 3 years, which resulted in compartmentalization of the TA inputs and a reduction of its impact as a capability building intervention. Another major problem of BNFE/DECS was the frequent change in the positions of PNFEP/BNFE director and counterpart staff. This had a negative impact on continuity and on TA performance. However, toward the later years of TA implementation, BNFE managed the TA more effectively with a stronger leadership.

During the PNFEP’s midterm review in 1997 and subsequent review missions,17 adjustments were made in the TA’s components, implementation arrangements and strategies to better align with the requirements of BNFE. Detailed terms of reference and action plans for each consultant were developed, BNFE counterpart staff were identified and their roles clarified, and mechanisms for more regular and effective reporting were instituted. Under the reconfigured TA, the performance of the international consultants for accreditation and equivalency system and FELP was satisfactory. The CBP consultant was able to assist BNFE develop training designs for various training workshops.

Prior to the midterm review, efforts to field domestic consultants at the division and district levels were unsuccessful because of lack of interest and support from the DECS’ NFE division and its regional offices. This resulted in the local capacity building sub-component of the TA not being realized fully and effectively. To redress the difficulties experienced, the BNFE decided with ADB’s endorsement to terminate the services of domestic consulting firms and contracted the services of individual domestic consultants for (i) advocacy and social mobilization, (ii) NFE A&E system, and (iii) the FELP manual of operations. BNFE also strengthened its supervision of the consultants’ work. Consequently, the performance of the individual consultants was adequate for producing the defined outputs. The TA was supervised by ADB twice year and the performance of ADB was rated satisfactory. Due to extensions of loan closing date of the PNFEP, the TA completion date was extended until 31 July 2001.

15 Now renamed as the Department of Education (DepEd). 16 A detailed assessment of the contribution to the Project of the TA consultants funded by the Norwegian grant can be found in the

Final TA Report of World Education, submitted to ADB and the BNFE on 18 December 2001. 17 ADB fielded review missions twice a year.

46 Appendix 17

Due to extensions of loan closing date of the PNFEP, the TA completion date was extended until 31 July 2001.

Evaluation of Outputs

The TA established a new NFE model using a decentralized delivery approach with local service providers, which comprise nongovernment organizations, local government units, academic institutions, local schoolteachers, and civic groups. This changed the role of the Government from directly providing NFE services to setting an overall strategic framework and taking responsibility for such oversight and regulating functions as curriculum design and quality assurance. The NFE A&E system became operational in all administrative regions and was successful in terms of flexible program delivery, improved NFE learning methodologies, authentic learner assessment tools, and new experiential training approaches. This constitutes an important contribution of the TA to the NFE system.

Under FELP, the TA supported development of a new model and its operationalization for the decentralized delivery of literacy programs through the Literacy Service Contracting Scheme. The Literacy Service Contracting Scheme has been institutionalized by the PNFEP. Under CEP, key elements of a NFE A&E system were developed including frameworks for policy formulation and curriculum development, learning materials, learning support delivery system , and assessment and certification mechanisms covering elementary and secondary levels. The learning modules developed are of high quality and are being adopted in non-PNFEP sites funded by local government units, and other international aid agencies.

Under CBP, after a series of difficulties, the TA was instrumental in strengthening the capacity of BNFE. It completed the staff development activities and training workshops. Unfortunately, the training handbooks remained unfinished and a number of learning models were completed too late for utilization within the Project’s life cycle. The performance of domestic and international consultants responsible for the management information system (MIS), and monitoring and evaluation was less than satisfactory. They were unable to develop an effective monitoring and evaluation system and the NFE MIS.

Overall Assessment and Rating

The TA was relevant to the Government’s poverty reduction program through support for nonformal education. Its design was too ambitious because the TA’s components, expected outputs , planned activities, and the range of consultants’ inputs were too complex to be implemented during a 5-year period.

While the TA accomplished most of its activities, it is rated as partly successful in view of (i) design flaws, (ii) inadequate institutional capacity assessment resulting in ineffective implementation of the TA activities, (iii) nondelivery of some TA outputs, (iv) some TA outputs being completed too late for utilization during the TA’s implementation period, and (v) reduced impact of the capacity-building initiatives.

Major Lessons Learned

The TA design was innovative but highly complex with too many activities, which was beyond BNFE’s capacity to handle. Future innovative TAs need first to be pilot-tested before being implemented nationwide. Adequate up-front analysis of the institutional capacity of the executing agency is critical to the appropriate design of ADB-supported TA and the successful achievement of the TA objectives and impact. Decisive actions should be taken to redress poor performance of the consultants during implementation. To use TA resources effectively, the Government should have adequate, up-front orientation/socializing initiatives for all parties involved in TA implementation to clearly define and clarify their respective roles and responsibilities.

Recommendations and Follow-Up Actions

The TA is a good model for a decentralized NFE delivery system. The TA final report analyzed major issues and provided elaborate recommendations for further strengthening NFE in the country. Some of the recommendations have been mainstreamed into the regular BNFE activities. To further improve the quality and impact of NFE, BNFE needs to vigorously review and adopt the remaining recommendations, including advocacy for public support and adequate budgetary allocation. ADB and the Government should actively explore the feasibility of a follow-up ADB-assisted NFE II Project to consolidate and build on the gains from the PNFEP and the TA.

Prepared by:

Hai Yan Zhai

Designation:

Senior Project Specialist, SESS

Appendix 18 47

ASSESSMENT OF OVERALL PROJECT PERFORMANCE

Criterion Weight Assessment Rating Weighted (a) (b) (c) Value Rating

(d) (b x d) 1. Relevance 20% Highly Relevant 3

Relevant 2 Partly Relevant 1 Irrelevant 0 0.40

2. Efficacy 25% Highly Efficacious 3 Efficacious 2 Less Efficacious 1 Inefficacious 0 0.25

3. Efficiency 20% Highly Efficient 3 Efficient 2 Less Efficient 1 Inefficient 0 0.20

4. Sustainability 20% Most Likely 3 Likely 2 Less Likely 1 Unlikely 0 0.20

5. Institutional Development 15% Substantial 3 and Other Impacts Moderate 2

Little 1 Negligible 0 0.30

Overall Ratinga 1.35 a Sum of the weighted ratings. Note: Highly Successful (HS): Overall weighted average (OWA) is >2.5 and none of the 5 criteria has a score of less than 2; otherwise, the rating would be downgraded by one level. Successful (S): OWA is between 1.6 = S = 2.5 and none of the 5 criteria has a score of less than 1; otherwise the rating would be downgraded by one level. Partly Successful (PS): OWA is between 0.6 = LS = 1.6 and number of criteria receiving a rating of less than 1 should not exceed 2; otherwise the lowest rating would be given. Unsuccessful (U): OWA is < 0.6.