Document of

The World Bank

FOR OFFICIAL USE ONLY

Report No. 15807

IHPLEMENTATION COMPLETION REPORT

PORTUGAL

TECHNOLOGY EDUCATION PROJECT(LOAN 286 7-PO)

June 19, 1996

Human Resources Sector Operations DivisionCentral and Southern Europe DepartmentsEurope and Central Asia Region

This document has a restricted distribution and may be used by recipients only in the performance oftheir official duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EOUIVALENTS

APPRAISAL (1986) US$1 = 150 EscudoCOMPLETION (1994) US$1 147 Escudo

WEIGHTS AND MEASURES

Metric System

FISCAL YEAR OF BORROWER

January I - December 31

ABBREVIATIONS AND ACRONYMS

EU European UnionGOP Government of PortugalMOE Ministry of Education

FOR OFFICIAL USE ONLY

Table of Contents

Preface

Evaluation Summary ............. i

Part I: Implementation Assessment from the Bank's Perspective . .A. Introduction .B. Summary and Evaluation of Project Objectives. 2C. Implementation Experience. 3D. Cancellation of Funds .4E. Project Outcome and Sustainability. 5F. Bank Performance .5G. Borrower Performance. 6H. Summary of Key Lessons Learned. 6

Part II: Statistical Annex. 8

Appendices:

A. Letter dated March 31, 1995, to the Government of Portugal regarding ICR preparationB. Borrower's Project Summary

Tis document has a restricted distribution and may be used by recipients only in the performancc of theiroffcil duties. Its contents may not otherwise be disclosed wiLhout World Bank authori=tion.

IMPLEMENTATION COMPLETION REPORT

PORTUGAL

TECHNOLOGY EDUCATION PROJECT(Loan 2867-PO)

Preface

This is the Implementation Completion Report (ICR) for the Technology Education Project inPortugal for which Loan 2867-PO in the amount of US$32.0 million equivalent was approved on July 21,1987, and made effective on December 8, 1987.

The loan was closed on March 31, 1995, the original closing date. A total of US$7,010,901.05million was disbursed, and US$24,989,098.95 cancelled (of which over US$23 million were cancelledin 1991 at the request of the Borrower). The last disbursement was paid on December 15, 1993, and allsubsequent applications received through May 1994 were applied to recovery of the undocumented specialaccount allocation. The final transaction under the loan--refund of the balance of US$321,445.61remaining in the special account--took place on February 2, 1995, and the balance of US$1,289,098.95was cancelled at closing.

The ICR was prepared by EC1/2HR staff. It was reviewed by Mr. Brad Herbert, PortfolioManager/Sr. Operations Officer (EC1/2HR), and Mr. George Zaidan, Projects Advisor (EC1DR). Mr.J. Paviuk (LEGEC), and Ms. Bridie Champion (LOAEL) also reviewed the report.

The ICR was prepared on the basis of documents in the project files. Final project costinformation, requested from the Borrower at the time of loan closing, was received on May 6, 1996.The Borrower's implementation summary, received in January 1996, is included as Appendix B to thisdocument.

:

-i-

IMPLEMENTATION COMPLETION REPORT

PORTUGAL

TECHNOLOGY EDUCATION PROJECT(Loan 2867-PO)

Evaluation Summary

Introduction

i. A major objective of the Government of Portugal's economic restructuring and developmentstrategy was to increase the competitiveness of Portuguese industry in the world market, and particularlyin the European Union (EU). The Government of Portugal thus focussed its efforts on strengthening thecountry's export- and technically-oriented light industry, and exploiting its comparative advantages, suchas a lower salary scale than its northern neighbors and its coastal location. To support this strategy, theestablishment of a scientific and technical education and training system in accordance with quantitativeand qualitative standards of other EU countries was accorded high priority. High-level skilled manpowerand structural technological changes to support EU integration were particularly crucial in the agriculturalsector in Portugal.

Proiect Objectives

ii. The Technology Education Project was the last in a series of education projects designed toupgrade and modernize Portugal's manpower development capacity in pursuit of the Government'sdevelopment objectives. This project aimed to improve the quality and relevance of: (i) scientific andtechnological research and training through support to three applied research institutes (in biotechnology,materials science, and water resources); and (ii) agricultural education through support to two agriculturalpolytechnic institutes. The Project also sought to strengthen general education in science andmathematics, and increase the number of graduates qualified for entry into higher levels of science andtechnology studies by supporting 140 secondary and 30 primary schools in the modernization of theircurricula through the use of computers as tools for teaching mathematics, science and language skills,and for increasing overall computer literacy.

iii. At preparation, the project's objectives to strengthen the scientific and technical education systemsfell directly in line with the Government's development strategy of seeking to increase competitivenessin the world market place and within the EU. The Government's commitment to these overalldevelopment objectives as supported by the initial design of the Technology Education Project wasconstant throughout project implementation. It did, however, emerge shortly after project startup thatthe means by which these objectives would be principally achieved--i.e., through a Bank-financed projector with support from other sources--would evolve as the project began.

Implementation Experience and Outcomes

iv. As was the experience under the earlier Manpower Training and Development Project (Loan2456-PO, closed on December 31, 1992; Project Completion Report No. 13137, dated June 13, 1994),access to grant funds through membership in the EU brought about shifts in attention and efforts awayfrom implementation of planned Bank-financed activities, resulting in substantial cancellation of projectfunds. For the most part, Bank funds (about US$6.0 million) were used for the development of two

-ii-

agriculturalpolytechnic institutes (about US$6.0 million). The Borrower's assessment indicates that theseinstitutes did in fact benefit from the support provided through the Bank project toward theirestablishment. Most of the project funds disbursed in support of this component were for civil works andequipment (about US$4.9 million), although some resources (about US$1.1 million) were allocated forconsultants and fellowships. As regards the applied research institutes, very little Bank financing wasallocated toward the development of these institutes (about US$1.0 million, mostly for equipment, withvery limited support to training--some US$0.1 million). There was no disbursement of project funds insupport of the third component focussing on computers in schools. Although these last two componentsappear to have benefitted from funding from other sources, namely the EU, the degree to which theywere successful in meeting their respective objectives of developing a scientific, technological andresearch capacity and improving the quality of general education through increased use of computertechnology cannot be assessed from information available in the project files; and the Borrower'ssummary provides no additional insight on this question.

vi. Overall performance was mixed at the outset. Borrower counterpart financing intended foractivities financed by the Bank project was not forthcoming in a timely manner. In the first supervisionreport for the project, counterpart financing delays were highlighted, and similar concerns were raisedin subsequent supervision reports. (This problem also arose during implementation of previous educationprojects in Portugal.) The components most affected were those supporting the establishment of threeapplied research institutes, and the expansion of computers in schools activities--both of which began toreceive support from alternative sources. (These components were eventually dropped from the project(see below)). Activities in support of the two agricultural polytechnics proceeded somewhat moresatisfactorily, albeit with some substantive concerns in terms of rapid increases in student intake andstaffing number at the Beja institute well above levels initially envisaged, and apparent less than fullapplication of appropriate qualifications (agricultural background and practical experience) for studentsand staff. At Ponte de Lima, implementation took place more along the lines and scope defined in originalproject description in SAR. Teachers were selected for their practical farming experience rather thanacademic qualifications; and a high proportion of student intake had farming backgrounds.

vii. Activities in support of computers in schools never entered the active implementation phase. NoBank funds were disbursed against this component. At the research institutes, some activities began withEU financing, while those slated for Bank support did not, given lagging counterpart funds. InNovember 1990, a Bank mission noted that continued support to this component was increasingly difficultto justify, given the low level of commitment of the Borrower through lack of counterpart support.

viii. As of May 1991, only about US$5 million had been disbursed (38% of SAR estimates).Performance was downgraded to an Overall Status of "3". By October, the Borrower formally requestedcancellation of a large portion of loan proceeds including remaining funds earmarked for the three appliedresearch institutes, as well as the computers in schools component, since alternative financing wasavailable. The total cancellation from the Bank loan at that time amounted to some US$23.7 millionequivalent. The only remaining component in the project was that supporting the agriculturalpolytechnics at Beja and Ponte de Lima.

ix. Subsequent to cancellation, the volume of activities under the project was very small. Betweenspring of 1992 and 1993, some $200,000 were disbursed from the loan. Under these circumstances,judging that the much-reduced project was substantially completed, the Bank proposed to the Borrower(as early as the supervision mission in March 1993) to advance the closing date to March 31, 1994 (fromMarch 31, 1995). As there were a number of outstanding applications, however, early closure was notaccepted by the Borrower. The last disbursement was paid in December 1993, and all subsequentapplications received through May 1994 were applied to recovery of the undocumented special account

-iii-

allocation. Upon return to the Bank of over US$300,000 in nondocumented funds from the specialaccount, the loan was closed as of the original closing date of March 31, 1995, and the balance ofUS$1,289,098.05 was cancelled.

x. The assessment of project performance drawn from this review of events as chronicled in theproject files would indicate an overall unsatisfactory project outcome. The project was, with substantialdelays, finally reduced in size and scope and its impact significantly curtailed vis-a-vis initial objectives.At the same time, availability of support from alternative sources like the EU suggests improved prospectsfor sustainability of the investments to which the project committed some US$7 million. Indeed,indications support the Government's general commitment to the overall sector orientation as defined inthe developmental objectives of the Technology Education Project, despite the shift in the primary sourceof support which served to finance activities to meet those objectives.

Summarv of Performance and Lessons Learned

xi. The Bank's assessment of the potential for success of this project was overly optimistic, failingto pay adequate attention to the real risks that had emerged under other projects in the sector. Previousproblems, already well-established, in obtaining timely and sufficient counterpart funding were notdirectly addressed and no recourse taken on the part of the Bank to suspend project support until aneffective remedy could be found. Adequate attention was not paid to quickly emerging alternativefinancing sources, and the resultant strain on the limited capacity at the local implementation level. Giventhe great potential for such problems, the stated means to mitigate those risks in the SAR, primarilythrough various assurances on the part of the Borrower, were inadequate.

xii. The Borrower's performance, based on the review of project files, was cooperative at all stagesof preparation and implementation, and initially satisfactory in pursuit of project objectives. At the sametime, a more decisive handling of the conflicting priorities which began to emerge soon after projectstartup would have been desirable. The lack of sufficient and timely counterpart financing posedsignificant implementation obstacles to activities envisaged under the Bank project from the outset. Suchdifficulties were not new, as they had appeared under other projects, along with the influx of EU grantfinancing and the accompanying conflicts in priorities and limited implementation and managerialcapacity.

xiii. Portugal graduated from the Bank in 1989. Nevertheless, several lessons can be derived fromthis experience of this last in a series of education projects: First, it is critical to candidly assess,throughout implementation, Government's commitment and capacity to implementing the components ofa given project, and take heed to the emergence of alternative sources of financing which may be moreadvantageous, and which may strain the Borrower's implementation and managerial capacity. Part of thiscommitment is demonstrated through provision of timely and sufficient counterpart financing: if notforthcoming, definitive actions should be taken to quickly remedy the situation. Second, non-perforningcomponents should be quickly and effectively dealt with early on, with action taken to restructure or todrop them after giving reasonable opportunity to progress. Third, those responsible for monitoringimplementation must be more alert, and thus better able to react appropriately, to the emergence ofconflicting priorities as public budgets evolve and demands shift. Finally, when having the benefit ofinsight from multiple projects in a given sector, better efforts should be made to directly and openly takeinto account lessons learned from those earlier experiences.

IMPLEMENTATION COMPLETION REPORT

PORTUGAL

TECHNOLOGY EDUCATION PROJECT(Loan 2867-PO)

Part I: Implementation Assessment from the Bank's PersRective

A. Introduction

1. Prior to the Revolution in 1974, Portugal's economy was dependent on its overseas possessions,and its education and training system was largely geared to their requirements. This colonial period wascharacterized by a large expatriate work force and underdeveloped industrial and agricultural sectors.The southern part of the country was the site of large estates, often owned by absentee landlords, whichemployed poorly skilled peasant farm workers, while in the north, economic activity was largely basedon small subsistence farms and large vineyards. Participation in education and training was low, withmost children attending school for only four years. Students continuing on to higher education typicallychose fields oriented toward the administrative, economic and political management needs of the colonies,and few pursued scientific and technological streams of study. In agriculture, traditional practicesprevailed, generating little recognition of any need for technical skills.

2. After 1974, most enterprises were nationalized, and in agriculture, many large estates weredivided into cooperatives, collective farms, and small holdings, the operation of which was seriouslyhandicapped by the lack of agricultural technicians, extension workers, and farm managers. By 1977,Portugal experienced a sharp deterioration in the balance of payments, and economic performancecontinued to decline into the early 1980s as most areas were affected by accelerated inflation rates,declining levels of migrant worker remittances, dependence on imported oil and food, and slowed growthin manufacturing and productive services. The stabilization program undertaken by the Government in1983, however, reversed the balance of payments situation, which went from a record deficit of US$3.2billion in 1982 to a surplus of US$400 million in 1985.

3. The education sector witnessed a noticeable increase in enrollments in primary education afterthe Revolution, while formerly parallel academic and technical secondary streams were merged into asingle, comprehensive secondary education program. Although viewed as a means to diminish theperceived societal division represented by the two parallel secondary education streams, the merger tookplace at the expense of technical secondary and agricultural schools -- the only source of skilled labor andvalued highly by employers. As the number of secondary school graduates increased and universityadmission standards were lowered, enormous pressure was placed on university infrastructure andteaching staff resulting in a decrease in education quality, and overemphasis on theoretical, rather thanpractical, education. The programs provided were outdated and insufficiently oriented to the country'seconomic needs.

4. Given that a major objective of the Government's economic restructuring and developmentstrategy was to increase the competitiveness of Portuguese industry in the world market, and particularlyin the European Union (EU)'/, the Government of Portugal (GOP) focussed its efforts on strengtheningthe country's export-oriented industry, and exploiting its comparative advantages, such as a lower salaryscale than its northern neighbors, and its coastal location. The GOP's development strategy included theexpansion of investment in technically-oriented light industry. To support this strategy, the establishment

1/ Portugal became a member of the EU in 1986.

-2-

of a scientific and technical education and training system in accordance with quantitative and qualitativestandards of other EU countries was accorded high priority. High-level skilled manpower and structuraltechnological changes to support EU integration were particularly crucial in the agricultural sector inPortugal. The potential impact on this sector of the country's entry into the EU were great, and majorchanges were required in agricultural practices not only to produce more food for the home market andthus reduce imports, but also to exploit Portugal's fertile land and mild climate. In addition to thegradual reduction of support prices and elimination of subsidies, the development of appropriately trainedmanpower, technologically advanced agricultural practices, and management and marketing skills forprivate farms became primary objectives.

B. Summary and Evaluation of Project Objectives

5. The Technology Education Project aimed to improve the quality and relevance of: (i) scientificand technological research and training through support to three applied research institutes (inbiotechnology, materials science, and water resources); and (ii) agricultural education through supportto two agricultural polytechnic institutes. The Project also sought to strengthen general education inscience and mathemnatics, and increase the number of graduates qualified for entry into higher levels ofscience and technology studies by supporting 140 secondary and 30 primary schools in the modernizationof their curricula through the use of computers as tools for teaching mathematics, science and languageskills, and for increasing overall computer literacy.

6. At the time of project preparation, the project's objectives as outlined in the Staff AppraisalReport fell directly in line with the GOP's development strategy of seeking to increase competitivenessin the world market place and within the EU through strengthening the scientific and technical educationand training systems. Previous vocational training was not specifically geared toward producing a skilledwork force necessary for an industrial economy, and it became clear that manpower development waskey to reaching the country's development potential. The Government's commitment to these overalldevelopment objectives as supported by the initial project design of the Technology Education Project wasconstant throughout project implementation. It did, however, emerge shortly after project startup thatthe means by which these objectives would be principally achieved--i.e., through a Bank-financed projector with support from other sources--would change. As was the experience under the Manpower Trainingand Development Project (Loan 2456-PO, closed on December 31, 1992), access to grant funds throughmembership in the EU brought about shifts in attention and efforts away from implementation of initiallyenvisaged Bank-financed activities. With limited implementation capacity available to be distributedamong various initiatives, the redirection in priorities not surprisingly focussed on obtaining and utilizinggrant funding at the expense of activities financed with borrowed funds. Eventually, significantcancellations under the Bank project (discussed further in the paragraphs below) would be requested.

7. Thus, although the overall developmental orientation supported through the objectives of theTechnology Education Project appears to have been maintained, the extent to which Bank-financedactivities furthered the attainment of such objectives was significantly reduced. For the most part, Bankfunds were used for the development of two agricultural polytechnic institutes (about US$6.0 million),and the Borrower's assessment indicates that these institutes did in fact benefit from the support providedthrough the Bank project toward their establishment. Most of the project funds disbursed in support ofthis component were for civil works and equipment (about US$4.9 million), although some resources(about US$1.2 million) were allocated for consultants and fellowships. As regards the applied researchinstitutes, very little Bank financing was allocated through the project toward the development of theseinstitutes (about US$1.0 million, mostly for equipment with limited support to training--some US$0.1million), and there was no disbursement of project funds in support of the third component focussing oncomputers in schools. Although these components did benefit from funding from other sources, namely

-3-

the EU, the degree to which they were successful in meeting their respective objectives of developing ascientific, technological and research capacity and improving the quality of general education throughincreased use of computer technology cannot be assessed from information available in the project files;and the Borrower's summary provides no additional insight on this question.

8. As noted above, given the development context of Portugal at the time of project preparation, theproject's objectives appeared relevant and in line with the development and investment priorities of theGovernment. The project's design included loan covenants intended to promote achievement of objectivesand their sustainability, including: the establishment of representative boards to strengthen collaborationbetween industry and university research, and the selection of teachers and students at the agriculturalpolytechnics with appropriate practical experience and relevant backgrounds. However, with thesignificant reduction in project scope in October 1991, limiting Bank support to one of the initial threecomponents, the initial covenants cannot be said to have contributed in a meaningful way to therealization of project objectives. Furthermore, financial reporting covenants were consistently leftpending throughout the project period despite constant reminders from Bank staff. Many audit reportswere received well after the due dates, if at all, with no repercussions on the Borrower.

C. Implementation Experience

9. At startup, implementation performance as recorded in the first supervision report of May 1988was assessed as satisfactory. Architectural designs, equipment lists and other elements of the physicalexecution of the project progressed. Startup of some activities progressed more quickly as a result ofalready available, parallel EU grant funds--for example, establishment of the biotechnology researchinstitute. Actual disbursement of loan funds was initially slow as a result of delays in opening the specialaccount attributed to communication problems between the Ministry of Education (MOE) and the Ministryof Finance.

10. Performance became mixed relatively early on. As already mentioned, this evolution inimplementation appears, to a large degree, to have been the result of shifting priorities on the part of theBorrower and reorientation of financial and other resources among activities as availability of fundingsupport from other sources, namely from the EU, increased. Borrower counterpart financing intendedfor activities financed by the Bank project was not forthcoming in a timely manner. In the firstsupervision report for the project, counterpart financing delays for Bank-financed project activities werehighlighted as a potentially major impediment to implementation of the project as envisaged. Similarconcerns were also raised in subsequent supervision reports. It should be noted that insufficient and/ordelayed release of counterpart funds was not unknown to the Bank in implementation of previouseducation projects in Portugal (see Manpower Training and Development Project, Project CompletionReport No. 13137, dated June 13, 1994).

11. The components most affected by lagging counterpart support were those supporting theestablishment of three applied research institutes, and the expansion of computers in schools activities.These components would eventually be dropped from the project and remaining funds cancelled (seebelow). Activities in support of the two agricultural polytechnics proceeded somewhat moresatisfactorily, these components benefitting from more consistent counterpart funding support tocomplement Bank financing. There remained substantive concerns, however, in terms of thesustainability of investments, particularly the Beja school. In Beja, student intake quickly leapt ahead ofthat foresee in the SAR: from a first year intake of 26 in 1987 to 723 in May 1992 (SAR estimate 200),and 910 in March 1993. Rapid increases in student numbers, as well as in staff at the school, failed totake into account the necessity to judiciously select pupils and teachers on the basis of their agriculturalbackground and practical experience. Furthermore, instruction was characterized by excessive contact

-4-

hours: 32 hours per week versus the norm of 18-20. With the overly ambitious intake, Bank staffexpressed concern that the MOE would have trouble achieving the objective of producing a limitednumber of farm technicians -- this objective being jeopardized by overexpansion and the increasinglytheoretical approach of instruction at the school. No information is available in the project file regardingthe current status of the school and whether or not expansion was eventually brought back in line withinitial targets.

12. At Ponte de Lima, implementation took place more along the lines and scope defined in originalproject description in SAR. Teachers were selected for their practical farming experience rather thanacademic qualifications; and a high proportion of student intake had farming backgrounds. The lastavailable information recorded intake at around 126, with contact hours staff limited to 20 hours perweek.

13. With respect to the project's envisaged activities in support of computers in schools (acomplementary activity to the Government's ongoing MINERVA project--see project files for moreinformation), problems emerged early on. The Annual Review of Implementation Status, dated August1990 noted: "The computers in schools component is virtually not being implemented, managementarrangements for this component have changed, and investment budget is insufficient." Moreover, theproject files also refer to disagreements among various parties over the scope of Bank-financed effortsto support this initiative. As noted above, the development of the applied research institutes was blockedby counterpart financing issues from the outset. Some activities, such as construction and equipmentprocurement, were carried out with EU financing (at least some channelled through the Ministry ofAgriculture), while intended complementary Bank-financed activities (both physical elements as well asenvisaged technical assistance and training) suffered major delays. Some support to the development ofthe research institutes was eventually obtained by the GOP from the National Institute of ScientificResearch and the National Board for Scientific and Technical Research, however, such support wasinadequate given the level of requirements foreseen under the project.

14. In November 1990, a Bank mission noted that continued support to this component wasincreasingly difficult to justify, given the apparent low level of commitment demonstrated by theBorrower through lack of forthcoming counterpart support. The mission concluded that development ofthese research institutes must no longer be among the investment priorities of the Government, at leastnot among those slated for support with Bank funding. The Bank also reminded the Borrower of theunnecessary burden of commitment fees on loan funds which appe;.red to be in excess of what theyintended to use. As a last attempt to find funding from other sources, the Bank wrote to the Secretaryof State for Science and Technology in May 1991 requesting that necessary funds be committed in orderthat the research institutes become viable and sustainable. Funds were not forthcoming, thus cancellationwas recommended.

D. Cancellation of Funds

15. Although a suggestion to consider cancellation of loan funds was first introduced in a letter to theBorrower following the November 1990 mission, it was not until the supervision report of May 1991 thatmore definitive action was taken. At that time, some US$5 million had been disbursed (38% of SARestimates, and over 70% of final total disbursements). Performance was downgraded to an Overall Statusof "3". The mission's findings clearly highlighted the Borrower's failure to make the research institutesa funding priority, at least a priority for Bank funds. In October 1991, MOE requested cancellation ofa large portion of loan proceeds including funds earmarked for the three applied research institutes, aswell as the computers in schools component. Indications from the Borrower were such that theseactivities would instead be supported through funding from the EU. The total cancellation from the Bank

-5-

loan at that time amounted to some US$23.7 million equivalent. The Bank agreed to this request forcancellation on October 29, 1991. Afterward, the only remaining component was that supporting theagricultural polytechnics at Beja and Ponte de Lima.

16. As a result of the substantial cancellation, the volume of activities during the two years was verysmall. Between spring of 1992 and 1993, some $200,000 were disbursed from the loan. Given that themuch-reduced project was for all intents and purposes completed, the Bank proposed to the Borrower (asearly as the supervision mission in March 1993) to advance the project's closing date by one year toMarch 31, 1994 (from March 31, 1995). Initially this suggestion was refused by Borrower, citing theneed for more time in order to complete certain civil works underway at the agricultural institutes. Itwould emerge that part of these works, at Beja, had been undertaken without the Bank's prior review ofbidding documentation, the documentation having failed to be transmitted to the Bank as a result of"administrative error". A subsequent request for retroactive approval of contracts amounting to US$6.23million, for which Bank financing of 40% of that amount was requested, was refused - in part given thatonly US$1.57 million remained of the loan balance. The last disbursement was paid in December 1993,and all subsequent applications received through May 1994 were applied to recovery of the undocumentedspecial account allocation.

17. Recovery of the special account took quite some time in the end. Despite frequentcommunications between the Bank and the Borrower, both from the operations and loan departmentsconcerned, those funds were not returned to the Bank until February 1995. Thus the loan was closed asof the original closing date of March 31, 1995, and the balance of US$1,289,098.05 was cancelled at thattime.

E. Project Outcome and Sustainability

18. The assessment of project performance drawn from this review of events as chronicled in theproject files indicates an overall unsatisfactory project outcome. The project was, with substantial delays,reduced in size and scope and its impact significantly curtailed vis-a-vis initial objectives. At the sametime, however, the availability of support from alternative sources, namely the EU, would indicateimproved prospects for sustainability of the investments to which the project committed some US$7million. All indications appear to support the Government's general commitment to the overall sectororientation as defined in the developmental objectives of the Technology Education Project, despite theshift in the primary source of support which served to finance activities to meet those objectives.

F. Bank Performance

19. Overall, it would appear that the Bank's assessment of the potential for success of this project wasoverly optimistic, failing to pay adequate attention to the real risks that had emerged under other projectsunder implementation during the same period (particularly the Manpower Training and DevelopmentProject (Loan 2456-PO; ICR No. 13137)). Past problems, already well-established, in obtaining timelyand sufficient counterpart funding were not directly addressed and no recourse taken on the part of theBank to suspend project support until an effective remedy could be found. Furthermore, it would appearthat adequate attention was not paid to quickly emerging alternative financing sources, and the resultantstrain on the limited capacity at the local implementation level. Again, these were issues which had alsocome to the forefront of implementation under earlier projects in the 3ector, most recently the ManpowerTraining and Development. Other project risks outlined in the SAR, such as the potential for theexcessively academic orientation of training at the agricultural institutes, the incongruence betweenresearch orientation of the applied research institutes and industry, and the failure to integrate computereducation into active curricula, all emerged during project implementation to some degree. Given the

-6-

great potential for such problems, the stated means to mitigate those risks in the SAR, primarily throughvarious assurances on the part of the Borrower, were inadequate.

20. Throughout implementation, it would appear that the Bank was accommnodating and available,and responsive to the changing project environment. The Bank did take the initiative in suggesting to theBorrower cancellation of funds allocated to the research institutes in an effort to reduce the cost ofborrowing to the client. Nevertheless, the hard evidence of commitment to closely monitoring projectprogress on the Bank's side through concentrated supervision efforts is sketchy at best: supervisionmissions occurred with decreasing frequency during implementation, were for the most part carried outby the task manager alone, with only a single mission taking place in each year of 1991, 1992 and 1993despite very poor project ratings. Devoting such minimal resources and attention to implementationmonitoring while well aware of the already known risks under other project, and obviously aware of thoseissues which emerged during implementation, would suggest a very low priority level accorded to theoperation by the Bank and an almost laissez-faire approach to supervision. At the same time, it shouldalso be noted that throughout the final implementation years of this and previous education projects inPortugal, it became increasingly clear that the EU was recognizably the dominant investment partner forPortugal. Under those circumstances, the Bank would have done well to act more decisively at a muchearlier stage in implementation in moving to limit itself to a more realistic set of agreed activities andtargeted objectives.

G. Borrower Performance

21. The Borrower's performance, based on the review of project files, was cooperative at all stagesof preparation and implementation, and initially satisfactory in pursuit of project objectives. At the sametime, a more decisive handling of the conflicting priorities which began to emerge soon after projectstartup would have been desirable. The project files indicate a willingness to collaborate with the Bankon the part of component managers, although often overtaken by conflicting priorities as EU fundingbecame more available and priorities shifted in response.

22. At the same time, the lack of sufficient and timely counterpart financing posed significantimplementation obstacles to activities envisaged under the Bank project from the outset. Such difficultieswere not new, as they had appeared under other projects, largely a result of the influx of EU grantfinancing and the accompanying conflicts in priorities and limited implementation and managerialcapacity. A more desirable approach would have been to take specific actions and firm decisions earlyon in recognition that some investments initially planned for under the Bank project would, in fact,benefit more greatly from alternative sources of support, thus reducing repayment commitments to theBank. Furthermore, it would appear that the Borrower met its reporting obligations under the loan withsignificant inconsistency, and with little if no meaningful recourse taken on the part of the Bank.

H. Summary of Key Lessons Learned

23. Portugal graduated from the Bank in 1989. Nevertheless, several lessons can be derived fromthis experience of the last in a series of education projects: First, it is critical to candidly assess,throughout implementation, Government's commitment and capacity to implementing the components ofa given project, and take heed to the emergence of alternative sources of financing which may be moreadvantageous, and which may strain the Borrower's implementation and managerial capacity. Part of thiscommitment is demonstrated through provision of timely and sufficient counterpart financing: if notforthcoming, definitive actions should be taken to quickly remedy the situation. Second, non-performingcomponents should be quickly and effectively dealt with early on, with action taken to restructure or todrop them after giving reasonable opportunity to progress. Third, those responsible for monitoring

-7-

implementation must be more alert, and thus better able to react appropriately, to the emergence ofconflicting priorities as public budgets evolve and demands shift. Finally, when having the benefit ofinsight from multiple projects in a given sector, better efforts should be made to directly and openly takeinto account lessons learned from those earlier experiences.

-8-

Part II: Statistical Annex

Table 1: Summary of AssessmentsTable 2: Related Bank Loans/CreditsTable 3: Project TimetableTable 4: DisbursementsTable 5: Project Costs and FinancingTable 6: Loan CovenantsTable 7: Key Performance IndicatorsTable 8: Bank Resources -- Staff InputsTable 9: Bank Resources -- Missions

-9-

Table 1: Summarv of Assessments

A. Achievement of Objectives Substantial Partial Neelizible N/A

Macroeconomic policies XSector policies XFinancial objectives X

Institutional development XPhysical objectives XPoverty reduction X

Gender concerns XOther social objectives XEnvironmental objectives X

Public sector management XPrivate sector development XOther - Research X

B. Project sustainabilitv Likl Unlikely Uncertainx

C. Bank D erformance Hiehly Satisfactory Deficientsatisfactory X

D. Borrower nerformance Highly Satisfactory Deficientsatisfactory X

E. Assessment of outcome High&v Satisfactory Unsatis-satisfactory factor

x

-10-

Table 2: Related Bank Loans

Loan Title Purpose Year of Statsl ~~~~~~~~~~~~~~~approval

To create a technician 1978 Closed.First Education Project training system and improve(Loan 1559-PO) teacher and management

education. l

To increase the training 1980 Closed.capacity of the Ministries of

Second Education Project Education and Agriculture(Loan 1793-PO) by expanding vocational

training at secondary andhigher education levels.

To expand capacity to train 1984 Closed.Manpower Training and manpower needed toDevelopment Project (Loan reorient the economy2456-PO) towards tourism and other

export industries.

Table 3: Proiect Timetable

Step in Project Cycle Date Planned * Actual Date

Identification ** March 1986

Preparation April 1986

Appraisal September 1986

Negotiations J June 1987

Board presentation _ July 21, 1987

Signing __ November 6, 1987

Effectiveness December 8, 1987

Completion September 30, 1994 September 30, 1994

Closing March 31, 1995 March 31, 1995

* Earlier timetable data not available.** Sector dialogue was well established through earlier investments, facilitating preparation of thisoperation.

-11-

Table 4: Loan Disbursements: Cumulative Estimated and Actual

FY88 FY89 FY90 FY91 FY92 FY93 FY94 FY95

Appraisal estimate 2.0 5.2 9.2 14.2 19.6 24.6 29.0 32.0

Actual 0.1 2.4 3.5 5.7 6.8 * 6.9 7.0

Actual as % of estimate 1 7 11 18 97 * 99 100

Date of Final Disbursement 12/93

* In October 1991, US$23.7 million cancelled.

Table SA: Proiect Cost by Categorv of Expenditure(US$ million, rounded)

Appraisal GOP IBRD LoanCategory Estimate Counterpart *

Civil Works 21.4 4.6 3.1

Equipment 32.5 0.4 2.5

Consultants & 23.2 0.0 1.4Training

Total 77.1 5.0 7.0

* Estimates provided by Government.

Table 5B: Proiect Financing(US$ million, rounded)

Appraisal Estimate Actual

Local Forei2n Total Local Foreign Total

IBRD -- 32.0 32.0 -- 7.0 7.0 *

Government 29.6 15.5 45.1 5.0 ** -- 5.0

Total 29.6 47.5 77.1 5.0 7.0 12.0

* US$23.7 million cancelled in October 1991, reducing the total loan amount to US$8.3 million of whichUS$7.0 million was disbursed and balance of US$1.2 million cancelled at loan closing.** Local/Foreign breakdown of government counterpart not available.

-12-

Table SC: Disbursements by Categorv and by Component (actual) *(US$ million)

Civil Works Equipment Consultants and Totall _______________ _________________ _________________ T raining

Water Resources 0.1 0.07 0.02 0.2Research Institute

Biotechnology 0.0 0.3 0.1 0.4Research Institute

Materials Science 0.0 0.4 0.0 0.4Research Center

Agricultural 0.4 0.8 0.3 1.5Polytechnic atBeja

Agricultural 2.6 1.1 0.8 4.5Polytechnic atPonte de Lima _

Total 3.1 2.5 1.4 7.0

* Numbers are rounded and thus may not total exactly.

-13-

Table 6: Status of Legal Covenants

LA Provi- Not Partial Met

plied ance____________ ____I~~With I_ _ _ _ _

Maintain PCU with experienced Section Xand qualified staff. 3.03

Establish boards for biotechnology Section X Advisory committeesresearch, materials science 3.04 (a) established for 2 of 3research, and water resources institutes.research, with representation fromconcerned industries, public sector,and universities.

Research institutes will carry out Section X Applied for contractsprogram for industry under 3.04 (b) awarded.contracts which provide foradequate cost recovery.

Recruit polytechnic teachers based Section Xon CV analysis and interim 3.05 (a)evaluation, and providesupplemental orientation seminarsto teachers.

Select 30% of polytechnic students Section X Applied at bothfrom applicants with farming 3.05 (b) institutions.backgrounds.

Include secondary school students Section Xwith farming backgrounds in 3.05 (c)recruitment, and provide remedialinstruction for student admittedthrough special entry procedures.

Prepare curricula emphasizing Section Xpractical skills for polytechnics and 3.05 (d)equipment lists for such curricula.

Establish an administrative support Sched.1 Committee and groupgroup reporting to a central (para. established. However,steering committee, for 3(b)) component wasadministration of the computers in cancelled.schools component -- prior todisbursements.

-14-

Carry out interim evaluation of Section Component cancelled.computers in schools program by 3.06 (a)12/31/91, and review results withthe Bank by 6/30/92.

Carry out a summative evaluation Section Component cancelled.of computers in schools program 3.06 (b)by 12/31/94, and review resultswith the Bank by 3/31/95.

Carry out a feasibility study on Section Component cancelled.resupplying computer facilities in 3.07schools through replenishmentfunds and furnish findings tothe Bank.

Furnish to Bank, by 3/31/88, Section Xevidence of allocation of 3.07appropriate land to agriculturalpolytechnic at Beja. l

Have records and accounts audited Section X ECA files' log includesannually in accordance with 4.01 (b) one financial report forappropriate auditing principles and this loan. Status of otherfurnish reports to the Bank 6 audits not clear.months after end of fiscal year.

For SOEs, retain documents for at Section Xleast one year and obtain separate 4.01 (c)auditor's option about theserecords.

-15-

Table 7: Key Performance Indicators

Item Appraisal Target Achieved *

Introduction of computers into 30 schools Component cancelled.primary schools

Introduction of computers into 140 schools Component cancelled.secondary schools

Establishments of applied 3 institutes Component cancelled (however,research institutes institutes were established).

Development and training ofuniversity research staff:- specialist assistance 1,296 months 36 months completed.- fellowships 2,044 months Some fellowships implemented.- training of post-graduateresearch workers 110 workers 20 being trained.

Establishment of agricultural 2 institutes Institutes functioning.polytechnic institutes

Evaluation of initial phase of December 1991 Component cancelled.computers in schools

In-service teacher training in use Some training implemented underof computers in education: other initiatives (MINERVA- specialist assistance 320 months project). Component cancelled.- fellowships 1,852 months

Summative evaluation of December 1994 Component cancelled.computers in schools.

Feasibility study of resupplying December 1994 Component cancelled.computer facilities throughreplenishment funds.

* Last update of indicators took place during supervision of 1992. No further information is available.

-16-

Table 8: Bank Resources -- Staff Inputs

Stage Planned Revised ActualStaffweeks Staffweeks Staffweeks

No data No data 44.5Throughappraisal

Appraisal No data No data 40.0through Board

Supervision * No data No data 33.9

Completion 11.0 11.0 4.0Report

Total Inputs 122.4

* Information on planned allocations available beginning only FY94.

-17-

Table 9: Bank Resources -- Missions

Performance RatingStage of Project Month/ # of Days Skills Types ofCycle Year per- in field repre- Implemen- Develop- Problems

sons sented tation/ ment

Overall ObjectivesStatus

Through 2/85 1 8 " not not not not applicableappraisal specified applicable applicable

Appraisal 3/86 4 15 not not not not applicablethrough Board specified applicable applicable

Supervision 5/88 2 12 21 TM, 1 1 Availability ofCOMED funds(counterpart

& spec. account)

11/88 1 14 3/ TM I 1 Counterpartfunds

7/89 1 14 3/ TM 1 2 Counterpartfunds; TA,training andprocurementprogress;financialperformance.

10/90 2 10 3/ TM, 2 2 CounterpartOPO funds; fmnancial

performance.

4/91 2 10 3/ TM, 3 2 CounterpartOPO funds;

procurement andTA progress.

4/92 1 1031 TM 3 2 Counterpartfunds;procurement andTA progress;audit reporting.

3/93 1 10 TM 3 2 Counterpartfunds;procurement andTA progress;audit reporting.

Completion none

I Combined mission wit supervision of two ongoing projets2/ Joint mission with Second Education Project supervision and Manpower Training and Development Project supervision.3/ Joint mission with Manpower Training and Development Project supervision.

TM: Task Manager/Mission LeaderOPO: Operations Officer COMED: Computer Education Specialist

APPENDIX AThe World Bank 1818 H Street, N.W. i202) 477-1234INTERNATIONAL BANK FOR RECONSTRUCTION AND DEVELOPMENT Washington, D.C. 20433 Cable Address: INTBAFRADINTERNATIONAL DEVELOPMENT ASSOCIATION U.S.A. Cable Addross: INDEVAS

Fax: (202) 477-3372

March 31, 1995

Mr. Rogerio da FonsecaDeputy Director GeneralMinistry of FinanceSecretary of State for TreasuryLisbon, Portugal

Reference: Technologv Education Proiect(Loan 2867-PO)

Dear Mr. da Fonseca:

I am writing in reference to the closing of the above-referenced loan scheduled for today,March 31, 1995. As you may recall from our exchange of correspondence one year ago, agreement hadbeen reached to advance the closing date of this loan to March 31, 1994. However, because of a delayin resolving the issue of an outstanding balance due to the Bank from the initial special account deposit-- the US$321,445.61 still outstanding was returned to the Bank only on February 2, 1995 -- loan closingwas not possible before now. Now that this outstanding issue has been settled, I would like to advise youthat Loan 2867-PO, Technology Education Project, will be closed effective today, March 31, 1995, andthat the undisbursed balance of USSI,289,098.95 will be cancelled.

The next step is the preparation of the Implementation Completion Report (ICR), which willeventually be circulated to the Bank's Board of Executive Directors. This report comprises two chapters:(i) an overview of the project implementation from the Bank's perspective; and (ii) a collection of variousstatistical tables on project costs (projected and actual), timetables for preparation and supervision,internal staff resources allocated and utilized, etc. Annexed to the ICR is an evaluation of projectimplementation prepared by the Borrower. This annex should include an assessment of project executionfrom the Borrower's perspective, project benefits and outcomes, Bank and Borrower performance of theirrespective obligations under the Loan Agreement, and the extent to which the purposes of the loan wereindeed achieved.

We would be grateful if responsible staff could begin preparation of this document as soonas possible in order that it be submitted to the Bank by the end of May 1995. This annex, prepared byyour staff, will complement preparation of the Bank's implementation assessment and accompanyingstatistical tables, on the basis of documents available in the project file here at headquarters; preparationof these chapters is already underway. Later in the spring, we will contact you to propose a short visitby our team -- most likely Mr. Brad Herbert, Portfolio Manager, and Ms. Sabrina Huffman, OperationsAnalyst -- with a view to collecting any outstanding project-related data and to conducting visits of projectsites which may be necessary to complete our assessment of project outcomes.

In addition to your assisting us in the ICR process by preparing the Borrower's assessmentof implementation performance, my staff would greatly appreciate any assistance your team may be ableto provide in the collection and compilation of actual project cost data (both in foreign and local currency)by project year, by category, and by component. Submission of this information to the Bank by the end

RCA 248423* WUI 68145

of May would also be most helpful in finalizing the initial ICR draft. Any followup on outstandingquestions could then be answered during the visit of the Bank's team to Lisbon later in the spring.

We would appreciate your reactions to our request for assistance in preparing the ICR.forthis project. Should your team have any questions with respect to this task, they should not hesistate tocontact Ms. Huffmnan at 202-473-5016 (telephone) or 202-477-0288 (fax).

Sincerely,

/- -

Ralph W. HarbisonDivision Chief

Human Resources Operations DivisionCentral and Southern Europe Departments

Europe and Central Asia Region

Cogy to:

Ms. Ana Luisa Gomes, Ministry of Finance, Lisbon, Portugal(fax 351-1-887-7524)

Mr. Brad Herbert and Ms. Sabrina Huffnan, EC1/2HR

,9' FM,nIatFo d EducAo APPENDIX BOeeflnmnl{eo di ProgagmuCJ°

e Geslio Ftnancera

RECEIVED January 1996

TECHNOLOGY EDUCATION PROJECT

FINAL REPORT

(Cooperation with the IBRD)

The Technology Education Project was developed with the financlalassistance of the World Bank under Loan Agreement Number 2867JPO,signed on November 6, 1987.

The total amount of the loan for the project was Initially U. S.$32,000, 000 (thirty-two million dollars). Owing to difficulties Inthe utilisation of the above mentioned amount, and according to proposallater submitted, that amount was cut down to U. S. S8,3.

The total costs of the project were U. S. $7,010,901.05. The remaining(U. S. $1,289,098.95) was remitted to the World Bank In the presentcivil year. (Of this remaining amount, U.S. $321,445.61 were in theSpecial Account).

OBJECTIVES OF THE PROJECT:

The objectives of the project were:-to expand and to improve the quality of training and of scientific andtechnological research in the areas of Biotechnology, of MaterialsSciences, of Water Resources and of Agricultural Higher Education;-to promote modernisation of educational provision by introducing theuse of computers in basic and secondary schools.

In accordance with those objectives, the project initially included thefollowing components):

-Biotechnology Research Institute of Lisbon (CB)-Materials Sciences Research Institute of Oporto (CM)-Water Research Institute of Coimbra (IA)-Higher Agricultural School of Bela (AB)-Higher Agricultural School of Ponte de Lima (AP)-Program for Information Technology in Schools (IC).

The Higher Agricultural Schools of Beja and Ponte de Lima are stillbenefitting from this loan; contrarywise, the remaining components arereceiving financial assistance from PRODEP and from EEC Fundsnegotiated for research.

The Closing Date was March 31, 1995.

0

,>G -M/itstFiO da Educao000"am.nRo de PfrogrAmao

e Geatdo Finance.ru

The tables In annex show the following: (**)

1 -There are no expenditures for the Program for InformationTechnology In Schools (IC);

2 -The highest expenditures were for the Higher Agricultural School ofPonte de Lima (AP); there follow, In decreasing order, the HigherAgricultural School of Bela (AB), the Biotechnology Research Instituteof Lisbon (CB), the Materials Sciences Research Institute of Oporto(CM) and the Water Research Institute of Coimbra (IA);

3 -For the Higher Agricultural School of Ponte de Lima (AP), thehighest expenditures were In Category I (civil work); Category 11(equipment) ranks second and the lowest expenditure was In CategoryIII (consultants' services).For the Higher Agricultural School ot Beja (AB), the highestexpenditures were in Category I, civil work; Category II( equipment)ranks second but close; Category IlIl (consultants' services) has thelowest expenditure.For the Biotechnology Research Institute of Lisbon (CB), the higherexpenditure was In Category II (equipment), the lower being Categoryill (consultants' services).For the Materials Sciences Research Institute of Oporto (CM)expenditures were only In Category II (equipment) and for the WaterResearch Institute of Coimbra (IA) expenditures were only In CategoryIlIl (consultants' services).

For the Higher Agricultural Schools of Ponte de Lima (AP) and of Beia(AB), expenditures were regular, from 1989 to 1994; expenditures forthe other components were irregular, amounts being allocated at giventimes only.

a) Note to table In page 3, Category III: 1,313,459.04, adding to U.S.$7,004,741.12 the amount of U.S.$6,159.93 from the Special Account.

(**) Available in the project file.

0

TOTAL EXPENDITURES (Expressed In Dollar equivalent) BY COMPONENT,BY CATEGORIES

CATEGORY I a) CIVIL WORK 3,183,651.01

CATEGORY 2 EQUIPMENT 2,507,631.07

CATEGORY 3 CONSULTANTS' SERVICES 1,313,459.04

TOTAL 7.004.741.12

Lisbon, December 28, 1995

Director

(Jose Manuel Prostes da Fonseca)

In annex: 9 tables

0

IMAGINNG

Report No: 15807Type: ICR