ASIAN DEVELOPMENT BANK PPA: INO 15059

PROJECT PERFORMANCE AUDIT REPORT

ON THE

MARINE SCIENCES EDUCATION PROJECT (Loans 894-INO/895-INO[SF])

IN

INDONESIA

July 2000

CURRENCY EQUIVALENTS

Currency Unit – Rupiah (Rp)

At Approval At Completion At Operations Evaluation

Rp1.00 = $0.000607 $0.000425 $0.0002 $1.00 = Rp1,647 Rp2,348 Rp4,185

ABBREVIATIONS

ADB – Asian Development Bank AR – appraisal report BAPPENAS – Badan Perencanaan Pembangunan Nasional (National Development Planning Agency) BME – benefit monitoring and evaluation CPIU – Central Project Implementation Unit DGHE – Directorate General of Higher Education EA – executing agency IPB – Institut Pertanian Bogor (Institute of Agriculture) LPIU – local project implementation unit MSc – master of science MSE – marine sciences education MSEP – Marine Sciences Education Project O&M – operation and maintenance OEM – Operations Evaluation Mission PBME – project benefit monitoring and evaluation PCR – project completion report PhD – doctor of philosophy PPAR – project performance audit report REPELITA IV – Rencana Pembangunan Lima Tahun IV (Fourth Five-Year Development Plan) RRP – report and recommendation of the President S1 – bachelor’s degree S2 – master’s degree S3 – doctoral degree TA – technical assistance UNDIP – University of Diponegoro UNHAS – University of Hasanuddin UNPATTI – University of Pattimura UNRI – University of Riau UNSRAT – University of Sam Ratulangi

NOTES

(i) The fiscal year (FY) of the Government ends on 31 March. (ii) The school year (SY) in Indonesia starts in September and ends

in June of the following year. (iii) In this report, “$” refers to US dollars.

Operations Evaluation Office, PE - 547

CONTENTS

Page BASIC PROJECT DATA ii EXECUTIVE SUMMARY iii MAP v

I. BACKGROUND 1 A. Rationale 1 B. Formulation 1 C. Objectives and Scope at Appraisal 1 D. Financing Arrangements 2 E. Completion 2 F. Operations Evaluation 3

II. IMPLEMENTATION PERFORMANCE 3 A. Design 3 B. Contracting, Construction, and Commissioning 5 C. Organization and Management 5 D. Actual Costs and Financing 6 E. Implementation Schedule 7 F. Technical Assistance 7 G. Compliance with Loan Covenants 7

III. PROJECT RESULTS 7 A. Operational Performance 7 B. Institutional Development 9 C. Financial Performance 10 D. Socioeconomic and Sociocultural Results 10 E. Gender and Development 11 F. Environmental Impacts and Control 11 G. Gestation and Sustainability 11

IV. KEY ISSUES FOR THE FUTURE 12 A. Project Design 12 B. Local vs. Central Ownership 12 C. Program Sustainability 13 D. Resource Utilization 13 E. Management 13 F. National Coordination 14

V. CONCLUSIONS 14 A. Overall Assessment 14 B. Lessons Learned 15 C. Follow-Up Actions 16

APPENDIXES 17

BASIC PROJECT DATA Marine Sciences Education Project (Loans 894-INO and 895-INO[SF])

PROJECT PREPARATION

Person- TA No. TA Name Type Months Amount Approval Date

844-INO Marine Sciences PPTA 17.5 $350,000 12 January 1987 Education As per ADB KEY PROJECT DATA ($ million) Loan Documents Actual Total Project Cost 91.6 75.6 Foreign Currency Cost 54.5 46.8 ADB Loan Amount/Utilization 73.3 66.3 ADB Loan Amount/Cancellation 8.8

KEY DATES Expected Actual

Fact-Finding 9-27 Nov 1987 Appraisal 7-25 Mar 1988 Loan Negotiations 14-22 Jun 1988 Board Approval 14 Jul 1988 Loan Agreement 4 Nov 1988 Loan Effectiveness 2 Feb 1989 24 Feb 1989 Project Completion 30 Sep 1994 30 Nov 1995 Loan Closing 31 Mar 1995 28 Jan 1997 Months (effectiveness to completion) 68 82

BORROWER Republic of Indonesia

EXECUTING AGENCY Directorate General of Higher Education, Ministry of Education and Culture (now Ministry of National Education)

MISSION DATA

Type of Mission Missions (no.) Person-Days (no.) Fact-Finding 1 19 Appraisal 1 18 Loan Inception 1 10 Project Administration - Loan Review 12 74 - Loan Disbursement 1 9 - Project Completion 1 15 Operations Evaluation 1 26 ________________________ ADB = Asian Development Bank, PPTA = project preparatory technical assistance, TA = technical assistance.

EXECUTIVE SUMMARY

Great expectations, mixed outcomes

The shortage of marine scientists and technicians has hindered the full utilization of Indonesia’s ample marine resources. Under the Fourth Five-Year Development Plan special attention was to be given to tertiary level programs in science and technology, particularly in the outer islands. The Marine Sciences Education Project was formulated to diversify the economy away from oil products, improve nutrition, and train professional-level human resources in marine sciences to optimize the utilization of marine resources under sustainable conditions. Strategies to achieve these objectives were to establish a new marine sciences education program, improve academic and other support services, develop research capacity, and upgrade existing facilities and construct new facilities at six universities. Beneficiaries were to be undergraduate students, university personnel, public and private employers of marine science graduates, coastal communities, and the population as a whole. Components included consultant services, fellowships, civil works, equipment, educational media and software, and funds for operation and research. Two loans were provided for a total of $73.3 million on 14 July 1988—$43.2 million from ordinary capital resources and $30.1 million equivalent from Special Funds resources. The Project was completed on 30 November 1995 after a delay of 14 months. The project completion report (PCR), circulated in September 1997, rated the Project as successful. The geographic spread of project universities suited the Government’s thrust to develop the outer islands. Significantly more fellowships and consultant inputs were provided than specified, and research funds were provided in excess of the target. Civil works, furniture, and equipment were provided essentially as designed, but with some buildings being poorly constructed and even dangerous. Vessels and vehicles may have all been procured, but some have been lost to the Project. Operation and maintenance obligations of the Government were not fully met, partly due to the effects of the Asian financial crisis. The Project did not meet expectations in the development of employable human resources. On average, only about half of the graduates have been able to find jobs in marine sciences or related fields; the other half are either unemployed or work in unrelated fields. Research output has been variable, and benefit monitoring and evaluation (BME) deficient. The Operations Evaluation Mission (OEM) had no success at some universities, nor at the Directorate General of Higher Education, in obtaining basic data. Fellows’ feedback indicated overall satisfaction at having obtained an overseas qualification. While they were critical of their inability to participate in equipment ordering and facilities design, they rated highly the level of encouragement from their superiors, the appropriateness of project teaching materials, and improvements in conservation stemming from the Project. They rated poorly the maintenance of equipment and facilities, and cooperation with the private sector. At some universities, the OEM found a high incidence of absence of students, faculty, and administrators, and serious underutilization and deterioration of civil works, equipment, books, and vessels. The worst case was the destruction of one of the six field stations due to civil unrest. Very

iv

little research was ongoing, and libraries were often devoid of users. The OEM did not see much evidence of a fertile academic atmosphere promoting significant research and excellence in teaching. Returned fellows occupied generally the same positions and did basically the same work as before their advanced studies. The few productive links with other institutions are ad hoc. There is no integrative element among project universities. Employment generated by the Project has been marginal, and the hoped-for modernization of marine industries is not occurring. The Project’s effects on gender and development have been minimal. While the PCR made recommendations to ensure project benefits, the OEM could find little evidence that they had been heeded. A key issue stemming from the Project’s design is the efficacy of advanced degree training as preparation for teaching and research management. This issue had been researched and discussed in the literature long beforehand but was not addressed in project formulation. Another issue is centralization versus localization. The universities were not consulted sufficiently during centralized procurement. And the local project implementation units had an anomalous position, reporting to the Central Project Implementation Unit in Jakarta but ostensibly being under their university administrations. The appraisal report gives almost no attention to management issues, which proved to be critical. A positive lesson learned from the Project is the crucial role of selecting, supervising, and preparing fellows for training. This aspect was well implemented.

Failure to comply with several loan covenants, viz., those on midterm review, BME, and private sector involvement, hurt the Project. While its relevance was high and its efficacy satisfactory, in terms of efficiency and sustainability of project benefits and the extent to which its major purposes and goals have been achieved, the OEM has reservations and thus rates the Project as partly successful.

I. BACKGROUND

1. As early as 1984, the Asian Development Bank (ADB) was exploring the possibility of a project in Indonesia for developing higher education in marine sciences for the utilization of marine living resources, aiming at a possible loan in 1985 to strengthen marine sciences education (MSE) at five universities. Both the National Development Planning Agency (BAPPENAS) and the Directorate General of Higher Education (DGHE) were keen to receive technical assistance (TA) for a feasibility study. Concurrently, DGHE and ADB were discussing the Second Agricultural Education Project for construction of agricultural polytechnics. Because both potential projects would be with the same executing agency (EA), for a while they were considered for processing as two components of a single project. The TA Fact-Finding Mission that visited Indonesia in September 1986 for the combined project recommended that the two projects be processed separately because of their focus on two different levels within the higher education system: the university system for marine sciences, and the polytechnic level for agricultural education.

A. Rationale

2. Indonesia has ample marine resources, but the shortage of professional marine scientists and senior technicians had hampered the full utilization of those resources. Before the Project, Indonesian universities produced a maximum of 100 marine science graduates per year (mostly in fisheries), but the estimated total need was for more than 3,000 bachelor’s degree (S1) and 4,000 diploma1 holders. There were only about 25 master’s degree (S2) and 8 doctoral degree (S3) holders in the field in all the universities. No university owned a research or training vessel. Laboratory facilities, equipment, and reference materials were seriously deficient, and research capability was thus very low. To remedy these and other deficiencies, the Government advocated expanding the educational facilities and improving the quality of teaching. The Fourth Five-Year Development Plan (REPELITA IV: 1984/85-1988/89), operative at the time of project formulation, emphasized expansion and improvement of secondary and tertiary vocational and technical education, particularly high level (tertiary) human resources in science and technology. Special attention was to be given to outlying regions. The ultimate goals of the Marine Sciences Education Project (MSEP) were to diversify the economy away from overreliance on traditional commodities (principally oil), improve the nutrition of the large and rapidly increasing population, and train professional-level human resources in marine sciences to optimize the use of marine resources under sustainable conditions.

B. Formulation

3. The project preparatory TA feasibility study was carried out under ADB TA 844-INO in the amount of $350,000, which was approved on 12 January 1987.

C. Objectives and Scope at Appraisal

4. Despite inconsistencies in terminology between the appraisal report (AR) and the report and recommendation of the President (RRP), and even within the AR, regarding project objectives, strategies, and components, the essential objective of the Project was “to assist the

1 A lower qualification than a four-year S1 degree.

2

Government in achieving an optimal utilization of living marine resources in Indonesia under sustained yield conditions.” 2 The strategies to achieve this objective were to (i) establish a new MSE program, (ii) improve academic and other support services, (iii) develop research capacity, and (iv) upgrade existing facilities and construct new ones at six existing universities/institutions (henceforth “universities”). Beneficiaries were envisaged to be about 3,500 existing and new S1 (undergraduate) students; about 215 academic, administrative, and technical staff of the six universities; public and private employers of marine science graduates; coastal communities (because of increased employment opportunities); and the population as a whole (due to additional food supply).

5. The components of the Project were (i) consulting services for academic matters and for architectural and engineering design, (ii) a fellowship and training program for upgrading academic and support staff, (iii) a civil works program for upgrading existing facilities and constructing new ones, (iv) provision of training and research equipment including vessels, (v) provision of educational media and software including libraries and instructional materials, and (vi) funds for operation and research.

D. Financing Arrangements

6. ADB approved two loans on 14 July 1988: 894-INO for $43.2 million from ordinary capital resources, and 895-INO for $30.1 million equivalent from Special Funds resources, for a total of $73.3 million. Of this, $54.1 million was to be used to cover 99 percent of the foreign exchange cost, including $9.5 million for interest and service charges during construction; and $19.2 million equivalent was to cover 52 percent of the local currency cost. ADB financing covered 80 percent of the total project cost of $91.6 million estimated at appraisal. The Borrower was the Government of Indonesia. Prior to these loans, ADB had provided four loans for higher education in Indonesia, disbursing a total of $102.0 million, plus part of the Special Project Implementation Assistance Loan amounting to $8.8 million, and five TAs totaling $1.1 million (all for project preparation). Subsequent to the MSEP loan, ADB has financed three other higher education projects for a total of $303.8 million, plus seven other TAs totaling $3.6 million.

E. Completion

7. Loan closing was on 28 January 1997 compared with the closing date of 31 March 1995 envisaged at appraisal. The project completion report (PCR), prepared by ADB’s Education, Health and Population Division (East) and circulated in September 1997, rated the Project as successful. The PCR is generally well prepared but does not mention any deterioration and underutilization of project facilities, which was observed only two years later by the Operations Evaluation Mission (OEM). Nor did the OEM see evidence of much joint activity with other agencies and the private sector, a problem that had been noted in the PCR. The PCR places responsibility for poor benefit monitoring and evaluation (BME) on the BME consultants, but the OEM believes that there was also a lack of commitment by the Central Project Implementation Unit (CPIU) and local project implementation units (LPIUs) to this component. The PCR also postulates spin-off effects of the MSEP: higher yields of marine resources, their effective management and conservation, concomitant additional revenues, and improved nutrition—none of which are obvious, especially in the absence of effective BME.

2 As cited in the AR’s Project Focus, Design, and Rationale section.

3

F. Operations Evaluation

8. This project performance audit report (PPAR) presents the findings of the OEM during September 1999. The PPAR assesses the Project’s effectiveness in terms of realizing its objectives, generating benefits, and sustaining the benefits. The PPAR is based on a review of the PCR, AR, RRP, and other materials in ADB files; data gathered in the field by the OEM; discussions with staff members of ADB, DGHE, and other agencies of the Borrower; and interviews and focus group discussions with administrators, academic staff, technicians, and students during site visits to five of the six universities (Appendix 1).3 Thirty-four fellows at four universities filled out a questionnaire assessing all aspects of the Project (Appendix 2). Copies of the draft PPAR were provided to the Borrower, the EA, concerned ADB staff, and the six universities for review and comments. Comments received were taken into consideration in finalizing the report.4

II. IMPLEMENTATION PERFORMANCE

A. Design

9. The geographic spread of project universities suited the Government’s thrust to develop the outer islands, especially the eastern ones. Universities were chosen that already had partly developed marine sciences programs and so would have a base to build on. The project components went beyond constructing buildings and providing equipment to embrace reference materials, staff development, funding for research, and BME. During project processing there was worry that the large number of individuals to be served by the overseas fellowship program, identified as the most challenging project component, might not be placed in appropriate master of science (MSc) and doctoral (PhD) programs in other countries; prequalification of fellows was also a potential issue. In fact, more fellows were sent than originally planned, and at lower cost. The problem became one of finding candidates proficient enough in English to send abroad, and in May 1992, DGHE requested for flexibility to send fellows for MSc and PhD programs within Indonesia. Permission was granted in June, which saved even more money. Despite these positive aspects, the OEM has several concerns about project design.

10. The AR, para. 22, states that one of the major issues in the higher education subsector was the need for qualitative improvement in the teaching and learning process. However, the MSEP design did not address the teaching and learning issue. Instead, it assumed that the satisfactory development of an undergraduate MSE program could be based on fellows following formal discipline-based academic study and research to obtain a master’s or doctoral degree. Formal training in teaching was never considered. At the time of appraisal in 1988, this kind of thinking about improving the quality of university education was already outdated, and substantial changes to educational policies and practices had been under way in many countries for at least a decade.5 But critical educational issues in the development and 3 The OEM could not visit the University of Pattimura (UNPATTI) in Ambon because of the security situation there.

UNPATTI’s field station at Hila, Ambon Island, had been destroyed during civil unrest. 4 Copies of the draft PPAR were forwarded to the Borrower, the EA, and the six universities on 19 April 2000 with a

request that comments be provided by 11 May 2000. Although the request was followed up subsequently, no comments were received; the OEM therefore assumes that none of these parties wishes to comment on the PPAR.

5 Between 1970 and 1975, over 1,000 colleges and universities in the United States began some type of program to assist staff to improve their teaching, while similar developments occurred in Europe and Australasia. By the time the MSEP was being designed, considerable literature on teaching and its improvement was available. Examples include E.L. Boyer, College: The Undergraduate Experience in America (New York: Harper and Row, 1987); R.A. Cannon, “The Professional Development of Australian University Teachers” in Higher Education (12.1,1983), 19-33; and K.P. Cross, Accent on Learning (San Francisco: Jossey Bass, 1988).

4

implementation of the MSEP, such as preparing fellows for their professional duties in curriculum development and teaching, were given short shrift. The 24 person-months of planned and 19.5 person-months of actual input from the curriculum coordination and development specialist were not adequate to address the teaching and learning development needs of the large number of geographically dispersed fellows. This issue pervades the Project, the primary strategy of which was establishment of a new MSE program.

11. In a similar way, the design for the management of the marine environment was deficient. There was no overall plan to address this major objective. Only one PhD and three MSc fellowships in environmental management were awarded to fellows employed at only two universities, with the probable assumption that formal higher degree study in technical disciplines somehow engenders better management outcomes. The OEM, however, found scant evidence of improved environmental management. Furthermore, nothing was built into the project design to facilitate interaction among the six universities in this crucial field.

12. Cofinancing was discussed during project processing, but none was found. There was concern that the MSEP might overlap with projects of other aid agencies, but this proved not to be a problem. Nevertheless, attention given at appraisal to the results of other projects was slight. Lessons learned from the Surabaya Institute of Technology Project (Loan 244-INO) included staff development for improving educational quality, and the need to establish links with industry. However, consideration of the full implications of these lessons, and the strategies adopted in the MSEP design, were inadequate to capitalize on past lessons. Lessons from other projects and sectors, especially education projects, could also have been drawn upon.

13. One major change in the appraised design was made early in project implementation: The University of Pattimura (UNPATTI) field station was originally planned to be in the Kai Islands, more than 600 kilometers by sea east of the main UNPATTI campus in Ambon. This was strongly recommended by the TA consultants based on the particularly good marine resources around the Kai Islands. The TA consultants also suggested that 300-ton vessels be provided by the Project. In February 1991, the project director requested permission, granted in March 1991, to relocate the station to Hila, Ambon Island, 40 minutes by car from the main campus. Aside from the obvious time difference (it being 48 hours by sea to the Kai Islands), the reason for the change was that it would take at least a 250-ton vessel to make the trip safely, but only a 15-ton vessel was specified in the final report of the Loan Fact-Finding Mission, despite the TA recommendation. This problem should have been foreseen at appraisal (in 1988).6 Furthermore, during site visits to the project universities it was suggested to the OEM that the project design would have been strengthened had local scientists been more fully involved in the process, particularly aspects related to their own institutions and to local marine environments.

14. The Loan Fact-Finding Mission also recommended “reevaluation”—i.e., reconsideration—of the inclusion of the Institut Pertanian Bogor (IPB) in the Project. However, IPB was retained. Given the low level of utilization of project facilities at IPB (Appendix 1, paras. 1,3; Appendix 3, photo), reconsideration may have been justified, as project objectives appear not to have been served fully at IPB.

6 Ironically, the relocated station at Hila was destroyed in the civil unrest. So, while the relocation made it more

possible to achieve project objectives by reducing travel time and freeing up funds that would have been spent on a large vessel and its operation, project success was diminished because the station was later razed.

5

B. Contracting, Construction, and Commissioning

15. The civil works component of the Project was generally well completed and in compliance with ADB’s Guidelines for Procurement, and many impressive facilities were built. However, one deficiency was the almost universal failure of tiled flooring to withstand use (Appendix 1, paras. 1, 11, 16, 18, 25); that the tiles are all the same indicates central procurement of substandard material. The collapsed roof at the Pelabuhan Ratu Field Station (Appendix 1, para. 3), while not a major problem, may point to poor construction. More seriously, the poor construction and the deterioration of buildings and associated works at the Likupang Field Station are deficiencies that were not identified in the PCR and are not only substandard but dangerous (Appendix 1, para. 29; Appendix 3, photo); while no records were found of any anomalies in the contracting and construction, the state of these buildings suggests poor construction practices. Moreover, some of the fixtures and equipment at the University of Hasanuddin (UNHAS) and University of Sam Ratulangi (UNSRAT) field stations appear to have never been completed or commissioned (Appendix 1, paras. 13, 29).

16. Most of the engineering and academic consultants seem to have done a creditable job. The fellows interviewed (Appendix 2, Table A2.1) gave them their third highest rating (Appendix 2, Table A2.2). The only criticism the OEM heard was that the program for the academic consultants could not be synchronized with the timing of their counterparts’ schedules. While unfortunate, this criticism would have been difficult to alleviate: the fellows left for long-term training, and the consultants had to act on matters such as equipment procurement in their absence.

17. The BME consultants, who were to measure internal and external efficiency, satisfaction of social demand, and cost effectiveness, were not rated as highly, but the problem may have also lain with the failure of the universities to provide the necessary support. The summative tracer study by the Institute of Educational Management Development, Jakarta, collected data only indirectly by sending questionnaires to the six universities; data was returned by only 4 institutions and concerned only 41 MSEP graduates, rendering it of questionable use.

C. Organization and Management

18. The division of labor between the CPIU, which was responsible for site acquisition, employment and supervision of engineering consultants, procurement of equipment and furniture, implementation of the fellowship program, and employment and management of academic consultants, and the LPIUs had advantages and disadvantages. Centralized procurement allowed economies of scale but perpetuated problems as in the purchase of substandard floor tiles (para. 15). The decision-making process during project implementation was two-tiered: projectwide decisions were made by the CPIU; decisions that involved only an individual project institution were made by the LPIU concerned. Most former LPIU and CPIU personnel with whom the OEM talked felt that this system worked reasonably well and were happy to have policy matters decided upon by the CPIU. However, in certain instances, LPIU officials felt they were not listened to by the CPIU, e.g., concerning what books and equipment to order. Some of the equipment did not match local needs, being either too large to be practical (e.g., autoclaves) or incomplete (Appendix 1, para. 18).

19. On more than one occasion during the OEM, former LPIU personnel observed that the LPIU did not fit in well with the university system. The institution’s hierarchy felt short-circuited by the LPIU’s reporting directly to the CPIU in Jakarta. Moreover, there was apparently no effective private sector representation at either the CPIU or LPIU level.

6

20. The large consultant component was for the most part effectively used, and transfer of technology to the generally capable counterparts was effective in many areas. However, little provision was made for transfer of skills in teaching, administration, or resource management (para. 52). And no effective management information system was developed.7

21. Regular ADB supervision through 12 review missions and 1 loan disbursement mission was adequate, but the planned midterm review should not have been canceled (paras. 27 and 32). Loan review missions as late as March 1993 reported that the midterm review was “not yet due.” There was a long break before the next review mission—until March 1994; that mission reported, “As a part of PBME exercise a mid-term evaluation was carried out during November 1993-March 1994 by two PBME consultants.” Apparently both the Government and ADB ignored their covenanted responsibility to hold a midterm review in favor of this report from consultants. As reported in the September 1994 Special Loan Review Mission’s back-to-office report, that midterm evaluation emphasized procurement of equipment and books, training, consultants, and publicity to assist in the employment of MSE graduates. It did not closely scrutinize the project, which should have been done during a midterm review.

D. Actual Costs and Financing

22. The PCR noted “sizeable savings.” This came partly from the allocation for consultants and overseas fellowships, despite the increase in their number. The consultant services component utilized 52 more person-months than envisaged at appraisal (607); both international and domestic consultant services were increased (from 510 person-months to 542 for international, and from 97 to 117 for domestic). The staff development component realized a savings of more than 10 percent of the appraised amount (actual cost of $9.739 million as compared with appraised amount of $10.848 million), but also with increased output; the appraisal called for 94 MSc fellowships, and the PCR says 130 were awarded (an increase of 38 percent), although data given to the OEM indicates that only 110 were awarded. These savings are attributable to overbudgeting for both consulting services and fellowships at appraisal. Fellowships were estimated at high-end United States costs, but most of the fellows went to less expensive training sites, and DGHE was able to negotiate favorable rates for consultants.

23. The largest savings occurred in taxes and duties, which was projected at $10.950 million but actually cost $2.403 million. The Government’s contribution to incremental recurrent costs (operation and maintenance [O&M]) was $2.8 million less than the appraised amount; given the poor state of some of the project assets, this was probably a false economy.

24. Only civil works went significantly over budget, in the amount of more than $2 million, but almost all of this was financed by the Government. The main reason was that the MSE facilities at IPB were integrated into a large, sprawling new building. To harmonize the MSE wings with the rest of the building, it was necessary to add another floor, which was done at government expense.

7 A report prepared for the OEM by UNHAS has three different figures for the same parameter, but at least UNHAS

made a report. Other universities could not readily produce basic data on MSEP enrollments, graduates, fellows, etc. Nor could DGHE produce requested data.

7

E. Implementation Schedule

25. Project effectiveness was delayed only three weeks. Project completion, expected on 30 September 1994, was delayed 14 months until 30 November 1995. Loan closing, expected on 31 March 1995, was delayed 22 months until 28 January 1997. Delays were caused by the Government's requirements for a legal opinion on the Loan Agreement, its tardy appointment of staff and setting up of implementation units, and delayed recruitment of consultants.

F. Technical Assistance

26. The project preparatory TA could have recommended more functional relationships among the CPIU, LPIUs, and university administrations, an area in which some misunderstanding occurred. The basic approach to training faculty to teach and to manage resources by sending them for graduate degrees can also be traced to the TA. Otherwise the TA was well conducted.

G. Compliance with Loan Covenants

27. The poor state of certain project facilities raises the following questions:

(i) Regarding the Loan Agreement for Loan 894-INO, Schedule 6, para. 9(b) on maintenance (PCR, p. 27), although the project schedule was complied with, maintenance has not been sustained. Many project facilities are in dire need of maintenance.

(ii) Regarding the Loan Agreement for Loan 895-INO(SF), Schedule 4, para. 7 on civil works contracts (PCR, p. 22), the poor condition of the UNSRAT field station at Likupang (Appendix 1, para. 29; Appendix 3, photo) raises concern about the prequalification, selection, and supervision of contractors.

(iii) Regarding the Loan Agreement for Loan 894-INO, Schedule 6, para. 8 on the midterm review (PCR, p. 27), the midterm review did not take place despite being specifically covenanted (AR, para. 114[iv]).

28. Otherwise loan covenants were followed.

III. PROJECT RESULTS

A. Operational Performance

29. Some targets identified at appraisal were exceeded: Significantly more fellowships and consultant inputs were provided than specified. Research funds were provided in excess of the target. Civil works, furniture, and equipment were provided essentially as designed, although the quality and maintenance of certain items were deficient. Vessels and vehicles may have been all procured, but some have been lost to the project faculties. Some vehicles were reportedly never delivered to the universities; at least one was apparently delivered and then taken back to Jakarta (Appendix 1, para. 28). The OEM boarded and witnessed the start-up of several research vessels and saw them to be shipshape. Other vehicles and vessels were not available for inspection, raising questions about their whereabouts. Some vessels the OEM saw were in

8

very poor shape, pointing to neglect and misuse (Appendix 3, photo). The Government’s undertaking to operate and maintain facilities (AR, para. 93) has not been fully met.

30. Where the Project did not meet appraisal expectations is in its ultimate product—human resources in MSE. The AR postulates a demand for 10,674 marine scientists in 1999, when the project institutions would be fully operational (AR, para. 50), but a supply of only 3,090 without the Project. The shortage of 7,854 S1 (bachelor’s degree) graduates should have been partly alleviated by the Project. The OEM thus expected to find that MSE graduates from the project universities were being readily absorbed into the job market. Instead, officials at the universities reported that, on average, roughly half of the graduates find jobs in marine sciences or in related fields where a science background is useful (e.g., drug company sales); the other half are either unemployed or work in unrelated fields. (DGHE was unable to provide any statistics on student output.) While the curtailment of demand has been partly caused by the Asian financial crisis, the AR human resources projections were probably unrealistic.

31. A major result of the Project was intended to be research output, and many articles have been written for the journals published by the institutions. Blind reviews by recognized authorities showed that they are variable in quality, ranging from solidly scientific down to undergraduate term paper level, and those published in English suffer from a lack of language editing. Production quality varies from fairly sophisticated to abysmal. Several universities have more than one journal, which may be too much for the level of research at this stage.

32. An area where project implementation was deficient was monitoring and evaluation. The PCR suggests that matters may not have been proceeding smoothly at midterm and that a substantial and systematic review was indicated, such as in statements about BME, civil works, the variations in cost estimates and the sizeable savings, delays in the appointment of consultants, contractors’ performance, and lack of evidence of improvements in industry performance. Moreover, the OEM noted discrepancies in some of the PCR observations about such matters as the quality of civil works (PCR, para. 13) and contractors’ performance (PCR, para. 27). Some facilities had apparently never been completed and therefore could not have been in working order (e.g., at the UNSRAT field station), and the full complement of vehicles had not been provided to some universities. Furthermore, the quality of the BME report dated March 1996 is poor. Since project completion, there has been no follow-up of BME either by DGHE or by the universities. Record keeping also seems to have been abandoned, as at some universities the OEM could not obtain even basic data such as the names of fellows and the overseas universities they attended.

33. Fellows are important direct beneficiaries of the MSEP, and their feedback indicated overall satisfaction at having obtained an overseas qualification. However, they were critical of their inability to participate fully in several aspects of the MSEP, including local design issues, where they felt their local and scientific knowledge may have been helpful. They were also critical of the timing of activities such as the procurement and commissioning of equipment while they were undertaking out-of-country training. Fellows’ attitudes to a limited number of project elements were evaluated in focus group discussions during the OEM (Appendix 2, Table A2.2). Asked about their assessment of the Project before they began their studies, they rated highest (i) their level of motivation to undertake studies, and (ii) the level of encouragement they received from their heads of department or deans to do so. During project implementation, and since completion, the following were rated most highly: (i) the appropriateness of the teaching materials (books, software, etc.) provided by the Project; and (ii) improvements in the conservation of marine resources as an outcome of the MSEP. Poor ratings, listed in rank order (from those rated most poorly), were given to (i) the monitoring of repairs and maintenance by

9

DGHE, (ii) the sufficiency of government funds for maintaining equipment and facilities, (iii) the actual maintenance of equipment at the university, and (iv) cooperation between the Project and the private sector.

B. Institutional Development

34. The MSEP enhanced the capacity of the project institutions to deliver a program in marine sciences, conduct studies on the marine environment, publish scholarly material, assist community-level organizations such as fish farm cooperatives, and produce graduates capable of contributing productively to marine industries. Physical assets were installed and, in the main, commissioned to support these capabilities. There is some evidence that project institutions are continuing to develop MSE such as at IPB, where a PhD program is now offered.

35. However, the OEM has several reservations regarding institutional development. One is the high incidence of absences of students, faculty, and even administrators, particularly at IPB. Related to that is the underuse—sometimes admitted, sometimes merely observed—of civil works, equipment, books, and vessels. No active experiments were observed at IPB and the University of Riau (UNRI). At only two universities (University of Diponegoro [UNDIP] and UNSRAT) were any students seen in the library. Only at UNDIP were students observed working at the marine station. Faculty offices were generally devoid of books or other evidence of work in progress; at UNRI the condition of the offices indicated they had not been used for many months at least. Some of the research vessels (Appendix 3, photo) are reportedly used as few as 45 days per year. With some exceptions at UNSRAT and UNDIP, the OEM failed to see much evidence of a fertile academic atmosphere promoting significant research and excellence in teaching.

36. The training component produced staff with advanced degrees from foreign universities. Those staff are now occupying generally the same positions and doing basically the same work that they were doing before their degrees. Other than allowing the universities to open faculties of marine science or to add marine science to an existing faculty, the training has done little to change the institutions. Because the fellows were out of the country pursuing their degrees when the equipment was ordered by the consultants, they missed the opportunity to learn from the consultants and have an impact on the type of research and teaching activities to be undertaken at their universities.

37. The productive links with industry and other institutions, particularly overseas universities, are ad hoc and lack a coordinated strategic focus even at the local level. Small-scale applied research studies are being done and the results published. Some of these studies have been commissioned by the Indonesian navy or the private sector, but such links are not pursued proactively. There are opportunities for further institutional development. With growing pressures in overseas universities to internationalize their curricula, there is potential for Indonesian marine science institutions to capitalize on their strategic and competitive advantages such as low costs, ready access to tropical marine environments, reasonably equipped vessels, acceptable accommodation, basic libraries, and some research equipment. This strategy will require support from at least the faculty level and possibly from the university level as well as from DGHE. Such a strategy would have at least two benefits. First, it would underpin the sustainability of project outcomes. Second, it would enhance the quality of teaching and research at project universities.

38. The lack of an integrative element among project universities is disappointing, e.g., a nationwide program of marine pollution monitoring or of publishing. The research studies and

10

publications are done in isolation. There is no sharing of teaching materials. Whatever conferences used to be held (e.g., among UNHAS, UNPATTI, and UNSRAT) have been discontinued for budgetary reasons. Academic staff seldom visit other universities, much less transfer to them. While the MSEP did not specifically set out to establish such integration, cross-fertilization should be an attendant institution-building impact of such an endeavor even if not programmed.

39. The OEM is concerned about the maintenance of established capabilities and infrastructure and the lack of planning for further institutional development (apart from an often-repeated hope for more development assistance). This concern is linked strongly to a related concern with management and leadership (para. 58), and to the issue of sustainability (paras. 48-51 and 55-56).

C. Financial Performance

40. The project design called for chartering large research vessels (250-300 tons) for long trips, but this was never done, apparently for budgetary reasons.

41. There is limited capacity to finance the ongoing O&M of the marine science facilities at the universities. Frequent mention was made by university staff of inadequate levels of funding from the Government; in fact, fellows rated issues of maintenance most negatively among all aspects of the Project (Appendix 2, Table A2.2). Most institutions had requested additional funding from DGHE, but DGHE was not keen to provide it, believing that such funds should come from local initiatives and local sources. In a few instances, money was raised locally for scholarships for S1 (undergraduate) students or for equipment repair. The funding issue has been exacerbated by the Indonesian monetary crisis. The crisis has had a particular impact on science-based disciplines such as marine sciences, where the cost of materials has increased several times. There does not appear to have been any negative impact of the crisis on enrollments, which actually have increased in many disciplines including marine sciences. But the crisis may result in more university staff being constrained to take second jobs (“moonlighting”).

42. Limited funding is a major issue, particularly in relation to the increased expectation of supporting academic quality in the universities arising from the overseas experiences of fellows, and during the present financial crisis. National research funds are not being accessed strategically to support research and the existing research infrastructure; choices about expenditure of government funds are being made within universities without necessarily reflecting high priority on academic activities (such as local decisions to spend on buildings for the administration rather than on research); strategies to develop income-generating links with industries and Government are meager; and the installed infrastructure is being used well below capacity.

D. Socioeconomic and Sociocultural Results

43. The extent to which the MSEP is having an effect on local communities is poorly understood due to the lack of BME surveys, but it seems to be marginal. Some local employment has been generated, as at the UNHAS field station, which is on an island and employs people from the island to feed and care for the students and staff. But these benefits are minor compared with the projections in the AR. And not enough MSE graduates have been absorbed into industry. Thus, the hoped-for modernization of marine industries is not occurring.

11

The outcomes of the small-scale research studies at all sites could have impacts on environmental issues and decision-making, but the extent of the impact on social behavior or local economies is unknown, partly because of inadequate BME at the local level.

E. Gender and Development

44. The interests and role of women were not taken into account in this Project in a planned way. The OEM found no evidence that women’s issues were addressed in the project design.

45. The captain of UNDIP’s research vessel is a woman. In the fellowship program, 24 percent of participants in the MSc and 32 percent in the PhD program were female. The only two women fellowship recipients the OEM met were suppressed in the group interview by their male counterparts, who spoke for them; the women could air their views effectively only one-on-one with OEM members. In private they noted that women who wished to avail of fellowship training had difficulties arranging for care of their families during their overseas studies.

F. Environmental Impacts and Control

46. The fellows rated environmental issues among the most positive project outcomes and processes. These included improvements in the conservation and management of marine resources (Appendix 2, Table A2.2). Positive signs of attention to environmental concerns were found in most institutions, where studies were being undertaken to establish at least baseline data or monitoring of impacts of marine and upstream activity. Occasional advice is solicited from and given to various government agencies.

47. As with other elements of the Project, the OEM was unable to identify any coordinated activity among institutions or any strategic plan to address environmental matters at either the local or national level.

G. Gestation and Sustainability

48. While many target levels had been achieved by the time the Project had been completed and in some instances, such as fellowship training, had even been exceeded, the capacity of capital assets, human resources, and organizational structures to remain intact and generate future benefits is being eroded through lack of maintenance and planned development. Because UNPATTI's Hila Field Station had been destroyed, considerable effort and funding will be needed to restore the MSE program properly at that university. At the other project sites, physical assets are in many instances not being well maintained, there is no program to manage the continuing professional development of highly trained human resources, and there is no overarching organizational structure to coordinate and integrate marine science capacity at the local or national level to support the general objectives of the MSEP. The one asset expected to be in poorer condition (due to use)—the books—turned out to be in excellent condition (presumably due to nonuse) in most cases.

49. Links with some outside institutions are providing modest support for marine science activities. Japanese sources of funds for research have been accessed, and the Japanese alumni association is providing support for Indonesian students. Staff at UNHAS recently visited several Canadian universities, and there is hope that this may lead to links with positive financial implications. Overall, however, fellows did not rate the continuation of cooperation with overseas

12

training institutions highly, although there are apparent variations in this perception between project institutions, with those in the eastern islands viewing this more positively.

50. Part of the difficulty is that the AR gives scant attention to strategies that might support sustainability. “Long-term development” of living marine resources is identified as an expected benefit of the MSEP (AR, para. 106), but how this long-term goal is to be achieved and supported to generate benefits is not considered.

51. The PCR (para. 44) makes recommendations to ensure continued benefits to institutions and intended beneficiaries. The OEM could find little or no evidence that these recommendations had been or were intended to be implemented by the Borrower, DGHE, or the project universities.

IV. KEY ISSUES FOR THE FUTURE

A. Project Design

52. In any educational system, one of the major activities of a university professor, if not the paramount one, is teaching. In Indonesia, academics are supposed to spend roughly equal times in three activities: teaching, research, and community service. To prepare them for their duties, the project fellows were sent to do research degrees. This system has been criticized within the education profession as not conducive to producing good instructors (para. 10), but it has nevertheless been replicated in Indonesia through this Project and others. Furthermore, as researchers, the fellows are expected to manage research, including equipment, facilities, and other researchers (students). They were also expected, somehow to absorb the skills to facilitate the sustainable development of marine science and of the marine environment—the ultimate goal of the Project. Their advanced degrees generally included little formal training in such matters. This is an example of a project being designed in isolation from current thinking and best practices. ADB’s procedures for project design and appraisal did not include review of current literature and engagement of consultants at the cutting edge of their disciplines.

B. Local vs. Central Ownership

53. Centralization is needed to realize economies of scale in purchasing and to coordinate components like curriculum development. But too much centralization takes ownership out of the hands of those who will do the actual work of the project in this case research and the teaching of courses in marine sciences. Staff and administrators of the project institutions told the OEM several times that they wish they had been more involved in consultant specification and selection and in drawing up the equipment lists (para. 36), and thus in the acquisition of equipment more appropriate to their needs. An example of inadequate consultation occurred at the UNSRAT field station at Likupang: The local people would have warned against building a wooden jetty, which was destroyed by worms within two years and had to be replaced by a concrete one at UNSRAT's expense. Another ownership issue is the relation of the LPIU to the university (para. 18). Some university administrations felt bypassed by the LPIUs reporting to the CPIU in Jakarta. Appendix 16 of the AR shows the LPIUs reporting through the university rectors to the CPIU, but it did not work that way. The university rectors and the CPIU were essentially two bosses for the LPIUs, with the CPIU taking precedence. Now that the project administrative structure no longer exists, there is a conflict of expectations: The university structure is not well attuned to the needs of the MSE program, and the MSE faculties are used

13

to having all their needs supplied from an external source. The university administrations and MSE faculties should do a joint exercise in expectations and goal-setting.

54. Both conceptual and physical ownership are important. UNDIP has never received the title to its research vessel from the Ministry of Education and Culture (now the Ministry of National Education). As a result, it cannot take out insurance. An accident at sea with two dozen students, faculty, and crew aboard could create extreme liability problems for the university.

C. Program Sustainability

55. Sustainability refers to the capacity of project assets to remain intact and to continue to generate benefits in the longer term. The observed and reported outcomes clearly show that neither project design nor project implementation gave sufficient attention to this matter. The evidence is in the visible degradation of physical assets; low utilization of equipment, books, and vessels; lack of attention to continuing professional development; and unsatisfactory management and coordination at the local and national levels. The AR gives little specific attention to sustainability other than expressing the hope that the whole population will benefit from the MSEP and that there will be long-term development. Sustainability is a complex matter that requires careful planning, management, and beneficiary participation. There is no evidence at any stage of any planning for sustainability, or preparation to help develop local capability through consultancy services or fellowship training in management disciplines that might have assisted to achieve the long-term ambitions. And, as indicated in paragraph 33, beneficiary participation was minimal during planning and implementation.

56. These oversights had impacts on sustainability. For example, the technologies employed were not always suitable for local conditions, leading to disuse of equipment or complete failure as in the case of the jetty at Likupang (para. 15). Moreover, there was little apparent planning or training in techniques to develop sustainable strategies to generate income for financing O&M costs. Finally, the OEM saw no evidence that the Government had created a marine policy environment that would have stimulated the attainment of long-term goals.

D. Resource Utilization

57. The OEM observed that project assets, both physical and human, were underused. Many buildings were empty and showed lack of any recent use. This contrasts strongly with the situation in Indonesian secondary schools and technical education facilities, which are alive with activity. Much of the MSEP equipment and practically all of the vessels were unused, underused, or unusable, and the books showed little or no evidence of use. Human resources also appear to be underutilized. At some campuses visited there was very little or no evidence of fellows being actively engaged in teaching or research work of any kind, with the exception of UNSRAT and UNDIP. At IPB no academic activity was in progress, a matter also observed in 1997 by the consultant to the OEM. A great deal of money and effort was put into establishing the MSE program at the six universities, and not enough can be seen as a result. Part of the reason for this lies in management and leadership (para. 58).

E. Management

58. Many of the problems of this Project come down to the issue of management. The reason for this can be found in the AR, which gives almost no attention to management matters. The AR assumed that the combination of research training, consulting services, and asset

14

development would suffice to achieve project objectives, and that somehow management skills and leadership would flow naturally from this. The same error was made with respect to teaching (para. 10). The need for significantly better management is reflected, indirectly, in the recommendations of the PCR. All the recommendations address management issues, but the clear need for assistance in developing management knowledge and skills is entirely overlooked.

F. National Coordination

59. Coordination is needed to improve the research environment and the teaching of basic courses at the project universities and at other institutions that offer courses in marine sciences. Economies of scale suggest that something done independently in several locations can be more efficiently done if centralized, even in an archipelagic country. Electronic mail gives the opportunity to coordinate with very little expense. There does not have to be complete uniformity, but the marketplace for S1 (bachelors’ degree) graduates in a given discipline has the right to expect that they have similar backgrounds.

60. Much of the research published in the journals is of a low level scientifically and duplicates previous and/or contemporary work. And because there are so many journals, the efforts are diffused. Concentration into one or two journals of the better research efforts, and doing so in good English for at least some of the articles, would ensure better quality and better exposure abroad.

V. CONCLUSIONS

A. Overall Assessment

61. As described in the AR (para. 56) and RRP (para. 68), the Project had two main parts or components: (i) a MSE development program, and (ii) upgrading and augmenting of facilities at the six universities. Inputs for the MSE development program consisted of academic consultants, staff training and development, a research fund, and BME. Inputs for facilities upgrading and augmenting included civil works, equipment and furniture, and training and research vessels. The upgrading and augmenting of facilities could be viewed as an input to support the MSE development program. An additional input was incremental budgetary support for project implementation. Inputs that produced largely positive effects were the academic and engineering consultants, fellowships, research fund, civil works, furniture, and incremental budgetary support. Inputs with mixed results were the equipment and vessels. The input for BME was generally of poor quality.

62. Relevance. The Project was highly relevant to Indonesia, which has huge marine resources. These resources were not being scientifically used, partly due to the dearth of trained people. The Project provided a new program and a quantitative expansion of facilities, much of it in the less developed outer islands. The ultimate goals of diversifying the economy, improving nutrition, and utilizing marine resources sustainably were and are good, albeit somewhat beyond the capacity of one project. The relevance of the project components to these goals was good, particularly the funds for operation and research.

63. Efficacy. The efficacy of the project components in reaching project goals and objectives was satisfactory. Positive outputs include the students trained, research done, journals published, community service projects accomplished, and increased environmental

15

awareness engendered. The funds for operation and research were particularly efficacious. The fellowships and training component was more mixed in that it produced many individuals with advanced degrees but did not make a large difference in their roles or their management of the teaching and research situation.

64. Efficiency. The efficiency of the Project was lower than desired. While more consulting services, fellowships, and training were provided because of efficient use of designated funds, the civil works, equipment, and vessels are significantly underutilized, leading to deterioration from lack of periodic maintenance. Much more could be done with them if sufficient O&M funds were available. Furthermore, ADB supervision countenanced the omission of the midterm review, which had important ramifications on efficiency.

65. Sustainability. In terms of sustainability of project benefits, the MSEP exhibits some serious limitations. Through no known fault of ADB, the Government, the EA, or UNPATTI, the field station at Hila was destroyed during civil unrest. Because of that unrest, UNPATTI has closed down. So, one of the six project universities is not functional at present, and the MSE program there will take time to get back on track whenever UNPATTI reopens, as an indeterminate amount of MSE facilities, equipment, books, vessels, etc. will have to be replaced—at considerable expense. Several of the other campuses have problems that impede the sustainability of the Project. UNSRAT's Likupang Field Station is apparently nonfunctional, and the inactivity at both the main IPB campus and the Pelabuhan Ratu Field Station is very disturbing. Facilities at the UNHAS Barang Lompo Field Station are deteriorating. Of the six universities, only UNDIP evidenced a lively and busy academic atmosphere at both the main campus and the field station (Appendix 3, photo).

66. The MSEP has had successes and produced benefits and thus has had positive impacts. However, this must be qualified by negative aspects: the poor condition of some of the equipment and vessels; underutilization of books, equipment, and many buildings; absence of meaningful BME; destruction of the Hila Field Station; closing of UNPATTI’s main campus; nonfunctionality of the Likupang Field Station; inactivity at IPB; deterioration of the Barang Lompo Field Station; almost universal absence of a fertile academic atmosphere; difficulty of graduates in finding relevant employment; lack of coordination among project universities and staffs; failure of overseas training to make a meaningful change in the fellows’ roles; uneven quality of research; meager interface with the private sector; and failure to have an effect on the use of marine resources, nutrition, or the economy. For these reasons the OEM rates the Project partly successful.

B. Lessons Learned

67. Feedback from the survey of the MSEP fellows points to a critical factor in achieving satisfactory project outcomes: the process of fellow selection, supervisor support of fellows, and preparation of fellows prior to their formal training.8 The interviews with fellows and on-site inspections suggest that this component of the Project was successfully implemented, although it could have been improved by more language training before the fellows went overseas.

68. Project success begins with sound design. A fundamental deficiency in the design of this Project was the confusion between concepts such as objectives and strategies. Rigorous design

8 This important issue is explored in more detail in SST: INO 97023: Special Study of the Effectiveness and Impact

of Training in Education Projects in Indonesia, Special Studies Series No. 29 (ADB: Manila), 1997. The study includes material derived from five MSEP universities—IPB, UNDIP, UNHAS, UNPATTI, and UNSRAT.

16

guides implementation, facilitates monitoring and evaluation, and ultimately determines success or otherwise. Design should not be limited to one-shot projects but should look at long-term involvement in a subsector to increase sustainability, at the same time considering the involvement of other external sources. Even for relatively narrow projects such as this one, increased aid coordination is desirable.

69. Strategies need to be adopted that have a reasonable prospect of assisting in the attainment of all objectives. The analysis of this project demonstrates a failure to implement logical, best-practice ideas to achieve such objectives. This is particularly the case in matters as critical as improved teaching and management outcomes. Research training and academic coursework in technical subjects alone are ineffective for improving teaching or management. Furthermore, the policy environment must be conducive to stimulate growth in a sector in order for academic programs in that sector to flourish.

70. The analysis of compliance with loan covenants suggests some failure in the performance of both the Borrower and ADB. These failures indicate the necessity to ensure the midterm review is accomplished as required; proper BME is performed as designed; meaningful private sector involvement occurs; and rigorous evaluation techniques are implemented at project completion and include, as a minimum, full inspection of all project sites and associated field stations. Specific guidelines are extant for the responsibility of review missions to ensure compliance with loan covenants,9 including consultation with the Office of the General Counsel.

71. To maximize the likelihood of sustainable project outcomes, ADB procedures should include deliberate, research-based strategies for supporting sustainability in the project implementation phase. If none is readily available, such strategies should be developed by conducting a specifically focused sector study in the country and by examining the relevant research literature. Governments must genuinely commit themselves to O&M and BME.

C. Follow-Up Actions

72. The MSEP’s human and physical resources are in general decline. Many physical assets are deteriorating or have broken down completely, and human resources need professional development and research support. Some physical facilities present a danger to students and to staff. ADB should explore with the Borrower strategies to rehabilitate existing resources within a sustainable framework to protect project assets from further deterioration and to set them on course for continuing development. These strategies could include a joint goal-setting exercise for DGHE, the universities, and the marine science faculties; pedagogical training for staff members of the marine science faculties; and creation of schemes for greater utilization of MSEP facilities, including inputs from potential employers of graduates and potential users of research output. Rationalization and consolidation of equipment can be considered within a larger look at rationalizing the academic programs at the six universities. Given that many of the UNPATTI facilities have been destroyed, it might be best to cancel the MSE program there and transfer remaining staff, students, equipment, vessels, and vehicles to other institutions. If a system can be devised to inject needed funds into the universities’ marine science faculties to reverse the degradation, there is hope for improvement. Unsafe facilities should be immediately repaired, and degraded vessels and equipment should be rehabilitated. A concerted effort should be made to offer to the private sector services such as fish-finding, research studies, testing services, and even secondment of personnel to develop a clientele that can finance repairs and refocus efforts to meet project expectations. 9 Project Administration Instructions No. 5.04; Guidelines on Operational Procedures GP 40.

17

APPENDIXES

Cited on Number Title Page (page, para.) 1 Conditions at Project Institutions 18 3, 8 2 Results of Survey of Project Fellows 26 3, 8 3 Photographs 29 4, 14

Appendix 1, page 1

CONDITIONS AT PROJECT INSTITUTIONS A. Observations at Institut Pertanian Bogor (IPB) 1. Civil works provided under the Marine Sciences Education Project (MSEP) at the main IPB campus at Bogor constitute two wings of a very large building having five courtyards. This component was generally in good condition, although many flooring tiles were missing or cracked. Water and electricity were functional. To establish the project wings at the same standard as the rest of the building, viz., to Overseas Economic Cooperation Fund specifications, the Government made additional funds available. Furniture was well constructed and sufficient. The Operations Evaluation Mission (OEM) saw very little equipment, and no project vehicles. The two libraries were clean and the books well arranged, but most of the books, which do not circulate, seemed unused. Periodicals were haphazardly stored. Very few students were in evidence during the OEM’s visit to the main campus; the OEM was informed that they were in the field. All MSEP fellows were reported to be on staff, but only one was around during the OEM’s visit. 2. The OEM was told that 60 percent of IPB marine sciences college graduates (S1) find jobs in marine industries; the others are happy to be able to get any job at all and so are potentially lost to the field. This information was provided verbally, and the OEM saw no evidence that the planned local project benefit monitoring and evaluation (BME) system was in place. Reasonable standards of entry to marine science are reportedly indicated by the competitive selection of 60-70 students each year from 100-150 applications. However, the OEM was told that almost all students graduate from the program. This is an unusual outcome for any higher education institution, despite the local provision of orientation and mentoring programs. 3. The field station at Pelabuhan Ratu was practically deserted except for two classrooms being rented out to a private secondary maritime academy. Although 10 staff are reported to work at the station, the OEM saw only 5 IPB people during the visit, and all but one had gone when the OEM returned one hour later. That staff member was staying at the station with his family, but the other staff houses and the dormitory were locked. Civil works were generally clean and well maintained, but some areas needed appreciable repair: metal ladders had rusted out, and part of a roof had collapsed. The laboratory buildings were locked, and the only equipment seen was through the window of the repair shop (also locked). The repair shop was disorganized, had equipment and rubbish strewn around the floor, and seemed not to have been recently used. The two Boston whalers, in a covered drydock, looked worn and damaged, with their outboard motors in the repair shop. The large ponds appeared to be in good condition; two were dry and two were filled. The large research vessel is kept in the town’s harbor and seemed functional if somewhat in general need of cleaning and care (the engine started and ran smoothly). Other equipment was installed in the vessel, but the OEM could not ascertain its condition. No rubber or glass-bottom boats were seen. 4. An oft-repeated complaint of IPB personnel was that there is very little money available for repair and maintenance, thus explaining the poor state of much of the Project’s physical resources. Only Rp80 million (currently about $10,000) has been provided for maintenance each year for the past three years, and this amount has been seriously undercut by inflation. IPB has rented out the Boston whalers to sport fishers to raise money for maintenance. 5. Two types of linkages have been pursued with the private sector: consultancies and community service. Concerning the former, there is a plan to set up a marine information

18

Appendix 1, page 2

system to sell marine data (compiled by oceanographic, satellite imagery, and marine acoustic investigations) to companies. However, other than private companies sending their employees for short courses, industry links are not yielding any significant benefit academically or financially. Concerning community service, students work with Sunda Straits fisherfolk in a kind of extension service. Some private sector scholarship money has come from the Lampung tambak1 area. There are links with the Indonesian navy, but without any monetary consideration. Stock assessment is done in cooperation with the Directorate General of Fisheries. Linkages have also been established with universities in Australia, Japan, and United States (US), and locally with the University of Indonesia. 6. The Faculty of Marine Science and Technology produces two journals, one twice a year and one three times. The latter is in the Indonesian language. The former, the Indonesian Journal of Coastal and Marine Resources, which began publication in 1998, presents articles in either Indonesian or English and contains contributions of both local and foreign scientists. The print run for each journal is 1,000 copies. 7. As concern for the environment is increasing, IPB staff have been active in monitoring pollution in Jakarta Bay and at Pelabuhan Ratu. Specifically, the station has monitored heavy metal pollution through satellite imagery and has reported its results to the authorities. But the program has had little effect on marine industries. 8. There is now a Graduate School of Marine Science in Bogor. It awarded its first MSc in 1994 and its first PhD in 1996. Also at IPB is the Centre for Coastal and Marine Resources Studies with funding from the Department of Domestic Affairs. The proximity of these institutions, the Marine Sciences Education Program, and IPB’s generally high standing in Indonesia should theoretically put IPB in the forefront of marine science. However, the low level of activity at the campus and field station, the general observations made, and the lack of response to the OEM’s attempts to gather questionnaire data from fellows give grounds to indicate otherwise. B. Observations at the University of Hasanuddin (UNHAS) 9. During the OEM’s two-day visit to UNHAS and its associated field station at Barang Lompo no academic activity was going on in offices, laboratories, classrooms, or the library, it being graduation time. A few experiments were in progress, and only two library staff and one technician were at work. Several laboratories were locked and unavailable for inspection. 10. The limited academic atmosphere was underscored by an absence of the paraphernalia one associates with a university environment—books and academic materials in staff offices, staff working in laboratories or offices, students milling around or meeting with staff, staff discussing their work together, and so on. The one computer terminal in active use was being used by students to play games. The library held a good collection of books in excellent condition, suggesting little use by students or staff. The faculty publish a journal, Torani, two times per annum. The OEM was told that sufficient papers are submitted for publication to enable editorial selection to take place. 11. The civil works provided under the Project at the main campus were in reasonable, if well-worn, condition, with many floor tiles missing. The OEM consultant observed some improvements from 1997, when he had visited UNHAS as part of an Asian Development Bank 1 Brackishwater aquaculture pond.

19

Appendix 1, page 3

(ADB) special study—the problem of dust entry into rooms and laboratories had been addressed by the installation of heavy plastic sheets over air ventilation louvers, and some air-conditioning had been installed. There are problems in providing basic utilities such as water, which is available only intermittently. One project vehicle was apparently taken over by the university administration. 12. The field station at Barang Lompo, an island about 45 minutes from Ujung Pandang City by motorboat, was generally in good condition. Rooms were clean and orderly, and the furniture was in good shape. However, the buildings have deteriorated somewhat. Some cracking is evident, and roofing sheets have rusted out, leading to major damage to ceilings. There is apparently no budget to address this kind of problem, but staff are holding discussions with the University of British Columbia to establish a cooperative program, which may lead to an income stream in the future. 13. The field station has two generators (both broken), a seawater pump, fishtanks, and a nonoperational rainwater collection system in addition to offices, two classrooms, four laboratories, and a dormitory that holds more than 100 students. The main water pump is broken. The laboratory equipment (microscopes, weighing scales, and very little else) and SCUBA gear looked in poor shape. There was no activity at the main part of the station; no students were around, nor were there any experiments in progress. A technician-caretaker lives on the premises. No courses are scheduled at the station, but more than two dozen courses have their practical work and some lectures there. 14. A 10-minute walk from the main station is the hatchery, which showed more activity. Giant clams were being raised by the three technicians based there. The seawater pump at the hatchery worked. 15. The large research vessel, kept in the city harbor, was in good shape. The harbormaster maintains it in return for the right to use it. The ship contained a full set of navigational manuals, was clean, and had apparently functional equipment. The other vessels are kept at Barang Lompo. One Boston whaler was in reasonable shape, but the other was derelict. One rubber boat was nonfunctional because of valve problems; the other was not seen. The glass-bottom boat was in fair shape. C. Observations at the University of Diponegoro (UNDIP) 16. The main campus at Semarang is an active, vibrant institution. Before the Project, marine sciences had been part of the Faculty of Fisheries; now it is a separate faculty with its own curriculum. The large marine sciences building constructed under the Project was alive with students, attending classes and standing by. While many floor tiles were broken, the building seemed sound and functional. It is used partly for staff offices. It is also used for classrooms for first-year students for two months in the first semester and two months in the second semester under the block system. The other two months of each semester for first year students cover practical work and are held at the marine station at Jepara. Second through fourth year students have all classes and practical work at Jepara. Thus the building at the main campus is utilized only half-time. Nevertheless, the offices showed signs of being used. 17. The main library at the Semarang campus, a functioning building, has some MSEP books, mostly introductory texts. The original plan was to have three copies of each book—one at the main library, one in the Faculty of Marine Sciences building at the main campus, and one at the Jepara marine station. In 1994 the students asked that most of the books be transferred

20

Appendix 1, page 4

to the Jepara marine station, which the local project implementation unit (LPIU) was able to do on its own cognizance. 18. The station sits on 11 hectares, about 2.5 hours drive to the east of the main campus, with the project buildings spread out. The Faculty of Marine Sciences has an ambitious plan to add marine engineering to the curriculum and has produced a site plan that would add some 30 buildings. The present structures, most of which were funded by ADB, are reasonably well maintained, with some broken floor tiles. The library is clean and neat. The furniture is sturdy and in good shape. The laboratories are well equipped but, according to UNDIP staff, underutilized. While the geographic information system (GIS) laboratory had produced some impressive maps, only Rp100 million (currently about $12,500) is provided annually for laboratory maintenance. Sensitive equipment such as microscopes and computers is kept locked in air conditioned rooms (although the air-conditioning is not always turned on), but ordinary laboratory equipment is kept on benches and in unlocked cabinets. Some of the equipment arrived incomplete, and some did not follow the specifications drawn up by the LPIU, having been changed by the Central Project Implementation Unit (CPIU) for the sake of bulk ordering. Very few experiments were ongoing, but the semester had just begun, and the main mode was lecture under the block system. Chemicals and other consumables seemed in short supply, implying that not much experimentation or research can be done. One problem is that drains from the laboratories empty through open canals into the sea. 19. The hatchery is functional. Some research was ongoing, with students actively participating. Water and electricity worked. The large research vessel was shipshape; all equipment provided by the Project was present and was demonstrated to the OEM, and the engine turned over smoothly. Staff indicated that the vessel is used 45-60 days per year, mostly for teaching, sometimes for research. The captain was a woman. The rubber boats and SCUBA gear looked very well cared for. However, the two Boston whalers were in bad shape, with holes in the deck; one appeared nonfunctional. The Ministry of Education and Culture (MOEC, now the Ministry of National Education) has never formally transferred the vessels’ titles to UNDIP. Consequently UNDIP cannot insure the boats. 20. The OEM arrived at the Jepara station after classes had finished for the day, but students were in evidence at the four large dormitories. Some were studying, some listening to music in their rooms, some playing basketball. The water and electricity in the dorms worked, and the toilets were reasonably clean. The canteen was preparing for dinner, which smelled delicious. Part of one dormitory is regularly rented for seminars and meetings in order to raise money. The OEM inspected two occupied staff housing units and found them to be well designed and comfortable. About 20 of the 48 units were in use. The two guest houses are being rented to a foreign businessman to generate additional income. 21. There has been some collaboration with other sectors—joint research projects with Ryukyus University, the Marine Research Center of Tokyo University, Curtin University in Australia, the National Oceanographic Research Institute, the Brackishwater Development Center of the Directorate General of Fisheries, the Indonesian navy, and a Swiss food products firm. However, not much collaboration has happened recently with the private sector. Staff are involved in the Government’s coral reef and mangrove programs. 22. A journal, Ilmu Kelautan, has been produced four times a year beginning in 1996—14 issues to date. Articles are mostly in Indonesian (some are in English), all by Indonesian scientists, but with significant representation from outside UNDIP.

21

Appendix 1, page 5

23. The OEM met with quite a few staff members, finding them intelligent, articulate, and forthcoming. An alumni organization for marine sciences graduates helps them keep in touch with each other and UNDIP. The OEM was told that tracer studies have shown that about 18 percent of alumni are jobless, 38 percent work outside their field, and the rest are in jobs more or less science related (e.g., drug companies, veterinary medicine, and sales). 24. In summary, UNDIP is a functional marine sciences program in which project funds appear to have been well spent. D. Observations at the University of Sam Ratulangi (UNSRAT) 25. During the OEM’s two-day visit to the main campus, the academic environment was lively and busy. Classes were in progress, and the library was being well used for academic purposes. A large teachers’ room was buzzing with activity—students consulting their mentors, etc. Civil works provided at the main campus consist of a main building with offices, classrooms, and laboratories in generally good condition, although there was some evidence of deterioration, especially of the eaves and floor tiles. Local repairs had addressed the commonly observed problem of cracked titles. Water and electricity were functional although inadequate for routine operations. Inspection of laboratories and the library revealed that the equipment, books, and resources were being actively used by students and staff for research and teaching activities. All but two fellows (one transferred to the Faculty of Medicine at UNSRAT, the other to a teachers college) are still at UNSRAT. There is some continuing staff development. A few master’s degree fellows have completed PhD studies, and some support for research and conference attendance is available. Staff are contributing to the literature on marine science through their research and active contribution to international journals as well as to locally published journals at UNSRAT (a general university journal and a specialist marine science journal). As with the other project institutions, however, there was no evidence of a systematic plan to maintain and enhance knowledge and skills through a continuing program of professional development. 26. High standards of entry to marine science are indicated by the competitive selection of students from a large pool. Most graduates find jobs in local marine industries. This information was provided verbally, and the OEM was unable to locate any evidence that the planned local BME (which should be generating such data) was in place or had even been considered. Staff questioned about this project element avoided the issue. 27. Linkages with industry and other universities had been established. Industrial links are with the canning, fishing, cold storage, and mariculture industries. These links are based largely on companies hiring marine science graduates and providing opportunities for student practicums. Linkages with overseas universities in Denmark and Japan, particularly, have been maintained. These links include student exchanges and research cooperation, but they appear to be based on individual contacts rather than institutional agreements. 28. Two vehicles, a second-hand bus and a car, had been provided by the Project. Other vehicles identified in the appraisal report had not been delivered. However, the car had been returned to Jakarta. The main vessel was reported to have sunk in a storm at Likupang. It was repaired and is now at Bitung, about two hours drive from the main campus. One Boston whaler is in very poor condition; the other appears to be operational and in reasonable condition, although the material supporting the outboard engine is cracking. No other boats were seen.

22

Appendix 1, page 6

29. The field station at Likupang shows consistent signs of very low standard construction and materials, poor design, and minimal maintenance. A wooden jetty was destroyed by worms within two years and has been replaced by a substantial concrete structure funded by UNSRAT. Embankments and dikes in the pond area are collapsing because they were constructed of unsuitable materials. The walls of a classroom block are severely cracked, and a gap of at least 4 centimeters has opened between the floor and the external wall, warranting an urgent engineering inspection to ensure that any future users of the room are in no physical danger. Concrete surfaces exposed to the weather have deteriorated so much that paths and drains crumble when stepped on, and the boat repair building has been abandoned. The slipway linked to this building has become so barnacle encrusted that it is useless for its purpose. Seawater is pumped from inlets in the intertidal zone, but the inlets do not work well even at high tide. Classrooms, dormitories, and lavatories are in poor condition (e.g., broken windows, damaged ceilings, and nonfunctional plumbing) and are littered with rubbish and disarranged furniture. No teaching, research, or other equipment was sighted. Numerous tanks were scattered about the site and had never been properly installed or used. The research facility comprising small tanks cannot be used because the connection with the seawater inlet has never been completed. The access road, although paved, is in poor condition, and the electricity line to the connection at the main road is down. 30. Some of the deficiencies in civil works and in other project elements were attributed to a lack of consultation with local scientists during project design and implementation. Examples cited included boat and jetty design and construction. 31. As at each other project institution visited, there were concerns about funding levels. It was claimed that the faculty receives only “standard” funding, which does not take into account the need for chemicals, water, electricity, etc. The often-repeated complaint of UNSRAT staff was that there is very little money available for repair and maintenance, thus explaining the poor state of the Project’s physical resources at the marine station. Yet there was no reflection on ways of addressing this serious problem, such as building better links with industry or developing a strategic approach to winning research grants. 32. UNSRAT academic staff have been active in contributing their expertise to monitoring environmental impacts of various industrial activities in North Sulawesi. This includes studies on the toxic effects of local gold mining and collecting data on biodiversity in collaboration with the Government. Staff recognize, however, that they need to develop more effective approaches to local communities, such as those directly involved in fishing, where changes to practices are indicated. E. Observations at the University of Riau (UNRI) 33. The project civil works at the main campus seemed structurally sound but needed general maintenance and cleaning. Offices appeared to have been unoccupied for years, witnessed by the presence of dust and cobwebs and the absence of work material, and classrooms were not being used. The furniture is strong and in good shape. 34. The OEM could not visit the marine station at Dumai, as it is about five hours drive from the main campus. All the books and equipment supplied by the Project were said to be at the station. Photographs taken in April 1998, given to the OEM, show a large, good-looking building, students aboard the large vessel wearing life jackets, and one of the rubber boats being paddled (i.e., minus its outboard motor). Fellows’ concerns with equipment centered on it never having been fully installed and the lack of training to operate it. No strategy for managing these

23

Appendix 1, page 7

concerns was presented. The frequency with which fellows volunteered written comments in their questionnaires about problems with maintenance funding, particularly for vessels, (compared with their colleagues elsewhere in the MSEP) causes the OEM to record its serious concern about the facilities at Dumai. 35. The OEM was shown one Toyota vehicle at the main campus and was told that the bus was at the marine station. 36. Interestingly, the rector of UNRI is a marine scientist, with a PhD from a US university. As a consequence, the Department of Marine Science theoretically enjoys a somewhat advantageous position, although no physical or documentary evidence of this was presented to the OEM. There were no significant links with other institutions, either with local industry or with overseas universities, that might be expected to flow from the rector’s patronage. 37. More than a dozen staff, formerly fellows under the Project, met with the OEM, filled out the questionnaire, and talked with OEM members. The two women staff who met with the OEM were the only women seen during the OEM. When the OEM talked privately with them, they discussed the special needs of women in a fellowship program. One of these needs, for married women, is support to address the problems of leaving family and especially children when a fellowship is offered. Another problem concerns the way women’s contribution to the academic work of the institution is devalued. Although this is possibly cultural, it is nevertheless antiacademic. The problem was apparent several times during the meeting with fellows: whenever the women attempted to express a view, their male colleagues intervened and attempted to present the women’s views for them. This issue needs to be addressed if the full value of fellowship training is to be realized and should be an element in a continuing professional development program, which the OEM noted was absent at UNRI as it was elsewhere. 38. Enrollment in the Marine Sciences program at UNRI was 265 (of whom 75 were women) in 1997/98 and is currently 358 (55 women). Thus, while total enrollment is up, female enrollment is dropping. F. Report on the University of Pattimura (UNPATTI) 39. The OEM was unable to go to Ambon for security reasons, but some information was learned from UNDIP staff who had been in contact with UNPATTI officials as recently as three weeks before and who were in current contact with UNPATTI students who had fled to Semarang. This information, however, is unverifiable. 40. While the main campus is still intact, the field station has been destroyed, presumably with all its equipment, furniture, books, and other contents. Fortunately, most of the project equipment was at the main campus. The informants did not know about the status of the vehicles and vessels. 41. The vice rector for student affairs of UNPATTI was in hiding in the hills because of death threats against him. He had called UNDIP three weeks earlier using a cellular telephone. Two academic staff members, including the son of the former rector, a marine sciences graduate, were killed on a boat from Ambon to Jakarta under unknown circumstances. 42. About 10 UNPATTI students have come to UNDIP to try to enroll. They say that their houses in Ambon were burned down. However, they mostly cannot produce the documentation

24

Appendix 1, page 8

(transcripts, etc.) that UNDIP needs to accept them. The UNPATTI dean and vice dean of the Faculty of Marine Sciences were unavailable to provide the documents. However, one student who had proper papers has been taken in at UNDIP, and two more are in process. Reports were received that other students are continuing their studies at UNSRAT and UNRI.

25

Institution FemaleShort

Training

UNDIP 3 10 38 9 4 0 1 9 3UNHAS 3 5 35 5 2 1 0 8 0UNRI 2 7 42 5 2 2 0 9 0UNSRAT 4 39 2 2 0 1 1 2Totals/Average 8 26 38 21 10 3 2 27 5

UNDIP = University of Diponegoro , UNHAS = University of Hasanuddin, UNRI = University of Riau, UNSRAT = University of Sam Ratulangi.Source: Operations Evaluation Mission.

Gender

Research Teaching

RESULTS OF SURVEY OF PROJECT FELLOWS

Table A2.1: Characteristics of the Fellows Surveyed

Master's PhD

Degree Major Work ActivityOthers/ Missing

Average AgeMale

Institution Australia Canada Denmark France Indonesia Japan USA

UNDIP 3 2 2 2 4 13UNHAS 2 5 1 8UNRI 1 1 1 2 3 1 9UNSRAT 1 2 1 4Totals 3 3 2 7 2 2 13 2 34

UNDIP = Diponegoro University, UNHAS = University of Hasanuddin, UNRI = University of Riau,UNSRAT = University of Sam Ratulongi.

United Kingdom TotalWhere Qualification

Appendix 2, page 2

Table A2.2: Fellows’ Attitudes to Their MSEP Training and Project Outcomes

Question No.

Question UNDIP (n=13)

UNHAS (n=8)

UNRI (n=9)

UNSRAT (n=4)

TOTAL (n=34)

Before you began your studies under MSEP, how would you have assessed each of the following?

(Scale: 5 = Excellent; 1 = Very Poor)

7 Your subject knowledge and skills necessary to undertake your studies

3.46 3.88 3.67 2.75 3.63

8 Your competence in English or another foreign language to undertake your studies

2.85 3.63 3.67 3.25 3.43

9 Your preparation to live and work in the overseas culture where you studied

3.18 3.38 4.00 3.00 3.59

10 Your level of motivation to undertake your studies

4.23 4.75 4.25 5.00 4.62

11 The level of encouragement you received from your head of department /dean to undertake your studies

4.08 4.38 4.25 5.00 4.50

During MSEP implementation and since the Project has been completed, how would you assess each of the following?

12 Your work with MSEP consultants 3.27 3.57 4.00 3.50 3.55 13 The quality of construction/ civil

works 3.08 3.38 3.11 3.00 3.15

14 The appropriateness of the equipment provided by the Project

2.92 4.14 3.78 3.00 3.42

15 The appropriateness of the vessels provided by the Project

2.75 4.13 4.22 3.25 3.55

16 The appropriateness of the teaching materials (books, software, etc) provided by the Project

3.69 4.38 4.22 3.25 3.94

17 Cooperation between MSEP and other aid programs (e.g., from Australia, Canada, Germany, and World Bank)

3.13 3.33 2.00 4.00 3.05

18 Cooperation between MSEP and the private sector

1.89 2.60 2.83 3.00 2.43

19 The sufficiency of Government of Indonesia funds for routine operations

2.45 2.29 2.78 3.00 2.58

20 The sufficiency of Government of Indonesia funds for the maintenance of equipment and facilities

1.89 1.88 2.33 3.00 2.17

21 The actual maintenance of equipment at the university/field station

2.38 2.43 2.22 2.50 2.36

27

Appendix 2, page 2

Table A2.2: Fellows’ Attitudes to Their MSEP Training and Project Outcomes

(continued)

Question No.

Question UNDIP (n=13)

UNHAS (n=8)

UNRI (n=9)

UNSRAT (n=4)

TOTAL (n=34)

22

The actual maintenance of vessels at the university/field station

2.33

3.17

2.25

3.00

2.57

23 The level of support you received from your head of department/dean to use your new knowledge and skills in your work.

3.23 3.88 3.33 4.00 3.48

24 The monitoring of repairs and maintenance by Directorate General of Higher Education

2.57 2.60 1.56 2.00 2.13

25 The continuation of cooperation programs with overseas training institutions

2.67 3.67 1.38 3.67 2.62

26 The usefulness of field stations for your work

3.31 4.25 3.22 3.33 3.52

27 The modernization of marine industries as an outcome of MSEP

3.08 3.00 2.00 2.50 2.81

28 Improvements in the management of marine resources as an outcome of MSEP

3.25 3.43 3.29 3.00 3.29

29 Improvement in the conservation of marine resources as an outcome of MSEP

3.64 3.57 3.78 3.33 3.63

MSEP = Marine Sciences Education Project, UNDIP = University of Diponegoro, UNHAS= University of Hasanuddin, UNRI = University of Riau, UNSRAT= University of Sam Ratulangi. Source: Operations Evaluation Mission.

28