GCSP/INS/073/JPN

SPECIAL PROGRAMME FOR FOOD SECURITY

INDONESIA

Report of Joint Final Evaluation

Masa Kato (FAO, Team Leader)

Minoru Nakano (Japan)

Arif Haryana (Indonesia)

Food and Agriculture Organization of the United Nations (FAO)

Rome, October 2007

Table of Contents

Pages

Executive Summary 1 - 4 Main Findings 1 - 2 Main Issues and Recommendations 3 - 4 I. Introduction and Background 5 – 6 A. Introduction 5 B. Background 5 – 6 II. Project Approach and Design 6 – 7 III. Assessment of Project Implementation, Efficiency and Management 8 – 12 A. Project Implementation 8 – 11 B. Implementation Efficiency and Management 11 – 12 C. Support Given by Three Parties 12 IV. Assessment of Emerging Results 13 – 18 A. Development of FGs 13 B. Capacity Building of DSTs 14 C. Improving Agricultural Production 14 – 16 D. Contribution to Improved Food Security and Income Generation 16 – 17 E. Sustainability of Project Results 17 - 18 V. Main Conclusions, Issues and Recommendations 19 – 23 A. Conclusions 19 – 21 B. Emerging Issues and Lessons 21 – 22 C. Recommendations 22 - 23 Annex I. Terms of Reference 24 – 29 Annex II. Mission Itinerary and Persons Met 30 Annex III. Project Budget Status 31

SPFS Project Indonesia (GCSP/INS/073/JPN) - Final Evaluation Report

Executive Summary

As the terminal date (16 September 2007) of this project approached, the final evaluation mission visited Indonesia between 8- 17 August 2007. The mission comprised M. Kato (FAO), M. Nakano (Donor) and Arif Haryana (Government). This summarizes the main findings and conclusions of the evaluation report.

Main Findings

i. Relevance of project and approach: The objectives have been very relevant to the national effort for food security among the rural poor. In particular, the re-articulated project approach developed in the initial implementation stage has provided a coherent and appropriate approach, more clearly aligned to the emerging policy orientation strategy (emphasis on participation and decentralization). In particular, the project approach provided opportunities for the target groups to organize themselves as Farmer Groups (FGs) and to shape the project interventions to address their priority needs through participatory identification, planning and implementation processes (Farmer Group Development Plans and Revolving Fund(RFS) in collaboration with district SPFS teams (DSTs); ii. Implementation efficiency. This has been quite satisfactory, including the quality of project management. Not only implementing largely on schedule a very complex project operating with 36 FGs located in five districts and provinces (actually across the Indonesian archipelago), the project management has also played, with support of the RSC, a key role in developing and operationalizing the main tools used by three other SPFS projects; iii. Overall effectiveness in achieving the planned results. Generally, the 36 FGs have made good progress towards achieving self-reliant management capacity, they made considerable improvement in raising food and agricultural productivity as well as in evolving farming systems adapted to their agro-ecological conditions. They have not only successfully applied new technical packages, taking advantage of the new irrigation systems, but also initiated some innovative activities, such as the goat milk production and processing in Banjar. With good progress in earning income, many households and FGs have strengthened their capital asset, individually and collectively (through RFS), with potentials for making investment for further productivity improvement. Similarly, the local DSTs and officers have been strengthened in their capacity, many with strong interest in working with farmer communities; iv. Sustainability of Project Results. Many aspects of the results show strong prospects for sustainability, including the institutional arrangements established by the project (the

2

majority of FGs and DSTs, and active participation and support of national and especially the local authorities) as well as the use of much of the improved technical packages introduced. In particular, the policy level support provided by the Agency for Food Security (initiation of the Desa Mandiri Pangan – DMP, “village self reliance programme” with an approach very similar to that of the SPFS project) has added a significant impetus for sustainability; v. Cost-effectiveness. This is a complex and controversial concept, especially in quantitative terms. The project cost per FG household is estimated at US$ 1,942. However, qualitatively, the outcome of the project is substantial and highly satisfactory, probably as good as practically feasible and certainly much better than many projects of similar nature and scope. Further, the role played by the project in developing the methods applied by other three SPFS and coordination with the Government on the DMP programme should be taken into account. Thus, on the whole, the project performance has many cost-effectiveness elements; vi. Gender aspects. While gender has not been particularly highlighted in the approach, the project has worked, closely and in a participatory manner, with women and men of the FGs, taking into account of their functions, roles and their local culture. In fact, many women groups have been actively involved in livestock activities, especially in Banjar and Central Lombok, and some women members have played leading roles in their FGs; vii. Pilot model for participatory rural development and food security. The project approach has proven to be effective in promoting self-reliance, empowering the rural poor, improving their productivity, as well as linking effectively the farmers and local service providers for rural and agricultural development. As a pilot project, its performance and achievement show much to recommend itself as an effective model, but this needs to be systematically demonstrated through an in-depth and systematic assessment of its effectiveness. viii. Critical factors affecting the project performance. Following factors have been particularly significant:

• positive factors – (i) coherent and relevant project approach, (ii) good reception by the target groups (members of FGs), including their strong sense of ownership of their FGs, (iii) good collaboration between the FGs and local DSTs, (iv) professional and efficient project leadership and management team, especially the PMU, (v) good technical support from the Regional SPFS Coordinator, and strong policy support from Government and the Donor;

• negative factors – no major weaknesses exist, but some aspects could have been improved: (a) complexity in the project scope, especially in geographical coverage, (b) provision of a preparatory phase for one - two years for developing operational approach for project implementation, and (c) more systematic monitoring throughout the implementation period.

3

Main Issues and Recommendations

A. Some Issues and Lessons

ix. Issues. Because of the Government’s strong policy support to food security, especially the initiation of the DMP Programme, no pressing issues exist in terms of follow-up to ensure sustainability of the project results. However, there are some issues needing attention:

a) issues for immediate attention following the project’ closure include:

• the need to maintain the PMU so as to apply its competence and experience in support of similar programmes and projects, and

• desirability of integrating the FGs formed by this project into similar programme or project for continued development (especially for these FGs which are still to reach sustainability);

b) issues for longer-term consideration:

• how to coordinate initiatives by local governments to replicate the experience of FGs in their areas (e.g. Kota Banjar, Barito Kuala), including their links with the DMP Programme. This also raises the question of how to ensure the necessary technical and managerial capacity for implementing such initiatives.

• need for refining and adjusting the project approach for developing FGs further in broad context, including the exit strategy for the approach, and strategy for assist them in expanding activities with high value-added activities (post harvest activities, processing and marketing areas, on or off farm).

x. Lessons. These include the following:

• for sustainability, empowerment of the rural poor for their self-reliance and self-management is imperative;

• holistic approaches are required, combining empowerment of the target groups and a range of support in improving their productivity;

• strong and well motivated substantive leadership for the project in the form of PMU and at the local level (DSTs) is essential;

• working with the rural poor for food security and poverty reduction requires systematic capacity building of the stakeholders, and over a long time;

• for complex projects or pilot projects, a functional monitoring system is essential to support the project management and to facilitate in-depth assessment of the effectiveness.

B. Recommendations

xi. For immediate follow-up action in conjunction with the project termination,

following are suggested:

4

a) It is strongly recommended that the Agency for Food Security transfer the PMU staff to its programme staff working for the national DMP Programme to assist in its planning and implementation;

b) It is similarly recommended that;

• the project FGs be incorporated into the DMP Programme immediately to ensure their continued development with coherent technical and institutional support;

• those FGs requiring further support be given a priority attention to help them achieve sustainability soonest - similarly, those FGs already with maturity should be encouraged to assist the other villages in the programme; and

• within the framework of the DMP Programme, further evolution of the project FGs be monitored to study their progress towards their sustainability – this would help gain insights into the conditions for defining an exit strategy under the DMP Programme.

xii. For longer-term follow-up action, following are suggested for consideration:

a) The Agency for Food Security should take initiative to coordinate the process of replicating the SPFS approach or launching similar projects by various local authorities within the framework of the DMP programme;

b) The Agency for Food Security should study ways and means for;

• deciding the exist conditions for withdrawing the programme support to a particular village, and

• expanding the development of rural communities beyond the simple poverty line by promoting agriculture and rural based enterprises with added-value (involving agro-processing, marketing and other post-harvest processes);

c) With the wealth of data available, the experience of the SPFS project be assessed systematically with focus on how the project approach and design might have contributed to achieving the planned results, i.e. verification of assumed cause and effects of factors involved in food security and poverty reduction.

5

I. Introduction and Background

A. Introduction

1. The SPFS project in Indonesia is one of the four country projects funded by Japan (also two regional projects1), all in support of implementing pilot projects in the four Asian countries during 2001 and 2007 (Bangladesh, Indonesia, Laos and Sri Lanka). This project in Indonesia, with original donor funding of US$ 2,500,000 became operational in September 2001 for five years (terminal date of September 2006) – this date has been extended by one year to 16 September 2007: the Donor contribution was increased by a total of US$ 599,537 to cover the one year extension and funding for two emergency projects in Ache and Java so that the total donor funding amounts to US$ 3,099,637. As the terminal dates of the four projects approached, the three parties concerned (the Governments, Donor and FAO) decided, in accordance with the FAO procedures, to conduct final evaluation of the projects during July and August. The independent evaluation mission comprising Messrs. Masa Kato (FAO designate, team leader) and Minoru Nakano (Donor designate, agricultural development) visited Indonesia during 8-17 August 2007 – the mission was joined by Dr. Arif Haryana (Government designate, BAPPENAS) during its stay in the country. The mission’s Terms of Reference and itinerary are given in Annexes I and II, respectively.

B. Background

2. Indonesia has one of the largest populations (217 million in 2003) in the world, and food security has been a major national goal since independence. Its concerted effort for rapid economic development since 1970 led to major economic growth, with GDP per capita doubling between 1980 and 1996 to over US$ 1,100 – in agriculture sector, the country achieved self-sufficiency in rice in mid-1990s. However, the Asian financial crisis of 1997-98 reversed the progresses, resulting in deterioration in poverty and food insecurity. While steady economic recovery has been made since (proportion of population below poverty line of US$ one per day reduced from over 20% in 1999 to 8% in 2003), the government has continued to strive for improved food security as a national priority (14% of population estimated as under-nourished in 2003). In order to address the multi-sectoral nature of food security through decentralized administrative system, the National Council for Food Security, chaired by the President and directed by the Minister of Agriculture, provides inter-ministerial policy coordination. Within the Ministry of Agriculture, the Agency for Food Security is responsible for operational coordination of various programmes in support of food security.

1 These projects are GCP/RAS/180/JPN financing the Regional SPFS Coordinator, located in Jakarta, Indonesia and GCP/RAS/182/JPN to finance the SPFS Asia Information Management System, operated from FAO Regional Office for Asia and the Pacific.

6

3. The present project, formulated with FAO assistance, has been intended to support the Ministry of Agriculture in shaping a national strategy for food security consistent with principles of participatory and decentralized development approaches. As a pilot project, it is designed to evolve an effective model for promoting food security in poorer rural areas through a participatory community development approach, including empowerment of poor farmers, in five selected sites in five districts and provinces (West Nusatenggara, South Sulawesi, South Kalimantan, Riau and West Java), each representing a major agricultural production systems in the country. 4. The project is executed by the Agency for Food Security (AFS), with its Director-General serving as National Project Director, under the overall policy coordination by the National Steering Committee (co-chaired by Minister of Agriculture and head of BAPPENAS). The project is implemented by the sub-district governments (Kabupaten), in particular, their agricultural service departments, which have established “District SPFS Teams” (DSTs) for the purpose of the project to support the farmers in the respective project sites. The project is housed within the AFS for overall coordination, and the Project Management Unit (PMU, comprising National Field Manager and three core specialists covering farming system, irrigation/water management, and livestock) provides technical support and coordination to the field implementation. Technically, the project has been supported by the Regional SPFS Coordinator, located in Jakarta.

II. Project Approach and Design

5. Project Objectives. These were defined in the original project document as:

a) Overall objective – to strengthen food security, revitalize the rural economy and alleviate rural poverty to enhance economic and nutritional well-being;

b) Immediate objectives – these covered (i) to identify socio-economic constraints on achieving food security and to strengthen community capacity to participate in planning, management and development of physical and natural resources for sustainable food security, (ii) to intensify and diversify crop production to increase food production and cash income by introducing and promoting new technologies and practices, (iii) to diversify farm production and increase household income and improve diets of villagers through introduction and promotion of intensified livestock and fisheries production systems, (iv) to establish improved post-harvest handling technology and small-scale agro-processing of agricultural produces, supported by marketing, credit and training, (v) to strengthen the capacity of district extension staff to support agricultural development.

6. These objectives are broadly relevant to the national objectives for food security as well as the National Poverty Reduction Strategy: these are generally very relevant to the needs of the target groups. In the project document, however, these objectives remained rather vague, without clear strategy for realizing them. Fortunately, in practice (without formal changes in the project document), they have been re-interpreted to stress more clearly the importance of strengthening the capacity of the local community for

7

participatory development in a decentralized manner, reflecting the ongoing shift in policy emphasis in the country.

7. Project Approach and Design. The modified approach and design, recognized clearly the project as a pilot designed for empowering poor rural communities in developing sustainable food security at household and community levels through participatory and innovative collaboration by the communities, local governments and civil society organizations. In particular, the focus has been on (i) establishing dynamic and self-reliant farmer groups, including their mind-sets, (ii) empowering the farmer groups (FGs) by strengthening their capacity to plan, implement and manage development actions through the use of Farmer Group Development Plans (FGDPs) and the Revolving Fund System (RFS) based on initial resources provided by the project in support of implementing FGDPs, (iii) strengthening collaboration between the FGs and villages with local government services ability, including strengthening of these government providers, (iv) improving village irrigation systems and water management, (v) increasing productivity of crops, livestock and fisheries through intensification and diversification, and (vi) improving livelihood and food security.

8. Within this overall approach, the project design has sought to evolve suitable models for improving food security and livelihood adapted to the five selected agro-ecosystems; (i) established and well developed irrigation systems (Banjar/Ciamis, West Java), (ii) newly developed irrigation scheme (Rokan Hulu, Riau), (iii) poor drainage agro-ecosystem (Barito Kuala, South Kalimantan), (iv) dry upland agro-ecosystem (Central Lombok, West Nusatenggara), and (v) coastal agro-ecosystem (Jeneponto, South Sulawesi). Accordingly, different patterns of technological interventions have been promoted to improve agricultural productivity and income generation. In practice, the project implementation in these sites has been undertaken like five distinct subprojects, each having its own focus and scope within the overall project framework. Another aspect very unique to this Indonesia project is that with intensive support of the RSC, it has played the role of piloting and developing the key operational tools (FGDP and RFS) to facilitate the implementation of the project’s approach. 9. The approach outlined above is very appropriate to the implementation of the national development strategy for rural and agricultural development targeted at poor rural communities, including the stress on empowerment and self-reliance of farmers through FGs for sustainable development. The approach is also coherent, especially its three main elements comprising: (i) empowerment of farmers through FGs, (ii) capacity building of the FGs through learning the FGDP method, coupled with establishing FGs’ financial basis through RFS to maximize sustainable use of the financial contribution from the project, and (iii) promoting collaboration between FGs and local governments in community development. This approach provides a clear and coherent institutional and methodological framework for defining, planning and implementing specific development interventions, including selection and application of technologies and practices.

8

III. Assessment of Project Implementation, Efficiency and Management

A. Project Implementation

10. Overall. The project implementation process can be divided into three stages. The first (September 2001 – May 2002) covered the preparation and mobilization period, culminating in the project launch workshop, which finalized the target sites, the implementation arrangements and overall implementation schedule. The second (June 2002 – early 2004) comprised initiation of field implementation, with focus on (i) training of the villagers and FGs as well as the local support teams (District SPFS Teams – DSTs), (ii) development of FGDP approach and RFS, and (iii) initiation of field development activities with livestock and irrigation system improvement works. The third stage (since early in 2004) may be called “intensive implementation period” when the FGDP and RFS mechanisms were improved and full implementation of production or income earning activities has been pursued. Given the relatively long gestation time before full implementation, it was well justified to have the terminal date extended by one year to September 2007, which has allowed some four years of intensive project implementation work. The main aspects are summarized below.

11. Formation and Strengthening of FGs in Five Project Sites. A total of 36 FGs in 20 villages2 in five districts (and provinces) were established in 2002 and since then these FGs with a total members of 1,402 have been supported towards developing into self-reliant organizations (see Table 1). The FGs generally represent some 25-30% of the village members (poorer sections), and they have their own organization arrangements, including constitutions agreed to by the members, executive committees elected by the members (usually chair-person, secretary and treasurer), several groups for separate activities with FGDP, and the RFS with its own account and management groups. The FGs have been assisted in participatory planning for community development, management and administration of FGs, especially RFS, the use of FGDP method for preparation and implementation of group activities, and specific technical subjects related to their development micro-projects. The technical training was provided mainly by the DSTs (sometimes also consultants and/or PMU staff), often employing farmer field school approach, with a number of farmers trained as farmer trainers in these subjects. Since 2002, a total of 21, 086 FG members (including double counting) have participated in training activities on various subjects.

2 These Fgs were selected in terms of several criteria: (i) being representative of the each district’s agro-ecological situation, (ii) vulnerability to food insecurity of more than three months of the year, (iii) prevalence of poverty with households below annual income of Rp. 1.0 million, (iv) access to market, production inputs and other supporting facilities, (v) the village community’s interest in development but not targeted by another development programme, (vi) the village and neighbouring villages have potentials for development.

9

Table 1: Project Target Beneficiaries and their Location

No.

District, Province

Number of Villages

Number of Farmer Groups

Beneficiary (household)

1 Central Lombok, West Nusatenggara

4 6 316

2 Jeneponto, South Sulawesi 3 7 226

3 Barito Kuala, South Kalimantan 3 6 180

4 Rokan Hulu, Riau 4 7 197

5 Ciamis, West Java 6 10 482

Total 20 36 1401

12. Capacity Building of DSTs. The implementation arrangement in the field is predicated on effective local support to FGs, especially in technical aspects of production and income generation activities. In each project districts, a five-member team with multi-disciplinary expertise has been organized as DST, backed up by the sub-district agricultural service (DINAS Kabupaten). Thus, the capacity building of DST entailed training of both the DST members and many staff of the sub-district agriculture services as well as ensuring liaison with the sub-district, district and provincial authorities for their support to the project. The training included participatory community development approaches including community empowerment, key tools of the project approach, such as FGDP and RFS, and technical subjects related to the project interventions in each site (e.g. irrigation development and water management, livestock, post-harvest and marketing). It is estimated that since 2002 a total of 930 district level officers has been trained by the project, including 134 in workshops for training of trainers. Similarly, over 90 provincial staff and over 130 staff of the Ministry have participated in various workshops and training activities.

13. Production Support Activities. As noted above, these activities started during 2002/03 with priority on livestock development and irrigation and water management components in all sites: while this priority reflected the needs of the FGs expressed in their community development action plans, it was considered necessary for PMU to conduct more detailed technical investigations with the DSTs and local agricultural services before concrete and appropriate intervention measures can be identified and implemented for intensification and diversification in food and agriculture production. This gradual pace at the early stages of implementation was made necessary because it took some time to build up the capacity of DSTs and because the compact size of PMU staff meant clear physical limitation on the implementation scope and pace in all five sites and at the same time. Since 2003, a total of 62 FGDP based activities have been implemented by FGs with direct inputs from the project, amounting to a total of Rp.

10

6,467 million or Rp.1,294 million per site. In terms of activities, most numerous has been rice farming (24 activities, including irrigation and hand tractors), followed by livestock farming (19 activities, for cows, goats, and ducks), fruits production (8 activities, citrus, oil-palms, rubber), fish culture (5 activities), secondary crops (4 activities, for soybeans, corn and ground nuts) and horticulture (2 activities, for vegetables and chilli) and handicrafts (2 activities). 14. As noted above, the FGDP activities, supported by RFS, have catered for improving productivity according to the production patterns adapted to potentials of each sites. These are summarized in Table 2, by the five project districts. Some clear patterns emerge for each site. For example, FGs in Banjar in West Java have concentrated on intensification of rice farming (by taking advantage of good irrigation systems (introducing three cropping in some cases)) and on livestock development to exploit the favourable marketing environment (dairy and meet production). In Central Lombok where rain-fed agriculture imposes sever water constraints, the FGs have opted for diversification in the production system with concentration on secondary crops and livestock, supplemented by handicrafts activities by the women groups for income earning. Similarly, in Jeneponto in South Sulawesi which faces water constraints but has access to marine fisheries, the FGs accorded priority to diversification in income earning through livestock development and coastal fisheries. The value of project support to these sites has been fairly even, with the major exception of Jeneponto, which received less than half of the amount of Banjar (the largest recipient) – apparently, project implementation in Jeneponto experienced some difficulties, including the development of the FGs (see para. 19 below). Table 2. Distribution of SPFS Projects Activities in Indonesia (2003 – 2007)

No. Activities Banjar (West Java)

Barito

Kuala (S.Kalimantan)

Central

Lombok (NTB)

Rokan

Hulu (Riau)

Jeneponto (S.Sulawesi)

Indonesia (Activities)

I. Rice Farm

1. Rice Intensification √√√√ √√√√ √√√√ √√√√ √√√√ 5

2. High Yielding Rice Var. - √√√√ - - - 1

3. Irrigation systems √√√√ √√√√ - - - 2

4. Water pumps √√√√ - - - - 1

5. Water Pump (9 hp) - - - - √√√√ 1

6. Water Pump (6.5 hp) - - - - √√√√ 1

7. Hand Tractors √√√√ √√√√ √√√√ √√√√ √√√√ 5

8. Hand Sprayers - √√√√ - - √√√√ 2

9. Threshers - √√√√ - - - 1

10. RMU √√√√ √√√√ - - √√√√ 3

11. Rice Warehouse - √√√√ - - - 1

12. Agro Input Kiosks - - √√√√ - - 1

III. Secondary Crops

13. Soybean Intensification - - √√√√ - - 1

14. Corn Intensification - - √√√√ - - 1

15. Ground nut intensification - - √√√√ √√√√ - 2

II. Horticulture

16. Vegetables - √√√√ - - - 1

17. Chili intensification - - - - √√√√ 1

11

Table 2 Cont’d

No. Activities Banjar (West Java)

Barito

Kuala (S.Kalimantan)

Central

Lombok (NTB)

Rokan

Hulu (Riau)

Jeneponto (S.Sulawesi)

Indonesia (Activities)

IV. Fruits Plantations

18. Citrus intensification - √√√√ - √√√√ - 2

19. Mini Chain Saw - √√√√ - - - 1

20. Salacca intensification - - - √√√√ - 1

21. Oil-palm intensification - - - √√√√ - 1

22. Rubber intensification - - - √√√√ - 1

V. Livestock Farm

23. Cows fattening √√√√ - - √√√√ - 2

24. Medicine for cows √√√√ - - - - 1

25. Male cows / bulls √√√√ - √√√√ √√√√ - 3

26. Cow reproductions √√√√ - √√√√ √√√√ - 3

27. Goat rising √√√√ √√√√ √√√√ - √√√√ 4

28. Medicine for goats √√√√ √√√√ - - √√√√ 3

29. Goat cage - - - - √√√√ 1

30. Duck - √√√√ - - - 1 31. Duck feed - √√√√ - - - 1

VI. Fish Culture

32. Fishponds for Gouramy √√√√ - - - - 1

33. Fishponds for Nila - √√√√ - - - 1

34. Fishpond for Patin - √√√√ - - - 1

35. Fish cage (Karamba) - - - √√√√ - 1

36. Boat machines - - - - √√√√ 1

VII. Others

37. Handicrafts - √√√√ √√√√ - - 2

Number of Activities 12 18 10 11 11 62

Total Funds (Rp 000) 1,665,539 1,353,331 1,402,790 1,226,928 818,349 6,466,936

Acc. Recovery Fund (%) 64% 54% 59% 92% 47% 63%

Source: VPA survey, 2007 15. Capacity Building of Stakeholders. Capacity building has been a priority component of the project and over two-thirds (68%) of the project expenditures have been for training and other capacity building. All together, over 22,700 persons have been trained or participated in workshops sponsored by the project. Besides the FG members (21,086) and district officers (930), this included officials from Provinces (91) and Ministry of Agriculture (132). B. Implementation Efficiency and Management

16. Efficiency. Despite the long gestation period at the beginning, the project is completing all the main activities planned on time in line with the revised implementation schedule. The extension of the project period by one year should not a surprise, given the complex nature of the project, addressing capacity building of FGs with membership of 1,400 households and five district support teams as well as guiding, coordinating and supervising them in the implementation of 62 micro-project activities, all located in five provinces in the major island of the country with long distance between

12

them. Furthermore, from qualitative viewpoint, the implementation has been satisfactorily. The training activities have generally been balanced in subject coverage and appropriate in methods used, centred on farmer field school approach. It also succeeded in assisting the FGs in articulating their development needs, translating them into appropriately designed tangible development activities, and implementing them through participatory but disciplined process with FGDP and RFS. The quality of work in the field observed by the evaluation team (e.g. rehabilitation of irrigation channels and their operation and maintenance) was generally of high standard. Thus, this aspect of project implementation has been very satisfactory. 17. Project management. The quality of management has been high throughout the implementation period. It started by re-interpreting the vague project objectives in the project document into a more meaningful and coherent approach. It managed to ensure coherence and technical soundness in planning and implementing the five sub-projects (sites) while providing varied package of assistance focused on the different needs and constraints. It is also a credit to the project management that the project served as a pilot for three other SPFS projects in the development of such tools and FGDP and RFS. The PMU staff has been hard working, and well led by the NFM and NPD, with support from the RSC. C. Support Given by the Three Parties

18. The Government has generally provided strong and constructive support to the project, both at the policy level and operational levels. In particular, the Agency for Food Security has given strong support in coordinating the project implementation with provincial, district, sub-district authorities, which has been essential for its implementation in the decentralized administrative structure. It also supported the project implementation at local level by coordinating various activities under the related government programmes/projects with project activities (e.g. workshops and training activities). Most importantly, it has closely monitored the project implementation results and associated the project in the development and execution the national food security programme for “village self-reliance”, launched in 2006. The Donor has similarly provided pro-active support, including agreement with the extension of the project period with additional funds (US$ 299,500) – it also channelled US$ 300, 000 through the project for the implementation of emergency assistance (for Tsunami relief in Aceh and Earth quake relief in Java). FAO has provided useful support to the project implementation, including operational support by the FAOR and technical support by the RSC and advice by the staff of the Regional Office and the Headquarters. However, this support has not always been well organized and consistently given over time and across various units involved.

13

IV. Assessment of Emerging Results

A. Development of FGs

19. The 36 FGs have made good progress in evolving as self-reliant groups, marked by strong sense of ownership among the members. An assessment by the PMU of FGs in terms of “group unity”, “leadership” and “RFS management” indicated the following. Overall, the 36 FGs were assessed to have reached “better than satisfactory” or “good” status, with only one FG considered on the borderline of being “less than satisfactory”. In particular, 19 of the FGs (over half) were judged “good” with sustainable prospects and 16 being between “satisfactory” and “good” but with some uncertainty on sustainability. On the whole, FGs were judged stronger in the area of group unity, and to somewhat lesser extent in group leadership. However, the PMU assessment on capacity to manage RFS was more mixed (12 of 36 FGs or one-third being considered not satisfactory), and this is one area where many FGs can benefit from further capacity building support. Comparing the five sites, Rokan Hulu (Riau) and Central Lombok had higher assessment scores, followed by Banjar, and Barito Kuala, and Jeneponto: the main reason for the poor mark on the FGs in Jeneponto was that most of FGs there (6 out of 7) were judged as unsatisfactory in the capacity to manage RFS.

20. Capacity building in FGDP and RFS has been a priority area of empowerment of FGs, but this has proven to be a challenging task. There seems to be particular difficulty with mastering the RFS as financial aspects are very new subject for poor villagers. This is also reflects the difficulty of building confidence in each other among the members, including the RF committee itself, which in some cases, poor financial outcome of FGDP activities has exacerbated. Nevertheless, as shown in Table 2 above, repayment under the RFS (payments into the RF against the original inputs and repaying the credits drawn from the RF) has been generally reasonable at the average rate of 63%. This rate varied between 92% for Rokan Hulu and 47% for Jeneponto. The contrast between the two extreme performances seems to reflect two salient differences between the two sites: (i) strength of FGs -those in Rokan Hulu are assessed as the best of all FOs while those in Jeneponto are seen as the weakest; (ii) level of household income – the two sites have lowest rates of growth (37% for Rokan Hulu and 53% for Jeneponto, but the former has much higher level of income, Rp 49.5 million vs Rp 11.4 million). In fact, there seems to be a strong co-relation between the strength of FGs and the rates of repayment for RFS.3 Obviously, these rates could be raised through further training of FG members. However, one issue relating to RFS frequently raised by FG members is that the amount of resources available in RFS (repayments plus savings of 10% from the profits of FGDP activities) is not only too low but does not grow fast enough, as the original project inputs

3 For example, the ranking of five sites in terms of RFS repayment rate is Rokan Hulu>Banjar>Central Lombok>Barito Kuala>Jeneponto while that in terms of FGs’ strength is Rokan Hulu>Central Lombok>Banjar>Barito Kuala>Jeneponto. While Central Lombok compares well in these criteria, its household income level (Rp. 7.6 million) is the lowest among the five sites, even lower than Jeneponto (Rp.14.3 million).

14

for RFS have been fairly modest (and high returns on investments are not always feasible). B. Capacity Building of DSTs

21. Generally, the DSTs have been strengthened over the last five years, and while there is a variation in their performance level, they have been able to provide useful support to the FGs in their district. Their performance is well appreciated by the local authorities and they are regarded as an essential source of technical support to planning and implementation of similar future programmes in their districts. For example, the governors of both sub-districts visited by the final evaluation team (Banjar and Barito Kuala) stressed this point, saying that the DST staff, with their experience and competence, would serve as the core team of the local staff in extending the project approach in their districts. As after the termination of the project, many of the DST staff can be expected returns to their local services and play key roles wherever opportunity is available, in the implementation of the programme for village self-reliance. However, in some sites, especially in Jeneponto, further effort appears to be needed to strengthen and maintain competence level of the DST.

C. Improving Agricultural Productivity

22. The aim of improving the productivity in each of the project sites have been pursued through the combination of developing a suitable farm production systems and introduction of appropriate technical packages adapted to each location and the specific farming system. Thus, as indicated in Table 2 above, project interventions were targeted on selected production aspects, and following systems have been established:

• Banjar - intensive irrigated farming (three cropping of rice and secondary crops) and livestock development,

• Barito Kuala – improved irrigation farming (primarily one crop of rice and/or vegetables, due to tidal water intrusion) and fruit plantations, supplemented by livestock, aquaculture production and handicrafts,

• Central Lombok – improved upland farming systems (basically rain-fed) with emphasis on rice, secondary crops and especially livestock production, supplemented by handicrafts,

• Rokan Hulu – diversified production systems with emphasis on rice estate crops/fruits and livestock, supplemented by cash crops and aquaculture,

• Jeneponto – mixed systems for irrigated rice production, livestock and coastal fisheries.

23. The evolution of productivity improvements, especially for crops, in selected sites can be seen in Table 3. Firstly, areas planned increased in all four sites, but by variable extents. Banjar gained most, with a total of 525 ha for rice (325 ha) and for secondary crops (200ha), while others achieved less area expansion - Barito Kuala 100 ha (for rice intercropped with citrus), Jeneponto 70 ha (50 for rice and 20 for other crops), and Lombok 40 ha (for secondary crops). Secondly, although data are missing for some crops, yield levels increased in all sites by some 25% for rice (season one), and also by 25% for

15

secondary crops in Jeneponto. In water management, the introduction or re-activation of Water User Groups (WUGs) in these sites helped improve the operation and management of the irrigation systems. Improved water management has enabled WUGs to collect fees, and some in Banjar have been able to accumulate as much as Rp 4.8 - 1.4 million (these had no such group funds before the project).

Table 3. Impact of Irrigation System Improvement

Paddy Season one Paddy Season two Secondary Crops Project Sites Area

(ha) Yield (t/ha)

Area (ha) Yield (t/ha)

Area (ha)

Yield (t/ha)

Banjar B* 900 4-5 800 4 400 0.5

After* 1125 5-6 900 4.5 600 Na

Barito Kuala B*

389 2.0 NA NA NA Na

After* 489** 2.5-3.0 NA NA NA Na

Lombok B*

193 NA Tobacco NA 200 2.0

After* 193 NA Tobacco/soybean*** NA 240 2.5

Jeneponto B*

150 3.5 30 (Maize) 4.2 NA Na

After* 200 4-4.5 50 (Various crops)****

NA NA Na

* “B” indicates before the project and “after” as of 2006 ** Introduction of intercropping of rice in citrus *** Crop rotation of tobacco and soybean introduced **** Vegetables (e.g. chili, onion) introduced with tube well irrigation 24. Livestock production is another area where the FOs have achieved considerable progress. Whereas these FOs had no significant livestock activities, by mid-2007, they have succeeded in increasing the number of main livestock animals as follows: nearly 800 cattle, over 2,200 goats, 18 horses (only in Jeneponto) and 274 ducks (Barito Kuala only). In particular, the FOs in Lombok site now own 442 cattle and 667 goats, while those in Banjar have 77 cattle and 900 goats. One very notable development has been in Banjar where a thriving goat milk production and processing industry has been started. The ten FGs acquired a total of 360 goats (240 female), and have increased the herd to over 820 (650 female) with some 12% in lactation, today producing some 45 litre of fresh milk daily. A small processing factory was established (supported by the sub-district government) and operated by a cooperative of FGs, selling some 5,500 litres of processed milk in 2006. The milk product is now being registered with a trade mark and being sold to grocery stores in the district.

25. Similarly, aquaculture has taken off as a newly organized activity by the FGs in three sites. In Banjar, production of gouramy fish has reached nearly 15,000 tons in the first half of 2007: in Barito Kuala, production amounted to 7,500 tons of catfish and

16

11,600 tons of tilapia, while 5,600 tons of carp was produced in Rokan Hulu. In many cases, aquaculture also provided employment to landless or un-employed in these villages, besides income to the FG members concerned (in the three sites, aquaculture is estimated to have provided a total net income of Rp 69.8 million in 2006.

D. Contribution to Improved Food Security and Income Generation

26. Food Security. It has not been possible to take direct measurement of food security indicator such as amount of food intake in calories. However, there are several reasons for believing that the project has had an important positive effect on food security of the target groups (FGs members’ households). Firstly, production of rice and other food produces has increased among the FGs at each project site, and it can be assumed that a good portion of the production increment has been consumed by the household, in the first instance, with the surplus being marketed. Diversified production also meant that the FGs households have gained access to more diverse food commodities, especially animal protein from expanded livestock production and aquaculture as well as vegetables and fruits. Secondly, increases in income from various food and agricultural production as well as from income earning activities provided greater possibilities for buying additional food items as necessary. In fact, household income of the FG members increased substantially (see below), and the survey carried out by PMU show that on average, household expenditures increased by some 61% between 2001 and 2006 (to Rp 9.9 million per year). As share of total household expenditure in 2006, food accounted for 69%, slight decline from 70% in 2001. The same survey also indicated that the FG members considered improvement in their family food security as one of the major benefits of the project. 27. Income Generation. Table 4 summarizes changes in household income in the five project sites between before (2001) and after (2007) the project intervention. Overall, the average total household income increased by 58%, and at much higher rates for Barito Kuala (127%) and Central Lombok (115%): even the lower rates for Jeneponto and Rokan Hulu were substantial at 53 and 37%. It is pertinent to note that in all sites, on-farm income had the highest rate of increase, higher than off-farm or non-farm incomes, and given the preponderant share of on-farm income in the total at all sites, it is clear that the rapid rise in on-farm income contributed to the overall rise in household income in project villages. The socio-economic survey undertaken by the PMU looked at changes in non-project households in the comparable villages and found that the income increases in FG household were higher than their counterparts in non-project villages (by about 10%). Thus, it is likely that the project indeed contributed materially to the rise of household income at the five sites.

17

Table 4. Annual Household Income of FG members (before and after, in Rp’000)*

District On farm income Off-farm income

Non-farm income

Total income

Banjar

Before 2,808 625 2,567 5,960

After 6,466 (130%) 812 3,636 10,913 (83%)

Barito Kuala

Before 5,271 192 1,015 6,477

After 12,594 (139%) 271 1,820 14,683 (127%)

Jeneponto

Before 7,431 150 1,783 9,365

After 11,412 (54%) 257 2,614 14,282 (53%)

Cent. Lombok Before 1,815 255 1,458 3,529

After 4,989 (175%) 615 1,967 7,571 (115%)

Rokan Hulu

Before 23,310 5,137 7,589 36,035

After 32,210 (38%) 5,724 11,566 49,500 (37%)

Average (5 sites)

before 8,070 1,232 2,853 12,153 After 13,416 (66%) 1,480 (20%) 4,298 (51%) 19,193 (58%)

* Figures in parentheses mean percentage difference between before and after. 28. The same survey mentioned above also showed that the FG members felt their livelihood had improved significantly, including better income, job opportunity, and food consumption as well as better hygienic and health standards. At the same time, the FG members expressed (i) strong self-confidence, (ii) sense of ownership of FG and institutional cooperation, and (iii) appreciation of the importance of technology and saving habit as well as business orientation to work, and education and gender equity. This indicates that the FG members have experienced a clear change in mind-set for greater self-reliance and initiatives. E. Sustainability of Project Results

29. In general, the prospects for sustainability of the project results are quite promising for several reasons:

a) the great majority of FGs has developed well, with strong sense of ownership and demonstrated capacity for self-management. While some FGs (especially those in Jeneponto) have not yet achieved such a sustainability capacity, most of them are developing towards sustainability - the problem with the FGs in Jeneponto appear to involve much deeper issues of political and cultural nature, perhaps beyond the influence of this project;

b) they have shown increasing capacity for planning and undertaking development activities of their choice, generally showing good success in

18

adopting and applying technology and business capacity. Although some FGs suffered some setbacks, like some FGs in Central Lombok with their goat rearing activity, they managed to overcome the difficulties;

c) they have also made considerable advances in improving productivity through the adoption of improved technologies and practices as well as in diversifying their production systems. Thus, they have succeeded in initiating the process of changing the farming systems, taking the advantages of their agro-ecological potentials and addressing key constraints;

d) above all, successes in their production activities have enabled them to increase household income and strengthen their agricultural asset basis, including greater areas for cultivation with irrigation, diversified production patterns, animals, machineries and equipment. The rising income has been expanding their financial assets individually and collectively – individually, their disposable net income has increased, while they have been contributing to expanded resources of RFS through repayment and group savings. However, the repayment performance has varied considerably between FGs, and for some FGs there is a much room for improvement in this area;

e) institutionally, FGs have developed closer cooperative relationship with the DSTs and other local service providers, in many cases with strong policy support from the local authorities. At the same time, local officials also benefited from capacity building support provided by the project, contributing to the improved effectiveness of agricultural extension and other services through more participatory approaches;

f) in particular, some local governments are taking strong interest in replicating the project approach in their respective sub-districts. For example, the mayor of Banjar told the final evaluation mission that he is initiating a food security programme applying the project model starting next year in the sub-district – initially the programme would cover 24 villages with investment of Rp 100 – 200 million per village. The deputy -Governor of Barito Kuala also expressed strong support to replicating the project approach, possibly in coordination with the national programme of village self-reliance (Desas Mandiri Pangan - DMP), started by the Ministry last year; and

g) finally, at the national level, the Ministry of Agriculture through the Agency of Food Security has given strong policy support to promoting food security among the poor rural villages, and as noted above, initiated in 2006 the DMP programme, which has a strategy and approach very similar to that of this SPFS project. The national programme, which is entirely funded by the Ministry budget (US$ 10 million for 2007), is now under way in 604 villages in 180 districts. The Agency’ plan is to incorporate the PMU and the villages covered by the project into the national programme after its termination in September this year.

19

V. Main Conclusions, Issues and Recommendations

Conclusions and Issues

A. Conclusions

30. Project’s Relevance, Efficiency and Effectiveness. Conclusions on these aspects can be summarized as follows:

• Relevance of Project Objectives and Approach – despite some weaknesses in the original project document, the objectives have been broadly relevant to the national effort for food security among the rural poor. However, the re-articulated project approach developed in the initial implementation stage has provided a coherent and appropriate approach, more clearly aligned to the emerging policy orientation strategy (emphasis on participation and decentralization). In particular, the project approach provided opportunities for the target groups to shape the project interventions to address their priority needs through participatory identification, planning and implementation processes. In this way, the target groups as organized in the FGs have grown strong sense of ownership on the project. The other important elements of the approach included the combined use of the FGDP for planning and implementation of micro-projects, formation of RFS to build FGs’ own resources and building the collaboration of local service providers to FG activities through DSTs. Finally, the approach developed by this project has been shown to be also appropriate and applicable to the other three SPFS projects in Asia;

• Implementation Efficiency – this has been quite satisfactory, including the quality of project management. Not only implementing largely on schedule a very complex project operating with 36 FGs located in five districts and provinces (actually across the Indonesian archipelago), the project management played, with support of the RSC, the key role in developing and operationalization of the key tools used by three other SPFS projects;

• Effectiveness in Achieving the Results – generally, FGs have made good progress towards achieving self-reliant management capacity and considerable improvement in raising food and agricultural productivity. In the process, they have been evolving farming systems adapted to their agro-ecological conditions. They have been successful in applying much of new technical packages with the benefits of new irrigation systems, and some cases they have initiated innovative activities, such as the goat milk production and processing activity in Banjar. With good progress in earning income, many households and FGs have strengthened their capital asset, individually and collectively (through RFS), for improving their productivity further. Similarly, the local DSTs and officers have been strengthened in their capacity, many with strong interest in working with farmer communities;

• Sustainability of Project Results – many aspects show strong prospects for sustainability, including the institutional arrangements established by the

20

project (the majority of FGs and DSTs, and active participation and support of national and especially the local authorities) as well as the use of much of the improved technical packages introduced. In particular, the policy level support provided by the Agency for Food Security (initiation of the DMP Programme) has added a significant impetus for sustainability;

• Cost-effectiveness – this is a complex and controversial concept, especially in quantitative terms. The project cost per FG household (the total project cost minus FAO support cost divided by the number of households) works out to be US$ 1,942, which is not low. However, qualitatively, the outcome of the project is highly satisfactory, probably as good as practically feasible and certainly much better than many projects of similar nature and scope. Further, the role played by the project in developing the methods applied by other three SPFS as well as its contribution to the Government’s DMP Programme should be taken into account. Thus, on the whole, the project has many cost-effectiveness elements;

• Gender Aspects – while this aspect was not particularly highlighted in the approach, the project has worked, closely and in a participatory manner, with women and men of the FGs, taking into account of their functions, roles and their local culture. In fact, FGs have many women groups, which have been actively involved in livestock activities, especially in Banjar and Central Lombok, and some women members have played leading roles in their FG activities.

31. Towards a Model for Participatory Rural Development and Food Security. The project approach has proven to be effective in promoting self-reliance, empowering the rural poor, and improving their productivity, while maintaining a balance between its focus on the poor but also collaboration with the villagers and local service providers for rural and agricultural development. As a pilot project, its performance and achievement show much to recommend itself as an effective model, but verifying the project effectiveness as a model would require a systematic analysis of not only the achievements but also the process of how the results have been achieved, including the specific contribution of the project approach. This has not yet been done, and for this purpose, it is desirable to conduct such an in-depth and systematic assessment, perhaps in a few years after the project termination.

32. In the meantime, following aspect of the approach are noteworthy:

• Participatory approach for empowerment and self-reliance of the target groups (self-managing Farmer Groups);

• Disciplined approach to enable farmers to learn collectively to plan, implement and manage their production activities (Farmer Group Development Plans), i.e. not only learning to use individual technologies but also how to run farming like a business;

• Stress on building farmers’ own financial asset through the Revolving Fund System (saving and investment);

• Provision of access to new technologies and better practices for improved productivity through collaboration with the local service providers (organizing

21

and training the local service staff as special support teams to work with farmer groups);

• Ensuring good working relation with the government and other stakeholders at various levels in support of the rural development;

• Its clear and coherent design, with a basis for sharing its experience with other programmes and projects.

33. Critical Factors Affecting the Project Performance. Following factors have been particularly significant:

a) positive factors – (i) coherent and relevant project approach, (ii) good reception by the target groups (members of FGs), including their strong sense of ownership of their FGs, (iii) good collaboration between the FGs and local DSTs, (iv) professional and efficient project leadership and management team, especially the PMU, (v) good technical and policy support from the RSC, Governments, Donor and to a lesser extent FAO, (vi) an attempt made during the final year to assess project results and impact (Vectorial Project Analysis) ;

b) negative factors – no major weaknesses exist, but some aspects could have been improved for better performance, such as: (i) complexity in the project scope, especially in geographical coverage, (ii) provision of a preparatory phase for one-two years for developing an operational approach for project implementation, and (iii) more systematic monitoring throughout the implementation period.

B. Emerging Issues and Lessons

34. Issues. As the Ministry has proactive in supporting the project in its framework for addressing food security and inasmuch as it has recently initiated the DMP Programme) with an approach very similar to the SPFS project, no pressing issues exist in terms of follow-up to ensure continuity and sustainability of the project results. However, there are some issues needing attention:

a) issues for immediate attention following the project’ closure – these include:

• the need to maintain the competence and experience of project PMU as a resource for further application in planning and implementing similar projects and programme (in fact, it is under consideration by the National Project Director to integrate the core PMU staff in to the national team managing the DMP Programme);

• how should the project FGs be integrated into similar programme or project for continued development (especially for these FGs which are still to reach sustainability), particularly in relation to the DMP Programme?

b) issues for longer-term consideration:

• some local governments are interested in replicating the experience of FGs in their administrative areas (e.g. Kota Banjar, Barito Kuala), but how should these activities be linked to the Desa Mandiri Pangan Programme? Similarly, if these initiatives do not become part of the

22

national programme, how can adequate technical and managerial capacity be ensured for their effective implementation?

• Whether part of the national programme or by itself, there is a need to consider possible refinements and adjustments in the project approach for developing FGs, including such points as (i) clarifications on the exit strategy for the approach (especially in terms of institutional maturity of FGs for sustainability), (ii) how to assist their future development by expanding activities with high values-added (post harvest activities, processing and marketing areas, on or off farm)?

• The SPFS project has collected lots of data and information, but these have not been adequately exploited to assess in depth the project performance and achievements, and to draw lessons for future, the rich information should be analyzed.

c) lessons. These include the following:

• for sustainability, it is imperative to empower the rural poor for their self-reliance and self-management through some form of group formation, at least initially;

• similarly, for the above purpose, holistic approaches are required, combining empowerment of the target groups and enabling them to learn to become more productive through access to technological solutions, inputs required, financial means, marketing and reliable technical advice;

• it is essential to have strong and well motivated substantive leadership for the project in the form of PMU and at the local level well organized local service providers (like DSTs);

• working with the rural poor for food security and poverty reduction requires systematic capacity building of the stakeholders, and over a long time;

• for complex projects or those intended to serve as pilot function, it is very important to provide them with a functional monitoring system to support the project management for informed in-course correction as well as to facilitate in-depth assessment of the effectiveness at the terminal stage.

C. Recommendations

35. For immediate follow-up action in conjunction with the project termination, following are suggested:

a) It is strongly recommended that the Agency for Food Security transfer the PMU staff to its programme staff working for the national programme (Desa Mandiri Pangan - DMP) as an important technical resources for planning and implementing the programme;

b) It is similarly recommended that;

• the FGs formed by the project be incorporated into the DMP Programme immediately so that future development of these FGs takes place with coherent technical and institutional support;

23

• those FGs requiring further support be given a priority attention so that they may achieve sustainability as soon as possible, and that those FGs already with maturity be given a role of acting as resource points to support other villages in the programme; and

• within the framework of the DMP Programme, the evolution of project FGs be monitored to study prospects for their sustainability – this would help gain insights into the conditions for defining an exist strategy under the DMP Programme.

36. For longer-term follow-up action, following are suggested for consideration:

a) Some local governments have expressed interest in replicating the experience of the SPFS project as noted above, and it is important that such local initiatives are linked with the national programme to ensure consistency in the approach, cost-effectiveness and synergy, and sound institutional and technical support basis for such local projects. Thus, it is suggested that the Agency for Food Security provides an opportunity to identify interest of local governments and to coordinate various local initiatives within the framework of the DMP Programme;

b) Support to rural development, while needed to be sufficiently long-term, must terminate at some point and it also requires constant efforts for improvement. Thus, the Agency for Food Security should study ways and means for;

• deciding the exist conditions for withdrawing the programme support to a particular villages, and

• promoting expanded development of rural communities beyond the simple poverty line so that these communities develop full capacity for vigorous rural development with strong economic capacity. In particular, the issues of promoting agriculture and rural based enterprises with high added-value (involving agro-processing, marketing and other post-harvest processes) need attention;

c) To strengthen the empirical basis of its programme, it would be important to analyse the experience of related programmes and projects for drawing strategic lessons and identifying key issues. As an initial step, it is suggested that because of the wealth of data available, the experience of the SPFS project be assessed systematically with focus on how the project approach and design might have contributed to achieving the planned results, i.e. analysis of assumed cause and effects of factors involved in food security and poverty reduction. It may be particularly useful to conduct such an assessment in one or two years after the termination of project support.

24

Annex I. Terms of Reference (Final )

Joint Final Evaluation Mission

by

Government of Japan, FAO, Government of Lao PDR, Indonesia,

Sri Lanka and Bangladesh

GCSP/LAO/011/JPN, GCSP/INS/073/JPN, GCSP/SRL/ 049/JPN and GCSP/BGD/033/JPN

1. Background

The projects addresses the problems of food security and malnutrition of rural and peri-urban communities of different Agro Ecological Zones (AEZ), and increased involvement of target beneficiaries at all levels of decision-making. The projects are intended to demonstrate effective local and improved technology for water control, intensification of cultivation and diversification of farming systems and carry out capacity building for the planning, implementation and monitoring of interventions, all of which is aimed at achieving self reliance for sustainable development. The development objective of the SPFS projects is to assist in improving national, household/individual food security on an economically and environmentally sustainable basis by rapidly increasing food production and productivity, reducing year to year variation of production and improve access to food as a contribution to equity and poverty alleviation. SPFS projects funded by the Government of Japan namely Lao PDR (GCSP/LAO/011/JPN), Indonesia (GCSP/INS/073/JPN), Sri Lanka (GCSP/SRL/049/JPN) and Bangladesh (GCSP/BGD/033/JPN) have been implemented consecutively since 2001. The allocated total budget, the EOD and the NTE of each project are determined as below: Table 1. Project budget, EOD and NTE

Project symbol Total budget (US$)

EOD NTE

Lao PDR GSCP/LAO/011/JPN 2,900,279 23/05/01 22/05/07 (31/08/07)4

Indonesia GCSP/INS/073/JPN 3,099,637 17/09/01 16/09/07

Sri Lanka GCSP/SRL/049/JPN 1,812,518 01/03/02 30/06/07 (31/07/07)

Bangladesh GCSP/BGD/033/JPN 3,296,075 01/07/02 30/06/07 (31/08/07)

The target beneficiaries of the project are primarily the poorest of the poor and the most disadvantaged segments of the population in the selected project villages. They belong to the landless, marginal, small and medium categories with particular emphasis on woman and other disadvantaged groups.

4 NTE of Lao, Bangladesh and Sri Lanka will officially be extended after approval by the donor. All the budget revision will be treated by budget neutral extension.

25

Table 2. Beneficial HH and FG

The SPFS Projects are directed by Steering Committees in Indonesia, Sri Lanka and Bangladesh, and a Task Force in Lao PDR. The project review mechanisms undertaken thus far include: one Consultative Review Mission in Bangladesh and Midterm Evaluations in each of the four countries, as per the schedule below: Table 3. Project Evaluation Histry

MTE/CRM Time Mission member

Lao PDR MTE Feb 2004 Bernd Bultemeier (PBEE): Mission leader Toshimitsu Nakaya (Japan) Bounkouang Souvannaphanh (Lao)

Indonesia MTE Oct 2003 Masa Kato (PBEE): Mission leader Takahito Misaki (Japan) Arif Hariyana (Indonesia)

Sri Lanka MTE Oct 2004 Robert Moore (PBEE): Mission leader Hiroshi Suzuki (Japan) D.G. Samarasinghe (Sri Lanka)

CRM Oct 2003 R.B. Singh (FAO): Mission leader Minoru Nakano (Japan) Shin Imai (FAO)

Bangladesh

MTE May 2005 Daniel Shallon (PBEE): Mission leader Hiroshi Okudaira (Japan) Mohd. Sher Ali (BGD) Nasrin Naher (BGD) Shohelur Rahman Khan (BGD) Mohd. Mizanul Haq (BGD)

MTE : Mid-term evaluation CRM : Consultative review mission

Each of the SPFS programmes has adopted the Farmers Group Development Plan (FGDP) and Revolving Fund System (RFS) as a major function. The FGDP is a method to analyze whether proposed interventions are technically feasible, economically viable and environmentally acceptable. RFS is another tool for assessing household level project sustainability.

5 Total beneficial village in Indonesia is increased up to 25 villages. 6 Total beneficial village in Sri Lanka is increased up to 49 villages.

Beneficial HH Province District Village Farmers group

Number of FGDP

Lao PDR 1,283 5 8 11 8 265

Indonesia 1,759 5 5 20 (25)5 36 150

Sri Lanka 5,234 6 17 17 (49)6 17 250

Bangladesh 4,600 6 Administrative

Divisions

16 21 21 600

Total 12,876 22 46 69 82 1,265

26

Project impact assessments have been conducted in each country. The results of these impact studies will serve as an important input to the final evaluation and will be made available to the team before the start of the evaluation.

2. Purpose of the Evaluation The SPFS programme is believed to have generated interest in each of the participating countries and there are indications that there is a desire in each country to institutionalize and internalize the SPFS concepts into government systems linking them with National Programmes for Food Security (NPFS). The present evaluation, scheduled as all of the projects come to a close, is intended to make an in-depth assessment of the progress made propose recommendations for further development of the SPFS programme to each government involved, as well as to FAO and the donor as necessary.

3. Scope of the Evaluation The joint tripartite evaluation mission (hereinafter refer to as “the mission”) will for each project assess the: a) Relevance of the project to development priorities and needs of the country. b) Clarity and realism of the project's development and immediate objectives, including

specification of targets and identification of beneficiaries and prospects for sustainability. c) Quality, clarity and adequacy of project design including:

• clarity and logical consistency between, inputs, activities, outputs and progress towards achievement of objectives (quality, quantity and time-frame);

• realism and clarity in the specification of prior obligations and prerequisites (assumptions and risks);

• realism and clarity of external institutional relationships, and in the managerial and institutional framework for implementation and the work plan;

• cost-effectiveness of the project

d) Efficiency and adequacy of project implementation including: availability of funds as compared with budget for both the donor and national component; the quality and timeliness of input delivery by both FAO and the Government; timeliness and quality of activities; managerial efficiency in dealing with implementation difficulties; adequacy of monitoring and reporting; the extent of national support and commitment; and the quality and quantity of administrative and technical support by FAO from Headquarters, the Regional Office and from coordination projects (GCP/RAS/JPN/180 and GCP/RAS/JPN/182).

27

e) Actual and potential project results, including a full and systematic assessment of outputs

produced to date and progress made towards achieving the immediate objectives:

• The communities develop self-reliance in diagnosing opportunities and constraints, and testing and implementing practical solutions to achieve sustainable food security.

• Strengthening institutional capacity all levels.

• The production and productivity of rice and other crops of resource poor farmers increased crop intensification through improved water control and integrated nutrient management.

• Promote farming system diversification and nutrition and energy conservation awareness and develop location specific integrated farming system of rural house hold especially women and vulnerable groups to enhance production, income generation and availability of nutritional food and to improve nutritional status of the people.

• Promote small scale agro-processing, post harvest handling technology and on-farm and off-farm income generating activities through promotion of community based produce handling methods, supported by marketing, credit and other agriculture support services and identification of constrain.

f) The prospects for sustaining the project's results by the beneficiaries and the host

institutions after the termination of the project, particularly regarding:

• intensive livestock and fisheries production system;

• marketability of agricultural products;

• strengthened Government extension, communication and training services;

• organization and management of farmers groups, including management of village revolving fund;

• general socio-economic improvements (in particular for groups, women and disadvantaged people);

• capacity of Government to sustain the project initiatives (central, provincial and district Government) and;

• replicability of the project results by the respective government. g) The overall cost-effectiveness of the project, especially in terms of serving as piloting

various development activities, including the identification of appropriate approaches and lessons/issues for future.

In conducting the evaluation, the mission will pay particular attention to the following issues: project ownership, partnerships among stakeholders, utility of analytical concepts introduced through the projects (i.e. FGDP and RFS), quality of the impact studies carried out, effectiveness of South-South cooperation, gender awareness, appropriateness of site selection, relevance of planning and training approaches, and viability of the micro credit and revolving fund systems.

28

Based on the above analysis the mission will draw specific conclusions and make proposals for any necessary further action by Government and/or FAO/donor to ensure sustainable development, including any need for additional assistance and activities of the project prior to its completion. The mission will draw attention to lessons of general interest derived from the projects as a whole.

4. Composition of the Mission The mission will comprise:

• A Team leader (Evaluation Specialist), appointed by FAO;

• Specialist in “Rural Development” appointed by the donor (Japan)7;

• Specialist in “food security” in each participating country. In each country, the mission should be briefed and debriefed by the FAO Representative. The mission will debrief RAP, PBEE and TCOS on the mission’s findings and recommendations. Mission members should be independent and thus have no previous direct involvement with the formulation, implementation or backstopping of the projects.

5. Timetable and Itinerary of the Mission The timetable and Itinerary is shown as follows. This trust funded programme is supported by the Government of Japan therefore the courtesy call to Embassy of Japan and discussion is planed. (Annex-I)

6. Consultations The mission will maintain close liaison with the representatives of the donor and FAO and the concerned national agencies, as well as with national and international project staff. Although the mission should feel free to discuss with the authorities concerned anything relevant to its assignment, it is not authorized to make any commitments on behalf of the Government, the donor, or FAO.

7. Reporting The mission is fully responsible for its independent report which may not necessarily reflect the views of the Government, the donor or FAO. The mission will prepare a general report, based on a comparative examination of each of the projects under review, to draw lessons for the future. The general report will also specifically assess the quality and quantity of technical support given to the projects by all sources within FAO. The Mission

7 Perhaps two individuals covering different countries

29

Leader will decide on an appropriate format for the general report. The individual country reports will be written in conformity with the headings as given in the Annex-II. The country reports will be completed, to the extent possible, in the country and the findings and recommendations fully discussed with all concerned parties and wherever possible consensus achieved. The mission will also complete the FAO Project Evaluation Questionnaire. The mission leader bears responsibility for finalization of the report, which will be submitted to FAO within two weeks of mission completion. FAO will submit the report to Government(s) and donor together with its comments.

Annex II.

Table of Contents

Executive Summary

I. Introduction and Background

A. Introduction B. Background

II. Project Approach and Design

III. Assessment of Project Implementation, Efficiency and Management

A. Overall Status B. Main Activities Implemented C. Adequacy of Project Management D. Support Given by the Three Parties

IV. Assessment of Emerging Results

V. Main Conclusions, Issues and Recommendations

A. Main Conclusions (including identification of constraints and issues) B. Main Recommendations

30

Annex II. Mission Itinerary and Persons Met (to be inserted) Mission Itinerary (8-17 August 2007) 8 August – Wednesday Arrival Jakarta 9 August – Thursday Meeting with FAOR Meeting with MOA and PMU Depart for Kota Banjar, West Java (by car) 10 August – Friday Meeting with Mayor of Banjar City Field Visit 11 August – Saturday Return to Jakarta (by car) 12 August – Sunday Depart for Banjamasin, South Kalimantan (by plane) 13 August – Monday Visit for Barito Kuala (by boat and car) Meeting with Deputy Governor of Kabupaten Return to Banjamasin 14 August – Tuesday Return to Jakarta (by plane) 15 August – Wednesday Wrap-up Meeting with DG of Food Security SPFS, NPD and MOA Meeting with SPFS Task Force 16 August – Thursday Meeting with FAOR Meeting at Japanese Embassy Meeting with JICA 17 August – Friday Leave Jakarta Persons Met Mr. Kaman Nainggolan, Director General, Agency for Food Security, Ministry of Agriculture Ms. Ratna Kusuma Dewi, Section Head, Agency for Food Security Mr. Ardi Jayawinata, Centre for Food Availability and Vulnerability (in charge of Desa Mandiri Pangan), Agency for Food Security Mayor of Kota Banjar, West Java Deputy Governor, Barito Kuala, South Kalimantan Mr. Takashi Seo, First Secretary – Agriculture, Embassy of Japan Mr. Takashi Sakamoto, Resident Representative, JICA Mr. Jotaro Tateyama, Assistant Resident Representative, JICA Mr. Man Ho So, FAO Representative Mr. Benni H. Sormin, Assistant Representative, FAO

31

Annex III. Project Budget Status

Project Total

Budgets Expenses Balance

Contributions Received in Advance (Parent)

0

<3,099,637>

3,099,637

3,099,637 3,075,365 24,271

Salaries Professional (Parent) 29,443 29,443 0

Salaries General Service (Parent) 152,340 154,104 <1,764>

Consultants (Parent) 426,104 418,993 7,112

Contracts (Parent) 379,903 376,727 3,176

Locally Contracted Labour (Parent) 18,245 16,706 1,539

Travel (Parent) 187,920 182,085 5,835

Training (Parent) 375,658 362,858 12,800

Expendable Procurement (Parent) 769,410 771,069 <1,659>

Non Expendable Procurement (Parent) 163,041 172,154 <9,114>

Technical Support Services (Parent) 54,545 55,511 <966>

General Operating Expenses (Parent) 186,434 183,056 3,378

Support Costs (Parent 356,595 352,661 3,934

Total Expenditure 3,099,637 3,075,365 24,271

<24,272>