kenya: monitoring and evaluation of agricultural and rural development strategies project...
TRANSCRIPT
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
1/70
A
UNITED NATIONS DEVELOPMENT PROGRAMME
KENYA
MONITORING AND EVALUATION OF
AGRICULTURAL AND RURAL DEVELOPMENT
STRATEGIES PROJECT
KEN/88/034
PROJECT COMPLETION REPORT
by
John Thinguri Mukui (Consultant/Team Leader)
Elphas Nyaga Njeru (Consultant)
and
Nelson Muturi(Office of the Vice-President and Ministry of Planning and National
Development)
July 1995
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
2/70
i
ABBREVIATIONS AND ACRONYMS
ACC-SCN United Nations Administrative Committee on Coordination, Sub-Committee on
NutritionADDO Assistant District Development Officer
AIE Authority to Incur Expenditure
APS Agricultural Production Survey, 1986/87
BRP Budget Rationalization Programme
CBS Central Bureau of Statistics
CSD Child Survival and Development, UNICEF
CSRP Cereal Sector Reform Programme
CTA Chief Technical Advisor
DAO District Agricultural Officer
DDC District Development Committee
DEC District Executive Committee
DFRD District Focus for Rural Development
DHWMES District Household Welfare Monitoring and Evaluation System
DIDC District Information and Documentation Centre
DPU District Planning Unit
DRSRS Department of Resource Surveys and Remote Sensing (Ministry of Planning)
DSO District Statistical Officer
ERD External Resources Department (Ministry of Finance)
FAO United Nations Food and Agriculture Organization
FNPU Food and Nutrition Planning Unit (Ministry of Planning)
GCS Government Computer ServicesGOK Government of Kenya
IDA International Development Association of the World Bank
IFAD International Fund for Agricultural Development
ILO International Labour Office
IMEC Inter-Ministerial Monitoring and Evaluation Committee
ISCO International Standard Classification of Occupations
KANU Kenya African National Union
KNOCS Kenya National Occupational Classification System
M&E Monitoring and Evaluation
MISD Management Information Services Division (Ministry of Planning)MOA Ministry of Agriculture
MOALDM Ministry of Agriculture, Livestock Development and Marketing
MOF Ministry of Finance
MOLD Ministry of Livestock Development
MPND Ministry of Planning and National Development
NASSEP National Sample Survey and Evaluation Programme
NCPB National Cereals and Produce Board
NGO Nongovernmental Organization
NHWMES National Household Welfare Monitoring and Evaluation System
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
3/70
ii
NPC National Project Coordinator
OOP Office of the President
OPS United Nations Office for Project Services
PIMS Project Implementation Monitoring System
PIP Public Investment Programme
PMEC Provincial Monitoring and Evaluation CommitteePPEU Project Planning and Evaluation Unit (Ministry of Finance and Planning)
PROREG Project Registry
RDF Rural Development Fund
RHBS Rural Household Budget Survey, 1981/82
RMRD Resource Management for Rural Development (Ministry of Planning)
RPD Rural Planning Department (Ministry of Planning)
RSCTS Rural Services Coordination and Training Section(Rural Planning Department)
RSD Rural Services Design Project
SIMS Social Impact Monitoring System
SPAD Strategy and Policy Analysis Division (Ministry of Planning)SPSS Statistical Package for the Social Sciences
SRDP Special Rural Development Programme
TPR Tripartite Review (UNDP)
UHBS Urban Household Budget Survey, 1982/83
UNDP United Nations Development Programme
USAID United States Agency for International Development
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
4/70
iii
TABLE OF CONTENTS
ABBREVIATIONS AND ACRONYMS .................................................................................... i
EXECUTIVE SUMMARY ......................................................................................................... 1
CHAPTER 1: BACKGROUND TO THE MISSION ................................................................ 9
CHAPTER 2: BACKGROUND TO THE PROJECT .............................................................. 10
CHAPTER 3: AN INTERPRETATION OF THE PROJECT DOCUMENT ......................... 13
INTRODUCTION ............................................................................................................................ 13
THE CONCEPTUAL FRAMEWORK ............................................................................................ 13
INSTITUTIONAL ARRANGEMENTS ........................................................................................... 14
IMPACT SURVEY DATA ............................................................................................................... 15
PROJECT IMPLEMENTATION MONITORING .......................................................................... 19
CHAPTER 4: IMPLEMENTATION OF IMPACT STUDIES PROJECT COMPONENT .... 20
RURAL HOUSEHOLD BUDGET SURVEY, 1981/82 .................................................................... 20
CEREAL SECTOR REFORM PROGRAMME ............................................................................... 22
MID-TERM REVIEW OF THE RURAL SERVICES DESIGN PROJECT .................................... 23
CHAPTER 5: PROJECT IMPLEMENTATION MONITORING .......................................... 24
BACKGROUND............................................................................................................................... 24
OUTLINE OF THE ACTIVITY....................................................................................................... 24
DATA CAPTURE ............................................................................................................................ 25
SERVICE DELIVERY TO THE PARTICIPATING DISTRICTS ................................................... 26
OUTPUTS ........................................................................................................................................ 28
CHAPTER 6: PROJECT MANAGEMENT ............................................................................ 30
PROJECT COORDINATION ARRANGEMENTS......................................................................... 30
PROJECT FINANCING ................................................................................................................... 31
PROJECT IMPLEMENTATION PROCESS ................................................................................... 32
CHAPTER 7: PROJECT RESULTS AND EFFECTS .............................................................. 35
FINDINGS ........................................................................................................................................ 35
PROJECT INSTITUTIONALIZATION AND SUSTAINABILITY ................................................ 36
CHAPTER 8: THE WAY AHEAD ......................................................................................... 38
INTRODUCTION ............................................................................................................................ 38
MINISTRY OF PLANNING AND NATIONAL DEVELOPMENT HEADQUARTERS .............. 38
CENTRAL BUREAU OF STATISTICS ........................................................................................... 40
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
5/70
iv
ANNEX 1: TERMS OF REFERENCE ..................................................................................... 42
ANNEX 2: ITINERARY FOR THE EVALUATION MISSION ............................................ 45
ANNEX 3: REPORT OF THE FIELD VISITS ........................................................................ 47
MACHAKOS DISTRICT ................................................................................................................. 47
KITUI DISTRICT ............................................................................................................................. 48
BARINGO DISTRICT...................................................................................................................... 49
BUNGOMA DISTRICT ................................................................................................................... 50
KISUMU DISTRICT ........................................................................................................................ 51
NYERI DISTRICT ............................................................................................................................ 52
KWALE DISTRICT ......................................................................................................................... 53
ANNEX 4: DISTRICT INSTITUTIONS PARTICIPATING IN THE PROJECT .................. 55
DISTRICT DEVELOPMENT COMMITTEE .................................................................................. 55
DISTRICT EXECUTIVE COMMITTEE ......................................................................................... 55
DISTRICT PLANNING UNIT ......................................................................................................... 56
DISTRICT INFORMATION AND DOCUMENTATION CENTRE ............................................. 56
PROVINCIAL MONITORING AND EVALUATION COMMITTEE .......................................... 57
ANNEX 5: LIST OF PERSONS INTERVIEWED .................................................................. 58
ANNEX 6: REFERENCES ....................................................................................................... 61
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
6/70
1
EXECUTIVE SUMMARY
INTRODUCTION
1. The project (UNDP KEN/88/034) forms part of the GOK/IDA umbrella activity - the RuralServices Design Project - and was implemented by UNDP under the Fourth UNDP Programme Cycle.
The project is the result of a project preparatory assistance by the UNDP KEN/87/015 (Monitoring and
Evaluation of Impact of Agricultural and Rural Development Strategies). The project (UNDP
KEN/88/034) was to build on the experiences accumulated under UNDP/FAO project KEN/85/001
(Monitoring Impact of Agricultural and Rural Development Projects on Food and Nutrition). Building
on the previous UNDP projects, the project was to develop and institutionalize monitoring and
evaluation (M&E) system for use at the district and central levels to assess the effectiveness of the
agricultural and rural development strategies contained in Sessional Paper No 1 of 1986 on Economic
Management for Renewed Growthand the sixth National Development Plan (1989-93). The specific
aims of the project were:a) To establish an effective and sustainable system for the monitoring and evaluation of
agricultural/rural development activities at the district and central level;
b) To make effective use of already collected survey data and new M&E data for improving the
implementation and design of agricultural and rural development policies and strategies; and
c) To strengthen district-level policy analysis and decision-making capacity in order that
programme/project adjustment and refinement may be undertaken at this level, where
appropriate, in line with the district-focus strategy of Government.
2. The purpose of the evaluation is to determine:
a) Whether the project design was logical and coherent, focusing on the appropriateness of the
institutional framework for the project; whether objectives and outputs were realistic,
measurable and achievable; and the strong and weak points in conceptualization and design of
the project;
b)
Review policy workshops, consultancies and staff development training undertaken under the
project;
c) Select a sample of six districts where M&E system has been established under the project and
review its viability, sustainability and replicability.
d) Assess the impact of the project on the development of a comprehensive monitoring and
evaluation system in Kenya.
e) Articulate the Governments policy in the area of monitoring and evaluation and make
recommendations on future activities, strategies and approaches for implementation as acomponent of the sixth country programme and on a long-term basis.
INTERPRETATION OF THE PROJECT DOCUMENTS
3. At the institutional-level, the Government in 1983 adopted the District Focus for Rural
Development (DFRD) strategy as an attempt to decentralize various management responsibilities to
the districts. Building on the previous UNDP projects in the M&E territory, the project was to develop
and institutionalize M&E system for use at the district and central levels to assess the effectiveness of
the agricultural and rural development strategies contained in Sessional Paper No 1 of 1986 on
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
7/70
2
Economic Management for Renewed Growthand the sixth National Development Plan (1989-93).
4. The agricultural sector strategies included increased crop production, improved research,
study of land use patterns for switch to more profitable farming, improved marketing and storage, and
efficient provision of farm inputs. Rural development sector strategies included rural-urban balance,
strengthening of administrative capacities of lower-level authorities, and promotion of non-farmemployment. The eleven Social Impact Monitoring System (SIMS) questionnaires (social
development, nutrition, health, energy, water, public works and environmental components)
developed under the preparatory assistance to the project were to be used to capture data for
monitoring.
5. The project document stated that the Ministries and Departments to be directly involved in
the project implementation were (a) Ministry of Planning and National Development: Rural Planning
Department (RPD), Management Information Services Division (MISD), and Central Bureau of
Statistics (CBS); and (b) Ministry of Agriculture, Livestock Development and Marketing (MOALDM).
The major responsibility for implementing the district-level M&E project sub-component was vested
in the Rural Planning Department of MPND as the institutional linkage between the project
district-level activities. A newly created division to house the project, Strategy and Policy Analysis
Division (SPAD), would ensure the smooth coordination and implementation of the projects various
components. To facilitate inter-ministerial coordination, an Inter-Ministerial Monitoring and
Evaluation Committee (IMEC) was to be established consisting of participating line ministries. The
division of responsibilities gave MPND a liaison role with the Office of the President, the line
ministries, and districts.
6. The project activities were intended to fall under three major categories: (a) district-level
agricultural production/marketing and household welfare monitoring surveys under the Social Impact
Monitoring System; (b) analysis of existing impact surveys data; and (c) project implementationmonitoring. The three components were jointly supposed to monitor strategies/policies and how
policies are translated into actual projects.
ANALYSIS OF IMPACT SURVEY DATA
7. The already collected survey data included (a) the Agricultural Production Survey (APS),
1986/87, (b) the Rural Household Budget Survey (RHBS), 1981/82, (c) the Urban Household Budget
Survey (UHBS), 1982/83, and (d) the Social Impact Data. The data was supposed to be analyzed, used
in the preparation of policy and analytical reports, and summary results displayed using computerized
mapping software.
8. Some of the 1986/87 APS data was lost. The 1982/83 UHBS data tapes were found in 1993 to
have been overwritten with customs data. It is not possible to judge whether the project would have
carried out these activities if the data was available. However, the project assisted in the transfer of
1981/82 RHBS data from mainframe to microcomputers and in the preparation of a basic report based
on the database. The draft basic report of the 1981/82 RHBS was discussed in a workshop but has not
been finalized. The project successfully completed the baseline study of grain market liberalization.
However, the project has not prepared policy papers based on the grain marketing study.
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
8/70
3
9. The impact studies have been coordinated by SPAD but have not had substantial involvement
of other units of Government e.g. the Ministry of Agriculture, Livestock Development, and
Marketing. There has also been too much delay between submission of draft reports and policy
workshops to discuss the findings, making the reports unusable in the intervening period. It is difficult
for a consultant to do substantial revision on a draft report prepared over two years earlier.
10. The project document also included analysis of social impact survey data. The project
KEN/85/001 finalized the Social Impact Monitoring System (SIMS) and created an enabling
environment for its institutionalization under KEN/87/015 through: installing the information
management system (ARIEL) on the Government Computer Services (GCS) IBM 37 mainframe
computer and in four microcomputers procured under the project. Indicators of social impact included
(a) nutrition and health: food consumption patterns, health status, malnutrition; (b) socioeconomic
status: household demographics, employment, literacy, housing; (c) access to basic social services:
physical access and intensity of use; and (d) food production and food security: crop and livestock
production patterns.
11. The SIMS project component was expected to collect district-level data and prepare reports
based on the findings. During the first Tripartite Review meeting in February 1991, the Ministry of
Planning and National Development stated that it was not aware of the existence of the Social Impact
Survey Data. A more detailed look at the preparatory assistance documents would have assisted in the
identification of the activities that were envisaged under the SIMS project sub-component.
12. A consultant was in 1986 funded by FAO under the project UNDP KEN/85/001 to assist in the
preparation of a sampling frame for the purpose of developing a monitoring system. The activities
performed under the project sub-component included the development of a computer program for
determining the sample size and actual sample selection using the hierarchically nested spatial and
administrative organization of the nation (i.e. from nation, province, district, division, location tosub-location). Other proposed codes were ecological zones and seasonal regimes due to their
explanatoryrole in rural households food security. The project, UNDP KEN/88/034, was expected to
install and manage a computerized mapping system which would show the regional differences in the
impact of sector-wide development policies and strategies. Such data would come from analyzed
survey data, Government allocations of recurrent and development expenditures, population censuses,
etc. No computerized mapping software was installed during the lifetime of the project.
13. The project was expected to assist the Rural Planning Department in the analysis of
infrastructure inventory data. In the first Tripartite Review meeting dated February 1991,
uncertainty was expressed as to whether this exercise should be incorporated into the project. Theproject budget was not used to support the Rural Planning Department in its maintenance of the
infrastructure inventory. The new division (SPAD) created to house the project managed the
substancerather than the process, hence the inability to implement project activities that could
not be transferred from their host departments to SPAD.
14. The responsibility for coordination of mid-term evaluation of the umbrella Rural Services
Design project was vested in the M&E sub-component. The study was commissioned by the Ministry
of Planning and National Development in July 1992 to assess the successes and failures of the 13
sub-projects under the RSD project, and to offer guidelines on the future direction of the
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
9/70
4
sub-components. The RSD impact study was completed in November 1993. The findings of the study
have not been discussed in a policy workshop. However, since the World Bank financed the RSD
impact study and the umbrella RSD project closed in April 1995, the consultant report can be
considered final.
15. On the M&E sub-component, the RSD impact study found out that, despite the late start, thesub-project had made substantial progress in establishing a system for monitoring development
projects. However, a comprehensive system for monitoring and evaluation and for explicitly linking
data collection and analysis with key strategies and programmes outlined in the Sessional Paper No. 1
of 1986 and the sixth National Development Plan (1989-93) had not been developed. In addition, the
impact study recommended the formation of an inter-ministerial steering committee and
identification of one counterpart from the headquarters of each participating line ministry to work in
close collaboration with SPAD on a regular basis in completing development and institutionalization
of the M&E system. The recommendations, if acted on, could have redirected the project to its original
objectives and ensured ownership of the M&E activity by the entire central Government.
PROJECT IMPLEMENTATION MONITORING SYSTEM PIMS)
16. The project was to facilitate data collection at the district-level, to enable the districts to utilize
the information, and serve the data demands at the central level. In February 1991, MPND held a
workshop in Naivasha with the prime objective of introducing basic concepts of the monitoring and
evaluation to key planning and budgeting staff within the ministries who will play the major role in
ensuring the effectiveness and the success of the system. The workshop did not allude to the
extensive literature prepared under the project preparatory activities (KEN/87/015) and the projects
predecessor (KEN/85/001).
17. Efforts to develop and institutionalize project implementation monitoring dates back to theearly 1970s. The Project Implementation Monitoring process had its roots in the Special Rural
Development Programme (SRDP) which was phased into a new national system of district
development planning (Belshaw and Chambers, 1973; Institute for Development Studies, 1973;
Livingstone, 1976;Holtham and Hazlewood, 1976). Before PIMS was taken over by SPAD, submission
of project information sheets was by headquarters of line ministries rather than the districts. The
PIMS project component was viewed as a pilot activity since the source of information was the district
headquarters. The project implementation information sheets largely correspond to the ones used by
the Office of the President in the early 1980s. However, the participating districts have made useful
recommendations on the design of PIMS sheets and the PROREG (Project Registry) computer
program, which could form the basis for the revision of the instruments.
18. The main data gaps encountered in the districts are background data on project cost and
project financial and physical scheduling. District departmental heads stressed the importance of
creating institutional arrangements to facilitate districts to receive original and revised project
documents. The project purchased six computers without printers. Sensitization workshops have been
held to acquaint participants with the project implementation monitoring sheets, but does not cover
technical issues e.g. the project cycle, and the logical framework for M&E of projects and programmes.
19. Two issues of concern on service delivery relate to the efficiency/costs and coordination
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
10/70
5
arrangementswith the headquarters of line ministries and other departments within the Ministry of
Planning and National Development. The project does not coordinate its activities with headquarters
of line ministries and other departments in the Ministry of Planning and National Development. This
reduces commitment of district departmental heads to the project implementation monitoring activity
as there are no sanctions for noncompliance. The project concept was not sold to the District
Executive Committees (DEC) and could not therefore be assumed to have beendelivered
from the
Ministry of Planning headquarters to the districts. Workshops on computer training are not
coordinated with other agencies, e.g. UNICEF which also provides similar computer training in its
Child Survival and Development (CSD) districts.
PROJECT INSTITUTIONALIZATION AND SUSTAINABILITY
20. Throughout the lifetime of the project, the project has not facilitated other departments and
ministries to carry out the project components. The project therefore concentrated on project
implementation monitoring at the district level, but did not finance, say, the Rural Planning
Department to maintain the District Infrastructure Inventory. The lack of coordination between
SPAD and other stakeholders in the project document therefore compromised the number of activities
that could be undertaken by the project, and the success of those activities it undertook.
21. The success of the project implementation monitoring was adversely affected by high staff
turnover due to minimal coordination of the project with headquarters of line ministries (for district
departmental heads) and the Rural Planning Department in MPND (for district development officers).
Second, the lack of institutionalization of the project implementation system within the entire central
Government with clear technical guidelines and sanctions for noncompliance reduces institutional
memory when a participating staff member is transferred. Third, since the project did not cover all the
districts, some staffs trained under the project were transferred to districts which were not in the
projects catchment districts.
22. According to the World Bank staff appraisal report for Rural Services Design project dated
November 1988, UNDP funds will be on a parallel (vis--vis IDA funds) and grant basis in accordance
with standard UNDP terms, and would contribute to primarily the technical assistance and training
requirements of these sub-projects. It is agreed that the Government implementing agency concerned
will execute the sub-projects co-financed with UNDP. In April 1990, the World Bank clarified in an
amendment to the Rural Services Design project that it would open special accounts with Central
Bank of Kenya to cater for the operational costs of all RSD sub-projects in the IDA credit agreement.
However, the UNDP project document states that IDA inputs, to be administered through UNDP,
will cover all equipment and miscellaneous costs. There was therefore apparent misunderstandingbetween the World Bank and UNDP on the mode of administering the IDA contribution to the
project.
23. The project was expected to hire experts, namely, an internationally-recruited Senior
Planner/Data Analyst, consultant on Evaluation Surveys to finalize the analysis of the APS data,
consultant on Agricultural Strategies to work with the ministry of agriculture (MOA) in analysis of
farm management data, consultant on Agricultural Growth Potential to assist MOA in agricultural
commodity analysis, consultant on Multi-Market Analysis to assist in preparation of policy papers on
the bases of analyzed impact survey data, and a consultant on Spatial Mapping. The project did not
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
11/70
6
hire consultants to undertake the abovementioned activities on the basis that there was sufficient
capacity within Government.
24. The chief technical advisor (CTA) was hired in May 1990 to assist in the district-level M&E
activities. His tenure was reduced from three to two years, and the Government counterpart to the
CTA took over the management of the project.
25. Technical training was provided to five officers in SPAD. However, the short courses should
be more appropriately considered as workshops, rather than the advanced training in agricultural
policy analysis and project implementation monitoring envisaged in the project document. The skills
demanded to undertake various activities in the project required higher level training in agricultural
policy analysis and the logical framework for analysis of implementation of strategies/polices and
projects/programmes, especially after it was decided that it was imprudent to recruit national
consultants. The project did not therefore transfer the appropriate capacity to supervise analytical
reports based on analysis of impact survey data, and project appraisal/logical framework as the
theoretical foundations for project implementation monitoring.
26. The tripartite review process was responsive to the changing needs of the project emanating
from the early expiry of project financing from the World Bank. UNDP provided six computers
without printers and with no budgetary provisions for operation (e.g. stationery) and repair and
maintenance of computers. The computers were intended to assist the initial six participating districts
in analysis of project implementation monitoring data. The UNDP non-response on the printers and
expendables raised project costs by probably more than the cost of the facilities due to the need to
transport diskettes from the districts for the tables to be printed in SPAD.
27. The effectiveness of the tripartite review process was somewhat compromised by the quality
of project performance review reports, inadequate follow-up by Government, and UNDP/OPS delayin response to requests by the project. The quality of reporting progress was not adequate, especially
on impact studies.
28. The PIMS sheets were not introduced to the entire central Government through a Treasury
Circular, which is the initial step in introducing systems of Government s budgetary data collection
using administrative budget records. The project information sheets are not known in the
headquarters of line ministries and are only known to departmental heads of participating districts.
The district-level activities are not coordinated with headquarters of line ministries and provincial
planning offices. The involvement of the latter would have been crucial in the institutionalization of
the activity, since the activity was in every province except North Eastern province.
29. The activity is important in the planning process and in assessing the quality of public sector
service delivery. It is, however, important to institutionalize the system in future M&E activities with
the districts. This would guarantee wider participation, and introduce sanctions for noncompliance
among the departmental heads in the participating districts.
PROJECT OUTPUTS AND EFFECTS
30. Analysis of Impact Survey Data: Some of the 1986/87 APS data was lost, while the
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
12/70
7
1982/83 UHBS data tapes were overwritten with customs data. It is not possible to judge whether the
project would have carried out these activities if the data was available. The project assisted in the
preparation of a basic report based on the 1981/82 RHBS database but the draft basic report has not
been finalized. The project successfully completed the mid-term review of the Rural Services Design
Project. The findings of the study have not been discussed in a policy workshop, and the
recommendations of the study were not acted upon. The project facilitated the baseline study of grainmarket liberalization, but did not prepare policy papers based on the study. The project was expected
to assist the Rural Planning Department in the analysis of infrastructure inventory data. The project
budget was not used to support the Rural Planning Department in its maintenance of the
infrastructure inventory.
31. The project was expected to install and manage a computerized mapping system which would
show the regional differences in the impact of sector-wide development policies and strategies. Such
data would come from analyzed survey data, Government allocations of recurrent and development
expenditures, population censuses, etc. No computerized mapping software was installed during the
lifetime of the project.
32. The SIMS Project Component: The SIMS project component was expected to collect
district-level data and prepare reports based on the findings. The SIMS project component was not
implemented, due to a problem in the interpretation of the project document occasioned by little
reference to the documents prepared under the project preparatory phase.
33. Project Implementation Monitoring: Before PIMS was taken over by SPAD, submission of
project information sheets was by headquarters of line ministries rather than the districts. The project
implementation information sheets largely correspond to the ones used by the Office of the President
in the early 1980s. However, the participating districts have made useful recommendations on the
design of PIMS sheets. There are also data gaps in the districts on project cost and project financial andphysical scheduling.
34. Two issues of concern on service delivery relate to the efficiency/costs and coordination
arrangements. The project does not coordinate its activities with headquarters of line ministries and
other departments in the Ministry of Planningand National Development. The project concept was
not sold to the District Executive Committees and could not therefore be assumed to have been
delivered from the Ministry of Planning headquarters to the districts. Workshops on computer
training are not coordinated with other agencies e.g. UNICEF.
35. Overall, the project did not address the broad goals envisaged in the project documents. One ofthe initial failures was the loose link between the vision of the project (as outlined in the project
preparatory assistance documents) and the actual project implementation. The lack of coordination
between SPAD and other stakeholders in the project document compromised the number of activities
that could be undertaken by the project, and the success of those activities it undertook. The three
project activities, namely, district-level agricultural production/marketing and household welfare
monitoring surveys, analysis of existing impact surveys data, and project implementation monitoring,
were jointly supposed to monitor strategies/policies and how policies are translated into actual
projects. Therefore, project implementation monitoring alone did not fulfil the vision of the project.
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
13/70
8
RECOMMENDATIONS
36. The current three-year Development Plan (1994-96) underscores the importance of
monitoring and evaluation of strategies, policies and projects/programmes. The Governments stated
policies in the seventh National Development Plan (1994-96) and the Sessional Paper No 1 of 1994 onRecovery and Sustainable Development to the Year 2010demonstrates a commitment to the centrality
of M&E in the process of economic management. The Mission supports the M&E system as
conceptualized in the project document.
37. However, it is important to increase managerial and technical capacity of the Ministry of
Planning to undertake the enhanced role of M&E under a deregulated economy. Within the
organization of the Kenya Government, one of the major constraints to effective coordination of
planning function is the proliferation of ministries, which brings conflict and problems of horizontal
coordination between ministries. Within the Ministry of Planning, there is need for reorganization of
departments to minimize overlap of responsibilities. The reorganization should ensure that Divisions
are organizationally nested within Departments, i.e. a divisional head reports to a departmental head,
who in turn reports to the Director of Planning.
38. The functional classification of departments should also focus on the Central Bureau of
Statistics vis--vis other departments at the Ministry of Planning headquarters. For example, the
activities undertaken by SPAD (for the UNDP KEN/88/034 project) and the Human Resources and
Social Services Department involve support to the Central Bureau of Statistics to carry out surveys and
analysis of data. The fact that this support is not direct may conceal the strengths and weaknesses of
CBS in a standalone environment and hence delay the implementation of remedial measures that may
be necessary to increase CBS capacity to collect and analyze survey data.
39. The M&E activity envisaged in the current National Development Plan (and which
corresponds to the vision in project UNDP KEN/88/034) requires personnel with background
academic training in: (a) project appraisal, including project cycle and logical framework for project
implementation monitoring; and (b) economic theory and econometrics and their application in
empirical work, i.e. the transformation of theoretical paradigms into real world applications.
Administratively, the head of M&E activity should be a chief economist, so as to have sufficient
authority to coordinate line ministries and other departments in MPND.
40. CBS should undertake the following activities as a matter of priority. First, a technical report
on the NASSEP ought to be published once the inclusion of the currently non-NASSEP districts iscompleted. There is also need to publish the National Statistical Needs Assessment Project report
completed in 1992 (Kenya, 1992b) and the Directorate of Personnel Management s (Office of the
President) Report on the Organization, Operations and Staffing of the Central Bureau of Statistics
(Kenya, 1992c). This would help to focus donor attention on CBS activities, and invite debate on the
institutional developments necessary to strengthen CBS capacity to undertake its role in a more
demanding socio-political and economic environment.
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
14/70
9
CHAPTER 1: BACKGROUND TO THE MISSION
1.1. A UNDP/GOK Mission consisting of John T. Mukui (Consultant/Team Leader), E.N. Njeru
(Consultant) and Nelson Muturi (Government Counterpart) carried out a terminal evaluation of the
UNDP project: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project(KEN/88/034) from Wednesday, March 22 to Friday, May 19, 1995. The Mission interviewed project
staff in the Ministry of Planning and National Development and United Nations Development
Programme, and thereafter travelled to seven district headquarters to interview district development
officers and district departmental heads of line ministries. The Mission received tremendous assistance
from the offices of UNDP, Ministry of Planning and National Development, Ministry of Finance, the
project staff, and the participating districts that were visited.
1.2. The project forms part of the GOK/IDA umbrella activity - the Rural Services Design Project -
and implemented by UNDP under the Fourth UNDP Programme Cycle. According to the Project
Document (November 1989) and IDA Rural Services Design Staff Appraisal Report (November 1988),the specific aims of the project were:
a) To establish an effective and sustainable system for the monitoring and evaluation of
agricultural/rural development activities at the district and central level;
b) To make effective use of already collected survey data and new M&E data for improving the
implementation and design of agricultural and rural development policies and strategies; and
c) To strengthen district-level policy analysis and decision-making capacity in order that
programme/project adjustment and refinement may be undertaken at this level, where
appropriate, in line with the district-focus strategy of Government.
1.3. The purpose of the evaluation is to determine:
a)
Whether the project design was logical and coherent, focusing on the appropriateness of the
institutional framework for the project; whether objectives and outputs were realistic,
measurable and achievable; and the strong and weak points in the conceptualization and
design of the project.
b) Review policy workshops, consultancies and staff development training undertaken under the
project.
c) Select a sample of six districts where M&E system has been established under the project and
review its viability, sustainability and replicability.
d) Assess the impact of the project on the development of a comprehensive monitoring and
evaluation system in Kenya.
e) Articulate the Governments policy in the area of monitoring and evaluation and make
recommendations on future activities, strategies and approaches for implementation as a
component of the sixth country programme and on a long-term basis.
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
15/70
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
16/70
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
17/70
12
2.10. The workshops covered the data collection instruments to be used in the full project
(KEN/88/034). The instruments included (a) project implementation monitoring forms, and (b)
framework for monitoring and evaluation of impact and effects of development strategies and policies.
Item (b) included nationwide surveys conducted under the national sample frame and the proposed
district-level social impact monitoring to be undertaken under the full project using larger districtsample sizes than the national surveys. The workshops also discussed the institutions to be involved in
project implementation e.g. CBS, Department of Resource Surveys and Remote Sensing (DRSRS),
Ministries of Agriculture, Health, Livestock Development, and district-level institutions e.g. District
Information and Documentation Centres (DIDC) and District Planning Units (DPU). A manual to be
used in data collection titled Monitoring and Evaluation of National Development Policies and
Strategies: Illustrative Version(July 1989) was prepared.
2.11. As noted in the sixth National Development Plan (1989-93), Currently, there is no effective
monitoring and evaluation system that can provide information necessary to indicate the extent to
which the process of development programming meets set objectives. In a situation where structural
adjustment programmes are to be implemented based on policy reforms, with the demands emanating
from the District Focus for Rural Development strategy and the envisaged increase in the
responsibility of communities through cost-sharing, the need for monitoring and evaluation of
programmes and projects has become more urgent.The Government prepared the implementation
version of the sixth National Development Plan as a basis for monitoring progress in achieving targets
set out in the Plan. The Government also included the proposed project in the GOK Programme
Review and Forward Budget for the period 1988/89-1990/91 and in the Revised Estimates for
1987/1988. The importance of the project to the Government was therefore patent.
2.12. According to the World Bank Presidents report on the Second Agricultural Sector
Adjustment Operation, one of the experiences with the first ASAO included focusing on financingthe preparatory steps for adjustment (studies, elaboration of policy papers, and restructuring plans)
rather than on implementation. A new initiative under ASAO II was strengthening planning and
management capacity in selected Ministries which deal with major subsector issues, and undertake
strategic studies which cut across subsectors, or have macroeconomic and environmental implications.
The Report further stated that for detailed monitoring it [the ASAO II programme] will depend on
the Development Planning Divisions of MOA and MOLD, and the Policy Analysis and Monitoring
Unit of MPND; the Unit will have overall responsibility for monitoring and evaluation of the key
actions, as part of an ongoing project funded by UNDP and IDA; and the agricultural data component
will be funded by the Monitoring and Evaluation Project, using IDA funds to be channelled to the
Central Bureau of Statistics and MOA. A consultants report was prepared to provide a framework forthe monitoring and evaluation of the Second Agricultural Sector Adjustment Operation (see K.
Altemeier, A Suggested Framework for the Monitoring and Evaluation of the Agricultural Sector
Adjustment Operation - ASAO II, World Bank, July 1990).
2.13. The above summary of historical roots of the project has helped to focus on the initial terms of
reference of the project in both areas of developing systems for district-level monitoring and
evaluation and the impact studies expected under the project. A further analysis of the Project
Document will shed light on the objectives, and the degree of consistency between objectivesand
visionof the project.
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
18/70
13
CHAPTER 3: AN INTERPRETATION OF THE PROJECT DOCUMENT
INTRODUCTION
3.1. The above summary of the background to the project provides a basis for interpretation of thevision of the project. This is important because, for some program components, there was divergence
between the visionof the project (as demonstrated in the projects preparatory activities) and the
objectives(as narrowly defined in the project document). M.E. Harvey, a consultant who worked
under the chief technical advisor (CTA) in the project that preceded KEN/88/034, in his December
1986 report to the FAO complained that the CTA is not getting support needed for the volume of
work that needs to be done. He was involved in all aspects of the project. Who, besides him,
understands what is going on? Presently, there is nobody.The lack of effective counterparts in the
preceding project may have contributed to the lapse in institutional memory.
3.2. This chapter provides the Missions interpretation of the project document. For the projectactivities that were not implemented, the chapter gives details of the data collected and uses of the
data in planning, and monitoring and evaluation.
THE CONCEPTUAL FRAMEWORK
3.3. At the institutional-level, the Government in 1983 adopted the District Focus for Rural
Development (DFRD), as an attempt to decentralize various management responsibilities to the
districts1. Building on the previous UNDP projects in the M&E territory, the project was to develop
and institutionalize M&E system for use at the district and central levels to assess the effectiveness of
the agricultural and rural development strategies contained in Sessional Paper No 1 of 1986 on
Economic Management for Renewed Growthand the sixth National Development Plan (1989-93).
3.4. The agricultural sector strategies included: increased crop production, improved research,
study of land use patterns for switch to more profitable farming, improved marketing and storage, and
efficient provision of farm inputs. Rural development sector strategies included: rural-urban balance,
strengthening of administrative capacities of lower-level authorities, and promotion of non-farm
employment. Annex III of the project document contained a sample of the various agricultural/rural
development strategies and programmes that the project was to monitor and the performance
indicators that would be evaluated. The eleven Social Impact Monitoring System (SIMS)
questionnaires (social development, nutrition, health, energy, water, public works and environmental
components) developed under the preparatory assistance to the project were to be used to capture datafor monitoring.
3.5. The indicators proposed largely focused on efficiency in the agricultural sector, infrastructure
1. There has been debate on whether DFRD was deconcentration or devolution of power.
Deconcentrationis the mere dispersal of central government offices outside the national capital without tangible
transfer of decision-making authority and financial and management responsibilities to the local level.
Devolution is generally understood to mean the transfer of authority for decision-making, development
planning and management to independent or semi-autonomous local governments, the functions and
responsibilities of which are outside the direct control of national governments.
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
19/70
14
supply and utilization, and household welfare and access to basic services (water, etc.). This was in line
with the development theme of the Sessional Paper No 1 of 1986on Economic Management for
Renewed Growth, namely, to increase efficiency in the public sector (through budget rationalization
and other measures to enhance delivery of public sector services) and private sector (through
strategies and policies that increases the private sectors role in development). The project therefore
focused on measures of effects at the household level. However, since both Governments recurrent
and development expenditures have implications on delivery of public services and the private sector
costs of doing business, it is not immediately apparent whether the project was to concentrate on
monitoring public sector development projects, and ignore recurrent expenditures.
INSTITUTIONAL ARRANGEMENTS
3.6. The project document stated that the Ministries and Departments to be directly involved in
the project implementation were:
(a) Ministry of Planning and National Development: Rural Planning Department (RPD),Management Information Services Division (MISD), Central Bureau of Statistics (CBS) and
Department of Resource Surveys and Remote Sensing (DRSRS);
(b) Ministry of Agriculture: Development Planning Division, Farm Management Division and
Crop Production Division; and
(c) Ministry of Livestock Development: Development Planning Division, Livestock Production
Division, and Department of Veterinary Services.
3.7. The major responsibility for implementing the district-level M&E project sub-component was
vested in the Rural Planning Department of MPND as the institutional linkage between the project
district-level activities. A newly created division to house the project, Strategy and Policy Analysis
Division (SPAD), would ensure the smooth coordination and implementation of the projects various
components. To facilitate inter-ministerial coordination, an Inter-Ministerial Monitoring and
Evaluation Committee (IMEC) was to be established consisting of participating line ministries. The
division of responsibilities gave MPND a liaison role with the Office of the President, the line
ministries, and districtsDPUs and DECs. The Office of the President had administrative authority on
the Provincial Monitoring and Evaluation Committee (PMEC), District Development Committee
(DDC), DEC/DPU and project managers. The line ministries had administrative authority on district
departmental heads. The project was supposed to establish close links with the USAID-financed
project in Rural Planning Department in the field of district planning and Ford Foundation $1 millioncomponent for training district personnel on monitoring and evaluation techniques.
3.8. The project was expected to hire experts, namely:
(a) An internationally-recruited Senior Planner/Data Analyst to assist MPND and participating
line ministries to design and implement a system for monitoring and evaluating Kenyas
agricultural and rural development projects/programmes, policies and strategies;
(b) Consultant on Evaluation Surveys to finalize the analysis of the Agricultural Production
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
20/70
15
Survey (APS), 1986/87 data;
(c) Consultant on Agricultural Strategies, to work with the Ministry of Agriculture (MOA) in the
processing and analysis of farm management data, including fertilizer distribution and its
pricing;
(d) Consultant on Agricultural Growth Potential, to assist the Ministry of Agriculture in
agricultural commodity analysis;
(e) Consultant on Multi-Market Analysis to assist in preparation of policy papers on the bases of
impact survey data e.g. APS 1986/87, RHBS 1981/82, UHBS 1982/83 and Farm Management;
and
(f) Consultant on Spatial Mapping.
3.9. The project distributed responsibilities and gave financial allocations to the stakeholders (i.e.
participating institutions). The project therefore envisaged that the newly created division would
manage the process rather than the substance of the project components. In particular,
district-level M&E was to be closely coordinated with participating ministries, while economy-wide
strategic policy issues were vested in the relevant institutions e.g. MOA.
IMPACT SURVEY DATA
3.10. The main objective of the activity was effective use of already collected survey data and new
M&E data for improving the implementation and design of agricultural and rural development policies
and strategies. The already collected survey data included (a) the Agricultural Production Survey,
1986/87, (b) the Rural Household Budget Survey, 1981/82, (c) the Urban Household Budget Survey,
1982/83, and (d) the Social Impact Data. The data was supposed to be analyzed, used in the preparation
of policy and analytical reports, and summary results displayed using computerized mapping software.
The maps were to be disseminated to central, provincial and district planning units to be used in policy
and project design.
3.11. The basic objectives of the Urban Household Budget Survey 1982/83 were to provide basic
information on the incomes and patterns of consumption of different socioeconomic groups, and to
provide weights on which to base revision of the cost of living index. The APS collected data on
household size and holdings, land use, and the use of agricultural inputs. The results of the
Agricultural Production Survey 1986/87, which covered 24 districts mostly in high and mediumpotential areas, were highlighted in capsule form in Economic Survey 1989.
3.12. While many authors have decried (and is implicit in some of the UNDP KEN/88/034 projects
objectives) the inability of CBS to analyze survey data and release survey results on time, this has
probably been caused by lack of a clear policy on data archival system. For example, the 1986/87
Agricultural Production Survey was combined with the Fourth Rural Child Nutrition Survey so as to
provide interrelationships between nutritional indicators and food security. However, some of the
APS data was lost, and the original intentions of the survey planners were never realized. The 1982/83
Urban Household Budget Survey data tapes were found in 1993 to have been overwritten with
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
21/70
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
22/70
17
addition, the Poverty Profiles (1982-92) and the Basic Report of RHBS were coordinated by a data
analyst/statistician from SPAD and an economist from HRSSD because (a) of overlap of functions of
SPAD and HRSSD, and (b) both studies used the RHBS database.
Computerized Mapping System
3.17. A consultant was in 1986 funded by FAO under the project UNDP KEN/85/001 to assist in the
preparation of a sampling frame for the purpose of developing a monitoring system. The activities
performed under the project sub-component included the development of a computer program for
determining the sample size and actual sample selection using the hierarchically nested spatial and
administrative organization of the nation (i.e. from nation, province, district, division, location to
sub-location). Spatial codes were devised so that the scheme could be used on diverse projects ranging
from national surveys to location-based studies. Other proposed codes were ecological zones and
seasonal regimes due to their explanatoryrole in rural households food security. The report also
recommended the development of capability to display information using, say, maps, charts, and
histograms, as an integral part of the National Food Monitoring System. A mapping computer program
was installed for that purpose.
3.18. The project, UNDP KEN/88/034, was expected to install and manage a computerized mapping
system which would show the regional differences in the impact of sector-wide development policies
and strategies. Such data would come from analyzed survey data, Government allocations of recurrent
and development expenditures, population censuses, etc. No computerized mapping software was
installed during the lifetime of the project.
District Infrastructure Inventory
3.19. A major objective of the Governments Budget Rationalization Programme (BRP) is to ensurethat districts use development resources efficiently. The District Infrastructure Inventory is an
important input in the attainment of this objective, since it incorporates information on location,
condition and utilization of infrastructure facilities. The infrastructure inventory is a valuable input in
the preparation of district development plans, as the database is used to support arguments for the
need to construct or maintain particular facilities in different locations and sub-locations in a district.
3.20. The facilities included in the infrastructure inventory are:
a) Roads (classified and unclassified) and airfields;
b) Primary schools;
c) Secondary schools;
d) Service and training institutions (teachers training colleges, Colleges of arts and Technology or
Harambee Institutes of Technology, medical training centres, farmers training centres, village
polytechnic, craft centres, district development centres, etc);
e) Water facilities;
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
23/70
18
f) Health facilities (hospitals, health centres, clinics/dispensaries, maternity centres, nursing
homes, mobile clinics, etc);
g)
Cattle dips;
h)
Livestock holding grounds and auction facilities;
i)
Grain storage facilities (off-farm); and
j) Cooperative societies (coffee factories, milk collection and cooling stations, housing
cooperatives, savings and credit societies, etc)
3.21. According to the guidelines issued by the Rural Planning Department (RPD) of MPND, the
District Development Officer (DDO) is supposed to ensure that the inventory schedules are completed
by the district head of the Ministry responsible for the facility being surveyed. The completed
infrastructure schedules should have: (a) clear description of the nature and function of the facility, (b)
location of the facility, (c) careful evaluation of the extent to which the facility is utilized, and (d)
specific statements on the condition of the facility and what improvements are necessary to maximise
the facilitys service delivery.
3.22. The Management Information Services Division of MPND developed a computer program
written in Dbase, which allowed the data collected to be processed and returned to the districts. This
provides the district departmental heads and District Development Committees with well-organized
and readable data, which is also used in support of district planning activities. The Inventory shows
the facilities which are underutilized and the spatial distribution of the facilities. It therefore guides
the Government on whether new projects should be undertaken or existing projects rehabilitated. TheInventory is therefore useful during planning and budgeting stages. However, the Inventory was not
intended to be used as a monitoring and evaluation tool2. The monitoring and evaluation tool was the
Annual Annex to the district development plan, detailing (a) a status report on projects implemented
in the previous financial year, (b) the annual work plan for the current year on projects and
programmes included in the Development Estimates, and (c) the forward budget that gives priority
projects drawn from the project bank of the District Development Plan.
3.23. The project was expected to assist the Rural Planning Department in the analysis of
infrastructure inventory data. In the first Tripartite Review meeting dated February 1991,
uncertainty was expressed as to whether this exercise should be incorporated into the project. Themanagement of RPD were not aware that the project was supposed to assist the RPD in district
planning and the maintenance of the infrastructure inventory. The project budget was not used to
support the Rural Planning Department in its maintenance of the infrastructure inventory. The new
division (SPAD) created to house the project was supposed to manage the processrather than the
2. The Inventory is related to strategies and policies. As MPND noted in a background document before
the project became operational, cattle dips are built to eradicate ticks and promote meat production; schools are
built to educate children and raise their potential for productive employment; and heath facilities are built to
provide treatment and prevent illness. The Inventory gives insights into whether the facilities are contributing
to these strategies/policies.
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
24/70
19
substance, hence the inability to implement project activities that could not be transferred from
their host departments to the new division (SPAD).
Other Impact Surveys
3.24. The project was expected to support the analysis offarm management data, agricultural
inputs [e.g. fertilizer and seed use], pricing policies, marketing and storage strategies. Farm
management data are collected annually by the Farm Management Division in the Ministry of
Agriculture. It has information on the prices of farm inputs, production costs, transportation costs and
yields, among others. The data provides insight on the trends of yields, costs and prices, and therefore
provides useful indicators of efficiency of farm production.
3.25. According to the draft Project Implementation Report (August 1990), the project was to
coordinate special studies on the effects of implementation of policy reforms in agriculture. Based on
the discussion paper prepared by K. Altemeier, consultant for the World Bank, the three studies
agreed on with the World Bank were on maize market reform, fertilizer market liberalization, and
beef price decontrol. The project was also expected to carry out an impact evaluation of all the
components of the Rural Services Design Project. The consultancy inputs for the three studies, RSD
mid-term review, and establishment of indicators of efficiency in the agricultural sector were to be
financed under the World Bank contribution to the project.
PROJECT IMPLEMENTATION MONITORING
3.26. During the projects preparatory phase, project implementation monitoring was conducted by
the Office of the President. The formats were filled at the ministerial level every six months. The
system was not therefore used for evaluation of programmes at the district-level. Neither the Ministry
of Planning (other than MISDs support in computerizing the data) nor Treasury was involved in this
process; hence there was weak linkage in reflecting the results of this activity to the planning and
budgetary processes.
3.27. The purpose of the project implementation monitoring system was to:
a) Enable implementing ministries to know whether approved projects have adequate funding;
b) Detect whether projects are being implemented;
c)
Provide information on rate of disbursement by donor; and
d) Enable the Ministry of Planning in the compilation of Compendium on Donor Support and
update this on an annual basis.
The project was expected to transfer data collection responsibilities for district projects from line
ministry headquarters to the districts, and for line ministries to coordinate data collection for national
and multi-district projects.
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
25/70
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
26/70
21
4.5. The legend of the codes for occupation of the household head was not availed to the
consultants who prepared the Basic Report and Poverty Profiles, making it difficult to generate
meaningful socioeconomic groupings. Under each occupation of the household head (e.g.
professional, technical class), there were different codes (numbers) rather than a single code as would
be expected. A frequency count of occupation of the household head under each occupation category
did not give a lead on the meaning of the codes (numbers) in the aggregated database. The occupationcodes in the database do not tally with those of the Kenya National Occupational Classification System
(KNOCS) used in the 1981/82 Rural Household Budget Survey and which was adapted from the
International Labour Offices (ILO) International Standard Classification of Occupations (ISCO-1968).
For example, agricultural, forestry and related workers not working on their holdings (KNOCS
category 55) is shown to represent 74 percent of all household heads who responded to the survey
questionnaires, although the figure probably represents all agricultural workers working on and
outside their holdings. This put a binding constraint on the use of the 1981/82 Rural Household
Budget Survey database, since it was initially envisaged that poverty measurements and
interpretations would be meaningful with socioeconomic groupings based on occupation of the
household head. The problems encountered in using the data in preparing a basic report and the
poverty profiles may have been due to inadequate guidance to the systems analyst on the data needs
for the abovementioned outputs.
4.6. The 1981/82 RHBS database provided a baseline for comparison with the 1992 National
Household Welfare Monitoring Survey. In addition, since consumption data was disaggregated in the
1981/82 RHBS, the RHBS consumption basket was used in the computation of poverty lines for rural
1981/82 and both rural and urban 1992. It was not feasible to compute itemized food expenditure
using the 1992 survey data since consumption from own-production was not itemized.
4.7. The 1981/82 Rural Household Budget Survey was used in the preparation of a basic report
using UNDP contribution to the project. Two local consultants were contracted for the activity, andstarted the analysis of the data in May 1993. The consultants were supervised by a data
analyst/statistician from SPAD and an economist from Human Resources and Social Services
Department (HRSSD). The draft basic report was completed in April 1994 and was discussed in a
workshop in June 1994. The grouping of data and exclusion of some variables collected in the survey
delayed the analysis and finalization of the basic report and reduced the soundness of the conclusions
emanating from the analysis. The workshop made extensive recommendations on the need to:
a) Reorganize the sequence of ideas in the draft report;
b)
Disaggregate the analysis to the district-level for some variables to serve as a pivot for districtplanning; and
c) The need for in-depth analysis for some of the datasets and the comparison of the results with
those of other surveys previously conducted in Kenya.
4.8. The basic report was accepted with minimum improvements. According to the projects
terminal report, one of the key outputs from the workshop was production of terms of reference for
finalization of the report. The exercise was supposed to commence in September 1994. However,
arrangements have not been made to act on the recommendations so that a final report could be
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
27/70
22
prepared. It is necessary for funds to be identified to complete the activity.
CEREAL SECTOR REFORM PROGRAMME
4.9. The cereal sector reform programme (CSRP) was initiated in 1988 with the objectives of
minimizing Government intervention in grain marketing and creating an enabling environment forparticipation of private traders. In order to understand whether the policy reforms were generating
the intended effects, the Government, with the support of the World Bank, initiated a system for
periodic assessment of the effects of reforms on vulnerable groups. A consultant was hired for the
purpose.
4.10. The study was conducted using (a) a household survey from a stratified sample of 1,670
households covering 8 districts, and (b) secondary data supplied by CBS, Ministry of Agriculture and
the then Ministry of Supplies and Marketing (e.g. area and production of maize, quantities and prices
of marketed production). The implementation of the survey was in three phases. Phase 1 involved the
development and refinement of a conceptual framework for M&E of the effects of maize marketreforms on vulnerable groups, covering identification of the poor, methodology for linking policies
and the poor, and the monitoring system and its data requirements. Phase 2 piloted the survey
instruments in two districts (Muranga and Machakos) and conducted a rapid assessment of maize
market performance in five districts. The pilot survey was used to refine the survey instruments. The
final phase covering 8 districts was conducted during March-April 1992 in Bungoma, Siaya, and South
Nyanza (surplus stratum), Kiambu and Nyeri (marginal stratum) and Kitui, Taita Taveta and Kilifi
(deficit stratum). The survey period coincided with the relaxation of maize movement controls, since
traders were officially allowed to transport up to 88 bags of maize across districts without permit in
April 1992. The final report was submitted in July 1992, but was discussed two years later (in June
1994).
4.11. The results showed that:
a)
Maize was the dominant item in the households consumption basket in all the districts;
b) In all the districts, the main marketing outlets were private traders followed by the National
Cereals and Produce Board (NCPB), and private traders paid higher producer prices in all
districts except Nyeri and Kitui where the prices were almost at the same level; and
c) The incidence and severity of calorie-deficit (food poverty) was highest in the surplus and
deficit strata compared with the marginal stratum.
4.12. However, on questionnaire design, some food items were omitted which might bias results on
calorie-deficiency as an indicator of household vulnerability; and the supply side was not adequately
captured in the survey (quality of fertilizer and seeds). In addition, the results of some of the variables
collected are not reported in the final report. The study was coordinated by SPAD and the Central
Bureau of Statistics, and had little direct supervisory links with the Ministry of Agriculture, due to the
fact that the entire M&E project did not co-opt the stakeholders (headquarters of line ministries)
envisaged in the project document.
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
28/70
23
4.13. As reported in the projects Terminal Report dated July 1994, the study on Monitoring and
Assessing the Effects of Maize Marketing Liberalization on Vulnerable Groups in Kenya (Kenya,
1992d) could not be discussed in a policy workshop nor could the recommendations be immediately
implemented as restrictions on maize movement were reintroduced soon after the completion of the
report3. This frustrated the success of the projects sub-component in two ways. First, although a
sequence of studies were planned so as to create a time series of the indicators, new studies could notbe undertaken while the first study was officially in draft form since it had not been discussed.
Secondly, the outcome of the study was supposed to influence the direction of policy but the reverse
was now true: policy was influencing the research studies. The projects terminal report (July 1994)
states that a more comprehensive impact study on the performance of the maize sector since
liberalization was reintroduced is scheduled for September 1994 and would cover 25 randomly
selected districts. However, the process has not been set in motion e.g. identification of consultants
for the activity.
MID-TERM REVIEW OF THE RURAL SERVICES DESIGN PROJECT
4.14. The responsibility for coordination of mid-term evaluation of the umbrella Rural Services
Design project was vested in the M&E sub-component. The study was commissioned by the Ministry
of Planning in July 1992 to assess the successes and failures of the 13 sub-projects under the RSD
project, and to offer guidelines on the future direction of the sub-components. The RSD impact study
was completed in November 1993. The findings of the study have not been discussed in a policy
workshop. However, since the World Bank financed the RSD impact study and the umbrella RSD
project closed in April 1995, the consultant report can be considered final.
4.15. On the M&E sub-component, the impact study found out that, despite the late start, the
sub-project had made substantial progress in establishing a system for monitoring development
projects. However, a comprehensive system for monitoring and evaluation and for explicitly linking
data collection and analysis with key strategies and programmes outlined in the Sessional Paper No. 1
of 1986 and the sixth National Development Plan (1989-93) had not been developed. In addition, the
impact study recommended the formation of an inter-ministerial steering committee and
identification of one counterpart from the headquarters of each participating line ministry to work in
close collaboration with SPAD on a regular basis in completing development and institutionalization
of the M&E system. The recommendations, if acted on, could have redirected the project to its original
objectives and ensured ownership of the M&E activity by the entire central Government.
3. The Government abolished maize marketing controls in late 1993.
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
29/70
24
CHAPTER 5: PROJECT IMPLEMENTATION MONITORING
BACKGROUND
5.1. The 1971 Report of the Commission on Public Service Structure and Remuneration (Kenya,1971) and the third National Development Plan (1974-78) argued that the Government should make
sustained efforts to strengthen its appraisal and monitoring of public investment activities. The then
Ministry of Finance and Plannings Project Planning and Evaluation Unit (PPEU), with substantial
technical assistance from the Canadian Government, strived from 1970 to 1983 to design a system to
provide a list of public sector projects and programs and produce guidelines for an integrated system
for appraising and evaluating projects and programs. The Canadian-supported project was
implemented by York University.
5.2. The PPEU produced a number of studies and reports e.g. the 1976 Public Sector Project List
and the 1976 Public Sector Projects Handbook. A Project Registry Information Sheet was developed(see Public Sector Projects Handbook, 1976, for description of the system and a sample of the project
registry information sheets). As a logical follow-up to these initial efforts, the Ministry of Planning in
1982 issued a circular to all line ministries titled Implementation Scheduling and Monitoring of
Development Projects, on monitoring the physical and financial progress of project implementation.
The World Bank (1986) notes that the Ministry of Economic Planning and Development (MEPD)
issued a circular in January 1982 establishing a project implementation monitoring system for the
Government, whose framework included (a) a forward implementation schedule, which would set out
physical and performance targets for the financial year, and would be used by the Estimates Working
Groups in the budget formulation process; (b) project implementation reportsissued on a quarterly
basis, which would summarize project expenditures and progress in implementation, and describe
major project issues and recommendations from technical ministries; and (c) a project registry to store
this information, located in both MEPD and the implementing ministries. This was followed by
Guidelines for the Preparation, Appraisal and Approval of New Public Sector Investment Projects
issued in January 1983. The initial efforts to establish an effective project monitoring system were not
successful (World Bank, 1986).
5.3. However, in the mid-1980s, a joint initiative of the Office of the President (OOP) and MPND
attempted to gather information on the implementation status of Government projects and programs,
beginning with July-December 1984 reporting period. A Project Implementation Review report for
each reporting period was published, roughly one year after the end of the reporting period, e.g.
January-June 1985 data was published in May 1986. In 1987, the responsibility for operating thesystem was passed solely to MPNDs Project Monitoring Unit in MISD. Without the OOPs
commanding authority, the Project Implementation Monitoring System (PIMS) was unable to gather
comprehensive data in a timely manner. The information sheets were filled by the line ministries
rather than the districts.
OUTLINE OF THE ACTIVITY
5.4. The major responsibility for implementing the project rested on the Ministry of Planning, in
the newly created Strategy and Policy Analysis Division (SPAD). To facilitate inter-ministerial
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
30/70
25
coordination and spread the ownership of the project to the entire central Government, an
Inter-ministerial Monitoring and Evaluation Committee (IMEC) was to be established, consisting of
participating line ministries and chaired by MPND. The project coordinator was to be the secretary to
IMEC.
5.5. According to the Project Plan Implementation Report (August 1990), the project was tofacilitate data collection at the district-level, to enable the districts to utilize the information and serve
the data demands at the central level. In February 1991, MPND held a workshop in Naivasha with the
prime objective of introducing basic concepts of the monitoring and evaluation to key planning and
budgeting staff within the ministries who will play the major role in ensuring the effectiveness and the
success of the system. The presentations included write-ups on the roles of the Project
Implementation Section (PMS), the Management Information Services Division (MISD) and the
District Information and Documentation Centres (DIDCs); and briefs on the Rural Development Fund
(RDF) project and EEC Micro-Projects project. The workshop did not allude to the extensive literature
prepared under the project preparatory activities (KEN/87/015) and the project s predecessor
(KEN/85/001).
DATA CAPTURE
5.6. The project implementation information sheets are divided into seven sections: project
identification; terms and dates; finance/funding; project expenditure; budget information; project
revision; and remarks on funding, project management, physical environment and
manpower/supplies.
5.7. The project implementation information sheets largely correspond to the ones used by the
Office of the President in the early 1980s. In the section on project identification, the only addition is a
question on project rank - core, high, medium, otherso as to conform with the submissions under
the Public Investment Program (PIP). The section on Terms and Dates was amended by deleting the
item actual start date, which, when compared with planned start date would be important in
detecting delays in project start-up. The sections on finance/funding, project expenditure, budget
informationand project revisionhave largely been left intact. However, the section on remarks
was elaborated by giving codes/legends to use in filling in information on problems affecting project
implementation.
5.8. Blank forms are submitted to the districts to be filled in by the district departmental heads
representing the line ministries. The main data gaps encountered in the districts are background data
on project cost and project financial and physical scheduling. This information is expected to be in theproject documents and agreements with donors, which are not forwarded to the districts. It therefore
becomes difficult to use the data collected to establish the financial and physical implementation of
projects vis--vis original designs. During the field visits, district departmental heads stressed the
importance of creating the institutional arrangements to facilitate districts to receive original and
revised project documents. They argued that this would facilitate project implementation monitoring,
and give consideration to district concerns/interests when budget reallocation decisions are made at
the line ministry headquarters.
5.9. Departmental heads reported that they are inclined to consider PIMS as an inventory of
-
8/9/2019 Kenya: Monitoring and Evaluation of Agricultural and Rural Development Strategies Project (KEN/88/034): Project C
31/70
26
investment projects, while Development Estimates contain donor projects whose expenditures,
especially on operations and maintenance, are largely recurrent in nature. This is because donor
projects are not included in the Recurrent Estimates. This is one of the reasons behind differences in
the number of projects reported under the PIP and PIMS. This factor may also have a distortionary
impact on the economic classification of Government expenditures.
5.10. The district departmental heads fill in the information sheets and forward them to the District
Development Officer (DDO), who organizes for the data to be keyed in at the district using clerks from
the DDOs office or the District Information and Documentation Centre (DIDC). The computer
diskettes containing data files are then forwarded to the Ministry of Plannings headquarters for
printing. The printed forms containing the original data entered by project and analytical tables from
the PROREG computer program are forwarded back to the districts. An analytical report based on the
districts submissions is also prepared by the project staff, and forwarded to the district. These
analytical reports form the background information for feedback workshops where the d