1

Tearfund

Tearfund Zimbabwe, 19 Broadlands Road, Avondale, Harare

Jan-Feb 2015 OTC- Impact Evaluation

Field work carried out from January 26 through February 7, 2015

Report author: Chris Woodring

Contributors: Ken Flower and Nicole Senderayi

2015

Tearfund-Operation Trumpet Call

2

Contents

ContentsGlossary......................................................................................................................................................................................

EXECUTIVE SUMMARY................................................................................................................................................................

Background................................................................................................................................................................................

Methodology..............................................................................................................................................................................

FINDINGS..................................................................................................................................................................................

CONCLUSIONS..........................................................................................................................................................................

RECOMMENDATIONS...............................................................................................................................................................

ANNEX 1: Draft Action Plan......................................................................................................................................................

ANNEX 2: Evaluation Team.......................................................................................................................................................

ANNEX 3: Map of OTC Sites......................................................................................................................................................

ANNEX 4: Map of Selected OTC Sites.......................................................................................................................................

ANNEX 5: Selected OTC Sites Characterizations.......................................................................................................................

ANNEX 6: Sample Community Mapping...................................................................................................................................

ANNEX 7: Key Informants.........................................................................................................................................................

ANNEX 8 Documents reviewed................................................................................................................................................

ANNEX 9: Interview Guides-FGDs, Farm Visits, Stakeholder Interviews..................................................................................

ANNEX 10: Pfumvudza and Actual Sample Cost Analysis.........................................................................................................

ANNEX 11: Terms of Reference for the Evaluation...................................................................................................................

ANNEX 12: Evaluation Schedule...............................................................................................................................................

ANNEX 13: Persons participating in the Evaluation..................................................................................................................

ANNEX 14: Field data used during the Evaluation, including baselines....................................................................................

ANNEX 15: Bibliography...........................................................................................................................................................

ANNEX 16: A self-evaluation of the evaluation using the BOND evidence principles...............................................................

3

Glossary

AER Agro Ecological Region

CA Conservation Agriculture

CYMMT International Maize and Wheat Improvement Centre

EFZ Evangelical Fellowship of Zimbabwe

FAO Food and Agriculture Organization

FfF Foundations for Farming

FGD Focus Group Discussion

Ha Hectare

HH House Hold

MT Metric Tonne

NGO Non-Governmental Organisation

NFCZ New Frontiers Church in Zimbabwe

OTC Operation Trumpet Call

RoL River of Life

4

EXECUTIVE SUMMARY

Background (page 8)

Operation Trumpet Call (OTC) is a conservation agriculture (CA) project supported by Tearfund New Zealand, UK, Australia, and Netherlands, World Vision Canada; and a number of Zimbabwe church partners. OTC uses the Foundations for Farming faith-based approach to promote conservation agriculture and is in the last year of a six year intervention. This evaluation focuses on the impact of the agricultural techniques, the methodology of implementation through the local church and the evidence for holistic transformation, providing quantitative and qualitative evidence of the impact of Foundations for Farming in Zimbabwe.

Evaluation Methodology (pages 8 to 11)

This evaluation was carried out by a team of 8 evaluators, including OTC staff, Tearfund partner staff and independent evaluators. The team used 4 tools: Focus Group Discussions, farm visits, key informant interviews, and literature review.

Findings (pages 11-25)

1. OTC AS COMPARED TO PEER ORGANIZATIONS: OTC promotes similar CA technology as its peers but differs in that it also promotes the management principals of timeliness, to standard, without wastage, and with joy; originally it provided free inputs but has discontinued this practice. It is unique in that it organizes all programming through local churches instead of secular farmer groups, possibly a more sustainable model. While most NGOs peers target resource poor families, OTC invites all community members to participate regardless of need.

2. PROJECT IMPACT: Farmers who adopted CA through OTC have increased yields. The OTC farmers visited during the evaluation averaged 717 kg of maize per farmer in 2014, providing sufficient maize to cover a typical family’s maize consumption for a year. The yield per Ha was 1.5 to 2.4 MT, depending on the point of reference.

CA production comes at a high labour cost. Farmers reported that CA requires more labour than conventional production per Ha and per MT of grain produced.

Increased production has led to improvements in schooling, household supplies, livestock holdings, and increased vegetable gardening.

Women specific impacts included improvements in food production and income while men were better able to improve family food security.

Girls and boys particularly benefited from the increased income being applied to their school fees

5

3. EVIDENCE OF LONG TERM CHANGE, ADAPTATION, AND LOCAL OWNERSHIP: About 14% of OTC farmers are now using CA for all grain production. The average farmer has 2.5 Ha and is using 0.48 HA for CA. During the 2015 evaluation the average crop appearance for the CA farmers was very similar to that of their conventional neighbours. Several critical CA components were poorly adopted, or not adopted at all, especially crop rotation, permanent soil cover and weed control. Little signs of adapting the system to the local environment were observed.

4. MONITORING AND EVIDENCE COLLECTION: The monitoring platforms did not consistently report on project indicators. Some project objectives were not reported on or tracked while weak indicators of change were sometimes used. There were significant variations between the OTC official participant lists and those used in the community.

5. THE CHURCH AS AN EXTENSION VEHICLE AND VEHICLE FOR HOLISTIC CHANGE: Evaluation participants were positive about the church as a vehicle for change while there were a few cautionary observations. Almost all Focus Group Discussions (FGDs) identified at least one holistic change (any HH change beyond increased yields). Most of the reported changes were a direct result of increased income. There were a small number of changes reported that reflected changes in values, including increased family harmony, better relationships between spouses, and less drunkenness.

Conclusions (pages 25 to 27)1. The FfF approach to CA used by OTC is achieving similar results as its peers promoting

hoe based CA in Zimbabwe and abroad. Most farmers are adopting minimal tillage but neglecting or not adopting permanent soil cover and rotations. Most farmers are adopting CA on a small portion of their land. Most farmers are increasing the amount of labour needed to farm but are more food secure as a result.

The technological limitations are the same and similar results are being achieved when looking at three key results areas:

yield per hectare has increased dramatically over conventional, often more than 100%

total land area adopted is usually a small portion of the total available, between 0.15 and 0.5 of a Ha

total production (or value of production) through CA is often about the amount the family actually consumes in a year; 800-1000 kg

2. The key limitation to increasing the impact of OTC programming is the high labour

demand of the system. The labour demand manifests itself most critically in weed control because weed

control must be done in a short window of time during the growing season to achieve good yields. Farmers cannot plant an area larger than they can effectively weed.

6

The preparation of basins, preparation of composts and the external sourcing of mulch also all imply large labour demands

3. Both OTC and peers report holistic changes as a result of their programming. Farmers have better health, their children have better schooling, their homes

are more comfortable, they have more cattle, and they often have more businesses.

Holistic changes related to changes in values and attitudes were reported sparingly by both OTC trained and non-OTC trained CA farmers. In particular CA appears to offer increased gender harmony and family well-being. However, conflicts do arise from CA adoption in families and communities, often in response to the struggle for scarce resources, especially land, manure and fodder.

4. Project Reporting covers yields per Ha extensively and the level of adoption of some of the components of CA but does not report on the increase in the amount of food or income attributable to FfF, probably the most important measurable result of programming.

Yields per Ha do not indicate how much was actually produced, or if it is more profitable than the previous system.

The cell phone platform has not gathered monitoring information in the quantities and timeliness needed to be credible for reporting purposes. Project participation is not reflected faithfully by the official registration lists.

Recommendations (pages 27 to 37)

GENERAL RECCOMENDATION #1: Tearfund and River of Life should continue to develop and promote CA with small holder farmers in Zimbabwe, Africa and around the world.

CA is a powerful tool that often enables a food insecure family to become much more food secure in just one season (Conclusion 1, bullets 1 &3; Finding #2) .

GENERAL RECCOMENDATION #2: Tearfund and River of Life should explore ways to make the CA system more profitable.

CA using the current basin and mulching technology is very accessible (simple, only labour required), but farmers quickly reach the limitation of labour availability, limiting their productivity and potential profit (Conclusion #1&2; Findings #3).

SPECIFIC RECOMMENDATIONS:

7

1. CA programming should explore technologies that diminish labour, enhance farmers’ ability to increase their total land area under CA and adopt key CA principals (Conclusion 1, bullet 1 &3; Finding #2 &3). Adaptation of CA to the local context is an integral part of this process. Recommended paths include:

o low external input options such as green manure cover crops, live fencing, and fodder production

o high external input options such as herbicides use paired with ox drawn rippers, planters and manual jab planters

o Promoting drought tolerant crops in drought prone areas

2. OTC/FfF programming should strengthen existing local structures (work with local authorities) to increase CA adoption. Relationships with local traditional leadership and government authorities should be cultivated and nourished (Finding #5).

3. OTC/FfF programming should actively seek to diminish points of friction in families and communities, especially related to the struggle for scarce resources, especially land, manure and fodder. Local churches should be engaged throughout programming to assume the vision of the program at the community level (Findings #5).

4. RoL should not take programming to new geographical areas until key challenges have been successfully addressed, especially innovations that will enable adopting CA to larger farm areas and effectively incorporating rotation and soil cover (Conclusion #1&2; Findings #3).

5. OTC Monitoring and Evaluation Systems should be significantly improved. Project impact indicators should be more focused, the cell phone platform reviewed and either abandoned or made functional, registration lists should be updated regularly and systematically. OTC monitoring and evaluation capacity should be increased (Conclusion #4; Finding #4).

6. OTC project capacity building design should be field based. Training and monitoring staff should be more locally based. Pastor and trainer events should be carried out locally or regionally (Finding #1).

7. The design used for this evaluation is recommended for replication for other projects. Key issues that should be considered include ensuring strong representation of women on the evaluation team, ensuring the representation of women in FGDs, including youth and children in the evaluation process, providing alternative interview formats for key informant interviews, and ensuring good transportation logistics.

8

BackgroundThis evaluation of Operation Trumpet Call (OTC) was jointly commissioned by the 3 Tearfund funding partners and RoL, under the authority of Ben Nicholson, Tearfund Zimbabwe country representative, and in close coordination with Tearfund UK, Australia, New Zealand, and Netherlands. OTC is a project of River of Life (RoL), the social action wing of the New Frontiers Church in Zimbabwe. Tearfund has partnered with RoL since 2002 and supports RoL in training farmers in Foundations for Farming (FfF) through OTC, now in its sixth and final year of programming.

FfF is a faith-based approach to Conservation Agriculture (CA) based on the three key CA principles (minimum tillage, crop rotation and permanent soil cover) combined with 4 management principles that address the need for holistic transformation. FfF is often implemented through local churches.

There is evidence that CA is an effective approach for sustainable agriculture but there are challenges to CA from academic sources. To respond to these challenges, improve effectiveness and provide an evidence base for future programmes, The OTC stakeholders commissioned this study to understand the impact of FfF in Zimbabwe through the OTC programme. This evaluation focuses on the impact of the agricultural techniques of FfF, the methodology of implementation through the local church and the evidence for holistic transformation. It provides quantitative and qualitative evidence of the impact of FfF in Zimbabwe and is expected give guidance for future programming to all partners’.

Methodology

Evaluation Team Selection: The evaluation team was selected by Tearfund Zimbabwe, in close collaboration with OTC and RoL. Eight members were selected from diverse backgrounds: two OTC staff, two Tearfund supported program staff from Zambia and Mozambique, two independent evaluators from Zimbabwe, and two independent evaluators from outside Zimbabwe. See Annex 2 for a brief description of each team member.

Evaluation Site Selection: Twelve of 70 project sites were selected (17%). Before selection all sites were characterized by OTC according to province, years of participation, number of participants, change in participation over time, rural/urban location, climatic conditions, and type of land holding. Sites were selected to be representative of the larger group. For example, 50 of the 70 groups were formed in 2010 (71%); as a result, 8 of the 12 sites were selected randomly from these 50 sites formed in 2010 (67%). Of those sites chosen from 2010 a

9

proportional selection was made of sites based on a net gain or loss of members, climatic conditions (agro ecological regions 3, 4, and 5), rural/urban location, and type of land holdings.

Eighteen sites located in Mashonaland and the extreme north of Matabeleland North were removed from the selection in order to diminish travel time and make it possible to visit a larger number of sites. All other sites were included in the selection process.

The selections were made by the team leader, an external evaluator with no prior knowledge of sites. The initial selection was shared with OTC and it was observed that no sites from the higher rainfall regions had been selected. As a result two sites from low rainfall were replaced with sites from higher rainfall areas from the same region. No other changes were made in site selection. See ANNEX 3 for a map of all OTC sites and ANNEX 4 for a map of the selected sites. See ANNEX 5 for a summary of key characteristics of the sites selected. The initial site selection was carried out and shared with the implementing partner 18 days before starting field work and the final selection was confirmed 10 days before the evaluation team arrived in the field, diminishing the risk of manipulation of the sites by the local partner.

FGD Participant Selection: Thirty six FGDs were held, three per project site visited. The FGD participants were selected by the local OTC group leadership, usually a pastor or site trainer directly in charge of organizing the local training site. Four different kinds of FGDs were organized:

OTC trained CA adopters CA volunteer adopters inspired by OTC programming CA adopters trained by other agencies (government and other NGO trainers) Conventional farmers.

Every site had a FGD with OTC trained farmers, for a total of 12 FGDs with OTC trained farmers. The other three types were assigned to each site randomly with a total of eight each. 195 farmers participated in FGDs; 121 females and 74 males. There were 25 male/44 female OTC trained CA adopters, 13 male/31 female CA volunteer adopters inspired by OTC, 14 male/23 female CA adopters trained by other agencies, and 22 male/23 female conventional farmers.

Farmer Selection/Farm Visits: A total of 149 farmers were visited in their homes and fields. 75 were OTC trained farmers and 74 were conventional farmers. The farmers visited had the following socio-economic characteristics:

Gender: 69% of the OTC farmers were women; 31% were men. 68% of the conventional farmers were women; 32% were men.

HH Size: OTC farmers reported an average of 5.9 family members. Conventional farmers reported 6.3 family members.

10

Main Livelihood: 78% of the OTC farmers identified farming as their main livelihood. 68% of the conventional farmers identified farming as their main livelihood.

Land Holdings: OTC farmers reported holding an average of 2.5 Ha. Conventional farmers reported holding 2.7 Ha.

Livestock Holdings: OTC farmers reported holding an average of 3.6 cattle and 7.7 goats or sheep. Conventional farmers reported holding 6 cattle and 5.9 goats or sheep.

OTC Farmer Selection: OTC farmers targeted for field visits were selected randomly from the participant lists provided by the organization and updated by the local leadership. In most cases these farmers were not participants in the FGDs. In most cases a map was drawn to identify the physical locations of the different farmer clusters and then farmers were randomly chosen from different clusters. For FGDs and farm visits there were three evaluation sub-teams at each site. For farm visits by each sub-team, one OTC farmer was randomly chosen from the identified clusters, followed by their closest OTC adopting neighbour, giving a total of six OTC trained farmers visited at each site. The interval for selection was identified by confirming the age of the eldest person in the meeting and using their age to create an interval for selection. For example, if the eldest person present was 72 years old, every 9th person (7+2=9) on the list was selected for a farm visit. In this way the community also participated in the random selection of the farmers visited.

Conventional Farmer Selection: During each farm visit the closest conventional farmer to the selected OTC trained farmer was visited. A total of 6 conventional farmers per site were targeted for farm visits using this methodology. See ANNEX 5 for a sample of a community participant map and farmer selection.

Key Informant Interviews: Key Informant interviews were carried out with RoL staff, AGRITEX, Community leaders, peer organizations and government officials. See ANNEX 6 for a list of Key Informant Interviews. Interviews were usually carried out with at least two evaluation team members present. An interview guide was used to elicit relevant responses. A number planned key informant interviews were not achieved due to scheduling difficulties, including visits with CYMMYT, the Ministry of Agriculture, and Christian Care.

Literature Review: Key project documents were reviewed to provide baselines and context for the evaluation. Research and report documents were used to provide additional context. See ANNEX 8 for a complete summary of documents reviewed.

Arrangement of Data: for the purposes of this report the data is organized as follows:

FINDINGS: Data from Farm Visits and Focus Group Discussions; relevant information gathered from documents reviews

11

CONCLUSIONS: Concise conclusions based on analysis of the data from different information sources

RECOMMENDATIONS: Pathways for change with concise rational

FINDINGS

Specific Objective 1: OTC approach to conservation agriculture as compared with other approaches:

Key CA principals: OTC is similar to its peers in promoting the three key principals of CA: minimal soil disturbance, permanent soil cover, and crop rotation. Additional FfF principals: In addition to the three CA technology principals, OTC also embraces four management principles key to the Foundations for Farming methodology:

o on timeo to standardo without wastageo with joy

These additional principals are a holistic approach to address the need to change values and attitudes. Good farmers and businesses also apply three of these four principals. Most organizations promoting CA also promote planting on time and to standard, even though they would not identify these two principals as independent management principals to be actively promoted in their own right. The principals of “minimal wastage” and “with joy” are probably unique to the FfF/Farming God’s Way movement.

Technology: Like many of its peers, OTC has been promoting hoe and animal traction based CA as well as other complementary technologies including compost making, micro dosing using manure and chemical fertilizer, improved seed use, OPV seed propagation, small grains production, improved pest control, post-harvest grain storage, marketing, and others related to grain production. While OTC reported promoting animal traction versions of CA, only two of the practitioners visited or interviewed during the evaluation reported using an ox drawn ripper. All others reported using the hoe. These same two farmers reported that their principal source of training was German Agro Action and AGRITEX respectively. Complementary Programming: While some peer organizations focus programming only on CA production, OTC reported training farmers on unrelated food security activities, such as bee keeping as well as ‘Basic Life Skills’ (BLS). It is unclear how prevalent or helpful this

12

complementary training has been. It was mentioned by project staff and in reporting but not by project participants in FGDs. The project report for year 5 indicates that concepts taught through BLS training were learned by 40% of participants. No systematic reporting was provided on the application of the training but anecdotal stories were provided suggesting that there was some application in commercialization of vegetables, and an egg layer project. Many other development organizations also promote income generation activities, including market gardening, irrigation projects, and improved livestock rearing.

Free Inputs: OTC and peer organizations, including Oxfam, Christian Care, Christian Aid, the Brethren in Christ Church and FAO (to name only a few) have promoted CA through the free provision of inputs. OTC provided free seed in the first year of programming OTC is currently not providing free inputs Many other organizations continue to provide inputs (especially seed and fertilizer) or

subsidized inputs (Chinamasa, 2014), Oxfam (BISHOP, 2015), the FAO (Food and Agriculture Organization of the United Nations, 2010), and many others (World Bank, 2015).

OTC reporting shows a very strong link between the change in numbers of project participants and inputs provision.

o One year of inputs based programming resulted in a 42% decrease in participation over the first 4 years of programming.

o Programming without had an increase in participation of 72% in 2 years of programming.

Church Based Programming: While there are a number of faith based organizations promoting CA in Zimbabwe (Christian Aid, Christian Care, Caritas, Brethren in Christ Church, among others), OTC is different in that it promotes CA using the local churches as its framework. This framework is present in all Zimbabwe provinces. No other church based promoter of CA was encountered working at a national level. Most other organizations work closely with government authorities and organizations (AGRITEX, District Drought Relief Committees, District Authority, etc.).

Formation of Farmer Groups: OTC and most other organizations promoting CA form community farmer groups for training and monitoring purposes. While most of the organizations promoting CA form temporary secular groups that disband at project end, OTC works through church based farmer groups, a strategy that offers the possibility of continued CA farmer support groups after the project has ended. While most NGOs target families with the least resources, OTC programming invites all farmers to participate, regardless of available resources.

Harare-based training: OTC carries out several training activities for pastors and lead trainers for all sites at the Harare Foundations for Farming training centre. Other training is carried out at field level and regional centres in Harare and Bulawayo.

13

Specific Objective 2: Measurable Impact of OTC programming: in terms of the impact of FfF’s conservation agriculture approach and the role of the church as an extension agent and vehicle for holistic change.

Improved Yields & Food Availability: The most often cited impact of CA adoption was increased food availability. This was cited by 68% of the FGDs. 65% of the FGDs also cited improved yields as one of the key reasons for CA adoption.

The below table summarizes yield information for both CA and conventional farmers from the farm visit data:

-Average farm size: 2.5 and 2.7 Ha for OTC and conventional farmers, respectively; OTC farmers planted an average of 0.48 Ha under CA

Total Harvest (kg)

Yield per Ha (kg)

Yield Per HH member (kg)

-Average HH size: 5.9 and 6.3 for OTC and conventional farmers, respectively

CA-average per Farmer

717

2361 134

CA-average per Ha planted 1447 121

Conventional-Average per farmer

1488

902 294

Conventional-Average per Ha 729 238

A 3 year study in Zimbabwe from 2008-2010 showed similar increases in yields under CA, going from about 800 kg per Ha under conventional to 1600 per Ha under CA (Mazvimavia, Ndlovu, An, & Murendo, 2012). These yield are similar to those encountered by this writer

“Now I have enough food for my family!”

~CA farmer from Gwanda FGD

Observations:

The smallest fields for both CA and conventional farmers had the highest yields per Ha. As a result, yields per Ha per farmer are higher than total yield per Ha for the total area under CA and Conventional

More than 33% of OTC farmers had fields smaller than 0.2 Ha; about 25% had plots 0.1 or smaller, amplifying this effect for CA farmers

CA FARMERS PRODUCE ABOUT THE SAME TOTAL HARVEST AS CONVENTIONAL NEIGHBORS.

14

in visits to similar programs in Zimbabwe and in other countries in Africa.

The above table does not take into account the conventional plot most CA farmers planted. On average, CA farmers also planted 1.1 Ha under conventional tillage and harvested 743 kg per farmer/559 kg per Ha. As a result the total amount of maize available to CA and Conventional farmers was almost identical, 1460 for CA farmers as compared to 1488 for Conventional farmers.

There appears to be an incoherence between the farmers claim to be more food secure and the fact that they produce almost exactly the same total amount as their conventional neighbors.

There is no incoherence. The families claim to be significantly more food secure in FGDs, individual interviews, and anecdotally, and this is almost certainly the case. They are poorer than their neighbors, as shown by their livestock holdings and as reflected by their lower conventional yields per Ha (559 kg/Ha for CA farmers as compared to 729 kg/Ha for conventional farmers). Their original total harvest before adopting CA was almost certainly significantly lower than after adoption. Now they are able to produce similar amounts of grain as their neighbors on less land.

Labour: 68% of the FGDs observed that CA requires more labour, and harder labour. See below table comparing labour requirements for CA and conventional farming in terms of labour per Ha and labour per MT of grain produced:

Labour per Ha in person days

Labour per MT harvested in person days

CA-average per farmer 298.5 days 201.1 days

CA-average per Ha 172.2 days 130.9 days

Conventional-average per farmer

83.0 days 173.6 days

Conventional-average per Ha 59.5 days 81.6 days

For comparison purposes:

15

An evaluation of Christian Care CA programming carried out in 2011 reported that between 75 and 277 days are required to prepare a Ha of CA, depending on the number of years of adoption and other factors, as compared to 49 to 51 for conventional tillage (Woodring & Braul, Conservation Farming in Zimbabwe Evaluation Report, 2011).

A referenced study in 2009 showed that between 109 and 122 days are required per Ha for CA while 69-77 are required for conventional tillage (Mazvimavi & Twomlow, 2009).

The possible significance of this finding cannot be overstated. First, this is a question that warrants further study since it is a key challenge to CA adoption. CA probably is more labour intensive than conventional under poor management; for example, the labour cost is almost identical, whether a farmer applies fertilizer or not; but there is a huge difference in labour per MT produced depending on just that factor. If CA is only more profitable than conventional when all high standards are adhered to, our programming will fail because the participating farmers have not been able to achieve these high standards.

Farmers may still find CA a valuable option, even if their return on labour is lower than conventional. If they do not have other options for employment, they may choose CA because it provides them with a return for their time as compared to idleness. However, farmers will abandon CA as soon as they encounter a more profitable option.

Long Term Impacts:

Improved family economy: 32% of the FGDs reported that CA improved the family economy.

Economic impacts included improved cattle holdings, ability to pay school fees, increased household supplies, among many others reported. As a

Key Finding: Many families now have enough maize and are no longer balancing on the knife’s edge of hunger.

“CA changed my life. I used to pay others to farm, now I have money for school fees, to invest in other businesses.”

~CA Farmer, Nkayi FGD

OTC promoted CA: less efficient than conventionalThe data provided by the farmers suggests that:

CA farmers produced 4.9 Kg of maize per day of labour

Conventional farmers produced 5.8 kg per day of labour.

16

result of increased yields and farm profitability some FGD participants reported investments and activities in other productive areas such as commercial vegetable gardens and livestock.

Improved education: 45% of FGDs reported that families are now better able to pay school fees and other educational expenses. In FGDs this was the most cited impact of CA for girls although boys also benefited.

Quality of Family Life: Most FGDs reported different kinds of quality of life impacts due to CA adoption, including:

• Peace of mind for women and men as the family economy improves and as there is a greater probability of a harvest, even in drought years

• Greater purchasing power & increased consumption of clothes, purchased foods, kitchen ware, and other HH goods

• An increased ability to produce food without relying on others, especially for oxen for ploughing.

o For women this has been liberating. For abandoned, widowed, and otherwise single women this has given them the ability to produce their own food without relying on men for tillage.

o For married women CA has provided them with their own source of income. o For men without cattle this has enabled them to improve their family food

security in spite of their diminished resources.• Improved teamwork between husband and wife as they work together on their fields

with improved results.• Increased opportunities for boys to contribute to productive labour, deviating them

from antisocial activities (such as theft) to farm production, enhancing family and community quality of life

• Two FGDs reported that CA adoption can bring conflict to families, especially when only one of the HH heads attempts to adopt:

o There is sometimes a struggle for land access as men usually control land use and are reluctant to provide women with land for production.

o When women have success and increased resources, at times their husbands become jealous.

o When increased amounts of labour and labour outside the traditional agricultural season are required, resentment and family pressure can result.

Profitability: CA clearly increases yields per Ha, however, this is not the same thing as profitability. Some data generated by the evaluation process suggests that the current package of technology frequently promoted may not be profitable in many parts of the country and may not

Key Finding: CA maize production may not provide a living wage of even $5 a day; or may produce maize at a loss.

17

produce a living wage ($5 a day or more). Using the cost data provided by the Foundations for Farming Pfumvudza initiative as a point of reference, farmers in higher rainfall regions (AER 1-3) may generate a wage of $4.41 a day (based on an average yield of 6MT and maize sale price of $265/MT). In lower rainfall areas with yields of 3MT per Ha or less the farmer will not cover the cost of seed and fertilizer. Under the assumptions of zero cash inputs (the farmer produces their own seed and fertilizer for ‘free’) and using the actual reported yields and labour costs from this evaluation and project reporting farmers might generate a wage of between $2 and $4 a day (see ANNEX 8 for a detailed worksheet).

INCRISAT data from 232 farmers from 12 districts using hoe based CA in 2006-2007 showed a return on labour of between $5.22 and $5.26 per days labour (Mazvimavi & Twomlow, 2009)

FAO data from 2012 reports a return on labour in CA of between 10 and 15 cents an hour, or $0.8 to $1.20 for an 8 hour day (Marongwe, Nyagumbo, Kwazira, Kassam, & Friedrich, 2012)

The main issue with profitability in CA is labour. The promoted CA system is labour intensive in 4 key areas:

Holing out Mulching Weeding Compost making

It is unclear if all of these are profitable practices.

Mulching by bringing in external stover doesn’t make logical sense; it amounts to teaching our farmers to steal the ‘gold’ from other fields (even if it is their own conventional fields). Furthermore, bringing in grass and forest leaves has never been adopted by small holder farmers in the long term, except in a few rare cases where farmers receive constant monitoring visits. It is probably not profitable!

Composting in dry land field crops (crops other than vegetables and wet land rice) has never been adopted by small holder

External Sourced Mulch & Composting:

Probably not profitable!

“Most people hesitate in

adopting CA because it is

laborious. Time required in CA is

too much.”

~OTC farmer, Bezha FGD

18

farmers in the long term, except in a few rare cases (Bunch, 2012). It is probably not profitable!

Specific Objective 3: Evidence of long term change, adaptation, and local ownership

Long term adoption of a technology can be measured by the degree to which practitioners adopt it, in exclusion of competing technologies. Long term adoption of CA can be measured by the amount of land dedicated to the 3 key principals of CA production as compared to the amount of land dedicated to production using conventional tillage. In the OTC 2014 year-end report 60 to 70% of beneficiaries were “modelling FfF to acceptable standards,” suggesting a similar level of adoption. The adoption of specific principles were not reported on, although some information was available from the complementary monitoring and evaluation report from the same year. During this evaluation adoption of the specific principles were encountered as follows:

Minimum Tillage

• About 14% of OTC farmers are now using minimum tillage for all grain production • OTC farmers are currently planting an average of 0.48 Ha of grain under minimum

tillage. o This is above the average of 0.31 reported in the 2014 Complementary

Monitoring and Evaluation Report and well above the average of 0.17 per CA farmer reported in the Christian Care Evaluation report from 2011.

• OTC farmers are currently planting an average of 1.1 Ha of grain under conventional production

• For OTC farmers there has been no significant change in the amount of land dedicated to minimum tillage and conventional tillage when comparing the current season with the previous season.

Permanent Soil Cover: Data from the evaluation showed that 3% had adopted mulch on more than 75% of their plot, while the average plot was 24% mulched. The September 2014 Complementary Monitoring and Evaluation Report for OTC reported

Key Finding: Two of the three CA pillars have not been widely adopted:

Permanent Soil Cover Rotation

19

that 17% of farmers had mulching in 2014. It is unclear from that report if 17% of OTC farmers had mulching to standard or applied any amount of mulch.

Crop Rotation: 32% of OTC trained CA farmers showed evidence of an active rotation in their fields as compared to 21% of conventional farmers who reported or demonstrated rotations in their fields. The Complementary

Monitoring and Evaluation Report from September 2014 reported that 75% of participants used crop rotation.

In addition OTC has promoted 4 management principles: on time, to standard, without wastage, and with joy. No attempt was made by the evaluation to gather quantitative information on wastage and joy; however, information was gathered as concerns timeliness, and to standard.

Timeliness: there was an average variation of 15.3 days between the optimal planting date (based on farmer’s identification of the first effective rains for their community) and the actual planting date as reported by the farmers. Ideally the difference would be ‘0’.

To Standard:

• The level of weed control is a reflection of work done to standard. Using a scale of 1 to 4 (1 being very well controlled weeds and 4 being uncontrolled weeds) weeding by OTC Farmers was rated at 2.0, between ‘poorly controlled’ and ’well controlled’. Weeding by conventional farmers on the same scale was very similar, 1.9.

• Crop Stand: the appearance of the crop in the field is the sum of the environment and many different aspects of management to standard, including the quality of the seed, the plant population, fertility, and weed control. Both OTC trained and conventional farmers’ fields were compared and

“THOSE WHO ARE PRACTICING CA ARE NOT BETTER THAN US!”

Conventional Farmer, Rathanyana FGD

CA without rotation & soil cover is like a one-

legged stool!

20

scored as to their crops’ general appearance on the day of the visit. On a scale of 1 to 5, with 1 being the best and 5 being the worst:

o OTC trained farmers had an average score of 2.1

o Conventional farmers had an average score of 2.3

Adoption of other promoted farm practices:

Compost: 14% of the OTC trained farmers used compost fertilizer as compared to 1% of the conventional farmers from the same communities.

Winter Weeding: 61% of OTC trained farmers reported carrying out winter weeding. 20% of the conventional farmers reported winter weeding.

Stover Use: 26% of the OTC trained farmers used stover for mulch and 21% used stover for compost, while 59% used stover as livestock fodder. 93% of the conventional farmers used stover as fodder and 4% of conventional farmers used stover for compost making.

Adaptation to the local environment: Conservation agriculture must be adapted to the local conditions for best results. Adaptation to the local context is one indicator of the degree to which local farmers have adopted the technology sustainably. No significant adaptation of the CA system to the local context was observed. Maize appears to have been promoted equally across all regions, even in the lowest rainfall areas (AER 4 & 5) where millet and sorghum and other crops are more reliable in the drought conditions frequent in those areas.

Challenges with Rotation and Soil Cover

Why don’t our farmers rotate their crops in CA?

To date FfF is mainly focused on growing maize for home consumption. Families consume a MT of maize a year; they cannot consume 0.5 Ha of any legume they might rotate with their maize (or millet or sorghum)

The market for maize is weak in many rural areas. It is hard to sell 5 MT locally. It is often far weaker for beans, cow peas, soya beans, etc.

Why don’t our farmers mulch?

The labour involved in bringing in external mulch is often very high

The stover from the crop is generally not enough for a robust mulch cover, enough to control the weeds

Farmers love their cattle; in many dry areas cattle may be a more reliable, drought tolerant form of food security then CA maize

21

Specific Objective 4: Effectiveness of monitoring and evidence collection

Indicator Reporting:

Monitoring and evaluation should provide timely, accurate, and relevant information covering the indicators of change of a particular intervention as well as other relevant information.

The project identified two key impact indicators/targets:

• 75% of Participating households produce 1,5 tonnes/hectare• FfF yield is higher than yields of traditional cultivation techniques by 50% by 2015

There are a number of indicators for change in practice that have been reported on by various platforms, even though these were not identified in the approved proposal, including:

• Adoption of permanent soil cover (mulching)• Adoption of crop rotation• Adoption of composting• Numerous others (not covered directly in this report)

There are a number of expected project impacts that were identified in the planning documents that were not covered in reporting by the various platforms, including:

• awareness of nutrition and health• access to education and health • reduce the need for labour migration

There are four key sources to compare project progress: the December 2, 2014 OTC year 5 annual report, the September, 2014 Complementary Monitoring and Evaluation Report, the Operation Trumpet Call Cell Phone Data Report (April – July 2014) covering 122 farmers and this evaluation report. The below table compares the information from these sources:

Target/Indicator

December 2, 2014 OTC year 5 annual report to Tearfund UK

2014 Complementary Monitoring and Evaluation Report

Mobile Phone Platform- Jan-July 2014.

Jan-Feb 2015 OTC- Impact Evaluation

Key Findings:

There are large reporting gaps The mobile phone platform hasn’t

measured many key indicators

22

75% of participating households produce 1.5 tonnes/hectare

68% in 2013/2014

% of participating households producing 1.5 MT or over not reported

-OTC farmers averaged 2.5 MT per Ha on their CA plots.

Not Available 60% of the visited households produced at 1.5 MT or more per hectare in 2013/2014 season.

FfF yield is higher than yields of traditional cultivation techniques by 50% by 2015

OTC FfF yield is 320% of the national average yield in 2013/2014.

-OTC national average for 2013/2014 was 2.66 t/ha compared to 0.83 t/ha (ZIMVAC).

OTC yield is 141% higher than the yields of conventional farmers from the same communities in 2013/2014

Not Available OTC FfF average yield per Ha per farmer was 160% higher than the yield per farmer of conventional farmers from the same communities

OTC FfF yield per total Ha planted was 98% higher than the yield per Ha of conventional farmers from the same communities

Practice Outcomes:

December 2, 2014 OTC year 5 annual report to Tearfund UK

2014 Complementary Monitoring and Evaluation Report

Mobile Phone Platform- Jan- July 2014.

Jan-Feb 2015 OTC- Impact Evaluation

% of OTC farmers using mulch.

Not part of Proposal/Reporting Framework

86% of OTC farmers collected mulch in 2014.

Data available but not include in report

56% of OTC farmers used at least some mulch in 2014/2015 planting season. 3% mulched their entire field.

% of OTC farmers using compost

Not part of Proposal/Reporting Framework

54% of OTC farmers had started a compost in 2014.

46% of OTC farmers had started a compost by July of 2014.

12% of OTC farmers used compost in 2014/2015 planting season.

23

% of beneficiaries using crop rotation

Not part of Proposal/Reporting Framework

75% of beneficiaries use crop rotation, compared to 47% of non-beneficiaries. (2013/2014)

32% of OTC participants used crop rotation, compared to 21% of non-beneficiaries. (2014/2015)

Participant Registration

During the evaluation there was a significant variation between the official list of participants registered for a particular site and the actual participants identified at the local level. In three sites, Dakamela, Ratanyana and Lonely Mine, this was tracked methodically. Between errors of inclusion (including people who moved away, withdrew or died) and exclusion (excluding active participants) there were marked differences between the official list of participants and the locally recognized ones. In Dakamela there were 45 variances identified by the local leadership of 139 participants. In Lonely Mine there were 12 variances of 44 official participants; in Ratanyana there were 17 variances of 12 listed participants, between inclusion and exclusion.

Specific Objective 5: The church as an extension vehicle and vehicle for holistic change

Church Vehicle

In focus group and stakeholder discussions there were a number of different comments made as concerns programming through the local church. The vast majority of the respondents were positive or very positive about the church promoting change while there were a few cautionary observations.

22% of FGDs reported that the church support or the faith component of OTC programming was beneficial for supporting local adoption; 2 groups reported that programming encouraged their faith growth.

2 FGD suggested that the churches would sometimes limit programming to church members; that the church saw programming as a way of strengthening its membership and church giving, but didn’t open up programming to ‘rival’ churches or the non-churched; project staff also shared that in some cases local pastors saw the OTC programming as an opportunity for them to build their own church membership and this had been an issue in a few cases.

24

It was observed by the evaluation team that while the project is ostensibly open to all Christian churches, no effective engagement has been made to engage the leadership of the Vapostori churches (Masowe Apostles and similar churches), even though this may be the single largest Christian segment in Zimbabwe (RelZim, 2015).

Several FGDs and stakeholders observed that OTC did not engage effectively with local and regional government representatives. Project reports and staff, coordinate with local and region authorities as a component of programming; however, staff suggested that engaging with government entities has been a low priority.

Holistic Impacts

The evaluation terms of reference identify ‘holistic’ changes as those “which address the need to change values and attitudes focusing on whole life transformation and not solely sustainable agriculture.” Changes in values and attitudes were reported in small numbers during FGDs, including (by the percentage of FGDs that mentioned them):

19% - ‘greater peace of mind’; men and women working together, families working together

14% - improved family harmony 11% - empowered women better able to care for their families 11%-CA adopters are stigmatized by poverty 6% - improved care of orphans and the vulnerable; increased time with family; more

faith growth; less alcohol abuse; 6%- increased family conflict, ‘even divorce’ 3% - men abuse their families less; increased community tensions due to newfound

prosperity

There were other changes in families and communities shared in FGDs that do not reflect changes in attitude and values but do reflect other non-agriculture changes due to increased income from CA production. These are summarized below:

44%-increased ability to pay school fees for girls and boys 19%-increased livestock holdings, 17%-improved household supplies (clothes, kitchen utensils) 3%-more timely bride price payments,

Key Finding: Holistic Impacts have been:

Most prevalent as economic impacts

Modest in terms of changes in values and attitudes

25

3%-more time for other productive activities

CONCLUSIONS1. The FfF approach to CA used by OTC is achieving similar results as its peers promoting

hoe-based CA in Zimbabwe and abroad. The church is an effective extension vehicle for change in farm practice Most farmers are adopting minimal tillage but neglecting or not adopting

permanent soil cover and rotations. Most farmers are adopting CA on a small portion of their land. Most farmers are increasing the amount of labour needed to farm but are more

food secure as a result, with the benefits that increased food security brings. In spite of the promotion of animal traction options, most farmers are using hoe-

based CA.

The technological limitations are the same and as a result similar results are being achieved when looking at three key results areas:

yield per hectare has increased dramatically over conventional, often more than 100%

total land area adopted is usually a small portion of the total available, often between 0.15 and 0.5 of a Ha

total production (or value of production) through CA often about the amount the family actually consumes in a year; 800-1000 kg

2. The key limitation to increasing the impact of OTC programming is the high labour

demand of the system. The labour demand manifests itself most critically in weed control because

weed control must be done in a short window of time during the growing season to achieve good yields. A farmer cannot plant an area larger than they can effectively weed.

The preparation of basins, preparation of composts and the external sourcing of mulch also all imply large labour demands

3. Both OTC and peers report holistic changes as a result of their programming. Farmers have better health, their children have better schooling, their

homes are more comfortable, they have more cattle, and they often have more businesses.

Holistic changes related to changes in values and attitudes were reported sparingly by both OTC trained and non-OTC trained CA farmers.

26

In particular CA appears to offer increased gender harmony and family well-being. However, conflicts do arise from CA adoption both in families and in communities, often in response to the struggle for scarce resources, especially land, manure and fodder.

4. Reporting covers yields per Ha extensively and the level of adoption of some of the components of CA but does not report the increase in the amount of food or income attributable to FfF, probably the most important measurable result of programming.

Yields per Ha do not indicate how much was actually produced, or if it is more profitable than the previous system.

The cell phone platform has not gathered monitoring information in the quantities and timeliness needed to be credible for reporting purposes. Project participation is not reflected faithfully by the official registration lists.

RECOMMENDATIONS

GENERAL RECCOMENDATION #1:

Tearfund and River of Life should seek to continue to support, develop and promote CA with small holder farmers in Zimbabwe, Africa and around the world. CA is one of the best investments in small holder food security.

CA is a powerful tool for transforming a food insecure farm family into a much more food secure one in a very short time and with little or no external resources.

GENERAL RECCOMENDATION #2:

Tearfund and River of Life should explore ways to make the CA system more profitable and more attractive as a business opportunity.

“I used to worry about food but now I have enough, even when there are dry spells.”

~OTC farmer, Bubi FGD

27

CA using the current basin and mulching technology is very accessible (simple, only labour required), but farmers quickly reach the limitation of labour availability, limiting their productivity and potential profit. This makes CA unattractive for youth and the progressive element in communities as compare to migration and immigration.

SPECIFIC RECCOMENDATIONS:

1. Explore Alternative CA Technologies (Findings #2,3 & 4 on p. 15-24;

Conclusions #2 &3 on p. 25-28)

CA programming should explore alternative technologies that diminish labour and enhance the farmers’ ability to adopt CA in terms of total land area and incorporating the key CA principals of rotation and permanent soil cover.

CA has been adopted by most OTC participants to a small scale and without permanent soil cover or rotation. This is not an anomaly; the biggest limitation to upscaling and rotation under CA (and conventional) is weed control. Under conventional practice farmers control weeds at the start of the season at a low labour cost by ploughing; later they use a cultivator. CA farmers are either tasked with applying huge amounts of mulch, which effectively diminishes the size of field they can adopt, or must hoe/weed by hand, which also limits their field size to the amount of land they can manually control weeds on.

Farmers and stake holders have requested options for mechanizing and upscaling CA. One of the technologies promoted is the use of the ox-drawn ripper. While the ripper enables farmers

to quickly prepare their land, it does not control weeds, the main barrier to upscaling.

Future CA programming should explore weed control options that provide opportunities for farmers to upscale using two tracks of technology: A. those that take advantage of available local resources and require very little

Key Assumptions: farming should be profitable and attractive, not just an escape from starvation. CA should ve a smart vocational choice for youth.

The CA system promoted must be profitable and provide a living wage

The biggest limitation to upscaling and rotation

under CA is weed control.

28

as concerns external inputs and B. technologies that require capital intensive external inputs.

Technologies for very low external input weed control/upscaling include opportunities through:

the use of green manure/cover crops to supress weeds, provide fertility, provide an effective rotation and increase soil cover

live fencing (to protect actual mulched fields and decrease the labour needed to bring additional mulch); sisal has been adopted by some farmers in Zimbabwe and Zambia and may be a good option in appropriate areas.

fodder production (to reduce livestock pressure on stover/mulch)

Technologies for weed control that depend on capital include opportunities through:

herbicide use paired with ox drawn rippers or jab planters

Both of these pathways have had positive results elsewhere in Africa. Both offer significant opportunities for farmers, while they also have inherent risks in their promotion. To enhance sustainable adoption, it is recommended that programming carry

Near Karonga, Malawi farmers use lab lab bean as a cover crop associated with maize and pigeon pea. Lab lab provides as much as 60 Mt of organic matter and 200 kg of nitrogen per Ha with complete soil cover, effective weed control, and a delicious bean.

In 2015 I visited 50 small holder farmers in Kenya, Tanzania, Malawi, Zimbabwe and Zambia. 27 used herbicides and 23 used manual weed control. The average plot size for those using herbicide was 2.2 Ha while the average for those using manual weed control was 0.5 (almost identical to the OTC average from this evaluation).

29

out trials on a small scale with a small number of innovative and motivated farmers. These same farmers should then be the focus of replication efforts in the case that a particular system is found to be viable.

Adaptation must be made according agro ecological region. Currently programming seems to be almost identical from region to region.

In agro-ecological regions 4 & 5 alternative crops that are more drought tolerant need to be explored. This is not a simplistic or gender neutral issue and requires an interactive and community response. Issues of marketing, diminishing losses to birds, crop protection, diminishing (women’s) labour in processing, and supporting a culture of consumption should all be explored.

Attractive rotation options need to be identified as appropriate, according to climate, local diet, and market factors, as well as the compatibility with the predominant cereal crop.

Cash crops should be explored and effectively promoted. Farmers, especially youth and young farmers need cash income in order for them to reach their aspirations.

Fertility options should continue to be a priority and should be focused according to the crop needs and local resources, including local access to cattle manure, poultry bedding, subsidized and free chemical fertilizers, and green manure cover crops.

The organization should promote self-reliance and ownership while promoting adaptation.

Farmer experimentation should take place on the farmer’s fields, not on community plots (to ensure ownership)

Little or no inputs should be provided. o Only green manure seed should be provided to a small number of initial

experimenting farmers; upscaling should depend on the ability of the community to reproduce or purchase the seed. Farmers who receive free seed should pass that gift forward to other farmers; this should be planned from the outset.

o Hardware should not be purchased for farmers, although it may be helpful to facilitate their purchase (a ripper can be delivered at cost)

o Herbicide can be purchased for a farmer trial plot but the sprayer should be provided by the farmer

Technology Trials:To enhance sustainable adoption, it is recommended that programming carry out trials on a small scale with a small number of innovative and motivated farmers. These farmers will be the focus of replication efforts in the case that a particular system is found to be viable.

30

Initial adopters will most likely be farmers with above average economic resources. Farmers with more resources are better able to take risks and often reflect other social virtues that enable community transformation.

Mulching & Composting must be profitable: Mulching and compost use enhance yields, especially in the long term, when water is scarce. However, importing mulch from off field and making compost may not be profitable, a view point held by notable expert Roland Bunch (Bunch, 2012). This question should be explored with

farmers. The organization should be open to the possibility that imported mulch and making compost is only profitable under certain circumstances; or may not be a profitable or sustainable practice. For example, if the value of the increase in yields per area is more than the value of the labour required to mulch that same area, the practice may be profitable. If the value of the labour is greater than the increased yield, the practice is probably not profitable. There is also a literature review and research opportunity for the organization to determine the impact of different levels of crop residue on maize yields and weed control.

2. Strengthen existing local structures (work with local authorities) to increase CA adoption (Findings #1, 5 on p. 14 & 25; Conclusions #1, 5 on p. 26 & 30)

The connection with local traditional leadership and government authorities should be cultivated and nourished to enhance results. OTC has done this to varying degrees but needs to strengthen this component of programming. This will be embraced and spurned at different times and places, depending on personalities and local dynamics; but a concerted attempt should be made to engage more productively. Specifically, during planning and implementation:

Local and district officials should be contacted and given regular updates on

PROGRAMMING SHOULD ANSWER KEY QUESTIONS OF PROFITABILITY:

DOES MULCHING PAY? DOES COMPOSTING PAY IN

MAIZE?

Inviting traditional leaders and officials to

yearly training events or fields days can go far in improving their buy-in

31

programming. MOUs and yearly reports should be done routinely. Inviting officials to yearly training events or fields days could go far to improving their buy-in to programming.

AGRITEX workers should be incorporated into programming as is possible, especially in training, field days if any, in evaluations and impact assessments.

Traditional authorities and religious leaders of all kinds should be invited and embraced by programming. They can be key in facilitating adoption of CA and should be targeted because community members greatly respect them. They should be encouraged to participate in programming events, to bless, and to adopt programming

None of these entities should be paid cash or provided with material incentives to participate or support programming. They should be encouraged to understand their participation as part of their current mandate. If supporting agriculture and community development in the community is not part of their mandate they should not participate.

3. Strengthen the Holistic Impact of Programming (Findings #1, 2, 3 and 5 on p. 14,18, 18-21, and 24 & 25; Conclusions #1, 3 & 5 on p. 26, 27 & 28 and 30)

Diminish Points of Friction: Friction within the family, church and community caused by CA adoption should be anticipated and mitigated.

During initial community and family contacts, in training, and during monitoring husbands, wives, and community members should be engaged on issues of resource allocation (the amount of land dedicated to CA, the quality of land dedicated to CA, the resources dedicated to CA production, the labour dedicated to CA production) with the goal of achieving full buy in to CA by husbands, wives, and other family & community members. The encouragement of full gender buy-in should be incorporated into training, monitoring mentoring, and in engagement with traditional, government and religious authorities.

Issues related to livestock control & mulch protection should be addressed with local and traditional authorities proactively

Jealousies between trainers, practitioners and churches should be minimised and proactively mitigated through messaging during trainings, meetings, monitoring, etc.

o Preaching a gospel of humble servanthood, of working in a kingdom that is far greater than the initiatives or labours of any one person, organization, or church may be helpful.

Local churches should be encouraged and enabled to organize holistic training

Training on other locally relevant projects should be encouraged through the structure of the OTC site leadership and structure (broilers, bee keeping, improved livestock keeping, marketing, etc.) This is not to suggest that OTC should carry out these trainings, but rather should encourage and enable the local groups to take ownership and organize these kinds of trainings.

32

Training should be carried out by OTC on money management and marketing with trainers and for replication with farmers. This is a key business skill that will help enable farm management as a business and enable upscaling.

Throughout programming site leadership should be enabled to carry out continued holistic development, including CA training and monitoring, in the absence of OTC trainers

Site leadership should identify their realistic goals as an entity as well as the activities they plan on carrying out into the future in the absence of OTC monitoring and support.

Individual sites should be encouraged to identify structures for continued monitoring and supporting of CA uptake. They should also be freed to identify their own path forward in supporting holistic change in their communities from a multi-church platform even if this does not include CA.

The last 6 months of programming should include a well-organized component of closure.

4. OTC-RoL should not take programming to new communities until key challenges have been addressed. (Findings #2 & 3 on p. 15-17, and 18-21; Conclusions #2 & 3 on p. 26-27 & 27-28)

Programming should explore innovations that will enable adopting CA to larger land areas and effectively incorporating key components (rotation, soil cover) before upscaling programming.

While the need for improved food security has not been resolved across Zimbabwe, CA offers one of the most immediate and sustainable contributions to food security in the rural areas. OTC programming has made a large contribution to improved food security through CA adoption, however, current CA standards by trained farmers are often poor across geographical areas and lead to a very limited contribution.

Future programming should focus initially on exploring together with farmers innovations that will enable upscaling CA to larger land areas. This upscaling should enable larger areas under minimum tillage, rotation, and permanent soil cover. Upscaling will require that the system promoted be more profitable. The technologies suggested in Recommendation #1, ‘Explore Alternative CA Technologies’ may be helpful in refocusing technical components of programming, especially as concerns more efficient weed control.

5. Monitoring and evaluation systems should be significantly improved (Finding #4 on p. 21-24; Conclusion #4 on p. 28-30)

33

Outcomes reporting should cover the most relevant indicators.

The total amount of grain contributed by CA per family member should be adopted as a key indicator for change.

For knowledge focused outcome areas such as composting, mulching and others there should be a focus on total production and total application.

Baselines used should be the most relevant.

When comparing CA farmer production to conventional production, conventional farmers from the specific community where the CA farmers are located should be used as the baseline, rather than provincial or district averages (although the provincial and district averages can and should be used as additional points of reference).

Composting, rotation practices, weed control, planting timeliness, and other promoted practices can all benefit from conventional and practicing farmer baselines.

Barriers to an effective mobile phone platform should be identified

The cell phone monitoring platform is not functional. While in theory the cell phones can provide instant data across many regions and geographical areas, in practice data has been gathered from very small populations at irregular intervals.

This evaluation did not explore the challenges with the implementation of the cell phone platform. There could be many different barriers to functionality such as:

The person identified for gathering the data may not have the mobility, time, or motivation to effectively gather the data

The person identified for gathering the data may not have the technical expertise to use the system

The cost of airtime may be a barrier to implementation There may be a need for additional organization and training

The use of good indicators is critical to understanding project impact:

“Number of farmers attending CA Basic Training” tells us nothing of impact.

“Yield per Ha” is a good indicator of the value of the technology promoted, but is a weak indicator of impact on HH.

“Change in total harvest” gets closer to the heart of immediate impact.”

34

The cell phone platform should be reviewed and its goals identified. Its design will be different if it is expected to provide field trouble shooting data, monitoring data for reporting, key project participation data, or other kinds of information. The cell phone platform will be more functional if it is more focused on fewer key areas and gathers less subjective data for reporting (e.g. harvest data, CA plot size, amount of land mulched, rotations, and actual planting date). The selection process for the sample and the sample size should be identified and reported on. Cell phone monitoring may not be a cost effective way to gather accurate information from the field via volunteer personnel. Data gathering via cell phone may need to be a paid activity to become effective. Data gathering via cell phone technology may not be a superior or cost effective way of data gathering as compared to paper formats.

Participant Registration should be accurately updated twice a year, at planting and after harvest.

Currently the registration lists do not reflect the actual participation in the field. Site leadership should be engaged twice a year to remove inactive participants from the project registers and include newly active participants.

Additional monitoring and evaluation capacity needs to be gained by OTC

Currently 4 staff members are titled ‘M&E Officers’ or ‘M&E Assistants;’ however their actual activities correspond more to that of field trainers. Capacity can be gained either by additional training & mentoring of current staff so that they can develop better tools and improve their monitoring and evaluation function or through the provision of an M&E specialist on staff. A part time or shared M&E specialist could be an appropriate option.

6. Project capacity building design should be field based (Finding #1 on p. 13-14)Training and Monitoring Staff should be more locally based and training should be carried out locally or regionally instead of nationally.

Currently the training of pastors and lead trainers in Harare is a large time and economic investment. At the same time, the Harare site doesn’t best reflect the agro-ecological, economic, and social realities of most project sites, those located in remote areas of agro-ecological regions 3, 4 and 5. It is recommended that training and meetings be held in regional sites as geographically accessible as possible. For

LET’S TAKE TRAINING TO THE FARMER!

35

example, instead of carrying out training in Harare the sites from Manicaland could meet at a local site; the sites from Midlands and Mat North could meet in a single site for training pastors and community leaders.

The current site in Harare celebrates CA in a high rainfall area on some of the country’s best soils, based on the work of food secure hired or salaried workers: conditions that are alien to most project sites.

The project may find that there is incredible power in rotating through different local project sites for hosting regional meetings, showcasing local achievement, learning, and progress.

There should be a significant savings in resources, both time and money. In the most recent budget year just over 30% of the budget was for transport and lodging for lead trainers, pastors, and site leaders to Harare or Bulawayo.

7. The design used for this evaluation is recommended for replication for other projects after incorporating a number of key adjustments (Based on evaluation team post field discussions)Key Evaluation Components recommended for replication include:

A. Evaluation team: a diverse evaluation team, including independent members, members from the local organization being evaluated, and peer Tearfund partner representatives

• Key Recommended Adjustment: women should be well represented on the evaluation team

B. Focus Group Discussion (FGD): Site Selection should be targeted based on key criteria (location, membership, uptake, years in the program, climate/rainfall, crop, etc.). Women should be well represented in FGDs.

• Key Recommended Adjustment: Youth should be involved in the evaluation process. It may be better to have 3 FGDs types: project CA adopters, non-adopters and youth

C. Farm visits: randomly selected adopters and randomly selected conventional farmers are key to gathering representative information on adoption. Women should be well represented in farm visits.

• Key Recommended Adjustment: Children from the Households visited should be included in the interview process as a way of incorporating the voice of the youth. If this is not possible during home visits due to school scheduling, alternatives should be sought out to give them a voice (such as meeting with them at the school)

D. Key informants: gathering information from key informants needs sufficient planned time.

36

• Key Recommended Adjustment: Alternative interview modes should be considered when key informants are not accessible personally (the ideal format), including telephone/skype interviews and simplified written surveys

E. Logistics: Good transportation is key to the evaluation design and should be well organized from the outset. A back-up vehicle should be identified in the case of breakdown and this contingency planning.

• Key Recommendation: Some participants travelled long distance at their own expense for the evaluation; the evaluation should mitigate for long distances by maximizing vehicle use, strategic identification of meeting sites, and where appropriate, reimbursement for travel expenses.

37

Annexes

ANNEX 1: Draft Action Plan

1. Explore Alternative CA Technologies

What: Gain capacity as an organization with existing experiences with alternative forms of CA based on low and high external inputs CA, especially using green manure cover crops (not conventional green manures which are ploughed under!), living fences, and improved fodders, and herbicide matched with jab planters or ripper technologies. The issue of the profitability of mulching should also be explored together with the communities and future programming should investigate this key question.

Much of this work can be done via literature review and as such should be very low cost. Opportunities for exchanges with other programs experimenting or with relevant

experience with these technologies should be sought out and financing via donor partners should be explored.

At end of project dialogues and celebration meetings with project participants, this on-going research should be shared and community input, observations and initial feedback should be gathered.

Person responsible: OTC Field Staff and Administrative Staff

When: April-June 2015

2. Strengthen existing local structures (work with local authorities) to increase CA adoption

What: During the closure phase of the final 3 months of programming community and regional secular authorities should be actively engaged in closure activities, including any celebrations, official acts, and in the final reporting/feedback process. A written final impact report should be submitted to district authorities and signed for. Authorities that should be engaged include:

Traditional community leaders (village heads, chiefs, etc.) Local political leaders (councillors) AGRITEX workers District

Person responsible: OTC Field Staff

38

When: May-June 2015

3. Strengthen the Holistic Impact of Programming

Diminish Points of Friction

What: During final dialogue processes with community leaders and FfF promoters, areas of negative friction caused by CA adoption within the family, the church and larger community should be identified and specific measures for mitigation identified. A record of issues identified and mitigation steps identified should be included in the final project report. Issues should be explored at different levels, including:

Family & spouse issues Neighbour & community issues Church & interfaith dialogue issues

Person responsible: OTC Field Staff

When: May-June 2015

Local churches should be encouraged and enabled to organize holistic training

What: During the final 3 months of programming local church organizations supporting FfF programming should be enabled by the OTC network to identify community issues that would benefit from a coordinated response, prioritize them, and identify an action plan to begin working through a church based platform. While the local faith body should be enabled to identify the issues most important in their community, issues that may be relevant could include: continued promotion of CA, alternative sources of income, money management, health, water access, sanitation, production challenges, family violence and education, among many others.

Person responsible: OTC Field Staff

When: May-June 2015

4. OTC-RoL should not upscale programming until key challenges have been addressed.

What: The promotion of FfF should not be carried out at the current scale (at 70 sites across all provinces with 5,000 to 10,000 HH) until key issues have been resolved as concerns upscaling

39

challenges, specifically adoption on a larger scale in terms of Ha per farmer and efficiency in production per MT produced. Resources should be dedicated to capacitating the technology package to better respond to the farmers’ needs and visions. A proposal that seeks to reach this end should be developed and resources sought to support implementation.

Of the current 70 sites, between 10 and 20 sites should be chosen Sites should be chosen based on the positive attitude of local leadership and farmers

needed to support farmer research Sites should be chosen to reflect different climates (different agro-ecological regions) 5 to 10 innovative farmers per site should be identified on a volunteer basis These same 5 to 10 farmers should form tight clusters (be located in close geographic

proximity to each other, with at least 3 farmers per ‘cluster’)

Person responsible: OTC Field Staff; proposal writer

When: May-June 2015

5. Monitoring and Evaluation Systems should be refined

More relevant indicators should be used

What: Final reporting should report on more relevant indicators:

The total amount of grain harvested under CA The total amount of grain contributed by CA per family member The total amount of land protected by mulch The degree of mulch thickness Total compost produced The total amount of land prepared with minimum tillage

Person responsible: OTC Field Staff

When: May-June 2015

The barriers to an effective mobile phone platform should be identified

What: OTC staff should identify the issues preventing a more effective implementation of the mobile phone monitoring platform. Discussions should be held with the community members tasked with gathering the information as well as internally with OTC field and administrative staff to identify gaps between capacity and expectations. As a result of this dialogue recommendations should be made as concerns replicating the methodology: improving it, abandoning it, and identifying the limitations and parameters for success.

Person responsible: OTC Field and Administration Staff in consultation with Tearfund M&E specialist

40

When: May-June 2015

Participant Registration should be accurately updated twice a year, at planting and after harvest.

What: The participants recognized at the community level should be reflected in the final report. Each site should be provided with the official list and provide feedback with all needed corrections/updates.

Person responsible: OTC Field and Administration Staff

When: May-June 2015

Additional monitoring and evaluation capacity needs to be gained by OTC

What: Field and Administration staff should be provided with support in refining the tools to be used for the end of project monitoring and evaluation to ensure that they cover all of the approved project indicators as well as any up-dated indicators adopted by the project. For example, if the project adopts ‘number of kg of grain produced per HH member’ as an impact indicator, before gathering data in the field the tool should be reviewed to ensure that it will capture the relevant data. The outcomes from the proposal that currently have no identified indicators should be reviewed and a decision made as to their relevance and the value of identifying indicators for each.

The rationale for adopting any new indicators should be shared with donor partners

Person responsible: OTC Field and Administration Staff in consultation with donors

When: April-June 2015

6. Project capacity building design should be field based

What: Future programming should plan and implement leader training (pastors, trainers) at the regional level rather than at the national level. A new model that enables this should be be explored, including the hosting of such trainings/events on a rotational basis among sites (joining 3 to 5 sites in a functional ‘circuit’ for training/support purposes) or strategic sites at the district level may be some of the models to be considered. This model should be discussed with the leadership as part of the end of project evaluation dialogue for local input.

Person responsible: OTC Field and Administration Staff

When: May-June 2015

41

ANNEX 2: Evaluation Team

Caleb Ndazi-Ncube (OTC Staff Member)– Caleb is an OTC Monitoring and Evaluation Officer, fluent in Ndebele, Shona, and several other local languages. Is currently providing technical and monitoring support to OTC programming.

Chris Woodring (External Consultant)–International Agriculture Development Consultant specializing in rural community development monitoring and evaluation.

Gerson Machevo (Tearfund Mozambique staff)– Projects Coordinator, Tearfund Mozambique

Joseph Chitopo (External Consultant)–Joseph is a FfF trainer, farmer, fluent in Shona and conversant in Ndebele

Kenneth Ludaka (Tearfund Zambia Partner staff member)–Kenneth holds a diploma in general agriculture and works as an agriculture officer for the Evangelical Fellowship of Zambia working with the farming communities in promoting conservation Agriculture. He has been a pioneer of this process.

Dr. Ken Flower (External Consultant)—Ken is a lecturer in agronomy and farming systems with the University of Western Australia. He is a specialist in sustainable farming systems. Main research areas include soil water relations, crop agronomy, crop residue retention and improving soil organic carbon levels in no-till systems.

Nicole Senderayi (External Consultant)-specialist in monitoring and evaluation, fluent in Shona and conversant in Ndebele

Simon Simbeya (OTC Staff Member)– OTC Monitoring and Evaluation Officer, fluent in Shona and Ndebele. Is currently providing technical and monitoring support to OTC programming.

42

ANNEX 3: Map of OTC Sites

OTC Farmer Sites across Zimbabwe 2014

43

ANNEX 4: Map of Selected OTC Sites

ANNEX 5: Selected OTC Sites Characterizations Group

Name/Location

Year Started

# of Farmers in Group 2010/2011

# of Farmers in Group 2011/2012

# of Farmers in Group 2012/2013

# of Farmers in Group 2013/2014

Change in Participant # per group:

Rural/Urban

Agro-ecological Region

Average Arable Land holding per far

Type of land holding (communal, A1, A2, etc.) (NB2)

Dakamela

Honde

Silobela

Nkayi

Ratanyana

Bezha

Masvingo

Ngundu

Bonda

Nkayi/Zinyangeni

GutuLonely Mine

44

mer

Manicaland

5 Bonda2010 301 166 172 180 60% Rural 2

0.64 Communal

6 Honde2010 159 87 89 97 61% Rural 2

0.04 Communal

Masvingo

27Masvingo

2010 163 80 155 150 92%

Urban and Rural

3 or 4

0.53

Old Resettlement & Communal

28 Ngundu2012 n/a n/a 130 145

112% Rural 4

0.31 Communal

32 Gutu2010 280 40 26 65 23% Rural

3 or 4

0.05 Communal

Mat North

37 Nkayi2010 165 163 168 188

114% Rural 4

0.19 Communal

38Dakamela

2010 108 107 110 117

108% Rural 4

0.36 Communal

40Lonely Mine

2010 39 8 20 38 97% Rural 4

0.62

A1 & Old Resettlement

44

Nkayi/Zinyangeni

2012 n/a n/a 0 38 DNA Rural 4

0.28 Communal

Mat South

49 Bezha2010 72 83 81 184

256% Rural 4

0.16 Communal

57Ratanyana

2013 n/a n/a 40 25 63% Rural 5

0.16

Old Resettlement

Midlands

59 Silobela2010 302 101 129 130 43% Rural

3 or 4 1.1 Communal

Key Selection Criteria:

Year Started: Group Size Change in Group Size:Year 2010 9

75%

Average: 113 Increase 4

33%

Year 2012 2

17% >100 7 Decrease 8

67%

45

Year 2013 1 8%

50-100 2 Increase > 50% 1 8%

<50 3Decrease >

50% 217%

2013 Total Participation: %

Groups Selected %

Agroecological Region

Bulawayo15

8 4% 0% AER 2 217%

Harare28

9 7% 0%AER 3 or 4 3

25%

Manicaland60

7 14% 2 17% AER 4 650%

Mashonaland Central

576 13% 0% AER 5 1 8%

Mashonaland East

216 5% 0%

Mashonaland West

305 7% 0%

Masvingo63

6 14% 3 25%

Matabeleland North

1101 25% 4 33%

Matabeleland South

325 7% 2 17%

Midlands23

2 5% 1 8%

Total:4445

46

ANNEX 6: Sample Community Mapping

47

ANNEX 7: Key Informants1. Alan Norton, River of Life Staff member

2. Janelle, Penny, and Abraham World Vision national office staff members

3. Karsto Kwazira, FAO National Office, Project Officer

4. Sehliselo Nkomo, Village Head, Nkayi District

5. Stanley Zondo, AGRITEX officer Umzingwani District

6. Million Togara, AGRITEX officer Bubi District

7. Dumisu Mathutu- Ex AGRITEX officer and Ex Christian Care staff member Nkayi District

8. Mandazenkosi Tshuma- Councillor, Ward 16, Dakamela

9. Brian Oldreive, Foundations for Farming

10. Craig Deall, Foundations for Farming

48

ANNEX 8 Documents reviewed.

A. OTC Project documents-Planning

o OTC Budget yr 5 June’13 Micah Updated Budget for 2013-2014

o OTC Comp M and E- Proposal –Final Rvsd 20 March’14[1] Complementary Monitoring and Evaluation Proposal for 2014

o OTC Proposal Years 4-6 (General) Updated Narrative Proposal from May 2012

o OTC Proposal Years 4-6 rev’d 11 June’13-final Updated Narrative Proposal from June 2013

o Complementary OTC Report Sept 2013 2012 Proposal for Complimentary Monitoring and Evaluation programme

B. Project documents-Reporting

o Cell Phone M&E Summary of Impact February 2014 report on cell phone platform use

o Cell Phone Report –January 2014 Report on data gathered from August 2013 through January 2014

o Complementary M & E Objectives Report Sept 2014 Monitoring report for October 2013 through September 2014

o Evaluation Targeted Communities Map 6-1-2015 Zimbabwe map showing the location of evaluation target sites

o Operation Trumpet Calll – Cell Phone Data Report_FINAL Sept 1 2014 April – July 2014 Report of Cell Phone Data

o OTC evaluation sites region Zimbabwe rainfall map showing the OTC sites and the location of

evaluation target siteso OTC Evaluation Sites

Zimbabwe provincial map showing the OTC sites and the location of evaluation target sites

o OTC HEA-Impact Report 2014_FINAL Sept 1 2014

49

July 2014 report on household economic assessment and project impacto OTC Monitoring Score sheet for Cell app

Field Visit monitoring tool o OTC Post-harvest report_FINAL Sept 2014

2013-2014 growing season post-harvest report from July 2014o OTC results for Eddie

Graphic comparative summary of OTC yields by province and yearo OTC Yr 5 Annual Narrative report TearUK with edits 02Dec14 bn (2)_OTC (3)

Year 5 Annual report to Tearfund UK from September 1, 2014o Tearfund OTC only Financial report Yr5 rev’d 8Sep14

2013-2014 OTC financial report

Other Resource Materials-see Bibliography in ANNEX 15

50

ANNEX 9: Interview Guides-FGDs, Farm Visits, Stakeholder Interviews

FGD GUIDE QUESTIONS: OTC Trained--OTC Spontaneous Adopters--Other trained CA---Conventional Farmers

(Circle one)

FGD Facilitators_________________________________ Date_____/0_/2015

Community/District______________________/___________________________

1. General Information

Respondent: 1 2 3 4 5 6

Male/Female

How many people eat from the same pot in your home?

What are your family’s main livelihoods? Which is the most important (circle)? (probe poultry, remittances, etc.)

How much land does your family hold?

How many head of cattle/goats and sheep) do you have? / / / / / /

2. Has the life of your family changed because of adopting FfF and CA? If so, how?

3. How has the adoption of FfF impacted women, men, boys, and girls differently?

Women?

Men?

Girls?

51

Boys?

4. What are the main challenges in adopting FfF and CA? (Identify importance as ‘high’, ‘medium’, or ‘low’)

5. What has made FfF and CA adoption possible or easier? (Identify importance as ‘high’, ‘medium’, or ‘low’)

6. How has working through the local church impacted the project? Have there been benefits to this strategy? Does I t cause challeges?

7. Do you have any suggestions for improving the FfF programme? (Identify value attached to each suggestion as ‘high’, ‘medium’, or ‘low’)

52

FARM VISIT GUIDE: OTC Adopter

Visit Facilitators___________________________________ Date_____/__ /2015

Community/District_______________________/___________________________

Homestead Representative/s____________________________________Male/Female_________

1. General Information

How many people eat from the same pot in your home?

What are your family’s main livelihoods? Which is the most important (circle)? (probe poultry, remittances, etc.)

How much land does your family hold?

How many head of cattle/goats and sheep) do you have? /

2. CA Adoption

How did you learn about CA? (probe for various different sources of knowledge)(Circle most valuable source)

How much land do you have under CA this season? How much did you have last season? / How do you prepare your CA

plantings? (hoe-ripper-ox drawn ripper-planter-tractor pulled planter-other ?) Who does the work? (man-women-girl-boy) man-women-girl-boy

How long did it take to prepare land this year? (in terms of person days) (holing out/ripping, mulching, manure/fert application, planting)

What date did you first plant on this year?

When is the ideal planting date, based on the rains? /

What do you use for fertilizer? (manure-compost-AN-Urea-Compound D-anthill soil-humus-etc)Do you use herbicide to control weeds every year? (not spot spraying)

Do you weed manually? If so: How many times a year do you weed? /How long does it take per weeding (in person days)

53

Who does the weeding? (man-women-girl-boy)

man-women-girl-boy

Did you keep your field weed free over the winter?

OBSERVATION: How well are weeds controlled in the plot?

Very well—well—poorly---not at all

OBSERVATION: How do the crops look? (on a scale of 1-5)

1---2---3---4---5

OBSERVATION: Is there a rotation in evidence? IDENTIFY ROTATION IF ANY. (IDENTIFY CURRENT CROPS AND 2013/2014 CROPS)OBSERVATION: Is the CA plot mulched? If so, how much of the plot is mulched?

(0---1/4----1/2----3/4----all)

OBSERVATION: How thick is the mulch?

(In cm)_____________

OBSERVATION: How much maize does the family have in stock now? (specify in 50-90-100 kg bags)What crops did you plant on your CA plot last season?

What land size was planted to each crop?

How much did you harvest from your CA plot last season? (per crop; identify bag weight)

How did you get your seed?(purchase-saved-gift-donation)

Did you use hybrid seed?

How much seed did you use? (in the case of replanting, the amount from last planting)

What did you do with your stover from last season’s crop?

What is the soil type (sand, loam, clay)

3. Conventional tillage

54

How much land do you have under conventional plantings this season? How much did you have last season? /How do you prepare your conventional plantings? (hoe- ox drawn plow-tractor plowed-other ?)Who does the work? (man-women-girl-boy)

man-women-girl-boyHow long did it take to prepare land this year? (in terms of person days) (plowing, planting, applying manure/fertility)

What do you use for fertilizer on your conventional plantings? (manure-compost-AN-Urea-Lime-Compound D-anthill soil-humus-etc.)Do you use herbicide to control weeds?

Do you weed your conventional plantings manually? If so: How many times a season do you

weed? /How long does it take (in person days)

Who does the weeding? (man-women-girl-boy)

man-women-girl-boy

Did you keep your conventional field weed free over the winter?

OBSERVATION: How well are weeds controlled in the plot?

Very well—well—poorly---not at all

OBSERVATION: How do the crops look? (on a scale of 1-5)

1---2---3---4---5

OBSERVATION: Is there a rotation in evidence? IDENTIFY ROTATION IF ANY. (IDENTIFY CURRENT CROPS AND 2013/2014 CROPS)What are your main crops under conventional planting

Are they sole cropped or intercropped?

How much did you harvest from your conventional plots last season? (per crop)

Do you plant the same crops in the same place every year?

How did you get your seed?(purchase-saved-gift-donation)

Did you use hybrid seed?

55

How much seed did you use? (in the case of replanting, the amount from last planting)

What did you do with your stover from last season’s conventional crop?

What is your soil type (sand, loam, clay)

OBSERVATION: How much GRAIN does the family have in stock now? (specify in 50-90-100 kg bags)

INFORMAL INTERVIEW WITH FAMILY MEMBERS:

HOW HAS CA ADOPTION IMPACTED THE FAMILY OF THOSE WHO HAVE ADOPTED?

56

FARM VISIT GUIDE: Conventional Farmer

Visit Facilitators___________________________________ Date_____/__ /2015

Community/District_______________________/___________________________

Homestead Representative/s____________________________________Male/Female_________

4. General Information

How many people eat from the same pot in your home?

What are your family’s main livelihoods? Which is the most important (circle)? (probe poultry, remittances, etc.)

How much land does your family hold?

How many head of cattle/goats and sheep) do you have? /

5. Conventional tillage

How much land do you have under conventional plantings this season? How much did you have last season? /How do you prepare your conventional plantings? (hoe- ox drawn plow-tractor plowed-other ?)Who does the work? (man-women-girl-boy)

man-women-girl-boyHow long did it take to prepare land this year? (in terms of person days) (plowing, planting, applying manure/fertility)

What do you use for fertilizer on your conventional plantings? (manure-compost-AN-Urea-Lime-Compound D-anthill soil-humus-etc.)Do you use herbicide to control weeds?

Do you weed your conventional plantings manually? If so: How many times a season do you

weed? /How long does it take (in person days)

Who does the weeding? (man-women-girl-boy)

man-women-girl-boy

Did you keep your conventional field weed free over the winter?

OBSERVATION: How well Very well—well—poorly---not at all

57

are weeds controlled in the plot?OBSERVATION: How do the crops look? (on a scale of 1-5)

1---2---3---4---5

OBSERVATION: Is there a rotation in evidence? IDENTIFY ROTATION IF ANY. (IDENTIFY CURRENT CROPS AND 2013/2014 CROPS)What are your main crops under conventional planting

Are they sole cropped or intercropped?

How much did you harvest from your conventional plots last season? (per crop)

Do you plant the same crops in the same place every year?

How did you get your seed?(purchase-saved-gift-donation)

Did you use hybrid seed?

How much seed did you use? (in the case of replanting, the amount from last planting)

What did you do with your stover from last season’s conventional crop?

What is your soil type (sand, loam, clay)

OBSERVATION: How much GRAIN does the family have in stock now? (specify in 50-90-100 kg bags)

INFORMAL INTERVIEW WITH FAMILY MEMBERS:

HOW HAS CA ADOPTION IMPACTED THE FAMILY OF THOSE WHO HAVE ADOPTED?

GUIDE QUESTIONS-Key Informant

Facilitator/s________________________________ Date_______/________ /2015

Name & Post of Key Informant_____________________/______________________

58

1. What main impact has the adoption of FfF-CA had in the life of the families and communities that have adopted them? (especially in terms of food security and/or hunger coping mechanisms)

2. What are the main challenges in adopting FfF-CA?

3. What impact have you observed on community relations due to FfF-CA? Have you observed any positive impacts? Negative impacts?

4. What were the most important factors that give success in promoting FfF-CA?

5. What are key factors that improve the adoption of FfF-CA that are not related to Operation Trumpet Call activities?

6. How do you perceive the impact of the church as an agent of change in the communities? Have you observed particular benefits or challenges with this approach?

7. What concrete suggestions do you have for improving programming in FfF-CA?

59

ANNEX 10: Pfumvudza and Actual Sample Cost Analysis

Pfumvudza SEED-FERT COST per pack: $50 Estimated Days Labour per Ha: Cost per Ha (16 packs) $800 172.2Realistic Maize sale price: $265

Estimated Income per days labour:Agroecological Region 3 Average Yield: 6 MTTotal Sale per HA: $1,590Income after Seed Fert Cost: $790 $4.59

Agroecological Region 4 Average Yield: 3 MTTotal Sale per HA: $795Income after Seed Fert Cost: ($5) ($0.03)

Agroecological Region 4 Average Yield: 2 MTTotal Sale per HA: $530Income after Seed Fert Cost: ($270) ($1.57)

CA Profitability Based on Pfumvudza costs & estimated yields

60

0 SEED-FERT COST per pack: $0 Estimated Days Labour per Ha: Cost per Ha (16 packs) $0 172.2Realistic Maize sale price: $265

Estimated Income per days labour:OTC Average Yield (2014 as per Eval.:) 1.447 MTTotal Sale per HA: $383Income after Seed Fert Cost: $383 $2.23

OTC Average Yield (2014 as per OTC Reporting:)2.472 MTTotal Sale per HA: $948Income after Seed Fert Cost: $946 $3.57

CA Profitability Based on 0 costs & actual estimated yields

61

7. The design used for this evaluation is recommended for replication for other projects after incorporating a number of key adjustments

What: This evaluation design is recommended for use in evaluating similar Tearfund supported food security projects in Zimbabwe and other countries.

Person responsible: To be determined (Tearfund)

When: As needed; to be determined by Tearfund

62

ANNEX 11: Terms of Reference for the Evaluation

Foundations for Farming Impact Evaluation Terms of Reference

Approval Title: n/aSummary: Evaluating the impact of Foundations for

Farming in Zimbabwe to support replication of a church based agriculture approach in the East and Southern Africa region

Location and Region: Zimbabwe, ESATConsultancy Coordinator/Manager:

Ben Nicholson (Zimbabwe), Claire Hancock (UK based),

Country Representative/ Programme Director:

Ben Nicholson (Zimbabwe)

Evaluation lead (internal/external)

(TBC)

Main Partner and/or Operational Team:

River of Life, Evangelical Fellowship of Zimbabwe and Foundations for Farming

Associate Partner: n/aDoes this assignment require the lead evaluator/evaluation team to have either one-to-one contact, regular or frequent contact with children, young people (under the age of 18) or vulnerable people? 1

Yes No

BudgetFor further guidance on number of days required and costs see the IPMS manual

Ledger Code: n/a Fund Code: n/a

Description £ GBP Flights and visa Consultancy fees (30 days)2

1 Please note: A DBS /police check is required when an external consultant has either one-to-one contact, regular or frequent contact with children and/or young and vulnerable people in the course of their assignment.

2 Notes: If the consultant charges VAT, ensure that this is included for all costs claimed by the consultant. Any individual consultancy agreement fees worth over £10,000 in total that is commissioned by anybody within the International Group must be approved and signed off by the Geographical / Deputy Geographical Head.

63

Travel TBCAccommodation and meals (?? days @ £??/day)

TBC

Equipment None neededMiscellaneous TBCTOTALTotal £XXX

Terms of Reference

1.BackgroundTearfund Zimbabwe has been supporting Foundations for Farming in Zimbabwe for over 10 years. Foundations for Farming (FfF) is an approach developed by Brian Oldreive in Zimbabwe in the 1980s in response to increasing soil infertility and degradation on the commercial farm he was managing3. Foundations for Farming is also the name of the training organisation run by Brian Oldreive.Foundations for Farming have developed an approach to conservation agriculture which is based on the 3 principles of conservation agriculture as agreed by FAO; minimum tillage, crop rotation and permanent soil cover, combined with 4 additional management principles. Foundations for Farming is a holistic approach which addresses the need to change values and attitudes focusing on whole life transformation and not solely sustainable agriculture. FfF is often outworked through the local church who train pastors and communities on the Foundations for Farming approach. River of Life is the ‘social action’ wing of the New Frontiers church in Zimbabwe who Tearfund has partnered with since 2002. Tearfund supports them to train farmers in Foundations for Farming in partnership with Evangelical Fellowship of Zimbabwe (EFZ) through a programme named Operation Trumpet call which is entering into its sixth and final year of activities.Foundations for Farming also run a separate training centre in Harare teaching their approach to conservation agriculture to secular and Christian organisations. Tearfund has encouraged staff and partners from East and Southern Africa region, including Mozambique, Malawi and Zambia, to attend the regular trainings run by FfF. Partners have then been replicating FfF’s approach to conservation agriculture in their own countries, with some adaptation to the local context.Led by the FAO there is a growing body of evidence for Conservation Agriculture as an effective approach for sustainable agriculture development, but there is also continual criticism and challenges of conservation agriculture from other academic sources. To respond to these challenges, continually improve effectiveness and provide an evidence base for future programmes, Tearfund is commissioning a study to

3 www.foundationsforfarming.org

64

understand the impact of Foundations for Farming in Zimbabwe through the Operation Trumpet Call programme, in collaboration with RoL and EFZ. This study will focus on the impact of the agricultural techniques of FFF, the methodology of implementation through the local church and the evidence for holistic transformation.A desk based review with be followed by a visit to Zimbabwe to review FFF's approach to conservation agriculture and extension in country and to advise Tearfund on what elements of FFF Tearfund should be supporting/replicating and also if Tearfund should be encouraging its partners to adjust the FFF approach in light of the critics.Tearfund has also recently completed a global corporate process of analysing sectors and approaches which are delivering impact for the communities Tearfund supports. These have been named core competencies and will be used to enable Tearfund to become more focused and effective in its work. Under the core competency for Food Security Foundations for Farming has been named as an approach which is already delivering impact. This commissioned study will provide further quantitative and qualitative evidence of the impact of FFF in Zimbabwe to feed into this corporate area of work and provide learning for other country programmes to inform their programme design.Previous analysis of the work of RoL/EFZ and Foundations for Farming are covered in the 2013 Zimbabwe Appeal Evaluation and the 2010 Zimbabwe Appeal evaluation. Various other reviews have been undertaken of varying quality, some internally by RoL in compliance with donor requirements. Tearfund has done its own analysis of data from the first 4 years of OTC.

2.Purposea) Evaluation Goal and ObjectivesThe aim of this assignment is to conduct an impact evaluation of Foundations for Farming in Zimbabwe. The goal of the study is to: Understand the impact of Foundations for Farming in Zimbabwe.and the key objectives are: Review FfF’s conservation agriculture approach and compare with alternative

conservation agriculture approaches. Document the impacts of Foundations for Farming in Zimbabwe, through

EFZ/RoL Operation Trumpet call and FfF research data, both in terms of the impact of FfF’s conservation agriculture approach and also the role of the church as an extension agent and vehicle for holistic change.

Review the effectiveness of the FFF approach in terms of evidence of a long term change at the farmer level and assess the role of OTC in establishing local ownership and the sustainability of the approach

Recommendations on whether RoL/OTC should scale up the FFF approach to increase coverage and how this could be implemented

65

Understanding the effectiveness of EFZ/RoL monitoring and evidence collection systems including their mobile phone platform

Advise Tearfund on key principles for replication and any adaptations that should be made – both in terms of the impact of FfF’s conservation agriculture approach and also the role of the church as an extension agent and vehicle for holistic change.

Provide a framework for analysing the impact of FfF’s conservation agriculture approach that can be used to replicate the study in other East and Southern Africa countries, including Malawi and Mozambique.

b) Tearfund OutcomesIn addition the evaluation will assess the contribution made by the intervention towards the 4 Tearfund corporate outcomes: Church Envisioned Communities Developed Polices Changed and Disasters Responded Toc) OECD-DAC CriteriaThis study will focus on the following OECD-DAC criteria. The full list of criteria is included here as a useful reference. Impact: The positive and negative changes produced by a development

intervention, directly or indirectly, intended or unintended. This involves the main impacts and effects resulting from the activity on the local social, economic, environmental and other development indicators. The examination should be concerned with both intended and unintended results and must also include the positive and negative impact of external factors, such as changes in terms of trade and financial conditions.

Sustainability: Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially sustainable.

Effectiveness: A measure of the extent to which an aid activity attains its objectives.

Relevance: The extent to which the aid activity is suited to the priorities and policies of the target group, recipient and donor.

[Additional areas that may be considered depending on the nature of the project/programme being assessed, to be discussed with consultant:] Efficiency: Efficiency measures the outputs -- qualitative and quantitative --

in relation to the inputs. It is an economic term which signifies that the aid uses the least costly resources possible in order to achieve the desired results. This generally requires comparing alternative approaches to

66

achieving the same outputs, to see whether the most efficient process has been adopted.

[Not deemed appropriate for this evaluation, listed as reference only] Coordination: The extent to which different actors’ interventions are

harmonised, promote synergy, and avoid gaps, duplication, and resource conflicts. Coordination can also be included in the effectiveness criterion rather than treated as a separate criterion. It is particularly relevant to humanitarian situations where there are multiple actors responding.

Coherence: The extent to which there is consistency across security, developmental, trade, military, and humanitarian policies, and to which all policies take into account humanitarian and human-rights considerations. Coherence is a policy-level issue that may not be relevant in single-agency, single-project evaluations.

3. Methodology1. Desk review of grey/published literature on the impact of other conservation

agriculture approaches and comparison of methodology to FFF’s conservation agriculture approach.

2. Desk review of existing impact data from Tearfund, RoL/EFZ and FFF highlighting areas where existing data meets and/or fails to address criticisms of the approach.

3. A field visit to Zimbabwe to review the FFF approach on the ground and engage directly with FFF and Rol/EFZ's agriculture team.

4. Collecting data to fill the gaps in the evidence base or, if more data is required, providing recommendations to Tearfund, FFF and RoL/EFZ on how to put the monitoring and evidence collection systems in place that will allow these gaps to be filled.

Further detail on the methodology, including how data will be gathered from communities and a dissemination plan back to the farmers and wider stakeholders, will be supplied by the selected evaluator.

4. Timings Briefing and debriefing of consultant Desk review – XX days? Travel and field visit – XX days? Dissemination of key findings in Zimbabwe to FfF, RoL, Tearfund and other

key stakeholders Submission of final report and presentation to Tearfund

Itinerary of planned visit: (TBC)

67

X days in Harare with FFF and meeting other stakeholders (including Ministry of Agriculture, other NGOs)X days visiting RoL/EFZ projects in the fieldX days with Tearfund staff including Jono Simpson and Ben Nicholson

5. Evaluation Leadership and Management The name of the Client who is commissioning and approving the work: Ben Nicholson and the East and Southern Africa team.Person(s) responsible for practical arrangements, travel arrangements, hotels etc in Zimbabwe: Priscilla Nyathi.Person(s) responsible for support in the UK: Claire Hancock

6.StakeholdersStakeholders who should be involved/consulted during the evaluation process:

Foundations for Farming, Rol/EFZ, Tearfund staff; Zimbabwe, Malawi, Mozambique and UK based External peer agencies and partners in Zimbabwe Government officials including Ministry of Agriculture staff and Project participants, community members and EFZ church pastors and lead

farmers

7.Evaluation OutputThe expected outputs of the evaluation are:

A stand-alone evaluation summary (see the Evaluation Summary Template) An evaluation report (see Tearfund recommended reporting format - being

updated) with the following sections: Executive Summary (no more than four A4 sides) Introduction / Background Methodology Context Analysis For each Aspect, a section in the form

o Findingso Conclusionso Assessment

68

Specific Actionable and Prioritised Recommendations Annexes (indicative)

o Draft Action Plano Terms of Reference for the Evaluationo Profile of the Evaluation Teamo Evaluation Scheduleo Protocols for the Evaluationo Documents consulted during the Evaluationo Persons participating in the Evaluationo Field data used during the Evaluation, including baselineso Bibliography

A self-evaluation of the evaluation using the BOND evidence principles

8.Assessment CriteriaThe following criteria are used to assess the contribution that the intervention has made to the Tearfund Outcomes and the OECD-DAC criteria. The intervention is to be assessed online using this google form as this ensures that the data is automatically captured. The form is also attached at Annex A.

a) Tearfund Outcomes: Tearfund outcomes are to be assessed using the scale below:

N/A 0 1 2 3

Intervention was not intended to contribute to the outcome

Intervention has made no contribution to the outcome

Intervention has made little contribution to the outcome

Intervention has made some contribution to the outcome

Intervention has made a significant contribution to the outcome

b) OECD-DAC Criteria: The intervention is to be assessed against the DAC criteria using the following scale:

0 1 2 3 4

Low or no visible contribution to this criteria

Some evidence of contribution to this criteria but significant improvement required

Evidence of satisfactory contribution to this criteria but requirement for continued improvement

Evidence of good contribution to this criteria but with some areas for improvement remaining

Evidence that the contribution is strong and/or exceeding that which was expected of the intervention

69

9.Utilisation of evaluation findings and recommendations

a) Dissemination of Findings

b) Action Plan: A draft action plan is to be developed as part of the evaluation report using the linked template. This is also attached at Annex B.

10. Assessment of the Evaluation The evaluation is to be assessed against the Bond Evidence Principles (Voice and Inclusion, Appropriateness, Triangulation and Contribution and Transparency). The assessment is to be completed using this google form as this ensures that the data is automatically captured. The form is also attached at Annex A:

70

ANNEX 12: Evaluation ScheduleJanuary-February 2015 FfF/OTC Draft Evaluation Itinerary-Zimbabwe

Notes:Sun

01/25/15 Travel to Harare; PM Briefing with Ken at Queensgate

No Tearfund or FfF staff are required; you are more than welcome, but it is a day of rest.

Mon

01/26/15

Introductions and Briefing with FfF/OTC staff; planning meetings with Evaluation Team/Review (trial) of Tools

1-2 hour briefing with FfF; most of the day dedicated to meeting with the 8 evaluation team

facilitators reviewing tools & methodologies

Tue01/27/1

5AM Interviews with peer organization/s promoting CA in Zimbabwe-To be determined; travel to Kwekwe.

Tearfund and FfF are expected to identify the best contacts with peer organizations, schedule

these interviews, and line up logistics; the interviews are to be carried out by the 8 Evaluation team facilitators in pairs (four

interview teams). At most 8 can be scheduled (2 per team).

Wed

01/28/15

Community Evaluation Process; Evaluation Team meeting & tool/methodology review

Pending confirmed field schedule; FfF/Tearfund expected to manage logistics; interviews with

local/regional

Thu01/29/1

5 Community Evaluation Process

Fri01/30/1

5 Community Evaluation Process

Sat01/31/1

5 Community Evaluation ProcessSun

02/01/15 Travel; Process Review

Mon

02/02/15 Community Evaluation Process

Tue02/03/1

5 Community Evaluation ProcessWed

02/04/15

Return to Harare; Evaluation Team Debriefing/Discussions/Identification of Findings

The 8 Evaluation Facilitators Are expected to identify the key findings

Thu02/05/1

5 Evaluation Team Meetings/Discussions/FindingsThe 8 Evaluation Facilitators Are expected to

identify the key findings and reccomendations.

Fri02/06/1

5 Debriefing with Tearfund, RoL, FfF, OTC staff

Debriefing will include a review/evaluation/feedback on the evaluation

process and reccomendations for replication

Sat02/07/1

5Data Analysis; Availability for work with Ken or other briefings/meetings Data Crunching (Ken and Chris)

71

ANNEX 13: Persons participating in the Evaluation

For Evaluation Team Members see ANNEX 1 on p. 34

For Stakeholders see ANNEX 6 on p. 41

For FGD Participants Summary see p. 7 & 8

For Farm Visit Participant Summary see p. 8 & 9

For individual farm visit participant data see Excel work sheet ‘Field Visit and FGD Data’ (available separately)

For individual focus group discussion summaries see ‘Field Visit and FGD Data’ (available separately)

72

ANNEX 14: Field data used during the Evaluation, including baselines

For OTC farmer data see Excel spread sheet ‘Field Visit and FGD Data’; see ‘OTC Farms’ worksheet (available separately)

For baseline data see Excel spread sheet ‘Field Visit and FGD Data’; see ‘Conventional Farms’ worksheet (available separately)

For individual focus group discussion summary data see Excel spread sheet ‘Field Visit and FGD Data’; see ‘FGDs’ worksheet (available separately)

ANNEX 15: Bibliography

BibliographyBISHOP, D. (2015). Livilhoods in Zimbabwe, Evaluation of food security and strengthening livelihood

options. Oxford: OXFAM.

Bunch, R. (2012). Restoring the Soil: . Winnipeg: Canadian Foodgrains Bank.

Chinamasa, P. (2014, November 27). Budget Policy Statements. Retrieved from The Ministry of Finance and Economic Development: http://www.zimtreasury.gov.zw/budget-policy-statements

Food and Agriculture Organization of the United Nations. (2010). Zimbabwe: Transitioning Emergency into Rehabilitation and Development Plan of Action 2010-2015. Harare: FAO.

Giller, K. E., Witter, E., Corbeels, M., & Tittonell, P. (2009). Conservation agriculuture and smallholder farming in Africa: The heretics view. 114.

Johansen, C., Haque, M., Bell, R., C, T., & R.J., E. (2012). Conservation Agriculture for small holder rain fed farming: Opportunities and constrains of new mechanized seeding systems. 132.

Kassam, A., Friedrich, T., Shaxson, F., Bartz, H., Mello, I., Kienzle, J., & Pretty, J. (2014). The spread of Conservation Agriculture: policy and institutional support for adoption and uptake. 7.

Mazvimavi, K., & Twomlow, S. (2009, June). Socioeconomic and institutional factors influencing adoption of conservation farming by vulnerable households in Zimbabwe. Agricultural Systems, 20-29.

73

Pannel, D. J., Llewellyn, R. S., & Corbeels, M. (2013). The farm level economics of conservation agriculture for resource-poor farmers.

Pedzisa, T., Rugube, L., Winter-Nelson, A., Baylis, K., & Mazvimavi, K. (2015). Abandonment of Conservation Agriculture by Smallholder Farmers in Zimbabwe. Journal of Sustainable Agriculture, 69-82.

Pittelkow, C. M., Liang, X., Linquist, B. A., van Groenigen, K. J., Lee, J., Lundy, M. E., . . . van Kessel, C. (2014). Productivity limits and potentials of the principles of conservation agriculture. (13809).

Woodring, C., & Braul, A. (2011). Conservation Agriculture in Zimbabwe Evaluation Report. Winnipeg: Canadian Foodgrains Bank.

Woodring, C., & Braul, A. (2011). Conservation Farming in Zimbabwe Evaluation Report. Canadian Foodgrains Bank.

Word Bank. (2015, February 13). Major Contracts FY 11. Retrieved from finances.worldbank.org: https://finances.worldbank.org/fr/Procurement/Major-Contracts-FY-11/5uuz-kv7b/alt?&page=242

ANNEX 16: A self-evaluation of the evaluation using the BOND evidence principles

This is pending the feedback and contributions of OTC, Tearfund, and other stakeholders.