consortium iea management response...dcl (dryland cereals and legumes afs) with eight legumes and...

24 March 2016

CGIAR Consortium Board Twenty-Fourth Meeting CB24-11 Virtual, 29 March 2016 of 1

CB24-11 For Information

IEA CRP Evaluation of Roots Tubers and Bananas – Consortium Management Response

Purpose: This document annexes the proposed Consortium Management response to the Independent Evaluation Arrangement (IEA)) evaluation of the Roots Tubers and Bananas (RTB) CGIAR Research Program (CRP), presented as Annex 1 of this document. The document presents two additional annexes for reference:

Annex 2: Executive Summary of the IEA evaluation

Annex 3: CRP Management Response

1. The evaluation was commissioned according to Section 23 of the CGIAR Consortium-Fund Council Joint Agreement1.

2. The management response has been prepared in line with the Consortium Office’s standing processes for review of IEA final evaluation reports.

1 The April 15, 2011 agreement entered into between Bioversity International (on behalf of CGIAR before

establishment) and the Fund Council (as represented by the International Bank for Reconstruction and Development), and approved by the Fund Council on April 5, 2011 and the Consortium Board on May 5, 2011.

CB24-11 Annex 1


Consortium Management Response to the External Review of RTB The CGIAR Research Program on Root, Tubers and Bananas (RTB) is one of the commodity CRPs dealing with specific challenges such as (i) integrating six different crops - all of them clonally propagated - in a unique program; (ii) identifying key priorities for research in the multiplicity of combinations between [crop x traits x users] for breeding in parallel with the diversity of related crop management approaches; and (iii) setting up and implementing a substantial restructuring of the program, moving from a thematic-based structure to a multi-disciplinary, integrating and outcome-based program. The recommendations of the evaluation have taken into account these specificities and the successful RTB realignment in Phase 1, very transparently mentioned in the RTB Annual Reports 2012-14 and the Extension Proposal for 2015-16. Substantial analysis and concrete proposals have been synthesized by the panel and are introduced adequately and contextualized in the text of the Evaluation Review. Four of the recommendations have specific relevance for the Consortium: Recommendation 3: More strategic allocation of W1/2 funds based on program priorities and performance Consortium response: This recommendation will need to be part of the review process for the new RTB proposal submission to ensure strategic allocations of W1/W2 funds. The Consortium considers that there are several examples where this may be appropriate, e.g. with the main objective of boosting innovative pre-breeding approaches, conducting field screening of genebank accessions for specific crops, selecting traits and locations of interest (where these are not necessarily W3 or bilaterally funded), testing multi-disciplinary integrated approaches combining host plant resistance and on-farm management practices, understanding the mechanisms and practicalities for up-scaling RTB technologies or securing the implementation of RTB Result-Based Management framework. Recommendation 5: Increase high quality science publications Consortium response: Agreed, and will need to be closely monitored as part of the ongoing CRP Performance Monitoring Program. The Elsevier bibliometric analysis (2014) identified this as a weakness. In addition, the Elsevier Report identified that between 2012 and 2014, the number of program-generated publications (147) by researchers (60) gave an average of 2.5 papers per RTB researcher, which is quite low when compared with other CRPs. In this analysis, 14 program-related researchers have not yet produced any publication within RTB, probably because of the quite recent starting date of this program (January 2012). Taking into account that quality of science will be even more important for monitoring CRP performance and budget allocation, this is a crucial matter to be followed up by the System Office in phase 2.

CB24-11 Annex 1


Recommendation 7: Modernizing and strengthening RTB breeding programs Recommendation 8: Harmonizing breeding approaches within crops and transferring lessons across crops Consortium response: These two complementary recommendations are a matter of importance for the proposed 8 AFSs and will need to be closely scrutinised by the Consortium and subsequently the System Council. It will include the clear connectivity of the AFS programs with all three proposed platforms. The Remaining Recommendations of the Evaluation The Consortium appreciates the CGIAR-IEA (2016) Evaluation of CGIAR Research Program on Root Tubers and Bananas and strongly concurs with the majority of the recommendations of the panel, summarized as follows: Recommendation 1: Enhance integration beyond individual time-bound Centers’ projects on the same topic (e.g. crop or trait) Consortium response: Agreed. The essence of the CRP in bringing together research on these several commodities is to look for complementarities between Centers’ activities – and other stakeholders’ inputs - with the aim of creating synergies and avoiding redundancies to improve efficiency at the program level. This recommendation is of importance to RTB with 4 CGIAR Centers participating (CIP, Bioversity, CIAT and IITA) and a 6-crop mandate. It is similarly applicable to this multi-crop AFS CRP in phase 2 and - more generally - to emerging CRPs as DCL (Dryland Cereals and Legumes AFS) with eight legumes and four cereals in phase 2. Recommendation 2: Improved congruence between Clusters of Activities and the FP problem definition Consortium response: Agreed. The restructuring of the program into inter-disciplinary and integrated FPs compared to the old Theme-based structure was a long and complex process that satisfactorily concluded with a more coherent RTB CRP pre-proposal for phase 2. The resulting FP2 and FP3 are extremely inter-disciplinary with CoAs mixing breeding approaches and agronomic practices. For example, FP2 on “Adapted productive varieties and quality seed of RTB crops” proposes to develop new hybrids in banana (BA 2.2), cassava (CA 2.3), sweet potato (SW 2.6) or yam (YA 2.7) in parallel with actions focusing on markets’ needs and preferences in banana (BA 2.2) or Integrated approaches for potato seed quality (PO 2.4). The congruence between CoAs and FPs has also been improved for FP1, 4 and 5. Recommendation 4: RTB should use priority assessment results for setting program priorities and in program planning, including fundraising. Consortium response: Agreed. A substantial prioritization exercise was carried out by RTB in 2012 through an online expert consultation in coordination with the most important regional organisations in Africa (ASARECA, CORAF, CCARDESA), Latin America (IICA) and Asia. Five key

CB24-11 Annex 1


options combining crops and traits/technologies were identified (more than 1,680 responses) and have guided the selection of prioritized CoA in RTB proposals for phase 2. The results of this survey are integrated and translated into the new articulation of the RTB preproposal for phase 2, based on RTB stakeholders’ prioritization effort. The follow up of these priorities at the regional level will be facilitated in phase 2 by the new proposed FP5 (RTB preproposals) which includes four geographically-based CoAs on mix RTB crop-tree-livestock farming systems, which are inherited from the disrupted HumidTropics CRP. Recommendation 6: Increase monitoring of quality of science implemented and generated by the program Consortium response: Agreed. For the RTB program to be responsible for the quality of science implemented and generated by the program, (i) additional strategic alignment needs to take place between the participating Centers’ strategy – including for quality of science - and the strategy of the CRP to which the Center contributes, and, (ii) the repartition of roles and responsibilities has to be very clearly designed between the individual Centres responsible for the performance of their contracted scientists. Better alignment will probably result in future bilateral grants being designed to contribute to a CRP – RTB in that case - and would help to reduce the ‘tension’ between the use of bilateral funds and the use of W1-2 funds by the CRP. Overall performance is yet broader than this, including other external stakeholders (ARIs, NARS, private companies, etc...) not just Centers’ scientists. In terms of management, the CRP Management Unit will need to include FP leaders, experienced senior research staff and top quality collaborations with ARIs with the aim of delivering high impact publications. The MEL specialist (Monitoring, Evaluation and Learning) – theoretically a member of the management unit - will need to be deeply involved in the CRP performance assessment, including obviously the Quality of Science as a key criterion. Recommendation 9: More efficient last mile delivery Consortium response: Agreed. By being more selective regarding the number of [crops x traits x users] the program will be able to commit jointly with NARS on the delivery of a few products (2 to 3) for each of the RTB mandate crop. The corresponding adoption strategy will be decided jointly with NARS with a very clear and approved partition of roles and responsibilities. In terms of capacity building/strengthening the needs for developers with multidisciplinary skills (market analysis, niche, innovation brokers, seed specialist, etc...) able to bridge the gap between research outputs (after the concept test is proven) and their adoption at higher scale as research and development outcomes, will need to be addressed by the NARS with a stronger mid- and long-term support and commitment from the CRP and - a point probably seconded by most of the development agencies - with the corresponding sustainable W 1/2 funding. This is a common matter of importance for all the AFS CRPs that will need to be followed up by the Consortium/System Office in phase 2.

CB24-11 Annex 1


Recommendation 10: Setup a researchers’ community of practice cutting across all RTB crops Consortium response: Agreed. We strongly support the setup of such a community within RTB breeders and scientists with the aim of sharing ideas on methods, data, results and user feed-back. That could lead to integrated data platforms, inter-Center working groups on traits or enhanced inter-disciplinarity between lab-genomics and field-breeding. This community should include the new pre-breeders in order to secure the fluent connection with the genebanks’ scientists at the platform and Centers/CRP levels and with the genetic gains and Big Data platforms. Adopting a similar approach, a community of scientists covering the 7 commodity CRPs was created in June 2014 during the CRP leaders meeting (Montpellier; France) with the aim of sharing information, insights, experience and tools in an area of common interest linked to breeding or plant science. Based on its ToR, the proposed “Genebank & Commodity CRPs CoP”, focused either on a professional discipline (managers, principal investigators, pre-breeders, breeders, other scientists, etc...), on a skill (program/project management, molecular biology, cell biology, quantitative genetics, pathology, bioinformatics, IT systems/data management, genebank collection/characterization, etc...) or on a topic (specific crop, Genomic Selection, GWAS, QTL mapping, cryo-conservation, double haploid production, TILLING, etc...). The Consortium recommends that linkages amongst RTB scientists are enhanced within the framework of a Community of Practice Recommendation 11: Establishment of economically sustainable seed systems for RTB crops Consortium response: Agreed. That is a key requirement for RTB as well as for all the current commodity and next AFS CRPs. For RTB the CO strongly supports the idea to hire a senior specialist in seed system analysis with technical skills in multiplication of vegetatively propagated crops and expertise in quality control for RTB seeds. Pathogen detection, germination and vigor testing, genetic purity control, seed lots homogeneity, seed treatment, packaging, labelling and logistic for distribution need to be addressed in order to establish a sustainable seed systems for RTB crops in phase 2. Key strategic questions on seed system also need to be handled by this expert - probably in collaboration with the private sector. Region-specific information on market demand for RTB seed/crops will help to determine the types of products to be developed, or to identify farmer response to the range of products under development. The combination of seed-system analyses and country case studies along the ‘seed value chain’ will allow RTB to identify systemic bottlenecks, and facilitate jointly with NARS the formulation of robust policies and strategies for specific country situations. The Consortium recommend that this position and the coverage of these areas should be made evident in the forthcoming phase 2 CRP.

CB24-11 Annex 1


Recommendation 12: Better integrate research on crop improvement (breeding) and crop management Consortium response: Agreed. The CO appreciate the huge effort that has been made in rebalancing crop improvement (breeding) and agronomic practices related to crop management in the new RTB preproposal for phase2, mainly for FP2, FP3 and even in FP4 by including the contribution of post-harvest and processing technologies for bio-fortified cassava and sweet potato. We also fully agree with the panel on the key principle indicating that “narrowing the yield gap for farmers may require rebalancing the RTB portfolio towards agronomic and soil fertility research”. Unfortunately the second research topic on soils is still missing in the RTB pre-proposal and could be developed, potentially in collaboration with the Water, Land & Ecosystems CRP. Recommendation 13: Increased focus on post-harvest research on the crop-specific aspects of value chain improvements Consortium response: Partially agreed. The urgency for further work in this area seems reduced, mainly because, as previously explained in response to recommendation 12, the CO considers that the program has made huge effort to include innovative, demand-led, postharvest technologies in the newly proposed FP4 (“Nutrition food and value added through postharvest innovation”) in their new proposal for phase 2. Recommendation 14: Secure adequate resources to develop and implement the needed strategy for communication, KM and CapDev. Consortium response: Agreed. The last three years, RTB has made a commendable effort to increase its visibility by communicating on program’s activities, outputs and outcomes, through an efficient communication and knowledge sharing website. The next step will probably be a better coordination between the RTB knowledge management (KM) strategy and the implementation of the Open-Access and Open-Data policy approved by the Centers in 2014. In phase 2, RTB will probably have a leading role in the design of the best tools/platforms and processes for integrating the genebank information with their genomic databases (e.g. excellent Cassavabase, Musabase, etc...) probably in collaboration with ARIs (e.g. BTI) and including the phenotyping and agronomic data. Recommendation 15: RTB impact assessment plan at the CRP level. Consortium response: Agreed. This IA plan has to be probably designed by the MEL specialists and directly linked with the monitoring and evaluation criteria. RTB should also apply lessons learnt from the SIAC program (Strengthening Impact Assessment in the CGIAR, 2015 Progress Report) - mainly on the advance methodologies for tracking the uptake and adoption of improved cassava varieties in Ghana – which shows that GBS (Genotyping by Sequencing) is becoming increasingly affordable in variety identification for adoption studies. Recommendation 16: More clarity to the respective roles, relationships and accountabilities of FP leaders, cluster leaders and project leaders within the management structures

CB24-11 Annex 1


Consortium response: Partially agreed. RTB is probably one of the clearest and most efficient program in terms of CRP Management Unit governance and role. The PMU and CRP leader should be congratulated for their transparent and fair management system which was well appreciated by FP leaders, IPs from external partners (CIRAD) and Centers in phase 1 and should be taken as a model for phase 2. Recommendation 17: Greater description of partners’ involvement in management & governance Consortium response: Partially agreed. In phase 1, RTB put emphasis on strengthening partnership amongst the 4 Centers and CIRAD making up the programmatic approach. Partners are well represented in the CRP management unit and the steering committee. Collaborations with NARS for sharing technicians or jointly publishing with national scientists is well-established. Obviously the program could go further in phase 2 by handling of joint appointments, handling joint undertakings and codes of conduct in program participation. The conclusion by the panel that boundary partners of RTB are not sufficiently aware of RTB’s roles and activities, is very likely applicable for a number of other CRPs. The Consortium recommends that boundary partners as well as donors (Fund donors and bilateral donors) have to be better included in priority-setting and implementation of the CRP in phase 2 through an action plan aimed at (i) raising the awareness of boundary partners, (ii) better understanding the concrete needs of strategic partners and boundary partners and (iii) better communicating with its key donors about the synergies the CRP creates through closer integration of bilaterally funded projects with W1-2 funded work. Consortium assessment of the CRP Management Response With one exception we concur with the RTB management response to the IEA report and associated action plan. Indeed many of the recommendations have been translated into actions incorporated into the phase 2 RTB full proposal. However the RTB management response is silent on recommendations 7 and 8 as described above; both these recommendations emphasise the need to adopt novel breeding techniques based on genome editing and reverse breeding. Adoption and testing of these new technologies will also help improve the quality of science publications (recommendations #5). As indicated earlier, this will require careful consideration by the Consortium and System Council. The Consortium thanks the evaluation panel Chair and his team for producing a well-argued and readable report of utility to both the RTB CRP and its staff and stakeholders, together with clear guidance for the development of phase 2 programs.

vii

Evaluation of CGIAR Research Program on Roots, Tubers and Bananas

cgiar.iea.org

Executive Summary

Background and context

Root and tuber crops and bananas are an essential staple food for the poor in developing countries.

With a mean production of 685 million tons on 55 million ha in 2006–2008, the RTB crops represent

the second most important set of crops in developing countries after cereals. Some 400 million tons

are consumed as fresh or processed food; the remainder is used as animal feed, planting material, or

industrial raw material. Production and use of RTB crops tends to be concentrated in countries with

lower per‐capita incomes. RTB crops are excellent sources of cheap energy and some varieties are

rich in vitamins and essential minerals. They are true food security crops and are mostly produced,

processed, and traded locally, contributing to the sustainability of cropping and production systems

and helping to reduce the risk of food shortages and nutritional shortfalls. RTB crops also play an

important role in the livelihoods of women.

The CGIAR Research Program (CRP) on Roots, tubers and bananas (RTB) is led by the International

Potato Center (CIP) and brings together the RTB crop‐related work of CIP, Bioversity International

(Bioversity), the International Center for Tropical Agriculture (CIAT), the International Institute of

Tropical Agriculture (IITA) and the French agricultural research and international cooperation

organization, CIRAD, as well as about 300 partners. The aim of RTB is to more fully realize the

potential of RTB crops (banana/plantain, cassava, potato, sweetpotato, yams, and other tropical and

Andean root and tuber crops—sometimes termed ‘vegetatively propagated staple crops’) for

improving nutrition, income generation, and food security—especially among some of the world’s

poorest and most vulnerable populations.

The CRP on RTB was initially approved for three years and began to operate in January 2012. In 2014

an extension phase 2015-2016 was approved. The original structure of RTB was made up of seven

disciplinary-based Themes each with a mix of existing, expanded and new Product Lines (PL) as well

as cross-cutting activities. A detailed priority assessment (PA) exercise was carried out in 2013-2014

and RTB began to reorganize its program structure into Flagship Projects (FP) and Clusters of

Activities (CoA) in 2014. The FP program structure presented in the extension proposal was

expanded and refined in the recently submitted pre-proposal for Phase II to be initiated in 2017. In

2016 RTB will begin to transition to this new structure. The program structure will be further revised

in the preparation of the full proposal.

Purpose, scope and objectives of the evaluation

The primary purpose of the evaluation is to enhance the contribution that RTB is likely to make to

reaching CGIAR goals, in particular food security for nutrition and health. The evaluation aims to

inform decision-making and planning by program management, CRP sponsors, partners and other

stakeholders with respect to program performance and the potential options for the future. It is also

intended to serve the interests of the broader group of RTB partners and stakeholders.

ocussen

Typewritten Text

CB24-11 Annex 2: IEA Final Report Executive Summary

viii


cgiar.iea.org

The evaluation includes both summative and formative aspects. The summative part encompasses

an assessment of research outcomes, primarily from pre‐CRP research and outputs from current

research and the period leading to RTB. The formative aspects focus on current research and

evolution of RTB over the past four years from the perspective of program design and governance

and management (G&M) arrangements. The evaluation was also informed by the pre-proposal.

Furthermore, the Evaluation looks at the extent to which current research is influenced by feedback

from impacts derived from pre‐CRP research. The evaluation also addresses four cross‐cutting issues

as part of programmatic performance: gender, capacity development, partnerships and

communication and knowledge management (KM).

The Evaluation Team developed evaluation questions at two levels: overarching questions and

questions in connection with the key evaluation criteria of relevance, quality of science, efficiency,

effectiveness, impact and sustainability. Programmatic and organizational aspects of RTB were

considered as distinct entities for defining the criteria‐specific evaluation questions.

The evaluation addressed ten overarching questions related to CGIAR reform principles and RTB in

the new CGIAR structure:

How well does RTB operate as an integrated program - programmatic-level thinking,

strategy, inter-center research activities and management?

Has the implementation of RTB realized anticipated synergies and complementarities from

centers working more closely together especially on the same crop, added value to research

on RTB and improved its prospects of achieving its objectives and contributing more

efficiently and effectively towards the program’s IDOs and SLOs?

Are the CGIAR reforms assisting RTB to deliver its objectives, achieve program IDOs and

contribute to SLOs?

Is RTB priority setting effective in terms of program coherence and focus of research on its

intended objectives?

To what extent shall the new RTB program structure based on discrete “business cases” of

crop-specific activity clusters contribute to or impede the program’s ability to achieve IDOs

and SLOs?

How is the long and continuing process of change in RTB to Flagship Projects affecting the

management burden and transaction costs, and affecting relationships with partners?

Is RTB designing and shaping future partnerships to articulate a sustainable research project

portfolio?

Are the impact pathways in the RTB structure sufficiently specified regarding target

beneficiary groups and alternative research and industry providers, and are they clearly

formulated and used in program monitoring and management?

In the current complex funding environment, has RTB been able to manage multiple sources

of funding to assure strategic coherence around highest priority areas of research?

To what extent do the G&M structures and practices of RTB contribute to or impede the

achievement of program coherence and effectiveness?

ix


cgiar.iea.org

Questions for the evaluation of G&M focused on legitimacy, accountability, transparency, conflicts

of interest and efficiency, and aspects of program management, including effectiveness, financial

management, resource mobilization, monitoring and reporting, collaboration, and risk management.

Moreover, the evaluation explored the effects of CGIAR reform on the efficiency and likely success of

the program.

Approach and methodology

The main impetus to bring research on root and tuber crops and bananas together in a single

program was to capture synergies due to commonality of the crops and their value chains and

potential efficiency gains. The Evaluation therefore focused on three main research areas where

there is greatest potential for integration of research across crops. These areas were: the breeding

pipeline; high quality planting material; and post-harvest management, value chains and marketing.

The Team based its findings, conclusions and recommendations on data collection and analysis from

several sources:

Case studies – PLs within Themes

Review of essential documents of the program, its approval, planning and reporting.

Documents for sample projects

More than 300 interviews with RTB policy makers, managers, researchers, partners, donors

and other stakeholders

Field visits to Belgium, Colombia, France, Kenya, Nigeria, Peru, Philippines, Tanzania, Uganda

and Vietnam

Virtual attendance at one Independent Steering Committee (ISC) and one Management

Committee (MC) meeting

Survey among RTB researchers

Bibliometric analysis of RTB publications

In-depth analysis of 79 journal articles

H-index analysis of 106 RTB researchers

Impact narratives prepared by CIP, CIAT, IITA and Bioversity

As the evaluation took place when RTB had been operational for less than four years, many research

results and impacts were attributed, at least in part, to pre‐CRP research done at the partner

centers.

Main findings and conclusions

Overall, the Evaluation Team concludes that in spite of the complexities and challenges of

successfully implementing a multi-crop and multi-partner CRP, RTB has made notable progress in the

past four years and is already delivering results, in spite of budget cuts. RTB is well-directed and

reaching a reasonable number of its near-term milestones and is working towards achieving its

goals, particularly those concerning productivity and nutritional improvement for some of its crops.

RTB is successfully delivering added value across centers and crops in several areas mainly through

x


cgiar.iea.org

strategically important complementary projects that are funded from Windows 1 and 2 (W1/W2).

The momentum achieved through this initial progress needs to be further developed into fully

integrated research programs on cross-center crops. RTB strongly warrants being continued as CRP

beyond the extension period.

Generally, RTB is a coherent program with a strong global comparative advantage based in its

stewardship and access to extensive well-characterized global germplasm collections of major RTB

crops; scientific capacity in human resources and research infrastructure; strategic locations in

production environments for next users; and broad range of often long-term partnerships with

public, private, and development organizations. RTB should continue to assess its role and priorities

in the light of the growing strengths of some National Agricultural Research Systems (NARS) and

increasing involvement and interest of the private sector in RTB crops, particularly for basic seed

production and processing.

The quality of RTB science is sound, overall, and novel approaches are being used to generate IPGs

but greater efforts are needed to improve the quality of research publications. The incorporation of

skills and expertise from noted Advanced Research Institutes (ARI) has enhanced the quality of the

outputs produced. The program’s effectiveness can be further improved through closer

collaboration among breeders; strengthening CRP expertise in seed systems; improved integration of

crop improvement and management (agronomy and soil fertility management) technologies; and

enhanced focus of post-harvest research on the crop-specific aspects of value chain improvements

that can deliver added value. Links with some CRPs are well-developed e.g. with Livestock and Fish

(L&F) and Humidtropics CRPs, for developing cassava peel-based feed but others are a work in

progress. NARS are appreciative of their enhanced roles and greater equity in RTB.

RTB has been successful in building new program G&M arrangements that largely meet the

challenges of integrating five crops, four centers and CIRAD. The management of RTB is well-

regarded throughout the CRP and the CGIAR. Good progress has also been made in implementing a

gender strategy.

Overall, RTB is a strong program that is addressing appropriate research issues for meeting the

challenges to increasing food and nutrition security, poverty reduction and gender and social equity.

A speedy move to using the outcomes of the PA for more strategic allocation of the budget across

crops and further programmatic integration will further enhance its strength.

Relevance

RTB’s research is strategically relevant to addressing the CGIAR’s objectives. Lessons learnt from

implementing the old program structure have been useful in informing the design of the new

structure based on inter-disciplinary and integrated FPs and CoAs. Although the new structure will

potentially improve coherence and should be more effective in the delivery of RTB’s outputs and

outcomes, RTB needs to further review and revise the CoAs for improved congruence with the FP

problem definition and the PA.

xi


cgiar.iea.org

RTB’s revised impact pathways should better facilitate it to reach CGIAR goals, the System Level

Outcomes (SLOs), measured through Intermediate Development Outcomes (IDO) at sub-level, thus

addressing one of the core principles of CGIAR reform1. Furthermore, the complementary projects

have been important in facilitating programmatic integration across CGIAR core skills, adding value

and capturing synergies and complementarities across centers and across crops. However, RTB

needs to make further efforts to enhance integration beyond individual time-bound projects. In

particular, further value would be gained by fully integrating the IITA and CIAT cassava breeding

programs and all RTB research on banana and plantain by IITA and Bioversity.

The strategic use of W1/W2 funds through sound and productive complementary projects has

enhanced the relevance of RTB research and strengthened partnerships for more effective

generation of priority research outputs, in spite of cuts to this funding. The Evaluation Team

encourages RTB to move rapidly towards greater strategic allocation of these funds based on

program priorities and highlights the need for the CGIAR to put more effort into raising W1/W2

funds for such projects. RTB should build on the PA already completed, and use results for setting

program priorities and in program planning, including fundraising. In doing so, RTB should also plan

for continuous improvements in the data and estimates that support PA.

RTB has a strong global comparative advantage as a main supplier of RTB research outputs relevant

and useful for small holders in Sub-Saharan Africa (SSA), Asia and Latin America. This comparative

advantage has been strengthened by selected partnerships with ARIs with complementary skills and

strong partnerships with NARS whose research has been developed by RTB centers. However, RTB

needs to carefully consider alternative research suppliers in future assessment of its comparative

advantage. The roles and responsibilities of each member of such partnerships will change over time

and RTB should continue to focus on the generation of IPGs and hand-over areas of research to

capable partners, enabling it to move on to other priority research.

Quality of science

The quality of science is generally good both from the centers themselves and through partnerships

with noted ARIs. The current RTB breeding programs are using both sound and novel approaches but

should continue to improve through the use of techniques such as gene editing, genetic

modification, marker assisted breeding (MAB) and genomic selection and utilizing new knowledge

generated from genomics research. Attracting young skilled scientists trained in these techniques

will strengthen the quality of science while training the next generation of plant breeders will help to

ensure continuity and sustainability.

Research on quality planting material is producing useful outputs for the development of

economically sustainable seed systems for major RTB crops and has provided opportunities for

developing cross-center and cross-crop partnerships and cross-crop learning. The incorporation of

skills and expertise from ARIs has enhanced the quality of the outputs produced. The addition of

1 Confirmation of the potential of the new program structure to more effectively contribute to the sub-IDOs awaits the implementation of the Phase II.

xii


cgiar.iea.org

CIRAD as a collaborating center has brought new intellectual energy to post-harvest work,

generating high quality efforts and publications. The complementary projects are bringing together

lessons from different countries and contexts to better inform technology development and

potential market interventions thus contributing to research quality. The nutrition research is well-

designed and results have the potential to provide a model for efforts in other crops or regions.

Continued attention to generating knowledge that provides global public goods for RTB will be

necessary.

RTB has published some excellent research papers in high impact journals. At the same time, the

percentage of publications in non-IF journals is disturbingly high – 39 percent. While it is recognized

that most RTB crops are at a disadvantage in terms of the range of journals willing to publish on

these crops and the need to target African journals to promote research findings to the most

appropriate stakeholders, RTB should endeavour to produce higher quality science in order to

publish in higher quality journals for greater international impact. Furthermore, in future, the MC

and ISC should play a more active role in monitoring the quality of science produced by RTB.

Program effectiveness

RTB has made notable progress towards outcomes in important research areas of its program: the

breeding pipeline; quality seed systems; resilient cropping systems; and post-harvest, value chains,

marketing and nutrition. Furthermore, the new program structure with its focus on outcomes and

plausible Theories of Change at FP level is likely to enhance program effectiveness in achieving

outcomes and contributing to sub-IDOs, IDOs and the SLOs. The multi-center and multi-crop

complementary projects have fostered a strong willingness and commitment among scientists to

work together towards greater research integration for program effectiveness. RTB must now

capitalize on this progress and momentum as it moves forward.

At the same time, the Evaluation Team has identified a number of areas where RTB could improve its

effectiveness. These include the need for improved understanding of both the capabilities of NARS

for breeding RTB crops and of end-user needs for specific products to better direct the focus – hence

effectiveness - of RTB breeding efforts. Furthermore, the team has also recommended a number of

ways in which RTB’s breeders could work more effectively together through a community of

practice.

RTB has also laid a solid foundation for future research on developing systems for quality seed,

making notable progress both in improving access and supply of planting material to seed producers

and small holders. The effectiveness of this important research could be further improved by

recruiting an RTB seed systems expert who would lead priority research on assessing demand for

clean high quality planting material; on understanding the incentives for small holders to purchase

quality planting material; as well as mechanisms for strengthening the supply chain with links to

marketing and processing.

Some important aspects of crop management research have not been well-supported which

compromises the ability to integrate crop improvement and management technologies to effectively

xiii


cgiar.iea.org

realize improved yields on-farm. In future, RTB needs to consider rebalancing the RTB portfolio

towards agronomic and soil fertility research to address this gap.

Moreover, in future RTB should focus post-harvest research on the crop-specific aspects of value

chain improvements that can deliver added value, as these are most likely to generate global public

goods. Lessons from the Theme 6 research on cassava should help to identify transferable strategies

and options for other RTB crops. Collaboration with the Agriculture for nutrition and health CRP

(A4NH) on nutrition research should also be strengthened.

Gender, capacity development, partnerships and knowledge management and

communication

RTB has made substantial progress on increasing the gender responsiveness of research activities.

The W1/W2 funding allocated for the development of the Gender strategy, capacity development on

gender, and support for the integration of gender analysis in project design and implementation has

been essential to achieve these gains. The recent RTB focus on gender capacity strengthening has

delivered impressive results over a relatively short period. Much of the gender work has been in the

form of case studies, and while some attempts were made to consolidate findings, the work is

somewhat fragmented. It is anticipated that the collaborative analysis of the case studies in the

GENNOVATE project will contribute significantly to strengthening capacity for rigorous consolidation

of case study research, to continue to build the evidence base for the use of gender analysis and

gender responsive project planning. It will be important to ensure that gender and social equity

considerations continue to feature prominently in RTB research activities in future.

While RTB has made strong contributions to capacity development, particularly with regard to

enhancing individuals’ research capacity, training end-users, and implementing innovative learning

strategies, the efforts remain largely project based, and heavily reliant on W3/bilateral funding.

RTB’s accountabilities for capacity development outcomes are not yet clearly defined. The initiatives

need to be coordinated with others, including regional initiatives aimed at addressing high-level

human resource constraints in agriculture. RTB could consider using a rapid assessment approach to

identify and prioritize specific focus areas for capacity development in future.

Partnership is a major strength of RTB. It has capitalized on established center-based relationships

with many institutions and broadened them where opportunity existed. Engaging partners in RTB

research planning and implementation has contributed to stronger bonds and commitment in

research. Similarly, partnership platforms and continent-wide projects have facilitated collaboration

with all types of stakeholders. Partnerships with private tissue culture facilities in SSA have proven to

be important for servicing the production of clean planting material for RTB projects. RTB

partnerships with other CRPs vary from excellent to a work in progress but a solid foundation has

been laid for enhanced collaboration in future.

RTB recognizes that a comprehensive and adequately resourced knowledge management and

communication strategy is essential to support the shift in focus towards achieving greater impact

through research and the proposed future focus on the use of ICT-smart communication strategies,

xiv


cgiar.iea.org

and scaling RTB’s knowledge management experiences through advancing networks, portals and

other innovative methods is appropriate. However, it will be important to allocate adequate

resources to develop and implement the needed strategy for communication and knowledge

management.

Impact and sustainability

Impact studies on RTB crops show clear evidence of global impact in cassava and potato, and to a

lesser extent in sweetpotato. Cassava modern varieties (MV) appear to have had major impacts in

SSA and in East Asia. New RTB research efforts are focusing on whether these impacts can be carried

forward to more marginal production environments in SSA and the importance of processing and

taste characteristics as constraints to adoption. Potato and sweetpotato programs have had

significant global impact and positive economic returns as well, although the extent of adoption is

more limited. Some of the most notable documented impacts are in China for both of these crops.

Impact studies of orange-fleshed sweetpotato (OFSP) in SSA show preliminary promise for

nutritional benefits, and these studies may provide lessons and models for biofortification efforts in

cassava and banana.

There has been limited documented impact in bananas and yams, although IITA studies show the

potential for increased productivity. Strategically directed studies of barriers to adoption for these

crops are needed, which could then inform research planning for greater future impact. Impact

assessment work is also uneven in both approach and quality across centers and regions.

Governance and management

RTB has made significant progress in establishing G&M structures and processes that create new

ways of working to promote the inter-dependence of the members of the RTB alliance. The creation

of the ISC has improved governance and works well with the Program Management Unit. However,

currently, there is no overarching business framework that can help guide these relationships

beyond common goals and mutual trust. There is therefore a need for RTB centers to develop and

agree on an alliance compact that would bring clarity and greater understanding to critical

partnership questions including the allocation of W1/W2 funds, handling of W3/bilateral projects,

participation in RTB G&M, alignment of management processes, and handling of joint undertakings

and appointments. The RTB leadership is well-appreciated by stakeholders and considered to be

amongst the best across CRPs. However, RTB needs to bring clarity to the respective roles,

relationships and accountabilities of FP leaders, CoA leaders and bilateral project leaders within the

management structures of RTB and the centers.

Added value

RTB is successfully delivering added value across centers and crops in several areas, with

collaborations and new science that did not exist prior to 2012. In this process it is capturing

synergies and taking advantage of complementarities through novel research on problems common

to crops and/or centers. Many examples are given in the report’s Chapters on relevance, quality of

xv


cgiar.iea.org

science and effectiveness. Furthermore, there was general agreement among RTB researchers that

the program is adding value by creating or enhancing synergies between the centers, becoming

strategically better focused on development outcomes and improving the way that gender issues are

integrated into research. However, there are still areas where synergies and complementarities from

centers working more closely together especially on the same crop will add more value to RTB

research. In particular, further value would be gained by fully integrating the IITA and CIAT cassava

breeding programs and integrating and consolidating all RTB research on banana and plantain by

IITA and Bioversity.

Going forward

The Evaluation Team’s overall assessment is that RTB plays a unique and much needed role in

research generating international public goods (IPG) on important root and tuber crops as well as

banana and plantain grown by small holders and consumed by the poor. Its potential to contribute

to food security and nutrition as well as enhancing the role of women in RTB crop value chains has

been established.

RTB has achieved a notable level of programmatic integration and captured synergies and taken

advantage of complementarities which would not have been possible before the creation of RTB, in

spite of budget cuts. However, a program-wide philosophy and commitment to programmatic

integration is still not fully realized. Center-level thinking still predominates largely due to the fact

that decisions on a considerable amount of RTB funds are made by the centers based on RTB

priorities, centers have a stronger brand than RTB and researchers identify primarily with centers.

Further effort is needed to enhance integration beyond individual time-bound projects.

These are several areas where improvements are needed for RTB to add value as a program and

contribute more effectively to its objectives. Several of these are addressed in the recommendations

below. It is important that RTB develops its research agenda and project portfolio by using and

continuously building on its recent PA, allocating W1/W2 funds more strategically and managing

different funding sources as one portfolio for enhancing strategic thinking. Program level

partnerships can be strengthened; partnership with A4NH on nutrition issues and with the private

sector particularly on seed systems and processing are but a few examples. Where national partners

are strong and RTB has limited further opportunity for generating substantial IPGs, a strategy for

handing over is needed to ensure long term local sustainability. This will enable RTB to concentrate

of other important priorities. RTB needs to be pro-active in enhancing its visibility as a unique CRP

amongst policy makers but also donors. This is important for expanding the currently narrow donor

base and thus enhancing funding sustainability.

A strengthened alliance between center partners will enhance RTB’s ability to contribute to its IDOs

and ultimately to the CGIAR’s goals for reduced poverty, improved food and nutritional security and

improved natural resource systems and ecosystem services.

xvi


cgiar.iea.org

Recommendations

The Evaluation Team makes a total of 16 recommendations presented below by the main evaluation criteria.

Relevance

Recommendation 1: RTB needs to make further efforts to enhance integration beyond individual

time-bound projects. In particular, further value would be gained by fully integrating the IITA and

CIAT cassava breeding programs. There is also clear potential for integration and consolidation of all

RTB research on banana and plantain by IITA and Bioversity. This would likely result in rationalization

of staff positions, allow better targeting of scarce W1/W2 funds and improve the ability to approach

donors as an integrated program.

Recommendation 2: The restructuring of the program into inter-disciplinary and integrated FPs adds

coherence to RTB compared to the Theme-based structure. However in some cases the definition of

clusters of activities lacks coherence and consistency with the FP problem definition e.g. the banana

disease clusters, single yam cluster and complex clusters in FP4. RTB should review and revise the

clusters for improved congruence with the FP problem definition.

Recommendation 3: During 2012-2015, with the exception of complementary and cross-cutting

funds, RTB allocated W1/W2 funds to partner centers based on historical funding. RTB has now

recognized the need to move towards more strategic allocation of these funds based on program

priorities and performance. In the current environment of decreasing W1/W2 funds, RTB should

ensure that W1/W2 funds are directed at the highest program priorities as informed by priority

assessment and performance evaluation.

Recommendation 4: RTB should use priority assessment results for setting program priorities and in program planning, including fundraising. In doing so, RTB should also plan for continuous improvements in the data and estimates that support priority assessment.

Quality of Science

Recommendation 5: RTB has published some excellent research papers in appropriate journals with

high impact. At the same time, the percentage of publications in non-IF journals is disturbingly high –

39 percent. While it is recognized that most RTB crops are at a disadvantage in terms of the range of

journals willing to publish on these crops and the need to target African journals to promote

research findings to the most appropriate stakeholders, RTB should endeavour to assure that its

science quality is consistently high in order to target and publish in higher quality journals for greater

international impact.

Recommendation 6: Although individual Centers are responsible for the performance of their

scientists, RTB is responsible for the quality of science implemented and generated by the program.

The MC in consultation with the FP leaders should play a more active role in monitoring the quality

xvii


cgiar.iea.org

of science produced by RTB with oversight by the ISC. Reviving Commissioned External Evaluations

at CRP-level would be of greatest value.

Recommendation 7: RTB needs to further modernize and strengthen its breeding programs. Within

the current funding climate, highest priority should be given to:

a) Adoption of the best breeding strategies for its crops that involve harmonizing breeding approaches within crops and transferring lessons across crops, where possible;

b) A benchmark study of its utilization of genomic technologies with the most adopted ones by the private sector to identify opportunities for improvement in the deployment of techniques such as gene editing and MAB;

c) Deployment of precise high-throughput phenotyping methods, novel breeding techniques and modelling for traits such as drought and temperature stress through engagement with best practice in ARIs;

d) Attracting young scientists working in genomics-led breeding, bioinformatics or omics research for both accelerating breeding and increasing genetic gains; and

e) Placing more emphasis on training the next generation of plant breeders so that breeding will continue after the termination of the short term bilateral project funding.

Effectiveness

Recommendation 8: RTB should better target client needs by delivering only two to three achievable

product profiles for each mandate crop per country or region and placing even greater emphasis on

farmer and consumer needs. RTB and NARS should decide together on the division of labour based

on NARS capability in each target country. This will allow RTB to provide appropriate back-stopping

to NARS in further development of the products into cultivar(s) for release to farmers.

Recommendation 9: RTB should develop a community of practice of researchers across all crop

breeding undertakings for enhancing effectiveness through better synergy. It will allow the sharing

of ideas on methods, data, results and user feed-back, thus leading to integrated data platforms,

developing inter-center working groups on traits, enhancing the inter-disciplinarity between lab-

genomics and field-breeding, establishing single RTB breeding programs for banana and cassava, and

sharing experiences among those engaged in transgenic breeding.

Recommendation 10: The establishment of economically sustainable seed systems for RTB crops is

of core importance for program effectiveness. Priority should be given to assessing demand for clean

high quality planting material throughout the seed value chain; on understanding the incentives for

small holders to purchase quality planting material; as well as mechanisms for strengthening the

supply chain with links to marketing and processing. Due to the importance of seed systems research

for impact, as RTB moves into scaling-up and scaling-out seed systems activities, it should recruit an

expert in RTB seed systems rather than relying on short term inputs from consultants and partners

as was noted in Chapter 4.

Recommendation 11: Some aspects of crop management research, for example agronomy and soil

fertility research in Theme 5, have not been well-supported. Narrowing the yield gap for farmers

xviii


cgiar.iea.org

may require rebalancing the RTB portfolio towards agronomic and soil fertility research. In order to

improve the realized yields in farmer’s fields, RTB needs to better integrate research on crop

improvement and crop management which have been implemented in different Themes to date and

will be implemented in different FPs in the new program structure.

Recommendation 12: RTB should focus post-harvest research on the crop-specific aspects of value chain improvements that can deliver added value, as these are most likely to generate global public goods. Assessing lessons from the emerging cassava Theme 6 research results should help to identify transferable lessons and strategies for other RTB crops, providing a basis for scalability of lessons learned.

Gender, capacity development and partnerships

Recommendation 13: It is recommended that RTB management ensures that adequate resources are made available to develop and implement the needed strategy for communication and knowledge management. Flagship and cluster leaders as well as bilateral project leaders will need access to communication and knowledge management expertise, and be enabled to incorporate knowledge management ‘experiments’ into the design of new projects to achieve the ambitious intentions laid out in the pre-proposal. A strategy similar to the one proposed on capacity development (and drawing on lessons from the approach adopted to address gender issues) is recommended.

Impact and sustainability

Recommendation 14: Impact assessment is strategically important for demonstrating impact, justifying resources, and informing program planning. RTB needs a clear strategy of how priority and impact assessments will be linked over time, and how the results from ex-post assessments, complementing ex-ante assessment, will inform program planning. This may lead to changes in the design of ex-post assessments. In formulating an impact assessment plan, RTB should scale up activities and apply lessons from the SIAC projects they are currently engaged in. It should also ensure comparable quality of efforts across crops and regions.

Governance and management

Recommendation 15: RTB should bring clarity to the respective roles, relationships and

accountabilities of FP leaders, cluster leaders and bilateral project leaders within the management

structures of RTB and the centers.

Recommendation 16: RTB partners should develop and agree on an alliance compact building on the progress already made in inter-center collaboration. Such an alliance would bring clarity and greater understanding to critical partnership questions such as: allocation and use of W1/W2 funds, handling of W3/bilateral projects, participation in RTB governance and management, alignment of management processes, handling of joint appointments, handling joint undertakings and codes of conduct in program participation.

1

RTB Management Response and Action Plan to the CGIAR-IEA (2015), Evaluation of CGIAR Research Program on Roots, Tubers and Bananas (RTB). Rome, Italy: Independent Evaluation Arrangement (IEA) of the CGIAR

(Version 24 January 2016)

1) Overall response

We appreciate that this is a very well-crafted and thorough review, and we would like to thank the team and IEA. Overall we think it’s a fair and balanced evaluation. It is very constructive in the way it is written, carefully crafted and looking to support the development of the RTB 2nd phase. The IEA evaluation team did a terrific job by visiting a wide selection of sites, analyzing a large array of supporting documentation, keeping close interaction with the RTB team and handling the process in a coherent and inclusive way.

RTB management notes with satisfaction the team’s conclusions, in particular:

Notable progress in past 4 years

Strongly warrants continuing

Well directed, and achieving reasonable number of milestones

Adding value across crops and centers mainly through complementary funded projects

Science sound

NARs appreciative

Good progress gender strategy

Good program governance and management

There were some areas where we did not fully agree with the report.

RTB breeding programs have made good progress in introducing modern breeding approaches and this has been a central thrust of complementary funding. Recommendation #7 is somewhat misleading in this regard, and does not capture the good progress already made and reflected in the in the Executive summary assessment of breeding programs as “using both sound and novel approaches but should continue to improve”.

Window1&2 funding is passing through a rather dramatic downturn at the moment, hopefully this situation will change from 2017 as there is a huge value in having a more integrated CRP which this makes possible. At the current decreasing levels of overall W1&2 funding it will be

ocussen

Typewritten Text

ocussen

Typewritten Text

ocussen

Typewritten Text

CB24-11 Annex 3

ocussen

Typewritten Text

ocussen

Typewritten Text

ocussen

Typewritten Text

ocussen

Typewritten Text

2

challenging to implement several recommendations including Recommendation 1 for further integration of breeding programs given increased dependence on bilaterally funded projects.

2) Responses by recommendation

Evaluation Recommendation Management Response to the

Recommendation

Management Follow up

Action to be taken

Who Responsible for Action

Timeframe

Is additional funding required to

implement recommendation?

RELEVANCE

Recommendation 1 (integrated breeding programs):

RTB needs to make further efforts to enhance integration beyond individual time-bound projects.

In particular, further value would be gained by fully integrating the IITA and CIAT cassava breeding programs.

There is also clear potential for integration and consolidation of all RTB research on banana and plantain by IITA and Bioversity.

This would likely result in rationalization of staff positions, allow better targeting of scarce W1/W2 funds and improve the ability to approach donors as an integrated program.

Partially accepted

Agree with the general point that RTB needs to enhance integration beyond individual projects and this will occur with design and implementation of the new RTB Program structure (based on clusters), rationalization of use of W1&2 funds and also with a CGIAR genetic gains platform currently under development which will link all Agri-Food Systems CRPs including RTB.

Cassava program: Cassava breeders in IITA and CIAT note that “fully integrating” cassava breeding comes with some caveats. Breeding program objectives are different in the three important cassava growing regions, and LAC cassava material is very sensitive to viruses prevalent in Africa and quarantine issues prevent transfer of material. Hence full integration is desirable, but within the context of recognized regionally specific objectives

1.1 Joint leadership of varietal development clusters of cassava and banana, as well as cross-cutting clusters, under FP1 (Discovery Flagship)

1.2 Evaluation of progress in cross center clusters for varietal development in cassava and bananas and FP 1 clusters

1.3 Active and effective breeding platform cluster

1.1 IITA, CIAT, Bioversity (and PMU)

1.2 PMU, Flagship and cluster leaders in reporting

1.3 PMU, Flagship and cluster leaders

Beginning 01/16

Yes – especially for

integrating mechanisms as: Frequent exchange visits between involved centers (including CIRAD)

shared training among regions

3


Recommendation


Action to be taken


Timeframe



and constraints.

Banana program: More interaction between the IITA breeders with the Bioversity scientists, within the framework of RTB, will ensure that breeder and end user preferred traits are characterized and integrated into breeding programs.

Recommendation 2 (coherence/ consistencies of clusters):

The restructuring of the program into inter-disciplinary and integrated FPs adds coherence to RTB compared to the Theme-based structure.

However in some cases the definition of clusters of activities lacks coherence and consistency with the FP problem definition e.g. the banana disease clusters, single yam cluster and complex clusters in FP4.

RTB should review and revise the clusters for improved congruence with the FP problem definition

Fully accepted

We fully agree that the clusters need to be revised for congruence with FP problem definition.

In the light of reduction in W1&2 and donor requirements for structuring CRP Phase II, there is also a need to reduce the number of clusters. We have already reduced from 32 to 25 clusters, based on five Flagships, including a reduction from three to two banana disease clusters.

Given this general reduction in numbers of flagships and clusters (coherence in aggregation levels, funding size, etc.) we feel that a single yam cluster is appropriate. In comparison, there are only two clusters each for the geographically much more widely cultivated crops of potato and sweetpotato.

2.1 Prepare consolidated content for 2 banana disease clusters

2.2 Improve coherence of clusters in FP4 during preparation of full proposal

2.3 Continue congruence analysis and fit of resources to FP problem definition and numbers of beneficiaries and size of benefits

2.1 PMU, Bioversity, IITA and FP/cluster leaders

2.2 PMU, Priority Assessment Focal points in each Center

2.3 PMU, Flagship and cluster leaders

2.1 03/2016

2.2 03/2016

2.3 12/2016

No

4


Recommendation


Action to be taken


Timeframe



Recommendation 3 (fund allocation):

During 2012-2015, with the exception of complementary and cross-cutting funds, RTB allocated W1/W2 funds to partner centers based on historical funding.

RTB has now recognized the need to move towards more strategic allocation of these funds based on program priorities and performance.

In the current environment of decreasing W1/W2 funds, RTB should ensure that W1/W2 funds are directed at the highest program priorities as informed by priority assessment and performance evaluation.

Fully accepted

The concept of maturing the fund allocation methodology in support of the evolution of the overall system and RTB objectives makes sense. In our view, refining the methodology should consider diverse elements such as: lessons learned, impact potential, programmatic performance (e.g. against indicators), financial performance (e.g. expenditure or absorption rates), as well as set aside needs, mechanisms to apportion funds and risks to build a sustainable program and attain an effective return on investment.

We agree that priority assessment can give some insights across crops and technologies but it is one element of several and would not e.g. address how much to assign to cross cutting activities or to gender research or CapDev.

Also, should recognize that diminished W1&2 reduces ability to direct investment and abrupt shifts in funding may be counter to the notion of RTB as an “alliance”.

3.1 Finalize congruence analysis from priority assessment and take this into consideration in setting budgets by cluster/flagship in full proposal

3.2 Implement more systematic performance evaluation of base and complementary funding and at cluster level as pilot in 2016 and rom 2017

3.3 Investment flow through projectized investment at the cluster level.

3.1 PMU, Priority Assessment Focal points in each Center

3.2 PMU, FP/cluster leaders

3.3 PMU

3.1 in process

3.2 & 3.3 02/2016

No

Recommendation 4 (Priority Assessment):

RTB should use priority assessment

Partially accepted

The assumptions behind the priority assessment results need to be cross

4.1 Upgrade full set of priority assessment data every three years

4.1 & 4.2 PMU, Priority Assessment/ Impact

4.1 2018

4.2 2021

Yes,

priority assessment and ex post impact

5


Recommendation


Action to be taken


Timeframe



results for setting program priorities and in program planning, including fundraising.

In doing so, RTB should also plan for continuous improvements in the data and estimates that support priority assessment.

checked in light of more recent evidence and unless sound and upto-date assumptions are used for ex-ante estimates, investment decisions could be misleading.

Also need to be aware of cost implications of continuous improvements under very limited W1&2 funding. Need also to invest scarce resources in ex-post impact studies.

Continuous improvement of data and estimates: Improved information should be available through M&E. But there are cost implications of continuous data improvements. More reasonable to upgrade every three years and major priority assessment every 6 years.

Using priority assessment results for fundraising has its limitations as donors in majority have their own mechanisms for setting priorities.

4.2 Major priority assessment every 6 years

Assessment focal points

assessment are costly activities, and require substantial time input both from the social scientist who lead this and the biological scientists who provide key information.

Continuous improvement in data and estimates will require additional funding.

QUALITY OF SCIENCE

Recommendation 5 (publications):

RTB has published some excellent research papers in appropriate journals with high impact.

At the same time, the percentage of publications in non-IF journals is

Fully accepted

As the reviewers mention, there are good reasons for where RTB science is published. With emphasis on delivery, this kind of work does not usually go to high impact journals. Giving emphasis to discovery research in Phase II will lead to

5.1 Insist on (clearer) attribution/acknowledgement/branding and tagging of key words of RTB in publications.

5.1 PMU, MC/ DDGs Research of each Center, communication Center Focal points

Yes,

if resources are used to cover open access costs in high impact journals.

6


Recommendation


Action to be taken


Timeframe



disturbingly high – 39 percent.

While it is recognized that most RTB crops are at a disadvantage in terms of the range of journals willing to publish on these crops and the need to target African journals to promote research findings to the most appropriate stakeholders, RTB should endeavor to assure that its science quality is consistently high in order to target and publish in higher quality journals for greater international impact.

high-impact publications, but with a longer time-frame.

One topic for analysis at the CGIAR-wide level is the trade off between publication in the higher impact journals and opting for open access journals which may have lower impact factors.

5.2 Create incentives for more publications through reporting and performance evaluation.

5.2 PMU, MC/DDGs Research of each Center

Recommendation 6 (monitoring quality of science):

Although individual Centers are responsible for the performance of their scientists, RTB is responsible for the quality of science implemented and generated by the program.

The MC in consultation with the FP leaders should play a more active role in monitoring the quality of science produced by RTB with oversight by the ISC.

Reviving Commissioned External Evaluations at CRP-level would be of greatest value.

Fully accepted

This is a good suggestion. ToRs of both flagship project and cluster leaders will include this aspect.

We also agree on the value of CRP commissioned External Evaluations; however, the frequency and intensity of these evaluations should be related to the level of available W1&2 funding, with perhaps an evaluation of one flagship per year under higher levels of funding or in alternate years under a lower funding scenario. It is not only the direct cost of implementation but also the cost of the scientists’ time involved which now has to be fully budgeted for.

6.1 CRP commissioned evaluations will be included in the Evaluation plan, developed for the 2nd Phase.

6.1 PMU 01/2017 Yes for CRP commissioned evaluations

7


Recommendation


Action to be taken


Timeframe



Recommendation 7 (strengthening breeding programs):

RTB needs to further modernize and strengthen its breeding programs. Within the current funding climate, highest priority should be given to:

a) Adoption of the best breeding strategies for its crops that involve harmonizing breeding approaches within crops and transferring lessons across crops, where possible;

b) A benchmark study of its utilization of genomic technologies with the most adopted ones by the private sector to identify opportunities for improvement in the deployment of techniques such as gene editing and MAB;

c) Deployment of precise high-throughput phenotypin5g methods, novel breeding techniques and modelling for traits such as drought and temperature stress through engagement with best practice in ARIs;

d) Attracting young scientists working in genomics-led breeding, bioinformatics or omics research for both accelerating breeding and increasing genetic gains; and

Partially accepted

We agree with the idea of strengthening the breeding programs. However this recommendation understates good progress already made and the continuous aim to use up-to-date breeding methods.

The new genetic gain platform under discussion should help to address all of these points.

However, also need to recognize that strengthening breeding programs requires funding and infrastructure to improve the level of excellence. A complementary option is to intensify partnership with private sector and ARIs and with other organizations to increase access to the latest advances in breeding. techniques and infrastructure

The training of next generation of plant breeders is critical. However, this may be best achieved by partnering and leveraging other institutions with capabilities and resources for training the next generation of breeders that can complement our limited W1&2 funding.

7.1 As part of the redesign of RTB structure a breeders’ community of practice is being set up.

7.2 RTB to strongly engage in design of proposal of new genetic gains platform and ensure articulation with RTB FP1 and the breeding community of practice.

7.3 Identify partnering opportunities for training next generation of breeders

7.1 PMU, FP and cluster leaders

7.2 PMU and FP1 Flagship and cluster leaders

7.3 Flagship 1 and 2 leaders and their respective clusters with DDGs Research and PMU

7.1 in process

7.2 03/2016

7.3 12/2017

Yes, especially on points:

c) Requires funding and infrastructure.. Collaborations work better for this, but need to consider long term sustainability (even if train local young scientists, they need support back in their country NARS to continue this kind of work). Perhaps a moment to acknowledge cross AFS synergies regarding a the Genetic Gains Initiative (to not call it a CRP or Platform) This could get at the comment below regarding limited funds

d) Same comment above.

Questions around resourcing this on

8


Recommendation


Action to be taken


Timeframe



e) Placing more emphasis on training the next generation of plant breeders so that breeding will continue after the termination of the short term bilateral project funding.

current budgets – training best achieved by leveraging others with deeper pockets and not from our limited W1&2.

Recommendation 8 (NARS/consumer needs):

RTB should better target client needs by delivering only two to three achievable product profiles for each mandate crop to NARS and placing even greater emphasis on farmer and consumer needs.

RTB and NARS should decide together on the division of labor based on NARS capability in each target country.

This will allow RTB to provide appropriate back-stopping to NARS in further development of the products into cultivar(s) for release to farmers.

Accepted

This is consistent with our approach to have better variety and technology delivery pipelines responding to user demand.

8.1 Revisit clusters to assure proper integration and visibility of consumer needs and feedback loops between FP1/FP2 and clusters of other FPs.

8.2 Strategic review of NARS breeding capability for key crops and targeted countries potentially linked with genetic gains platform

8.1 & 8.2 PMU, FP and cluster leaders

8.1 in process

8.2 12/2017

Yes,

for strategic review of NARS breeding capability

EFFECTIVENESS

Recommendation 9 (CoP Breeding):

RTB should develop a community of practice of researchers across all crop breeding undertakings for enhancing effectiveness through better synergy.

Partially accepted

Fully agree on the community of practice as noted under recommendation #1, we have reservation about a single RTB breeding program for cassava.

9.1 As part of the redesign of RTB structure a breeders community of practice is being set up

9.1 PMU, FP1 and FP leaders and cluster leaders

In process Yes, for joint workshops, face to face meetings, etc.

9


Recommendation


Action to be taken


Timeframe



It will allow the sharing of ideas on methods, data, results and user feed-back, thus leading to integrated data platforms, developing inter-center working groups on traits, enhancing the inter-disciplinarity between lab-genomics and field-breeding, establishing single RTB breeding programs for banana and cassava, and sharing experiences among those engaged in transgenic breeding.

All RTB center geneticists and breeders are involved in the planning of the breeding activities for 2016 and Phase II, thus promoting collaboration and a shared mission, enhancing inter-disciplinarity.

Recommendation 10 (seed system):

The establishment of economically sustainable seed systems for RTB crops is of core importance for program effectiveness.

Priority should be given to assessing demand for clean high quality planting material throughout the seed value chain; on understanding the incentives for small holders to purchase quality planting material; as well as mechanisms for strengthening the supply chain with links to marketing and processing.

Due to the importance of seed systems research for impact, as RTB moves into scaling-up and scaling-out seed systems activities, it should recruit an expert in RTB seed systems rather than relying on

Partially accepted

We agree on the importance of seed work in RTB because they are all clonally propagated – and have therefore set up FP2.

Seed system experts should be contracted via the relevant centers and not form part of PMU. Given the volatile and limited nature of W1&2 funds this should be encouraged via the relevant bilateral or W3 project opportunities.

10.1 Continue supporting work on assessing demand for quality planting materials and market driven approaches

10.2 Enable cross crop/cross Partner collaboration on variety release and dissemination for clonally propagated crops.

10.1 FP2 leader and relevant cluster leaders

10.2 PMU with FP1, FP2, FP5 Leaders and relevant cluster leaders

2017-2019 Yes

For 10.2 Yes for enabling collaboration, establishing guidelines and gathering best practices.

10


Recommendation


Action to be taken


Timeframe



short term inputs from consultants and partners as was noted in Chapter 4.

Recommendation 11 (agronomy/crop management):

Some aspects of crop management research, for example agronomy and soil fertility research in Theme 5, have not been well-supported.

Narrowing the yield gap for farmers may require rebalancing the RTB portfolio towards agronomic and soil fertility research.

In order to improve the realized yields in farmer’s fields, RTB needs to better integrate research on crop improvement and crop management which have been implemented in different Themes to date and will be implemented in different FPs in the new program structure.

Partially accepted

We agree that better integration and visibility in the cluster structure is needed.

Agronomic aspects are mainly focused in FP3 but due to the nature/logic of the cluster set up, integrated in different clusters also under other FPs.

Yield gap analysis shows a strong role for agronomy and soil fertility in improving yields. However, until recently this area of work in RTB crops often appears as more location specific and has not really demonstrated IPGs. And impact assessments with RTB have shown less clear evidence of impacts of agronomic/crop management work.

However, this is an important area and inclusion of more systems work with implementation as Agri-food System CRP, RTB should strengthen human resources and bring novel approaches to bear.

11.1 Revisit FP 3, its clusters and all other relevant clusters to assure proper integration and visibility of agronomic aspects and feedback loops with FP3.


In process Yes

Staffing in all Centers in crop management/improvement and agronomic aspects is quite weak and requires additional funding for new employments

Recommendation 12 (post-harvest research):

RTB should focus post-harvest research on the crop-specific aspects of value chain improvements that can deliver

Fully accepted

However, should note that cassava has by some distance the greatest potential currently for local value added from processing driven in part by rapid

12.1 Revisit FP 4, its clusters and all other relevant clusters to assure proper integration and visibility of


In process No

11


Recommendation


Action to be taken


Timeframe



added value, as these are most likely to generate global public goods.

Assessing lessons from the emerging cassava Theme 6 research results should help to identify transferable lessons and strategies for other RTB crops, providing a basis for scalability of lessons learned.

postharvest physiological deterioration and also by local food preferences. This explains why it has a cluster for post-harvest innovation. All crops have some potential.

Some value chain aspects are crop related and are not easily transferable as such.

post-harvest lessons learnt into other clusters.

GENDER, CAPACITY DEVELOPMENT AND PARTNERSHIPS

Recommendation 13 (communication/ knowledge management):

It is recommended that RTB management ensures that adequate resources are made available to develop and implement the needed strategy for communication and knowledge management.

Flagship and cluster leaders as well as bilateral project leaders will need access to communication and knowledge management expertise, and be enabled to incorporate knowledge management ‘experiments’ into the design of new projects to achieve the ambitious intentions laid out in the pre-proposal.

Fully accepted

We agree with the need to include communication and knowledge management aspects in the research program. However, given shrinking share of W1&2 more of this should be embedded in bilateral and W3 projects whilst providing a critical support and oversight function at the overall program level. The M&E capability of RTB will help to underpin the design and utility of both Recommendations 13 and 14 in an integrated way.

13.1 Communication (strategy/action plan) will be integral part of the 2nd Phase proposal.

PMU, Center communication and KM focal points

01/2017 Yes

Staffing in all Centers in communication/KM aspects – related to CRPs - is quite weak and requires additional funding for new employments

12


Recommendation


Action to be taken


Timeframe



A strategy similar to the one proposed on capacity development (and drawing on lessons from the approach adopted to address gender issues) is recommended.

IMPACT AND SUSTAINABILITY

Recommendation 14 (impact assessment):

Impact assessment is strategically important for demonstrating impact, justifying resources, and informing program planning.

RTB needs a clear strategy of how priority and impact assessments will be linked over time, and how the results from ex-post assessments, complementing ex-ante assessment, will inform program planning. This may lead to changes in the design of ex-post assessments.

In formulating an impact assessment plan, RTB should scale up activities and apply lessons from the SIAC projects they are currently engaged in. It should also ensure comparable quality of efforts across crops and regions.

Fully accepted

However see point on continuous updating of data for priority assessment (recommendation #4).

14.1 Finalize ongoing Impact Assessment

14.2 CRP commissioned evaluations; Impact Assessment will be included in the RBM set-up and Evaluation plan, developed for the 2nd Phase.

14.1 PMU & Impact Assessment Center focal points

14.2 PMU, MC

14.1 During 2016

14.2 01/2017

Yes

GOVERNANCE AND MANAGEMENT

Recommendation 15 (leadership Fully accepted Reformulation of the Terms PMU, MC and In process No

13


Recommendation


Action to be taken


Timeframe



roles/responsibilities):

RTB should bring clarity to the respective roles, relationships and accountabilities of FP leaders, cluster leaders and bilateral project leaders within the management structures of RTB and the centers.

The continuous improvement of leadership roles and responsibilities is seen as a natural progression in the evolution of the program. RTB intends to maximize its management effectiveness based on its institutional memory and lessons learned.

ToR of FP and cluster leaders, and work plans of individual scientists within centers contributing to RTB should be streamlined to clarify the contribution to RTB deliverables, and staff members should be assessed in terms of fulfillment of the commitments

of Reference for all these positions as we move to flagship based program structure

ISC

Recommendation 16 (alliance compact):

RTB partners should develop and agree on an alliance compact building on the progress already made in inter-center collaboration.

Such an alliance would bring clarity and greater understanding to critical partnership questions such as: allocation and use of W1/W2 funds, handling of W3/bilateral projects, participation in RTB governance and management, alignment of management processes, handling of joint appointments, handling joint undertakings and codes of conduct in

Partially accepted

We appreciate the acknowledgement that "RTB has made significant progress in establishing G&M structures and processes that create new ways of working to promote the inter-dependence of the members of the RTB alliance. The creation of the ISC has improved governance and works well with the Program Management Unit.”

Although the concept of an alliance compact is not fully developed in the evaluation, we concur with the basic concept that a soft contractual vehicle, among centers, to better define basic

16. 1 Identify potential governance, operational and business relationships which could further enhance the alignment of partner objectives for strategic goals and could be included in a soft contractual vehicle such as an alliance compact or similar alternative model.

16.2. Building on analysis under 16.1, agree Strategy Statements which would constitute “soft contracts”

16.1 ISC and PMU/MC followed by CIP BoT

16.2. ISC and PMU/MC followed by CIP BoT

16.1 from 06/2016 once the new Systems Office is in place

16.2. until 11/2016

14


Recommendation


Action to be taken


Timeframe



program participation. rules of engagement, could strengthen the partnership.

It should be noted that some of the topics indicated for this 'alliance compact' are currently under examination by the Transition Team (TT) working to establish the System Council, Systems Office and associated functions (notably the finance working group of the TT on aspects such as the allocation of W1/W2 funds, handling of W3 and bilateral projects, participation in CRP G&M, alignment of management processes…). The outcome of the TT work should be considered and be aligned with new mechanisms such as an ‘alliance compact’.

to be achieved through Compact or other mechanisms for improving Governance and Management and agree a timeline for implementation