PowerPoint Presentation

Monitoring stakeholder engagement The experience of developing the M&E strategy for PRISE

Simon Hearn, Research Fellow, ODI

2

What is PRISE?

Pathways to Resilience in Semi-Arid Economies

Research consortium led by ODI, primarily CEP and Water but other programmes also involved. Working with:LSE Grantham Research InstituteIED Afrique SenegalCCCS, University of Dar es Salaam TanzaniaSDPI Pakistan

5 years, funded by DFID, managed by IDRC as part of a larger programme on climate adaptation involving three other consortia: CARIAA.

This research will support the emergence of equitable, climate resilient economic development in semi-arid lands through research excellence and sustained engagement with business leaders, local and national government decision-makers, civil society, and regional economic communities.

Develop an evidence base on the risks posed to economic growth in semi-arid lands by extreme climate events, particularly droughts and floods;

Identify investment, policy and planning measures for inclusive climate resilient development and growth in semi-arid lands;

Leverage existing initiatives and networks in a stakeholder engagement process that co-creates knowledge, builds credibility with research users, and promotes the uptake of results. 3

Country focusEast AfricaTanzania KenyaWest AfricaSenegal Burkina FasoCentral / South AsiaPakistan Tajikistan

4

People benefitResilient economyStakeholders invest in and incentivise CRDStakeholders cooperate and coordinateStakeholders demonstrate understanding, capacity and evidence to act on CRDResearch and engagement

How to monitor and evaluate this?5

M&E strategy build up

6

Strategy and direction

Management

Outputs

Uptake and outcomes

Context are you doing the right thing? are you doing what you planned to do? are the outputs produced to standard and appropriate for the audience? are people aware of your work and what effect is it having? what is changing in your context that you should be aware of?Monitoring areas

7

Monitoring areaKey evaluation question1. Strategy and directionHow appropriate and relevant are PRISE strategies for meeting the goals of the consortium?2. Management and implementationHow well are internal systems working to implement the strategy (to time and budget)?3. Outputs and quality assuranceWhat has been the quality of outputs produced and communicated?4. Uptake and outcomesWhat outcomes have been produced and what contribution has PRISE made to them?5. ContextHow do the contexts within which PRISE is operating vary over time and space?

Evaluation questions

Mapped what data we would need to answer the questions and then what methods are available to collect, analyse and help interpret that data.8

Ongoing data collection: outputs, uptake and outcomesE.g. CARIAA output monitoring tool, event assessment, uptake log, media and social media monitoringPeriodic data collections: uptake and outcomesE.g. interviews, surveys, stories of change Data analysis and synthesisE.g. meetings, dashboards, reports, annual learning reviewsM&E processes_____

at country level at consortium level

PRISE M&E strategy relies on a combination of ongoing data collection and periodic data collection, at both country and consortium level.

These data will be analysed and synthesised in formal and informal ways for four different purposes:Management and decision makingLearningAccountabilityStrengthening stakeholder engagement

At country level this is managed by M&E focal points (one per region), although comms officers have responsibility for monitoring outputs and dissemination. At regional level this is managed by the M&E manager with support from the PRISE comms manager and PRISE Coordinator.9

Strategy and direction

Management

Outputs

Uptake and outcomes

Context happens intuitively at steering group level distributed within each partner organisation driven by donor requirements and systems emerging tools but need support difficult to focus until research areas are clearChallenges to model

10Challenges:It is difficult in any programme or organisations to marry M&E with strategic decision making. Each partner has their own contract with the donor which makes oversight and coordinated management systems difficultOutputs in the end what we measure is dictated by what the donor wants through the logframe but primarily through an online dashboard they set upThe methodological challenge is uptake and outcomes knowing where to look, how to record and how to know what is significantContext monitoring happens intuitively and specific measure are difficult until we know the specific research areas.

Have focussed on outputs so far. Getting the basics right.

Work this year on uptake and outcomes: - important to have a shared understanding of what we mean by outcome and where to look - using Outcome Mapping to develop rubrics - relies on observational data

11

Impact: Change in economy, Institutions,communities

Outputs:Research, comms and engagementOutcomes: Changes in behavior, relationships, policyDefining outcomes

13Given our understanding of the context, there are behaviours we wouldexpect to see.like to seelove to seeEarly positive responses to the researchActive engagement with the research resultsDeep transformation in behaviour

13

Attitudes Public opinionCapacityDiscourseProcedure/processContentBehaviour changeNetworks and systemsSource: Keck and Sikkink (1998) and Steven (2007) Defining outcomes

Policy change does not just mean change in legislation, budgets or programmes.

Attitudes of key stakeholders to get issues onto the agenda: How interested and open are policy actors to your issues? What kind of evidence will convince them?Public opinion: How are the public engaged in these issues?Capacity and engagement of other actors: Who else is engaging in this policy area? How influential are they? What can be done to involve others?Change in discourse among policy actors and commentators: What are the influential policy actors saying on this issue? What language are they using?Improvements in policy-making procedure/process: Who is consulted during policy-making? How is evidence taken into account? Change (or no change) in policy content: What new legislation, budgets, programmes or strategies are being developed?Behaviour change for effective implementation: Who is involved in implementing targeted policies? Do they have the skills, relationships, incentives to deliver?Networks and systems for supporting delivery: Are different actors working coherently together to implement policy? Are the necessary structures and incentives in place to facilitate this?14

What kind of behaviour would weExpect to see?Like to see?Love to see?Stakeholder 1Reads reports, asks for briefingTalks to a group theyve not engaged with beforeThey develop joint statements / approachesStakeholder 2Attends seminarA bit of a backlash: this is threatening to themDoesnt try to up-end the process

Defining outcomes

Think at the beginning who are you trying to influence, what do you want to see?

Develop mini theories of change for each stakeholder at country level.

This is where we have got to and now that the research agenda has been set and key stakeholders can be identified around these we can now begin the outcome mapping process.

e.g. KSI, DFID ESRC growth programme, CDKN, RIPPLE, SLRC.

OM or parts of it at least is becoming a standard for defining and identifying outcomes.15