managing for results a sampler of plan:net’s perspective and experience july, 2010 for more...

Managing for Results

A Sampler of Plan:Net’s Perspective and Experience

July, 2010

For more information go to www.plannet.ca

Commitments to Results Accountability

• Results Based Management (RBM) evolves as a

means of regulating the flow of public funds under

New Public Management.

• By the mid 90’s, All OECD countries have adopted

results based management approaches as a matter

of policy.

• Canadian Federal and Provincial Ministries,

multilateral organizations and philanthropic

organizations looking for evidence of results

achieved.

What does this mean in practical terms?

• When making a plan, or reporting on progress, we are challenged like never before to:

• Describe the connection between the work we do and the difference we make

Example…If you are developing…

• Applied Research on Eco Tourism…You would focus attention on… – the number of men and women participants; – what they are exposed to through their interactions;– what they contribute of their own knowledge and experience; – experiments started, finished, documented and sharedand also…– What participants DO with the knowledge, confidence, contacts

gained– eco-tourism business plans, financing and start ups– employment and revenue– protection of ecosystem – behaviour and attitude shifts within target market

Three RBM Tools

• Results Logic Model/Framework• Risk Analysis and Management Table• Monitoring and Evaluation Plan

Important Preparatory Steps

Before beginning a results based plan..

1. Looking Inward

- Mandate- Competencies- Experience

2. Looking Outward- Issues & Opportunities

3. Clarifying Priorities and Planning Perspective/Scale

Situation Assessment - Stakeholders

Influence

Inte

rest

or

Impo

rtan

ce High Interest/ Importance, High Influence

Low Interest/ Importance, High Influence

Low Interest/ Importance, Low Influence

High Interest/ Importance, Low Influence

Build coalition of support

Spend more time on the other Stakeholders

Consider ways to strengthen or protect interests

Caution - be aware of risks or obstacles

HIGH

LOW

HIG

H

LOW

Understanding your Working Environment - Stacey Matrix

Complex

Complicated

Complicated

Simple

Close to Certainty Far from Certainty

Far

fro

m A

gre

emen

tC

lose

to

Ag

reem

ent

Technical Aspects

Pol

itica

l Asp

ects

Where is your project or organization

on the Stacey Matrix? • Implications for planning and

management?

The Results Logic Model

Splash and Ripple!Splash and Ripple !

- a metaphor for RBM

Example: A Hypothetical Project to Support Entrepreneurs

Ultimate Outcomes

Intermediate Outcomes

Immediate Outcomes

Outputs

Activities

Inputs Trainers, Materials, Equipment, Lease/Rent, Travel, Communications….


Ultimate Outcomes


Immediate Outcomes

Outputs

Activities Recruit entrepreneurs

Assess training needs

Deliver training & coaching

Launch learning circles, organize networking events



Ultimate Outcomes


Immediate Outcomes

Outputs

Entrepreneurs get in touch/ apply for membership

Members identify critical business, personal and IT gaps

Participants complete training & participate in coaching sessions

Members participate in learning circles and network events


Assess training needs Deliver training &

coaching




Ultimate Outcomes


Immediate Outcomes

Diverse, expanding pool of entrepreneur members

Member participants apply learned skills and use new contacts in the development of their own businesses

Outputs







coaching




Ultimate Outcomes


Member businesses are webbed in mutually re-enforcing upstream/downstream relationships

Member participants manage viable, employment generating enterprises

Immediate Outcomes


Member participants apply learned skills and use new business contacts and ideas in the development of their own enterprises

Outputs






Assess training needs

Deliver training & coaching




Ultimate Outcomes

Micro and small enterprises gain prominence as an engine of development in ‘x’ region


Member businesses are webbed in mutually re-enforcing upstream/downstream relationships

Member participants manage viable, employment generating enterprises

Immediate Outcomes


Member participants apply learned skills and use new business contacts and ideas in the development of their own enterprises

Outputs







coaching



A Progression of Results - Typical Terms

What we Need - Staff Equipment Supplies Office Space Services

Inputs

What we Do - Research Build Train/Mentor Promote Procure Convene

Activities

What we Produce - Networks Courses Trained participants Assessments Publications Infrastructure

Outputs

Capacities Developed - Institutional exchanges/collaborations

Applied knowledge Functioning spaces/infrastructure

System improvements/adaptations New/Improved leadership

Aligned decision-making

Immediate Outcomes

Performance Improved - Institutional behaviour changes

End-user benefits Larger scale policy and programming shifts

Social mobilization


Conditions - Social Economic

Civic Cultural Environmental

Ultimate Outcomes

Logic Model FormatUltimate

Outcomes

How

What we want

Why

Inputs

ImmediateOutcomes

ImmediateOutcomes

ImmediateOutcomes

ImmediateOutcomes

Activities Activities Activities Activities Activities

Outputs Outputs Outputs OutputsOutputs




Mapping your logic should be a creative process

Risk Analysis and Management Table

Enablers and Constraints…• Are forces affecting

any part of your programme or project, both positively and negatively.

Examples of Enablers or Constraints (project/programme level)• Dominant attitudes or behaviours among stakeholders (positive or negative)• Related government policies or programmes (positive or negative)• International agreements or conventions (positive or negative)• Uncertain or cyclical weather conditions

Risk Analysis MatrixEffect

Significant (3)

Moderate (2)

Minor (1)

Low (1) Medium (2) High (3)

Likelihood

Monitor Risks

Acceptable Risks

Acceptable Risks

Monitor Risks

Monitor Risks

Risk Mitigation Required



Acceptable Risks

A Hypothetical Project to Support Entrepreneurs

Intermediate Outcome:Member participants manage viable, employment-generating enterprises

1. What could hamper progress?

2. What is the likelihood of a problem?

3. What impact would the problem have on the intermediate outcome?

A Typical Risk Assessment and Management Table

Description of Risk

Likelihood of Occurrence

(low, medium, high)

Effect on Planned Results(minor,

moderate, significant)

Rank in Importance

(1 = highest)

Owner of the Risk

(who takes responsibility)

Risk Mitigation Strategy

(how to address the trend)

Risk Indicators

(what to look out for)

Ultimate Impact level

Intermediate Outcome level

Immediate Outcome level

Output level

Caution…this is a summary tool - use creative methods to generate the content for this table.

Monitoring and Evaluation Plan

What indicators tell us about the wind…

Who has seen the wind?

Neither you nor I.

But where the trees bow down their heads,

The wind is passing by.-The Wind, Christina Rosetti, 1830-1894

Monitoring and Evaluation - The Distinction

• MonitoringA continuous (or regular) self-assessment of progress -collecting data and comparing current performance with planned activities and results.

• EvaluationIndependent (often external), periodic, strategically focused assessment of a programme/project’s continuing relevance, management, governance, results, coverage, external relationships, sustainability, etc.

Indicators…• Evidence that the

programme is producing expected results at the output and outcome levels.

• They can be expressed quantitatively and qualitatively.

• Information collected should help managers make wise decisions.

Quantitative IndicatorsExpressed as a percentage, ratio, proportion, comparison, or number that can be analyzed statisticallyExamples?

– Proportion of members (f,m) who say they have improved their business skills since joining the Entrepreneurship Programme

– Number of new jobs created and lost by programme members in the past year

– Composition of programme (f,m) membership, by area of business activity

– Value of goods sold in the past year– Percentage of members showing:

–Market expansion–Increased product range–Improved sales

….In the past year

Qualitative IndicatorsExpressed as a change or comparison between at least two states of understanding or experience that can be analyzed for patternsRelies on people’s judgment or perceptionExamples?

– Perceptions of men and women participants of the way in which the Entrepreneurship Programme is guiding members through the current global economic crisis

– Trends in the level and type of coverage of Entrepreneurship Program by local business writers and reporters

– Trends in the calibre of business plan submissions, with reference to: a) innovation, b) strength of case, c) clarity of presentation

Indicators, Baseline, and Target

Indicator

Baseline

Target

Meaning

The information to be collected

What the indicator would tell you at the beginning

What you want the indicator to tell you at a specific point in the future

Example

Net # of jobs created before FY 2012

1200

2500

Indicator Selection Checklist

Validity - Does it measure the result? Yes No

Is it cost-effective to collect the information?

Yes No

Does it help understand how the program or project is affecting men and women, and/or specific sub-groups of people differently?

Yes No

Does it provide useful information for management decisions?

Yes No

Does it communicate well to stakeholders (public, policy makers, donors)?

Yes No

A Typical Performance Measurement FrameworkIndicator Baseline Target Data Source Method Persons

Responsible




Output level

Caution…this is a summary tool - use creative methods to generate the content for this table.

Preparing Results Based Progress Reports-An introduction

From Activities to Results

• In RBM, we are challenged to describe what has changed NOT what was done

When Reporting on Results…

• Place the people affected, or the product generated by the activity in the centre of your statements, not the person doing the activity

• If you think you are describing an activity, ask the “so what” question

When Reporting on Results…

Be guided by your results framework• Outputs• Immediate Outcomes • Intermediate Outcomes• Ultimate Outcomes

More specifically, look to your indicators

Reporting Against Planned Results

Indicators Targets Baseline Actuals VariancePriorities for Next Period




Output level

Planning, Monitoring and Evaluation, Reporting

Key functions of Project Cycle Management…

Project Initiation

Proposal Development

Donor Appraisal

Start Up

Implementation

Monitoring / Evaluation

Phase Out

1.The basis for an introductory conversation with stakeholders

2. The centrepiece of your proposal or plan

3.The basis for workplans and job descriptions

4. A management reference during team meetings

5.A guide for monitoring and reporting on progress & adjusting strategies

6.An aid in designing an evaluation

RBM tools - Logic Model, Evaluation Framework (PMF) and the Risk Management Plan are relevant to all parts of the programme cycle…

managing for results a sampler of plan:net’s perspective and experience july, 2010 for more...

Documents

rbm slide

evaluation plan slide

results logic model

high influence low

low influence high

importance high

target market slide

accountability results