santa cruz international meet up mar 10
DESCRIPTION
Overview of Firelight’s Foundation's M&E training approach for grassroots organizationsTRANSCRIPT
Making M&E Accessible to Grassroots Organizations:
Lessons Learned from Firelight’s Capacity Building Program
Santa Cruz International
Non-Profit Meet-Up,
9 March 2010
Presentation Overview
• Overview of Firelight’s M&E training approach
• Engage in practical exercises to:
– Identify guiding principles for foundations’ M&E activities
– Identify how key M&E concepts and practicalities can be
better understood and utilized by grantees
• Sharing Firelight’s outcomes to date, challenges,
and way forward
M&E?
RESULTS?
Monitoring & Evaluation:The International Development Paradigm
Objectives
Statements
Performance
Indicator Statement
s
MeasurementMethods / Data
Sources
Goal
StrategicObjectives
IntermediateResults
Outputs
Activities
Critical Assumptions
Capacity Building Strategies
1) Training to build capacity of grantees, e.g. “traditional” M&E skills
2) Special grants to support organization-specific capacity development and learning needs
3) Partner learning exchanges and networking meetings
4) Weekly “Newsflash” to share information with grantees
5) As-needed requests
Guiding Principles(official)
• Nurture strong, sustainable grassroots
organizations.
• Meet groups “where they are.”
• Build partners’ ability and confidence to
monitor and evaluate their own work.
• Peer-to-peer technical assistance is most
effective.
Guiding Principles (unofficial)
• Grantees are already monitoring their work.
• Perfect beneficiary numbers are not our goal.
• Impact is NOT easy to prove due to:
• Timeframe
• Causality and attribution
• M&E activities should never detract from the
work at hand, which is serving children.
Cost
Complexity
Existing records
(e.g. household
lists)
Routinestatistics
Focus groups
Specific samplesurveys
Key informantinterviews
Observation
Special or ’point’ studies
Choosing M&E Methods
Keep expectations realistic.
M&E Training
• 43 partners trained in 3 countries by Insideout.
• Aims to build grantees’ capacity to measure their own progress in a more meaningful way.
• Training is practical and experiential, to apply principles to organizations’ day-to-day work.
What did we hope to see?
Intended Long-Term Outcomes for Trained Grantees
• Organizations clearly identify strategies and intended
outcomes of their work.Monitoring is paired with ongoing, day-to-day work.Organizations have ownership of their monitoring framework,
tools, and procedures.Management/leaders makes changes in the organization’s
programs and activities based on what is learnt.The organization shares what it learns with local communities.Reports to an organization’s donors and other stakeholders are
a more complete depiction of the outcomes of their work.
Making M&E Accessible
De-technicalize language.
ROADMAP TO
MONITORING
What are we trying to
change?
(problem analysis)
Where do we want to
get to?
(goal, objectives)
How are we going to get
there?
(strategy, activity)
What do we expect to happen along the
way?
(RESULTS)OUTPU
TOUTCOME
IMPACT
How do we know we are on the right
road?
(indicators,
baseline, targets)
OUTPUT
OUTCOME
IMPACTACTIVITI
ESHave the activities
taken place?
The very first result
of an activity.
Organizations have direct control
over this result.
What happened
next?
Change of behavior in participant
s.
Organizations have
less control over this result.
So what?
Change at population/
societal level.
Organizations have
very little control, if
any.
Making M&E Accessible
Making M&E Accessible
1. Collect
2. Compile
3. Compare
4. Share
Get to the practical.
Simple Steps in Data Analysis
Making M&E Accessible
Offer long-term,
ongoing support to grantees.
“I appreciate that there will be support after the training, and that we will not have carry on alone.”
Training Outcomes
Enthusiastic Feedback on the Training Curriculum/Approach
INCREASED CONFIDENCE - “We can see we are making progress.”
• “The simple words removed the fear and myths about M&E.”• “I used to get indicators, outputs and outcomes confused but now I know the difference.”• “I learnt that we are already doing monitoring, and we can build on that. But [we] need to make the process more systematic.”
RECOGNITION OF THE VALUE OF M&E• “I learnt to think about more than outputs – that it is important to
think about the ‘So whats?’ of what we do.”• “We need to monitor in order to be able to share our success
stories and learn from our mistakes!”• “We are used to only thinking about monitoring at the end of
a project – this workshop has changed that!”
Training Outcomes
Evidence of “uptake” thus far• Conceptualizing programming in terms of results
• Monitoring at the core of grantee’s planning - “We were doing
things randomly, jumping from here to there. Now we go forward
based on what our plan is.”
• Using the skills on projects not funded by Firelight - “The
knowledge we are getting is helping us improve our service
delivery, even on projects not funded by Firelight Foundation.”
• “Reporting is easier because you can go line by line, comparing
what you have done to what you planned.”
Training Outcomes
Evidence of “uptake” thus far
• Improved/adapted programming
• Strengthened fundraising - “[U.S. Embassy] officers were
shocked to find out that we are even capable of monitoring
and evaluating our project. Remember, we had written our
proposal by ourselves – there was no outside assistance.”
• Increased engagement with government frameworks
• Strengthened collaboration, networking, and advocacy
Challenges
• Buy-in at organizational level - staff turnover, quality of shared learning within organization, “Some people who were attended were not the right people.”
• Management issues within organizations - planning (strategic, workplans), human resources management, budgeting/financial management
• Efforts needed to ensure outcome indicators included in grantees’ M&E frameworks
• Some baggage/anxiety about M&E remains - pleasing the donor, some still searching for the “holy grail” of M&E
Challenges: Reality Check
“Absence of computers to process data and store the information so it can be easily accessed.”
“We now need to be able to cross the river more frequently.”
Challenges:Showing Progress
• Lack of baseline data
• Data quality issues - verification needs
• Interest in participatory M&E with communities
• Data management and recordkeeping
• Lack of resources to devote to M&E (time, staff, $)
What do these challenges reveal to Firelight?
Grantees are excited.
Grantees are invested.
Grantees are “taking on” M&E within their organizations.
Next Steps
1. Conduct review of M&E training outcomes. Produce publication on Firelight’s approach and lessons learned.
2. Test revised curricula for training in additional Firelight focus countries in 2010/11.
Remember:
It’s not about the indicators.
It’s about reflection and learning.
Good luck in your own M&E efforts!
THANK YOU