Controlling Your Evaluation Destiny

Marla SteinbergDirector of Evaluation

MSFHR & The CAPTURE Projectwww.thecaptureproject.ca

Can you spot the PEAR cake?

What does evaluation mean to you?

“Evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object”

Evaluation can be defined as the systematic collection and analysis of information on the performance of a policy, program, or initiative to make judgments about relevance, progress or success and cost-effectiveness, and/or to inform future programming decisions about design and implementation.

Evaluation means asking good, critical questions about programs to improve them and help them be accountable for the wise use of resources.

Don’t be put off by the language of evaluation

Planning TipsProgram Participant Practitioner or

ManagerFunder

Set up evaluation advisory committee

Jointly determine purpose of evaluation and generate evaluation questions

Jointly develop program logic modelso multiple perspectives are represented

Jointly determine data collection strategyChoose methods that meet required level of rigour and are acceptable to

participantsJointly identify other stakeholders and develop engagement and

communication strategy

Jointly develop reporting and dissemination strategyThink about different products for different audiences

Data Collection Tips

• Chosen data collection methods that meet required level of rigour and are acceptable to participants

• Include qualitative and quantitative data, get people to tell their stories

• Use multiple methods• If possible try to use engaging methods• Use lay or participant data collectors

Analysis and Interpretation Tips

• Hold interpretation sessions• Find creative ways to display data that can

speak to different audiences

Reporting and Dissemination Tips

• Don’t just create the one long evaluation report

• Video• Art• Brief reports

Using Results Tips

• Quality improvement• Management• Decision-making

Jointly develop action plan

Joint Action PlanningAction Planning WorksheetEvaluation Recommendation:Action Step(s) Person(s)

ResponsibleTimeline Plans for

monitoring results

Final Tip: Plan Evaluation Up Front

How many evaluators does it take to change a light bulb?– one to do a needs assessment– one to do a feasibility study– one to do a qualitative study to find out what bulb to change– one to empower the bulb to change– one to tender a contract for further study– one to write performance indicators for success– one to do a cost benefit analysis to determine the best to buy– one to do a meta-evaluation showing that all previous studies have

left everyone in the dark

SO.......how many evaluators does it take?