Showing it’s worth it: Measuring the impact of volunteering
@IVRtweets@VolunteeringEng
Nissa Ramsay11th February 2012
Introduction
How important is impact measurement to your volunteering?
• Very Important
• Not important
• I’m going to sit on the fence
Introduction• Who here thinks they could measure the impact of their
volunteering project?
• I’d be ace at this
• I could blag it
• I don’t know what you’re talking about
Today…
My vision:
To enhance the capacity of the student volunteering sector
Specifically,
To increase how you value impact measurement
To improve your understanding of impact measurement
Lesson 1… Always consider who is asking the question, to who, what
their relationship is and the context
Lesson 2… The best evaluation questions are part of the delivery
process: It should not be hard work
Lesson 3… Measuring impact is not an exact science but a balancing
act between the ideal world information you would want and the reality of working in your role
Why bother?You should do this if:
• You have a genuine desire to improve the volunteer experience
• You have to as a requirement of your funding
• You want to apply for funding
• You want to shout about how amazing your work is
• Your volunteers are always deserting you
• You can’t convince any one to volunteer for you
You should think twice about doing this if:
• You have no time/money/resources you’re willing to devote
• You are only doing this as a requirement of your funding
• You have no use/goal for the results
Lesson 4… Measuring impact is also about improving the quality of
your work and delivery (not just the end statistics)
Lesson 5… Measuring impact can help focus your project/
organisation on its core purpose (BUT can also take you off course)
Lesson 6… Measuring impact can work on a range of levels:
Project, Thematic area, organisation
Where to start1. Theory of change
Articulates how, when and why a change happens as a result of the project/intervention
= measure your impact by testing the assumptions behind this
= show if you create positive outcomes and why YOU have
= Can be on a project/programme or organisation level
Case Study: Student Hubs…www.studenthubs.org/xwiki/bin/View/main
http://www.youtube.com/watch?v=TLXTnc2YNuk
A mission statement/overall aimWhat?
• A mission statement
• Describes the overall impact of the programme
• The bigger picture – you aren’t measuring this directly
Why?• This is your impact: it sets the context as to why you are doing this
Student Hubs empowers students to create positive social change, now and in the future
Specific aimsWhat?
• Precise statements about change
• Break down mission in to components and target groups
• And/or consider your activities – why are you doing all of them and what do they amount to
Why?• This will help identify target groups and the core components of your work/ impact/beneficiaries
Student Hubs will inspire students to take social action
Outcome indicatorsWhat?
• Statements of change that we can measure, link to specific aim
• Achievable and a change measure (perceptions/feelings)
Why?• This will help decide who to involve in the evaluation, why and what questions you will ask them
Output indicatorsWhat?
• Activities and numbers/targets involved
Students feel more motivated to be socially active
500 students involved have taken on new volunteer roles
The connections
Vision
Aim 1Aim 2
Aim 3
Change = Outcomes
Activities = Outputs
If your aim is: To provide high quality volunteering opportunities• Volunteers’ views on support, management and training
• Their volunteer manager (approach / availability etc)• Perceptions of bureaucracy (e.g. CRB, interviews)• Importance of payment of out-of-pocket expenses• Recognition received• Workload and level of responsibility
•Whether they see their involvement as meaningful
• What they see as the impact of their volunteering• Whether they perceive their help as being needed• Whether they perceive any barriers limiting this
Your turn…Group discussion:
Choose either
- A project one of you are running
- Student Hubs
Identify
- Specific aims: To increase awareness/perception…
- Outcome indicators: Volunteers have increased skills
- Output indicators: 25 volunteers have been trained in…
Lesson 7… Measuring impact has to start with understanding what
you are trying to achieve and how
Methodologies and tools• Surveys
• Online (e.g. Surveymonkey)• Access to a sample group (e.g. email addresses for online) • Response rate and representativeness (using incentives)
• Focus groups• Do not have managers in the same room!
• In-depth interviews• Informal approaches (e.g. walking interviews)
• Observation• Volunteer testimonials and case studies• Life history interviewing• Mapping exercises
Involving volunteers in the process• Peer research and volunteer involvement in the process
• Accessing new skills and experiences
• Ability of volunteers to form different relationships with respondents, as
peers
• Quality of data vs. professional development of volunteers
• Project guidance and steering groups
• Volunteer / user perspective (e.g. the development of research tools)
Improving the quality of research• Gain organisational buy-in and support
• Ask only what you need to know
• Avoid leading questions
• Always pilot surveys
• Don’t cherry-pick respondents but pick randomly
• Opportunities for tracking / longitudinal research
Using toolkits• The Volunteer Impact
Assessment Toolkit• Hard copy and website
• Assesses impact on
volunteers, the organisation,
service users and wider
community
• Guidance on tools and
methods
• £29.99 plus postage
Limitations, challenges and pitfalls• Self-reported nature of satisfaction
• Unavoidable to a certain degree
• Can attempt to triangulate the results (e.g. speak to staff, clients)
• Ensuring a good response rate• Can the results speak on behalf of all volunteers if changes are to be
made?
• Working with volunteers with specific resource needs• e.g. disability, low confidence, illiteracy, English as a second language
• Cost and time implications but everyone needs to have the opportunity
to participate
Taking it forwards: options
Undertaking in-house Sub-contractedGreater understanding of specific volunteering context
Draws on expertise in volunteering from a wide variety of fields
Cheaper option (financially but not in staff time)
Requires limited in-house staff time and easy to manage
Allows independent verification and increases confidence in results
Volunteers may be more honest and at ease when not talking to staff
Communication and dissemination• Accessibility of findings
• Full report vs. summary
• Plain English and awareness of audience
• Warts and all (honesty and accuracy)
• Will increase trust in findings
• Feedback to the respondents and the volunteer body• Recommendations / changes need to be communicated as well as
reasons for not changing something
• Review of changes / developments
Conclusion• How important is impact measurement?
• Very Important
• Not important
• I’m going to sit on the fence
• Who here thinks they could measure the impact of volunteering within a project?
• I’d be ace at this
• I could blag it
• I’m not really sure
The results• At the start of my session
• 92% felt impact measurement was ‘very important’
• 0% felt it was ‘not important’
• 8% felt they were ‘going to sit on the fence’
• At the end of my session
• 96% felt impact measurement was ‘very important’
• 0% felt it was ‘not important’
• 4% felt they were ‘going to sit on the fence’
ConclusionAt the start of my session:
• 8% of you thought you’d be ace at measuring impact
• 45% of you thought you could blag measuring impact
• 28% of you ‘don’t know what I’m talking about’
At the end of my session:
• 10% of you thought you’d be ace at measuring impact
• 73% of you thought you could blag measuring impact
• 2% of you ‘don’t know what I’m talking about’
The results
As a result of me running this session today:
28% more people can blag impact measurement
[email protected] 520 8900www.ivr.org.uk
Thank you