measurement impact 101: practical techniques for capturing evp inputs, activities, outcomes &...
Post on 22-Dec-2015
217 views
TRANSCRIPT
Measurement Impact 101:
Practical Techniques for Capturing EVP Inputs, Activities, Outcomes &
Impacts
Official Business Track Sponsor:
Syreeta SkeltonAssociate Director of Evaluation &
Performance Measurement
Today’s Agenda1. Core Metrics for Employee Volunteer Programs
2. Corporate Perspectives on EVP Measurement & ReportingA. Credit Suisse –enterprise-wide case of
evaluation/measurement process development, framework & instruments
B. Gap Inc. Foundation – Job Readiness program level case of evaluation/measurement process development, framework & instruments
3. Measurement Expert Perspective A. EVP Performance Measurement Guidance – importance of
framing the research question; what do you want to know about your program and who wants to know it
4. Group/Panel Q & A
Who is our audience?Where is your EVP in its evaluation maturity?
a) Novices – Tracking inputs and effortb) Intermediates – Tracking outputs and counting activitiesc) Experts – Assessing outcomes and impact
What types of data do you currently collect?
a) Informal data (i.e. stories)b) Systematically collected data
Are the data you use in your evaluation…
a) Qualitativeb) Quantitativec) Both
What are your learning objectives for today’s session?
What is the appetite for evaluation & reporting in corporate America?
Now more than ever, companies are demanding better standards, tools, and best practices for measuring and demonstrating the efficacy of corporate civic engagement, and particularly employee volunteerism.
Measuring the magnitude, effectiveness, and especially the impact of employee volunteering remains a top concern for businesses.
1.A Points of Light Foundation 2005 survey among corporate members found that evaluation of employee volunteering was the greatest challenge for respondents.
2.A survey of 77 multinational companies conducted by The Conference Board (2006) found that more than one-third of responding companies cite measuring results and outcomes as the biggest challenge they will face in managing their corporate contributions programs (Lim, 2010).
What is the appetite for evaluation and reporting in corporate America?
We are here
Want to get here!
Who’s leading the charge?
Corporate Service Professions
Who’s leading the charge?2006 Corporate Volunteer
Reporting Standards2010 Employee Volunteer
Reporting Standards
AngelPointsBay Area Corporate Volunteer CouncilCommittee to Encourage Corporate PhilanthropyCenter for Responsible Business,Haas School of BusinessLevi Strauss & Co.McKessonNorthern California GrantmakersPoints of Light FoundationSan Francisco Business TimesThe Volunteer Center Serving SanFrancisco & San Mateo CountiesUnited Way of the Bay AreaVolunteerMatch
Credit SuisseThe Boeing CompanyThe Coca Cola CompanyThe Hitachi FoundationKPMG LLPLBG AssociatesPoints of Light/HandsOn NetworkTarget CorporationThe Timberland CompanyTrue Impact
What are the key data points to collect about my EVP?
New Employee Volunteer Reporting Standards•Standard measures for EVPs are particularly important for:
Understanding how programs and projects compare to one another
Identifying how EVP operations and processes are performing
Documenting areas of success and opportunities for improvement
Determining how to leverage program resources most effectively
EVP Reporting Standard Key Metrics
1. EVP Partner Organizations2. Volunteers3. Volunteer Activities4. Volunteer Hours5. Volunteer Frequency6. EVP Participation Rates7. Company-Paid Service Utilization
Rates8. Valuation of Volunteer Hours9. Dollar Rate of Return on Investment
(ROI)10.Dollar Rate of Social Return on
Investment (SROI)
Tools for Benchmarking EVPs
Tool Measures About the Instrument
Employee Volunteer MetricsBenchmarking Toolwww.volunteerreporting.com
How does they way we count/trackvolunteering compare to the wayother companies do (e.g., whatcounts as volunteering)?
Owned by Point of Light Instituteand HandsOn NetworkOpen and free to all Supports the Corporate VolunteerReporting Standards v2.0
Drivers of Effectiveness for EmployeeVolunteering and Giving Programswww.volunteeerbenchmark.com
Does our employee volunteerprogram have the components tomake it effective (as determined by itsperformance on the Drivers)?How does our employee volunteerprogram’s performance on the Driverscompare to other companies?
Owned by the Boston CollegeCenter for Corporate Citizenship, with sponsorship from Bank of AmericaOpen and free to all
EVP ROI/Impact Survey [email protected]
What is the return on investment/social return on investment in employee volunteerism for my company?How does the return on investment in employee volunteerism compare to other programs?
Owned by the True Impact, a private research firm and Points of Light partnerCost ranges from $500-$3,000 annually depending on company size
Official Business Track Sponsor:
Lalita Advani, Vice PresidentCredit Suisse Americas
Foundation
This Section’s Agenda
I. Tracking: Why Bother?
II. Evolution of Measurement
III. Sneak Peek of our System
IV. Next Steps for You
I. Tracking: Why Bother?
Dollars + Volunteer Hours Greater Value
1. Strategy
53%
36% 32%
47%
25%
25% 30%
25%
AM PB IB ShS
2009 Engagement Per Division
77%
62% 62%
72%
2. Motivation 3. Credibility
II. Evolution of Measurement
RudimentaryStaff/Grantee
Responsibility
DefiningKey Fields & Process
Garnering Resources
Building Global Consensus
System Development
& Launch
II. Evolution of MeasurementDefining Key Fields & Process
INPUTS:
OUTPUTS:
BUSINESS
IMPACT:
ENGAGEMENT = VOLUNTEERING + PARTICIPATION
DEMOGRAPHICS – FTE or Staff, Title, DepartmentTIME – On-Company or Off-Company
Number of Unique Employees, Total Hours
VOLUNTEERING – Company sponsored, direct servicePARTICIPATION – Fundraising, Donations, Attendance
Employee Morale, Recruitment/Retention, Career Development, Team Building, Cross-departmental Networking
SKILL LEVEL – Extra (Pair of) Hands or Skills-BasedLEVEL OF COMMITMENT – One-Time or Recurring
Activity Types, Sectors, Population Served, # of Lives Impacted
III. Sneak Peek – Team Leader View
KEY FEATURES:
• Team Leaders can build events for employees to view online
• Key Fields are mandatory, so that information is captured up front
III. Sneak Peek – Team Leader View
KEY FEATURES:
• Team Leaders can check off who attended and who did not attend
• Detailed employee information is captured for future analysis
III. Sneak Peek – Employee View
KEY FEATURES:
• Employees can view a list of events online and register to attend
• Philanthropy staff has access to global activity
III. Sneak Peek – Staff View
KEY FEATURES:
• Philanthropy Team can run standard reports
• Granular detail is also available by exporting data
• WHERE – Understand where in the cycle your organization stands
IV. Next Steps for You
• WHY – Develop the internal case and garner resources
• WHO – Identify organizations that have done this and learn
• WHAT – Define what key fields you need to measure
• WHEN – Develop near-term goals and plan for delays
Official Business Track Sponsor:
Marianne CampbellSenior Manager, Gap
Foundation
YouthDevelopment
Women’sAdvancement
CapacityBuilding
EmployeeEngagement
FOUR AREAS OF FOCUS
Preparing youth for life through the world of work
Job Readiness
Internships
Follow-on Support
Career Exploration
This Way AheadGap Inc. Youth Signature ProgramCareer Exploration
One Month+
•Individual career assessment
•Exploration of roles in various industries
•Long-term planning and goal planning
Career Exploration
One Month+
•Individual career assessment
•Exploration of roles in various industries
•Long-term planning and goal planning
Job Readiness
Four Months
• Weekly classes that teach hard and soft skills that prepare
youth for the world of work and life
•Gap Inc employees serve as volunteer
facilitators
Job Readiness
Four Months
• Weekly classes that teach hard and soft skills that prepare
youth for the world of work and life
•Gap Inc employees serve as volunteer
facilitators
Internship
Four Months/13 Hours per Week
•Two week preparatory course•Paid internships
that occur in Gap or Old Navy stores
•Monthly coaching sessions with Store
Manager
Internship
Four Months/13 Hours per Week
•Two week preparatory course•Paid internships
that occur in Gap or Old Navy stores
•Monthly coaching sessions with Store
Manager
Follow-On Support
Twelve Months
•Check-in with career coach
•Opportunity to interview for a
permanent part-time job at a store
•Access to other job opportunities,
outside of Gap Inc
Follow-On Support
Twelve Months
•Check-in with career coach
•Opportunity to interview for a
permanent part-time job at a store
•Access to other job opportunities,
outside of Gap Inc
This Way AheadEvaluation Process
• Created a Logic Model • Identified desired impact • Developed strategy to achieve that impact
• Designed an evaluation to measure progress to desired impact goal
• Determined who is evaluated and how• Youth• Nonprofit Organization• Employees
• Evaluation resource• TCC Group
Gap Inc. Employees:•Develop their own skills by
helping interns•Increased pride in store and Gap
Inc.•Deeper connections with co-
workers•Increased productivity
All Youth in Program:•Have expanded awareness of
career opportunities •Have a better understanding of what it takes to get on a career
track •Have more positive attitude
toward jobs, working, and their futures
•Are motivated to do what it takes to get internship and to get
career later i.e. stay in school, get good grades, etc
SHORT-TERM OUTCOMESSTRATEGIES LONG-TERM OUTCOMES IMPACTRESOURCES/INPUTS
Gap Inc. Resources
Gap Inc. Employee
skills, time,
experience
Partner CBO skills
and experienc
e
Youth interest, time, and readiness
Gap Inc. Youth Signature Program Logic Model
Underserved youth are
better prepared for
life and ultimately to
better contribute to
society
Gap Inc. is known
as an innovative corporate
citizen that
achieves real
results for society and the
business
Customers:•Positive associations with
Gap Inc. brands•Loyalty to Gap Inc. goods
Smaller # of Youth:• 4-month internship
• Follow on support post internship
So
cial
Im
pac
tB
usi
nes
s Im
pac
t
Gap Inc. Employees:•Volunteer at CBOs to provide
job readiness training•Act as mentors to interns
Participating Stores:•Train employees for program
•Communicate to all store staff about program
Gap Inc. communicates to: employees, Gap Inc.
leadership, and customers about program and its impact
Field Leadership participate in program
Interns: •Acquire new skills
•Have the opportunity to practice the new
skills•Take steps toward career or vocational
plan
All Youth in Program:• 4-month Job Readiness
Training at CBOs•Supplementary sessions with Gap Inc. employees
Field Leadership and store managers see
benefits; want to sustain the program
Participating stores:•Store leaders build own skills
•Higher employee morale in store
Participating stores have less turnover, better work
force
Evaluation Results:Used for continuous program enhancement
Shared with public, customers, Gap Inc. leadership, employees, shareholders
Evaluation Included Three Data Sources
Focus Groups Interviews Surveys
YouthFocus groups with youth interns at end of the Job
Readiness phase
A pre-post retrospective survey at completion of
job readiness phase
CBO Staff
Interviews with coaches from The Door at end of the Job Readiness
phase
Gap Inc. Employee Volunteers
Focus groups with Gap Inc. volunteers at end of the Job Readiness phase
Survey at the end of the job readiness phase
Youth
Gap Inc. Employees
CBO Staff
Findings
√ = outcome narrowly achieved
√√ = outcome somewhat achieved
√√√ = outcome solidly achieved
√√√√ = outcome strongly achieved
Outcome Category for YouthOutcome
Achievement
Youth Attained Skills Needed to Find and Secure a Job including Interviewing Skills √√√√
Youth Improved their On-the-Job Skills: Customer Service, Relating Well with Others, and
Communication Skills√√√√
Youth Gained Confidence in their Future Success √√Youth Experienced Personal Growth (Maturity, Responsibility, Greater Willpower, Leadership,
etc.)√√√√
Youth Strengthened their Goals Related to Educational Attainment √√
Youth Evaluation Summary
Official Business Track Sponsor:
Jan Brown, Senior Consultant
This Section’s Agenda
Framing the Evaluation of Corporate Volunteer/Service Efforts
Questions & Metrics
The Typical Project or Program Framework
The Corporate Volunteer/Service Project or Program Framework
Corporate Social
Responsibility
Inputs Strategies Outcomes Impact
Framing the Evaluation Questions: Outcomes• What social outcomes will result from our
volunteer/service program or effort?– CBO partner’s target audience (individuals and/or
groups)– General public (community, neighborhood, region, etc.)– CBO partner’s organizational capacity
• What business outcomes will result from our volunteer/service program or effort?– Employees (current & future)– Business units/departments– Corporate reputation
Framing the Evaluation Questions
• What was the quality of our volunteer/service project, program or event?– Volunteer/employee experience– Target audience (e.g., CBO or their clients) experience– Other partners’ & stakeholders’ experience– Resource needs & use
Framing the Evaluation Questions• What, specifically, about the
volunteer/service project, program or event worked, and what did not, in relation to achieving the outcomes?– What specific experiences did outcome achievers have
that non-achievers did not?– What background traits, environmental conditions and/or
readiness variables distinguished high achievers (with re to outcomes) from the rest?
– What project, program or event resources were critical to supporting the specific experiences that made a difference?
Exercise: What Are Your Company’s Volunteerism/Service Outcomes?
1. How will employees, CBOs and/or CBO’s clients be “directly” changed (improved) by the experience?
– Awareness– Knowledge
– Attitude– Motivation
– Skills– Opportunity
– Behavior2. What would people hear implementers, participants and
service recipients say or see them do if the outcomes were evident?
Contact: Peter YorkTCC Group http://www.tccgrp.com/
Questions
Panel Question & Answer Period
Official Business Track Sponsor:
Thank You