charity navigator 2.0 case study presentation
DESCRIPTION
A case study and discussion of Charity Navigator’s current efforts to movebeyond rating financial accountability to measuring outcomesTRANSCRIPT
![Page 1: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/1.jpg)
Charity Navigator CN 2.0 Case Study
Presentation at Managing to Outcomes Forum
Paul Brest, President, The William and Flora Hewlett Foundation With Ken Berger, President & CEO, Charity Navigator,
David Bonbright, CEO, Keystone Accountability, Xandy Brown, Pilot Project Coordinator, Charity Navigator,
And Professor David Campbell, SUNY Binghamton June 13, 2011
![Page 2: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/2.jpg)
The Data Proves Impact
Estimated 3.3 million distinct visitors per year (~5 million hits)
92% say evaluations affected their decision to support individual public charities
CN ratings influence decisions on billions in donations annually
![Page 3: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/3.jpg)
![Page 4: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/4.jpg)
Financial Health
Accountability/Transparency
Results More Comprehensive Rating System
CN 2.0
![Page 5: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/5.jpg)
Note: Strategic Plan is in draft and not yet finalized.
![Page 6: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/6.jpg)
• Why 10,000? Because ~70% of all revenues that come into the NP sector annually go to them.
• How to scale this effort? A new process to move beyond our staff and involve consumers in the rating of charities in cause areas they’re passionate about.
• now… 2.0…
10,000 CHARITIES RATED BY VOLUNTEER RATERS
![Page 7: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/7.jpg)
Reporting Results R&D Process
![Page 8: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/8.jpg)
![Page 9: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/9.jpg)
![Page 10: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/10.jpg)
What is the charity’s commitment to reporting results?
• Clear commitment to reporting results stated. • Specified time period for which results are
presented. • Some mission-related results in current period
compared to earlier period. • Reporting distinguishes between activities,
outputs and outcomes. • Credible intention to validate results evidence.
![Page 11: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/11.jpg)
How does the charity demonstrate the demand for its services?
• Reports indicating the aggregate numbers of individuals accessing the charity’s outputs in a given period.
• Evidence of demand for more than half of the charity’s outputs is provided.
• A statement of evidenced demand set against the larger demand that the charity does not presently have the capacity to reach.
![Page 12: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/12.jpg)
Does the charity report its activities and outputs?
• A clear description of the majority of the charity’s mission-related activities and outputs for a defined reporting period
• A significant portion of mission-related activities and outputs reported are compared to an earlier reporting period
• Activities and outputs reported can readily be related to reported expenditure
![Page 13: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/13.jpg)
Does the charity report its outcomes (medium and longer-term results)?
• A clear description of mission-related outcomes achieved by the charity in the reporting period.
• Outcomes are described for more than half of the charity’s mission-related activities or those mission-related outcomes described relate to activities that consume more than half of total charity’s expenditure.
![Page 14: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/14.jpg)
What is the quality of evidence for reported results?
• Evidence of most mission-related outputs includes some element of independent validation
• Low level of outcome evidence • Medium level of outcome evidence • High level of outcome evidence:
beneficiary feedback • High level of outcome evidence:
independent validation
![Page 15: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/15.jpg)
Does the charity adjust and improve in light of its results?
• Evidence that the organization assesses its mission-related performance in light of what was planned for the reporting period
• Admits mistakes and publicizes corrective actions • Admits mistakes, publicizes corrective actions and
commits itself to validating proposed corrective measures through dialogue with those affected
• Admits mistakes, publicizes corrective actions, and provides evidence that it has in fact validated proposed corrective measures through dialogue with those affected
![Page 16: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/16.jpg)
Framing Questions • Given the low level of current reporting of
results, what are the best initial rating criteria? • What is the best way to communicate and
implement rating criteria that will ratchet up over time?
• What initial criteria will be (a) possible to rate reliably by volunteers and (b) sufficiently achievable to have a basic level of compliance within 12 months, but (c) not be so easy as to be credible and widely gamed.
• Who else needs to be on the bus?
![Page 17: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/17.jpg)
What do student raters learn? • More questions raised than answered:
– How to differentiate activities, outputs, and outcomes when they are not explicitly labeled?
– How does a charity report results at this level and still have a website that is accessible and clear to lay people?
– How to decrease subjectivity?
![Page 18: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/18.jpg)
How was student feedback incorporated? • Materials
– Online platform – Centralized communication
• Rating – Illustrative examples of where to find
information – 4 level confidence scale
• Process – Thanking volunteers – Providing background
![Page 19: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/19.jpg)
Binghamton University Courses • Two courses
– Public Administration • 16 Students • Issues in Nonprofit Administration
– Social Work • Advanced Social Work Practice with Communities • 19 Students
• Unique Features – Local Organization Assessment – $3,000 in “Philanthropy Incubator” grants – Student Blog (“Navigating Southern Tier
Charity”)
![Page 20: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/20.jpg)
Course Structure • Public Administration
– Course Focus: Effectiveness and Accountability – CN Recommended Readings, Forces for Good
• Social Work – Forces for Good
• Inter-disciplinary Dimensions – Charity Navigator Assessment Teams – Local Organization Assessment Teams – Class Discussion
![Page 21: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/21.jpg)
CN: Students’ Practical Issues • Assessing Confidence Levels
• Positive Response to 5 Minute Rule
• Most Diligent: Assessments Time Consuming
• Importance of Preparation, Knowledge (SW vs. PA)
• Technology Challenges (CN Responsiveness)
![Page 22: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/22.jpg)
CN: Students’ Philosophical Issues
• Standards Incomplete – Mistrust – “Stars” system simplistic. – Good organizations negatively affected.
• Resource Limitations Constrain Performance • Assessing Transparency, not Performance
– Performance data depend on trust (not verifiable). • Discomfort with “Admitting Mistakes” • Donor vs. Beneficiary Focus
![Page 23: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/23.jpg)
Standards and Students’ Funding Decisions
• Real-world Challenge of Assessment • Charity Navigator Standards OK After All…
– Financial Measures – Transparency
• Role of Executive Compensation • Frustrations with Limitations of Secondary Data
– Assessment Incomplete • Creation of New Standards
– Role of Social Media – Student Concerns (small vs. large, effect of
contribution)
![Page 24: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/24.jpg)
End of Semester CN Rating Attitudes
• The perfect not the enemy of the good.
• Value of standards • Not all performance standards apply. • Create standards for stakeholder
dialogue • SW students: Limited utility for
beneficiaries
![Page 25: Charity Navigator 2.0 Case Study Presentation](https://reader034.vdocuments.net/reader034/viewer/2022052522/554ab1ecb4c905ec668b566d/html5/thumbnails/25.jpg)
End Semester Performance Assessment Attitudes
• Effectiveness a social construct • Negotiate standards with
stakeholders. • More questions than answers; “it
depends” • Importance of responsiveness to
performance perceptions.