results-based accountability
Post on 05-Jan-2016
35 Views
Preview:
DESCRIPTION
TRANSCRIPT
I N
N O
V A
T I
O N
Results-Based AccountabilityResults-Based Accountability
Discovery Communities TA InstituteDiscovery Communities TA Institute
April 1, 2008April 1, 2008
2
2
Results Accountability Decision-making
and Strategic Planning
Fiscal Policy Studies InstituteFiscal Policy Studies Institute
Santa Fe, New MexicoSanta Fe, New Mexico
WEBSITESWEBSITES
www.resultsaccountability.comwww.resultsaccountability.com
www.raguide.orgwww.raguide.org
www.charteroakgroup.comwww.charteroakgroup.com
BOOK ORDERS www.trafford.com
www.amazon.com
3
3
3
Why Are We Here?Why Are We Here?
4
4
SIMPLE
COMMON SENSE
PLAIN LANGUAGE
MINIMUM PAPER
USEFUL
5
5
Concepts to Take Away TodayConcepts to Take Away Today
Population accountability v. performancePopulation accountability v. performance accountability accountability
Ends v. meansEnds v. means
How to choose indicators and measuresHow to choose indicators and measures
The importance of a data development agendaThe importance of a data development agenda
How to move from talk to actionHow to move from talk to action
6
6
Results Accountabilityis made up of two parts:
Results Accountabilityis made up of two parts:
Performance Accountabilityabout the well-being of
CLIENT POPULATIONSFor Programs – Agencies – and Service Systems
Population Accountabilityabout the well-being of
WHOLE POPULATIONSFor Communities – Cities – Counties – States - Nations
7
7
7
Results and Performance Accountability
Results and Performance Accountability
COMMON LANGUAGE
COMMON SENSE
COMMON GROUND
8
8
THE LANGUAGE TRAPToo many terms. Too few definitions. Too little discipline
Benchmark
Target
Indicator Goal
Result
Objective
Outcome
Measure
Modifiers Measurable Core Urgent Qualitative Priority Programmatic Targeted Performance Incremental Strategic Systemic
Lewis Carroll Center for Language Disorders
9
9
The Humpty Dumpty Approach to LanguageThe Humpty Dumpty Approach to Language
““When When II use a word,” Humpty Dumpty said, use a word,” Humpty Dumpty said, in rather a scornful tone, “it means just in rather a scornful tone, “it means just what I choose it to mean—neither more nor what I choose it to mean—neither more nor less.” “The question is,” said Alice, less.” “The question is,” said Alice, “whether you “whether you cancan make words mean so make words mean so many different things.” “The question is,” many different things.” “The question is,” said Humpty Dumpty, “which is to be said Humpty Dumpty, “which is to be master—that’s all.”master—that’s all.”
LEWIS CARROLL, LEWIS CARROLL, Through the Looking-Through the Looking-Glass,Glass, chapter 6, p. 205 (1934). First chapter 6, p. 205 (1934). First published in 1872.published in 1872.
10
10
DEFINITIONS
Children born healthy, Children succeeding in school, Safe communities, Clean Environment, Prosperous Economy
Rate of low-birthweight babies, Rate of high school graduation, crime rate, air quality index, unemployment rate
1. How much did we do? 2. How well did we do it? 3. Is anyone better off?
RESULT
INDICATOR
PERFORMANCE MEASURE
A condition of well-being for children, adults, families or communities.
A measure which helps quantify the achievement of a result.
A measure of how well a program, agency or service system is working. Three types:
= Customer Outcomes
Popu
latio
nPe
rfor
man
ce
11
11
From Ends to Means
ENDS
MEANS
From Talk to ActionPo
pula
tion
Perf
orm
ance
RESULT
INDICATOR
PERFORMANCEMEASURE
Customer outcome = EndsService delivery = Means
12
12
Connecticut Glossary of RBA TermsConnecticut Glossary of RBA Terms
The Appropriations Committee standardized the The Appropriations Committee standardized the terms we use in Connecticut. You have a copy of the terms we use in Connecticut. You have a copy of the glossary.glossary.
We could have chosen other terms consistent with We could have chosen other terms consistent with Friedman’s RBA approach so this list is somewhat Friedman’s RBA approach so this list is somewhat arbitrary.arbitrary.
Its virtue is that everyone in Connecticut doing this Its virtue is that everyone in Connecticut doing this work – both executive branch and legislative branch – is work – both executive branch and legislative branch – is using the same vocabulary and meaning the same thing using the same vocabulary and meaning the same thing by it. At least we should be able to understand each by it. At least we should be able to understand each other.other.
13
13
Some Connecticut Early Childhood IndicatorsSome Connecticut Early Childhood Indicators
% Infants born at low birth weight% Infants born at low birth weight
% Births to mothers without a high school degree% Births to mothers without a high school degree
% of kindergartners with all or most pre-literacy and personal % of kindergartners with all or most pre-literacy and personal skillsskills
% 4% 4thth grade reading scores at mastery or above grade reading scores at mastery or above
14
14
IS IT A RESULT, INDICATOR OR PERFORMANCE MEASURE?IS IT A RESULT, INDICATOR OR PERFORMANCE MEASURE?
1. Safe Community
2. Crime Rate
3. Average Police Dept response time
4. A community without graffiti
5. % of surveyed buildings without graffiti
6. People have living wage jobs and income
7. % of people with living wage jobs and income
8. % of participants in job training who get living wage jobs
15
15
IS IT A RESULT, INDICATOR OR PERFORMANCE MEASURE?IS IT A RESULT, INDICATOR OR PERFORMANCE MEASURE?
9. % HS graduates enrolling in college
10. Traffic-related death rate
11. Clean environment
12. Air pollutants in parts per million
13. % participating cities fixing treatment plants
14. All children eat healthy
15. % students eligible for free lunch participating in free lunch
16
16
POPULATIONACCOUNTABILITY
POPULATIONACCOUNTABILITY
For Whole PopulationsFor Whole Populationsin a Geographic Areain a Geographic Area
17
17
17
MarylandResults for Child Well-Being
● Babies born healthy
● Healthy children
● Children enter school ready to learn
● Children successful in school
● Children safe in their families and communities
● Stable and economically independent families
● Communities that support family life
18
18
18
Connecticut Results StatementsConnecticut Results Statements
• A clean and healthy Long IslandA clean and healthy Long Island Sound Sound
• All children healthy and ready toAll children healthy and ready to learn by age 5 learn by age 5
• All children ready by five andAll children ready by five and fine by nine fine by nine
19
19
Leaking Roof(Results thinking in everyday life)
Experience:
Measure:
Story behind the baseline (causes):
Partners:
What Works:
Action Plan:
Inches of Water
? Fixed
Not OK
Turning the Curve
20
20
20
Seven Population Accountability QuestionsSeven Population Accountability Questions
What are the quality of life conditions we want for the What are the quality of life conditions we want for the children, adults and families who live in our community?children, adults and families who live in our community?
What would these conditions look like if we could see them?What would these conditions look like if we could see them?
How can we measure these conditions?How can we measure these conditions?
How are we doing on the most important of these How are we doing on the most important of these measures?measures?
Who are the partners that have a role to play in doing Who are the partners that have a role to play in doing better?better?
What works to do better, including no-cost and low-cost What works to do better, including no-cost and low-cost ideas?ideas?
What do we propose to do?What do we propose to do?
21
21
“We haven’tgot the money, so we’ve gotto think.”
Lord Rutherford1871 - 1937
22
22
22
Connecticut Population TemplateConnecticut Population Template Quality of Life Result All Connecticut children are healthy and ready for school success at age 5, contributing to a reduction over time in Connecticut’s “achievement gap” at Grade 4. Indicator 1: Infants born at Low Birth Weight (LBW) Indicator 2: Births to Mothers Without a High School Degree
% Births to Mothers without a High School Degree
0
5
10
15
20
25
2000 2001 2002 2003 2004 2005
Priority Distircts Other SRP Balance of Towns
Indicator 3: HUSKY A Enrollment and Participation Rates
Story Behind the Baselines 41,719 babies were born in CT in 2005. About 6,000 (14%) are at risk because their family income is at or below the Federal Poverty Level. About 28% of young children at each age (~12,000 children) are at risk of school un-readiness because their family income is at or below 185% of the Federal Poverty Level. Nearly eight in ten of these “at risk” children (78%) live in just 19 towns, CT’s Priority School Districts. Another 15% live in the balance of School Readiness Towns (39 towns). These 58 towns are listed at the back of this template. The rest of CT’s at risk children, defined (for now) as living in poverty, (7%) live in the remaining 111 communities. Other indicators point to developmental challenges for many of these same children: Seven in ten of all low birth weight babies
(71%) live in the 58 School Readiness towns, and half of them (52%) live in the 19 Priority School Districts. Of note, while the average rate of low birth weight babies is stable (but too high), it is rising among African American families.
About a quarter (23%) of mothers with young
children who live in Priority School Districts have not attained a high school degree, a rate much higher than for other communities in CT. The proportion of mothers without a high school degree is increasing in these 19 towns as well.
Nearly two-thirds (64%) of children from
Priority School District towns enter kindergarten without the pre-literacy skills needed for early school success.
Key Funding Information Total Current Funding 533.3 million Funding Distribution Total Federal Funds 262.9 million Total State Funds 266.3 million Capital Projects Subtotal Other Funding 4.1 million Percent of Total Funding Contracted to Third Parties
Dis tr ibution o f Bir ths and LBW In fan ts , 2003
40%
52%
22%
19%
35%
29%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Tot al B irt hs LB W
P rior it y Distric ts Ot her S R Distr ic ts A ll other Towns
HUSKY A Enrollment and Well-Care Participation, Children Under Six Years
65%
70%
75%
80%
85%
Per
cen
t
Enrollment of Eligible Children At Least 1 Well-care visit
Enrollment ofEligible Children
78% 80% 82% 83%
At Least 1 Well-
care visit
72% 74% 76% 80%
2002 2003 2004 2005
23
23
Criteria for
Choosing Indicators
as Primary vs. Secondary Measures
Communication Power
Proxy Power
Data Power
Does the indicator communicate to a broad range of audiences?
Does the indicator say something of central importance about the result?
Does the indicator bring along the data HERD?
Quality data available on a timely basis.
24
24
Choosing IndicatorsWorksheet
Outcome or Result_______________________
Candidate IndicatorsCommunication
PowerProxyPower
DataPower
H M L
H
Measure 1
Measure 2
Measure 3
Measure 4
Measure 5
Measure 6
Measure 7
Measure 8
HData
Development
Agenda
Safe Community
H M L H M L
H H
H L
25
25
Three Part Indicator List for each Result
Part 1: Primary Indicators
Part 2: Secondary Indicators
Part 3: Data Development Agenda
● 2 or 3 or 4 “Headline” Indicators● What this result “means” to the community● Meets the Public Square Test
● Everything else that’s any good (Nothing is wasted.)● Used later in the story behind the baseline
● New data● Data in need of repair (quality,timeliness etc.)
26
26
The Matter of Baselines
Baselines have two parts: history and forecast
H
M
L
History Forecast
Turning the CurvePoint to Point
OK?
27
27
Performance AccountabilityPerformance Accountability
For Programs, Agencies and Service For Programs, Agencies and Service SystemsSystems
28
28
Results Accountabilityis made up of two parts:
Results Accountabilityis made up of two parts:
Performance Accountabilityabout the well-being of
CLIENT POPULATIONSFor Programs – Agencies – and Service Systems
Population Accountabilityabout the well-being of
WHOLE POPULATIONSFor Communities – Cities – Counties – States - Nations
29
29
“All Performance Measures
that have ever existed
for any program
in the history of the universe
involve answering two sets of
interlocking questions.”
30
30
HowMuchdid we do?
( # )
HowWell
did we do it?
( % )
Quantity Quality
Program Performance Measures
31
31
EffortHow hard did we try?
EffectIs anyone better off?
Program Performance Measures
32
32
Effort
Effect
HowMuch
HowWell
Program Performance Measures
33
33
How much service did we deliver?
Program Performance Measures
How welldid we
deliver it?
How much change / effect
did we produce?
What quality of change / effect
did we produce?
Quantity Quality
E
ffect
Ef
fort
Out
put
Inpu
t
34
34
How much did we do?
Program Performance Measures
How welldid we do it?
Is anyonebetter off?
Quantity Quality
E
ffect
Ef
fort
# %
35
35
How much did we do?
Education
How well did we do it?
Is anyone better off?
Quantity Quality
Effe
ct
E
ffort Number of
studentsStudent-teacher
ratio
Number ofhigh schoolgraduates
Percent ofhigh schoolgraduates
36
36
How much did we do?
Education
How well did we do it?
Is anyone better off?
Quantity Quality
Effe
ct
E
ffort Number of
studentsStudent-teacher
ratio
Percent of 9th graders who
graduate on timeand enter college or
employment after graduation
Number of 9th graders who
graduate on timeand enter college or
employment after graduation
37
37
How much did we do?
Health Practice
How well did we do it?
Is anyone better off?
Number ofpatientstreated
Percent ofpatients treated
in less than1 hour
Incidence ofpreventable
disease(in the practice)
Rate ofpreventable
disease(in the practice)
Quantity Quality
E
ffect
Effo
rt
38
38
How much did we do?
General Motors
How well did we do it?
Is anyone better off?
# of production hrs
# tons of steel
# of cars produced
Employees pervehicle
produced
# of cars sold
$ amount of Profit
$ car value after 2 years
Quantity Quality
Effe
ct
E
ffort
Source: USA Today 9/28/98
% market share
Profit per share
% car value after 2 years
39
39
39
● # of people served
● % participants who got jobs
● staff turnover rate
● # participants who got jobs
● % of children reading at grade level
● cost per unit of service
● # applications processed
● % patients who fully recover
What Kind of PERFORMANCE MEASURE?
Upper Left
Lower Right
Upper Right
Lower Left
Lower Right
Upper Right
Upper Left
Lower Right
40
40
RBA Categories Account for All Performance Measures(in the history of the universe)
Quantity Quality
Efficiency, Admin overhead, Unit costStaffing ratios, Staff turnoverStaff morale, Access, Waiting time, Waiting lists, Worker safety
Customer Satisfaction(quality service delivery& customer benefit)
Cost / Benefit ratioReturn on investment
Client results or client outcomes
EffectivenessValue-addedProductivity
Benefit value
Product Output Impact
Process Input
Effe
ctEf
fort
Cost
TQM
Effectiveness
Efficiency
41
41
42
42
How much did we do?
The Matter of Control
How well did we do it?
Is anyone better off?
Quantity Quality
Effe
ct
E
ffort
LeastControl
PARTNERSHIPS
MostControl
43
43
The Matter of Use
1. Fundamental Purpose is to Improve Performance as a contribution to improving results
2. Avoid the Performance Measurement Equals Punishment Trap
● Acknowledge the experience as real.
● Work to create a healthy organizational environment
● Start small.
● Build bottom-up and top-down simultaneously.
44
44
How much did we do?
Program Performance Measures
How welldid we do it?
Is anyonebetter off?
Quantity Quality
E
ffect
Ef
fort
# %
45
45
45
Connecticut Program TemplateConnecticut Program TemplateConnecticut Appropriations Committee RBA Template
Part II, Program/Agency/System Accountability Summary Special Supplemental Nutrition Program for Women, Infants, and Children (WIC)
Program/Agency/System Purpose The purpose of the WIC Program is to provide supplemental foods, nutrition education and referrals for health and support services to eligible, low income pregnant, postpartum and breastfeeding women, and children 0-5.
Program/Agency/System The Special Supplemental Nutrition Program for Women, Infants and Children (WIC Program) / Department of Public Health, Public Health Initiatives Branch
Performance Measure 1 The incidence of low birth weight (LBW) among infants whose mothers were on the WIC Program for at least 6 months during pregnancy does not exceed 6%.
Performance Measure 2 At least 50% of infants whose mothers were enrolled in the WIC Program during pregnancy breastfeed.
Performance Measure 3 The prevalence of anemia among children enrolled in the WIC Program for at least one year does not exceed 10%.
Key Budget Information Total Current Program Year Funding Admin $9.38 M
Food $27.68 M
Funding as Percent of All Funding for Population Result
Program Funding As Percent of Total Agency Budget
Admin 4% Food 11%
Funding Distribution Total Federal Funds $37.06 M
Total State Funds $0
Capital Projects Subtotal
Other Funding $0
Percent of Total Current Funding Contracted to Third Parties
Admin 77%
Story Behind the Baselines Participation in the WIC Program is associated with Performance Measure 1: a reduction in rates of low birth weight (LBW) and Performance Measure 3: childhood anemia. It also is associated with better cognitive performance. Early and continuous enrollment in the program may result in further improvement in LBW and anemia rates and, hence, school readiness. Performance Measure 2: Children who were breastfed score higher on cognitive and IQ tests at school age. Breastfed infants experience fewer and less severe cases of infectious and non-infectious illnesses. Therefore, children who were breastfed are more likely to be healthy and ready to learn. Turning The Curves: What do you propose to do over the next two years and why? 1. Prepare and implement the USDA Value Enhanced
Nutrition Assessment (VENA) state plan to improve client services; enhance local agency program monitoring to improve program operations, nutrition counseling and breastfeeding promotion.*
2. Contingent upon the receipt of additional funding, conduct formative research to identify why
eligible women do not routinely enroll in WIC during their first trimester of pregnancy and
eligible children leave the program before age 5, and develop a targeted outreach initiative to address. 3. Contingent upon the receipt of additional funding,
increase WIC local agency staffing levels to the state-recommended staffing pattern to enhance outreach activities and nutrition and breastfeeding education.
4. Contingent upon the receipt of additional funding, replicate the existing model breastfeeding peer counseling program in a selected WIC local agency.
*Indicates, low-cost, no-cost action steps, including reallocation of existing resources.
46
46
How Population
&Performance Accountability
FIT TOGETHER
How Population
&Performance Accountability
FIT TOGETHER
47
47
POPULATION ACCOUNTABILITY
Healthy Births Rate of low birth-weight babiesStable Families Rate of child abuse and neglectChildren Succeeding in School Percent graduating from high school on time
Contributionrelationship
Alignmentof measures
Appropriateresponsibility
Child Welfare Program
THE LINKAGE Between POPULATION and PERFORMANCE
CUSTOMEROutcomes
POPULATIONRESULTS
# Foster ChildrenServed
% withMultiple
Placements
# RepeatAbuse/Neglect
% RepeatAbuse/Neglect
PERFORMANCE ACCOUNTABILITY
48
48
Contributionrelationship
Alignmentof measures
Appropriateresponsibility
THE LINKAGE Between POPULATION and PERFORMANCE
POPULATION ACCOUNTABILITY
Healthy Births Rate of low birth-weight babiesChildren Ready for School Percent fully ready per K-entry assessmentSelf-sufficient Families Percent of parents earning a living wage
CUSTOMEROutcomes
# personsreceivingtraining
Unit costper person
trained
# who getliving wage jobs
% who getliving wage jobs
PERFORMANCE ACCOUNTABILITY
POPULATIONRESULTS
Job Training Program
49
49
50
50
50
What is Happening in the Legislature?What is Happening in the Legislature?
Institutionalizing RBA in the legislature:Institutionalizing RBA in the legislature:
• The Appropriations Committee has created a new RBA sub-The Appropriations Committee has created a new RBA sub-committeecommittee
Early Childhood Cabinet must develop an accountability plan Early Childhood Cabinet must develop an accountability plan and make other recommendations for changes necessary to and make other recommendations for changes necessary to ensure coordination, service integration, and accountabilityensure coordination, service integration, and accountability
The 2007 budget requires new and expanded programs to The 2007 budget requires new and expanded programs to report to the General Assembly and OPM using an approved report to the General Assembly and OPM using an approved RBA framework, an effort lead by the Office of Fiscal RBA framework, an effort lead by the Office of Fiscal Analysis (OFA) and OPMAnalysis (OFA) and OPM
The co-chairs and rankings members of the other sub-The co-chairs and rankings members of the other sub-committees of Appropriations are identifying particular committees of Appropriations are identifying particular public policy issues (result statements) for RBA public policy issues (result statements) for RBA development development
All this month, the early childhood agencies that were part All this month, the early childhood agencies that were part of the RBA Phase II initiative last session reported back to of the RBA Phase II initiative last session reported back to the legislature on the progress they have madethe legislature on the progress they have made
51
51
What is Happening at State Agencies and in Communities?What is Happening at State Agencies and in Communities?
The Child Poverty and Prevention Council and The Child Poverty and Prevention Council and the Juvenile Justice Policy and Operating the Juvenile Justice Policy and Operating Coordinating Committee are both using RBA to Coordinating Committee are both using RBA to guide their implementation effortsguide their implementation efforts
Municipalities and non-profit groups, e.g., Municipalities and non-profit groups, e.g., workforce boards, are embracing RBA in areas workforce boards, are embracing RBA in areas beyond early childhoodbeyond early childhood
State agencies will be developing program State agencies will be developing program quality and outcome measures in response to the quality and outcome measures in response to the new Appropriations Committee initiativesnew Appropriations Committee initiatives
They will be passing the requirements along to They will be passing the requirements along to their vendorstheir vendors
52
52
52
What is Happening at State Agencies and in Communities?What is Happening at State Agencies and in Communities?
• Organizations that contract with the State will Organizations that contract with the State will have new data collection and reporting have new data collection and reporting requirements, including outcome measuresrequirements, including outcome measures
• Managing performance will be more Managing performance will be more complicatedcomplicated
• Likely to be greater requirements for Likely to be greater requirements for integration of programs and collaboration with integration of programs and collaboration with other agenciesother agencies
• Opportunity for contractors to have input on Opportunity for contractors to have input on measurementmeasurement
• Opportunity for program improvementOpportunity for program improvement
53
53
53
“If you do what you always did,
you will get what you always got.”
“If you do what you always did,
you will get what you always got.”
Kenneth W. JenkinsPresident, Yonkers NY NAACP
54
54
Thank YouThank You
The Charter Oak Group, LLCThe Charter Oak Group, LLC
www.charteroakgroup.comwww.charteroakgroup.com
Ron SchackRon Schack
ron_schack@yahoo.comron_schack@yahoo.com
(860) 478-7847(860) 478-7847
Bennett PudlinBennett Pudlin
bpudlin@charteroakgroup.combpudlin@charteroakgroup.com
(860) 324-3555(860) 324-3555
55
55
Turn the Curve Exercise: Population Well-being
5 min: Starting Points - timekeeper and reporter - two hats (yours plus partner’s)
5 min: Baseline
- forecast: Where is the trend line going? - turn the curve: Is forecast OK or not OK?
15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes
15 min: What works? (What would it take?) - what could work to do better
- each partner’s contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works
10 min: Report: Convert notes to one page
Two pointers to action
56
56
ONE PAGE Turn the Curve Report
Result: _______________Indicator
(Lay Definition)Indicator Baseline
Story behind the baseline --------------------------- --------------------------- (List as many as needed)
Partners --------------------------- --------------------------- (List as many as needed)
Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost ---------Off the Wall
SharpEdges
4.
top related