2012 cbhc conference panel september 28, 2012 breckenridge, co innovative evaluation: collaborating...
TRANSCRIPT
![Page 1: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/1.jpg)
2012 CBHC Conference PanelSeptember 28, 2012
Breckenridge, CO
Innovative Evaluation: Collaborating to Develop
Population-based Measurement for the Future
![Page 2: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/2.jpg)
CBHC Panel Abstract, Session # 605
This multi-disciplinary panel will discuss how the future direction in behavioral health care informs the need to develop and implement program outcome measures which reflect an integrated and collaborative approach to service delivery and affect positive change throughout Colorado’s community behavioral health system.
![Page 3: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/3.jpg)
CBHC Panelists & Facilitator Michael Allen, LCSW, CAC III, MBA – Vice President, Health
Network/TeleCare, AspenPointe
Sharon Raggio, LPC, LMFT, MBA - Chief Executive Officer, Colorado West, Inc.
Vicki Rodgers, MS, LPC – Vice President, Clinical Systems Administration, Jefferson Center for Mental Health
Richard Swanson, Ph.D., J.D. - Executive Director, Aurora Research Institute
John Mahalik, Ph.D., MPA – Director, Data & Evaluation, Colorado Division of Behavioral Health, Office of Behavioral Health, CDHS
![Page 4: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/4.jpg)
CBHC Panelists’ PresentationsClinical focus:
How do clinical staff know performance measures make a difference for providers and practitioners?
What are behavioral health service providers/clinicians using to monitor client progress?
AspenPointe – Wellness and chronic disease focus
Colorado West – BASIS-24, RCCO focus
Jefferson Center – CCAR, client-directed outcome informed treatment
Aurora Research Institute – Recovery and client treatment rating scales
![Page 5: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/5.jpg)
Past, Present and Future Paradigm shift from outputs and structure to
outcomes and results.Mental health and substance abuse performance indicators
task forcesAccess, customer satisfaction, continuity of care, quality and
appropriateness of care, outcomes (quality of life)
‘Future is now’ with foci on integration, prevention and wellness.Consumers lack information to select providers by quality
and performance.Without objective data to demonstrate value, behavioral
healthcare becomes vulnerable to becoming a commodity purchased solely based on price.
IHI’s ‘triple aim’ is to optimize the health system by accounting for:Experience of the individual; Health of a defined population; Cost for the population.
![Page 6: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/6.jpg)
C-Stat Initiative – What is it? CDHS Director Reggie Bicha strategic initiative for 2012
is a performance-based analysis strategy that will allow CDHS programs and services:To better focus on and improve performance outcomes; To make more informed, collaborative decisions; To align efforts and resources to affect positive change.
Collaborative approach to affect change at every level – in concert with provider leadership, Divisions determine strategies for improvement and implement strategies, while Executive Leadership help to reduce barriers to the Divisions’ success.
Goals are to collect timely data, increase transparency, conduct regular executive meetings to assess the effectiveness of the strategies, and to identify new performance measures, all in support of continuous quality improvement.
![Page 7: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/7.jpg)
DBH, Providers and C-Stat Process C-Stat initiative in developmental phase, as a
collaborative, iterative process.Leadership and Technical Advisory GroupsPerformance Improvement Plans (PIPs)
C-Stat changes through monthly updated data and presentations to CDHS leadership.Access to SUD services, reduction SU, lesser MH severity,
maintenance of housing and employment, reduce SUD and MH drop-out rates,
Benchmarks/goals and CY 2011 statewide averages
Broad sphere of influence with feedback from CDHS, Governor’s Office, OBH, DBH, and community behavior health provider network.
![Page 8: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/8.jpg)
AspenPointe
Michael Allen, LCSW, CAC III, MBA
![Page 9: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/9.jpg)
![Page 10: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/10.jpg)
Data and information sharing will be critical to successful integrated care
It will be expected that providers demonstrate quality outcomes and return-on-investment to funders
Data will be crucial to providing a common language/common lexicon between disparate healthcare systems
Integrated care will be an essential component in Medicaid re-procurement
Looking Through the Data ‘Crystal Ball’…
![Page 11: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/11.jpg)
“Services that engage and support clients in making behavioral changes related to diet, nutrition, smoking cessation, and physical activity should be included in the package of integrated services.”
CBHC, Input for the Design of the Request for Proposals for Behavioral Health Services Contracts, September 7, 2012
How can the Department (HCPF) achieve greater integration of services at the point of care?
![Page 12: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/12.jpg)
Whole Person Wellness
![Page 13: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/13.jpg)
An integrated/whole-person form of disease management
DCM is a confidential, clinically proven program for screening, managing, and supporting people with depression
CDM is a confidential, clinically proven program for screening, managing, and supporting people with chronic conditions and co-occurring mental health diagnoses
Telephonic
Depression Care Management (DCM)/Chronic Disease Management (CDM)
![Page 14: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/14.jpg)
DepressionAsthmaType II DiabetesHeart DiseaseChronic Pain (new)
Conditions Managed
![Page 15: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/15.jpg)
TeleCare Program Goals:
Help clients understand their disease and the impact both physically and emotionally
Educate clients on ways to self-manage their diseaseAssess for mental health needsProvide resources and information to help reduce
barriers to successWith the client leading the way, develop a plan with
achievable, measureable goalsEncourage and support each step of the wayCelebrate each success no matter how small
Focus is on strengths!
![Page 16: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/16.jpg)
Client Identification, Outreach and Engagement
Assessment
Self Management Support
Coordination of Care
Care Manager’s Role
![Page 17: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/17.jpg)
Functional health and well being using the Short Form 12 (SF-12 v2 ®), a 12-question health inventory.
Depression severity using a tool developed for the PCP setting, the Patient Health Questionnaire-9 (PHQ-9).
Condition specific tools using condition-specific tools designed to screen for asthma, diabetes, chronic pain and heart disease and monitor symptom severity.
Global Assessment of Functioning (GAF) recorded “pre-” and “post-” discharge.
Colorado Patient Assessment Record (CCAR) Survey recorded “pre-” and “post-” discharge.
Colorado Health Partnerships (CHP) Adult Outcome Questionnaire recorded “pre-” and “post-” discharge.
Client satisfaction with the program using the AspenPointe TeleCare satisfaction survey.
Evaluation Tools
![Page 18: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/18.jpg)
DIABETES
![Page 19: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/19.jpg)
Program ParticipantsInformation was analyzed for 40 clients participating in the AspenPointe TeleCare Diabetes Care Management Program. The information in this report represents approximately one year of program participation.
![Page 20: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/20.jpg)
Changes in General Physical and Mental HealthChanges in health over the 12 month period were evaluated by the clients’ responses to the SF-12 interview made up of 12 questions related to both general physical and mental health.
SF-12 ResultsNote: Higher Scores Indicate Improvement
** after domain name indicates statistically significant change
0
10
20
30
40
50
60
70
80
90
100
Sco
re
Pre Follow-up
![Page 21: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/21.jpg)
Changes in Self-Care Activities Changes in Diabetes Self-Care Activities after Approximately 12 Months of Program Participation
![Page 22: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/22.jpg)
Responses to Satisfaction ItemsScore mean calculated with 1=Strongly Disagree to 4=Strongly AgreeA higher score indicates stronger agreement with an item or group of items
StatementItem Mean
ScoreStrongly
Agree
Agree
Disagree
StronglyDisagree
Mean for All Items
3.68 69% 31% <1% --
1. I am treated with respect and compassion. 3.77 77% 23% -- --
2. My Care Manager listens to me and understands my situation.
3.66 66% 34% -- --
3. My Care Manager is knowledgeable about my disease and its treatment.
3.66 66% 34% -- --
4. My Care Manager considers any cultural consideration I may have when work with me and making recommendations.
3.77 77% 23% -- --
5. My Care Manager and I discuss when I need to talk with my doctor about my disease.
3.59 59% 41% -- --
6. My Care Manager helps me identify and manage my symptoms.
3.68 68% 32% -- --
7. I am offered telephone appointments at convenient days and times.
3.63 66% 31% 3% --
8. I believe my information was handled in a confidential manner.
3.71 71% 29% -- --
9. I am satisfied with the help/service I received through this Chronic Disease Management program.
3.69 69% 31% -- --
10. I would recommend this Chronic Disease Management program to another person.
3.74 74% 26% -- --
![Page 23: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/23.jpg)
IMPROVED POPULATION HEALTH
Decrease in severity of depression/chronic condition symptoms
Increase in functional health and well-being
Increase in self-care and self-management of condition
IMPROVED PATIENT EXPERIENCE
Excellent scores on Satisfaction Survey
DECREASED PER CAPITA COSTS
TBD = Opportunity!
Outcomes Summary – “Triple Aim”
![Page 24: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/24.jpg)
Contact Information
Michael Allen, LCSW, CACIII, MBA
Vice PresidentHealth Network and TeleCare6208 Lehman DriveColorado Springs, CO 80918Phone (719) [email protected]
![Page 25: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/25.jpg)
Colorado West, Inc.
Sharon Raggio, LPC, LMFT, MBA
![Page 26: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/26.jpg)
Colorado West, Inc. Measurement is important to our futureCW currently measures:
No shows-indication of engagementTime to first appt (intake)-indication of walk in accessTime to first therapy appt (second contact)-used with
productivity measures to determine staffing needsproductivityHospital recidivismQuarterly client satisfactionSuicide ratesBASIS 24
![Page 27: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/27.jpg)
![Page 28: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/28.jpg)
![Page 29: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/29.jpg)
In Development with RCCO
Outreach Specialist for Behavior ChangeCW employee paid by RCCOGoal is to outreach for behavior change, regardless of having a MH diagnosis
Have a 3 level triage tool used to ID need for this level of intensity
Go to people’s homes who are:Discharged from the acute care hospital
Identified by PCP
![Page 30: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/30.jpg)
Continued…..Other elements that the RCCO is tracking.
Cost of clients care pre ICC interventionsCost of client care post ICC interventionsNumber of Hospital admits pre ICC interventions
Number of Hospital admits post ICC interventions
Number of ER services pre ICC interventionsNumber of ER services post ICC interventions
![Page 31: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/31.jpg)
Jefferson Center for Mental Health
Vicki Rodgers, MS, LPC
![Page 32: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/32.jpg)
Data Informed. Results Driven.
![Page 33: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/33.jpg)
We have been busy…Overall, we have been working to improve the
quality of our use of data – including clinical outcomes.
We will know we have achieved ongoing improvement in our quality in use of data when we…Ask early in the decision making process, “What does
data tell us about that?”Use principles of a learning organization in our daily
work and planning.Use data to be accountable for our personal, team, and
organizational outcomes.Use data well for future planning, forecasting, and continuous quality improvement.
![Page 34: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/34.jpg)
Meaningful Change
There are two types of outcome measures that can describe meaningful change in the lives of behavioral health consumers at Jefferson Center:First, goals that are personalized to each
individual’s needs and desires for their life regarding their struggle with a behavioral health condition.
The other type of meaningful change is the kind that is measured across individuals, programs, and systems to provide statistically valid and reliable results about the outcomes that occur as a result of treatment.
![Page 35: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/35.jpg)
Using the CCARSometimes the CCAR gets a bad reputation because it is
required and many staff have not received feedback on the results that they have entered year after year. However, after careful review of results in CCAR change over time, we have found 3 valid and reliable measures that we use to monitor meaningful, statistically significant change the individual experiences and the health of various populations and have made these key performance indicators.CCAR Symptom Severity - Change between time 1 and
time 2 – Quality of service indicator.CCAR Level of Functioning – Change between time 1
and time 2 – Quality of service indicator.CCAR Hope Domain – Change between time 1 and time 2
- Quality of service indicator related to Resilience and Recovery.
![Page 36: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/36.jpg)
![Page 37: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/37.jpg)
CCAR ScoresSome of the good things about using CCAR scores
are that:They can be compared across individuals and the
organization by many different factors, such as, age or payer.
There are many years of data available for comparison, research, and review.
It is information that is collected already.It is in an easy to use a rating scale format and in
some programs we are comparing the client and clinician rating of these items.
Results may be compared between organizations.Clinicians understand these 3 measures and can see
the relevance to their day-to-day work with clients.
![Page 38: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/38.jpg)
Moving on to Client Directed Outcome Informed Treatment (CDOI)Behavioral health outcomes that measure
positive, meaningful change for consumers are those that focus on improving behavioral health and functioning and decreasing health risks and behavioral health symptoms that interfere with the individual’s ability to function in life domains.
Behavioral health outcomes for individuals need to be realistic, obtainable and measured on an on-going basis. In order for these outcomes to measure meaningful change for a consumer, they must include consumer input and involve on-going assessment.
![Page 39: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/39.jpg)
Barry Duncan’s presentations on a Client Directed Outcome Informed approach to treatment and change suggest:“The quality of the patient's participation . . . [emerges] as the
most important determinant of outcome."“The quality of the alliance is a more potent predictor of outcome
than orientation, experience, or professional discipline.”“Feedback improves Outcomes.”“When clients are not benefiting it provides the opportunity to do
your best work. It gives you the possibility of being helpful to everyone.”
“Change happens early.”
…and our Director of Effectiveness, Alan Girard, has posted on the Heart and Soul of Change website that:“CDOI provides a practical and realistic way to privilege the client’s voice, participate in and bear witness to the change process.”
Relevance of CDOI
![Page 40: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/40.jpg)
Every session feedback!CDOI Outcome Measures are done at every
session:Outcome Rating Scale (ORS): Individually,
Interpersonally, Socially, Overall – Client’s rating of their progress in treatment.
Session Rating Scale (SRS): Relationship, Goals & Topics, Approach or Method, Overall – Client’s rating of the alliance with therapist.
The measures are important and the conversations are essential. Change happens as a result of the conversation.
Asking these questions increases the engagement between client and therapist which increases the achievement of meaningful outcomes for the client.
![Page 41: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/41.jpg)
![Page 42: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/42.jpg)
Outcome Rating ScaleBlue line on graph: This line represents client ORS
scores for 10 sessions rating how they feel progress in treatment is going. The circled area indicates that most clients experience
significant change for the better in sessions 1 – 3. This data informs about the importance of fully engaging clients early in the therapeutic process.
Also, a change of 5 or more points between sessions 1 and 10 indicates meaningful change has taken place for the client. We are doing well in this area!
The client sees their graphs each week and can view their progress along the way. Research indicates that a client’s progress happens much more slowly after 10 sessions so our goal is that more than half of clients reach the clinical cutoff by the 10th session. We are achieving this goal.
![Page 43: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/43.jpg)
Session Rating ScaleRed line on graph: This line represents client SRS
scores for 10 sessions rating how they feel the strength of their alliance is with the clinician. The clinician also see the client’s Session Rating Scale
scores and can discuss how the clinician can better engage with the client. We find that this discussion reduces blaming clients for non-compliance, no-shows, and lack of response.
When the client is about done with treatment, their SRS scales may begin to fall.
Clinicians can also compare their client’s ORS and SRS scores to an international database of other clinicians. Trust us, although average is good in comparing scores – none of our clinicians want to be average!
![Page 44: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/44.jpg)
Next Steps:Using these measures and service costs to understand potential
cost reduction for this population or subpopulations.Using these measures for similar groups of people with various
behavioral health diagnoses and physical health diagnoses.Preparing baselines for new information that can be compared
to current CCAR and CDOI measures.Reviewing how to use client input in more meaningful ways to
further refine supports in resilience and recovery.Continue developing meaningful measures for certain
populations of clients, such as, persons 60 and better or persons with criminal justice involvement.
Continue development of comparison of client/clinician ratings on the CCAR items and as key performance indicators.
Develop better methods to measure homelessness, employment, and housing status.
![Page 45: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/45.jpg)
Feel free to give me a call!
Vicki K. Rodgers, MS, LPC
Vice President, Clinical Systems AdministrationJefferson Center for Mental Health4851 IndependenceWheat Ridge, CO [email protected]
![Page 46: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/46.jpg)
Aurora Research Institute
Richard Swanson, Ph.D., J.D.
![Page 47: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/47.jpg)
An Aurora Research Institute Presentation
Recovery and Client Treatment
Ratings
Recovery and Client Treatment
RatingsPRESENTED AT THE
Colorado Behavioral HealthCare Council
September 28, 2012
Richard M. Swanson, Ph.D., J.D.Aurora Research Institute
![Page 48: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/48.jpg)
48
RecoveryRecovery
At the heart of the recovery movement is the idea that instead of focusing on the disease or pathology of (serious mental illness) … emphasis is placed on the potential for growth in the individual.
Patrick McGuire, February, 2000. APA Monitor, 31, No. 2.
People facing life challenges (such as serious illness, trauma, disability, or disadvantage) are resilient and can significantly improve the healing process when they have access to knowledge, self-help resources, skilled professionals, sustaining environment, and social justice.
Courtenay Harding, vision statement of theInstitute for the Study of Human Resilience, Boston University
![Page 49: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/49.jpg)
49
Recovery ModelRecovery Model
New ModelCombination of traditional clinical and
recovery model
Process
Content
Empowerment of client
Client guided
Clinician facilitates
![Page 50: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/50.jpg)
50
Adult, Middle Childhood, and Early Childhood Client Treatment RatingsAdult, Middle Childhood, and Early Childhood Client Treatment Ratings
Client Information
Social Support
Hope
Empowerment
Community Involvement
Overall Recovery Involvement
Overall Symptoms
Family Functioning
The Adult and Child Client Treatment Rating forms are questionnaires that asks consumers and parents to rate their mental health functioning.
There are eight separate sections:
![Page 51: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/51.jpg)
51
Recovery Approach Overview Recovery Approach Overview
Purpose of Client Treatment Ratings Actively involve consumers in their own treatment. Consumers have opportunity to rate domains
(sections) from their perspective. Consumers also rate symptom recovery from their
own perspective.
Improve Therapeutic Experience Therapists want and welcome client input. Client feedback can be taken into account for
treatment planning. Client feedback may enhance rapport between
therapist and client.
![Page 52: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/52.jpg)
52
Social Support Rating ScaleSocial Support Rating Scale
![Page 53: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/53.jpg)
53
Hope Rating ScaleHope Rating Scale
![Page 54: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/54.jpg)
54
Empowerment Rating ScaleEmpowerment Rating Scale
![Page 55: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/55.jpg)
55
Community Involvement Rating ScaleCommunity Involvement Rating Scale
![Page 56: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/56.jpg)
56
Overall Recovery Involvement Rating ScaleOverall Recovery Involvement Rating Scale
![Page 57: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/57.jpg)
57
Overall Symptoms Rating ScaleOverall Symptoms Rating Scale
![Page 58: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/58.jpg)
58
Family Functioning Rating ScaleFamily Functioning Rating Scale
![Page 59: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/59.jpg)
59
Defining Mental Health RecoveryDefining Mental Health Recovery
A journey of healing and transformation
enabling a person with a mental health
problem to find a meaningful life in a
community of his or her choice while
striving to achieve his or her full potential.
![Page 60: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/60.jpg)
60
Client Treatment and CCARRatings of Recovery: PsychometricsClient Treatment and CCARRatings of Recovery: Psychometrics
Reliability Analysis
CTRAdult
CTRMiddle
CTREarly
CTRAll ages
CCARAdult
CCARMiddle
CCAREarly
CCARAll ages
Alpha .83 .81 .77 .82 .91 .90 .86 .90
Items 5 5 5 5 5 5 5 5
N 2710 1090 292 4092 3734 1960 346 6042
![Page 61: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/61.jpg)
61
Client Treatment Ratings of RecoveryClient Treatment Ratings of Recovery
MeanTime 1
MeanTime 2
SignificanceAverage
Change Score
CTR – Social Support 6.44 6.83 .000 .39
CTR – Hope 6.11 6.68 .000 .57
CTR – Empowerment 6.05 6.37 .000 .32
CTR – Community Involvement 4.76 5.29 .000 .53
CTR – Overall Recovery 6.06 6.52 .000 .46
CTR – Overall Symptoms 5.66 6.02 .000 .36
CTR – Family Functioning 6.18 6.49 .000 .31
CTR – Scale Score 29.33 31.66 .000 2.23
Means and Significance Level
N = 440
Modal 6 months
![Page 62: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/62.jpg)
62
CCAR Recovery Outcome DomainsCCAR Recovery Outcome Domains
Means and Significance Level
MeanTime 1
MeanTime 2
SignificanceAverage
Change Score
CCAR – Social Support 6.22 6.48 .000 .26
CCAR – Hope 6.34 6.65 .000 .31
CCAR – Empowerment 6.15 6.39 .000 .23
CCAR – Activity 5.61 5.91 .000 .29
CCAR – Overall Recovery 5.67 6.00 .000 .33
CCAR – Overall Symptoms 5.35 5.59 .000 .24
CCAR – Scale Score 29.99 31.42 .000 1.43
N = 1636
Model 12 months
![Page 63: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/63.jpg)
63
Client Treatment Ratings of RecoveryReliability AnalysesClient Treatment Ratings of RecoveryReliability Analyses
CTRSocial
Support
CTRHope
CTREmpowerment
CTRCommunityInvolvement
CTROverall
Recovery
CTROverall
Symptoms
CTRFamily
Functioning
CTRSocial Support
1
4110
CorrelationSig. (2-tailed)N
CTRHope
.484 **.000
4110
CorrelationSig. (2-tailed)N
CTREmpowerment
.380 **.000
4111
.516 **.000
4112
CorrelationSig. (2-tailed)N
CTR Community Involvement
.440 **.000
4100
.485 **.000
4094
.451 **.000
4093
CorrelationSig. (2-tailed)N
CTROverall Recovery
.425 **.000
4096
.494 **.000
4094
.592 **.000
4094
.522 **.000
4095
CorrelationSig. (2-tailed)N
CTR Overall Symptoms
.295 **.000
4095
.474 **.000
4093
.371 **.000
4092
.381 **.000
4095
.365 **.000
4092
CorrelationSig. (2-tailed)N
CTRFamily Functioning
.529 **.000
4099
.487 **.000
4094
.413 **.000
4092
.474 **.000
4100
.452 **.000
4095
.413 **.000
4095
CorrelationSig. (2-tailed)N
CTRScale Score
.704 **.000
4113
.768 **.000
4112
.765 **.000
4096
.765 **.000
4096
.788 **.000
4097
.490 **.000
4092
.611 **.000
4115
CorrelationSig. (2-tailed)N
** Correlation is significant at the 0.001 level (2-tailed)
Inter-Scale Correlations
![Page 64: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/64.jpg)
64
CCAR Reliability AnalysesCCAR Reliability Analyses
CCAR SocialSupport
CCARHope
CCAR Empowermen
tCCAR Activity
CCAR OverallRecovery
CCARSocial Support
1
6042
CorrelationSig. (2-tailed)N
CCARHope
.722 **.000
6042
CorrelationSig. (2-tailed)N
CCAREmpowerment
.640 **.000
6042
.685 **.000
6042
CorrelationSig. (2-tailed)N
CCARActivity
.761 **.000
6042
.734 **.000
6042
.777 **.000
6042
CorrelationSig. (2-tailed)N
CCAROverall Recovery
.504 **.000
6042
.507 **.000
6042
.553 **.000
6042
.648 **.000
6042
CorrelationSig. (2-tailed)N
CCARScale Score
.856 **.000
6042
.866 **.000
6042
.866 **.000
6042
.921 **.000
6042
.743 **.000
6042
CorrelationSig. (2-tailed)N
** Correlation is significant at the 0.01 level (2-tailed)
Inter-Scale Correlations
![Page 65: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/65.jpg)
65
Client Treatment Ratings of Recoveryby CCAR Validity AnalysisClient Treatment Ratings of Recoveryby CCAR Validity Analysis
** Correlation is significant at the 0.01 level (2-tailed)
* Correlation is significant at the 0.05 level (2-tailed)
CTR SocialSupport
CTRHope
CTR Empowerm
ent
CTR Community Involvemen
t
CTR OverallRecovery
CTR Overall Symptoms
CTR Family Functioning
CTR Scale Score
CCARSocial Support
.177 **.000
3359
.246 **.000
3347
.152 **.000
3350
.197 **.000
3337
.161 **.000
3307
.172 **.000
3319
.196 **.000
3337
.246 **.000
3358
CorrelationSig. (2-tailed)N
CCARHope
.161 **.000
3359
.188 **.000
3347
.147 **.000
3350
.192 **.000
3337
.145 **.000
3307
.151 **.000
3319
.183 **.000
3337
.219 **.000
3358
CorrelationSig. (2-tailed)N
CCAREmpowerment
.177 **.000
3359
.196 **.000
3347
.133 **.000
3350
.244 **.000
3337
.151 **.000
3307
.141 **.000
3319
.190 **.000
3337
.239 **.000
3358
CorrelationSig. (2-tailed)N
CCARActivity
.197 **.000
3359
.234 **.000
3347
.173 **.000
3350
.242 **.000
3337
.181 **.000
3307
.181 **.000
3319
.226 **.000
3337
.271 **.000
3358
CorrelationSig. (2-tailed)N
CCAROverall Recovery
.150 **.000
3359
.216 **.000
3347
.124 **.000
3350
.265 **.000
3337
.138 **.000
3307
.230 **.000
3319
.201 **.000
3337
.222 **.000
3358
CorrelationSig. (2-tailed)N
CCARScale Score
.206 **.000
3359
.257 **.000
3347
.174 **.000
3359
.257 **.000
3337
.185 **.000
3307
.208 **.000
3319
.236 **.000
3337
.285 **.000
3358
CorrelationSig. (2-tailed)N
CCAR Overall Symptom Severity
.248 **.000
3359
.162 **.000
3347
.108 **.000
3359
.177 **.000
3337
.133 **.000
3307
.148 **.000
3319
.215 **.000
3337
.218 **.000
3358
CorrelationSig. (2-tailed)N
![Page 66: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/66.jpg)
66
CCAR and CTR Scores by Division,Team, and TherapistCCAR and CTR Scores by Division,Team, and Therapist
CCAR CTR
Baseline Time 2 Baseline Time 2
Community Involvement 3 7 9 8
Empowerment 4 7 1 6
Hope 6 8 8 9
Overall Recovery Involvement 4 9 9 8
Social Support 4 9 9 9
Family Functioning 7 6
![Page 67: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/67.jpg)
67
CCAR and CTR Scores by Division,Team, and TherapistCCAR and CTR Scores by Division,Team, and Therapist
CCAR CTR
Baseline Time 2 Baseline Time 2
Community Involvement 5 8 7 9
Empowerment 5 7 8 9
Hope 7 7 9 9
Overall Recovery Involvement 5 7 8 9
Social Support 7 6 8 6
Family Functioning 8 9
![Page 68: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/68.jpg)
68
CCAR and CTR Scores by Division and TeamCCAR and CTR Scores by Division and Team
CCAR CTR
Baseline (151)
Time 2 (151)
Baseline (67)
Time 2 (67)
Community Involvement 5 5 4 4
Empowerment 5 5 6 5
Hope 6 6 6 6
Overall Recovery Involvement 4 5 6 6
Social Support 5 5 5 6
Family Functioning 6 5
![Page 69: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/69.jpg)
69
CCAR and CTR Scores by Division and TeamCCAR and CTR Scores by Division and Team
CCAR CTR
Baseline (43)
Time 2 (43)
Baseline (11)
Time 2 (11)
Community Involvement 5 5 5 5
Empowerment 5 6 6 6
Hope 6 6 6 6
Overall Recovery Involvement 5 5 6 6
Social Support 5 6 6 5
Family Functioning 4 6
![Page 70: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/70.jpg)
70
CCAR and CTR Scores by DivisionCCAR and CTR Scores by Division
CCAR CTR
Baseline (217)
Time 2 (217)
Baseline (121)
Time 2 (121)
Community Involvement 5 5 4 5
Empowerment 5 5 6 6
Hope 6 6 6 6
Overall Recovery Involvement 5 5 6 6
Social Support 5 6 6 6
Family Functioning 6 6
![Page 71: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/71.jpg)
71
CCAR and CTR Scores by DivisionCCAR and CTR Scores by Division
CCAR CTR
Baseline (301)
Time 2 (301)
Baseline (162)
Time 2 (162)
Community Involvement 5 6 5 5
Empowerment 6 6 5 6
Hope 6 6 6 7
Overall Recovery Involvement 5 6 6 6
Social Support 6 6 6 7
Family Functioning 6 7
![Page 72: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/72.jpg)
72
CCAR and CTR Scores by DivisionCCAR and CTR Scores by Division
CCAR CTR
Baseline (743)
Time 2 (743)
Baseline (240)
Time 2 (240)
Community Involvement 5 6 4 4
Empowerment 6 6 5 6
Hope 6 6 5 6
Overall Recovery Involvement 5 6 5 6
Social Support 6 6 6 6
Family Functioning 6 6
![Page 73: 2012 CBHC Conference Panel September 28, 2012 Breckenridge, CO Innovative Evaluation: Collaborating to Develop Population-based Measurement for the Future](https://reader038.vdocuments.net/reader038/viewer/2022110323/56649d1f5503460f949f2f98/html5/thumbnails/73.jpg)
Contact MeContact Me
11059 East Bethany Drive, Suite 105 • Aurora, Colorado 80014 • TEL 303-617-2675 • FAX 303-617-2397
Aurora Research Inst i tute
Richard M. Swanson, Ph.D., J.D.
Executive DirectorAurora Research Institute
11059 E. Bethany Dr., Suite 105Aurora, CO 80014(303) 617-2574