fy 2019-20 medi-cal specialty mental health external ... and... · 1. fy 2019-20 medi-cal specialty...

1.

FY 2019-20 MEDI-CAL SPECIALTY MENTAL HEALTH

EXTERNAL QUALITY REVIEW

GLENN MHP FINAL REPORT

Behavioral Health Concepts, Inc. 5901 Christie Avenue, Suite 502 Emeryville, CA 94608

[email protected] www.caleqro.com 855-385-3776

Prepared for:

California Department of

Health Care Services (DHCS)

Review Dates:

August 6, 2019

- 2 -

Glenn County MHP CalEQRO Report Fiscal Year 2019-20

TABLE OF CONTENTS

INTRODUCTION .................................................................................................... 5

MHP Information ............................................................................................................. 5 Validation of Performance Measures .............................................................................. 6 Performance Improvement Projects ................................................................................ 6 MHP Health Information System Capabilities .................................................................. 6 Validation of State and MHP Beneficiary Satisfaction Surveys ....................................... 6

Review of Recommendations and Assessment of MHP Strengths and Opportunities .... 6

PRIOR YEAR REVIEW FINDINGS, FY 2018-19 ..................................................... 8

Status of FY 2018-19 Review of Recommendations ....................................................... 8

Key Recommendations from FY 2018-19 ....................................................................... 8

PERFORMANCE MEASUREMENT ....................................................................... 15

Health Information Portability and Accountability Act (HIPAA) Suppression Disclosure: ...................................................................................................................................... 17

Total Beneficiaries Served ............................................................................................ 18 Penetration Rates and Approved Claims per Beneficiary.............................................. 18 High-Cost Beneficiaries ................................................................................................. 22

Psychiatric Inpatient Utilization ...................................................................................... 22 Post-Psychiatric Inpatient Follow-Up and Rehospitalization .......................................... 23

Diagnostic Categories ................................................................................................... 24

PERFORMANCE IMPROVEMENT PROJECT VALIDATION ................................. 25

Glenn MHP PIPs Identified for Validation ...................................................................... 25 Clinical PIP—Intensive Home-Based Services (IHBS) .................................................. 25 Non-clinical PIP—Timely Access .................................................................................. 26

INFORMATION SYSTEMS REVIEW ..................................................................... 31

Key Information Systems Capabilities Assessment (ISCA) Information Provided by the MHP .............................................................................................................................. 31 Telehealth Services ....................................................................................................... 33

Summary of Technology and Data Analytical Staffing................................................... 34 Current Operations ........................................................................................................ 35 The MHP’s Priorities for the Coming Year ..................................................................... 35 Major Changes since Prior Year ................................................................................... 36

Other Areas for Improvement ........................................................................................ 36 Plans for Information Systems Change ......................................................................... 36 Current EHR Status....................................................................................................... 36

Personal Health Record (PHR) ..................................................................................... 37 Medi-Cal Claims Processing ......................................................................................... 38

CONSUMER AND FAMILY MEMBER FOCUS GROUP(S) .................................... 40

CFM Focus Group One ................................................................................................. 40

- 3 -


PERFORMANCE AND QUALITY MANAGEMENT KEY COMPONENTS............... 42

Access to Care .............................................................................................................. 42 Timeliness of Services .................................................................................................. 43 Quality of Care .............................................................................................................. 44 Beneficiary Progress/Outcomes .................................................................................... 46

Structure and Operations .............................................................................................. 47

SUMMARY OF FINDINGS ..................................................................................... 49

MHP Environment – Changes, Strengths, and Opportunities ....................................... 49

FY 2019-20 Recommendations ..................................................................................... 53

ATTACHMENTS ................................................................................................... 55

Attachment A—On-site Review Agenda ........................................................................ 56

Attachment B—Review Participants .............................................................................. 57 Attachment C—Approved Claims Source Data ............................................................. 60

Attachment D—List of Commonly Used Acronyms ....................................................... 61 Attachment E—PIP Validation Tools ............................................................................. 64

- 4 -


LIST OF TABLES AND FIGURES

Table 1: MHP Medi-Cal Enrollees and Beneficiaries Served, by Race/Ethnicity Table 2: High-Cost Beneficiaries Table 3: MHP Psychiatric Inpatient Utilization Table 4: PIPs Submitted by MHP Table 5: PIP Validation Review Table 6: PIP Validation Review Summary Table 7: Budget Dedicated to Supporting IT Operations Table 8: Distribution of Services, by Type of Provider Table 9: Contract Providers Transmission of Beneficiary Information to MHP EHR System Table 10: Technology Staff Table 11 Data Analytical Staff Table 12: Primary EHR Systems/Applications Table 13: EHR Functionality Table 14: Summary of CY 2019 Short-Doyle/Medi-Cal Claims Table 15: Summary of Top CY 2018 Top Three Reasons for Claim Denial Table 16: Access to Care Components Table 17: Timeliness of Services Components Table 18: Quality of Care Components Table 19: Beneficiary Progress/Outcomes Components Table 20: Structure and Operations Components Figure 1A: Overall Penetration Rates, CY 2016-18 Figure 1B: Overall Approved Claims per Beneficiary, CY 2016-18 Figure 2A: Latino/Hispanic Penetration Rates, CY 2016-18 Figure 2B: Latino/Hispanic Approved Claims per Beneficiary, CY 2016-18 Figure 3A: Foster Children Penetration Rates, CY 2016-18 Figure 3B: Foster Children Average Approved Claims per Beneficiary, CY 2016-18 Figure 4A: 7-day Post-Psychiatric Inpatient Follow-up Figure 4B: 30-day Post-Psychiatric Inpatient Follow-up Figure 5A: Beneficiaries Served, by Diagnostic Categories, CY 2018 Figure 5B: Total Approved Claims by Diagnostic Categories, CY 2018

- 5 -


INTRODUCTION

The United States Department of Health and Human Services (HHS), Centers for Medicare and Medicaid Services (CMS) requires an annual, independent external evaluation of State Medicaid Managed Care Organizations (MCOs) by an External Quality Review Organization (EQRO). External Quality Review (EQR) is the analysis and evaluation by an approved EQRO of aggregate information on quality, timeliness, and access to health care services furnished by Prepaid Inpatient Health Plans (PIHPs) and their contractors to recipients of State Medicaid Managed Care Services. The Code of Federal Regulations (CFR) specifies the requirements for evaluation of Medicaid MCOs (42 CFR, Section 438; Medicaid Program, External Quality Review of Medicaid Managed Care Organizations). These rules require an on-site review or a desk review of each Medi-Cal Mental Health Plan (MHP).

In addition to the Federal Medicaid EQR requirements, the California External Quality Review Organization (CalEQRO) also takes into account the State of California requirements for the MHPs. In compliance with California Senate Bill (SB) 1291 (Section 14717.5 of the Welfare and Institutions Code), the Annual EQR includes specific data for Medi-Cal eligible minor and nonminor dependents in foster care (FC).

The State of California Department of Health Care Services (DHCS) contracts with 56 county Medi-Cal MHPs to provide Medi-Cal covered Specialty Mental Health Services (SMHS) to Medi-Cal beneficiaries under the provisions of Title XIX of the federal Social Security Act.

This report presents the fiscal year (FY) 2019-20 findings of an EQR of the Glenn MHP by the CalEQRO, Behavioral Health Concepts, Inc. (BHC).

The EQR technical report analyzes and aggregates data from the EQR activities as described below:

MHP Information

MHP Size ⎯ Small-rural

MHP Region ⎯ Superior

MHP Location ⎯ Willows

MHP Beneficiaries Served in Calendar Year (CY) 2018 ⎯ 846

MHP Threshold Language(s) ⎯ Spanish

Threshold languages are listed in order beginning with the most to least number of eligibles. This information is obtained from the DHCS/Research and Analytic Studies Division (RASD), Medi-Cal Statistical Brief, September 2016.

- 6 -


Validation of Performance Measures1

Both a statewide annual report and this MHP-specific report present the results of CalEQRO’s validation of eight mandatory performance measures (PMs) as defined by DHCS and other additional PMs defined by CalEQRO.

Performance Improvement Projects2

Each MHP is required to conduct two Performance Improvement Projects (PIPs)—one clinical and one non-clinical—during the 12 months preceding the review. The PIPs are reviewed in detail later in this report.

MHP Health Information System Capabilities3

Using the Information Systems Capabilities Assessment (ISCA) protocol, CalEQRO reviewed and analyzed the extent to which the MHP meets federal data integrity requirements for Health Information Systems (HIS), as identified in 42 CFR §438.242. This evaluation included a review of the MHP’s Electronic Health Records (EHR), Information Technology (IT), claims, outcomes, and other reporting systems and methodologies for calculating PMs.

Validation of State and MHP Beneficiary Satisfaction Surveys

CalEQRO examined available beneficiary satisfaction surveys conducted by DHCS, the MHP, or its subcontractors.

CalEQRO also conducted 90-minute focus groups with beneficiaries and family members to obtain direct qualitative evidence from beneficiaries.

Review of Recommendations and Assessment of MHP

Strengths and Opportunities

The CalEQRO review draws upon prior years’ findings, including sustained strengths, opportunities for improvement, and actions in response to recommendations. Other findings in this report include:

1 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). Validation of

Performance Measures Reported by the MCO: A Mandatory Protocol for External Quality Review (EQR), Protocol

2, Version 2.0, September, 2012. Washington, DC: Author. 2 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). Validating

Performance Improvement Projects: Mandatory Protocol for External Quality Review (EQR), Protocol 3, Version

2.0, September 2012. Washington, DC: Author. 3 Department of Health and Human Services. Centers for Medicare and Medicaid Services (2012). EQR Protocol 1:

Assessment of Compliance with Medicaid Managed Care Regulations: A Mandatory Protocol for External Quality

Review (EQR), Protocol 1, Version 2.0, September 1, 2012. Washington, DC: Author.

- 7 -


• Changes, progress, or milestones in the MHP’s approach to performance management — emphasizing utilization of data, specific reports, and activities designed to manage and improve quality.

• Ratings for key components associated with the following three domains: access, timeliness, and quality. Submitted documentation as well as interviews with a variety of key staff, contracted providers, advisory groups, beneficiaries, and other stakeholders inform the evaluation of the MHP’s performance within these domains. Detailed definitions for each of the review criteria can be found on the CalEQRO website, www.caleqro.com.

http://www.caleqro.com/

- 8 -


PRIOR YEAR REVIEW FINDINGS, FY 2018-19

In this section, the status of last year’s (FY 2018-19) recommendations are presented, as well as changes within the MHP’s environment since its last review.

Status of FY 2018-19 Review of Recommendations

In the FY 2018-19 site review report, the CalEQRO made a number of recommendations for improvements in the MHP’s programmatic and/or operational areas. During the FY 2019-20 site visit, CalEQRO reviewed the status of those FY 2018-19 recommendations with the MHP. The findings are summarized below.

Assignment of Ratings

Met is assigned when the identified issue has been resolved.

Partially Met is assigned when the MHP has either:

• Made clear plans and is in the early stages of initiating activities to address the recommendation; or

• Addressed some but not all aspects of the recommendation or related issues.

Not Met is assigned when the MHP performed no meaningful activities to address the recommendation or associated issues.

Key Recommendations from FY 2018-19

PIP Recommendations

Recommendation 1: Measure results of interventions for both PIPs at least quarterly.

This allows the opportunity to adjust the PIP if barriers to success are discovered.

Status: Met

• The MHP runs data quarterly for both the clinical and non-clinical PIPS as is evidenced upon review of the Quality Improvement Committee (QIC) and System Improvement Committee (SIC) meeting minutes.

Recommendation 2: Clinical PIP: Add at least one new intervention and refine the PIP

to reach conclusion this year.

Status: Partially Met

• The MHP added two interventions to the clinical PIP focused on In-Home Based Services (IHBS). These include use of the Child Adolescent Needs and Strengths (CANS-50) outcome tool (July 2018) and staff training (May 2019); however, the newly implemented interventions do not meet the standard of PIP

- 9 -


interventions. Rather, the outcome tool is a measurement tool for PIP indicators, and staff training is focused on staff deficiencies, not beneficiary outcomes.

Recommendation 3: Non-clinical PIP: The PIP would benefit from beneficiary input.

Status: Met

• Beneficiaries were given the opportunity for input on this PIP through the consumer section of the monthly QIC and SIC meetings, and at Consumer Voice focus groups run monthly at the drop-in centers. Feedback from grievances related to timely access are also reviewed each quarter.

Recommendation 4: Re-measurement of percent of improvement achieved needs to

be done quarterly, at a minimum.

Status: Met

• As previously mentioned, data was run quarterly for both PIPs which also included examining the percent of improvement achieved.

Recommendation 5: Add a new intervention to allow for measurement of changes the

MHP has made in clinical assessments being the first service provided and refine the

PIP to reach conclusion this year.

Status: Met

• The MHP clarified that all clinical assessments occur during first appointment sessions and that this has been the process pre-dating the non-clinical PIP on timely access. All remeasurements in the PIP are compared against the baseline measure of the clinical assessment as the first appointment.

Recommendation 6: Consult with EQRO early and often during the continuation of

both PIPs and the design of a new PIP at the clinical PIP’s conclusion.


• The MHP attended PIP technical assistance webinars and attended the CalQIC session on creating a good PIP. The MHP sought limited technical assistance from the assigned CalEQRO reviewer in advance of the onsite review; however, the MHP would have benefited from ongoing consultation with CalEQRO over the last twelve months on its PIPs.

- 10 -


Access Recommendations

Recommendation 7: Resolve issue of psychiatry capacity through increase in

telehealth and/or recruitment of onsite psychiatrist.

Status: Met

• The MHP added five more telepsychiatry hours for FY 2018-19. The MHP’s non-clinical PIP focused on psychiatry timeliness and reflects an improvement in timeliness to psychiatric services. The MHP plans to evaluate psychiatrist caseloads to determine those who may be ready to step down to the mild-to-moderate level of care; however, a formal evaluation plan and timeline was not available. The MHP also reports that they have not received a plan of correction to increase psychiatry capacity through its network adequacy submissions to DHCS.

Recommendation 8: Continue discussion with Glenn Medical Center and North Valley Indian Health Center regarding the opportunity for health information exchange (HIE) pilot program collaboration.

Status: Met

• Glenn Medical Center (GMC) and North Valley Indian Health (NVIH) have not expressed interest in collaboration for health information exchange. The MHP continues to work closely with both entities on other projects, including an updated memorandum of understanding (MOU) for delivering crisis services at GMC and partnering with the local American Indian Health Facility as a choice of provider for beneficiaries under the new network adequacy standards issued by DHCS.

Timeliness Recommendations

Recommendation 9: Continue to work with Kings View to resolve psychiatry capacity

issue through increase in telehealth. Continue recruitment for onsite psychiatrist.

Status: Met

• Telehealth hours were expanded with five additional hours per month (for a total of 29 hours per week).

• The MHP continues to recruit for an onsite psychiatrist and is considering hiring a locum provider.

- 11 -


Quality Recommendations

Recommendation 10: Design and implement interventions to increase collaboration, transparency, and engagement with beneficiaries, specifically focusing on increasing treatment planning that is in direct collaboration with the beneficiary in the office.

Status: Met

• First appointments are a face-to-face treatment planning session. The MHP also practices the Wellness Team Meeting model for its Full-Service Partnership (FSP) beneficiaries. This model gathers individuals together in support of the beneficiary including service providers, family members, probation, attorneys, and public guardians to develop a treatment plan focused on the beneficiary’s work-life goals and action plan.

• Staff also received training specifically focused on treatment planning (June 2018 and June 2019). Training focused on collaboration, beyond documentation. While there was not an evaluation of the in-person training, the MHP reports a positive impact on its documentation data. Last year, before the training, 15 percent of charts had missing or had late treatment plan signatures, with 18 percent of planned services being provided without a signature. So far this year, 10 percent of charts at chart review had missing or late treatment plan signatures with no planned services being provided without a signature.

Recommendation 11: Research implementation options of standard level of care tool into EHR.

Status: Met

• The MHP researched the Level of Care Utilization System (LOCUS) tool and found it to be an overly extensive tool given the increased documentation and data tracking requirements. Alternatively, the MHP uses the CANS-50 as a level of care indicator for beneficiaries up to age 20. For adults, 21 and older, the MHP uses the Risk and Resiliency Factors Inventory, which produces outcomes, informs treatment planning and quantifies level of care at the start of treatment.

• The MHP amended its medical necessity form to build-in standardized indicators to screen for Specialty Mental Health Services (SMHS) and mild-to-moderate behavioral health levels of care. The amended form incorporates the same indicators used by both managed care plans. Also included in the medical necessity form update is a screening for In-Home Based Services (IHBS) and Intensive Case Coordination (ICC) services for all beneficiaries up to age 20.

- 12 -


Recommendation 12: Resolve issue of extracting co-occurring diagnoses this past year.

Status: Met

• The MHP now runs the co-occurring diagnoses report by diagnostic category rather than subunit of treatment. As a result, the MHP can extract co-occurring diagnoses.

Recommendation 13: Evaluate the low (2.4 percent) number of co-occurring diagnoses reported for year before last for accuracy. If incorrect, resolve data issue.

Status: Met

• The MHP changed its methodology from running the data by beneficiaries open to both mental health and substance use disorder (SUD) services, to looking specifically at diagnostic categories. On the mental health side, they are now able to run reports by diagnosis. On the SUD side, they are evaluating beneficiaries via the Addiction Severity Index (ASI). Referrals between SUD and mental health (collocated) are now being made according to American Society of Addiction Medicine (ASAM) criteria. The MHP now reports a co-occurring diagnoses rate of 16 percent.

Recommendation 14: Design and implement staff training for competency in diagnosing co-occurring mental health and substance abuse disorders.

Status: Met

• In partnership with the MHP’s managed care plan partners, the MHP organized a training on co-occurring disorders held during the January 2019 All Staff meeting. In attendance were counselors, clinicians, and case managers. The training included best practices, the recovery model, criteria from the Diagnostic and Statistical Manual of Mental Disorders (DSM–5) and was followed-up by supervision discussion.

• The need for continuous training was identified. The MHP hired two SUD certified counselors. Of note, the MHP anecdotally found that clinicians were reluctant to give diagnoses related to a substance use disorders as they believed that if beneficiaries were applying for Social Security Disability Insurance (SSDI), it could slow the process down for beneficiaries and delay approvals/payments.

- 13 -


Outcomes Recommendations

Recommendation 15: Research evidence-based practice outcomes tool to implement

for adult beneficiaries.

Status: Not Met

• The MHP elected to continue the use of the Risk and Resiliency Factors Inventory. Given increasing documentation and tracking demands, the MHP is waiting to see what direction DHCS will pursue for adult outcomes before deciding on another tool.

Foster Care Recommendations

Recommendation 16: Assess staffing capacity issues related to insufficient numbers

of personnel.

Status: Met

• The MHP was able to hire and fill open positions this last fiscal year. Currently they do not have any provider deficiencies as identified in the MHP’s network adequacy certification from DHCS.

Recommendation 17: Retrain staff on coding and service delivery types for ICC and

Intensive Home Based Services (IHBS) counts and conduct a post-training assessment.

Status: Met

• The MHP provided a formal training on coding in May 2019 to all mental health staff. The MHP identified this as an ongoing training need and will continue training on this topic in the future. Coding is also addressed in individual supervision with youth and family providers.

Recommendation 18: Design and implement a system that ensures parents and

caretakers have adequate information regarding their children’s medications.

Status: Met

• The MHP reports that parents and caretakers are offered information every time a child is prescribed or has a change in medication. Information is reviewed with the parent or caretaker by the psychiatrist; further, parents/caretakers acknowledge their receipt of this information through the medication consent process. For foster youth not residing with a parent, the MHP’s psychiatrist ensures that a caretaker is present. The MHP has worked closely with its tele-psychiatry program to ensure that medication consents are documented during any change in medication.

- 14 -


Information Systems Recommendations

Recommendation 19: Resolve the issue of extracting co-occurring diagnoses from the EHR.

Status: Met

• This recommendation was addressed previously.

Recommendation 20: Implement action schedules installed in the EHR with

collaboration of EHR vendor.


• The MHP reports that the design of action schedules is cumbersome resulting in significant time constraints on staff and little margin for error correction. The MHP consulted Kings View regarding the issue and were told that the issue will not be modified or considered until the Millennium upgrade is released from Cerner.

• The MHP has opted to continue use of an internal beneficiary list in Excel, with due dates of all documentation for staff, as an alternate to action schedules.

Recommendation 21: Continue work to install Cerner Community Behavioral Health

(CCBH) Promotions 227 and 228.

Status: Met

• As of June 19, 2019, the MHP has installed Promotions 227, 228 and 229 on their EHR and are fully operational.

Carry-over and Follow-up Recommendations from FY 2017-18

Recommendation 22: Continue discussion with GMC and NVIHC regarding the opportunity for HIE pilot program collaboration. (This recommendation is a follow-up to the partially met Recommendation 4, FY 2017-2018.)

Status: Met


Recommendation 23: Design and implement interventions to increase collaboration, transparency, and engagement with beneficiaries, specifically focusing on increasing treatment planning that is in direct collaboration with the beneficiary in the office. (This recommendation is a continuation of part of the met Recommendation 3, FY 2017-2018 and is based on the MHP’s response.)

Status: Met


- 15 -


PERFORMANCE MEASUREMENT

CalEQRO is required to validate the following eight mandatory PMs as defined by DHCS:

• Total beneficiaries served by each county MHP.

• Penetration rates in each county MHP.

• Total costs per beneficiary served by each county MHP.

• High-Cost Beneficiaries (HCBs) incurring $30,000 or higher in approved claims during a CY.

• Count of Therapeutic Behavioral Services (TBS) beneficiaries served compared to the 4 percent Emily Q. Benchmark (not included in MHP reports; this information is included in the Annual Statewide Report submitted to DHCS).

• Total psychiatric inpatient hospital episodes, costs, and average length of stay (LOS).

• Psychiatric inpatient hospital 7-day and 30-day rehospitalization rates.

• Post-psychiatric inpatient hospital 7-day and 30-day SMHS follow-up service rates.

- 16 -


In addition, CalEQRO examines the following SB 1291 PMs (Chapter 844; Statutes of 2016) for each MHP:4

• The number of Medi-Cal eligible minor and nonminor dependents.

• Types of mental health services provided to children, including prevention and treatment services. These types of services may include, but are not limited to, screenings, assessments, home-based mental health services, outpatient services, day treatment services or inpatient services, psychiatric hospitalizations, crisis interventions, case management, and psychotropic medication support services.

• Performance data for Medi-Cal eligible minor and nonminor dependents in FC.

• Utilization data for Medi-Cal eligible minor and nonminor dependents in FC.

4 Public Information Links to SB 1291 and foster care specific data requirements:

1. Senate Bill (SB) 1291 (Chapter 844). This statute would require annual mental health plan reviews to be conducted

by an EQRO and, commencing July 1, 2018, would require those reviews to include specific data for Medi-Cal eligible

minor and nonminor dependents in foster care, including the number of Medi-Cal eligible minor and nonminor

dependents in foster care served each year. The bill would require the department to share data with county boards

of supervisors, including data that will assist in the development of mental health service plans and performance

outcome system data and metrics, as specified. More information can be found at http://www.leginfo.ca.gov/pub/15-

16/bill/sen/sb_1251-1300/sb_1291_bill_20160929_chaptered.pdf

2. EPSDT POS Data Dashboards:

http://www.dhcs.ca.gov/provgovpart/pos/Pages/Performance-Outcomes-System-Reports-and-Measures-

Catalog.aspx

3. Psychotropic Medication and HEDIS Measures:

http://cssr.berkeley.edu/ucb_childwelfare/ReportDefault.aspx includes:

• 5A (1&2) Use of Psychotropic Medications

• 5C Use of Multiple Concurrent Psychotropic Medications

• 5D Ongoing Metabolic Monitoring for Children on Antipsychotic Medications New Measure

http://www.dhcs.ca.gov/dataandstats/Pages/Quality-of-Care-Measures-in-Foster-Care.aspx

4. Assembly Bill (AB) 1299 (Chapter 603; Statues of 2016). This statute pertains to children and youth in foster care and ensures that foster children who are placed outside of their county of original jurisdiction, are able to access mental health services in a timely manner consistent with their individualized strengths and needs and the requirements of EPSDT program standards and requirements. This process is defined as presumptive transfer as it transfers the responsibility to provide or arrange for mental health services to a foster child from the county of original jurisdiction to the county in which the foster child resides. More information can be found at

http://www.leginfo.ca.gov/pub/15-16/bill/asm/ab_1251-1300/ab_1299_bill_20160925_chaptered.pdf

5. Katie A. v. Bonta: The plaintiffs filed a class action suit on July 18, 2002, alleging violations of federal Medicaid laws, the American with Disabilities Act, Section 504 of the Rehabilitation Act and California Government Code Section 11135. The suit sought to improve the provision of mental health and supportive services for children and youth in, or at imminent risk of placement in, foster care in California. More information can be found at https://www.dhcs.ca.gov/Pages/KatieAImplementation.aspx.

http://www.leginfo.ca.gov/pub/15-16/bill/sen/sb_1251-1300/sb_1291_bill_20160929_chaptered.pdf

http://www.leginfo.ca.gov/pub/15-16/bill/sen/sb_1251-1300/sb_1291_bill_20160929_chaptered.pdf

http://www.dhcs.ca.gov/provgovpart/pos/Pages/Performance-Outcomes-System-Reports-and-Measures-Catalog.aspx

http://www.dhcs.ca.gov/provgovpart/pos/Pages/Performance-Outcomes-System-Reports-and-Measures-Catalog.aspx

http://cssr.berkeley.edu/ucb_childwelfare/ReportDefault.aspx

http://www.dhcs.ca.gov/dataandstats/Pages/Quality-of-Care-Measures-in-Foster-Care.aspx

http://www.leginfo.ca.gov/pub/15-16/bill/asm/ab_1251-1300/ab_1299_bill_20160925_chaptered.pdf

https://www.dhcs.ca.gov/Pages/KatieAImplementation.aspx

- 17 -


• Medication monitoring consistent with the child welfare psychotropic medication measures developed by the State Department of Social Services and any Healthcare Effectiveness Data and Information Set (HEDIS) measures related to psychotropic medications, including, but not limited to, the following.

o Follow-Up Care for Children Prescribed Attention Deficit Hyperactivity Disorder Medication (HEDIS ADD).

o Use of Multiple Concurrent Antipsychotics in Children and Adolescents (HEDIS APC).

o Use of First-Line Psychosocial Care for Children and Adolescents on Antipsychotics (HEDIS APP).

• Metabolic Monitoring for Children and Adolescents on Antipsychotics (HEDIS APM).

• Access to, and timeliness of, mental health services, as described in Sections 1300.67.2, 1300.67.2.1, and 1300.67.2.2 of Title 28 of the California Code of Regulations and consistent with Section 438.206 of Title 42 of the Code of Federal Regulations, available to Medi-Cal eligible minor and nonminor dependents in FC.

• Quality of mental health services available to Medi-Cal eligible minor and nonminor dependents in FC.

• Translation and interpretation services, consistent with Section 438.10(c)(4) and (5) of Title 42 of the Code of Federal Regulations and Section 1810.410 of Title 9 of the California Code of Regulations, available to Medi-Cal eligible minor and nonminor dependents in FC.

Health Information Portability and Accountability Act (HIPAA)

Suppression Disclosure:

Values are suppressed to protect confidentiality of the individuals summarized in the data sets when the beneficiary count is less than or equal to 11 (*). Additionally, suppression may be required to prevent calculation of initially suppressed data; corresponding penetration rate percentages (n/a); and cells containing zero, missing data or dollar amounts (-).

- 18 -


Total Beneficiaries Served

Table 1 provides details on beneficiaries served by race/ethnicity.

Penetration Rates and Approved Claims per Beneficiary

The penetration rate is calculated by dividing the number of unduplicated beneficiaries served by the monthly average Medi-Cal enrollee count. The annual average approved claims per beneficiary (ACB) served is calculated by dividing the total annual Medi-Cal approved claim dollars by the unduplicated number of Medi-Cal beneficiaries served during the corresponding year.

CalEQRO has incorporated the Affordable Care Act (ACA) Expansion data in the total Medi-Cal enrollees and beneficiaries served. Attachment C provides further ACA-specific utilization and performance data for CY 2018. See Table C1 for the CY 2018 ACA penetration rate and ACB.

Regarding the calculation of penetration rates, the Glenn MHP uses a different method than that used by CalEQRO.

Race/Ethnicity

Average Monthly

Unduplicated

Medi-Cal

Enrollees

%

Enrollees

Unduplicated

Annual Count

Beneficiaries

Served

% Served

White 4,488 35.2% 461 54.5%

Latino/Hispanic 6,795 53.3% 287 33.9%

African-American 89 0.7% * n/a

Asian/Pacific Islander 433 3.4% * n/a

Native American 259 2.0% 19 2.2%

Other 683 5.4% 50 5.9%

Total 12,746 100% 846 100%

Table 1. Medi-Cal Enrollees and Beneficiaries Served in CY 2018

by Race/Ethnicity

Glenn MHP

The total for Average Monthly Unduplicated Medi-Cal Enrollees is not a direct sum of the averages above

it. The averages are calculated independently.

- 19 -


Figures 1A and 1B show three-year (CY 2016-18) trends of the MHP’s overall penetration rates and ACB, compared to both the statewide average and the average for small-rural MHPs.

CY 2016 CY 2017 CY 2018

Glenn 7.42% 6.58% 6.64%

Small-Rural 7.66% 7.58% 7.71%

State 4.53% 4.52% 4.66%

0.00%

2.00%

4.00%

6.00%

8.00%

10.00%

12.00%

Figure 1A. Overall Penetration Rates Glenn MHP

CY 2016 CY 2017 CY 2018

Glenn $3,751 $3,408 $3,624

Small-Rural $3,845 $3,692 $3,794

State $5,978 $6,170 $6,454

$0

$1,000

$2,000

$3,000

$4,000

$5,000

$6,000

$7,000

Figure 1B. Overall ACB Glenn MHP

- 20 -


Figures 2A and 2B show three-year (CY 2016-18) trends of the MHP’s Latino/Hispanic penetration rates and ACB, compared to both the statewide average and the average for small-rural MHPs.

CY 2016 CY 2017 CY 2018

Glenn 4.67% 3.84% 4.22%

Small-Rural 4.77% 4.54% 4.79%

State 3.51% 3.35% 3.78%

0.00%

2.00%

4.00%

6.00%

8.00%

10.00%

12.00%

Figure 2A. Latino/Hispanic Penetration Rates Glenn MHP

CY 2016 CY 2017 CY 2018

Glenn $3,214 $2,598 $3,333

Small-Rural $3,499 $2,886 $3,146

State $5,588 $5,278 $5,904

$0

$1,000

$2,000

$3,000

$4,000

$5,000

$6,000

$7,000

$8,000

$9,000

$10,000

Figure 2B. Latino/Hispanic ACB Glenn MHP

- 21 -


Figures 3A and 3B show three-year (CY 2016-18) trends of the MHP’s FC penetration rates and ACB, compared to both the statewide average and the average for small-rural MHPs.

CY 2016 CY 2017 CY 2018

Glenn 59.09% 48.15% 46.34%

Small-Rural 39.90% 40.96% 40.58%

State 47.48% 47.28% 48.41%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Figure 3A. FC Penetration Rates Glenn MHP

CY 2016 CY 2017 CY 2018

Glenn $6,528 $6,723 $7,310

Small-Rural $7,290 $6,553 $6,630

State $9,521 $9,962 $9,340

$0

$2,000

$4,000

$6,000

$8,000

$10,000

$12,000

Figure 3B. FC ACB Glenn MHP

- 22 -


High-Cost Beneficiaries

Table 2 provides the three-year summary (CY 2016-18) MHP HCBs and compares the statewide data for HCBs for CY 2018 with the MHP’s data for CY 2018, as well as the prior two years. HCBs in this table are identified as those with approved claims of more than $30,000 in a year.

See Attachment C, Table C2 for the distribution of the MHP beneficiaries served by ACB range for three cost categories: under $20,000; $20,000 to $30,000; and above $30,000.

Psychiatric Inpatient Utilization

Table 3 provides the three-year summary (CY 2016-18) of MHP psychiatric inpatient utilization including beneficiary count, admission count, approved claims, and LOS.

MHP YearHCB

Count

Total

Beneficiary

Count

HCB %

by

Count

Average

Approved

Claims

per HCB

HCB

Total Claims

HCB % by

Total

Claims

Statewide CY 2018 23,164 618,977 3.74% $57,725 $1,337,141,530 33.47%

CY 2018 * 846 n/a $39,863 - n/a

CY 2017 * 842 n/a $40,032 - n/a

CY 2016 * 929 n/a $42,059 - n/a

Table 2. High-Cost Beneficiaries

Glenn MHP

MHP

CY 2018 36 45 10.88 $11,426 $411,351

CY 2017 36 48 14.38 $11,344 $408,385

CY 2016 38 58 10.43 $10,704 $406,761

Table 3. Psychiatric Inpatient Utilization - Glenn MHP

Year

Unique

Beneficiary

Count

Total

Inpatient

Admissions

Average

LOSACB

Total Approved

Claims

- 23 -


Post-Psychiatric Inpatient Follow-Up and Rehospitalization

Figures 4A and 4B show the statewide and MHP 7-day and 30-day post-psychiatric inpatient follow-up and rehospitalization rates for CY 2017 and CY 2018.

Outpatient MHP Outpatient State Rehospitalization MHP Rehospitalization State

CY 2017 59% 36% 3% 3%

CY 2018 47% 32% 0% 3%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Figure 4A. 7-Day Post-Psychiatric Inpatient Follow-up Glenn MHP

Outpatient MHP Outpatient State Rehospitalization MHP Rehospitalization State

CY 2017 76% 54% 6% 7%

CY 2018 75% 48% 3% 6%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Figure 4B. 30-Day Post-Psychiatric Inpatient Follow-up Glenn MHP

- 24 -


Diagnostic Categories

Figures 5A and 5B compare statewide and MHP diagnostic categories by the number of beneficiaries served and total approved claims, respectively, for CY 2018.

The MHP’s self-reported percent of beneficiaries served with co-occurring (i.e., substance abuse and mental health) diagnoses: 16 percent.

Depression Psychosis Disruptive Bipolar Anxiety Adjustment Other Deferred

MHP CY 2018 33% 7% 5% 5% 30% 12% 5% 3%

State CY 2018 28% 16% 9% 8% 14% 8% 14% 5%

0%

5%

10%

15%

20%

25%

30%

35%

Figure 5A. Diagnostic Categories, Beneficiaries Served Glenn MHP

Depression Psychosis Disruptive Bipolar Anxiety Adjustment Other Deferred

MHP CY 2018 28% 13% 8% 7% 28% 8% 7% 0%

State CY 2018 22% 20% 10% 6% 13% 5% 22% 1%

0%

5%

10%

15%

20%

25%

30%

35%

Figure 5B. Diagnostic Categories, Total Approved Claims Glenn MHP

- 25 -


PERFORMANCE IMPROVEMENT PROJECT

VALIDATION

A PIP is defined by CMS as “a project designed to assess and improve processes and outcomes of care that is designed, conducted, and reported in a methodologically sound manner.” CMS’ EQR Protocol 3: Validating Performance Improvement Projects mandates that the EQRO validate one clinical and one non-clinical PIP for each MHP that were initiated, underway, or completed during the reporting year, or featured some combination of these three stages.

Glenn MHP PIPs Identified for Validation

Each MHP is required to conduct two PIPs during the 12 months preceding the review. CalEQRO reviewed two PIPs and validated one PIP, as shown below.

Table 4 lists the findings for each section of the evaluation of the PIPs, as required by the PIP Protocols: Validation of Performance Improvement Projects.5

Clinical PIP—Intensive Home-Based Services (IHBS)

The MHP presented its study question for the clinical PIP as follows:

“Will increasing the delivery of IHBS services to children, youth, and caregivers who have an open Child Welfare Services (CWS) case improve the family’s resiliency and increase the number of children who are reunified with their family, as indicated by an improved score in the Life Domain Functioning section of the CANS-50?”

Date PIP began: July 2017

Projected End date: July 2019

Status of PIP: Inactive, developed in a prior year (not rated)

5 2012 Department of Health and Human Services, Centers for Medicare and Medicaid Service Protocol 3 Version

2.0, September 2012. EQR Protocol 3: Validating Performance Improvement Projects.

Table 4: PIPs Submitted by Glenn MHP

PIPs for Validation

# of PIPs

PIP Titles

Clinical PIP 1 Intensive Home-Based Services (IHBS)

Non-clinical PIP 1 Timely Access

- 26 -


The goal of this PIP, as stated last year, was is to increase the number of IHBS provided to beneficiaries with open CWS cases, increase beneficiary resiliency as evidenced by scoring on the Risk and Resiliency Factors Inventory, and, thereby, promote an environment that supports permanency and/or a stable living situation. The MHP revised its question to clarify that it aimed to increase reunification through increased IBHS, and to rate the success of the process via the CANS-50, in particular the life domain function. The MHP made few changes. Having implemented the CANS-50 in compliance with the State, the MHP named the outcome tool as an intervention. It also added staff training; however, both of its new interventions are not true clinical interventions with a direct impact on beneficiaries. The CANS-50 is a measurement tool, and staff training on codes and modalities is a process to improve quality and the integrity of data entry. Further, it was not clear why the MHP decided that children need more IHBS. The indicators presented need further explanation as to how they measure performance and are relevant to the study question.

Suggestions to improve the PIP: This PIP is not rated and is categorized as having been developed in a prior year. While there were changes that were recommended last year, it is advised that this PIP formally end, and a new one begins. For this PIP, CalEQRO reviewed study question criteria – that study questions need to be measurable and contain quantifiable goals (percentage or number). Also discussed was the need for the MHP to outline why they think that children need more IHBS, and how it relates to the cause of the identified problem. The indicators presented need further explanation as to how they measure performance and are relevant to the study question.

Relevant details of these issues and recommendations are included within the comments found in the PIP validation tool.

The technical assistance (TA) provided to the MHP by CalEQRO identified several steps of a PIP submission that need further detail. The MHP presented improving outcomes through better family functioning as an idea for its next Clinical PIP. CalEQRO reviewed the different phases of PIP development and the need to do adequate background work to set the PIP up for success. CalEQRO reviewed gathering data to define the scope and depth of a problem, and a barrier analysis to determine the cause(s) of the problem. CalEQRO reviewed the connection between identified barriers and selecting appropriate beneficiary centered interventions, which are likely to have the greatest impact. The MHP was encouraged to consult with EQRO early and often during the continuation of this PIP.

Non-clinical PIP—Timely Access

The MHP presented its study question for the non-clinical PIP as follows:

“Will implementing a triage assessment process help high-need beneficiaries with an urgent need access services within five days; ensure beneficiaries with a routine need

- 27 -


for services access services within 10 days; and beneficiaries requesting psychiatric services receive services within 15 days?”

Date PIP began: September 2016

Projected End date: September 2019

Status of PIP: Completed

The goal of this non-clinical PIP was to implement a triage assessment process to help high-need beneficiaries access services as quickly as possible, as well as ensure all beneficiaries have access to services in a timely manner. Annual External Quality Reviews have shown that in recent years Glenn County did not score well on timeliness to mental health and/or psychiatric services. The MHP identified this as an important area for improvement to ensure proper and consistent access to care for their beneficiaries.

In the initial year of the PIP, the MHP used 7, 14, and 21 days for prioritizing referrals for urgent, moderate, and routine beneficiaries, and have since updated the standards to include the Final Rule timeliness regulations of 15 business to psychiatric and 10 business days to mental health services.

The MHP initially developed interventions to directly impact the access processes and improve timeliness to services for beneficiaries. For this year, the MHP added the expansion of available appointments through hiring a dedicated staff person for assessments and adding assessment slots to supervisor schedules. The MHP also implemented a scheduling process which allowed the front office to schedule follow-ups efficiently.

For 5-day urgent requests for service, at baseline, 42.4 percent met the standard. At the time of remeasurement, 57 percent met the standard, representing an improvement of 9.4 percentage points.

Suggestions to improve the PIP: The PIP could have benefited from beneficiary inclusion on the PIP team. Also, re-measurement should occur quarterly at minimum rather than yearly. Although the initial PIP and problem were identified through data, further evaluation as to the related causes and/or barriers was not completed. Although the study question was measurable, its multipart structure was complicated. Simple, straight-forward questions which identify the problem, intervention and outcome measure might prove more answerable.


The TA provided to the MHP by CalEQRO consisted of recommending that the MHP measure results quarterly. Discussion onsite involved the MHP’s plan to end the PIP and begin a new Non-clinical PIP on the topic of improving beneficiary access to Harmony House, the MHP’s wellness center. The MHP was also encouraged to consult with EQRO early and often during the development of new PIPs. Recommendations for

- 28 -


new PIPs include using a simple, straight-forward question which identifies the problem, intervention and outcome measure might prove more answerable; moreover, the new PIP should include more in-depth data evaluation, a barrier analysis, to ensure that interventions are related to problem causes.

Table 5, on the following pages, provides the overall rating for each PIP, based on the ratings: Met (M), Partially Met (PM), Not Met (NM), Not Applicable (NA), Unable to Determine (UTD), or Not Rated (NR).

Table 5: PIP Validation Review

Item Rating

Step PIP Section Validation Item Clinical Non-

Clinical

1 Selected

Study Topics

1.1 Stakeholder input/multi-functional team

NR PM

1.2 Analysis of comprehensive aspects of enrollee needs, care, and services

NR PM

1.3 Broad spectrum of key aspects of enrollee care and services

NR M

1.4 All enrolled populations NR M

2 Study

Question 2.1 Clearly stated NR PM

3 Study

Population

3.1 Clear definition of study population NR M

3.2 Inclusion of the entire study population

NR M

4 Study

Indicators

4.1 Objective, clearly defined, measurable indicators

NR M

4.2 Changes in health states, functional status, enrollee satisfaction, or processes of care

NR M

5 Sampling Methods

5.1 Sampling technique specified true frequency, confidence interval and margin of error

NR NA

5.2 Valid sampling techniques that protected against bias were employed

NR NA

- 29 -



Item Rating


Clinical

5.3 Sample contained sufficient number of enrollees

NR NA

6 Data

Collection Procedures

6.1 Clear specification of data NR M

6.2 Clear specification of sources of data

NR M

6.3 Systematic collection of reliable and valid data for the study population

NR M

6.4 Plan for consistent and accurate data collection

NR M

6.5 Prospective data analysis plan including contingencies

NR PM

6.6 Qualified data collection personnel NR PM

7 Assess

Improvement Strategies

7.1 Reasonable interventions were undertaken to address causes/barriers

NR PM

8

Review Data Analysis and Interpretation

of Study Results

8.1 Analysis of findings performed according to data analysis plan

NR M

8.2 PIP results and findings presented clearly and accurately

NR PM

8.3 Threats to comparability, internal and external validity

NR M

8.4 Interpretation of results indicating the success of the PIP and follow-up

NR M

9 Validity of

Improvement

9.1 Consistent methodology throughout the study

NR M

9.2 Documented, quantitative improvement in processes or outcomes of care

NR M

9.3 Improvement in performance linked to the PIP

NR PM

- 30 -



Item Rating


Clinical

9.4 Statistical evidence of true improvement

NR NM

9.5 Sustained improvement demonstrated through repeated measures

NR NM

Table 6 provides a summary of the PIP validation review.

Table 6: PIP Validation Review Summary

Summary Totals for PIP Validation Clinical PIP Non-clinical

PIP

Number Met 0 15

Number Partially Met 0 8

Number Not Met 0 2

Unable to Determine 0 0

Number Applicable (AP)

(Maximum = 28 with Sampling; 25 without Sampling) 28 25

Overall PIP Ratings ((#M*2) +(#PM))/(AP*2) 0% 76%

- 31 -


INFORMATION SYSTEMS REVIEW

Understanding the capabilities of an MHP’s information system is essential to evaluating its capacity to manage the health care of its beneficiaries. CalEQRO used the written response to standard questions posed in the California-specific ISCA, additional documents submitted by the MHP, and information gathered in interviews to complete the information systems evaluation.

Key Information Systems Capabilities Assessment (ISCA)

Information Provided by the MHP

The following information is self-reported by the MHP through the ISCA and/or the site review.

Table 7 shows the percentage of MHP budget dedicated to supporting IT operations,

including hardware, network, software license, and IT staff for the past four-year period.

For comparative purposes, we have included similar size MHPs and statewide average

IT budgets per year for prior three-year periods.

Table 7: Budget Dedicated to Supporting IT Operations

FY 2019-20 FY 2018-19 FY 2017-18 FY 2016-17

Glenn 8.16% 4.42% 2.85% 2.78%

Small-Rural MHPs N/A 4.20% 3.70% 3.80%

Statewide N/A 3.40% 3.30% 3.40%

• The MHP IT budget has increased incrementally over the past four years.

The budget determination process for information system operations is:

☐ Under MHP control

☐ Allocated to or managed by another County department

☒ Combination of MHP control and another County department or Agency

- 32 -


Table 8 shows the percentage of services provided by type of service provider.

Table 8: Distribution of Services, by Type of Provider

Type of Provider Distribution

County-operated/staffed clinics 97.29%

Contract providers 2.71%

Network providers 0.00%

Total 100%*

*Percentages may not add up to 100 percent due to rounding.

Table 9 identifies methods available for contract providers to submit beneficiary clinical

and demographic data; practice management and service information; and transactions

to the MHP’s EHR system, by type of input methods.

Table 9: Contract Providers Transmission of Beneficiary Information to MHP EHR System

Type of Input Method Percent

Used Frequency

Direct data entry into MHP EHR system by contract provider staff

0% Not used

Electronic data interchange (EDI) uses standardized electronic message format to exchange beneficiary information between contract provider EHR systems and MHP EHR system

0% Not used

Electronic batch files submitted to MHP for further processing and uploaded into MHP EHR system

0% Not used

Electronic files/documents securely emailed to MHP for processing or data entry input into EHR system

50% Weekly

Paper documents submitted to MHP for data entry input by MHP staff into EHR system

50% Daily

Health Information Exchange (HIE) securely share beneficiary medical information from contractor EHR system to MHP EHR system and return message or medical information to contractor EHR

0% Not used

- 33 -


• MHP does not utilize contract providers that serve beneficiaries within county service area to provide outpatient services locally in a clinic/program setting.

Telehealth Services

MHP currently provides services to beneficiaries using a telehealth application:

☒ Yes ☐ No ☐ In pilot phase

• Number of county-operated sites currently operational: Two

Identify primary reason(s) for using telehealth as a service extender (check all that apply):

☒ Hiring healthcare professional staff locally is difficult

☐ For linguistic capacity or expansion

☐ To serve outlying areas within the county

☐ To serve beneficiaries temporarily residing outside the county

☐ To serve special populations (i.e. children/youth or older adult)

☐ To reduce travel time for healthcare professional staff

☐ To reduce travel time for beneficiaries

• Telehealth services are available with English and Spanish at two sites, Willows and Orland.

• Of the 1338 total telehealth services provided over a 12 month period, approximately nine telehealth sessions were conducted in Spanish and Hmong. The MHP’s capacity to serve non-English speaking beneficiaries is limited via telehealth.

• Recently, the MHP has increased telehealth services to 29 hours/week.

• Kings View is the MHP’s telehealth vendor.

- 34 -


Summary of Technology and Data Analytical Staffing

MHP self-reported IT staff changes by full-time equivalents (FTE) since the previous CalEQRO review are shown in Table 10.

Table 10: Technology Staff

Fiscal Year

IT FTEs (Include

Employees and Contractors)

# of New FTEs

# Employees / Contractors Retired,

Transferred, Terminated

Current # Unfilled

Positions

2019-20 2 0 0 0

2018-19 2 0 0 0

2017-18 2 1 1 1

MHP self-reported data analytical staff changes by FTEs since the previous CalEQRO review are shown in Table 11.

Table 11: Data Analytical Staff

Fiscal Year

IT FTEs (Include

Employees and Contractors)

# of New FTEs

# Employees / Contractors Retired,

Transferred, Terminated

Current # Unfilled

Positions

2019-20 8 0 0 0

2018-19 7 1 0 0

2017-18 3 0 0 0

The following should be noted with regard to the above information:

• The two technology FTEs are County IT staff and support county systems like email, they do not support the EHR.

• The eight data analytical FTEs are Quality Improvement (QI) staff who have multiple responsibilities, but all do some reporting for the MHP. The MHP estimated its data analytical staffing to be one FTE.

- 35 -


• Kings View provides data analytics support to the MHP such as producing the monthly penetration rates report.

• IDEA Consulting has also done work for the MHP such as geo-maps.

Current Operations

• The MHP uses CCBH as its EHR with Promotion 229.

• Kings View hosts CCBH as an Application Service Provider (ASP).

• Glenn County’s Parkside facility in Orland was Medi-Cal certified in April 2019. Billing for services from October 2018 to April 2019 were submitted in June 2019.

• Two QI staff are designated CCBH super users and serve as Kings View’s liaison.

• Youth for Change is the MHP’s contract provider.

• The MHP has secured funding for the Cerner Millennium upgrade.

Table 12 lists the primary systems and applications the MHP uses to conduct business and manage operations. These systems support data collection and storage; provide EHR functionality; produce Short-Doyle Medi-Cal (SDMC) and other third-party claims; track revenue; perform managed care activities; and provide information for analyses and reporting.

Table 12: Primary EHR Systems/Applications

System/Application Function Vendor/Supplier Years Used

Operated By

CCBH EHR Cerner 7 Kings View

The MHP’s Priorities for the Coming Year

• Network Adequacy geo-mapping.

• Client and Service Information (CSI) system for timeliness implementation and reporting.

• Create CANS-50 reports.

• Create data dashboards.

- 36 -


Major Changes since Prior Year

• CCBH was updated from Promotion 226 to 229.

• Completed CANS-50 and Family Advocacy and Support Tool (FAST) reporting.

• Developed Network Adequacy reporting.

• Built a new medical necessity form in the EHR that indicates which level of care (specialty mental health or mild to moderate) a beneficiary should be referred to.

• Developed a new Brief Assessment form in the EHR.

• Updated an Access Contact Log to support new CSI data requirements and to track more meaningful data.

Other Areas for Improvement

• The MHP reported 16 percent of active beneficiaries have co-occurring mental health and substance use diagnoses.

• The MHP initiated discussions with the Glenn Medical Center and North Valley Indian Health Center to set up a Health Information Exchange pilot; however, both providers have expressed no interest in the project.

Plans for Information Systems Change

• The MHP is in discussion with Kings View about upgrading to Cerner Millennium.

Current EHR Status

Table 13 summarizes the ratings given to the MHP for EHR functionality.

Table 13: EHR Functionality

Rating

Function System/Application Present Partially Present

Not Present

Not Rated

Alerts Cerner/CCBH X

Assessments Cerner/CCBH X

Care Coordination X

Document Imaging/ Storage

Cerner/CCBH X

- 37 -


Table 13: EHR Functionality

Rating

Function System/Application Present Partially Present

Not Present

Not Rated

Electronic Signature—MHP Beneficiary

Cerner/CCBH X

Laboratory results (eLab) X

Level of Care/Level of Service

X

Outcomes Cerner/CCBH X

Prescriptions (eRx) Cerner/CCBH X

Progress Notes Cerner/CCBH X

Referral Management X

Treatment Plans Cerner/CCBH X

Summary Totals for EHR Functionality:

FY 2019-20 Summary Totals for EHR Functionality:

8 1 3 0

FY 2018-19 Summary Totals for EHR Functionality*:

7 0 5 0

FY 2017-18 Summary Totals for EHR Functionality:

6 2 1 0

*Two new EHR functionalities were added to the list beginning in FY 2017-18. Progress and issues associated with implementing an EHR over the past year are summarized below:

• The MHP has made small incremental progress towards implementing a fully functional EHR system over the past three years.

Personal Health Record (PHR)

Do beneficiaries have online access to their health records through a PHR feature provided within the EHR, a beneficiary portal, or third-party PHR?

☐ Yes ☐ In Test Phase ☒ No

- 38 -


If no, provide the expected implementation timeline.

☐ Within 6 months ☐ Within the next year

☐ Within the next two years ☒ Longer than 2 years

Medi-Cal Claims Processing MHP performs end-to-end (837/835) claim transaction reconciliations: If yes, product or application:

Dimension Reports

Method used to submit Medicare Part B claims:

☐ Paper ☒ Electronic ☐ Clearinghouse

Table 14 summarizes the MHP’s SDMC claims.

Service

Month

Number

Submitted

Dollars

Billed

Number

Denied

Dollars

Denied

Percent

Denied

Dollars

Adjudicated

Dollars

Approved

TOTAL 14,034 $4,083,334 134 $47,856 1.17% $4,035,478 $2,988,654

JAN18 1,195 $289,289 4 $969 0.33% $288,320 $229,627

FEB18 1,069 $285,161 4 $938 0.33% $284,223 $213,366

MAR18 1,275 $312,261 7 $2,131 0.68% $310,130 $240,749

APR18 1,150 $287,934 1 $382 0.13% $287,552 $226,742

MAY18 1,249 $382,141 3 $572 0.15% $381,569 $289,562

JUN18 1,144 $391,867 6 $1,610 0.41% $390,257 $274,710

JUL18 1,086 $360,394 43 $15,786 4.38% $344,608 $245,762

AUG18 1,242 $383,157 31 $15,506 4.05% $367,651 $259,023

SEP18 1,957 $608,513 18 $5,593 0.92% $602,920 $443,006

OCT18 1,032 $302,536 14 $3,420 1.13% $299,116 $220,066

NOV18 845 $249,872 2 $537 0.21% $249,335 $180,900

DEC18 790 $230,208 1 $413 0.18% $229,795 $165,141

Table 14. Summary of CY 2018 Short Doyle/Medi-Cal Claims

Glenn MHP

Includes services provided during CY 2018 with the most recent DHCS claim processing date of June 7, 2019.

Only reports Short-Doyle/Medi-Cal claim transactions, does not include Inpatient Consolidated IPC hospital claims.

Statewide denial rate for CY 2018 was 3.25 percent.

☒ Yes ☐ No

- 39 -


Table 15 summarizes the top three reasons for claim denial.

Denial Reason DescriptionNumber

Denied

Dollars

Denied

Percent

of Total

Denied

Invalid procedure code and modifier combination OR single service

exceeds maximum minutes per day.6 $18,150 38%

Medicare or Other Health Coverage must be billed before submission

of claim.45 $12,107 25%

Service line is a duplicate and repeat service procedure modifer is not

present.55 $8,925 19%

TOTAL 134 $47,856 N/A

The total denied claims information does not represent a sum of the top three reasons. It is a sum of all denials.

Table 15. Summary of CY 2018 Top Three Reasons for Claim Denial

Glenn MHP

- 40 -


CONSUMER AND FAMILY MEMBER FOCUS

GROUP(S)

CalEQRO conducted one 90-minute focus group with consumers (MHP beneficiaries) and/or their family members during the site review of the MHP. As part of the pre-site planning process, CalEQRO originally requested two focus groups with 10 to 12 participants each, the details of which can be found in each section below. Through collaborative planning, the MHP and CalEQRO mutually decided that one focus group was sufficient.

The consumer and family member (CFM) focus group is an important component of the CalEQRO site review process. Feedback from those who are receiving services provides important information regarding quality, access, timeliness, and outcomes. The focus group questions emphasize the availability of timely access to care, recovery, peer support, cultural competence, improved outcomes, and CFM involvement. CalEQRO provides gift cards to thank the CFMs for their participation.

CFM Focus Group One

CalEQRO requested a culturally diverse group of adult beneficiaries who are mostly new beneficiaries who have initiated/utilized services within the past 12 months. CalEQRO also requested a culturally diverse group of parents/caregivers of child/youth beneficiaries who are mostly new beneficiaries who have initiated/utilized services within the past 12 months. As mentioned above, the review moved forward with one group. Group attendees included both adult male and female clients and parent/caregivers, who were mostly white. The focus group was held at Glenn County Behavioral Health, 1187 E. South St., Orland, CA 95963.

Number of participants: 14

The ten participants who entered services within the past year described their experiences as the following:

• Participants were referred for services through a variety of means including the wellness center, probation, walk-in and self-referral.

• Most participants accessed services in less than two weeks.

• The majority of participants indicated high satisfaction with services.

Participants’ general comments regarding service delivery included the following:

• Overall, participants received weekly or biweekly therapy. Most receive psychiatry services on a monthly basis.

• All participants stated that the frequency of service was sufficient to improve functioning.

- 41 -


• Support groups are available and regularly attended by group members.

• Most participants have a Wellness Recovery Action Plan (WRAP).

• All participants felt that extra services were available if need be and that they could be seen within a day or two if they were in crisis or had an urgent need.

Participants’ recommendations for improving care included the following:

• Increase transportation and drivers.

• Separate transportation for Harmony House and appointments. Being late or missing appointments can sometimes be an issue and the current two drivers drive all over the county and get very busy.

• Include a music program at the Transition Age Youth (TAY) drop-in center.

• Provide same day after-hours urgent care.

• Implement automatic refill system for medications.

• Update the office area so that it feels warm and welcoming, not clinical.

• Provide calming, peaceful music or television for children who are waiting for appointments.

• Include more services for men at Harmony House, especially support groups for homeless men, men in domestic violence situations, and men who are single parents.

• Offer closed groups, i.e. WRAP groups, at the Harmony House campus that will not be interrupted by people who are there for drop-in services only.

Interpreter used for focus group one: No

- 42 -


PERFORMANCE AND QUALITY MANAGEMENT KEY

COMPONENTS

CalEQRO emphasizes the MHP’s use of data to promote quality and improve performance. Components widely recognized as critical to successful performance management include Access to Care, Timeliness of Services, Quality of Care, Beneficiary Progress/Outcomes, and Structure and Operations. The following tables in this section summarize CalEQRO’s findings in each of these areas.

Access to Care

Table 16 lists the components that CalEQRO considers representative of a broad service delivery system that provides access to beneficiaries and family members. An examination of capacity, penetration rates, cultural competency, integration, and collaboration of services with other providers forms the foundation of access to and delivery of quality services.

Table 16: Access to Care Components

Component Maximum Possible

MHP Score

1A Service Access and Availability 14 14

The MHP provides beneficiaries a written Guide to Services. New clinicians attend orientation. The MHP provides beneficiaries a van schedule for the four drivers who provide transportation. The MHP has an IT team which monitors website usage to ensure links work and are current. The MHP’s System-Wide Mental Health. Assessment Response Treatment (SMART) team provides presentations on school violence prevention. This team is a collaboration with law enforcement.

1B Capacity Management 10 10

Last year there were six vacancies in both adult and children’s’ services. The MHP has now been fully staffed for a year; however, recently two therapists left (July 2019). The MHP is now in the process of advertising. To assist with recruitment, the MHP raised clinician salaries and now offers a 10 percent differential for licensure.

1C Integration and Collaboration 24 24

The MHP collaborates with several community-based organizations and agencies including managed care organizations, local churches and Indian Health Service. The MHP also holds outreach game nights locally in Willows and the following unincorporated areas – Orland, Hamilton City, Oak Creek, and Artois.

- 43 -


Timeliness of Services

As shown in Table 17, CalEQRO identifies the following components as necessary for timely access to comprehensive specialty mental health services.

Table 17: Timeliness of Services Components


MHP Score

2A First Offered Appointment 16 16

Last year, overall, 88.7 percent of offered appointments met the 10-day standard (as compared to the year before, 80 percent).

2B Assessment Follow-up and Routine Appointments 8 4

The MHP reported that it tracks and trends assessment follow-up and routine appointments; however, data was not included in the self-assessment.

2C First Offered Psychiatry Appointment 12 10

From FY 2017 to FY 2018, the MHP made improvements to timeliness of first offered psychiatry appointment, 28.8 percent to 60.4 percent; however, the State standard was not met.

The MHP explained that Colusa had a new county counsel who changed how services were authorized for FC. County Counsel revoked CWS deputies’ ability to sign consent. A new county counsel will take over soon and as a result, the MHP will revisit this.

The Plan must offer a psychiatric appointment within 15 business days. The MHP needs to comply with the state timeliness metric as per IN 18-011.

2D Timely Appointments for Urgent Conditions 18 12

Data was not provided on either types of urgent appointments including looking at the

average and median times, percentages of times the standard is met, and the range.

2E Timely Access to Follow-up Appointments after Hospitalization

10 10

MHP beneficiaries experienced an increase in hospitalizations (59 to 75) after the local Carr Fire and Camp Fire. Many residents, including MHP staff, suffered significant trauma and loss. Of 75 hospitalization discharge appointments, 93.5 percent met the 7-day standard.

- 44 -


Table 17: Timeliness of Services Components


MHP Score

2F Tracks and Trends Data on Rehospitalizations 6 6

The hospital readmission rate (within 30 days) was reported as 5.3 percent.

2G Tracks and Trends No-Shows 10 10

The MHP offers a single point of contact for scheduling for first assessment appointments and therapy appointments.

Individual health records contain no-show information. Cancellation type (by staff or beneficiary) is also tracked.

Average no-shows for psychiatrists are 14.3 percent. Average no-shows for clinicians are 11.1 percent. The MHP has a standard of 10 percent for both psychiatrists and clinicians.

Quality of Care

In Table 18, CalEQRO identifies the components of an organization that is dedicated to the overall quality of care. These components ensure that the quality improvement efforts are aligned with the system’s objectives and contributes to meaningful changes in the system to improve beneficiary care characteristics.

Table 18: Quality of Care Components


MHP Score

3A Beneficiary Needs are Matched to the Continuum of Care

12 12

The MHP provides a monthly support group for staff on secondary traumatization. Weekly support for clinical debriefing of difficult cases is also available. This practice improves staff retention and quality of care. The MHP uses an outside therapist for staff support groups.

The MHP held a treatment planning meeting in June 2019 for staff. Training covered how to collaborate with beneficiaries beyond documentation. While this was an in-person training, there was no evaluation. However, during chart review, 15 percent of charts had missing/late treatment plan signatures. So far for this year, chart review showed 10 percent missing/late notes.

- 45 -


The MHP has 15-20 conservatees in psychiatric facilities They were able to step 5-6 beneficiaries down to less intensive services.

3B Quality Improvement Plan 10 10

The MHP provided a QI Work Plan and an annual evaluation. The Work Plan has measurable goals and provided analysis of disparity in services. The MHP receives a Kings View report on penetration rates quarterly.

3C Quality Management Structure 14 14

The MHP’s QIC is headed by a compliance and quality improvement manager and is attended by all QI staff, billing, EHR, analysts and contracts divisions. Peer mentors and coaches also attend the QIC; moreover, they hold focus groups monthly called the “Consumer Voice” meeting. Information is brought back to the QIC. The QIC meets quarterly, along with the SIC.

3D QM Reports Act as a Change Agent in the System 10 10

The MHP’s utilizes data extraction to evaluate timeliness on a quarterly basis. Data guides scheduling and the number of assessment slots offered. Chart review is used to build training opportunities and sometimes make changes to the EHR.

The MHP reported that for billing Medi-Cal dollars and services provided, they have a disallowance rate of 2.9 percent.

3E Medication Management 12 9

The MHP’s psychiatrist provides medication information to parents and caretakers, who also sign a consent and an acknowledgement. For foster youth not residing with a parent, the psychiatrist ensures that a caretaker is present. The degree that collaboration with primary care providers occurs is unclear. The MHP does not track and trend prescribing practices at the aggregate level.

- 46 -


Beneficiary Progress/Outcomes

In Table 19, CalEQRO identifies the components of an organization that is dedicated to beneficiary progress and outcomes as a result of the treatment. These components also include beneficiary perception or satisfaction with treatment and any resulting improvement in beneficiary conditions, as well as capture the MHP’s efforts in supporting its beneficiaries through wellness and recovery.

Table 19: Beneficiary Progress/Outcomes Components


MHP Score

4A Beneficiary Progress 16 16

The MHP utilizes the CANS-50 and Pediatric Symptom Checklist (PSC-35) for children/youth. The MHP also uses an outside consultant to assist with the collection and analysis.

4B Beneficiary Perceptions 10 10

The MHP administers the DHCS-administered Consumer Perception Survey (CPS). The MHP also has regular opportunities for beneficiaries to provide feedback, at the wellness center, QIC focus groups, and those related to the MHSA stakeholder process.

4C Supporting Beneficiaries through Wellness and Recovery

4 4

The MHP’s wellness center, Harmony House, is a peer-run and peer-driven drop-in community wellness center. It is located at 343 Yolo Street in Orland. Drop-in hours are Monday 12:00 p.m. to 4:30 p.m. and Tuesday through Friday 10:00 a.m. to 4:30 p.m. Transportation is provided if needed by scheduling a day in advance. All support groups and events are open to the public, with some offered in Spanish.

The TAY Center is a youth peer-run, youth friendly wellness center environment offering peer support, expressive arts, mentoring and counseling for youth ages 14--24. The center hours are Monday through Thursday, 12:00 p.m. to 6:00 p.m. and Friday 12:00 p.m. to 5:00 p.m. The center is located at 612 4th Street in Orland.

- 47 -


Structure and Operations

In Table 20, CalEQRO identifies the structural and operational components of an organization that is facilitates access, timeliness, quality, and beneficiary outcomes.

Table 20: Structure and Operations Components

Component Quality Rating

5A Capability and Capacity of the MHP 30 28

The MHP’s Parkside location for adult services was certified in April 2019. The MHP received approval retroactive to September 2018.

The MHP is working with managed care providers to provide the required training and education for contracted providers. In that regard, the MHP changed its medical necessity form to closely match managed care forms to facilitate exchange of referrals.

The MHP’s SUD staff attended a Tribal Opioid Coalition. The MHP is in the process of creating a policy and procedure for Narcan.

The MHP held a Mindfulness training in January 2019. All adult staff are certified and using the modality with beneficiaries.

5B Network Adequacy 18 18

The MHP provides 29 hours per week of telehealth. Five hours were added per month starting July 2019. The MHP is recruiting a locum provider one day per week.

5C Subcontracts/Contract Providers 16 N/A

MHP does not have contract providers (less than 3 percent)

5D Stakeholder Engagement 12 12

Staff and focus group participants reported positive communication with MHP management and leadership.

5E Peer Employment 8 6

The MHP has four part-time peer coaches and two peer mentors employed at Harmony House and the TAY center. An established career ladder is in place and employees are aware of how to pursue employment opportunities. Employees start as peer mentors (TAY) or coaches (Harmony House) and can then become case managers. Those with further educational goals can pursue other employment

- 48 -


opportunities. Peer employees are regularly evaluated for job performance on a yearly basis. The MHP does not have supervisory or management positions designated for beneficiaries.

5F Peer-Run Programs 10 9

The MHP has two peer-run peer-driven programs located in Orland. Harmony House is located at 343 Yolo Street and the TAY center at 612 4th street (more information about the centers is provided above in section 4C). Beneficiaries are informed about these programs through informational brochures, flyers, calendars provided by the MHP, case managers, therapists and the Glenn county website. More wellness center locations would potentially help those beneficiaries who are in more isolated areas of the county.

5G Cultural Competency 12 12

The Ethnic Services Committee (ESC) monitors assessment slots for Spanish-speaking beneficiaries and a Spanish-speaking service provider is available. This also applies to telehealth. The ESC meets with monthly and then with clinicians weekly to see what that week’s need is. The ESC was formed to test potential bilingual staff; moreover, this year the ESC became a consultation group for the MHP, acting as a resource for interpreter services. They also review Spanish-language forms. The MHP is hiring a Clinician III to manage the ESC and their work. The MHP has 19 direct service staff, eight direct service speak Spanish.

The MHP uses the Kings View Penetration Report to produce housing fund requests along with transportation logs to identify underserved populations.

The Cultural Competence Committee (CCC) meets monthly, keeps minutes and has CFM representation.

- 49 -


SUMMARY OF FINDINGS

This section summarizes the CalEQRO findings from the FY 2019-20 review of Glenn MHP related to access, timeliness, and quality of care.

MHP Environment – Changes, Strengths, and Opportunities

PIP Status

Clinical PIP Status: Inactive, developed in a prior year (not rated)

Non-clinical PIP Status: Active and ongoing

Access to Care

Changes within the Past Year:

• The Parkside facility in Orland was Medi-Cal certified in April 2019 and it houses the adult unit that offers outpatient services.

• Treatment court started in June 2018 as a pilot and it is now fully operational with a coordinator and program manager.

• In the past year, the MHP added five hours of telehealth services per month to bring the total to 29 hours per week.

• A support group for Lesbian Gay Bisexual Transgender and Queer/Questioning beneficiaries (LGBTQ) was recently started.

• The MHP established an MOU with NVIH for a referral process between the systems and for beneficiary services on a single case-by-case basis.

Strengths:

• The MHP has a van that goes between Willows and Orland and it is supported by four drivers. This van provides transportation to beneficiaries. Case managers also drive beneficiaries to appointments using county vehicles when necessary.

Opportunities for Improvement:

• Stakeholders indicate a need for more drivers and transportation options.

• Stakeholders indicate the need for after-hours urgent care.

• Stakeholders identified an underserved group - men who are homeless who would benefit from increased support services.

- 50 -




• A process was established in March 2019 for reception to schedule all initial face-to-face appointments. This change ensures that all service entry data is tracked.

Strengths:

• Calls are made to beneficiaries to remind them about scheduled appointments.

• Some groups who need more timely access to treatment (e.g. beneficiaries released from jail) have priority to telehealth services.


• The MHP did not provide information on timeliness of assessment follow-up and routine appointments.

• 60.4 percent of the MHP's first offered psychiatry appointments meet the 15-day standard.

• The MHP did not provide data related to the tracking of timely appointments for urgent conditions.

Quality of Care


• In January 2019, adult services clinical staff were trained and certified on mindfulness, an evidence-based practice.

• In May 2019, providers received coding training on service delivery types ICC and IHBS.

• In June 2019, clinical staff were trained on how to build treatment plans.

Strengths:

• Using an outside consultant to run support groups for staff, the MHP provides ongoing support for secondary traumatization and clinical debriefing of difficult cases to prevent burnout and improve quality of care.


• The MHP does not track and trend prescribing practices at the aggregate level.

- 51 -


Beneficiary Outcomes


• Clinical staff were trained and certified on CANS-50.

Strengths:

• None noted.


• The MHP continues to use the Risk and Resiliency Factors Inventory. While the MHP is electing to not make any changes at this time, and instead is waiting for more information from DHCS, the MHP would benefit from researching evidence-based practice outcomes tools for adults.

• While the MHP provides relevant trainings to its staff, evaluation of the effectiveness of the training is lacking.

Foster Care


• The new county counsel changed how services were authorized for FC.

• County Counsel revoked the child welfare deputy’s ability to sign consent; however, a new County Counsel will be in this role soon. The MHP will be working to restore the right to consent with the new County Counsel.

Strengths:

• The MHP is in the process of standardizing FC referrals and screenings for all beneficiaries up to age 21 for ICC and IHBS services.


• Changes made to consent process for FC children delay assessment and treatment.

Information Systems


• The MHP updated its EHR promotion and added forms to support medical necessity determination and beneficiary brief assessment.

• With support from Kings View, the MHP was able to meet Network Adequacy and CANS-50 reporting requirements.

- 52 -


Strengths:

• The MHP has steadily increased its IT budget since 2016-17 to support EHR operations and DHCS requirements such as the implementation of CANS-50, PSC-35, Network Adequacy reporting and CSI timeliness reporting. The MHP dedicated 2.78 percent of its total budget in FY 2016-17 for IT. Subsequent year-over-year increases have brought the IT budget to 8.16 percent of the total budget for FY 2019-20.


• The MHP is aware co-occurring diagnoses are under-reported. Clinical staff received training in January 2019 on how to document co-occurrence. The MHP also corrected where co-occurrence data should be extracted from the EHR.



• The MHP had discussions with GMC and NVIHC regarding a HIE pilot but both providers expressed no interest in the project.

• The MHP filled six clinical staff vacancies in both adult and children services. They were fully staffed for one year, but in the week prior to the review two staff left.

• This last year, the MHP raised clinician salaries and added a 10 percent differential when interns become licensed.

Strengths:

• The MHP is continuing efforts within the county to co-locate Children’s Mental Health and CWS, as well as other agency partners (i.e., eligibility and probation).


• The MHP does not have supervisory or management positions designated for beneficiaries.

• Wellness center locations do not include the more rural areas of the county.

- 53 -


FY 2019-20 Recommendations

PIP Status

1. Title 42, CFR, §438.330 requires two PIPs; the MHP is urged to meet this requirement going forward.

2. The MHP added interventions to its clinical PIP; however, interventions were not beneficiary focused. For multi-year PIPs, a new intervention needs to be included. (This recommendation is a carry-over from FY 2018-19.)

3. The MHP did consult on occasion with CalEQRO; however, the MHP would have benefited from ongoing consultation with CalEQRO for PIP improvement as well as new PIP development. (This recommendation is a carry-over from FY 2018-19.)

Access to Care

4. Evaluate current transportation resources to determine any gaps. Implement a plan to improve identified gaps.

5. Evaluate current after-hours care options to determine areas for improvement and/or expansion. Implement a plan to remedy shortfalls.

6. Perform or access a needs assessment for the local homeless population. Provide outreach and supportive services as identified through the needs assessment.


7. Track and trend timeliness of assessment follow-up and routine appointments.

8. The MHP must offer a psychiatric appointment within 15 business days. The MHP should comply with the state timeliness metric as per IN 18-011.

9. Track and trend timeliness of urgent appointments in hours, 48 for those who do not need preauthorization and 96 hours for those requiring preauthorization.

Quality of Care

10. Track and trend prescribing practices and medication on an aggregate level.

Beneficiary Outcomes

11. Research evidence-based practice outcomes tools to consider implementing for adult beneficiaries. (This recommendation is a carry-over from FY 2018-19.)

12. Implement an evaluation for each staff training to determine the effectiveness and usefulness of training topics (i.e., pre/post training surveys).

- 54 -


Foster Care

13. Collaborate and explore consent options with county counsel to develop a consent process that eliminates the delay in accessing treatment.

Information Systems

14. Conduct periodic chart reviews on dually diagnosed beneficiaries against data reports to assess if the co-occurring under-reporting problem has been rectified.


15. Expand management opportunities for peer employees and incorporate management/supervisory position within career ladder for beneficiaries.

16. Investigate the potential benefit and need for wellness centers in more rural areas of the county. Where appropriate develop a plan and timeline for wellness center site expansion.

- 55 -


ATTACHMENTS

Attachment A: On-site Review Agenda Attachment B: On-site Review Participants Attachment C: Approved Claims Source Data Attachment D: List of Commonly Used Acronyms in EQRO Reports Attachment F: PIP Validation Tools

- 56 -


Attachment A—On-site Review Agenda

The following sessions were held during the MHP on-site review, either individually or in combination with other sessions.

Table A1—EQRO Review Sessions - Glenn MHP

Opening Session – Changes in the past year; current initiatives; and status of previous year’s recommendations

Use of Data to Support Program Operations

Cultural Competence, Disparities and Performance Measures

Timeliness Performance Measures/Timeliness Self-Assessment

Quality Management, Quality Improvement and System-wide Outcomes

Beneficiary Satisfaction and Other Surveys

Performance Improvement Projects

Clinical Line Staff Group Interview

Consumer and Family Member Focus Group(s)

Peer Employee/Parent Partner Group Interview

Validation of Findings for Pathways to Mental Health Services (Katie A./CCR)

Information Systems Billing and Fiscal Interview

Information Systems Capabilities Assessment (ISCA)

Wellness Center Site Visit

Final Questions and Answers - Exit Interview

- 57 -


Attachment B—Review Participants

CalEQRO Reviewers

Cyndi Lancaster – Lead Quality Reviewer Angela Kozak-Embrey – Quality Reviewer Caroline Yip – Information Systems Reviewer Gloria Marrin – Consumer/Family Member Consultant Additional CalEQRO staff members were involved in the review process, assessments, and recommendations. They provided significant contributions to the overall review by participating in both the pre-site and the post-site meetings and in preparing the recommendations within this report.

Sites of MHP Review

MHP Sites

Glenn County Behavioral Health 1187 E. South St. Orland, CA 95963

- 58 -


Table B1—Participants Representing the MHP

Last Name First Name Position Agency

Alvarez Nancy Staff Services Specialist GCBH

Anderson Shelby TAY Peer Mentor GCBH

Arnold Amber Compliance and QI Coordinator GCBH

Baillergeon Roxann Program Manager GCBH

Bobadilla Dago Staff Services Specialist GCBH

Cartes Priscilla Case Manager II GCBH

Christianson Travis Senior MH Counselor HHSA

Crowell Patti Senior Mental Health Counselor GCBH

Cull Lisa Senior Program Coordinator GCBH

Davis Mandi Senior MH Counselor GCBH

Diamond Lisa Harmony House Coach GCBH

Doyle Kristin Senior Program Coordinator GCBH

Enriquez Brenda Administrative Services Analyst GCBH

Flewelling Tara Deputy Probation Off. Probation

Gomez Maria Program Coordinator GCBH

Hallett Joe Compliance and QI Manager GCBH

Hanni Tami Administrative Analyst/HIPAA Officer GCBH

Jordan Michael HH PSE Coach GCBH

Kahl Julie Case Manager II GCBH

Lindsey Amy Behavioral Health Director GCBH

Noel Patrick Senior Program Coordinator GCBH

Priest David HH Case Manager GCBH

Prose Ellen Program Manager GCBH

Randolph Melanie HH Coach GCBH

Ross Cindy Senior Program Coordinator GCBH

- 59 -


Table B1—Participants Representing the MHP

Last Name First Name Position Agency

Rust Stephanie Program Manager II GCBH

Salinas Francisco Tay Intern GCBH

Smith Pamela Senior MH Counselor GCBH

Solis Linyu Senior Mental Health Counselor GCBH

Tarango Leo Peer Mentor GCBH

Thalken Steve Program Manger GCBH

Trevineo-

Tzintzun Ulises Case Manager II

GCBH

Watkins Irvin Case Manager II GCBH

Waygoner Thomas Senior MH Counselor GCBH

Wong Christina Senior Program Coordinator GCBH

- 60 -


Attachment C—Approved Claims Source Data

Approved Claims Summaries are provided separately to the MHP in a HIPAA-compliant manner. Values are suppressed to protect confidentiality of the individuals summarized in the data sets where beneficiary count is less than or equal to 11 (*). Additionally, suppression may be required to prevent calculation of initially suppressed data, corresponding penetration rate percentages (n/a); and cells containing zero, missing data or dollar amounts (-).

Table C1 shows the penetration rate and ACB for just the CY 2016 ACA Penetration Rate and ACB. Starting with CY 2016 performance measures, CalEQRO has incorporated the ACA Expansion data in the total Medi-Cal enrollees and beneficiaries served.

Table C2 shows the distribution of the MHP beneficiaries served by ACB range for three cost categories: under $20,000; $20,000 to $30,000, and above $30,000.

Entity

Average

Monthly ACA

Enrollees

Beneficiaries

Served

Penetration

Rate

Total

Approved

Claims

ACB

Statewide 3,807,829 152,568 4.01% $832,986,475 $5,460

Small-Rural 31,091 2,287 7.36% $6,904,894 $3,019

MHP 2,728 211 7.73% $528,459 $2,505

Table C1. CY 2018 Medi-Cal Expansion (ACA) Penetration Rate and ACB

Glenn MHP

Entity

Average

Monthly ACA

Enrollees

Beneficiaries

Served

Penetration

Rate

Total

Approved

Claims

ACB

Statewide 3,807,829 152,568 4.01% $832,986,475 $5,460

Small-Rural 31,091 2,287 7.36% $6,904,894 $3,019

MHP 2,728 211 7.73% $528,459 $2,505

Table C1. CY 2018 Medi-Cal Expansion (ACA) Penetration Rate and ACB

Glenn MHP

- 61 -


Attachment D—List of Commonly Used Acronyms

Table D1—List of Commonly Used Acronyms

ACA Affordable Care Act

ACL All County Letter

ACT Assertive Community Treatment

ART Aggression Replacement Therapy

CAHPS Consumer Assessment of Healthcare Providers and Systems

CalEQRO California External Quality Review Organization

CARE California Access to Recovery Effort

CBT Cognitive Behavioral Therapy

CDSS California Department of Social Services

CFM Consumer and Family Member

CFR Code of Federal Regulations

CFT Child Family Team

CMS Centers for Medicare and Medicaid Services

CPM Core Practice Model

CPS Child Protective Service

CPS (alt) Consumer Perception Survey (alt)

CSU Crisis Stabilization Unit

CWS Child Welfare Services

CY Calendar Year

DBT Dialectical Behavioral Therapy

DHCS Department of Health Care Services

DPI Department of Program Integrity

DSRIP Delivery System Reform Incentive Payment

EBP Evidence-based Program or Practice

EHR Electronic Health Record

EMR Electronic Medical Record

EPSDT Early and Periodic Screening, Diagnosis, and Treatment

EQR External Quality Review

EQRO External Quality Review Organization

FY Fiscal Year

HCB High-Cost Beneficiary

HIE Health Information Exchange

HIPAA Health Insurance Portability and Accountability Act

ehr Health Information System

HITECH Health Information Technology for Economic and Clinical Health Act

HPSA Health Professional Shortage Area

HRSA Health Resources and Services Administration

IA Inter-Agency Agreement

ICC Intensive Care Coordination

ISCA Information Systems Capabilities Assessment

- 62 -



IHBS Intensive Home Based Services

IT Information Technology

LEA Local Education Agency

LGBTQ Lesbian, Gay, Bisexual, Transgender or Questioning

LOS Length of Stay

LSU Litigation Support Unit

M2M Mild-to-Moderate

MDT Multi-Disciplinary Team

MHBG Mental Health Block Grant

MHFA Mental Health First Aid

MHP Mental Health Plan

MHSA Mental Health Services Act

MHSD Mental Health Services Division (of DHCS)

MHSIP Mental Health Statistics Improvement Project

MHST Mental Health Screening Tool

MHWA Mental Health Wellness Act (SB 82)

MOU Memorandum of Understanding

MRT Moral Reconation Therapy

NP Nurse Practitioner

PA Physician Assistant

PATH Projects for Assistance in Transition from Homelessness

PHI Protected Health Information

PIHP Prepaid Inpatient Health Plan

PIP Performance Improvement Project

PM Performance Measure

QI Quality Improvement

QIC Quality Improvement Committee

RN Registered Nurse

ROI Release of Information

SAR Service Authorization Request

SB Senate Bill

SBIRT Screening, Brief Intervention, and Referral to Treatment

SDMC Short-Doyle Medi-Cal

SELPA Special Education Local Planning Area

SED Seriously Emotionally Disturbed

SMHS Specialty Mental Health Services

SMI Seriously Mentally Ill

SOP Safety Organized Practice

SUD Substance Use Disorders

TAY Transition Age Youth

TBS Therapeutic Behavioral Services

TFC Therapeutic Foster Care

TSA Timeliness Self-Assessment

- 63 -



WET Workforce Education and Training

WRAP Wellness Recovery Action Plan

YSS Youth Satisfaction Survey

YSS-F Youth Satisfaction Survey-Family Version

- 64 -


Attachment E—PIP Validation Tools

PERFORMANCE IMPROVEMENT PROJECT (PIP) VALIDATION WORKSHEET FY 2018-19 CLINICAL PIP

GENERAL INFORMATION

MHP: Glenn

PIP Title: Intensive Home-Based Services (IHBS)

Start Date:07/01/2017

Completion Date: ongoing

Projected Study Period: 24 Months

Completed: Yes ☐ No ☒

Date(s) of On-Site Review:

08/06/2019

Name of Reviewer: Cyndi Lancaster

Status of PIP (Only Active and ongoing, and completed PIPs are rated):

Rated

☐ Active and ongoing (baseline established and interventions started)

☐ Completed since the prior External Quality Review (EQR)

Not rated. Comments provided in the PIP Validation Tool for technical

assistance purposes only.

☐ Concept only, not yet active (interventions not started)

☒ Inactive, developed in a prior year

☐ Submission determined not to be a PIP

☐ No Clinical PIP was submitted

Brief Description of PIP (including goal and what PIP is attempting to accomplish)

The goal of this PIP, as stated last year, was to increase the number of IHBS provided to beneficiaries with open CWS cases, increase beneficiary resiliency as evidenced by scoring on the Risk and Resiliency Factors Inventory, and, thereby, promote an environment that supports permanency and/or stable living situation. The MHP revised its question to clarify that it aimed to increase reunification through increased IBHS, and to rate the success of the process via the CANS-50. The MHP, having implemented the

- 65 -


CANS-50 in compliance with the State, named the outcome tool as an intervention. It also added staff training; however, both of its new interventions are not true clinical interventions with a direct impact on beneficiaries. The CANS-50 is a measurement tool, and staff training on codes and modalities is a process to improve quality and integrity of data entry; further, it was not clear why the MHP feels that children need more IHBS. The indicators presented need further explanation as to how they measure performance and are relevant to the study question.

ACTIVITY 1: ASSESS THE STUDY METHODOLOGY

STEP 1: Review the Selected Study Topic(s)

Component/Standard Score Comments

1.1 Was the PIP topic selected using stakeholder input? Did the MHP develop a multi-functional team compiled of stakeholders invested in this issue?

☐ Met

☒ Partially Met

☐ Not Met

☐ Unable to

Determine

The PIP includes a multifunctional team; however, no beneficiaries were listed as part of the team.

1.2 Was the topic selected through data collection and analysis of comprehensive aspects of enrollee needs, care, and services?

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

Data showed that IHBS services were lower than expected, while CWS reported higher rate of multiple placement and that mental health services were not sufficiently intensive.

Select the category for each PIP:

Clinical:

☐ Prevention of an acute or chronic condition ☐ High volume

services

☒ Care for an acute or chronic condition ☐ High risk

conditions

Non-clinical:

☐ Process of accessing or delivering care

- 66 -


1.3 Did the Plan’s PIP, over time, address a broad spectrum of key aspects of enrollee care and services?

Project must be clearly focused on identifying and correcting deficiencies in care or services, rather than on utilization or cost alone.

☐ Met

☒ Partially Met

☐ Not Met

☐ Unable to

Determine

The MHP is using the CANS-50 as a measurement tool. Accordingly, that the MHP should add a goal percentage or target number to the study question. They did not amend the question with a goal.

1.4 Did the Plan’s PIPs, over time, include all enrolled populations (i.e., did not exclude certain enrollees such as those with special health care needs)?

Demographics:

☐ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language

☐ Other

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

All open CWS cases.

Totals 2 Met 2 Partially Met 0 Not Met 0 UTD

STEP 2: Review the Study Question(s)

2.1 Was the study question(s) stated clearly in writing?

Does the question have a measurable impact for the defined study population?

Include study question as stated in narrative:

“Will increasing the delivery of IHBS services to children, youth, and caregivers who have an open CWS case improve the family’s resiliency and increase the number of children who are reunified with their family, as indicated by an improved score in the Life Domain Functioning section of the California CANS-50?”

☐ Met

☒ Partially Met

☐ Not Met

☐ Unable to

Determine

The MHP is using the CANS-50 as a measurement tool. The MHP did not add a goal percentage or target number to the study question.


- 67 -


STEP 3: Review the Identified Study Population

3.1 Did the Plan clearly define all Medi-Cal enrollees to whom the study question and indicators are relevant?

Demographics:

☒ Age Range ☐ Race/Ethnicity ☐ Gender ☐ Language

☐ Other

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

The PIP includes all children and youth with an open CWS case who meet Medical Necessity for specialty mental health services and who live in Glenn County. The MHP expects to include approximately 30 individuals each year.

3.2 If the study included the entire population, did its data collection approach capture all enrollees to whom the study question applied?

Methods of identifying participants:

☒ Utilization data ☐ Referral ☐ Self-identification

☐ Other: <Text if checked>

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine


- 68 -


STEP 4: Review Selected Study Indicators

4.1 Did the study use objective, clearly defined, measurable indicators?

List indicators:

1. Percent of IHBS beneficiaries who improved their overall Risk and Resiliency Factors (RRF) score

2. Percent of IHBS beneficiaries who received 10 or more hours of IHBS services per year

3. Percent of CWS youth who are open to Glenn County HHS and receiving IHBS services

4. Percent of IHBS beneficiaries with a CWS length of stay of less than 2 years

Added this year:

5. Percent of CWS beneficiaries open to GCBH who received a CANS-50 assessment

6. Percent of IHBS beneficiaries who had an improved score in the Life Domain Functioning section of the CANS-50

☐ Met

☒ Partially Met

☐ Not Met

☐ Unable to

Determine

The indicators presented need further explanation as to how they measure performance and their relevance to the study question.

In Step 4 of the PIP submission tool, there needs to be discussion in the narrative on why the MHP thinks that children need more IHBS.

- 69 -


4.2 Did the indicators measure changes in: health status, functional status, or enrollee satisfaction, or processes of care with strong associations with improved outcomes? All outcomes should be consumer focused.

☐ Health Status ☒ Functional Status

☐ Member Satisfaction ☐ Provider Satisfaction

Are long-term outcomes clearly stated? ☐ Yes ☒ No

Are long-term outcomes implied? ☒ Yes ☐ No

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

It is the MHP’s goal to ensure children and caregivers receive the services they need, to ensure positive outcomes including improving access to IHBS services, improved resiliency factors, reduced out of home placement, and reduced changes in living situation.


STEP 5: Review Sampling Methods

5.1 Did the sampling technique consider and specify the:

a) True (or estimated) frequency of occurrence of the event?

b) Confidence interval to be used?

c) Margin of error that will be acceptable?

☐ Met

☐ Partially Met

☐ Not Met

☒ Not

Applicable

☐ Unable to

Determine

- 70 -


5.2 Were valid sampling techniques that protected against bias employed?

Specify the type of sampling or census used:

☐ Met

☐ Partially Met

☐ Not Met

☒ Not

Applicable

☐ Unable to

Determine

5.3 Did the sample contain a sufficient number of enrollees?

______N of enrollees in sampling frame

______N of sample

______N of participants (i.e. – return rate)

☐ Met

☐ Partially Met

☐ Not Met

☒ Not

Applicable

☐ Unable to

Determine

Totals 0 Met 0 Partially Met 0 Not Met 3 NA 0 UTD

- 71 -


STEP 6: Review Data Collection Procedures

6.1 Did the study design clearly specify the data to be collected?

When a beneficiary is opened to IHBS services, the Eligibility Assessment Form is completed, as well as the Referral Form. Any services the beneficiary receives is then tracked in the EHR and online data system, Cerner.

Data was collected on the services received by beneficiaries with an open CWS case and the number who were referred and met the criteria for specialty mental health services. For those who were open to mental health, services were collected in the EHR.

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

6.2 Did the study design clearly specify the sources of data?

Sources of data:

☐ Member ☐ Claims ☐ Provider

☒ Other: Eligibility Assessment Form and Referral

Form

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

6.3 Did the study design specify a systematic method of collecting valid and reliable data that represents the entire population to which the study’s indicators apply?

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

- 72 -


6.4 Did the instruments used for data collection provide for consistent, accurate data collection over the time periods studied?

Instruments used:

☐ Survey ☐ Medical record abstraction tool

☒ Outcomes tool ☐ Level of Care tools

☒ Other: Assessment and referral forms

☐ Met

☒ Partially Met

☐ Not Met

☐ Unable to

Determine

6.5 Did the study design prospectively specify a data analysis plan?

Did the plan include contingencies for untoward results?

The MHP analyzed data on CWS youth who received services from Glenn County HHS and the number of IHBS services that were received, and if they improved their scores collected on the RRF.

Data was routinely discussed to determine if beneficiaries are receiving the right amount of IHBS services. Mental health staff record the services in the EHR. I.D.E.A. Consulting analyzed the data from the EHR for the performance indicators

☐ Met

☒ Partially Met

☐ Not Met

☐ Unable to

Determine

No contingencies were given for untoward results.

- 73 -


6.6 Were qualified staff and personnel used to collect the data?

Project leader:

Name:

Title:

Role:

Other team members:

Names:

☐ Met

☐ Partially Met

☒ Not Met

☐ Unable to

Determine

Names/titles of personnel used to collect the data were not given.


- 74 -


STEP 7: Assess Improvement Strategies

7.1 Were reasonable interventions undertaken to address causes/barriers identified through data analysis and QI processes undertaken?

Describe Interventions:

1. Hire qualified staff to fill clinical and case management positions to have enough staff to provide these services; utilize additional staff and conduct ongoing recruitment to have enough staff to provide the intensive, wrap around level of services.

2. Implement weekly staff meetings and offer training to increase knowledge and skills in delivering mental health services, quality of service contacts, IHBS services, and improve the frequency of face-to-face services; Train the entire children’s unit on the Core Practice Model, and provide updates in the new edition of the model coming out

3. Obtain a clear vision and communication from the Behavioral Health Director that all treatment staff will attend each Family Team Meeting.

4. Provide monthly data and feedback on the number of IHBS services delivered to CWS families at SIC/QIC and biweekly feedback of the data during staff meetings to identify barriers and celebrate success.

5. Provide staff training on collecting and scoring the RRF and monitor RRF completion monthly.

☐ Met

☐ Partially Met

☒ Not Met

☐ Unable to

Determine

There needs to be at least one intervention that touches the beneficiary and speaks to the clinical outcome for the beneficiary.

The MHP added:

• Implemented and certified staff to utilize the CANS-50 to replace the RRF.

• Trained Mental Health staff on appropriate code usage and modalities of service, including IHBS.

However, these are not interventions. The CANS-50

is a tool for measuring outcomes, and staff training is

for quality control of data entry.

- 75 -


Totals 0 Met 1 Partially Met 0Not Met 0 UTD

STEP 8: Review Data Analysis and Interpretation of Study Results

8.1 Was an analysis of the findings performed according to the data analysis plan?

☐ Met

☒ Partially Met

☐ Not Met

☐ Not

Applicable

☐ Unable to

Determine

MHP staff record results in the EHR and I.D.E.A. Consulting analyzes the data.

8.2 Were the PIP results and findings presented accurately and clearly?

Are tables and figures labeled?

☒ Yes ☐ No

Are they labeled clearly and accurately?

☒ Yes ☐ No

☒ Met

☐ Partially Met

☐ Not Met

☐ Not

Applicable

☐ Unable to

Determine

- 76 -


8.3 Did the analysis identify: initial and repeat measurements, statistical significance, factors that influence comparability of initial and repeat measurements, and factors that threaten internal and external validity?

Indicate the time periods of measurements: ___________________

Indicate the statistical analysis used: _________________________

Indicate the statistical significance level or confidence level if available/known: ____percent ______Unable to determine

☐ Met

☐ Partially Met

☒ Not Met

☐ Not

Applicable

☐ Unable to

Determine

No statistical analysis has been conducted on this data yet.

8.4 Did the analysis of the study data include an interpretation of the extent to which this PIP was successful and recommend any follow-up activities?

Limitations described:

Conclusions regarding the success of the interpretation:

Recommendations for follow-up:

☐ Met

☐ Partially Met

☒ Not Met

☐ Not

Applicable

☐ Unable to

Determine



- 77 -


STEP 9: Assess Whether Improvement is “Real” Improvement

9.1 Was the same methodology as the baseline measurement used when measurement was repeated?

Ask: At what interval(s) was the data measurement

repeated?

Were the same sources of data used?

Did they use the same method of data collection?

Were the same participants examined?

Did they utilize the same measurement tools?

☐ Met

☐ Partially Met

☐ Not Met

☐ Not

Applicable

☐ Unable to

Determine

9.2 Was there any documented, quantitative improvement in processes or outcomes of care?

Was there: ☒ Improvement ☐ Deterioration

Statistical significance: ☐ Yes ☒ No

Clinical significance: ☒ Yes ☐ No

☒ Met

☐ Partially Met

☐ Not Met

☐ Not

Applicable

☐ Unable to

Determine

To summarize, the baseline for this indicator was May 2017- April 2018, and there were 210 requests identified as five-day indicators of which 89 met the timeliness standard (42.4 percent). This improved to 146 out of 256 (57 percent) upon close of the PIP, representing an improvement of 9.4 percent.

9.3 Does the reported improvement in performance have internal validity; i.e., does the improvement in performance appear to be the result of the planned quality improvement intervention?

Degree to which the intervention was the reason for change:

☐ No relevance ☐ Small ☒ Fair ☐ High

☐ Met

☒ Partially Met

☐ Not Met

☐ Not

Applicable

☐ Unable to

Determine

While there was improvement, it is unclear which intervention over the last two years had the greatest impact. Remeasurement after each intervention would provide more information.

- 78 -


9.4 Is there any statistical evidence that any observed performance improvement is true improvement?

☐ Weak ☐ Moderate ☐ Strong

☐ Met

☐ Partially Met

☒ Not Met

☐ Not

Applicable

☐ Unable to

Determine

9.5 Was sustained improvement demonstrated through repeated measurements over comparable time periods?

☐ Met

☐ Partially Met

☒ Not Met

☐ Not

Applicable

☒ Unable to

Determine


ACTIVITY 2: VERIFYING STUDY FINDINGS (OPTIONAL)


Were the initial study findings verified (recalculated by CalEQRO) upon repeat measurement?

☐ Yes

☒ No


- 79 -


ACTIVITY 3: OVERALL VALIDITY AND RELIABILITY OF STUDY RESULTS: SUMMARY OF AGGREGATE VALIDATION FINDINGS

Conclusions:


Recommendations:

Add interventions that are more clearly clinical outcomes of beneficiaries.

State a measurable goal in percent or numbers in the study question.

Analyze the results at least quarterly.

Explain why more children need services and how many more, as per Step 4.2 of the PIP validation.

Indicate the clinical significance to which the indicator measure or relate.

Check one: ☐ High confidence in reported Plan PIP results ☐ Low confidence in reported Plan PIP results

☐ Confidence in reported Plan PIP results ☐ Reported Plan PIP results not credible

☒ Confidence in PIP results cannot be determined at this time

- 80 -


PERFORMANCE IMPROVEMENT PROJECT (PIP) VALIDATION WORKSHEET FY 2018-19 NON-CLINICAL PIP

GENERAL INFORMATION

MHP: Glenn

PIP Title: Timely Access

Start Date: 09/26/16

Completion Date: 09/01/2019

Projected Study Period: 24

Completed: Yes ☒ No ☐

Date(s) of On-Site Review: 08/06/2019

Name of Reviewer: Cyndi Lancaster

Status of PIP (Only Active and ongoing, and completed PIPs are rated):

Rated

☐ Active and ongoing (baseline established and interventions started)

☒ Completed since the prior External Quality Review (EQR)

Not rated. Comments provided in the PIP Validation Tool for technical

assistance purposes only.

☐ Concept only, not yet active (interventions not started)

☐ Inactive, developed in a prior year

☐ Submission determined not to be a PIP

☐ No Non-clinical PIP was submitted

Brief Description of PIP (including goal and what PIP is attempting to accomplish):

The goal of this non-clinical PIP was to implement a triage assessment process to help high-need beneficiaries access services as quickly as possible, as well as ensure all beneficiaries have access to services in a timely manner. Annual External Quality Reviews have shown that in recent years Glenn County did not score well on timeliness to mental health and/or psychiatric services. The MHP identified this as an important area for improvement to ensure proper and consistent access to care for their beneficiaries.

- 81 -


In the initial year of the PIP, the MHP used 7, 14, and 21 days for prioritizing referrals for urgent, moderate, and routine beneficiaries, and have since updated the standards to include the Final Rule timeliness regulations of 15 business to psychiatric and 10 business days to mental health services.

The MHP initially developed interventions to directly impact the access processes and improve timeliness to services for beneficiaries. For this year, the MHP added the expansion of available appointments through hiring a dedicated staff person for assessments and adding assessment slots to supervisor schedules. The MHP also implemented a scheduling process which allowed the front office to schedule follow-ups efficiently.

For 5-day urgent requests for service, at baseline, 42.4 percent met the standard. At the time of remeasurement, 57 percent met the standard, representing an improvement of 9.4 percentage points.

ACTIVITY 1: ASSESS THE STUDY METHODOLOGY

STEP 1: Review the Selected Study Topic(s)


1.1 Was the PIP topic selected using stakeholder input? Did the MHP develop a multi-functional team compiled of stakeholders invested in this issue?

☐ Met

☒ Partially Met

☐ Not Met

☐ Unable to

Determine

The MHP assembled a multifunctional team; however, no beneficiary representation was noted.

- 82 -


1.2 Was the topic selected through data collection and analysis of comprehensive aspects of enrollee needs, care, and services?

☐ Met

☒ Partially Met

☐ Not Met

☐ Unable to

Determine

Annual External Quality Reviews have shown that in recent years Glenn County did not score well on timeliness to outpatient mental health and/or psychiatric services. They analyzed data for timeliness to mental health services and psychiatric services for January – May 2016 and verified that there was an issue with access for beneficiaries requesting initial services.

Although the initial PIP and problem were identified through data, further evaluation as to the related causes and/or barriers was not completed.

Select the category for each PIP:

Non-clinical:

☐ Prevention of an acute or chronic condition ☐ High volume services

☐ Care for an acute or chronic condition ☐ High risk conditions

☒ Process of accessing or delivering care

1.3 Did the Plan’s PIP, over time, address a broad spectrum of key aspects of enrollee care and services?

Project must be clearly focused on identifying and correcting deficiencies in care or services, rather than on utilization or cost alone.

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

The MHP developed interventions to directly impact the access process.

- 83 -


1.4 Did the Plan’s PIPs, over time, include all enrolled populations (i.e., did not exclude certain enrollees such as those with special health care needs)?

Demographics:


☒ Other All consumers of all ages.

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

The PIP includes all individuals, of all ages, who request mental health, including psychiatric services, from Glenn County Behavioral Health.


STEP 2: Review the Study Question(s)

2.1 Was the study question(s) stated clearly in writing?

Does the question have a measurable impact for the defined study population?

Include study question as stated in narrative:

“Will implementing a triage assessment process help high-need beneficiaries with an urgent need access services within five days; ensure beneficiaries with a routine need for services access services within 10 days; and beneficiaries requesting psychiatric services receive services within 15 days for at least 75 percent of all beneficiaries?”

☐ Met

☒ Partially Met

☐ Not Met

☐ Unable to

Determine

Although the question is measurable, its multipart structure may be too complicated to answer definitively.

Simple, straight forward questions which identify the problem, intervention and outcome measure might prove more answerable.

“Will (intervention X) improve (stated problem Y) as measured by (outcome measure Z)?”


STEP 3: Review the Identified Study Population

3.1 Did the Plan clearly define all Medi-Cal enrollees to whom the study question and indicators are relevant?

Demographics:


☒ Other All beneficiaries of all ages

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

All individuals, of all ages, who request mental health including psychiatric services from Glenn County Behavioral Health.

- 84 -


3.2 If the study included the entire population, did its data collection approach capture all enrollees to whom the study question applied?

Methods of identifying participants:

☒ Utilization data ☐ Referral ☐ Self-identification

☒ Other: Internal Access Log

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

All individuals, of all ages, who request mental health including psychiatric services from Glenn County Behavioral Health.


STEP 4: Review Selected Study Indicators

4.1 Did the study use objective, clearly defined, measurable indicators?

List indicators:

1. Percent of the urgent mental health service requests with an assessment scheduled within 5 calendar days

2. Percent of the routine mental health services requests with an assessment scheduled within 10 calendar days

3. Percent of the psychiatric services requests with an assessment scheduled within 15 calendar days

4. Percent of beneficiaries who received an assessment who received a second mental health service within 14 calendar days after their Assessment

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

- 85 -


4.2 Did the indicators measure changes in: health status, functional status, or enrollee satisfaction, or processes of care with strong associations with improved outcomes? All outcomes should be consumer focused.

☒ Health Status ☐ Functional Status

☐ Member Satisfaction ☐ Provider Satisfaction

Are long-term outcomes clearly stated? ☒ Yes ☐ No

Are long-term outcomes implied? ☒ Yes ☐ No

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine


STEP 5: Review Sampling Methods

5.1 Did the sampling technique consider and specify the:

a) True (or estimated) frequency of occurrence of the event?

b) Confidence interval to be used?

c) Margin of error that will be acceptable?

☐ Met

☐ Partially Met

☐ Not Met

☒ Not

Applicable

☐ Unable to

Determine

- 86 -


5.2 Were valid sampling techniques that protected against bias employed?

Specify the type of sampling or census used:

☐ Met

☐ Partially Met

☐ Not Met

☒ Not

Applicable

☐ Unable to

Determine

5.3 Did the sample contain a sufficient number of enrollees?

______N of enrollees in sampling frame

______N of sample

______N of participants (i.e. – return rate)

☐ Met

☐ Partially Met

☐ Not Met

☒ Not

Applicable

☐ Unable to

Determine


STEP 6: Review Data Collection Procedures

6.1 Did the study design clearly specify the data to be collected?

Utilized internal Access Log of service requests for mental health services and psychiatry services.

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

6.2 Did the study design clearly specify the sources of data?

Sources of data:

☒ Met

☐ Partially Met

☐ Not Met

- 87 -


☐ Member ☐ Claims ☐ Provider

☒ Other: Other: Internal Access Log and EHR

☐ Unable to

Determine

6.3 Did the study design specify a systematic method of collecting valid and reliable data that represents the entire population to which the study’s indicators apply?

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

All requests for services are in Internal Access Log and second appointments are recorded in the EHR.

6.4 Did the instruments used for data collection provide for consistent, accurate data collection over the time periods studied?

Instruments used:

☐ Survey ☐ Medical record abstraction tool

☐ Outcomes tool ☐ Level of Care tools

☒ Other: Internal Access Log

☒ Met

☐ Partially Met

☐ Not Met

☐ Unable to

Determine

Front office staff logged the service requests on the Access Log. Clinical staff performed the assessments and administrative staff updated the appointments on the Access Log.

6.5 Did the study design prospectively specify a data analysis plan?

Did the plan include contingencies for untoward results?

☐ Met

☒ Partially Met

☐ Not Met

☐ Unable to

Determine

Data was routinely discussed to determine the timeliness of services and ways to expedite the process. I.D.E.A. Consulting analyzed the data from the Access Log.

Who collected the data? How often was it collected? Who analyzed the data? How often was it analyzed? What tools were used to analyze it? Why was that person qualified to analyze the data? What did the MHP expect the data to show?

- 88 -


6.6 Were qualified staff and personnel used to collect the data?

Project leader:

Name:

Title:

Role:

Other team members:

Names:

☐ Met

☒ Partially Met

☐ Not Met

☐ Unable to

Determine

Specific names and titles were not given in the submission tool. I.D.E.A. consulting and front office staff are mentioned.


STEP 7: Assess Improvement Strategies

7.1 Were reasonable interventions undertaken to address causes/barriers identified through data analysis and QI processes undertaken?

Describe Interventions:

1. Create different levels of need on the Referral Form. Schedule and assess urgent acuity beneficiaries within five calendar days and routine beneficiaries within 10 calendar days.

2. Implement a wait list so when a beneficiary cancels an assessment appointment, the spot can be filled.

3. If a beneficiary does not show up within 15 minutes of their appointment time, staff call beneficiaries on the wait list to fill the spot. Staff also call and remind beneficiaries of their appointment the day prior to minimize “no-show” appointments.

☐ Met

☒ Partially Met

☐ Not Met

☐ Unable to

Determine

The MHP added the expansion of available appointments through hiring and adding assessment available to supervisor schedules. The MHP also implemented a scheduling process which allowed the front office to schedule follow-ups efficiently.

Interventions launched in 2017 are not considered active. Also, a review of data collection methods and improved tracking is not an intervention but rather an approach to data integrity. A barrier analysis on the causes of the problem would have allowed a more fine-tuned approach to intervention selection.

- 89 -


4. Expedite the assessment process for high-need medication beneficiaries with a shortened Treatment Plan completed by one of the referring staff or Managers.

Newly added:

5. Add a bi-weekly psychiatric evaluation time slot to be scheduled for persons with an urgent medication need, and fill with standard evaluations if not filled one week prior (July 2017)

6. Hire a PSE clinician to focus on providing additional assessment appointments. (August 2018) (active intervention).

7. Review Data collection methods at QIC and implement tracking of business days rather than calendar days, when to start the date of request, and looking at first offered appointment in addition to first accepted. (October 2018)

8. Create a scheduling process for all clinicians so that front office can schedule all follow-up appointments after assessment and help to get the beneficiary into treatment more efficiently and consistently. (November 2018)

9. Have clinical supervisors who have an expectation to see beneficiaries also provide assessment appointments to increase the number of appointments overall. (May 2019) (active intervention)

- 90 -



STEP 8: Review Data Analysis and Interpretation of Study Results

8.1 Was an analysis of the findings performed according to the data analysis plan?

☒ Met

☐ Partially Met

☐ Not Met

☐ Not

Applicable

☐ Unable to

Determine

The MHP analyzed data to show the number and percent of beneficiaries who received an assessment within the 5-day goal for urgent needs, 10-day goal for routine needs of mental health services, and 15-day goal for psychiatric services. Upon implementation of the interventions, the MHP analyzed the data again. They added a new indicator to measure the time from receiving an assessment to receiving a second mental health service. May 2017 – April 2018 was the baseline with remeasurement July 2018 through May 2019.

8.2 Were the PIP results and findings presented accurately and clearly?

Are tables and figures labeled?

☒ Yes ☐ No

Are they labeled clearly and accurately?

☐ Yes ☒ No

☐ Met

☒ Partially Met

☐ Not Met

☐ Not

Applicable

☐ Unable to

Determine

They did not use sampling as all new beneficiaries are included in this PIP. The MHP did not use statistical significance for this PIP. Once they fully implemented the various levels of need for requests, they analyzed the data separately for each level. The PIP committee routinely reviewed the analyses; however, the remeasurement findings contained calculation errors. The MHP provided a corrected table.

- 91 -


8.3 Did the analysis identify: initial and repeat measurements, statistical significance, factors that influence comparability of initial and repeat measurements, and factors that threaten internal and external validity?

Indicate the time periods of measurements: ___________________

Indicate the statistical analysis used: ___________n/a______________

Indicate the statistical significance level or confidence level if available/known: ____percent ______Unable to determine

☒ Met

☐ Partially Met

☐ Not Met

☐ Not

Applicable

☐ Unable to

Determine

8.4 Did the analysis of the study data include an interpretation of the extent to which this PIP was successful and recommend any follow-up activities?

Limitations described:

Conclusions regarding the success of the interpretation:

Recommendations for follow-up:

☒ Met

☐ Partially Met

☐ Not Met

☐ Not

Applicable

☐ Unable to

Determine


- 92 -


STEP 9: Assess Whether Improvement is “Real” Improvement

9.1 Was the same methodology as the baseline measurement used when measurement was repeated?

Ask:

At what interval(s) was the data measurement repeated?

Were the same sources of data used?

Did they use the same method of data collection?

Were the same participants examined?

Did they utilize the same measurement tools?

☒ Met

☐ Partially Met

☐ Not Met

☐ Not

Applicable

☐ Unable to

Determine

The MHP analyzed the baseline data for services delivered in FY 2016-17. The MHP then implemented the interventions, and analyzed the data again for FY 2017-18, and reviewed in our EQRO that year. This PIP was continued for FY 2018-19, and data was analyzed quarterly at the Quality Improvement Committee meetings, and then overall upon close of the Fiscal Year. Since beneficiaries need a second CANS-50 score to analyze improvement for this indicator, we added measurement results at the end of the study period.

Baseline measurements were:

January – May 2016; May 2017 – April 2018

Remeasurement occurred:

July 2018 – May 2019

9.2 Was there any documented, quantitative improvement in processes or outcomes of care?

Was there: ☒ Improvement ☐ Deterioration

Statistical significance: ☐ Yes ☒ No

Clinical significance: ☒ Yes ☐ No

☒ Met

☐ Partially Met

☐ Not Met

☐ Not

Applicable

☐ Unable to

Determine

The baseline was for May 2017-April 2018, and there were 210 requests for services identified as 5-day urgent indicators. 89 requests met the timeliness standard (42.4 percent). This improved to 146 out of 256 (57 percent) upon close of the PIP, representing an improvement of 9.4 percent.

- 93 -


9.3 Does the reported improvement in performance have internal validity; i.e., does the improvement in performance appear to be the result of the planned quality improvement intervention?

Degree to which the intervention was the reason for change:

☐ No relevance ☐ Small ☒ Fair ☐ High

☐ Met

☒ Partially Met

☐ Not Met

☐ Not

Applicable

☐ Unable to

Determine

While there was improvement, it is unclear which intervention over the last two years had the greatest impact. Remeasurement after each intervention would provide more information.

9.4 Is there any statistical evidence that any observed performance improvement is true improvement?

☐ Weak ☒ Moderate ☐ Strong

☐ Met

☐ Partially Met

☒ Not Met

☐ Not

Applicable

☐ Unable to

Determine

9.5 Was sustained improvement demonstrated through repeated measurements over comparable time periods?

☐ Met

☐ Partially Met

☒ Not Met

☐ Not

Applicable

☐ Unable to

Determine


- 94 -


ACTIVITY 2: VERIFYING STUDY FINDINGS (OPTIONAL)


Were the initial study findings verified (recalculated by CalEQRO) upon repeat measurement?

☐ Yes

☒ No


Conclusions:

PIP Validation #s

Met 15

Partially Met 8

Not Met 2

UTD 0

# Not applicable 3

Score 76.00%

#applicable 25

Suggestions to improve the PIP: The PIP could have benefited from beneficiary inclusion on the PIP team. Also, re-measurement should occur quarterly at minimum. Although the initial PIP and problem were identified through data, further evaluation as to the related causes and/or barriers was not completed. Although the study question was measurable, its multipart structure was complicated. Simple, straight-forward questions which identify the problem, intervention and outcome measure might prove more answerable.

- 95 -



Recommendations:

Conclude the PIP and begin a new Non-clinical PIP.


The TA provided to the MHP by CalEQRO consisted of recommending that the MHP measure results quarterly. Discussion onsite involved the MHP’s plan to end the PIP and begin a new Non-clinical PIP on the topic of improving beneficiary access to Harmony House, the MHP’s wellness center. The MHP was also encouraged to consult with EQRO early and often during the development of new PIPs. Recommendations for new PIPs include using a simple, straight forward question which identifies the problem, intervention and outcome measure might prove more answerable; moreover, the new PIP should include more in-depth data evaluation, a barrier analysis, to ensure that interventions are related to problem causes.

Check one: ☐ High confidence in reported Plan PIP results ☐ Low confidence in reported Plan PIP results

☒ Confidence in reported Plan PIP results ☐ Reported Plan PIP results not credible

☐ Confidence in PIP results cannot be determined at this time

fy 2019-20 medi-cal specialty mental health external ... and... · 1. fy 2019-20 medi-cal specialty...

Documents