measuring value and outcomes for continuous quality ... value and outcomes for continuous quality...

Measuring Value and Outcomes for Continuous Quality Improvement

Noelle Flaherty MS, MBA, RN, CCM, CPHQ1

Jodi Cichetti, MS, RN, BS, CCM, CPHQ

Leslie Beck, MS 1

Amanda Abraham MS 1

Maria Uriyo, PhD, MHSA, PMP 1

1. Johns Hopkins Healthcare LLC, Baltimore Maryland

Corresponding Author: Noelle Flaherty, MS, MBA, RN, CCM, CPHQ

Biographical Sketches:

Noelle Flaherty, MS, MBA, RN, CCM, CPHQ is Director of Quality Improvement at Johns

Hopkins HealthCare LLC. (JHHC). Ms. Flaherty earned her BA from Bryn Mawr College, BS

from Johns Hopkins School of Nursing, MS in Health Services Leadership and Management

from the University of Maryland School of Nursing, and MBA from the University of Baltimore.

Jodi Cichetti, MS, RN, BS, CCM, CPHQ is the Senior Director of Quality and Clinical

Improvement, WellSpan Health System, and past Senior Director Medical Management, Johns

Hopkins Health System.

Leslie Beck, MS, is the Health Services Project Manager at JHHC. Ms. Beck served as Co-Chair

on the JHHC Diversity Management Leadership Committee and is a member on the Johns

Hopkins Berman Institute of Bioethics Patient Advisory Group. Ms. Beck holds a Bachelor of


2

Science degree in Business Management and a Master of Science degree in Healthcare

Management from the Johns Hopkins University Carey Business School in Baltimore, Maryland.

Amanda Abraham, MS, is the Data Analyst for Quality Improvement at JHHC. She received her

Bachelor’s degree from Penn State University in Health Policy & Administration and a Master’s

degree from George Mason University in Health Systems Administration with a concentration in

Executive Management Her primary role as an analyst is to provide actionable data on the

Value-Based Purchasing project and Medicaid line of business (Priority Partners MCO) to the

administrative team.

Maria Uriyo, PhD, PMP, is the Project Manager for NCQA Accreditation at JHHC. She received

her PhD in Food Science, Master’s in Food Chemistry from Virginia Polytechnic Institute &

State University and a Masters in Health Systems Administration from Georgetown University.

In her current role Dr. Uriyo manages the NCQA accreditation process for JHHC.

Acknowledgement: The Quality Improvement Team developed the framework under direction

of Dr. Chester Schmidt, Chief Medical Officer, JHHC.

Disclosures: No grants or external support funded this work.


3

Abstract

Quality of care is critically important to health plans, systems and care providers striving

for a healthier population at a decreased cost. Prioritizing action related to population health

and quality of care is a challenge for leaders in the healthcare industry. Poor performance on

quality outcome measures can cost health plans and providers millions of dollars in missed

revenue, fines, penalties and other related health care costs. Challenges for the health care

industry include hundreds of National Quality Forum (NQF) endorsed measures, variations

between government, payer and provider quality indicators and related definitions, as well as

current or future financial penalties linked to different measures by regulatory and accreditation

bodies. In response to the need for meaningful and actionable health care initiatives, the Johns

Hopkins HealthCare LLC Quality Improvement (QI) Department developed a framework for

quality project planning based on a mathematical model in order to easily identify, forecast and

target population health measures. This effort is in alignment with Johns Hopkins Medicine’s

mission to continuously reduce preventable harm, improve patient outcomes and enhance the

value and equity of care around the world by advancing the science of patient safety and quality

through discovery, implementation, education, evaluation, and collaborative learning.

Keywords: Value Based Purchasing; Performance Improvement; Pay for Performance (P4P)

Introduction

Johns Hopkins HealthCare LLC (JHHC) administers health care benefits and services for

over 408,000 lives enrolled in managed care organizations, government and employer sponsored

programs. JHHC supports plan members through added benefits, including outreach, disease


4

management, complex case management and health education. Quality Improvement (QI)

Program activities support and promote the JHHC mission to improve the lives of our plan

members by providing access to high quality, cost effective, member-centered healthcare.

Additionally, the JHHC QI program supports the Johns Hopkins Medicine (JHM) mission to

improve the health of the community and the world by setting the standard of excellence in

medical education, research, and integration. The QI Department works collaboratively with

Johns Hopkins entities to ensure that members are receiving exceptional health care.

Annually, health outcomes for our covered members are measured through the

Healthcare Effectiveness Data and Information Set (HEDIS®). Based on the HEDIS

® results, the

JHHC Quality Improvement (QI) Department will propose projects to improve patient outcomes.

A standard work method was needed to efficiently and effectively focus and allocate resources to

measure directed projects that will improve care quality, and also provide optimal impact to

quality ranking results for the health plan. In response, the QI Department developed a

framework for quality project planning based on a mathematical model to easily identify,

forecast and target population health quality measures.

Healthcare outcomes and patient/member satisfaction data are important indicators in

measuring care quality, and when monitored with due diligence, support good financial

stewardship. Quality measurement and improved outcomes are a high priority due in part to the

importance of care quality for the patient population, and also to the potential financial impact of

pay for performance programs including value based purchasing (VBP), the Centers for

Medicare and Medicaid Services (CMS) Five-Star Quality Rating System (Stars), and other pay

for performance (P4P) programs. From a financial perspective, the return on investment for


5

improving health outcomes and patient/member satisfaction can be estimated in the millions of

dollars for some organizations.

Health care systems and leadership are challenged with developing strategies for the

management of quality improvement projects due to the sheer number of quality measures and

related data metrics, and the competing priorities for resources in the healthcare environment.

The National Quality Forum (NQF) has endorsed over 700 measures1 that have undergone peer

review, and are considered clinically relevant to population health and quality of care. Measure

types include process, outcome, intermediate clinical outcome, efficiency and cost/resource

utilization.1 Quality measures that are used by health plans and federal and state agencies are

based on NQF endorsed measures. For example, the Maryland Department of Health (MDH) has

a VBP program for HealthChoice, which is the Medicaid managed care program2, ten (10) of the

thirteen (13) VBP measures are NQF endorsed measures, while the other three (3) were

developed by the MDH. In addition to the NQF measures, the Agency for Healthcare Research

and Quality (AHRQ) National Quality Measure Clearinghouse includes over 2,000 measures,

including some measures that overlap with NQF. 3

In order to manage the volume and variety of measures, a standard framework is needed

to support a focus on clinically relevant opportunities with the greatest positive impact on

population health.4 Meltzer & Chung evaluated thirteen (13) AHRQ quality measures and

proposed a framework for prioritization called “net health benefit”. This clinical framework

focuses on the evaluation of cost and population health outcomes. Porter’s model is another

method for prioritization that focuses on the value for the patient5. Value is quantified through an

analysis of health outcomes relative to costs. Similar to the model proposed by Meltzer &

Chung, Porter’s model is population health based and clinically driven5. The framework


6

developed by our health plan includes the clinical and cost elements, but also includes

administrative and effort scoring with the goal of the best impact on patient health, patient

experience and cost of care.

Methods

In 2012, the QI Department analyzed patient outcomes data as defined by HEDIS® and

identified many opportunities for improvement. The QI Department had limited resources to

impact all of the measures, and identified a need to develop a targeted approach to quality

improvement initiative planning. The measures that were targeted by the QI team were specific

to the health plan accreditation requirements from the National Committee for Quality Assurance

(NCQA®

), which is a key indicator of health plan excellence. The QI team created an initiative

planning framework to focus on decision making and appropriately allocate resources.

Institutional Review Board Approval (IRB) was not required for the framework development or

associated data analysis.

The framework developed by the QI Department has three (3) phases, and includes

consideration of multiple factors, including national and regional benchmarks, administrative

effort, individual circumstances of the business, and variations in population demographics. This

framework includes tools for quality improvement reporting, planning and intervention. The

framework is cost effective and does not require technology beyond standard data collection of

quality measures.

In the first phase of the framework, previous year results are used to prioritize measures

with the greatest opportunity for improvement when compared to national or regional

benchmarks. The evaluation of previous year results includes the following:


7

1. Basic population evaluation components: age, gender, geography, and ethnicity.

2. Determine multifaceted population evaluation components identified in previous quality

program evaluations, targeting best practices and opportunities for improvement.

3. A barrier analysis.

4. Identify population targets by geography and population size.

5. Literature review of best practices.

Figure 1: Example of an evaluation of a behavioral health measure

After completing the evaluation and barrier analysis, measures are color coded to

highlight opportunity for improvement. Green indicates maximum improvement opportunity.

Yellow is the next best point, and red indicates minimal movement or need to maintain. Minimal

or need to maintain usually indicates measures that are in the goal range (threshold or target

when compared to benchmarks). The use of color coding and benchmarks in this first phase are

shown in the example in Figure 2. Although NCQA®

benchmarks are used in the example; other

national or regional benchmarks can be applied using the same process. The color coding can be

modified to meet the business or population health requirements of the organization.


8

Figure 2

After the measures are color coded, the methodology focuses on measures that are

identified as having the best or next best opportunities for improvement (color coded green or

yellow). The JHHC QI department may also decide to focus on high performing measures (color

coded red) if there is a business reason (P4P) or population health rationale to focus on those

measures.

During phase two of the methodology, a percentage to goal value is calculated for each

measure using current year (prospective) data to identify likelihood of success based on current

data. The percent to goal (PG) of a measure moving to the next threshold percentile level is

calculated using the following formula: PG= CCR/NBPTR. In this formula, PG is the percent to

goal; CCR is current compliance rate and NBPTR is the national benchmark percentile target

rate. This calculation helps focus attention to the amount of effort that will be needed to meet

the next percentile benchmark. Some measures might be identified as potentially reaching the

target based on little to no effort. Figure 3 is an example of the second phase of the framework

for CY 2018 planning. The measures in this example have demonstrated improvement, so a

control plan can be put in place for these measures and other measures can be targeted in C 2018.

HEDIS® Measures2017 Final HEDIS

Rate

2017 NCQA®

National

Percentile

Benchmark

Threshold

Additional 2018

HEDIS® Points (if

next benchmark

attained)

Breast Cancer Screening 76.15 75th 0.180

Chlamydia Screening in Women 43.07 50th 0.285

Follow-up After Hospitalization for Mental

Illness - within 7 days66.67 90th 0.000


9

Figure 3

During phase three, the QI Department meets internally to review the priority measures

identified in the first two phases. The QI Director then meets with business leaders, physicians

and other stakeholders to learn about any business needs or strategies already underway that

could positively (or negatively) impact the quality outcomes for the current year. Factors to

consider prior to scoring measures include specifics related to the financial history of project

planning, patient/member population, company workforce, and historical quality approach.

After gathering input from the QI team, business leaders and other stakeholders, the QI

Director scores each measure using four pre-defined categories: 1) administrative effort; 2)

population impact / relevance; 3) reporting requirements; and 4) anticipated expense.

Definitions of the categories are as follows:

1) Administrative effort is defined as the level or intensity of the total business work effort

related to quality or care management programs;

2) Population impact or relevance is defined as the number of opportunities within the

measure or the relevance assigned. The lower the point score, the higher the anticipated

impact on the population or higher relevance;

3) Reporting effort is defined as the level of complexity for ongoing monitoring and

evaluation of effectiveness of quality projects; AND


10

4) Anticipated expense includes projected costs for the quality program and potential

financial risks such as VBP penalties, potential loss of contracted members, or other

regulatory fines. The amount of expense is compared with the potential financial gain or

loss.

Each category is scored on a scale of 0-5. Effort scoring is a total cumulative score

(maximum of 20 points) where the higher point value represents the higher/heavier work

effort comparatively for corporate consideration and project planning. Measures identified as

having a lower point value may be selected for rapid cycle quality improvement projects.

Measures with a higher point value that are selected for quality initiatives may need more

time for planning and budgeting to successfully manage the work effort. The cumulative

score can be considered in addition to the other factors in determining the most valuable

quality measure for corporate focus and quality planning for improved clinical and reported

patient outcomes and enhanced value. Figure 4 is an example of the third phase of the

framework that was used for initiative planning in CY 2016.

Figure 4

Results

In 2012, the QI team used the framework and identified breast cancer screening as a

measure for improvement for the commercial line of business. A women’s health screening


11

brochure was sent to members with opportunities in 2011 and 2012. A significance validation

was calculated for women who received health reminders. The evaluation of the members who

received a mailing had a significantly better rate of compliance (p value <0.0001) than those who

did not receive the mailing in 2011. Based on evaluation, the brochure mailings likely served as

a benefit to remind the member to schedule the appointment for breast cancer screening. HEDIS

® scores increased for this measure from 70.82% (50th

percentile) in Calendar Year (CY) 2012

to 76% (75th

percentile) in CY 2013. The QI Initiative Project expanded in 2014 and 2015 and

the plan has maintained 75th

percentile since CY 21013, indicating that a control plan is in place

to maintain performance for this measure.

Another measure that was identified for improvement using this framework was the

quality measure for follow up care after a mental health hospitalization. A project was initiated

in 2013, and was refined and updated in 2016. As a result of the ongoing focus on this measure,

the results in 2016 demonstrated a statistically significant improvement (9.93%). This

improvement also resulted in an increase in overall accreditation score for the plan, contributing

to the US Family Health Plan ongoing “Excellent” accreditation. Figure 5 demonstrates the

improvement in this measure over time and with a sustained effort to improve the coordination

of care project for mental health hospitalization within seven (7) days.


12

Figure 5

In addition to the measure specific successes referenced above, overall quality results for

JHHC health plans have been above national standards. Results for 2017 (CY 2016) that

demonstrates the overall success of the framework for one of our programs includes:

– NCQA® Excellent accreditation for US Family Health Plan, the highest level of

health plan accreditation

– NCQA® Rating 5 out of 5, indicating that the US Family Health Plan is a highly

rated national plan.

– The highest possible percentile ranking (NCQA® 90th percentile) for the

following HEDIS® measures identified through the framework: Cervical Cancer

Screening, Comprehensive Diabetes Care, Follow-Up After Hospitalization for

Mental Health – 7 days


13

– Ongoing compliance for the Breast Cancer Screening measure at the 75th

percentile level.

– Significant improvement (6.90 %) of compliance with chlamydia screening for

women.

Limitations

The framework and associated tools are not automated and do not incorporate statistical

analysis using large data sets. Access to quality data, including administrative, pharmacy,

laboratory, claims, encounters and electronic data is needed to maximize the use of the

framework. Changes within health care, including changes to clinical practice guidelines, quality

measure specifications, payer plan benefits or decreased access to care can result in lower than

expected results. Although leadership input is engaged for the effort scoring, strategic decisions

may be made by leadership that could result in selection of quality measures that are not

identified as priority through the framework.

Discussion

For most quality measures, the framework is applied to the current year using past results

and a minimum of three months’ of quality data. When there is a new quality measure, or when

there are significant revisions to the specification for an established measure, projected trended

rate and percentage to goal are not applicable because there is no historical data available to

calculate the projected trend rate. In these circumstances, the goal percentile ranking is listed as

requiring the greatest effort.

There are measures that require chart audit/abstraction of a sample population, so

prospective administrative data is not available. An example is the HEDIS® measure for

controlling blood pressure6, which cannot be easily measured for a health plan throughout the


14

year because the measure is based on a representative sample of the entire population, and

measures the last blood pressure of the calendar year. For this type of measure, the projected

rate is derived from the two basic assumptions that past patterns will persist into the future, and

measurable fluctuations in past trends will recur regularly and can be projected into the future.

Previous year’s eligible reporting population and information on total plan membership is

referenced.

Conclusions

The framework provides a standard approach to prioritizing quality improvement projects

with the goal of improving patient outcomes. Executive decisions are made easier by clear

recommendations from subject matter experts that are supported by data. The final product for

executive decision making is an objective report inclusive of a pre-determined and consistent

methodology, that once applied, supports quality improvement prioritization and focus for

intervention. The model developed by the JHHC QI Department has been effective over the

past five years, and quality measures identified through the framework have maintained or

improved each year when the proposed projects were approved and funded. In addition to

proven maintenance or improvement of quality measure performance (by measured year)

identified by the framework, there is a correlated improvement in overall quality ranking.

Implications

The NQF has recognized the importance of developing meaningful measures as well as

improving and prioritizing existing measures, as evidenced by the NQF 2016-2019 Strategic

Plan to answer “an unmet need for NQF to lead, prioritize, and collaborate to drive measurement

that can result in better, safer and more affordable healthcare for patients, providers, and


15

payers.”1

CMS has also recognized the need for a core measure set to focus on patient outcomes.

CMS, in collaboration with America’s Health Insurance Plans (AHIP), released seven set of

quality measures in 2016.7 The work between CMS and AHIP is ongoing, but does not address

the need for health plans, health systems and providers to prioritize quality improvement

activities and related interventions to improve patient health care. The framework developed

and implemented by the JHHC QI department supports both the NQF and CMS goals related to

identifying key measures for prioritization to improve health outcomes. This framework can be

easily modified for adoption for use in a variety of settings, including health plan/ payor,

Accountable Care Organization (ACO), hospital and provider.

References

1 National Quality Forum (NQF). Healthcare Measurement. Retrieved from

http://www.qualityforum.org/NQF_Strategic_Direction_2016-2019.aspx on June 7th, 2016.

2 Maryland Department of Health and Mental Hygiene. HealthChoice. Retrieved from

https://mmcp.dhmh.maryland.gov/healthchoice/pages/Home.aspx on July 22, 2016.

3 Agency for Healthcare Research and Quality. National Quality Measures Clearinghouse.

Retrieved from http://www.qualitymeasures.ahrq.gov/index.aspx on June 7th, 2016.

4 Meltzer, D. O., & Chung, J. W. (2014). The population value of quality indicator reporting: a

framework for prioritizing health care performance measures. Health Affairs, 33(1), 132-139.

5 Porter, M. What is Value in Health Care? The New England Journal of Medicine 363:26

(2015): 2477-2481.

6 National Committee for Quality Assurance (NCQA). HEDIS ® & Performance Measurement.

Retrieved from http://www.ncqa.org/tabid/59/Default.aspx on June 7th, 2016.

https://mmcp.dhmh.maryland.gov/healthchoice/pages/Home.aspx

http://www.qualitymeasures.ahrq.gov/index.aspx


16

7 Centers for Medicare & Medicaid Services. (2016). CMS and major commercial health plans,

in concert with physician groups and other stakeholders, announce alignment and simplification

of quality measures, Retrieved from CMS.gov on July 22, 2016.

measuring value and outcomes for continuous quality ... value and outcomes for continuous quality...

Documents