1
Long Term Care Luncheon:
The CMS Five-Star Quality
Rating System
April 12, 2018
Quality Ratings of
U.S. Nursing Homes
on Nursing Home
Compare
Jennifer Pettis, MS, RN, WCC
Nurse Researcher / Associate
Abt Associates
Abt Associates | pg 3
Session Objectives
After attending this session participants will be able to:
List the domains included in the Five-Star Quality Rating System,
Define recent changes to the Five-Star Quality Rating System, including how these changes have improved the quality of data being publicly reported to US consumers, and
Discuss potential changes in provider behavior and changes in provider performance related to safety, quality, and value since the implementation of the Five-Star Quality Rating System.
This work was supported through a contract with the Centers for Medicare & Medicaid Services (CMS), Center for Clinical Standards and Quality, Survey and Certification Group.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
2
Abt Associates | pg 4
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 5
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 6
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Nursing Home Compare
February 2015Quality Measure
(QM) domain rebased
Antipsychotic QMs incorporated into
rating system
2008
Five-Star Quality Rating System introduced
1998
Nursing Home Compare launched
2018
Staffing Rating based on Payroll-Based Journal (PBJ)
September 2016
Final Rule to Reform Requirements for Long-Term Care Facilities
July 2016
New QMs introduced
Methodological changes to Five-Star Quality Rating System
3
Abt Associates | pg 7
Three Domains of Quality in Rating
System
Overall Rating
Health Inspections
Staffing
Quality Measures
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 8
Health Inspections
Nursing homes that participate in the Medicare
and/or Medicaid programs have an onsite
recertification (standard) (“comprehensive”)
inspection annually, on average.
Inspections are unannounced and provide a
comprehensive assessment of the nursing home.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 9
Health Inspection Domain
Ratings are based on the number, scope, and
severity of deficiencies identified during:
– The two most recent annual recertification inspections
occurring prior to November 28, 2017, and
– Substantiated findings from complaint investigations
occurring in the 24 months prior to November 28, 2017.
All deficiency findings are weighted by scope and
severity.
This measure also takes into account the number of
revisits.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
4
Abt Associates | pg 10
Health Inspection Domain (cont.)
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 11
Health Inspection Domain (cont.2)
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 12
Health Inspection Domain (cont.3)
CMS calculates a total health inspection score for
each facility calculated as the facility’s weighted
deficiency score.
A lower health inspection score corresponds to better
performance.
More recent health inspections are weighted more
heavily than earlier health inspections.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
5
Abt Associates | pg 13
Health Inspection Domain (cont.4)
Complaint inspections are assigned to a time period based on the 12-month period in which the complaint survey occurred.
Complaint inspections that occurred between November 28, 2016 and November 27, 2017 receive a weighting factor of 60 percent, and those occurring between November 28, 2015 and November 27, 2016 have a weighting factor of 40 percent.
Facilities with only one standard health inspection prior to November 28, 2017 are considered not to have sufficient data to determine a health inspection rating and are reported as “Too New to Rate” for the health inspection domain.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 14
Health Inspection Domain (cont.5)
CMS bases Five-Star quality ratings in the health
inspection domain on the relative performance of
facilities within a state.
Top 10 percent ★★★★★
Middle 70 percent
(approximately 23.33
percent in each rating)
★★★★
★★★
★★
Bottom 20 percent ★
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 15
Health Inspection Domain (cont.6)
Rating thresholds are re-calibrated each month.
– However, the rating for a given facility is held constant until
there is a change in the weighted health inspection score for
that facility.
While changes to health inspection scores (and thus
rating changes) during the time period when surveys
conducted under the new process will be rare, there
are a few reasons why facilities may have a change
(some refer to this as a “freeze” of the survey rating,
beginning February 2018).
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
6
Abt Associates | pg 16
Health Inspection Rating “Freeze”
Beginning in February 2018, for a period of 12
months, CMS will not use deficiencies cited on
surveys on or after November 28, 2017 in calculating
the Five-Star health inspection rating
Standard surveys and complaint surveys that
occurred on or after November 28, 2017 (under the
new survey process) will be displayed on the Nursing
Home Compare website but will not be utilized to
calculate the Five-Star health inspection rating.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 17
Staffing Domain
In 2018, CMS will replace the existing staffing
measures posted on Nursing Home Compare and
used in the Five-Star Quality Rating System with
measures using Payroll-Based Journal (PBJ) data
submitted for the October 1 to December 31, 2017
reporting period by the February 14, 2017 deadline.
March Provider Preview Reports included facility-
specific PBJ data that was submitted to CMS by the
deadline.
Physical therapist data are reported on NHC but not
included in calculations for the Five-Star ratings.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 18
Staffing Domain (cont.)
The Nursing Home Compare website indicates
whether providers have submitted data by the
required deadline, and, if providers have submitted,
whether those data are complete, incomplete, or
inaccurate.
Information about staffing data submission is
available on the CMS website. Go to:
https://www.cms.gov/Medicare/Quality-Initiatives-
Patient-Assessment-
Instruments/NursingHomeQualityInits/Staffing-Data-
Submission-PBJ.html
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
7
Abt Associates | pg 19
Benefits of PBJ Data
The staffing data submitted on the CMS-671 captures only a two-week period prior to the facility’s annual recertification survey, whereas the PBJ data captures facility staffing for every day in a quarter.
The definitions of certain job categories have been revised for PBJ data submission compared to the instructions on the CMS-671 form, resulting in more precise reporting.
The PBJ data includes the census for each day in a quarter.
The PBJ program requires facilities to submit only those hours that staff are paid to work.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 20
PBJ Fiscal Quarter Reporting
Periods
Fiscal
Quarter
Reporting
Period
Due Date
1October 1 –
December 31February 14
2January 1 –
March 31May 15
3April 1 –
June 30August 14
4July 1 –
September 30November 14
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 21
QM Domain
The facility rating for the QM domain is based on
performance on:
– A subset of 13 (out of 24) of the Minimum Data Set (MDS)-
based QMs and
– Three Medicare claims-based QMs
Nine QMs are for long-stay residents.
Seven QMs are for short-stay residents.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
8
Abt Associates | pg 22
Measures for Long-Stay Residents
1. Percentage of residents whose need for help with activities of daily living has increased
2. Percentage of residents whose ability to move independently worsened (New, 7/2016)
3. Percentage of high risk residents with pressure ulcers (sores)
4. Percentage of residents who have/had a catheter inserted and left in their bladder
5. Percentage of residents who were physically restrained
6. Percentage of residents with a urinary tract infection
7. Percentage of residents who self-report moderate to severe pain
8. Percentage of residents experiencing one or more falls with major injury
9. Percentage of residents who received an antipsychotic medication
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 23
Measures for Short-Stay Residents
1. Percentage of residents whose physical function improves from admission to discharge (New, 7/2016)
2. Percentage of residents with pressure ulcers (sores) that are new or worsened
3. Percentage of residents who self-report moderate to severe pain
4. Percentage of residents who newly received an antipsychotic medication
5. Percentage of residents who were re-hospitalized after a nursing home admission (New, 7/2016)
6. Percentage of residents who have had an outpatient emergency department visit (New, 7/2016)
7. Percentage of residents who were successfully discharged to the community (New, 7/2016)
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 24
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
QM Domain Calculation
CMS calculates ratings for the QM domain using the
four most recent quarters for which data are
available.
Values for the three claims-based measures are
calculated in a similar manner, except that the data
used to calculate the measures use a full year of
data.
9
Abt Associates | pg 25
Missing Data and Imputation
MDS-based measures are reported if the measure can be calculated for at least 20 residents’ assessments.
The claims-based measures are reported if the measure can be calculated for at least 20 nursing home stays over the course of the year.
For facilities with missing data or an inadequate denominator size for one or more QMs, all available data from the facility are used. The remaining assessments (or stays) are imputed to get the facility to the minimum required sample size.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 26
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Scoring Rules for the QM Domain
For each measure, points are assigned based on facility
performance relative to the national distribution of the
QM.
For all of the measures, the groupings are based on the
national distribution of the QMs, prior to any imputation.
After any needed imputation for individual QMs, the points
are summed across all QMs based upon the scoring rules
above to create a total score for each facility.
Once the summary QM score is computed for each facility
as described above, the five-star QM rating is assigned.
Abt Associates | pg 27
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Overall Nursing Home Rating
(Composite Measure)
Step 1: Start with the health inspection rating.
Step 2: Add one star to the Step 1 result if the staffing rating is four or five stars and greater than the health inspection rating; subtract one star if the staffing rating is one star. The overall rating cannot be more than five stars or less than one star.
Step 3: Add one star to the Step 2 result if the quality measure rating is five stars; subtract one star if the quality measure rating is one star. The overall rating cannot be more than five stars or less than one star.
Note: If the health inspection rating is one star, then the overall rating cannot be upgraded by more than one star based on the staffing and quality measure ratings. If the nursing home is a Special Focus Facility (SFF) that has not graduated, the maximum overall rating is three stars.
10
Abt Associates | pg 28
Change in Nursing Home Ratings
Events that could change the health inspection score include:
– A new health inspection (prior to 11/28/17).
– New complaint deficiencies (prior to 11/28/17).
– A second, third, or fourth revisit (for a survey before 11/28/17).
– Resolution of Informal Dispute Resolutions (IDR) or Independent IDR (IIDR) resulting in changes to the scope and/or severity of deficiencies (for surveys before 11/28/17).
New staffing data for the facility.
Changes in the QM rating for the facility (mid-month January, April, July, and October).
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 29
July 2016 Changes to Five-Star
Introduction of five new QMs, including the first
measures that utilize Medicare claims, rather than
only self-reported MDS assessments.
– These measures greatly increase the number of short-stay
measures, and add domains not well covered by other
measures.
Several methodological changes to the calculation of
the QM ratings.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 30
Overall Quality Ratings:
2012 – 2018
Note: Several changes to 5-Star rating methodology were made in February 2015.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
11
Abt Associates | pg 31
Health Inspection Ratings:
2012 – 2018
Note: By design, the distribution of health inspection ratings is fixed.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 32
Staffing Ratings:
2012 – 2018
Note: Starting in February 2015, facilities required to have a 4- or 5-star staffing rating on either RN or total nurse staffing to have an overall staffing rating of 4-stars.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Abt Associates | pg 33
Quality Measure Ratings:
2012 – 2018
Note: In February 2015, the quality measure scale was re-set, increasing the standard for nursing homes to receive a high rating.
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
12
Abt Associates | pg 34
Questions and Contact Information
Copyright © 2017, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)
Jennifer Pettis, MS, RN, WCC
Nurse Researcher/Associate
Abt AssociatesEmail: [email protected]
Office phone: 617.520.3072
www.abtassociates.com
Abt Associates | pg 35
References
Abt Associates. (2014). Nursing Home Compare Five-Star Quality Rating System: Year five report. Retrieved from https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/CertificationandComplianc/Downloads/NHC-Year-Five-Report.pdf
Abt Associates. (2016). Nursing Home Compare Five-Star Quality Rating System: Year Seven Annual Report. Currently in draft.
Centers for Medicare & Medicaid Services. (2018). Design for Nursing Home Compare Five-Star Quality Rating System: Technical users’ guide. Retrieved from https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/CertificationandComplianc/downloads/usersguide.pdf
Centers for Medicare & Medicaid Services. (2018). Staffing Data Submission PBJ . Retrieved from https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/NursingHomeQualityInits/Staffing-Data-Submission-PBJ.html
Copyright © 2018, Abt Associates Inc., All Rights Reserved (prepared with data obtained under HHS/CMS contract HHSM-500-2013-13001I)