auditing your staff for performance improvement: tips to ... · tips to manage underperformers...

1

This is the Full Title of a Session

Auditing Your Staff for Performance Improvement: Tips to Manage Underperformers

Rachel Mack, RN, MSN, CCDS, CDIPCDI Quality & Performance Improvement Analyst SCL Health SystemLittleton, CO

Kari Thie, RN, BSN, CCDS CDI Quality & Performance Improvement AnalystSCL Health SystemLakewood, CO

2018 Copyright, HCPro, an H3.Group division of Simplify Compliance LLC. All rights reserved. These materials may not be copied without written permission.

2

Learning Objectives

• At the completion of this educational activity, the learner will be able to:– Identify need for a CDI auditing program and outline prioritization methods

– Define goals of the CDI auditing program and demonstrate creation of the auditing tool(s)

– Incorporate audit results into new CDI staff evaluations and use audit results to identify strengths and weakness for long‐term staff


3

Learning Objectives

• At the completion of this educational activity, the learner will be able to:– Illustrate findings of two rounds of CDI audits– Evaluate unexpected outcomes of the auditing program and integrate protocols to improve CDI and coding processes

– Examine CDI staff feedback from auditing program


4

Who Is SCL Health?

• Faith‐based, nonprofit healthcare organization

• 8 hospitals and > 190 ambulatory service centers

• 16,000 full‐time associates and > 500 employed providers

• $2.4 billion health network

• Available beds: 1,813• Acute yearly

admissions: 78,458


5

SCL CDI Staffing Information

• 3 leadership positions– CDI director, CDI staff manager, CDI data manager

• 20 permanent CDI specialists– 62% of CDI staff with 2+ yrs experience have CCDS credential – 2 CDI team leads on‐site– Varying levels of CDI experience (3 months to 11 years)

• Average CDI experience: 4.4 years • 5 CDI support positions

– 2 educators, 2 auditors, 1 coding liaison• Assist with weekly CDI and coding education sessions, monthly staff meetings, and weekly site huddles


6

SCL CDI – How Did We Get Here?

2007

• Formation of CDI teams independently at care sites without formal CDI leadership/expertise• Training and tools different at each site: 3M, JA Thomas, Huron, Claro•Reporting structure different at each site: Quality, care mgmt, HIM

•SCL Health Centralization under HIM•Huron engagement, advisory board engagement • Implementation of 3M CDIS Program •Positions created: Director of CDI, system‐level CDI (manager, educator)

• CDI program brought under system budget • First CDI retreat (groups from 3 states met for the first time)• Positions created: CDI team leads • SCL stewardship involvement – CDI leadership (director) change

•Blue chip spotlight (CDI became a top 5 focus for our health system) •Denver CDI training conference and CDI team retreat• Positions created: 2nd CDI educator, 2 CDI auditors • CDS individual audits with deliverables, other audits (peds, PSI, malnutrition, DRG‐focused)

2012 &2013

2014 &2015

2016 & 2017


7

Identifying Need for CDI Auditing Program

• Varying levels of CDI performance – Barriers: Technical difficulties and IT issues, MD education issues, query follow‐up, side projects

• Unclear baseline and varying knowledge gaps for individual CDI staff

• Process gaps • Staff desired individualized feedback for growth

End Goals of Auditing Program:Maximize performance for individual CDSs

Identify individual and group education needs Identify best ways to improve and grow SCL CDI program


8

CDI Auditing ProcessStarting from scratch


9

Creation of the Auditing Program

• Where do we start? • Who needs to be audited first? Why?• What are the priorities of the audit? • What identifiable outcomes can we measure?• How will we showcase results?


10

Defining SCL CDI Productivity Standards

STANDARDS

Developing Experienced*Best

Practice Initial Reviews/Day 6–7 8 10

Initial Reviews/Month 120–140 160 200

Total Reviews/Month (Initial + Re‐reviews) 360–420 480 600

Total Queries/Month 36–42 48 60

Query Rate 30% 30% 30%

Response Rate 90% 90% 90%

Agreement Rate 90% 90% 90%

3M® CDIS Error Rate < 10% < 10% < 10%

*New CDI will shift from Developing to Experienced after 9 months.


11

Comparing Performance & Productivity

9 Months of CDI Data (Before Audit) Low

PerformerMedium #1Performer

Medium #2Performer

High Performer

Years of CDI Experience 10 years 9 years 8 years 2 years

Avg Initial Reviews/Month 147 164 174 156

Avg Queries/Month 32 53 60 132

Avg Query Rate/Month 22% 33% 35% 89%

Avg Response Rate/Month 94% 95% 93% 98%

Avg Agree Rate/Month 84% 94% 92% 88%

Avg Financial Impact/Month $47,000 $55,200 $100,300 $237,900

There will always be variation in performance.We saw opportunity for performance improvement in

multiple aspects of CDI.


12

Who Gets Audited First?

Critical Audits:Not meeting productivity standards

(new reviews, queries)Noticeable knowledge gaps

Intermediate AuditsLow‐

PriorityAudits


13

Targeted Selection Process

• Chose 30 cases to review based on the following:– High‐risk DRGs (think 312, 313, 948, etc.)– Medical & surgical DRGs with LOS > GLOS– Medical & surgical DRGs with no CC or MCC– Review up to 10 cases with CDI queries

• Assess CDI query compliance• Assess “unable to determine” and “disagreed” query responses

• LOS “maxed out” at 10 days


14

CDI Auditing Process

• Goals:– Review entire record and identify any missed query, reconciliation, education, or process opportunities.

– What counts as a missed opportunity?• Enhanced DRG assignment (CC, MCC, PDX, procedure)• Improved SOI/ROM scoring• Established a 2nd CC/MCC

• Auditors also assessed:– # of reviews compared to length of stay– If the case was followed through to patient discharge – If the case was reconciled per SCL CDI protocol (7 days post‐DC)


15

Creation of the Auditing Tool

• What’s our platform?• How do we deliver results to our CDSs post‐audit?

– Brainstorming sessions, team work, investigating other audits online (not just CDI audits!)

• We initially considered utilizing a traditional spreadsheet …– 2 auditors completed the same audit (reviewed the same records) to assure findings were comparable

– A spreadsheet was NOT going to work for us … see next slide


16

Delivering Results …?


17

So … What Would Work?

• Discussed concerns with the HIM Data team at SCL• They introduced us to Microsoft Access – this tool utilizes information to create PDF documents


18


19


20

Audit Summary Page


21

Audit Summary Page


22

Step One:Auditor #1 chooses 30 cases, reviews all cases, IDs cases w/missed query opportunity.

Step Two:Group meeting: Auditor #1, auditor #2 and CDI data manager meet to discuss cases.

Step Three:Auditor #1 refines cases based on previous meeting, updates audit PDF.

Step Four:Group meeting: Auditors, CDI data manager, and staff manager.

Step Five:CDI staff manager meets with CDS to discuss audit findings.

Step One:Auditor #1 chooses 30 cases, reviews all cases, IDs cases w/missed query opportunity.

Step Two:Group meeting: Auditor #1, auditor #2 and CDI data manager meet to discuss cases.

Step Three:Auditor #1 refines cases based on previous meeting, updates audit PDF.

Step Four:Group meeting: Auditors, CDI data manager, and staff manager.

Step Five:CDI staff manager meets with CDS to discuss audit findings.

Growth!CDI Auditing Process – Summary


23

Current Auditing Timeline

• Round #1 – Completed – High Risk Medium Risk Low Risk

• Round #2 – Completed*– Query‐focused – Was there progress made?

• Decreased missed query opportunities?• Improved follow up/re‐reviews/post‐discharge review?• Improved reconciliation process?

• Future: Plan for yearly audits for each long‐term CDS• New CDS audit plan – occur at 6 months and 1 year • Incorporated into their 6‐month and 1‐year evaluations


24

CDI Audit FindingsRound #1


25

Round #1 – Findings

Classification (30 cases) Breakdown

Total Cases with Any Missed Query Opportunity 266/600

Average # of Cases per CDS with any Missed Query Opportunity 12/30

Average Total Miss Rate per CDS 40.3%Average # of Cases per CDS with Missed DRG Shift Opportunity 9

Average # of Cases per CDS with Missed SOI Opportunity 5.8Average # of Cases per CDS with Missed ROM Opportunity 3.7

Average # of Cases per CDS with Missed 2nd CC Opportunity 2.1Average # of Cases per CDS with Missed 2nd MCC Opportunity 0.5

Large component of missed query opportunities was due to the CDS not following the record through to the discharge summary.


26

Round #1 – Missed Query Topics

Topic # of CasesRespiratory Failure (acute, acute‐on‐chronic, chronic) 30

AKI/CKD 26

Altered Mental Status & Encephalopathy Specificity 26

Sepsis 25

SIRS (Noninfectious, with or without organ dysfunction) 25

Malnutrition 17

CHF 16

Treatment, no Diagnosis (variety of conditions) 14


27

Round #1 – Encouraging Findings

• 20 CDS audits completed – Excellent queries

• Queries sent were found to be warranted, compliant, clear, and included supportive clinical indicators and treatment

• Query templates utilized were appropriate

– Outstanding communication and one‐on‐one interactions with providers

– Excellent peer‐to‐peer communication, even amongst remote staff


28

Round #1 – Needs Improvement

• Identified multiple missed query opportunities in common – Amongst all CDSs, regardless of experience

• Recognized need for improvement in consistently following up on cases – Not following patients all the way through discharge

• Reconciliation process– Timeliness – Escalating cases to CDI and/or coding leadership


29

Suggestions for Our CDSs

• Read the discharge summary when it is available*– May not be available – Auditing helped develop a new process of “post‐discharge

review”

• Educated the CDSs to better utilize our CDI tools and update their reviews accordingly

• Educated the CDSs to utilize the follow‐up tool in our CDI program to help better prioritize patients

• Friendly reminder to not make assumptions during record review

• Re‐educated on use of coding guidelines and Coding Clinics


30

CDI Audit FindingsRound #2


31

Round #2 – Similar but Different Focus

• Goal was (still) to identify any missed query, reconciliation, education, or process improvement opportunities that would have accurately enhanced DRG assignment, improved SOI/ROM scoring, or established a 2nd CC/MCC to better protect us from future outside audits.

• Round #2 audits: Similar objectives with new emphasis:– Queries (chose more cases with queries already placed)– New process: Post‐discharge review– Critical thinking skills – Advanced CDI

• PSI• HAC• Moving principal diagnosis


32

Round #1 vs Round #2 – Findings

Classification (30 Cases) Round 1 Round 2

Total Cases with Any Missed Query Opportunity 266/600 166/600

Average # of Cases with any Missed Query Opportunity 12/30 8.7/30

Average Total Miss Rate 40.3% 29.1%

Average # of Cases with Missed DRG Shift Opportunity 9 6.2

Average # of Cases with a Missed SOI Opportunity 5.8 3.6

Average # of Cases with a Missed ROM Opportunity 3.7 3.4

Average # of Cases with a Missed 2nd CC Opportunity 2.1 0.8

Average # of Cases with a Missed 2nd MCC Opportunity 0.5 0.4

Huge improvements!


33

Missed Query Opportunity Topics

Round #1 Round #2Topic # Cases Topic # CasesRespiratory Failure (all) 30 AMS/Encephalopathy 18

AKI/CKD 26 CHF 17

AMS/Encephalopathy 26 Malnutrition 17

Sepsis 25 Sepsis 12

SIRS 25 SIRS 12

Reconciliation** 25 TX no DX 12

Malnutrition 17 AKI/CKD/ATN 9

CHF 16 Chronic Respiratory Failure 8

TX no DX 14 Alternate PDX 7


34

Round #2 – Encouraging Findings

• New post‐discharge review process was strongly followed, leading to fewer missed query opportunities – Reduced # misses related to reconciliation/coding– Provided more accurate working DRGs

• Excellent queries being sent with strong impact (same)• Outstanding communication and one‐on‐one interactions with

providers (same) • Overall improved review process:

– CDSs better able to follow the record from their last review without having to re‐review the entire record

– Better choice of follow‐up prioritization (low/medium/high)


35

Audit Findings Summary

• Big decreases in missed query opportunities from Round #1 to Round #2!

• Overall process improvement

• Different queries being asked– Move PDX– PSI/HAC


36

Feedback About Auditing ProgramSurvey & results

“We all need people who give us feedback. That’s how we improve.” —Bill Gates


37

Auditing Survey

• SurveyMonkey – Several rating/scaled questions– 3 additional comment questions (free text) – Sent out 3–4 months s/p Round #2 audits – Anonymous

• 60% response rate (12/20 participants)


38

CDS Auditing – Survey Questions

The purpose and scope of my CDI audit was clearly defined (Round 1 and/or Round 2).

Average: 3.5 out of 5

1‐Star: Not clearly defined at all. 2‐Star: Only partially defined. 3‐Star: Defined appropriately. 4‐Star: Defined somewhat above my expectations. 5‐Star: Defined well above my expectations.


39


The audit was presented in a clear, easy‐to‐follow format.

1‐Star: Not clear, not easy to follow. 3‐Star: Appropriately readable. 5‐Star: Extremely clear and easy to follow.

3

Average: 3.8 out of 5


40


There is value and usefulness to the CDI audits and recommendations stemming from the audit – I came away with new ways to approach record review, query topics, and/or process improvement techniques.

1‐Star: No value or usefulness to my CDI practice. 2‐Star: Little value or usefulness to my CDI practice. 3‐Star: Some value or usefulness to my CDI practice. 4‐Star: Strong value or usefulness to my CDI practice. 5‐Star: Strong value or usefulness to my CDI practice and exceeded my expectations. Average: 4.0 out of 5


41

What did you dislike about the auditing process? How could the process be improved?

Free Text – Areas for Improvement

“Intimidating ‐ felt ‘under a microscope’…allow for judgment in prioritizing and time allowance."

“Because it was new it was

intimidating initially however the 2nd was much better.”

“It can be hard to take feedback even when offered in a non‐threatening

way.”

“I feel like the process is a negative thing although it has been discussed multiple times that it is not a negative ‐ but either

way it is a very stressful situation.”

“It didn’t make me cry, but it

made me sweat.”


42

What did you especially like or enjoy about the audit? What positive outcomes did you experience or perceive from the process?

Free Text – Positive Feedback

“Non‐threatening learning

experience.”

“I felt it reminded me about query opportunities I need to think

about.”

“It gave me some different areas to think about when reviewing charts.”

“I like the recommendations on query intro sentences, suggestions on improving. I loved the ‘great job’ ‘great query’ comments so I knew what I did well. Was very good to

have different auditors.”

“It is good to get feedback ‐positive and negative.”


43

CDI Auditing OutcomesExtras


44

CDI Auditing Outcomes

• Identifying topics for CDI and physician education– Great partnership with CDI educators for future education topics• Noninfectious SIRS, acidosis, depression, opioid dependence, etc.

• Query innovation group– Creation of query companion resources– Clinical validation query development and creation

• CDI‐CDS manager interaction– Individualized findings and 1:1 meetings fostered improved professional relationships

• PSI audits, malnutrition audits, etc.


45

CDI Auditing Outcomes

• Development of an ongoing positive relationship with our coding department – Post‐discharge review– Pre‐bill reconciliation process

• One‐on‐one communication with the coder • New process by which record is confirmed – by both CDS & coder – to “match” prior to billing (DRG, SOI, ROM, 2ndCC, 2nd MCC)

• All sites – July 2017


46

SCL CDI Audits – Plans for Growth 2018

• Expand on the CDI auditing program– Recently completed an “unable to determine” audit – Recently completed a mortality audit across the system for 2017

– Plan to audit by insurance coverage – Kaiser audit• Kaiser has dramatically expanded in Denver – recent audit completion to evaluate need for CDI

– Plan for DRG 195/194/193 audit (179/178/177??)– Continue CDS audits (one per year, new staff according to schedule)


47

Conclusion

• Highly recommend some sort of CDI auditing program– Auditors– Peer review– Outside agency

• Utilize the tools you have at your disposal– CDI reports, documentation reports, coding reports

• Develop the tools you need – get creative!• Befriend your coding department – may be able to help

prioritize DRGs (or query topics) that need to be audited• Set program goals and clarify expectations for auditing • Get your staff’s feedback – they are the reason your

program is successful (or not)


48

Thank you! Questions?

[email protected]@sclhs.net

In order to receive your continuing education certificate(s) for this program, you must complete the online evaluation. The link can be found in the continuing education section at the front of the program guide.


auditing your staff for performance improvement: tips to ... · tips to manage underperformers...

Documents