using nlp to identify physician documentation opportunities · anupam goel, vp, clinical...
TRANSCRIPT
1
Using NLP to Identify Physician Documentation Opportunities
Session #241, February 23, 2017
Anupam Goel, VP, Clinical Information, Advocate Health Care
2
Speaker Introduction
Anupam Goel, MD
Vice President, Clinical Information
Advocate Health Care
3
Conflict of Interest
Anupam Goel, MD has no real or apparent conflicts of interest to report.
4
Agenda• Learning objectives, STEPS
• Introduction
• The promise and reality of Natural Language Processing (NLP)
• Using NLP to maximize value of human input for appropriate case detection
• Identifying patients with stroke
• Progress to date
• Other opportunities
• Next steps
• Your questions
5
Learning Objectives
• Describe risks and benefits of using natural language processing for identifying gaps in physician documentation
• Recognize how variations in local practice that may impair the ability of natural language processing to address specific use cases at your facility or health system
• Identify methods to display quantitative information about physician documentation quality to highlight patterns and encourage physician behavior change
6
STEPS argument
7
STEPS – Treatment/Clinical• Identify patients sooner to initiate appropriate diagnostic or treatment
algorithms (stroke, CHF)
• NLP must be embedded within a clinical notification workflow that activates a team member to make a change in clinical care
8
STEPS - Savings• In its current state, the technology is probably best at highlighting
patterns for a human to verify.
• For high-risk or publicly reported conditions, many health systems employ multiple FTEs to sift through physician documentation. NLP could help redeploy those resources to complete other tasks.
http://www.integrity-data.com/improve-productivity-employee-satisfaction-deploy-better-technology/
9
Introduction
http://simsa.dsu.dal.ca/tag/orientation
10
What is Natural Language Processing?• Field of computer science and linguistics concerned with the interactions
between computers and human (natural) languages
• Extracting concepts from free-text
• In its most basic form, it is not a tool
• To identify improper diagnosis or treatment
• To judge documentation content quality
• To determine if health care resources are being wasted
11
Common error modalities with NLP• Misinterpreting errors in syntax and grammar
• Misinterpreting errors in content (voice-to-text errors)
• Misinterpreting context around specific terms
http://computerdocnc.com/computer-errors-indian-trail-nc/
12
The promise and reality of NLP
13
Positive impressions of NLP
https://developer.amazon.com/blogs/post/TxC2VHKFEIZ9SG/First-Alexa-Third-Party-Skills-Now-Available-for-Amazon-Echo
14
NLP sophistication
• Bag of words
• Concepts
• Negation, association
• Identifying themes
• Suggesting alternative pathways
15
What’s your gold standard?
https://www.macobserver.com/tmo/article/head-to-head-comparison-of-13-streaming-music-services
16
How do we measure how good NLP might be?
http://gru.stanford.edu/doku.php/tutorials/sdt
17
Vendor promises
• More complex diagnoses (clinical documentation improvement)
• Real-time feedback for documentation quality
• Improved billing accuracy
18
How good do you need NLP to be?• Immediate life-and-death decisions?
• Retrospective reviews
• Billing and documentation improvement
19
“Secret sauce” for NLP• How do you train an NLP algorithm?
• Variations in documentation style
– Pertinent negatives
– Pertinent positives
• “Black box” algorithm
20
Risks for NLP from a clinician’s perspective• Minimal documentation limits the software’s ability to meaningfully
discriminate between “wanted” and “unwanted” cases
• Voice recognition technology may enable more comprehensive documentation, improving NLP’s effectiveness
• Cases that are difficult for humans to distinguish will also be difficult for NLP algorithms to distinguish
21
Using NLP to maximize value for human input for appropriate case detection• Copy-and-paste
• Missing operative report elements
22
Example 1: Copy-and-paste
• Some physicians abuse computer technology to meet documentation requirements
• There is no easy way to manually review every note that is entered in an electronic medical record for “identical-ness”
• Technology could highlight high levels of identical text without determining if the similarity was appropriate or not
23
What’s the driver?
FRAUD
24
How much similarity is too much similarity?• Specialty
• Patients with a prolonged hospital stay
• Some sections of the note hardly change without adverse effects on patient care
• Absolute cutoff of >95% to trigger a manual review
25
Who should be targeted?
• High absolute numbers of similar notes
• High proportion of similar notes
26
So What?
• Is the copy-and-paste appropriate?
• Is there an escalation path?
• What about continued “bad behavior?”
27
Copy and paste
28
Copy and paste
29
Next steps
• Target sections of a note
• Identify “value-adding” team members
30
Example 2: Operative report completeness• Challenge: capture missing elements within an operative report before the
patient is discharged
• Data elements often scattered throughout the surgeon’s operative report or across multiple documents
31
Defining the note “population”
• What constitutes an eligible note?
• What euphemisms are used by different surgeons to imply different required elements?
• EBL
• Blood loss
• Approximate bleeding
32
Data display
• Number of missing elements
• Number of incomplete reports
• Surgeons with the highest percent of incomplete notes
33
Next steps
• Use the operative report extractions to identify quality metrics to drive quality improvement metrics
34
Identifying patients with stroke
35
How do you identify patients with a stroke?• Current state: manual review of patients admitted to specific floors
• About four FTEs
• Limited bandwidth to review patients across the hospital
• Public reporting implications
36
How do you identify a patient with stroke?
37
Order matters
• When reviewing 200 charts, wouldn’t it be great if the first 40 were the ones most likely to include a stroke?
38
Assessing the probability for stroke
• What terms are most consistent with a stroke?
39
Patient presentation changing over time• Day 1: not a stroke
• Day 2: not a stroke
• Day 3: a stroke
40
Progress to date
41
“Sharpening the tool”
• Ongoing improvements to algorithm by using reviewers to provide new “gold standard” information as additional cases are reviewed
42
Other opportunities
43
Next steps• Apply to other clinical scenarios
– Care pathways not followed
– Identify physicians who are outliers based on specific documentation elements
– Flag patients for aggressive follow-up after discharge
44
Summary
• NLP can provide technological support to help augment manual workflows instead of replacing existing workflows
• NLP may be most effective when targeting case-finding
• Gaining proficiency through low-risk scenarios may improve confidence in the technology without large-scale adverse events
45
Summary
• NLP is only as good as the information that is available for review
• Training an NLP algorithm can take time, but early investments can pay off in FTE savings
• Think about simplifying the data display of information from NLP analysis to facilitate action
• Rather than thinking of NLP as a tool with static operating characteristics, consider using new information to continually tweak the algorithm to improve the software’s yield
46
A Summary of How Benefits Were Realized for the Value of Health ITTreatment/Clinical – begin diagnostic and treatment pathways faster
Savings – FTEs allocated to more productive uses
47
Questions?
• Email: [email protected]
• Twitter: @anupam1623
• Linkedin: goelanupam
• Please complete the online session evaluation