![Page 1: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/1.jpg)
1
Using NLP to Identify Physician Documentation Opportunities
Session #241, February 23, 2017
Anupam Goel, VP, Clinical Information, Advocate Health Care
![Page 2: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/2.jpg)
2
Speaker Introduction
Anupam Goel, MD
Vice President, Clinical Information
Advocate Health Care
![Page 3: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/3.jpg)
3
Conflict of Interest
Anupam Goel, MD has no real or apparent conflicts of interest to report.
![Page 4: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/4.jpg)
4
Agenda• Learning objectives, STEPS
• Introduction
• The promise and reality of Natural Language Processing (NLP)
• Using NLP to maximize value of human input for appropriate case detection
• Identifying patients with stroke
• Progress to date
• Other opportunities
• Next steps
• Your questions
![Page 5: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/5.jpg)
5
Learning Objectives
• Describe risks and benefits of using natural language processing for identifying gaps in physician documentation
• Recognize how variations in local practice that may impair the ability of natural language processing to address specific use cases at your facility or health system
• Identify methods to display quantitative information about physician documentation quality to highlight patterns and encourage physician behavior change
![Page 6: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/6.jpg)
6
STEPS argument
![Page 7: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/7.jpg)
7
STEPS – Treatment/Clinical• Identify patients sooner to initiate appropriate diagnostic or treatment
algorithms (stroke, CHF)
• NLP must be embedded within a clinical notification workflow that activates a team member to make a change in clinical care
![Page 8: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/8.jpg)
8
STEPS - Savings• In its current state, the technology is probably best at highlighting
patterns for a human to verify.
• For high-risk or publicly reported conditions, many health systems employ multiple FTEs to sift through physician documentation. NLP could help redeploy those resources to complete other tasks.
http://www.integrity-data.com/improve-productivity-employee-satisfaction-deploy-better-technology/
![Page 9: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/9.jpg)
9
Introduction
http://simsa.dsu.dal.ca/tag/orientation
![Page 10: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/10.jpg)
10
What is Natural Language Processing?• Field of computer science and linguistics concerned with the interactions
between computers and human (natural) languages
• Extracting concepts from free-text
• In its most basic form, it is not a tool
• To identify improper diagnosis or treatment
• To judge documentation content quality
• To determine if health care resources are being wasted
![Page 11: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/11.jpg)
11
Common error modalities with NLP• Misinterpreting errors in syntax and grammar
• Misinterpreting errors in content (voice-to-text errors)
• Misinterpreting context around specific terms
http://computerdocnc.com/computer-errors-indian-trail-nc/
![Page 12: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/12.jpg)
12
The promise and reality of NLP
![Page 13: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/13.jpg)
13
Positive impressions of NLP
https://developer.amazon.com/blogs/post/TxC2VHKFEIZ9SG/First-Alexa-Third-Party-Skills-Now-Available-for-Amazon-Echo
![Page 14: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/14.jpg)
14
NLP sophistication
• Bag of words
• Concepts
• Negation, association
• Identifying themes
• Suggesting alternative pathways
![Page 15: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/15.jpg)
15
What’s your gold standard?
https://www.macobserver.com/tmo/article/head-to-head-comparison-of-13-streaming-music-services
![Page 16: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/16.jpg)
16
How do we measure how good NLP might be?
http://gru.stanford.edu/doku.php/tutorials/sdt
![Page 17: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/17.jpg)
17
Vendor promises
• More complex diagnoses (clinical documentation improvement)
• Real-time feedback for documentation quality
• Improved billing accuracy
![Page 18: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/18.jpg)
18
How good do you need NLP to be?• Immediate life-and-death decisions?
• Retrospective reviews
• Billing and documentation improvement
![Page 19: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/19.jpg)
19
“Secret sauce” for NLP• How do you train an NLP algorithm?
• Variations in documentation style
– Pertinent negatives
– Pertinent positives
• “Black box” algorithm
![Page 20: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/20.jpg)
20
Risks for NLP from a clinician’s perspective• Minimal documentation limits the software’s ability to meaningfully
discriminate between “wanted” and “unwanted” cases
• Voice recognition technology may enable more comprehensive documentation, improving NLP’s effectiveness
• Cases that are difficult for humans to distinguish will also be difficult for NLP algorithms to distinguish
![Page 21: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/21.jpg)
21
Using NLP to maximize value for human input for appropriate case detection• Copy-and-paste
• Missing operative report elements
![Page 22: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/22.jpg)
22
Example 1: Copy-and-paste
• Some physicians abuse computer technology to meet documentation requirements
• There is no easy way to manually review every note that is entered in an electronic medical record for “identical-ness”
• Technology could highlight high levels of identical text without determining if the similarity was appropriate or not
![Page 23: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/23.jpg)
23
What’s the driver?
FRAUD
![Page 24: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/24.jpg)
24
How much similarity is too much similarity?• Specialty
• Patients with a prolonged hospital stay
• Some sections of the note hardly change without adverse effects on patient care
• Absolute cutoff of >95% to trigger a manual review
![Page 25: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/25.jpg)
25
Who should be targeted?
• High absolute numbers of similar notes
• High proportion of similar notes
![Page 26: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/26.jpg)
26
So What?
• Is the copy-and-paste appropriate?
• Is there an escalation path?
• What about continued “bad behavior?”
![Page 27: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/27.jpg)
27
Copy and paste
![Page 28: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/28.jpg)
28
Copy and paste
![Page 29: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/29.jpg)
29
Next steps
• Target sections of a note
• Identify “value-adding” team members
![Page 30: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/30.jpg)
30
Example 2: Operative report completeness• Challenge: capture missing elements within an operative report before the
patient is discharged
• Data elements often scattered throughout the surgeon’s operative report or across multiple documents
![Page 31: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/31.jpg)
31
Defining the note “population”
• What constitutes an eligible note?
• What euphemisms are used by different surgeons to imply different required elements?
• EBL
• Blood loss
• Approximate bleeding
![Page 32: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/32.jpg)
32
Data display
• Number of missing elements
• Number of incomplete reports
• Surgeons with the highest percent of incomplete notes
![Page 33: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/33.jpg)
33
Next steps
• Use the operative report extractions to identify quality metrics to drive quality improvement metrics
![Page 34: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/34.jpg)
34
Identifying patients with stroke
![Page 35: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/35.jpg)
35
How do you identify patients with a stroke?• Current state: manual review of patients admitted to specific floors
• About four FTEs
• Limited bandwidth to review patients across the hospital
• Public reporting implications
![Page 36: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/36.jpg)
36
How do you identify a patient with stroke?
![Page 37: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/37.jpg)
37
Order matters
• When reviewing 200 charts, wouldn’t it be great if the first 40 were the ones most likely to include a stroke?
![Page 38: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/38.jpg)
38
Assessing the probability for stroke
• What terms are most consistent with a stroke?
![Page 39: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/39.jpg)
39
Patient presentation changing over time• Day 1: not a stroke
• Day 2: not a stroke
• Day 3: a stroke
![Page 40: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/40.jpg)
40
Progress to date
![Page 41: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/41.jpg)
41
“Sharpening the tool”
• Ongoing improvements to algorithm by using reviewers to provide new “gold standard” information as additional cases are reviewed
![Page 42: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/42.jpg)
42
Other opportunities
![Page 43: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/43.jpg)
43
Next steps• Apply to other clinical scenarios
– Care pathways not followed
– Identify physicians who are outliers based on specific documentation elements
– Flag patients for aggressive follow-up after discharge
![Page 44: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/44.jpg)
44
Summary
• NLP can provide technological support to help augment manual workflows instead of replacing existing workflows
• NLP may be most effective when targeting case-finding
• Gaining proficiency through low-risk scenarios may improve confidence in the technology without large-scale adverse events
![Page 45: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/45.jpg)
45
Summary
• NLP is only as good as the information that is available for review
• Training an NLP algorithm can take time, but early investments can pay off in FTE savings
• Think about simplifying the data display of information from NLP analysis to facilitate action
• Rather than thinking of NLP as a tool with static operating characteristics, consider using new information to continually tweak the algorithm to improve the software’s yield
![Page 46: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/46.jpg)
46
A Summary of How Benefits Were Realized for the Value of Health ITTreatment/Clinical – begin diagnostic and treatment pathways faster
Savings – FTEs allocated to more productive uses
![Page 47: Using NLP to Identify Physician Documentation Opportunities · Anupam Goel, VP, Clinical Information, Advocate Health Care. 2 Speaker Introduction Anupam Goel, MD Vice President,](https://reader033.vdocuments.net/reader033/viewer/2022042016/5e74cc80c675a0159769db59/html5/thumbnails/47.jpg)
47
Questions?
• Email: [email protected]
• Twitter: @anupam1623
• Linkedin: goelanupam
• Please complete the online session evaluation