improving patient’s electronic health record comprehension with noteaid balaji polepalli ramesh,...

IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID

Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu

Outline

Background The NoteAid System &

Evaluation Results Discussion Conclusion & Future Work

Background

Patients reading their clinical notes has the potential to Enhance medical understanding Improve healthcare management and

outcome

Background

“A patient with hx of active tobacco abuse, bronchitis, and psoriasis presented to ED earlier today with c/o SOB, mild wheezing, chest congestion and chills”

The NoteAid System

NoteAid system - automatically Identifies clinically relevant concepts Links concepts to their definitions

The NoteAid System

The patient will be scheduled for a repeat EGD in one year for surveillance purposes of Barrett esophagus. From a GI standpoint, we recommend to proceed with bariatric surgery. However, he will need to continue daily PPI administration to maximize acid reduction. Otherwise, there are no additional recommendations.

Knowledge Resources

(Medline Plus, UMLS, Wiki)

NLP Approaches

NoteAid

The NoteAid System

The patient will be scheduled for a repeat EGD in one year for surveillance purposes of Barrett esophagus. From a GI standpoint, we recommend to proceed with bariatric surgery. However, he will need to continue daily PPI administration to maximize acid reduction. Otherwise, there are no additional recommendations.

EGD – Acronym for Esophagogastroduodenoscopy.It is a test to examine the lining of the esophagus (the tube that connects your throat to your stomach), stomach, and first part of the small intestine. It is done with a small camera (flexible endoscope) that is inserted down the throat. EGD may be done if you have symptoms that are new, cannot be explained, or are not responding to treatment, such as: Black or tarry stool, Feeling full sooner than normal or after eating less than usual, Swallowing problems or pain with swallowing, heart burns and others.

Knowledge Resources

(Medline Plus, UMLS, Wiki)

NLP Approaches

NoteAid

The NoteAid System

A knowledge resource MedlinePlus UMLS – Unified Medical Language System Wikipedia

NLP approaches

NLP Approach

Two system components – Concept Identifier

Process input text and identify clinically relevant concept

Definition LocatorFetch definitions from MedlinePlus, UMLS and

Wikipedia

Input Split

Sentences

Mapped

Phrases

Mapped

Text

KnowledgeLinked Text

SentenceSplitter

ConceptMapper

ConceptFilter

DefinitionFetcher

COMPONENT 1 COMPONENT 2

CONCEPT IDENTIFIER DEFINITION LOCATOR

NLP Approach


Process input text and identify clinically relevant concepts


Wikipedia

Input Split

Sentences

Mapped

Phrases

Mapped

Text


SentenceSplitter

ConceptMapper

ConceptFilter

DefinitionFetcher



NLP Approach


Process input text and identify clinically relevant concept


Wikipedia

Input Split

Sentences

Mapped

Phrases

Mapped

Text


SentenceSplitter

ConceptMapper

ConceptFilter

DefinitionFetcher



Evaluation

Four NoteAid implementations MedlinePlus UMLS Wikipedia Hybrid (MedlinePlus+UMLS+Wikipedia)

Evaluation Data

From the Pittsburgh NLP repository 20 Progress Note reports 20 Discharge Summary reports

Subjects

The Amazon Mechanic Turk Has shown to be reliable for medical

annotations and evaluations Recruited 64 subjects

8 systems (4 implementations X 2 types of EHR notes)

8 subjects per system 59 subjects completed the evaluation

Demographic Information

Evaluation Process

Each subject read 20 EHR notes before and after NoteAid implementation Randomly assigned to an implementation Self-reported comprehension on a scale of

1 to 5

Readability

Flesch-Kincaid grade level (Grade Level)

Evaluation Data Statistics

Type Discharge Summaries

Progress Notes

No of Reports 20 20

Total (Avg) # of sentences

355 (17.8) 473 (23.7)

Total (Avg) # of words

2362 (118) 4862 (243)

Avg Flesch-Kincaid Grade

Level

8.8 9.8

Readability and Self-Reported Comprehension

Grade Level correlated with self-reported comprehension score (ρ=-0.47, p<0.05, Spearman rank

correlation)

Results

Average comprehension scores before and after each implementation

*p<0.05, Non-parametric Wilcoxon signed-rank test

System

Discharge Summaries Progress Notes

Before After diff (%) Before After diff (%)

MedlinePlus

3.52 3.49-0.03 (-

0.9)3.18 2.86

-0.32 (-10.1)

UMLS 3.80 3.81 0.01 (0.3) 3.75 4.01 0.26 (7.0)

Wiki 3.57 4.140.57

(16.0)3.45 4.53

1.08* (31.3)

Hybrid

3.86 4.02 0.16 (4.1) 3.40 4.54 1.14 (33.5)

Number of Concepts Identified Total number of concepts that were

recognized by three different NoteAid implementationsSystem Discharge

SummariesProgress Notes

MedlinePlus 37 53

UMLS 171 362

Wiki 190 427

Readability

UMLS Definition – Coagulopathy

“Hemorrhagic and thrombotic disorders that occur as a consequence of abnormalities in blood coagulation due to a variety of factors such as COAGULATION PROTEIN DISORDERS; BLOOD PLATELET DISORDERS; BLOOD

PROTEIN DISORDERS or nutritional conditions” Wiki Definition – Coagulopathy

“Coagulopathy is a condition in which the blood’s ability to clot is impaired. This condition can cause prolonged or excessive bleeding, which may occur spontaneously or following an injury or medical and dental procedures. The normal clotting process depends on the interplay of

various proteins in the blood”

Discussion

Text readability correlated with the comprehension scores

Wiki performed the best Content coverage Readability

Limitations

Limitation Lay people not patients performed evaluation Order-effect bias Score subjects’ EHR note comprehension but

not to what extent they accurately comprehended the EHR note content

Conclusion and Future Work

NoteAid improved EHR note self-reported comprehension

Future Work Improve concept coverage and filtering Evaluate quality of the definitions Evaluate content comprehension Evaluate system in which patient read their

own EHR notes

Acknowledgements

1R01GM095476 University of Massachusetts Medical

School National Center for Advancing

Translational Sciences of the National Institutes of Health under award number UL1TR000161

http://clinicalnotesaid.org

Thank You and Questions

improving patient’s electronic health record comprehension with noteaid balaji polepalli ramesh,...

Documents