Ontology-aware Clinical Abstractive Summarization

Sean MacAvaney*, Sajad Sotudeh*, Arman Cohan, Nazli Goharian, Ross Filice, Ish Talati

To appear at SIGIR 2019 (short paper)* Equal contribution

BackgroundINDICATION : Peripheral edema.COMPARISON : None.

FindingsThe XXXX examination consists of frontal and lateral radiographs of the chest. The cardiomediastinal contours are within normal limits . Pulmonary vascularity is within normal limits . There is a vague right suprahilar density with elevation of the XXXX fissure most XXXX mild subsegmental atelectasis though superimposed infection can not be entirely excluded. The remaining lungs are clear. The visualized osseous structures and upper abdomen are unremarkable.

ImpressionRight upper lobe subsegmental atelectasis.No evidence of heart failure.

Image and note from the Indiana University Chest X-Ray Collection (de-identified)2

There are many components of a radiology report.

Radiological Image(s)

Radiological Note

BackgroundINDICATION : Peripheral edema.COMPARISON : None.

FindingsThe XXXX examination consists of frontal and lateral radiographs of the chest. The cardiomediastinal contours are within normal limits . Pulmonary vascularity is within normal limits . There is a vague right suprahilar density with elevation of the XXXX fissure most XXXX mild subsegmental atelectasis though superimposed infection can not be entirely excluded. The remaining lungs are clear. The visualized osseous structures and upper abdomen are unremarkable.

ImpressionRight upper lobe subsegmental atelectasis.No evidence of heart failure.

3

There are many components of a radiology report.

Radiological NotePertinent information about a

patient’s conditions the procedure

Detailed textual description of observations in the images

can include precise measurements, etc.

Summary of the mostimportant observations

typically ~1/3 length of findings

4

Impressions often miss main points(i.e., they are incomplete)

* From our study on impression completeness, accuracy, and readability. MedStar radiology reports.

Why not apply automatic summarization to build impressions?

5

- Impressions tend to draw conclusions, not just copy sentences- Sentences in findings often

choppy, incomplete

- Reports contain technical jargon, which might not appear in training data

Abstractivegenerates text word-by-worde.g., neural Seq-2-seq networks

Extractiveselect (extract) sentences to build summarye.g., LSA, LexRank, etc.

Hybridboth generates or copies, word-by-worde.g., pointer-generator networks

- Can either generate novel words, or copy from findings (word-by-word)

6

Existing hybrid methods miss more important points than humans do.

7

ExperimentIncorporate medical ontological resources into a hybrid summarization model.

HypothesisExisting hybrid summarization techniques are limited by a lack of domain knowledge.

Medical OntologiesMedical terms occur in both findings and impression

• UMLS• Unified Medical Language System• Including various body parts,

procedures, conditions, etc.

• RadLex• 68,534 radiological concepts• Hierarchical ordering

8

postoperative changes of right total knee arthroplasty. mild lucency around the proximal tibia component and cement bone interface, measuring up to4 mm. no acute displaced fracture. diffuse bony demineralization. no dislocation. vascular calcifications.

1) mild lucency at the tibial component of right total knee arthroplasty. loosening not excluded. 2) no acute displaced fracture or dislocation

Impression

Findings

Pointer-Generator Network

9

postoperative changes of right total knee arthroplasty. mild lucency around the proximal tibia component and cement bone interface, measuring up to4 mm. no acute displaced fracture. diffuse bony demineralization. no dislocation. vascular calcifications.

1) mild lucency at the tibial component of right total knee arthroplasty. loosening not excluded. 2) no acute displaced fracture or dislocation

Impression

Findings

Step 1: Source text encoded into vectore.g., via RNN

Pointer-Generator Network

10

postoperative changes of right total knee arthroplasty. mild lucency around the proximal tibia component and cement bone interface, measuring up to4 mm. no acute displaced fracture. diffuse bony demineralization. no dislocation. vascular calcifications.

1) mild lucency at the tibial component of right total knee arthroplasty. loosening not excluded. 2) no acute displaced fracture or dislocation

Impression

Findings

Step 2: Impression iteratively generated from findings vectore.g., via RNN, using beam search

Pointer-Generator Network

11

postoperative changes of right total knee arthroplasty. mild lucency around the proximal tibia component and cement bone interface, measuring up to4 mm. no acute displaced fracture. diffuse bony demineralization. no dislocation. vascular calcifications.

1) mild lucency at the tibial component of right total knee arthroplasty. loosening not excluded. 2) no acute displaced fracture or dislocation

Impression

Findings

Step 2: Can either:a) generate from known vocabularyb) copy from source via attention weights

1)

generate

Pointer-Generator Network

12

postoperative changes of right total knee arthroplasty. mild lucency around the proximal tibia component and cement bone interface, measuring up to4 mm. no acute displaced fracture. diffuse bony demineralization. no dislocation. vascular calcifications.

1) mild lucency at the tibial component of right total knee arthroplasty. loosening not excluded. 2) no acute displaced fracture or dislocation

Impression

Findings

Step 2: Can either:a) generate from known vocabularyb) copy from source via attention weights

1) mild

generate

Pointer-Generator Network

13

postoperative changes of right total knee arthroplasty. mild lucency around the proximal tibia component and cement bone interface, measuring up to4 mm. no acute displaced fracture. diffuse bony demineralization. no dislocation. vascular calcifications.

1) mild lucency at the tibial component of right total knee arthroplasty. loosening not excluded. 2) no acute displaced fracture or dislocation

Impression

Findings

Step 2: Can either:a) generate from known vocabularyb) copy from source via attention weights

1)

copy

mild lucency

Pointer-Generator Network

14

postoperative changes of right total knee arthroplasty. mild lucency around the proximal tibia component and cement bone interface, measuring up to4 mm. no acute displaced fracture. diffuse bony demineralization. no dislocation. vascular calcifications.

1) mild lucency at the tibial component of right total knee arthroplasty. loosening not excluded. 2) no acute displaced fracture or dislocation

Impression

Findings

Step 2: Can either:a) generate from known vocabularyb) copy from source via attention weights

1)

copy

mild lucency

…

Ontology-Aware Pointer-Generator

15

postoperative changes of right total knee arthroplasty. mild lucency around the proximal tibia component and cement bone interface, measuring up to4 mm. no acute displaced fracture. diffuse bony demineralization. no dislocation. vascular calcifications.

1) mild lucency at the tibial component of right total knee arthroplasty. loosening not excluded. 2) no acute displaced fracture or dislocation

Impression

Findings

Step 1a: Ontology terms encoded into second vectore.g., via RNN, attention-weighted from findings RNN output

…

16

postoperative changes of right total knee arthroplasty. mild lucency around the proximal tibia component and cement bone interface, measuring up to4 mm. no acute displaced fracture. diffuse bony demineralization. no dislocation. vascular calcifications.

1) mild lucency at the tibial component of right total knee arthroplasty. loosening not excluded. 2) no acute displaced fracture or dislocation

Impression

Findings

Step 2: Impression iteratively generated from both vectorse.g., via RNN, using beam search

1)

generate

Ontology-Aware Pointer-Generator

17

postoperative changes of right total knee arthroplasty. mild lucency around the proximal tibia component and cement bone interface, measuring up to4 mm. no acute displaced fracture. diffuse bony demineralization. no dislocation. vascular calcifications.

1) mild lucency at the tibial component of right total knee arthroplasty. loosening not excluded. 2) no acute displaced fracture or dislocation

Impression

Findings

Step 2: Impression iteratively generated from both vectorse.g., via RNN, using beam search

1) mild

copy

Ontology-Aware Pointer-Generator

18

postoperative changes of right total knee arthroplasty. mild lucency around the proximal tibia component and cement bone interface, measuring up to4 mm. no acute displaced fracture. diffuse bony demineralization. no dislocation. vascular calcifications.

1) mild lucency at the tibial component of right total knee arthroplasty. loosening not excluded. 2) no acute displaced fracture or dislocation

Impression

Findings

Step 2: Impression iteratively generated from both vectorse.g., via RNN, using beam search

1) mild lucency

copy

Ontology-Aware Pointer-Generator

19

postoperative changes of right total knee arthroplasty. mild lucency around the proximal tibia component and cement bone interface, measuring up to4 mm. no acute displaced fracture. diffuse bony demineralization. no dislocation. vascular calcifications.

1) mild lucency at the tibial component of right total knee arthroplasty. loosening not excluded. 2) no acute displaced fracture or dislocation

Impression

Findings

Step 2: Impression iteratively generated from both vectorse.g., via RNN, using beam search

1) mild

copy

…lucency

Ontology-Aware Pointer-Generator

Our model tends to focus more on medical terms.

20

(Attention weight comparison between our model and unmodified pointer-generator.)

We evaluate on real radiology reports.• Radiology Department, Medstar Georgetown

University hospital. • 41,066 real-world radiology reports.• Different modalities• Average lengths:• findings: 136.6 Impression: 37.1

• 100 reports for expert evaluation• Evaluated on 5-point scale for completeness, readability,

and accuracy

21

22

Our approach makes big gains in terms of impression completeness.

23

There is still a gap between human parity, though.

Our approach does not reduce readability or accuracy of impressions.

24

OursPG

Automatic (ROUGE) results show there is better n-gram overlap with real reports.

Model Ontology ROUGE-1 ROUGE-2 ROUGE-L

LexRank (Erkan and Radev) - 28.02 14.26 26.24

LSA (Steinberger and Jezek) - 28.16 14.71 26.27

PG (See et al.) - 37.17 22.36 35.45

Background PG (Zhang et al.) - 36.95 22.37 35.68

Ontology PG (ours) UMLS 37.98 23.14 36.67

Ontology PG (ours) RadLex 38.42 23.29 37.02

* Both the UMLS and RadLex ontology PG models are statistically better than theother values (paired t-test, p<0.05). 25

Ontology-aware Clinical Abstractive Summarization

Sean MacAvaney*, Sajad Sotudeh*, Arman Cohan,Nazli Goharian, Ross Filice, Ish Talati

- Existing automatic clinical summarization techniques perform poorly in terms of completeness.- Including ontological resources in summarization models improves completeness without sacrificing readability or accuracy.

Extra Slides

27

28

Comparison between our approach and unmodified pointer-generator

29

Comparison between our approach and human-written impressions

30

Ontology Encoder

enteric tube courses subd... with tip projecting in distal stomach ...

enteric tube tip distal stomach ...

1 . enteric

tube

o

.

cc

Full network diagram

Diagram adapted from Zhang et al (2018)