![Page 1: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/1.jpg)
Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning
Yoshihiro Nakashima
Standardization and Management Group, Data Science,
Development
Astellas Pharma Inc.
September 4, 2018
![Page 2: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/2.jpg)
Agenda 2
I Introduction
II
III
IV
VMedDRA Mapping Process
Characteristics of Reported
Term and MedDRA
Building Deep Neural
Network
Summary
![Page 3: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/3.jpg)
Introduction
![Page 4: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/4.jpg)
Introduction
Today, it is said that artificial intelligence (AI) is the third boom, and not a day passed without hearing
AI. This boom is led by deep neural network (DNN) using technique of deep learning. DNN made
tasks (e.g. image recognition, natural language processing, speech recognition) more advanced than
ever.
In this presentation, I will implement deep learning to map reported term for adverse event into
MedDRA. The procedure consists of two steps. First, converts each word in reported term into
numeric vector produced by word2vec using Wikipedia data. Second, trains DNN using words
represented by numeric vectors. Through this implementation, I would like to examine the applicability
of AI technology to clinical trials.
4
![Page 5: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/5.jpg)
MedDRA Mapping Process
![Page 6: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/6.jpg)
MedDRA Mapping Process: Over View 6
Cramps in the
calves (right leg)
Investigator
eCRF: Reported Term
Muscle spasms
Rawdata: PT
Today’s Topic
![Page 7: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/7.jpg)
MedDRA Mapping Process: Basic 7
eCRF: Reported Term
Rawdata: PT
ManualMatching Existing
List (synonym)
![Page 8: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/8.jpg)
MedDRA Mapping Process: Today’s Topic 8
eCRF: Reported Term
Pre-Processing
Deep Neural Network
Word Embedding
Rawdata: PT
Absorb External Data
(e.g. Wikipedia)
![Page 9: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/9.jpg)
Pre-Processing
Word Segmentation
Original: “Cramps in the calves (right leg)”
-> Replace text: “cramps in the calves right leg ”
-> Separate by space: “cramps”, “in” “the”, ”calves”, “right”, “leg”
Drop Stop-words
“cramps”, “in” “the”, ”calves”, “right”, “leg”
-> “cramps”, ”calves”, “right”, “leg”
(#words = 4)
9
![Page 10: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/10.jpg)
Word Embedding
Vectorize the word using Word2Vec to “calculate” words (i.e.
Plus, Minus, Distance)
10
Dictionary Vocabulary Embeddable Words in
Reported Term / All
Words
Wikipedia*1 2.2 million 2300 / 2600
Google News*2 3 million 2000 / 2600
*1: https://dumps.wikimedia.org/enwiki/latest/
*2: https://code.google.com/archive/p/word2vec/
![Page 11: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/11.jpg)
Deep Neural Network
Structure of Neural Network
Simple Neural Network (NN)
• No structure
• Each layers are fully connected
Bidirectional Recurrent Neural Network (BRNN)
• Treat sequence of the data
• Network is trained by Bidirectional information (Past and Future
information)
11
![Page 12: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/12.jpg)
Characteristics of Reported Term and MedDRA
![Page 13: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/13.jpg)
Distribution of Pattern and Record 13
Pattern: Unique Pre-processed Reported Term
Record: CRF Record
![Page 14: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/14.jpg)
Distribution of #Words per Pattern
0
5
10
15
20
25
30
35
40
1 2 3 4 5 6 7≧
%
#Words / Pattern
![Page 15: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/15.jpg)
Building Deep Neural Network
![Page 16: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/16.jpg)
Split Data into Training Data, Validation Data, and Test Data
Split Data
• Training Data
• Validation Data
• Test Data (at least 1 reported term per each PT)
Build Neural Network
• Construct neural network using training data and
validation data
• Evaluate neural network by test data
16
![Page 17: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/17.jpg)
Accuracy of each Structure by Original Data 17
Accuracy for Test: 43% Accuracy for Test: 42%
BRNN Simple NN
![Page 18: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/18.jpg)
Improve Training
Data Acquisition
• Medical history data
• MedDRA dictionary (LLT is linked to PT)
Data Augmentation
Image data
• Rotation, Shift, etc.
Text data
• Replace with Synonyms
• Back translating
• Permutate the words
18
![Page 19: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/19.jpg)
Accuracy of BRNN by each Training Data 19
Accuracy for Test: 59%
Permutated Training Data Original Training Data
Accuracy for Test: 43%
![Page 20: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/20.jpg)
Accuracy of BRNN by Test Data 20
Improved accuracy using permutated training data
But is the accuracy the best measurement to your
purpose?
#Words
0
20
40
60
80
100
1 2 3 4 5 6
Permutated Not Permutated
![Page 21: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/21.jpg)
Summary
• Implemented MedDRA mapping process using DNN.
• BRNN is better than simple NN.
• Word permutation improved test accuracy, but still over-fitting.
• Consider process and performance measurement and based
on your purpose before implicate to real world.
21
![Page 22: Mapping Reported Term for the Adverse Event into MedDRA ... · Mapping Reported Term for the Adverse Event into MedDRA Using Deep Learning Yoshihiro Nakashima Standardization and](https://reader033.vdocuments.net/reader033/viewer/2022042214/5eb954072644aa170744a850/html5/thumbnails/22.jpg)
References
• 斎藤康毅, ゼロから作るDeep Learning 2 -自然言語処理編, オライリージャパン, 2018
• Keras document, https://keras.io/
• Mike Schuster and Kuldip K. Paliwal, Bidirectional Recurrent Neural
Networks, IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 45, NO. 11,
NOVEMBER 1997
• Rico Sennrich, Barry Haddow, and Alexandra Birch, Improving neural
machine translation models with monolingual data, arXiv preprint
arXiv:1511.06709, 2015.
• Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean, Efficient Estimation of
Word Representations in Vector Space, arXiv: 1301.3781, 2013