enhancing drug-drug interaction extraction from texts by ... · enhancing drug-drug interaction...
TRANSCRIPT
![Page 1: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/1.jpg)
Enhancing Drug-Drug Interaction Extraction from Texts by
Molecular Structure Information
Masaki Asada, Makoto Miwa, Yutaka Sasaki
Toyota Technological Institute, Japan
1
![Page 2: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/2.jpg)
Introduction
• Our target problem is the extraction of drug-drug interactions (DDIs) from biomedical texts
Grepafloxacin inhibits the metabolism of Theophylline
Mechanism
2
![Page 3: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/3.jpg)
Introduction
• Our target problem is the extraction of drug-drug interactions (DDIs) from biomedical texts
• We investigate the use of external drug database (DrugBank) information in extracting DDIs from texts
• We especially focus on molecular structure information
Grepafloxacin inhibits the metabolism of Theophylline
Mechanism
DrugBankdatabase
3
![Page 4: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/4.jpg)
Method Overview
• We obtain the representations of textual drug pairs using convolutional neural networks (CNNs) and molecular drug pairs using graph convolutional networks (GCNs)
• We concatenate text-based and molecule-based vectors
Text
Molecular structure
GCN
CNN
concat
concat
Grepafloxacin inhibits the metabolism of Theophylline
Grepafloxacin
Theophylline
DDI types
DrugBankDatabase
4
![Page 5: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/5.jpg)
Method
Text Corpus
Grepafloxacininhibits
themetabolism
ofTheophylline
word + positionembeddings
Molecular vector
concat
concat
Theophylline
DDI extraction from texts using molecular structures• Text-based DDI representation• Molecular structure-based DDI representation
DDI types
DrugBankDatabase
Grepafloxacin GCN
Textual vector
CNN
5
![Page 6: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/6.jpg)
Method
Text Corpus
Grepafloxacininhibits
themetabolism
ofTheophylline
word + positionembeddings
Molecular vector
concat
concat
Theophylline
DDI extraction from texts using molecular structures• Text-based DDI representation• Molecular structure-based DDI representation
DDI types
DrugBankDatabase
Grepafloxacin GCN
Textual vector
CNN
6
![Page 7: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/7.jpg)
Method: Text-based DDI Representation
Text Corpus
Grepafloxacininhibits
themetabolism
ofTheophylline
word + positionembeddings
Textual vector
CNN
• Our model for representing textual DDIs is based on the CNN model by Zeng et al. (2014)
• We use word and position embeddings as the input to the convolution layer
• We convert the output of the convolution layer into a fixed-size textual vector
7
![Page 8: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/8.jpg)
Method
Text Corpus
Grepafloxacininhibits
themetabolism
ofTheophylline
word vectorInput sentence
Molecular vector
concat
concat
Theophylline
Predict DDI
Grepafloxacin GCN
Textual vector
DDI extraction from texts using molecular structures• Text-based DDI representation• Molecular structure-based DDI representation
CNN
DrugBankDatabase
8
![Page 9: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/9.jpg)
Method: Molecular Structure-based DDI Representation• We represent drug pairs in molecular graph structures using
GCNs
• We pre-train GCNs using interacting (positive) pairs mentioned in the DrugBank and not mentioned (pseudo negative) pairs in the DrugBank
Molecular vector
GCN interactnot mentioned
prediction
Theophylline
Grepafloxacin
9
![Page 10: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/10.jpg)
Method: Molecular Structure-based DDI Representation
Graph Convolutional Network (GCN) [Li et al. 2016]
We use GCNs to convert a drug molecule graph into a fixed size vector by aggregating node vectors 𝒉𝑣
𝑇
Node 𝑣Edge 𝑒𝑣𝑤
GCN
molecular vectorgraph structure
Node 𝑤
𝒉𝑣𝑡 : node vector
𝑁 𝑣 : neighbors of 𝑣
GRU : gated Recurrent Unit
𝑖, 𝑗 : linear layer
⊙ : element-wise product[… ;… ] : concatenation
𝑨 : learned weight
𝒈
𝒎𝑣𝑡+1 = σ𝑤∈𝑁(𝑣)𝑨𝑒𝑣𝑤𝒉𝑤
𝑡
𝒉𝑣𝑡+1 = GRU([𝒉𝑣
𝑡 ;𝒎𝑣𝑡+1])
10
![Page 11: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/11.jpg)
Method: Molecular Structure-based DDI Representation
Graph Convolutional Network (GCN) [Li et al. 2016]
We use GCNs to convert a drug molecule graph into a fixed size vector by aggregating node vectors 𝒉𝑣
𝑇
Node 𝑣Edge 𝑒𝑣𝑤
GCN
molecular vectorgraph structure
Node 𝑤
𝒉𝑣𝑡 : node vector
𝑁 𝑣 : neighbors of 𝑣
GRU : gated Recurrent Unit
𝑖, 𝑗 : linear layer
⊙ : element-wise product[… ;… ] : concatenation
𝑨 : learned weight
𝒈
𝒎𝑣𝑡+1 = σ𝑤∈𝑁(𝑣)𝑨𝑒𝑣𝑤𝒉𝑤
𝑡
𝒉𝑣𝑡+1 = GRU([𝒉𝑣
𝑡 ;𝒎𝑣𝑡+1])
𝒈 = σ𝑣 𝜎 𝑖( 𝒉𝑣𝑇; 𝒉𝑣
0 ) ⊙ (𝑗 𝒉𝑣𝑇; 𝒉𝑣
0 )
11
![Page 12: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/12.jpg)
Method: DDI Extraction from Texts Using Molecular Structures
Grepafloxacininhibits
themetabolism
ofTheophylline
word + position embeddings
textualvector
CNN
12
![Page 13: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/13.jpg)
Method: DDI Extraction from Texts Using Molecular Structures• Link mentions in text corpus to drug database entries by relaxed
string matching
Grepafloxacininhibits
themetabolism
ofTheophylline
word + position embeddings
TheophyllineDrugBank
Grepafloxacin
textualvector
CNNRelaxedstring
matching
13
![Page 14: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/14.jpg)
Method: DDI Extraction from Texts Using Molecular Structures• Link mentions in text corpus to drug database entries by relaxed
string matching • Obtain molecular vectors via GCNs with fixed parameters
Grepafloxacininhibits
themetabolism
ofTheophylline
word + position embeddings
molecular vector
TheophyllineDrugBank
Grepafloxacin
GCN
textualvector
CNNRelaxedstring
matching
14
![Page 15: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/15.jpg)
Method: DDI Extraction from Texts Using Molecular Structures• Link mentions in text corpus to drug database entries by relaxed
string matching • Obtain molecular vectors via GCNs with fixed parameters• Predict DDIs from concatenated textual and molecular vectors
DDI types
Grepafloxacininhibits
themetabolism
ofTheophylline
word + position embeddings
molecular vector
concat
concat
Theophylline
Grepafloxacin
GCN
textualvector
CNN
DrugBank
Relaxedstring
matching
15
![Page 16: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/16.jpg)
Task Settings
SemEval2013 shared task 9.2The data set is composed of documents annotated with drug mentions and their 4 types of interactions (Mechanism, Effect, Adviceand Interaction) or no interaction
Statistics of the DDI SemEval2013 shared task
16
![Page 17: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/17.jpg)
Data for Pre-training GCNs
• We extracted 255,229 interacting (positive) pairs from DrugBank and generated the same number of pseudo negative pairs by randomly pairing DrugBank drugs
• We deleted drug pairs mentioned in the test set of the text corpus
17
![Page 18: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/18.jpg)
Molecular Structure Features
• To obtain the graph of a drug molecule, we took as input the SMILES string encoding of the molecule from DrugBank and then converted it into the 2D graph structure using RDKit
• For the initial atom (node) vectors, we used randomly embedded vectors for atoms, i.e., C, O, N, …
• We also used 4 bond (edge) types: single, double, triple, and aromatic
18
![Page 19: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/19.jpg)
Differences of Labels in Text and Database Tasks• Interacting drug pairs in database may not appear
as positive instances in the text task
• Text task define 4 detailed types, while database task has one positive type.
Grepafloxacin inhibits the metabolism of Theophylline
Mechanism
While the effect of Grepafloxacin on the metabolism of C.P.A substrates is not evaluated, in vitro data suggested similar effects of Grepafloxacin in Theophylline metabolism
No relation
No relation
19
![Page 20: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/20.jpg)
Training Settings
• Mini-batch training using the Adam optimizer with L2 regularization
• Word embeddings trained by the word2vec tool on the 2014 MEDLINE/PubMed baseline distribution– Skip-gram
– Vocabulary size: 215k
20
![Page 21: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/21.jpg)
Training Settings
Hyper-parameters
Hyper-parameters for text-based model Hyper-parameters for molecule-basedmodel
21
![Page 22: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/22.jpg)
Evaluation on Relaxed String Matching
• How much of drug mentions in texts are linked to DrugBank entries by relaxed string matching?– We lowercased the mentions and the names in the entries and
chose the entries with the most overlaps
– As a result, 92.15% and 93.09% of drug mentions in train and test SemEval2013 data set matched the DrugBank entries
22
![Page 23: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/23.jpg)
68 69 70 71 72 73
Text-Only
Text + Molecular Structure
Zheng et al. 2017
Lim et al. 2018
micro F-score (%)
Evaluation on DDI Extraction from Texts (SemEval2013 Shared Task)• We observe the increase of micro F-score by using
molecular structures
2.39 pp
23
![Page 24: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/24.jpg)
Can molecular structures alone represent DDIs in texts ?
- Low F-score (23.90%)- This might be because the drug pairs that interact can appear in the
textual context that does not describe their interactions
Analysis
Grepafloxacininhibits
themetabolism
ofTheophylline
Molecular vector
concat
Theophylline
interact
Grepafloxacin GCN
Textual vector
CNN
not interact
DrugBankDatabase
24
![Page 25: Enhancing Drug-Drug Interaction Extraction from Texts by ... · Enhancing Drug-Drug Interaction Extraction from Texts by Molecular Structure Information Masaki Asada, Makoto Miwa,](https://reader033.vdocuments.net/reader033/viewer/2022060402/5f0e60e87e708231d43ef520/html5/thumbnails/25.jpg)
Conclusions
• We proposed a novel neural method for DDI extraction using both textual and molecular information
• The molecular information has improved DDI extraction performance
• As future work, we will investigate the use of other information in DrugBank
25