recent advances in distantly supervised relation extractionwilliam/papers/part1_distant... ·...
TRANSCRIPT
![Page 1: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/1.jpg)
Recent Advances in Distantly Supervised Relation Extraction
William WangDepartment of Computer Science
University of California, Santa Barbara
Joint work with Jiawei Wu, Lei Li, Pengda Qin, Weiran Xu.
CIPS Summer School 2018
Beijing, China
1/20
![Page 2: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/2.jpg)
Agenda
• Motivation
• Challenges in Semi-Supervised Learning • Reinforced Co-Training (Wu et al., NAACL 18)• Reinforced Distant Supervision Relation Extraction
(Qin et al., ACL 18a)
• DSGAN (Qin et al., ACL 18b)• Conclusions
2
![Page 3: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/3.jpg)
Motivation
3
• Most of the existing successful stories of deep learning are still based on supervised learning.
• For example, object recognition, machine translation, text classification.
• However, in many applications, it is not realistic to obtain large amount of labeled data.
• We need to leverage unlabeled data.
![Page 4: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/4.jpg)
A Classic Example of Semi-Supervised Learning
• Co-Training (Blum and Mitchell, 1998)
4
![Page 5: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/5.jpg)
Challenges• The two classifiers in co-training have to be
independent.• Choosing highly-confident self-labeled examples
could be suboptimal.• Sampling bias shift is common.
5
![Page 6: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/6.jpg)
Our Approach: Reinforced SSL
• Assumption: not all the unlabeled data are useful.
• Idea: performance-driven semi-supervised learning that learns an unlabeled data selection policy with RL, instead of using random sampling.
• 1. Partition the unlabeled data space• 2. Train a RL agent to select useful unlabeled data• 3. Reward: change in accuracy on the validation set
6
![Page 7: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/7.jpg)
Reinforcement Learning
7
Environment
𝑎𝑡𝑠$%&
𝑟$%&
𝑠$𝑟$
Agent
Agent Environment
DeepQ-Network UnlabeledData
![Page 8: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/8.jpg)
Reinforced Co-Training(Wu et al., NAACL 2018)
8
![Page 9: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/9.jpg)
Deep Q-Learning
9
![Page 10: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/10.jpg)
Experiment 1: Clickbait Detection
10
![Page 11: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/11.jpg)
Experiment 1: Clickbait Detection
11
![Page 12: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/12.jpg)
Experiment 2: Generic Text Classification
12
![Page 13: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/13.jpg)
Experiment 2: Generic Text Classification
13
![Page 14: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/14.jpg)
Conclusion
• We proposed a novel RL framework for semi-supervised learning• Strong results in SSL text classification• Also showed effectiveness in relation extraction
14
![Page 15: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/15.jpg)
Deep Reinforcement Learning for Distantly Supervised Relation Extraction
15
![Page 16: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/16.jpg)
Outline
• Motivation• Algorithm• Experiments• Conclusion
16
![Page 17: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/17.jpg)
Outline
• Motivation• Algorithm• Experiments• Conclusion
17
![Page 18: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/18.jpg)
Plain Text Corpus(Unstructured Info)
Classifier Entity-relation Triple(Structured Info)
Relation Type withLabeled Dataset
Relation Type withoutLabeled Dataset
18
Relation Extraction
![Page 19: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/19.jpg)
“If two entities participate in a relation, any sentencethat contains those two entities might express thatrelation.” (Mintz, 2009)
19
Distant Supervision
![Page 20: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/20.jpg)
Data(x): <Belgium, Nijlen>Label(y): /location/contains
Target Corpus(Unlabeled)
DS Label(y): /location/contains
Nijlen is a municipalitylocated in the Belgianprovince of Antwerp.
20
Distant Supervision
DS Data(x):
![Page 21: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/21.jpg)
v Within-Sentence-Bag Level
v Entity-Pair Level
§ Hoffmann et al., ACL 2011.§ Surdean et al., ACL 2012.§ Zeng et al., ACL 2015. § Li et al., ACL 2016.
§ None
21
Wrong Labeling
![Page 22: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/22.jpg)
§ Place_of_Death
i. Some New York city mayors – William O’Dwyer, Vincent R. Impellitteri and Abraham Beame – were born abroad.
ii. Plenty of local officials have, too, including two New York city mayors, James J. Walker, in 1932, and William O’Dwyer, in 1950.
22
Wrong Labeling
v Entity-Pair Level
![Page 23: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/23.jpg)
v Most of entity pairs only have several sentences
1 Sentence55%2 Sentence
32%
Other4%
23
Wrong Labeling
v Lots of entity pairs have repetitive sentences
![Page 24: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/24.jpg)
Outline
• Motivation• Algorithm• Experiments• Conclusion
24
![Page 25: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/25.jpg)
Sentence-Level Indicator
Entity-Pair Level Wrong Labeling Problem
Learn a Policy to Denoise theTraining Data
25
General Purpose and Offline Process
Without Supervised Information
Requirements
![Page 26: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/26.jpg)
Negative set
Positive set
Negative set
Positive setFalse Positive Policy Based Agent
𝑅𝑒𝑤𝑎𝑟𝑑
𝐴𝑐𝑡𝑖𝑜𝑛
Classifier𝑇𝑟𝑎𝑖𝑛
False Positive
DS Dataset Cleaned Dataset
26
Overview
![Page 27: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/27.jpg)
v State
27
v Action
v Reward§ ???
Deep Reinforcement Learning
§ Sentence vector§ The average vector of previous removed sentences
§ Remove & retain
![Page 28: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/28.jpg)
v One relation type has an agent
28
v Sentence-level
v Split into training set and validation set
Deep Reinforcement Learning
§ Positive: Distantly-supervised positive sentences§ Negative: Randomly sampled
![Page 29: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/29.jpg)
RL Agent Train
29
Deep Reinforcement Learning
TrainRelationClassifier
RelationClassifier
𝐹&34&
𝐹&3× +𝓡3 + ×(−𝓡3)
Noisydataset𝑃$=>3
Cleaneddataset
Cleaneddataset
Removedpart
Removedpart
𝓡3 = 𝛼(𝐹&3 - 𝐹&34& )
RL Agent
Epoch i-1
EpochiNoisy
dataset𝑃$=>3
+𝑁$=>3
𝑁$=>3 +
![Page 30: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/30.jpg)
30
Positive Set Negative Set
§ Accurate
§ Steady
§ Fast
Reward
False Positive
§ Obvious
![Page 31: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/31.jpg)
31
Positive Set Negative Set
RelationClassifier
PreTrain RelationClassifier
Train
Calculate
𝐹&
Epoch 𝑖
Reward
False Positive
False Positive
Positive Negative
![Page 32: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/32.jpg)
Outline
• Motivation• Algorithm• Experiments• Conclusion
32
![Page 33: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/33.jpg)
v Dataset: SemEval-2010 Task 8v True Positive: Cause-Effectv False Positive: Other
Evaluation on a Synthetic NoiseDataset
v True Positive + False Positive: 1331 samples
33
![Page 34: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/34.jpg)
0.64
0.645
0.65
0.655
0.66
0.665
0.67
0.675
0.68
0.685
0 10 20 30 40 50 60 70 80 90 100
F1Score
Epoch
200 FPs in 1331 Samples
Evaluation on a Synthetic NoiseDataset
(198/388)
(197/339)
(195/308)(180/279) (179/260)
34
False PositiveRemoved Part
![Page 35: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/35.jpg)
Evaluation on a Synthetic NoiseDataset
0.68
0.69
0.7
0.71
0.72
0.73
0.74
0.75
0 10 20 30 40 50 60 70 80 90 100
F1Score
Epoch
0 FPs in 1331 samples
(0/258)
(0/150)
(0/121)
(0/59)(0/32)
35
![Page 36: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/36.jpg)
36
v CNN+ONE, PCNN+ONE§ Distant supervision for relation extraction via piecewise
convolutional neural networks. (Zeng et al., 2016)
v CNN+ATT, PCNN+ATT§ Neural relation extraction with selective attention over
instances. (Lin et al., 2016)
Distant Supervisionon NYT Freebase Dataset
![Page 37: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/37.jpg)
37
Distant Supervision
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4
CNN-basedCNN+ONE
CNN+ONE_RL
CNN+ATT
CNN+ATT_RL
![Page 38: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/38.jpg)
38
Distant Supervision
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4
PCNN-based
PCNN+ONE
PCNN+ONE_RL
PCNN+ATT
PCNN+ATT_RL
![Page 39: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/39.jpg)
Outline
• Motivation• Algorithm• Experiments• Conclusion
39
![Page 40: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/40.jpg)
vWe propose a deep reinforcement learningmethod for robust distant supervision relationExtraction.
v Our method is model-agnostic.
v Our method boost the performance of recentlyproposed neural relation extractors.
Conclusion
40
![Page 41: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/41.jpg)
DSGAN: Adversarial Learning for DenoisingDistantly Supervised Relation Extraction
(Qin et al., ACL 2018b)
41
![Page 42: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/42.jpg)
42
DS data space
DS Positive Data
DS Negative Data
Distant Supervision Data Distribution
![Page 43: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/43.jpg)
43
DS data space
DS False PositiveData
DS True Positive Data
DS Negative Data
The Decision Boundaryof DS Data
The DesiredDecision Boundary
Data Distribution
![Page 44: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/44.jpg)
44
Adversarial Training
True Positive
False Positive
Noisy Positive Set
Generator
True Positive
False Positive
Label1
Label1
Label0
Label1
![Page 45: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/45.jpg)
45
DSGAN (Qin et al., ACL 2018b)
Epoch 𝐢
𝐁𝐚𝐠𝐤4𝟏
𝐁𝐚𝐠𝐤
𝐁𝐚𝐠𝐤%𝟏
DSPositiveDataset
𝑠&𝑠I𝑠J
…
𝑠K
G
…
𝑝& = 0.57
𝑝I = 0.02
𝑝J = 0.83
𝑝T = 0.26
𝑝V = 0.90
Sampling
𝐥𝐚𝐛𝐞𝐥 = 𝟏
𝐥𝐚𝐛𝐞𝐥 = 𝟎D
𝒓𝒆𝒘𝒂𝒓𝒅
DS positivedataset
𝐥𝐚𝐛𝐞𝐥 = 𝟏
𝐥𝐚𝐛𝐞𝐥 = 𝟎
Pre-training
𝑝K= 0.7
High-confidencesamples
Low-confidencesamples
Generator Discriminator
DS negativedataset
DS negativedataset
![Page 46: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/46.jpg)
v Sentence-Level Noise Reduction
46
v Training Without Supervised Information
v Model-Agnostic
Characteristics
![Page 47: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/47.jpg)
47
Distant Supervision Relation Extraction
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4
CNN-basedCNN+ONECNN+ONE_GANsCNN+ATTCNN+ATT_GANs
![Page 48: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/48.jpg)
48
Distant Supervision Relation Extraction
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4
PCNN-basedPCNN+ONEPCNN+ONE_GANsPCNN+ATTPCNN+ATT_GANs
![Page 49: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/49.jpg)
Conclusion
• We introduce Reinforced Co-Training, a new approach that combines reinforcement learning and semi-supervised learning.• We show that in weakly-supervised relation
extraction, reinforcement learning can be utilized to de-noise the training signals.• Adversarial learning serves as a joint learning
framework, and it can also be applied to de-noising distantly supervised IE data.
49
![Page 50: Recent Advances in Distantly Supervised Relation Extractionwilliam/papers/Part1_Distant... · 2018-08-01 · R. Impellitteri and Abraham Beame – were born abroad. ii. Plenty of](https://reader033.vdocuments.net/reader033/viewer/2022041911/5e67a7eac22ddb3d8115c6d2/html5/thumbnails/50.jpg)
50
Thanks!
http://nlp.cs.ucsb.edu