l14-semantic-parsing - github pages · 2020. 12. 2. · 9 (slide credit: acl 2018 tutorial on...
TRANSCRIPT
![Page 1: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/1.jpg)
Semantic Parsing
Fall 20202020-11-13
CMPT 413/825: Natural Language Processing
SFUNatLangLab
Adapted from slides from Pengcheng Yin (with some content from ACL 2018 tutorial on Neural Semantic Parsing by
Pradeep Dasigi, Srini Iyer, Alane Suhr, Matt Gardner, Luke Zettlemoyer)1
![Page 2: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/2.jpg)
What is semantic parsing?
(figure credit: CMU CS 11-747, Pengcheng Yin)
Interpretable by a machine!
Logical form Formal representation
2
![Page 3: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/3.jpg)
What is semantic parsing good for?
• NLP Tasks • Question Answering
• Applications • Natural language interfaces • Dialogue agents • Robots
(figure credit: CMU CS 11-747, Pengcheng Yin)3
![Page 4: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/4.jpg)
Meaning representations
• Machine-executable representations: executable programs to accomplish a task • Meaning representation for semantic annotation: captures the semantics of the
natural language sentence
• Arithmetic expressions • Lambda calculus • Computer Programs:
• SQL / Python / DSLs
(slide credit: CMU CS 11-747, Pengcheng Yin)4
![Page 5: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/5.jpg)
Semantic ParsingSentence
Meaning Representation
Semantic Parser
Executor
Response
5(slide credit: ACL 2018 tutorial on semantic parsing,
Pradeep Dasigi et al)
![Page 6: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/6.jpg)
Semantic Parsing: QAHow many people live in Seattle?
Semantic Parser
Executor
620,778
SELECT Population FROM CityData where City=="Seattle”;
[Wong & Mooney 2007], [Zettlemoyer & Collins 2005, 2007], [Kwiatkowski et.al 2010, 2011], [Liang et.al. 2011],[Berant et.al. 2013,2014],[Reddy et.al, 2014,2016],[Dong and Lapata, 2016] ..... (slide credit: ACL 2018 tutorial on semantic parsing,
Pradeep Dasigi et al)6
![Page 7: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/7.jpg)
Go to the third junction and take a left
(do-seq(do-n-times 3 (move-to forward-loc (do-until (junction current-loc (move-to forward-loc))))
(turn-right))
SemanticParser
[Chen & Mooney 2011][Matuszek et al 2012][Artzi & Zettlemoyer 2013][Mei et.al. 2015][Andreas et al, 2015][Fried at al, 2018] ....
Semantic Parsing: Instructions
7(slide credit: ACL 2018 tutorial on semantic parsing,
Pradeep Dasigi et al)
![Page 8: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/8.jpg)
Language to Meaning
More informative
8(slide credit: ACL 2018 tutorial on semantic parsing,
Pradeep Dasigi et al)
![Page 9: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/9.jpg)
Language to Meaning
More informative
Information Extraction
Recover information about pre-specified
relations and entities
Relation ExtractionExample Task
is a(OBAMA,PRESIDENT )
9(slide credit: ACL 2018 tutorial on semantic parsing,
Pradeep Dasigi et al)
![Page 10: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/10.jpg)
Language to Meaning
More informative
Broad-coverage Semantics
SummarizationExample Task
Obama wins election. Big party in Chicago. Romney a bit down, asks for some tea.
Focus on specific phenomena
10(slide credit: ACL 2018 tutorial on semantic parsing,
Pradeep Dasigi et al)
![Page 11: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/11.jpg)
Language to Meaning
More informative
Semantic Parsing
Recover complete meaning
representation
Database QueryExample Task
What states border Texas?
OklahomaNew Mexico
ArkansasLouisiana
11(slide credit: ACL 2018 tutorial on semantic parsing,
Pradeep Dasigi et al)
![Page 12: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/12.jpg)
Semantic Parsing workflow
Denotation
(slide credit: CMU CS 11-747, Pengcheng Yin)12
![Page 13: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/13.jpg)
Semantic Parsing Components
Goal: learn parameters for a function that gives ascore(x, c, d) that judges how good a derivation d is wrt theutterance x and context c
θ
(figure credit: Percy Liang)13
![Page 14: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/14.jpg)
Supervised learning of Semantic Parsers
(slide credit: CMU CS 11-747, Pengcheng Yin)14
![Page 15: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/15.jpg)
Meaning Representations and Datasets
GeoQuery / ATIS / JOBS WikiSQL / Spider IFTTT
Django, HeartStone, CONCODE, CoNaLa, JuICe
(slide credit: CMU CS 11-747, Pengcheng Yin)15
![Page 16: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/16.jpg)
(slide credit: CMU CS 11-747, Pengcheng Yin)16
![Page 17: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/17.jpg)
Text-to-SQL Tasks
(slide credit: CMU CS 11-747, Pengcheng Yin)17
![Page 18: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/18.jpg)
(slide credit: CMU CS 11-747, Pengcheng Yin)18
![Page 19: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/19.jpg)
Supervised learning of Semantic Parsers
• Train a semantic parser with source natural language utterance and target programs
• Can use general structured prediction methods (similar methods as for constituency parsing and dependency parsing)
(slide credit: CMU CS 11-747, Pengcheng Yin)19
![Page 20: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/20.jpg)
Semantic Parsing as Sequence-to-Sequence Transduction
• Treat the target meaning representation as a sequence of surface tokens • Reduce the (structured prediction) task as another sequence-to-
sequence learning problem
(slide credit: CMU CS 11-747, Pengcheng Yin)[Dong and Lapata, 2016; Jia and Liang, 2016]
20
![Page 21: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/21.jpg)
(slide credit: CMU CS 11-747, Pengcheng Yin)
[Xu et al., 2017; Yu et al., 2018]
21
![Page 22: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/22.jpg)
Structure-aware Decoding for Semantic Parsing (Dong and Lapata, 2016)
22
![Page 23: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/23.jpg)
Structure-aware Decoding for Semantic Parsing (Dong and Lapata, 2016)
23
![Page 24: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/24.jpg)
Coarse-to-Fine Decoding(Dong and Lapata, 2018)
24
![Page 25: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/25.jpg)
Grammar/Syntax-driven Semantic Parsing
(slide credit: CMU CS 11-747, Pengcheng Yin)25
![Page 26: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/26.jpg)
Grammar/Syntax-driven Semantic Parsing
(slide credit: CMU CS 11-747, Pengcheng Yin)26
![Page 27: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/27.jpg)
Grammar/Syntax-driven Semantic Parsing
(slide credit: CMU CS 11-747, Pengcheng Yin)27
![Page 28: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/28.jpg)
Weakly Supervised Semantic Parsing
Learning from denotations
(slide credit: CMU CS 11-747, Pengcheng Yin)28
![Page 29: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/29.jpg)
Semantic Parsing Components
Hypothesize possible logical forms that may match the utterance x and execute to get denotation.
(figure credit: Percy Liang)29
![Page 30: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/30.jpg)
Weakly Supervised Semantic Parsing
(slide credit: CMU CS 11-747, Pengcheng Yin)30
![Page 31: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/31.jpg)
Weakly Supervised Semantic Parsing - Challenges
(slide credit: CMU CS 11-747, Pengcheng Yin)31
![Page 32: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/32.jpg)
Weakly Supervised Semantic Parsing
• Maximum Marginal Likelihood • Structured Learning Methods • Reinforcement Learning
32
![Page 33: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/33.jpg)
Maximum Marginal Likelihood• Given
• We want to optimize
• But the semantic parser defines a distribution over logical forms.
• So we marginalize over logical forms that yield
• could be the set of all valid logical forms, if we are using constrained decoding during training
• Even then, the summation could be intractable!
33(slide credit: ACL 2018 tutorial on semantic parsing,
Pradeep Dasigi et al)
![Page 34: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/34.jpg)
MML: Approximating Y•Perform heuristic search
•Search may be bounded, by length or otherwise
•Y is approximated as a subset of retrieved logical forms
Two options for search:
Online Search Offline Search
Search for consistent logical forms during training, as per model scores
Search for consistent logical forms before training
Candidate set changes as training progresses Candidate set is static
Less efficient More efficient
34(slide credit: ACL 2018 tutorial on semantic parsing,
Pradeep Dasigi et al)
![Page 35: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/35.jpg)
Structured Learning Methods• More commonly used with traditional semantic parsers
• Eg. Margin based models and Latent variable structured perceptron (Zettlemoyer and Collins 2007)
• Typically involve heuristic search over the state space like MML methods • Unlike MML, can use arbitrary cost function • Training typically maximizes margins or minimizes expected risks
35(slide credit: ACL 2018 tutorial on semantic parsing,
Pradeep Dasigi et al)
![Page 36: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/36.jpg)
Reinforcement Learning Methods• Comparison with MML:
• Like MML Y is approximated
• Unlike MML, the approximation is done using sampling techniques
• Comparison with structured learning methods • Like structured learning methods, the reward function can be arbitrary
• Unlike structured learning methods, reward is directly maximized
• Training typically uses policy gradient methods
Example from Liang et al., 2017, using REINFORCE
36(slide credit: ACL 2018 tutorial on semantic parsing,
Pradeep Dasigi et al)
![Page 37: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/37.jpg)
Weakly Supervised Semantic Parsing as Reinforcement Learning
(slide credit: CMU CS 11-747, Pengcheng Yin)37
![Page 38: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/38.jpg)
Maximum Marginal Likelihood
• Intuitively, the gradient from each candidate logical form is weighted by its normalized probability. The more likely the logical form is, the higher the weight of its gradient
(slide credit: CMU CS 11-747, Pengcheng Yin)38
![Page 39: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/39.jpg)
Retrieve and Edit(Hashimoto et al, 2018)
39
![Page 40: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/40.jpg)
Semantic Parsing via Paraphrasing(Berant and Liang, 2014)
Learn to map input to canonical utterance
One-to-one mapping between canonical utterance and logical form
40
![Page 41: L14-semantic-parsing - GitHub Pages · 2020. 12. 2. · 9 (slide credit: ACL 2018 tutorial on semantic parsing, Pradeep Dasigi et al) Language to Meaning More informative Broad-coverage](https://reader036.vdocuments.net/reader036/viewer/2022071605/6141b03ad64cc55ff0755471/html5/thumbnails/41.jpg)
Interactive Semantic Parsing(Wang et al, 2016)
41