natural language processing - uppsala universitynivre/master/nlp-conparsing.pdf · 2017-10-15 ·...
TRANSCRIPT
![Page 1: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/1.jpg)
Natural Language Processing
Constituency Parsing
Joakim Nivre
Uppsala UniversityDepartment of Linguistics and Philology
Natural Language Processing 1(10)
![Page 2: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/2.jpg)
Context-Free Grammar
! A context-free grammar (CFG) consists of! a finite set of nonterminal symbols! a finite set of terminal symbols! a distinguished nonterminal symbol S (for Start)! a finite set of rules of the form
A → α
where A is a nonterminal and α is a (possibly empty)sequence of nonterminal and terminal symbols
Natural Language Processing 2(10)
![Page 3: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/3.jpg)
Context-Free Grammar
S → NP VP Punct PP → Prep NP Adj → EconomicVP → VP PP Verb → had Adj → littleVP → Verb NP Noun → news Adj → financialNP → NP PP Noun → effect Prep → onNP → Adj Noun Noun → markets Punct → .
Adj
Economic
✟✟Noun
news
❍❍✑
✑✑✑
✑✑✑✑
✑✑✑✑
NP
Verb
had
$$
$$$$
$VP
S
Adj
little
✟✟Noun
effect
❍❍✧✧✧✧✧
❍❍
NP
NP
Prep
on
$$$
❍❍PP
Adj
financial
✟✟Noun
markets
❍❍❍❍NP Punct
.
◗◗
◗◗
◗◗
◗◗
◗◗
◗◗
Natural Language Processing 3(10)
![Page 4: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/4.jpg)
CFG Parsing
! Input:! Sentence: S = w1, . . . ,wn! Grammar: G
! Output:! Phrase structure tree for S generated by G
! Parsing:! Construct a derivation of S in G! Read off phrase structure tree from derivation
Natural Language Processing 4(10)
![Page 5: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/5.jpg)
Quiz
S → NP VP Punct PP → Prep NP Adj → EconomicVP → VP PP Verb → had Adj → littleVP → Verb NP Noun → news Adj → financialNP → NP PP Noun → effect Prep → onNP → Adj Noun Noun → markets Punct → .
! Which sentences can be derived in the grammar above?1. Economic news had effect .2. Economic news had little effect .3. Economic news had little effect on markets.4. Economic news had little effect on little effect on little effect .
Natural Language Processing 5(10)
![Page 6: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/6.jpg)
Top-Down Parsing
! Basic idea:! Start at the root node (start symbol)! Expand tree by matching the left-hand side of rules! Derive a tree whose terminal nodes match the input sentence
! Potential problems:! May use rules that could never match the input! May loop on recursive rules: VP → VP PP
Natural Language Processing 6(10)
![Page 7: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/7.jpg)
Bottom-Up Parsing
! Basic idea:! Start with the terminal nodes (words)! Expand tree by matching the right-hand side of rules! Build a tree whose with the start symbol at the root
! Potential problems:! May build structures that could never be in a tree! May loop on empty productions NP → ϵ
Natural Language Processing 7(10)
![Page 8: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/8.jpg)
Ambiguity
Adj
Economic
✟✟Noun
news
❍❍✑✑✑✑
✑✑✑✑
✑✑✑✑
NP
Verb
had
$$
$$$$
$VP
S
Adj
little
✟✟Noun
effect
❍❍✧✧✧✧✧
❍❍
NP
NP
Prep
on
$$$
❍❍PP
Adj
financial
✟✟Noun
markets
❍❍❍❍NP Punct
.
◗◗
◗◗
◗◗
◗◗
◗◗
◗◗
Adj
Economic
✟✟Noun
news
❍❍✑✑✑
✑✑✑✑
✑✑✑✑✑
NP
Verb
had
✡✡✡✡
VP
S
Adj
little
✟✟Noun
effect
❍❍❍❍
✦✦✦✦✦
NP
VP
Prep
on
$$$
❍❍❍❍
PP
Adj
financial
✟✟Noun
markets
❍❍❍❍NP Punct
.
◗◗
◗◗
◗◗
◗◗
◗◗
◗◗
! A sentence may be assigned more than one tree! This corresponds to syntactic ambiguity! What should the parser do in this case?
Natural Language Processing 8(10)
![Page 9: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/9.jpg)
Dealing with Ambiguity
! Combinatorial explosion:! The number of possible trees grows exponentially! We need a smart algorithm – dynamic programming
! Disambiguation:! We need some way of selecting the best tree! Probabilistic grammars rank trees by decreasing plausibility
Natural Language Processing 9(10)
![Page 10: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/10.jpg)
Quiz
! Which of the following sentences are syntactically ambiguous?1. I caught a bat2. I caught a bat with a hat3. I caught a bat happily4. I caught small bats and rats
Natural Language Processing 10(10)
![Page 11: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/11.jpg)
Natural Language Processing
Probabilistic Context-Free Grammar
Joakim Nivre
Uppsala UniversityDepartment of Linguistics and Philology
Natural Language Processing 1(10)
![Page 12: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/12.jpg)
Probabilistic Context-Free Grammar
G = (N,Σ,R , S ,Q)
! N is a finite (non-terminal) alphabet! Σ is a finite (terminal) alphabet! R is a finite set of rules A → α (A ∈ N, α ∈ (Σ ∪ N)∗)! S ∈ N is the start symbol
! Q is function from R to real numbers in the interval [0, 1]! For every nonterminal A ∈ N:
!
r∈R : LHS(r)=A
Q(r) = 1
Natural Language Processing 2(10)
![Page 13: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/13.jpg)
Example Grammar
S → NP VP Punct 1.00VP → VP PP 0.50VP → Verb NP 0.50NP → NP PP 0.25NP → Adj Noun 0.75PP → Prep NP 1.00
Punct → . 1.00Adj → Economic 0.33Adj → little 0.33Adj → financial 0.33
Noun → news 0.33Noun → effect 0.33Noun → markets 0.33Verb → had 1.00Prep → on 1.00
Adj
Economic
✟✟Noun
news
❍❍✑✑✑
✑✑✑✑
✑✑✑✑✑
NP
Verb
had
$$$$$
$$VP
S
Adj
little
✟✟Noun
effect
❍❍✧✧✧✧✧
❍❍
NP
NP
Prep
on
$$$
❍❍PP
Adj
financial
✟✟Noun
markets
❍❍❍❍NP Punct
.
◗◗
◗◗
◗◗
◗◗
◗◗
◗◗
Adj
Economic
✟✟Noun
news
❍❍✑✑✑
✑✑✑✑
✑✑✑✑✑
NP
Verb
had
✡✡✡✡
VP
S
Adj
little
✟✟Noun
effect
❍❍❍❍
✦✦✦✦✦
NP
VP
Prep
on
$$$
❍❍❍❍
PP
Adj
financial
✟✟Noun
markets
❍❍❍❍NP Punct
.
◗◗
◗◗
◗◗
◗◗
◗◗
◗◗
Natural Language Processing 3(10)
![Page 14: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/14.jpg)
Probability Model for Trees
! Probability of a tree T generated by G :
P(T ) =!
r∈RQ(r)f (r ,T )
! f (r ,T ) = the frequency of r in T
! Intuition:! The tree probability is the joint probability of rule instances! Rule instances are dependent only on their left-hand side! Thus, the joint probability is the product of rule probabilities
Natural Language Processing 4(10)
![Page 15: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/15.jpg)
Example Trees
S → NP VP Punct 1.00VP → VP PP 0.50VP → Verb NP 0.50NP → NP PP 0.25NP → Adj Noun 0.75PP → Prep NP 1.00
Punct → . 1.00Adj → Economic 0.33Adj → little 0.33Adj → financial 0.33
Noun → news 0.33Noun → effect 0.33Noun → markets 0.33Verb → had 1.00Prep → on 1.00
Adj
Economic
✟✟Noun
news
❍❍✑✑✑
✑✑✑✑
✑✑✑✑✑
NP
Verb
had
$$$$$
$$VP
S
Adj
little
✟✟Noun
effect
❍❍✧✧✧✧✧
❍❍
NP
NP
Prep
on
$$$
❍❍PP
Adj
financial
✟✟Noun
markets
❍❍❍❍NP Punct
.
◗◗
◗◗
◗◗
◗◗
◗◗
◗◗
0.000065
Adj
Economic
✟✟Noun
news
❍❍✑✑✑
✑✑✑✑
✑✑✑✑✑
NP
Verb
had
✡✡✡✡
VP
S
Adj
little
✟✟Noun
effect
❍❍❍❍
✦✦✦✦✦
NP
VP
Prep
on
$$$
❍❍❍❍
PP
Adj
financial
✟✟Noun
markets
❍❍❍❍NP Punct
.
◗◗
◗◗
◗◗
◗◗
◗◗
◗◗
0.000130
Natural Language Processing 5(10)
![Page 16: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/16.jpg)
Quiz
NP → Noun 0.1NP → Det Noun 0.4NP → Adj Noun 0.2NP → Det Adj Noun ?.?
! If these are all the NP rules, what is the missing probability?1. 0.12. 0.33. 0.64. 1.0
Natural Language Processing 6(10)
![Page 17: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/17.jpg)
Treebank Grammars! Given a treebank T = {(S1,T1), . . . , (Sn,Tn)}! Extract grammar G = (N,Σ,R , S):
! N = the set of all nonterminals occurring in some tree! Σ = the set of all terminals occurring in some tree! R = the set of all rules needed to derive some tree! S = the nonterminal at the root of every tree
! Estimate Q using relative frequencies (MLE):
Q(A → α) =
n!
i=1
f (A → α,Ti )
n!
i=1
!
r∈R:LHD(r)=A
f (r ,Ti )
Natural Language Processing 7(10)
![Page 18: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/18.jpg)
Limitations of Treebank Grammars
! Insensitive to structural context:
Tree Context NP PP DT NN PRPAnywhere 11% 9% 6%NP under S 9% 9% 21%NP under VP 23% 7% 4%
! Insensitive to lexical information:! She ate pizza with mushrooms! She ate pizza with a fork
Natural Language Processing 8(10)
![Page 19: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/19.jpg)
Parent Annotation Lexicalization
Adj
Economic
✟✟Noun
news
❍❍✑✑✑
✑✑✑✑
✑✑✑✑✑
NPˆS
Verb
had
$$$$$
$$VPˆS
SˆROOT
Adj
little
✟✟Noun
effect
❍❍✧✧✧✧✧
❍❍
NPˆNP
NPˆVP
Prep
on
$$$
❍❍PPˆNP
Adj
financial
✟✟Noun
markets
❍❍❍❍
NPˆPP Punct
.
◗◗
◗◗
◗◗
◗◗
◗◗
◗◗
Adj
Economic
✟✟Noun
news
❍❍✑✑✑
✑✑✑✑
✑✑✑✑✑
NP(news)
Verb
had
$$$$$
$$
VP(had)
S(had)
Adj
little
✟✟Noun
effect
❍❍✧✧✧✧✧
❍❍
NP(effect)
NP(effect)
Prep
on
$$$
❍❍PP(on)
Adj
financial
✟✟Noun
markets
❍❍❍❍
NP(markets) Punct
.
◗◗
◗◗
◗◗
◗◗
◗◗
◗◗
Natural Language Processing 9(10)
![Page 20: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/20.jpg)
Quiz
Adj
Economic
✟✟Noun
news
❍❍✑
✑✑✑✑
✑✑✑✑
✑✑✑
NP
Verb
had
$$$
$$$$VP
S
Adj
little
✟✟Noun
effect
❍❍✧✧✧✧✧
❍❍
NP
NP
Prep
on
$$$
❍❍PP
Adj
financial
✟✟Noun
markets
❍❍❍❍NP Punct
.
◗◗
◗◗
◗◗
◗◗
◗◗
◗◗
! Parent annotation increases the number of symbols.! How many new NP symbols do we need for the tree above?
1. 22. 33. 44. 5
Natural Language Processing 10(10)
![Page 21: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/21.jpg)
Natural Language Processing
PCFG Parsing
Joakim Nivre
Uppsala UniversityDepartment of Linguistics and Philology
Natural Language Processing 1(6)
![Page 22: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/22.jpg)
PCFG Parsing
! Given a sentence S and grammar G! derive all possible trees for S according to G! find most probable tree T ∗
! We can use dynamic programming to solve this efficiently! The Cocke-Kasami-Younger (CKY) algorithm! Earley’s algorithm
Natural Language Processing 2(6)
![Page 23: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/23.jpg)
Probabilistic CKY
0 I 1 saw 2 her 3 duck 4
! Parse tables:C[i , j ,B] = probability of best phrase of type B from i to jB[i , j ,B] = back-pointers to retrieve best B from i to j
! CKY assumes Chomsky Normal Form (CNF):A → B CA → a
! Any CFG can be converted to CNF
Natural Language Processing 3(6)
![Page 24: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/24.jpg)
Probabilistic CKYPARSE(G, x)for j from 1 to n do # j = end of word
for all A : A → a ∈ R and a = j−1w j
C[j − 1, j ,A] := Q(A → a)
for j from 2 to n do # j = end of phrasefor i from j − 2 downto 0 do # i = start of phrase
for k from i + 1 to j − 1 do # k = split pointfor all A : A → BC ∈ R and C[i , k ,B] > 0 and C[k , j ,C ] > 0
if (C[i , j ,A] < Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]) thenC[i , j ,A] := Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]
B[i , j ,A] := {k ,B ,C}return BUILD-TREE(B[0, n, S ]), C[0, n, S ]
Natural Language Processing 4(6)
![Page 25: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/25.jpg)
Probabilistic CKYPARSE(G, x)for j from 1 to n do # j = end of word
for all A : A → a ∈ R and a = j−1w j
C[j − 1, j ,A] := Q(A → a)
for j from 2 to n do # j = end of phrase
for i from j − 2 downto 0 do # i = start of phrasefor k from i + 1 to j − 1 do # k = split point
for all A : A → BC ∈ R and C[i , k ,B] > 0 and C[k , j ,C ] > 0if (C[i , j ,A] < Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]) then
C[i , j ,A] := Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]
B[i , j ,A] := {k ,B ,C}return BUILD-TREE(B[0, n, S ]), C[0, n, S ]
Natural Language Processing 4(6)
![Page 26: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/26.jpg)
Probabilistic CKYPARSE(G, x)for j from 1 to n do # j = end of word
for all A : A → a ∈ R and a = j−1w j
C[j − 1, j ,A] := Q(A → a)
for j from 2 to n do # j = end of phrasefor i from j − 2 downto 0 do # i = start of phrase
for k from i + 1 to j − 1 do # k = split pointfor all A : A → BC ∈ R and C[i , k ,B] > 0 and C[k , j ,C ] > 0
if (C[i , j ,A] < Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]) thenC[i , j ,A] := Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]
B[i , j ,A] := {k ,B ,C}return BUILD-TREE(B[0, n, S ]), C[0, n, S ]
Natural Language Processing 4(6)
![Page 27: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/27.jpg)
Probabilistic CKYPARSE(G, x)for j from 1 to n do # j = end of word
for all A : A → a ∈ R and a = j−1w j
C[j − 1, j ,A] := Q(A → a)
for j from 2 to n do # j = end of phrasefor i from j − 2 downto 0 do # i = start of phrase
for k from i + 1 to j − 1 do # k = split point
for all A : A → BC ∈ R and C[i , k ,B] > 0 and C[k , j ,C ] > 0if (C[i , j ,A] < Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]) then
C[i , j ,A] := Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]
B[i , j ,A] := {k ,B ,C}return BUILD-TREE(B[0, n, S ]), C[0, n, S ]
Natural Language Processing 4(6)
![Page 28: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/28.jpg)
Probabilistic CKYPARSE(G, x)for j from 1 to n do # j = end of word
for all A : A → a ∈ R and a = j−1w j
C[j − 1, j ,A] := Q(A → a)
for j from 2 to n do # j = end of phrasefor i from j − 2 downto 0 do # i = start of phrase
for k from i + 1 to j − 1 do # k = split pointfor all A : A → BC ∈ R and C[i , k ,B] > 0 and C[k , j ,C ] > 0
if (C[i , j ,A] < Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]) thenC[i , j ,A] := Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]
B[i , j ,A] := {k ,B ,C}return BUILD-TREE(B[0, n, S ]), C[0, n, S ]
Natural Language Processing 4(6)
![Page 29: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/29.jpg)
Probabilistic CKYPARSE(G, x)for j from 1 to n do # j = end of word
for all A : A → a ∈ R and a = j−1w j
C[j − 1, j ,A] := Q(A → a)
for j from 2 to n do # j = end of phrasefor i from j − 2 downto 0 do # i = start of phrase
for k from i + 1 to j − 1 do # k = split pointfor all A : A → BC ∈ R and C[i , k ,B] > 0 and C[k , j ,C ] > 0
if (C[i , j ,A] < Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]) then
C[i , j ,A] := Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]
B[i , j ,A] := {k ,B ,C}return BUILD-TREE(B[0, n, S ]), C[0, n, S ]
Natural Language Processing 4(6)
![Page 30: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/30.jpg)
Probabilistic CKYPARSE(G, x)for j from 1 to n do # j = end of word
for all A : A → a ∈ R and a = j−1w j
C[j − 1, j ,A] := Q(A → a)
for j from 2 to n do # j = end of phrasefor i from j − 2 downto 0 do # i = start of phrase
for k from i + 1 to j − 1 do # k = split pointfor all A : A → BC ∈ R and C[i , k ,B] > 0 and C[k , j ,C ] > 0
if (C[i , j ,A] < Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]) thenC[i , j ,A] := Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]
B[i , j ,A] := {k ,B ,C}return BUILD-TREE(B[0, n, S ]), C[0, n, S ]
Natural Language Processing 4(6)
![Page 31: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/31.jpg)
Probabilistic CKYPARSE(G, x)for j from 1 to n do # j = end of word
for all A : A → a ∈ R and a = j−1w j
C[j − 1, j ,A] := Q(A → a)
for j from 2 to n do # j = end of phrasefor i from j − 2 downto 0 do # i = start of phrase
for k from i + 1 to j − 1 do # k = split pointfor all A : A → BC ∈ R and C[i , k ,B] > 0 and C[k , j ,C ] > 0
if (C[i , j ,A] < Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]) thenC[i , j ,A] := Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]
B[i , j ,A] := {k ,B ,C}
return BUILD-TREE(B[0, n, S ]), C[0, n, S ]
Natural Language Processing 4(6)
![Page 32: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/32.jpg)
Probabilistic CKYPARSE(G, x)for j from 1 to n do # j = end of word
for all A : A → a ∈ R and a = j−1w j
C[j − 1, j ,A] := Q(A → a)
for j from 2 to n do # j = end of phrasefor i from j − 2 downto 0 do # i = start of phrase
for k from i + 1 to j − 1 do # k = split pointfor all A : A → BC ∈ R and C[i , k ,B] > 0 and C[k , j ,C ] > 0
if (C[i , j ,A] < Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]) thenC[i , j ,A] := Q(A → BC ) · C[i , k ,B] · C[k , j ,C ]
B[i , j ,A] := {k ,B ,C}return BUILD-TREE(B[0, n, S ]), C[0, n, S ]
Natural Language Processing 4(6)
![Page 33: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/33.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 4
0NP1 0.2 0S3 0.024 0S4 0.072
1Verb2 1.0 1VP3 0.12 1VP4 0.361VP4 0.006
2NP3 0.2 2NP4 0.62Det3 1.0 2S4 0.02
3VP4 0.13Noun4 1.0
Natural Language Processing 5(6)
![Page 34: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/34.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2
0S3 0.024 0S4 0.072
1Verb2 1.0 1VP3 0.12 1VP4 0.361VP4 0.006
2NP3 0.2 2NP4 0.62Det3 1.0 2S4 0.02
3VP4 0.13Noun4 1.0
Natural Language Processing 5(6)
![Page 35: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/35.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2
0S3 0.024 0S4 0.072
1Verb2 1.0
1VP3 0.12 1VP4 0.361VP4 0.006
2NP3 0.2 2NP4 0.62Det3 1.0 2S4 0.02
3VP4 0.13Noun4 1.0
Natural Language Processing 5(6)
![Page 36: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/36.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2
0S3 0.024 0S4 0.072
1Verb2 1.0
1VP3 0.12 1VP4 0.361VP4 0.006
2NP3 0.2
2NP4 0.62Det3 1.0 2S4 0.02
3VP4 0.13Noun4 1.0
Natural Language Processing 5(6)
![Page 37: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/37.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2
0S3 0.024 0S4 0.072
1Verb2 1.0
1VP3 0.12 1VP4 0.361VP4 0.006
2NP3 0.2
2NP4 0.6
2Det3 1.0
2S4 0.023VP4 0.1
3Noun4 1.0
Natural Language Processing 5(6)
![Page 38: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/38.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2
0S3 0.024 0S4 0.072
1Verb2 1.0
1VP3 0.12 1VP4 0.361VP4 0.006
2NP3 0.2
2NP4 0.6
2Det3 1.0
2S4 0.02
3VP4 0.1
3Noun4 1.0
Natural Language Processing 5(6)
![Page 39: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/39.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2
0S3 0.024 0S4 0.072
1Verb2 1.0
1VP3 0.12 1VP4 0.361VP4 0.006
2NP3 0.2
2NP4 0.6
2Det3 1.0
2S4 0.02
3VP4 0.13Noun4 1.0
Natural Language Processing 5(6)
![Page 40: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/40.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2
0S3 0.024 0S4 0.072
1Verb2 1.0 1VP3 0.12
1VP4 0.361VP4 0.006
2NP3 0.2
2NP4 0.6
2Det3 1.0
2S4 0.02
3VP4 0.13Noun4 1.0
Natural Language Processing 5(6)
![Page 41: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/41.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2 0S3 0.024
0S4 0.072
1Verb2 1.0 1VP3 0.12
1VP4 0.361VP4 0.006
2NP3 0.2
2NP4 0.6
2Det3 1.0
2S4 0.02
3VP4 0.13Noun4 1.0
Natural Language Processing 5(6)
![Page 42: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/42.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2 0S3 0.024
0S4 0.072
1Verb2 1.0 1VP3 0.12
1VP4 0.361VP4 0.006
2NP3 0.2 2NP4 0.62Det3 1.0
2S4 0.02
3VP4 0.13Noun4 1.0
Natural Language Processing 5(6)
![Page 43: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/43.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2 0S3 0.024
0S4 0.072
1Verb2 1.0 1VP3 0.12
1VP4 0.361VP4 0.006
2NP3 0.2 2NP4 0.62Det3 1.0 2S4 0.02
3VP4 0.13Noun4 1.0
Natural Language Processing 5(6)
![Page 44: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/44.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2 0S3 0.024
0S4 0.072
1Verb2 1.0 1VP3 0.12 1VP4 0.36
1VP4 0.006
2NP3 0.2 2NP4 0.62Det3 1.0 2S4 0.02
3VP4 0.13Noun4 1.0
Natural Language Processing 5(6)
![Page 45: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/45.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2 0S3 0.024
0S4 0.072
1Verb2 1.0 1VP3 0.12 1VP4 0.361VP4 0.006
2NP3 0.2 2NP4 0.62Det3 1.0 2S4 0.02
3VP4 0.13Noun4 1.0
Natural Language Processing 5(6)
![Page 46: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/46.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2 0S3 0.024
0S4 0.072
1Verb2 1.0 1VP3 0.12 1VP4 0.36
1VP4 0.006
2NP3 0.2 2NP4 0.62Det3 1.0 2S4 0.02
3VP4 0.13Noun4 1.0
Natural Language Processing 5(6)
![Page 47: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/47.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2 0S3 0.024 0S4 0.072
1Verb2 1.0 1VP3 0.12 1VP4 0.36
1VP4 0.006
2NP3 0.2 2NP4 0.62Det3 1.0 2S4 0.02
3VP4 0.13Noun4 1.0
Natural Language Processing 5(6)
![Page 48: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/48.jpg)
Probabilistic CKY
S → NP VP 1.0 NP → her 0.2VP → Verb NP 0.6 NP → I 0.2VP → Verb S 0.3 Verb → saw 1.0VP → duck 0.1 Noun → duck 1.0NP → Det Noun 0.6 Det → her 1.0
0 I 1 saw 2 her 3 duck 40NP1 0.2 0S3 0.024 0S4 0.072
1Verb2 1.0 1VP3 0.12 1VP4 0.36
1VP4 0.006
2NP3 0.2 2NP4 0.62Det3 1.0 2S4 0.02
3VP4 0.13Noun4 1.0
Natural Language Processing 5(6)
![Page 49: Natural Language Processing - Uppsala Universitynivre/master/NLP-ConParsing.pdf · 2017-10-15 · Economic news had little effect on little effect on little effect . Natural Language](https://reader033.vdocuments.net/reader033/viewer/2022050115/5f4c1c3258aee7618f7ba721/html5/thumbnails/49.jpg)
Quiz
! Which items are combined into 1VP3?1. 1Verb2 and 2NP32. 1Verb2 and 2Det33. 1Verb2 and 2NP4
Natural Language Processing 6(6)