advanced lecture on dependency parsing: dynamic oracles and …miryam/papers/5ln455-f10.pdf ·...
Post on 12-Aug-2020
6 Views
Preview:
TRANSCRIPT
Advanced Lecture on Dependency Parsing:Dynamic Oracles and Online Reordering
Miryam de Lhoneux
14 December 2017
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 1
Outline
1 Arc-Hybrid
2 Dynamic Oracles
3 Reordering
4 Dynamic Oracles and Reordering
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 2
Outline for section 1
1 Arc-Hybrid
2 Dynamic Oracles
3 Reordering
4 Dynamic Oracles and Reordering
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 3
Transition-Based Parsing with Arc-Hybrid
STACK BUFFER
Configuration:
friend home rootDrive your
Transitions:
Kuhlmann et al. (2011)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 4
Transition-Based Parsing with Arc-Hybrid
LEFT−ARC
STACK BUFFER
Configuration:
friend home rootDrive your
Transitions:
Kuhlmann et al. (2011)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 4
Transition-Based Parsing with Arc-Hybrid
LEFT−ARC
STACK BUFFER
Configuration:
friend home rootDrive your
Transitions:
Kuhlmann et al. (2011)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 4
Transition-Based Parsing with Arc-Hybrid
LEFT−ARC
STACK BUFFER
Configuration:
friend home rootDrive your
Transitions:
Kuhlmann et al. (2011)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 4
Transition-Based Parsing with Arc-Hybrid
RIGHT−ARC
LEFT−ARC
STACK BUFFER
Configuration:
friend home rootDrive your
Transitions:
Kuhlmann et al. (2011)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 4
Transition-Based Parsing with Arc-Hybrid
RIGHT−ARC
LEFT−ARC
STACK BUFFER
Configuration:
friend home rootDrive your
Transitions:
Kuhlmann et al. (2011)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 4
Transition-Based Parsing with Arc-Hybrid
RIGHT−ARC
LEFT−ARC
STACK BUFFER
Configuration:
friend home rootDrive your
Transitions:
Kuhlmann et al. (2011)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 4
Transition-Based Parsing with Arc-Hybrid
SHIFT
RIGHT−ARC
LEFT−ARC
STACK BUFFER
Configuration:
friend home rootDrive your
Transitions:
Kuhlmann et al. (2011)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 4
Transition-Based Parsing with Arc-Hybrid
SHIFT
RIGHT−ARC
LEFT−ARC
STACK BUFFER
Configuration:
friend home rootDrive your
Transitions:
Kuhlmann et al. (2011)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 4
Training a Transition-Based Parser
Figure : Figure taken from Goldberg and Nivre (2012)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 5
Training a Transition-Based Parser
Figure : Figure taken from Goldberg and Nivre (2012)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 5
Static Oracle for Arc-Hybrid
Drive your friend home **root**
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 6
Static Oracle for Arc-Hybrid
Drive your friend home **root**
[ ] [Drive your friend home **root**]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 6
Static Oracle for Arc-Hybrid
SHIFT
Drive your friend home **root**
[ Drive ] [your friend home **root**]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 6
Static Oracle for Arc-Hybrid
SHIFT
Drive your friend home **root**
[ Drive your ] [friend home **root**]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 6
Static Oracle for Arc-Hybrid
LEFT-ARC
Drive your friend home **root**
[ Drive ] [friend home **root**]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 6
Static Oracle for Arc-Hybrid
SHIFT
Drive your friend home **root**
[ Drive friend] [home **root**]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 6
Static Oracle for Arc-Hybrid
RIGHT-ARC
Drive your friend home **root**
[ Drive ] [home **root**]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 6
Static Oracle for Arc-Hybrid
SHIFT
Drive your friend home **root**
[ Drive home] [**root**]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 6
Static Oracle for Arc-Hybrid
RIGHT-ARC
Drive your friend home **root**
[ Drive ] [**root**]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 6
Static Oracle for Arc-Hybrid
LEFT-ARC
Drive your friend home **root**
[ ] [**root**]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 6
Exercise!
Write the transition sequence for this tree:
I love parsing so much ! *root*
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 7
Outline for section 2
1 Arc-Hybrid
2 Dynamic Oracles
3 Reordering
4 Dynamic Oracles and Reordering
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 8
Static Oracle: problem 1
Figure : Figure taken from Goldberg and Nivre (2012)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 9
Static Oracle: problem 1
Figure : Figure taken from Goldberg and Nivre (2012)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 9
Spurious ambiguity
Send him a flower
[ Send him ] [ a flower ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 10
Spurious ambiguity
RIGHT-ARC
Send him a flower
[ Send ] [ a flower ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 10
Spurious ambiguity
Send him a flower
[ Send him ] [ a flower ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 10
Spurious ambiguity
SHIFT
Send him a flower
[ Send him a ] [ flower ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 10
Spurious ambiguity
LEFT-ARC
Send him a flower
[ Send him ] [ flower ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 10
Spurious ambiguity
RIGHT-ARC
Send him a flower
[ Send ] [ flower ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 10
Static Oracle: problem 2
Figure : Figure taken from Goldberg and Nivre (2012)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 11
Static Oracle: problem 2
Figure : Figure taken from Goldberg and Nivre (2012)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 11
Training with a Dynamic Oracle
Figure : Figure taken from Goldberg and Nivre (2013)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 12
Training with a Dynamic Oracle
Figure : Figure taken from Goldberg and Nivre (2013)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 12
Training with a Dynamic Oracle
Figure : Figure taken from Goldberg and Nivre (2013)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 12
Dynamic Oracle for Arc-Hybrid
Drive your friend home **root**
[ Drive your ] [ friend home **root**]
Goldberg and Nivre (2013)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 13
Dynamic Oracle for Arc-Hybrid
RIGHT-ARC
Drive your friend home **root**
[ Drive your ] [ friend home **root**]
Goldberg and Nivre (2013)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 13
Dynamic Oracle for Arc-Hybrid
RIGHT-ARC
Drive your friend home **root**
[ Drive ] [ friend home **root**]
Goldberg and Nivre (2013)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 13
Dynamic Oracle for Arc-Hybrid
RIGHT-ARC
Drive your friend home **root**
cost+=1
[ Drive ] [ friend home **root**]
Goldberg and Nivre (2013)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 13
Dynamic Oracle for Arc-Hybrid
SHIFT
Drive your friend home **root**
[ Drive your ] [ friend home **root**]
Goldberg and Nivre (2013)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 13
Dynamic Oracle for Arc-Hybrid
SHIFT
Drive your friend home **root**
[ Drive your friend ] [ home **root**]
Goldberg and Nivre (2013)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 13
Dynamic Oracle for Arc-Hybrid
SHIFT
Drive your friend home **root**
cost+=1
cost+=1
[ Drive your friend ] [ home **root**]
Goldberg and Nivre (2013)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 13
Dynamic Oracle for Arc-Hybrid
Figure : Cost function
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 14
Exercise!
For the 7 first transitions in the correct transition sequence wedefined in the previous exercise, compute the cost of the otherlegal transitions.
I love parsing so much ! *root*
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 15
Outline for section 3
1 Arc-Hybrid
2 Dynamic Oracles
3 Reordering
4 Dynamic Oracles and Reordering
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 16
Arc-Hybrid Parsing with Reordering
SHIFT
RIGHT−ARC
LEFT−ARC
STACK BUFFER
Configuration:
friend home rootDrive your
Transitions:
Nivre (2009); de Lhoneux et al. (2017)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 17
Arc-Hybrid Parsing with Reordering
SWAP
SHIFT
RIGHT−ARC
LEFT−ARC
STACK BUFFER
Configuration:
friend home rootDrive your
Transitions:
Nivre (2009); de Lhoneux et al. (2017)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 17
Arc-Hybrid Parsing with Reordering
SWAP
SHIFT
RIGHT−ARC
LEFT−ARC
STACK BUFFER
Configuration:
friend home rootDrive your
Transitions:
Nivre (2009); de Lhoneux et al. (2017)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 17
Arc-Hyrid Parsing with Reordering
found best example ever1 2 4 3
(Thanks Carlos Gomez-Rodriguez for the example!)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 18
Arc-Hyrid Parsing with Reordering
found best example ever1 2 4 3
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 18
Arc-Hyrid Parsing with Reordering
found best example ever1 2 4 3
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 18
Arc-Hyrid Parsing with Reordering
SHIFT
found best example ever1 2 4 3
[ ] [ found1 best2 example4 ever3 ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 18
Arc-Hyrid Parsing with Reordering
SHIFT
found best example ever1 2 4 3
[ found1 ] [ best2 example4 ever3 ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 18
Arc-Hyrid Parsing with Reordering
SHIFT
found best example ever1 2 4 3
[ found1 best2 ] [ example4 ever3 ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 18
Arc-Hyrid Parsing with Reordering
SHIFT
found best example ever1 2 4 3
[ found1 best2 example4 ] [ ever3 ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 18
Arc-Hyrid Parsing with Reordering
SHIFT
found best example ever1 2 4 3
[ found1 best2 example4 ] [ ever3 ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 18
Arc-Hyrid Parsing with Reordering
SWAP
found best example ever1 2 4 3
[ found1 best2 ] [ ever3 example4 ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 18
Arc-Hyrid Parsing with Reordering
SHIFT
found best example ever1 2 4 3
[ found1 best2 ever3 ] [ example4 ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 18
Arc-Hyrid Parsing with Reordering
RIGHT-ARC
found best example ever1 2 4 3
[ found1 best2 ] [ example4 ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 18
Arc-Hyrid Parsing with Reordering
LEFT-ARC
found best example ever1 2 4 3
[ found1 ] [ example4 ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 18
Arc-Hyrid Parsing with Reordering
SHIFT
found best example ever1 2 4 3
[ found1 example4 ] [ ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 18
Arc-Hyrid Parsing with Reordering
RIGHT-ARC
found best example ever1 2 4 3
[ found1 ] [ ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 18
Exercise!
Write the transition sequence for this tree:
Stay away as1 far as2 possible . **root**1 2 3 6 4 5 7 8
(Example taken from the UD English treebank.)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 19
Outline for section 4
1 Arc-Hybrid
2 Dynamic Oracles
3 Reordering
4 Dynamic Oracles and Reordering
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 20
Dynamic Oracles and Reordering
Dynamic oracle for parsing with reordering:Open research question!
Partial solution:A static-dynamic oracle (de Lhoneux et al., 2017)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 21
Dynamic Oracles and Reordering
Dynamic oracle for parsing with reordering:Open research question!
Partial solution:A static-dynamic oracle (de Lhoneux et al., 2017)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 21
Dynamic Oracles and Reordering
Dynamic oracle for parsing with reordering:Open research question!
Partial solution:A static-dynamic oracle (de Lhoneux et al., 2017)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 21
Dynamic Oracles and Reordering
Dynamic oracle for parsing with reordering:Open research question!
Partial solution:A static-dynamic oracle (de Lhoneux et al., 2017)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 21
Dynamic Oracles and Reordering
Dynamic oracle for parsing with reordering:Open research question!
Partial solution:A static-dynamic oracle (de Lhoneux et al., 2017)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 21
Why is it hard?
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 22
Why is it hard?
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 22
Why is it hard?
Using information about the position of words in stack and buffer
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 22
Why is it hard?
Using information about the position of words in stack and bufferBut now words can move!
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 22
A Static-Dynamic Oracle
Figure : Figure taken from Goldberg and Nivre (2013)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 23
A Static-Dynamic Oracle
Figure : Figure taken from Goldberg and Nivre (2013)
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 23
A Static-Dynamic Oracle
found best example ever1 2 4 3
[ found1 best2] [ example4 ever3 ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 24
A Static-Dynamic Oracle
RIGHT-ARC
found best example ever1 2 4 3
[ found1 ] [ example4 ever3 ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 24
A Static-Dynamic Oracle
RIGHT-ARC
found best example ever1 2 4 3
cost+=1
cost+=1
[ found1 ] [ example4 ever3 ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 24
A Static-Dynamic Oracle
LEFT-ARC
found best example ever1 2 4 3
[ found1 ] [ example4 ever3 ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 24
A Static-Dynamic Oracle
LEFT-ARC
found best example ever1 2 4 3
cost+=1
[ found1 ] [ example4 ever3 ]
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 24
References
Miryam de Lhoneux, Sara Stymne, and Joakim Nivre. 2017. Arc-hybrid non-projective dependency parsing with astatic-dynamic oracle. In Proceedings of the 15th International Conference on Parsing Technologies.Association for Computational Linguistics, Pisa, Italy, pages 99–104.http://www.aclweb.org/anthology/W17-6314.
Yoav Goldberg and Joakim Nivre. 2012. A dynamic oracle for arc-eager dependency parsing. In Proceedings of the24th International Conference on Computational Linguistics (COLING). pages 959–976.
Yoav Goldberg and Joakim Nivre. 2013. Training deterministic parsers with non-deterministic oracles. Transactionsof the Association for Computational Linguistics 1:403–414.
Marco Kuhlmann, Carlos Gómez-Rodríguez, and Giorgio Satta. 2011. Dynamic programming algorithms fortransition-based dependency parsers. In Proceedings of the 49th Annual Meeting of the Association forComputational Linguistics (ACL). pages 673–682.
Joakim Nivre. 2009. Non-projective dependency parsing in expected linear time. In Proceedings of the JointConference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on NaturalLanguage Processing of the AFNLP (ACL-IJCNLP). pages 351–359.
Miryam de Lhoneux Advanced Lecture on Dependency Parsing 25
top related