Lecture 15: Dependency Parsing

Kai-Wei ChangCS @ University of Virginia

kw@kwchang.net

Couse webpage: http://kwchang.net/teaching/NLP16

1CS6501: NLP

How to represent the structure

CS6501: NLP 2

Dependency trees

vDependency grammar describe the structure of sentences as a graph (tree)vNodes represent wordsvEdges represent dependencies

v Idea goes back to 4th century BCin ancient India

CS6501: NLP 3

Phrase structure (constituent parse) trees

v Can be modeled by Context-free grammarsv We will see how constituent parse and

dependency parse are related

CS6501: NLP 4

Context-free grammars

CS6501: NLP 5

PP→ PNPPP→ PDTNPP→ inthegarden

Non-terminal:DT,N,P,NP,PP,…Terminal:the,a,ball,garden

Generate sentences by CFG

CS6501: NLP 6

Parse tree defined by CFG

CS6501: NLP 7

1234567 Rule2 Rule4&1

Rule6

Example: noun phrases

CS6501: NLP 8

Example: verb phrase

CS6501: NLP 9

Sentences

CS6501: NLP 10

Constituent Parse

CS6501: NLP 11

From:KevinGimpel

Constituent Parse

CS6501: NLP 12

Non-terminal

Terminal

Sà NPVPNPà DTNNNPà DTPP à INNPVPà VBDPPVPà NPVBDNPVPà NPVB

Nonterminal in Penn Treebank

CS6501: NLP 13

Probabilistic Context-free Grammar

CS6501: NLP 14

Non-terminal

Terminal

1.0Sà NPVP0.6NPà DTNN0.4NPà NPPP1.0PPà INNP0.5VPà VBDPP0.2VPà NPVBDNP0.3VPà NPVB

Probabilistic Context-free Grammar

vPCFG achieves ~73% on PTBvState-of-the art ~92%

v Lexicalized PCFG(Collins 1997)

CS6501: NLP 15

CS6501: NLP 16

How to decide head?

vUsually use deterministic head rules (e.g., Collins head rules)

vDefine heads in CFGvS → NP VPvVP → VBD NP PPvNP → DT JJ NN

CS6501: NLP 17

FromNoahSmith

Lexical Head Annotation

CS6501: NLP 18

Constituent parse → Dependency Parse

CS6501: NLP 19

Constituent parse → Dependency Parse

CS6501: NLP 20

CS6501: NLP 21

HeadrulescanbeusedtoextractdependencyparsefromaCFG

CS6501: NLP 22

Arrowtypesshowthenameofgrammaticalrelations

Dependency parsing

vCan be more flexible (non-projective)vEnglish are mostly projective

vSome free word order languages (e.g., Czech) are non-projective

CS6501: NLP 23

How to build a dependency tree?

vThere are several approachesvGraph Algorithms

v Consider all word pairsv Create a Maximum Spanning Tree for a sentence

v Transition-base Approachesv Similar to how we parse a program:

Shift-Reduce Parser

v Many other approaches…

CS6501: NLP 24

Sources of information for DP

v Lexical affinitiesv [ issues → the ]v [ issues → I ]

vDistances vWords usually depend on nearby words

vValency of headsv# dependents for a head

CS6501: NLP 25

Graph-Based Approaches[McDonald et al. 2005]

vConsider all word pairs and assign scoresvScore of a tree = sum of score of edgesvCan be solve as a MST problem

vChu-Liu-Edmonds

CS6501: NLP 26

Transition-based parser

vMaltParser (Nivre et al. 2008)vSimilar to a Shift-Reduce Parser

vBut “reduce” actions can create dependencies

vThe parser has:vA stack 𝜎– starts with a “Root” symbolvA buffer 𝛽– starts with the input sentencevA set of dependency arcs A– starts off empty

vUse a set of actions to parse sentencesvMany possible action sets

CS6501: NLP 27

Arc-Eager Dependency Parser

vShift:

v Left-Arc:

CS6501: NLP 28

ROOTJoe likesMary Joe

ROOTlikesMary→

JoeROOT

likesMary →ROOT

likesMary

Joe

Precondition:𝑤% ≠ Root&(𝑤(,𝑤%) ∉ 𝐴

Arc-Eager Dependency Parser

vRight-Arc

vReduce:

CS6501: NLP 29

likeROOT

MaryMarylikeROOT→

Joe Joe

MarylikeROOT

Joe

→ likeROOT

Joe

Precondition:(𝑤(, 𝑤%) ∈ 𝐴Mary

Arc-Eager Dependency Parser

vStart:vConduct a sequence of actions

vTerminate with 𝜎, 𝛽 = ∅

CS6501: NLP 30

CS6501: NLP 31

CS6501: NLP 32

CS6501: NLP 33

CS6501: NLP 34

CS6501: NLP 35

CS6501: NLP 36

CS6501: NLP 37

CS6501: NLP 38

CS6501: NLP 39

CS6501: NLP 40

CS6501: NLP 41

CS6501: NLP 42

CS6501: NLP 43

CS6501: NLP 44

CS6501: NLP 45

CS6501: NLP 46

CS6501: NLP 47

It’s your turn

vHappy children like to play with their friend .vShift→ Left-arc→ Shift→ Left-arc →Right-arc→ Shift →Left-arc →Right-arc →Right-arc → Shift →Left-arc →Right-arc →Reduce*3→Right-arc →Reduce*3

CS6501: NLP 48

CS6501: NLP 49

FromChrisManning

CS6501: NLP 50

Structured Prediction –beyond sequence tagging

Task Input Output

Part-of-speechTagging

They operate ships and banks.

Dependency Parsing

They operate ships and banks.

51

Pronoun Verb Noun And Noun

Root Theyoperateshipsandbanks.

Assign values to a set of interdependent output variables