treebanks and mwes (part 1) jan hajič, pavel straňák, jiří mírovský institute of formal and...

Treebanks and MWEs(Part 1)

Jan Hajič, Pavel Straňák, Jiří MírovskýInstitute of Formal and Applied Linguistics & LINDAT/CLARIN

School of Computer ScienceFaculty of Mathematics and Physics

Charles University in PragueCzech Republic

19.1.2015 PARSEME Training School Prague 2

Outline

• Treebanks– Phrase-(Constituency-) based: The Penn Treebank– Dependency: The Prague Dependency Treebanks

• The Penn Treebank (basics)• The Prague Dependency Treebank

– Layers of Annotation• Morphology• Syntax• Semantics• Valency

THE PENN TREEBANK



Phrase- vs. Dependency-Based Treebanks

• The original: The Penn Treebank– Phrase-based style; good for parsing by CFG grammars

• Followers– Almost all Penn-based treebanks

• Chinese, Arabic, Korean, …

– Negra (German), many others

• Now: dependency parsing prevails• Conversion from phrase-based treebanks

– Might lose information, heads added „ad hoc“

• “native” dependency treebanks: annotated as such– Considered “better”– Hindi/Urdu, TIGER (sort of); both styles manually annotated– PDT (of course) and similar ones

» PDT style treebanks: Danish, Croatian, Slovene, Greek, Latin

The Penn Treebank

• Published (first) in 1993, now LDC99T42 (www.ldc.upenn.edu)– First the Wall Street Journal part (1 mil. words, 2312 documents)

• Added other text types– ATIS corpus (dialogs, travel reservations)

– Brown corpus annotated for syntax

– Switchboard (spoken language, tel. conversations)



Penn Treebank Format

• ( (S • (NP-SBJ • (NP (NNP Pierre) (NNP Vinken) )• (, ,) • (ADJP • (NP (CD 61) (NNS years) )• (JJ old) )• (, ,) )• (VP (MD will) • (VP (VB join) • (NP (DT the) (NN board) )• (PP-CLR (IN as) • (NP (DT a) (JJ nonexecutive) (NN director) ))• (NP-TMP (NNP Nov.) (CD 29) )))• (. .) ))

Pierre Vinken, 61 years old, will join the board as a nonexecutive director Nov. 29.

POS tag (NNS)(noun, plural)

Phrase label (NP)

Noun Phrase

“Preterminal”

The Penn Treebank(s)

• Extensions– Annotation of named entities, co-reference (BBN)

– cf. also previous slides

– Function labels (SBJ, OBJ, TMP, ...)– PropBank

• Penn Treebank syntax + Predicate-argument relations, added “frame files” (predicate dictionary)

(S (NP-SBJ (PRP I Arg0) VP (VBD gave Pred) (NP-DOBJ (PRP him Arg1) (NP-IOBJ (DET the) (NN book Arg2))) ... )

– NomBank• Like PropBank, but for nouns and their “arguments”

• Other languages (Chinese, Arabic, ...)


THE PRAGUE DEPENDENCY TREEBANK



The Prague Dependency Treebanks: the Basics

• Original Treebank: PDT 1.0, 2001 (morf., dep. syntax)• First full release: PDT 2.0

– http://ufal.mff.cuni.cz/pdt2.0 • LDC2006T01, see http://www.ldc.upenn.edu

– Now: PDT 3.0: http://ufal.mff.cuni.cz/pdt3.0 • Basic general features

– Multilayered annotation, interlinked layers– Dependency-based syntax (both surface and deep)– Information structure of the sentence (topic/focus)– Grammatical and basic textual coreference– New: discourse relations, MWEs

• Languages: Czech, English (also parallel), Arabic– Student work on “samples”: Indonesian, Urdu, Russian, …– Spoken: work started on Czech and English (non-parallel, dialogs)


The Prague Dependency

Treebank

• Three basic layers of annotation– Morphemic layer– Surface syntax (“analytical”) layer– “Tectogrammatical” layer:

underlying syntax, semantic roles (valency), inf. structure, coreference

• Size– 830,000 words (tokens) = 50000 sentences in 3165 full

documents (texts)• Format

– Prague Markup Language (XML-based)

– Now also: .treex format• For smooth uise in the TreeX platform• http://ufal.mff.cuni.cz/treex


PDT (Czech) Data

• 4 sources:– Lidové noviny (daily newspaper, incl. extra sections)– DNES (Mladá fronta Dnes) (daily newspaper)– Vesmír (popular science magazine, monthly)– Českomoravský Profit (economical journal, weekly)

• Full articles selected– article ~ DOCUMENT (basic corpus unit)

• Time period: 1990-1995

• 1.8 million tokens (~110,000 sentences total)


PDT Annotation Layers

• L0 (w) Words (tokens)– automatic segmentation and markup only

• L1 (m) Morphology– Tag (full morphology, 13 categories), lemma

• L2 (a) Analytical layer (surface syntax)– Dependency, analytical dependency function

• L3 (t) Tectogrammatical layer (“deep” syntax)– Dependency, “functor”, grammatemes, ellipsis

solution, coreference, topic/focus (deep word order), valency lexicon; PDT 3.0PDT 3.0: mass, clauses, formemes, discourse, ...

PD

T 1

.0

(200

1)

PD

T 2

.0

(200

6)






• L3 (t) Tectogrammatical layer (“deep” syntax)– Dependency, functor (detailed), grammatemes,

ellipsis solution, coreference, topic/focus (deep word order), valency lexicon


Morphological Attributes

Tag: 13 categories Example: AAFP3----3N---- Adjective no poss. Gender negated Regular no poss. Number no voice Feminine no person reserve1 Plural no tense reserve2 Dative superlative base var.

Lemma: POS-unique identifierBooks/verb -> book-1, went -> go, to/prep. -> to-1

Ex.: nejnezajímavějším“(to) the most uninteresting”


Layer 2 (a-layer): Analytical Syntax

• Dependency + Analytical Function

dependent

governor

The influence of the Mexicancrisis on Central and EasternEurope has apparently been underestimated.


Analytical Syntax: Functions

• Main (for [main] semantic lexemes):• Pred, Sb, Obj, Adv, Atr, Atv(V), AuxV, Pnom• “Double” dependency: AtrAdv, AtrObj, AtrAtr

• Special (function words, punctuation,...):• Reflexives, particles: AuxT, AuxR, AuxO, AuxZ, AuxY• Prepositions/Conjunctions: AuxP, AuxC• Punctuation, Graphics: AuxX, AuxS, AuxG, AuxK

• Structural• Elipsis: ExD, Coordination etc.: Coord, Apos

Tectogrammatical Annotation

• Underlying (deep) syntax

• 5 sublayers (integrated and/or standoff annotation):– dependency structure, (detailed) functors

• valency annotation

– topic/focus and deep word order

– coreference (mostly grammatical only)

– discourse

– all the rest (grammatemes): • detailed functors• underlying gender, number, mass nouns, ...

• Total: 39 attributes (vs. 5 at m-layer, 2 at a-layer)



Tectogrammatical vs. analytical syntax

TR: Nofunction words

AR: All words

Predicate verb

“Location”

In practice, that procedure will require making of certified copies.

Re-inserted elided actorof “making”

Dependency Structure

• Similar to the surface (Analytical) layer... ...but:– certain nodes deleted

• auxiliaries, non-autosemantic words, punctuation• (some) multiword expressions -> 1 node

– some nodes added• based on word (mostly verb, noun) valency• some ellipsis resolution

– detailed dependency relation labels (functors)


Tectogrammatical Functors

• “Actants”: ACT, PAT, EFF, ADDR, ORIG

– modify: verbs, nouns, adjectives

– cannot repeat in a clause, usually obligatory

• Free modifications (~ 50), semantically defined– can repeat; optional, sometimes obligatory– Ex.: LOC, DIR1, ...; TWHEN, TTILL,...; RSTR; BEN, ATT, ACMP,

INTT, MANN; MAT, APP; ID, DPHR, CPHR, ...

• Special– Coordination, Rhematizers, Foreign phrases (#Forn),...


“syntactic” semantic

MW

Es

Deep Word Order Topic/Focus

• Example:

• Baker bakes rolls. vs. BakerIC bakes rolls.


Analyticaldep. tree:

Deep Word OrderTopic/Focus

• Deep word order:– from “old” information to the “new” one (left-to-

right) at every level (head included)– projectivity by definition (almost...)

• i.e., partial level-based order -> total d.w.o.

• Topic/focus/contrastive topic– attribute of every node (t, f, c)– restricted by d.w.o. and other constraints


Coreference

• Grammatical (easy)– relative clauses

• which, who– Peter and Paul, who ...

– control• infinitival constructions

– John promised to go home

– reflexive pronouns• {him,her,thme}self(-ves)

– Mary saw herself in ...


Johngo

he home

promisePRED

ACTPAT

ACT DIR3

Coreference

• Textual– Ex.: Peter moved to Iowa after he finished his PhD.


Peter Iowafinish

he PhD

movePRED

ACT DIR1TWHEN

ACT PAT

heAPP

Grammatemes

• Detailed functors (“subfunctors”)– needed for some functors:

• TWHEN: before/after• LOC: next-to, behind, in-front-of, ...• also: ACMP, BEN, CPR, DIR1, DIR2, DIR3, EXT

• Lexical (underlying)– number (Sg/Pl), tense, modality, degree of

comparison, mass-noun?; is_person_name, is_dsp_root, ...


MW

Es

VALENCY IN PDT


Prague Dependency Treebank & Valency

• Valency in the PDT– Valency lexicon for PDT– General valency lexicon

• Valency in deep vs. surface syntax– Links between the layers w.r.t. valency

• Valency and word sense– Sense-disambiguated occurrences:

• Links from data to the lexicon

• Valency in translation, text generation


Definition of Valency

• Ability (“desire”) of words (verbs, nouns, adjectives) to combine themselves with other units of meaning

• Properties of valency:– Specific for every word meaning (in general)

• leave: sb left sth for sb vs. sb left from somewhere• similar to PropBank leave.02 vs. leave.01

– Typically strongly correlates with surface form (Czech)• morphological case (~ ending), preposition+case, ...)

– Semantic constraints


Structure of Valency

• word (lemma) – word sense group 1

• valency frame:– slot1 slot2 slot3

• surface expression

– word sense group 2 • ...


vyměnit (to replace) vyměnit1

ACT PAT EFF

Nom. Acc.za+Acc.

vyměnit2

...

PDT-Vallex Entry

• dosáhnout: “to reach”, “to get [sb to do sth]”

• browser/user-formatted example:


MWEs in PDT-Vallex

• Types included:– Reflexive particle (se, si)

• smát se – to laugh• všimnout si – to notice

– Idiomatic constructions• dosáhnout svého - to achieve one’s goals• běhá mi mráz po zádech – to give me the shivers

– Light verb constructions (and similar)• uzavřit dohodu – to agree [on sth], strike an

agreement, ...• vzbuzovat pochybnosti – to doubt, to raise doubts


smát

_se

(t_l

emm

a)

DP

HR

(ar

gum

ent)

CP

HR

(ar

gum

ent)

Corpus ↔ Valency Lexicon


• Corpus:

ENTRY: uzavřít (to close) vf1: ACT(.1) CPHR({smlouva}.4)

ex: u. dohodu (close a contract)vf2: ACT(.1) PAT(.4)

ex.: u. pokoj (close a room, house)

Lexicon:

Sentence 2035: Sentence 15345: Sentence 51042:

Valency & Text Generation


• Using valency for...– ...getting the correct (lemma, tag) of verb arguments

• Example:

starat_se

PRED

Martin

ACT

tygr

PAT

Martin

....1..........

starat

V..............

o

...............

tygr

....4..........

VALLEX entry: starat (se) ACT(.1) PAT(o.[.4])

se

...............

Martin se stará o tygry.

“Martin takes care of tigers.”

“to take care of”

“tiger”

PARALLEL TREEBANK CZ-EN



Parallel Czech-English Annotation

• English text → Czech text (human translation)• Czech side (goal): all layers manual annotation• English side (goal):

– Morphology and surface syntax: technical conversion• Penn Treebank style -> PDT Analytic layer

– Tectogrammatical annotation: manual annotation• (Slightly) different rules needed for English

• Alignment– Natural, sentence level only (now)


English Annotation POS and Syntax

• Automatic conversion from Penn Treebank– PDT morphological layer

• From POS tags

– PDT analytic layer• From:

– Penn Treebank Syntactic Structure– Non-terminal labels– Function tags (non-terminal “suffixes”)

• 2-step process– Head determination rules– Conversion to dependency + analytic function

Czech-English Example


Dicku Darmane, zavolejte do své kanceláře! Dick Darman, call your office!

SUMMARY OF PART 1/1



PDT Treebanks at UFAL (written language)

• Czech– Prague Dependency Treebank

• Complex annotation, all levels, additional annotation

– Translation of Penn Treebank• Tectogrammatical layer only, no t/f

– Analytical, morphology: automatic tool

• English– Re-annotation of Penn Treebank

• Other languages– Arabic (own annotation)– Other: by conversion (HamleDT – 30 treebanks)

Prague Dependency Treebanks

• Annotation:– 4 layers:

• Words, lemmas/tags, surface dep. syntax, tectogrammatics

– Tectogrammatical layer:• No function words, semantic relations• Valency/verb arguments (some MWE features)

– Separate valency lexicon, fully linked from PDT nodes

• Coreference, Topic/focus, Discourse • Links back to analytical layer (parsing!)


Now also in the Universal Dependency format!

https://github.com/UniversalDependencies


Pointers• PDT 2.0 (the “Original”), newest version: PDT 3.0

– http://ufal.mff.cuni.cz/pdt2.0– http://ufal.mff.cuni.cz/pdt3.0

• PCEDT– http://ufal.mff.cuni.cz/pcedt2.0/

• PEDT– English side of PCEDT, additional: NE, coreference– http://ufal.mff.cuni.cz/pedt2.0/

• PADT (Arabic, morphology + surface syntax)– http://ufal.mff.cuni.cz/padt

• Other corpora, PDT-Vallex, EngVallex:– Search at http://lindat.cz

• LDC catalog numbers:– LDC2006T01 (PDT 2.0), LDC2004T23 (PADT 1.0), LDC2004T25 (PEDT 1.0)

• CoNLL 2009 shared task (7 languages, surface syntax + predicate arguments only)

– http://ufal.mff.cuni.cz/conll2009-st • HamleDT 2.0 (30 treebanks in unified format)

– http://ufal.mff.cuni.cz/hamledt

treebanks and mwes (part 1) jan hajič, pavel straňák, jiří mírovský institute of formal and...

Documents

pennbased treebanks

penn treebankdependency

adjp np cd

dictionarys npsbj prp

pred npdobj prp

arg1 npiobj det

phrasebased treebanksmight

basicsoriginal treebank