computational linguisticsnlpcl.kaist.ac.kr/~cs579_2020/slides/579-fall-2020-9.pdf · sentential...

Computational LinguisticsCS579: Fall Semester 2020

School of ComputingKorea Advanced Institute of Science and Technology

Jong C. Park

© All rights reserved.

We reviewed a first-order model checker.• Represent models and first-order formulas in Prolog;

Specify the evaluation process in Prolog The Satisfaction Definition in Prolog

• satisfy/4 for boolean connectives (negation, conjunction, disjunction, implication), existential and universal quantification, one-place and two-place predicates, equality

• Interpretation function We discussed the need to refine the Model Checker. We examined the relationship between First-Order

Logic and Natural Language.• Inferential and Representational Capabilities

Fall 2020 KAIST CS579: Computational Linguistics 2

Review of the Last Lecture

Compositionality Two Experiments The Lambda Calculus Implementing Lambda Calculus Grammar Engineering


Lambda Calculus

Goals Today• We study the following question:• How can we automate the process of

associating semantic representations with expressions of natural language?

• We discuss the idea of compositionality, experiment with ways of implementing compositional semantic construction, and lambda calculus.


Lambda Calculus

COMPOSITIONALITY

CS579: Computational Linguistics 5Fall 2020 KAIST

Question: • Given a sentence of English, is there a

systematic way of constructing its semantic representation?

A more specific question:• Is there a systematic way of translating simple

sentences such as “Vincent loves Mia” and “A woman snorts” into first-order logic?

• What do we mean by “systematic”?


Compositionality

Consider “Vincent loves Mia”. • Its semantic content is partially captured by the

first-order formula . • The proper name “Vincent” contributes the

constant to the representation.• The transitive verb “loves” contributes the

relation symbol , and “Mia” contributes . Can we generalize this process to the claim

that the words making up a sentence contribute all the bits and pieces needed to build the sentence’s semantic representation?


Another, related question:• From the symbols love, mia and vincent, why

can’t we also form love(mia,vincent) from the sentence “Vincent loves Mia” (in addition to, or instead of love(vincent,mia))?

• Are there any limitations?


Such limitations are explained by the notion of syntactic structure. • “Vincent loves Mia” isn’t just a string of words,

but presents a hierarchical structure: • a sentence (S) that is composed of the subject

noun phrase (NP) “Vincent” and the verb phrase (VP) “loves Mia”.

• This VP is in turn composed of the transitive verb (TV) “loves” and the direct object NP “Mia”.


The systematicity of the translation from natural language to first-order logic expressions amounts to using the additional information provided by syntactic structure to spell out exactly how the semantic contributions are to be gluedtogether.


Example


Vincent loves Mia (S)love(vincent,mia)

Vincent (NP)vincent

Mia (NP)mia

loves (TV)love(?,?)

loves Mia (VP)love(?,mia)

Compositionality• Semantic information flows from the lexicon,

thus each lexical item is associated with a representation.

• How is the information combined at all?• It is combined by making use of the hierarchy

provided by the syntactic analysis.


• Suppose the syntax tells us that some kind of sentential subpart (e.g., a VP) is decomposable into two sub-subparts (e.g., a TV and an NP). • Then the task is to describe how the semantic

representation of the VP subpart is to be built out of the representation of its two sub-subparts.

• If we succeed in doing this for all the grammatical constructions covered by the syntax, we will have given a compositionalsemantics for the language under discussion.


For the original question, we need to:• [Task 1] Specify a reasonable syntax for the

fragment of natural language of interest.• [Task 2] Specify semantic representations for

the lexical items.• [Task 3] Specify the translation composition-

ally.• We should specify the translation of all expressions in

terms of the translation of their parts, where “parts” refer to the substructures given to us by the syntax.


Natural Language Syntax via Definite Clause Grammars

Moreover, all the three tasks need to be carried out in a way that leads naturally to computational implementation.

As for Task 1, the syntactic analysis of a sentence will be identified with a tree, whose non-leaf nodes represent complex syntactic categories and whose leaves represent lexical items.

This approach will allow us to make use of Definite Clause Grammars (DCGs).


Refer to the files experiment1.pl, experiment2.pl, and experiment3.pl. s --> np, vp. noun --> [woman].np --> pn. noun --> [foot,massage].np --> det, noun. vp --> iv.pn --> [vincent]. vp --> tv, np.pn --> [mia]. iv --> [snorts].det --> [a]. iv --> [walks].det --> [every]. tv --> [loves].

tv --> [likes].


An example DCG implementation

The grammar accepts the following simple sentence:• Vincent walks.

In addition, by posing the query below, we can test whether “Mia likes a foot massage” is accepted by the grammar.?- s([mia,likes,a,foot,massage],[]).

The query below generates all grammatical sentences.?- s(X,[]).


TWO EXPERIMENTS


Experiment 1• Assume the following piece of DCG code:

pn(vincent) --> [vincent].pn(mia) --> [mia].iv(snort(_)) --> [snorts].tv(love(_,_)) --> [loves].


Two Experiments

How do we come up with the specific values?

• We choose to use the predicate arg/3.s(Sem) --> np(SemNP), vp(Sem),

{ arg(1,Sem,SemNP) }. np(Sem) --> pn(Sem). vp(Sem) --> tv(Sem), np(SemNP),

{ arg(2,Sem,SemNP) }. vp(Sem) --> iv(Sem).


pn(vincent) --> [vincent].pn(mia) --> [mia].iv(snort(_)) --> [snorts].tv(love(_,_)) --> [loves].

• Example interaction:?- s(Sem, [mia,snorts], []).Sem = snort(mia)

• We extend the DCG with the determiners “a” and “every”.

det(some(_,and(_,_))) --> [a].det(all(_,imp(_,_))) --> [every].noun(woman(_)) --> [woman].noun(footmassage(_)) --> [foot,massage].


pn(vincent) --> [vincent].pn(mia) --> [mia].iv(snort(_)) --> [snorts].tv(love(_,_)) --> [loves]. s(Sem) --> np(SemNP), vp(Sem),

{ arg(1,Sem,SemNP) }. np(Sem) --> pn(Sem). vp(Sem) --> tv(Sem), np(SemNP),


• We use arg/3 again for the determiners. np(Sem) --> det(Sem), noun(SemNoun),

{arg(1,SemNoun,X),arg(1,Sem,X),arg(2,Sem,Matrix),arg(1,Matrix,SemNoun)}.

• Sample interaction:?- np(Sem, [a,woman], []).Sem = some(X,and(woman(X),Y)).


missing piece of information

det(some(_,and(_,_))) --> [a].det(all(_,imp(_,_))) --> [every].noun(woman(_)) --> [woman].noun(footmassage(_)) -->

[foot,massage].

• We need to furnish VP with the missing piece of information.

• A proposal:s(Sem) --> np(Sem), vp(SemVP),

{ arg(1,SemVP,X), arg(1,Sem,X),arg(2,Sem,Matrix),arg(2,Matrix,SemVP) }.

• Problems:• Redundancy: We already have a rule ‘s --> np, vp’. • Wrong interaction: np(Sem) --> pn(Sem).• Another rule for quantified NPs in object position?


np(Sem) --> det(Sem), noun(SemNoun), {arg(1,SemNoun,X),

arg(1,Sem,X),arg(2,Sem,Matrix),arg(1,Matrix,SemNoun)}.

s(Sem) --> np(SemNP), vp(Sem), { arg(1,Sem,SemNP) }.

np(Sem) --> pn(Sem). vp(Sem) --> tv(Sem), np(SemNP),


Experiment 2• Lessons from experiment 1• We need to work with incomplete first-order

formulas to build representations.• We need a way of manipulating the missing

information. • We should take care to always associate

missing information with an explicit Prolog variable.


• Proposal:• We shall need three extra arguments: (1) one

for the bound variable, (2) one for the contribution made by the noun (restriction),and (3) one for the contribution made by the VP (nuclear scope).

det(X,Restr,Scope,some(X,and(Restr,Scope))) --> [a].

det(X,Restr,Scope,all(X,imp(Restr,Scope))) --> [every].


• Additional lexical changesnoun(X,woman(X)) --> [woman].iv(Y,snort(Y)) --> [snorts].tv(Y,Z,love(Y,Z)) --> [loves].

• Quantified noun phrases and proper namesnp(X,Scope,Sem) --> det(X,Restr,Scope,Sem),

noun(X,Restr).np(SemPN,Sem,Sem) --> pn(SemPN).


det(X,Restr,Scope,some(X,and(Restr,Scope))) --> [a].

det(X,Restr,Scope,all(X,imp(Restr,Scope))) --> [every].

noun(woman(_)) --> [woman].noun(footmassage(_)) --> [foot,massage].pn(vincent) --> [vincent].pn(mia) --> [mia].iv(snort(_)) --> [snorts].tv(love(_,_)) --> [loves].

• Redefined ruless(Sem) --> np(X,SemVP,Sem), vp(X,SemVP).vp(X,Sem) --> tv(X,Y,SemTV),

np(Y,SemTV,Sem).vp(X,Sem) --> iv(X,Sem).

• Problems?• Much of the work is done by the rules, requiring

rule-specific Prolog tricks such as variable doubling. • It is hard to think about the resulting grammar in a

modular way. • We need a better solution.


np(X,Scope,Sem) --> det(X,Restr,Scope,Sem), noun(X,Restr).np(SemPN,Sem,Sem) --> pn(SemPN).

THE LAMBDA CALCULUS


Lambda calculus works as a notational extension of first-order logic that allows us to bind variables using a new variable binding operator .• Occurrences of variables bound by should be

thought of as placeholders for missing information: they explicitly mark where we should substitute the various bits and pieces obtained in the course of semantic construction.

• An operation called –conversion performs the required substitutions.


The Lambda Calculus

A simple lambda expression:• x.man(x)

Notes• The prefix x. binds the occurrence of x in

man(x).• The prefix x. abstracts over the variable x. • We call expressions with such prefixes

lambda abstractions (or, more simply, abstractions).


We use the symbol @ to indicate the substitutions (or functional applications) we wish to carry out. • The expression x.man(x)@vincent gives rise

to man(vincent).

This substitution process is called -conversion, -reduction, or -conversion.


Additional process• -conversion

• the process of relabelling bound variables• -equivalence

• alphabetic-variants• 𝜆y.man(y) and 𝜆z.man(z)

• Why do we need -conversion?• It could happen that, when we apply 𝜆x. 𝐹 to an

argument A, some occurrences of a variable that is free in A becomes bound by a lambda operator or a quantifier when we substitute it into F.


A revised formulation• Assign lambda expressions to the different

basic syntactic categoriesevery: boxer: y.boxer(y)walks: z.walk(z)


every boxer (NP)𝜆u. 𝜆v. ∀x u@x → v@x @𝜆y. boxer y

every (DET)𝜆u. 𝜆v. ∀x u@x → v@x

boxer (Noun)𝜆y. boxer y

every boxer:

leading to:

every boxer walks: @ z.walk(z)

leading to: z.walk(z)

leading to: walk(x)





Full example


every boxer walks∀x boxer x → walk(x)




walks (VP)𝜆z.walk(z)

Proper names are handled similarly.Mia: 𝜆u.(u@mia)Vincent: 𝜆u.(u@vincent)


loves (TV)𝜆w.𝜆z.(w@𝜆x.love(z,x))

Mia (NP)𝜆u. u@mia

loves Mia (VP)𝜆z.(love(z,mia))

Vincent (NP)𝜆u. u@vincent

Vincent loves Mia (S)love(vincent,mia))

We assume a revised version of the three tasks (for the remainder of the course). • Task 1: Specify a DCG for the fragment of

natural language of interest.• Task 2: Specify semantic representations for

the lexical items with the help of the lambda calculus.

• Task 3: Specify the semantic representation R’of a syntactic item R whose parts are F and A , with the help of the lambda calculus.


SUMMARY


Lambda Calculus• Compositionality• Systematicity via syntactic structure• Definite Clause Grammars (DCGs)

• Two Experiments• The Lambda Calculus • operator, Lambda expression, lambda

abstraction, functional application, -conversion, -conversion


Summary

computational linguisticsnlpcl.kaist.ac.kr/~cs579_2020/slides/579-fall-2020-9.pdf · sentential...

Documents