double r theory january 2011 jerry ball human effectiveness directorate 711 th human performance...

Double R TheoryJanuary 2011

Jerry Ball

Human Effectiveness Directorate

711th Human Performance Wing

Air Force Research Laboratory

2

Theoretical FoundationsLanguage Representation and Processing

• Double R Grammar

– Cognitive Linguistic theory of the grammatical encoding of referential and relational meaning

• Double R Process

– Psycholinguistic theory of the processing of English text into Double R Grammar based representations

• Double R Model

– Computational implementation using the ACT-R cognitive architecture and modeling environment

DoubleRTheory.com

3






• Double R Model


DoubleRTheory.com

4

Theoretical FoundationsGrounding Language in Experience

• Symbol Grounding (Harnad)

– Ungrounded symbols are meaningless

– There must be a chain from abstract to perceptually grounded concepts that provides the grounding for abstract concepts

• Perceptual Symbol Systems (Barsalou)

– No purely abstract concepts

– The brain is a highly evolved perceptual (motor) organ

– Imagery simulates perceptual experience

• Embodied Cognition (Lakoff et al.)

– Abstract concepts are often understood via metaphorical association with more concrete concepts

• Good is up—Bad is down; Life is a journey

5

Theoretical FoundationsSituation Model

• Situation Model (Kintsch et al.)

– Originally viewed as a propositional text base (van Dijk & Kintsch)

• Elaboration of propositions in linguistic input

– Now viewed as a Spatial-Imaginal (and Temporal) representation of the objects and situations described by linguistic expressions and encoded directly from the environment (Zwann et al.)

• Non-propositional (in part)

• Non-textual

• No available computational implementations

– Provides grounding for linguistic representations

6

Abstract Concepts vs. Perceptually Grounded Language

“pilot”

XY-123

(aka PILOT)

Real World Mental Box

Cognition

The Prevailing “Cognitive Psychological” View

Concept ~ abstract amodal fixed point in conceptual space

“pilotpilot”

Perception

7

Abstract Concepts vs. Perceptually Grounded Language

“pilot” “pilotpilot”

Mental BoxReal World

perception

An Emerging “Embodied Cognition” View

gro

un

din

g

perception

Explicit(Perceptual)

Perceptual

Symbol

Do we really need

abstract concepts?

How are they learned?

Cognition is the

simulation of

perceptual experience

Concept ~ dynamic and tangled interconnections of associated experiences

8

Language is Grounded in a Situation Model

The horse runs

the

horse

runs

subj

headSRE

ORE

PRED

Dynamic mental simulation of horse running would be better!

SRE: Situation Referring Expression

ORE: Object Referring Expression

PRED: Predicate

refers

refers

9

Language is Grounded in a Situation Model

The paint runs

the

paint

runs

subj

headSRE

ORE

PRED

Dynamic mental simulation of paint running would be better!

refers

refers

Each experience of a running event changes the RUN concept!

10

Guiding Linguistic Principles

• Jackendoff’s (1983) Grammatical Constraint:

…one should prefer a semantic theory that explains otherwise arbitrary generalizations about the syntax and the lexicon…a theory’s deviations from efficient encoding must be vigorously justified, for what appears to be an irregular relationship between syntax and semantics may turn out merely to be a bad theory of one or the other

11

Guiding Linguistic Principles

• Langacker’s Cognitive Grammar (1987, 1991)

– Grammar is simply the structuring and symbolization of semantic content

– Exclusionary Fallacy – one analysis, motivation, categorization, cause, function or explanation for a linguistic phenomenon necessarily precludes another

– Rule/List Fallacy – the assumption, on grounds of simplicity, that particular statements (i.e. lists) must be excised from the grammar of a language if general statements (i.e. rules) that subsume them can be established

12

Construction Grammar (Fillmore, Goldberg, Sag, etc.)

• Constructions—the basic units of grammar—are pairings of form, function and meaning

the man hit the ballform

function

meaning

subject predicator object

HIT(AGENT:MAN PATIENT:BALL)

“concepts”uppercase word syndrome

semantic roles

13

Construction Grammar

• Declarative Clause + Intransitive Verb construction

– The woman sneezed

• Decl Clause + Transitive Verb construction

– The man hit the ball

• Wh-Question + Ditransitive Verb + Passive constr.

– Who was given the ball?

• Decl Clause + Intrans Verb + Causative constr.

– The woman sneezed the napkin off the table

14

X-Bar Theory

• Key element of Chomsky’s Generative Grammar from the 1970’s to the 1990’s

• Theory of the universal structure of all languages

– Autonomous from meaning

• X-Bar structure presumed to be innate (not learned)

• Replaced Phrase Structure Grammar component of earlier theory (e.g. S NP VP; NP Det N; …)

• Has gone thru several major revisions resulting in more and more complex syntactic representations

• Subsumed by other theoretical considerations in Chomsky’s Minimalist Program (circa. 1995)

15

X-Bar Theory (Chomsky 1970)

Specifier X-Bar

XP

X (head)

Complement(s)

Generalization over Syntactic Categories – NP, VP, AP, PP

XP Spec X-Bar

X-Bar X (Head) Comp(s)

Universal structure of all languages except that relative locations can vary (e.g. complements may occur before or after head)

Universal structure of all languages – very strong claim – generative linguists spent next 20+ years trying to demonstrate it!

16

X-Bar Theory ~ 1993

Universal structure of all languages Something went seriously wrong!

XP

(X’’)

Spec X-Bar

(X’)

X Comp

(YP)

Locally adheres to X-Bar Schema Globally very complex!

X-Bar schema Universal structure of clause

17

X-Bar Theory (adapted in Ball 2007)

Specifier X-Bar

XP

X (head)

Complement(s)

Generalization over grammatical categories – referring expression

referential

layer

relational

layer

What’s right about X-Bar Theory:

1.Referential layer

2.Relational layer

3.Grammatical functions: specifier, head, complement, modifier (but need to be semantically motivated)

head – semantically most significant element

specifier – indicates referential

function

complements – arguments of

relational head

18

Simpler Syntax(Culicover & Jackendoff 2005)

• Reaction against the complex syntactic representations of modern mainstream generative grammar

– Against syntactocentrism

• If there is a level of meaning representation, then syntactic representations can be simpler

– Flat as opposed to deeply nested syntactic representations

• Culicover & Jackendoff are former students of Chomsky

19

Comprehensive Grammars of English

• Cambridge Grammar (Huddleston & Pullum, 2002)

– Informed by linguistic theory, but attempts to cover most of English with all its exceptions

– Adds functional categories to syntactic representations

• Longman’s Grammar (Quirk et al., 1985)

– Focus on basic functions of linguistic elements

– In the spirit of Functional Grammar as opposed to Chomsky’s Generative Grammar

20

Double R Grammar

• Theory of the grammatical encoding of Referential and Relational meaning

• Derived from X-Bar Theory prior to the introduction of functional heads (Chomsky, 1970)

• Grammatical Functions (GFs) explicitly represented

– Phrase Level: Specifier, Head, Complement, Modifier

– Clause Level: Specifier, Head, Subject (Comp), Modifier

• Specifier + Head Referring Expression (Max Proj)

– All the grammatical info needed to support reference

• Specifier = locus of Referential meaning

• Head = locus of Relational meaning

21

Basic Nominal – X-Bar Theory (Chomsky 1970)

the captain

D

N

NP

Lexical Item

Syntactic Category

NP D N-Bar

N-Bar N

the captain

N-BarHead (implicit)

Specifier (implicit)

Maximal Projection

Grammatical Functions are implicit in syntactic representation

• Noun is head of nominal (NP)• N-bar level is required

Later – D reanalyzed

as head of DP

(functional head)

DP D-bar NP

22

Basic Nominal – Simpler Syntax

the captain

D N

NP

Double Line marks head

Lexical Item

• Noun is head of nominal (NP)• No N-bar level

Syntactic Category

One (explicit) phrase level GF:

1.Head

NP D N (head) the captain

23

Basic Nominal – Cambridge Grammar

the captain

Det:D

Head:N

NP

GF:

Syntactic Category

Lexical Item

Four phrase level (NP) GF’s:

1.Head

2.Determiner

3.Complement

4.Modifier

• Noun is head of nominal (NP)• N-Bar level allowed, but not required

Note: Nominal = N-bar, not NP

for H&P

NP Det:D Head:N the captain

24

Nominal ~ Referring Expression

John Lyons, Semantics, Vol 2, 1977, p. 445

“Looked at from a semantic point of view, nominals are referring expressions”

“They are expressions which have a certain potential for reference”

25

Basic Nominal – Double R

the captain

Spec Head

Object Referring Expression (ORE)

Referential pole Relational pole

D N

Grammatical

Function (GF)Grammatical/ Lexical Construction

Lexical Item

Four phrase level GF’s:

1.Head

2.Specifier

3.Complement

4.Modifier

• Nominal ~ Object Referring Expression• Noun is head of nominal (NP)• No N-bar level

ORE Spec Head; Spec D; Head N the captain

26

Basic Clause X-Bar Theory ~ 1970s

Joe run

VP

S

NP

N V

S NP VP

VP Specv V-Bar

V-Bar V

Joe runs

Structure of S

not explained

by X-Bar Theory

circa. 1970

-- no specifier or head of S

N-bar V-barSpecv

TENSEpres

Deep Structure gets transformed

into Surface Structure

(Transformational Grammar)

-- TENSEpres + run runs

Deep Structure

27

Basic Clause – Simpler Syntax

Joe run

VP

S

NP

N V

Syntactic Tier:

GF Tier: Subject

• Head of S not specified in Culicover (2009)

• In Jackendoff (2002), no lexical items in syntactic tier

S NP AUX VP Joe runs

Clause level GF’s:

1.Subject

2.Object

3.Second Object

CS: RUN(AGENT:X)

AUX

TENSEpres

affix hoppingVestige of

Transformational Grammar

28

Basic Clause – Cambridge Grammar

Joe runs

Predicate:VP

Clause

Subj:NP

Head:N

Predicator:V


1.Predicate ~ Head of Clause

2.Subject ~ External Complement

3.Modifier

Additional phrase level (VP) GF:

1.Predicator ~ Head of VP

Clause Subj:NP Predicate:VP

Predicate:VP Predicator:V

Joe runs

No equivalent to determiner at clause level!

29

Basic Clause – Double R

(Spec+)Head

Situation Referring Expression (SRE)

Subj|

ORE|

(Spec+)Head|

PN|

Joe

Vfin

|runs


1.Head

2.Specifier

3.Subject ~ External Complement

4.Modifier

• SRE ~ Clause or S

SRE Subj (Spec+)Head

Subj ORE

(Spec+)Head Vfin

Joe runs

Grammatical Construction

Specification fused with Head

30

Basic Clause X-Bar Theory ~ 1970s

VP

S

NP|

N-bar|N|

Joe

S NP VP

VP Specv V-Bar (head)

V-Bar V (head) NP (comp)

NP D (spec) N-bar (head)

N-Bar N (head)

Joe kicks the ball

N-bar|N|

ball

D|

the

NP

Later – VP reanalyzed as head of S & Subject NP

reanalyzed as

specifier of S – left of head so must be spec!

S NP (spec) VP (head)

Later – tense reanalyzed as head of IP;

S reanalyzed as CP (complementizer phrase) with C-bar = IP

CP IP = C-bar (head)

IP (inflection phrase) NP (spec) I-bar (head)

I-bar I (tense head) VP (comp)

Specv

TENSEpres

V-Bar

V|

kick

31

Basic Clause – X-Bar Theory ~ 1980s

VP

CP

NP|

N-bar|N|

Joe

CP IP = C-bar (head)

IP NP (spec) I-bar (head)

I-bar I (head) VP (comp)

VP V-Bar (head) NP (comp)

V-Bar V (head)

Joe kicks the ball

N-bar|N|

ball

D|

the

NP

IP = C-bar

Later – additional levels proposed:

AgrP (agreement) AgrSP, AgrOP

NegP (negation)

ModP (modality)

Etc.

Complement of I-bar

Spec of IP

(subject)

Head of CP

I-bar

V-Bar

V|

kick

I|

TENSEpres

Sentence now adheres to X-Bar Theory!

32

Basic Clause – X-Bar Theory ~ 1993

C’ = C-bar

TP = IP

Subj Agreement

Obj Agreement

VP way down here!

Structure below VP not shown

Universal clausal structure of all languages!

Some languages have object agreement, so universal, innate structure must have this layer!

|Joe?

kick the ball

TENSEpres

Joe kicks the ball

33

Basic Clause – Simpler Syntax

VP||V|

kick

S

NP||N|

Joe

Syntactic Tier:

GF Tier: Subject Object

S NP AUX VP

VP V (head) NP

Joe kicks the ball

CS: KICK(AGENT:X PATIENT:Y )

N|

ball

D|

the

NP

AUX

TENSEpres

affix hopping

34

Basic Clause – Cambridge Grammar

Predicator:V|

kicks

Clause

Subj:NP

|Head:

N|

JoeHead:

N|

ball

Det:D|

the

Predicate:VP

Obj:NP

Clause Subj:NP Predicate:VP

Predicate:VP Predicator:V Obj:NP

Joe kicks the ball

Additional phrase level (VP) GF:

1.Object ~ Complement

36

Basic Clause with Auxiliary – Simpler Syntax

VP[PROG-PART]

||V

[PROG-PART]

|kick

S

NP||N|

Joe

Syntactic Tier:

GF Tier: Subject

N|

ballObject

AUX

S NP AUX VP

VP V (head) NP

D|

the

NP

Joe is kicking the ball

CS: KICK(AGENT:X PATIENT:Y )

TENSEpres VAUX

|be

affix hopping

37

Basic Clause with Auxiliary – Cambridge Grammar

Predicator:V|

kicking

Clause

Subj:NP

|Head:

N|

Joe

Head:N|

ball

Det:D|

the

Predicate:VP

Obj:NP

Predicate:VP

Predicator:V|is

Clause Subj:NP Pred:VP

Pred:VP Pred-or:V Comp:Clausebare

Comp:Clausebare Pred:VP

Pred:VP Pred-or:V Obj:NP


Comp:Clausebare

|

catenative verbs

bare clause

(no subj or tense)

No specifier GF

head of

clause!

39

Possessive Nominal – Simpler Syntax

Joe book

N

NP

NP

NP NP’s N Joe’s book

’s

No label!

40

Possessive Nominal – Cambridge Grammar

Joe’s book

Head:N

NPPlain

Subj+Det:NPGen

Fused subject-determiner

H & P allow GF’s to be fused

– consistent with grammatical evidence

NPPlain Subj+Det:NPGen Head:N Joe’s book


1.Subj ~ Complement

41

Possessive Nominal – Double R

Joe ’s book

RefPt Head

Possessive Object Referring Expression (ORE)

Spec

Referential pole Relational pole

NPoss-Mkr

ORE – (Spec+)Head –

PN

Poss-ORE RefPt+Spec Head Joe’s book


1.Ref Pt ~ Complement


42

Clause without Main Verb – Simpler Syntax

PP||P|

on

S

N|

book

Syntactic Tier:

GF Tier: Subject

N|

table

S NP AUX PP

D|

the

NP

NP

D|

the

the book is on the table

CS Tier: BE(THEME:X, ON(THEME:Y))

AUX

TENSEpres VAUX

|be

45

Clause without Main Verb – Simpler Syntax

PP||P|

on

S

N|

book

Syntactic Tier:

GF Tier: Subject

N|

table

S NP AUX PP

D|

the

NP

NP

D|

the

the book’s on the table

CS Tier: BE(THEME:X, ON(THEME:Y))

AUX

TENSEpres VAUX

|be

46

Clause without Main Verb – Cambridge Grammar

Head:P|

on

Clause

Head:N|

book

Head:N|

table

Det:D|

the

Comp:PP

Obj:NP

Predicate:VP

Predicator:V|’s



Comp:PP Head:P Obj:NP

Det:D|

the

Subj:NP

Don’t see how H&P can allow

GF’s to be fused

– inconsistent with grammatical evidence


fused?

47

Clause without Main Verb – Double R

Head|P|

on

SRE

Head|N|

bookHead

|N|

table

Spec|D|

the

Head|

Pred-Prep

Obj|

ORE

Spec|

Aux|’s

SRE Subj+Spec Head

Subj ORE

Spec Aux

Head Pred-Prep

Pred-Prep Head Obj

Subj|

ORE

Spec|D|

the


48

Passive Clause – Simpler Syntax

VP

S

Syntactic Tier:

GF Tier: Subject

PP

bev-aux

S NP AUX VPbe

VPbe be (head) VP[PASS]

VP[PASS] V[PASS] (PPby)

PPby by NP

V[PASSIVE]

|

take

VP[PASSIVE]

NP

byp NP

the book was taken by Joe

CS: TAKE(AGENT:X, PATIENT:Y)

AUX

TENSEpast

the book

Joe

49

Passive Clause – Cambridge Grammar

Predicator:V|

taken

Clause

Head:N|

book

Comp:NP

Joe

Comp:P|

by

Predicate:VP

Comp:PP

Predicate:VP

Predicator:V|

was


Pred:VP Pred-or:V Comp:Clausebare

Comp:Clausebare Pred:VP


Det:D|

the

Subj:NP


Comp:Clausebare

|

58

Grammatical Features of Nominals in English

• Definiteness – definite, indefinite, universal

• Number – singular, plural

• Animacy – human, animate, inanimate

• Gender – male, female

• Person – first, second, third

• Case – subj, obj, gen (2)

59

Why We Need Grammatical Features

• Definiteness:

– Give me the ball (definite)

– Give me a ball (indefinite)

• Number

– The men (plural) kick the ball (sing). They (plural)…

• Animacy

– The man (human) kicks the ball (inanimate). It (inanimate)…

• Gender

– The man (male) likes the woman (female). She (female)…

or

60

Simple Nominal

the man

singular human male

“the” projects definite to obj-refer-expr

“man” projects singular, human and male

definite

61

Grammatical Features of Clauses in English

• Tense – present, past, non-finite

• Aspect – perfect, progressive

• Modality – “could”, “should”, “must”…

• Polarity – negative

• Voice – active, inactive, passive

62

Simple Clause

…could not have gone

finite

present

perfect

active

“could”

negative

“could” projects finite present tense and modality

“not” projects negative polarity

“have gone” projects perfect aspect and active voice

“could not” recognized as a multi-word unit

63

Summary

• Representations matter!

• Language is complex!

• In complex systems, overall coherence is more important than overall simplicity!

– Einstein: make your theory as simple as possible, but no simpler!

– Computational implementation necessitates coherence

• If axioms + logical reasoning incoherence or a system that is obviously false, then question your axioms or your “logical” reasoning

– E.g. if innateness assumptions lead to overly complex representations, then question the innateness assumptions or the reasoning

64






• Double R Model


DoubleRTheory.com

65

Double R Process

• Serial, incremental, pseudo-deterministic language processor with a non-monotonic context accommodation mechanism (with limited parallelism) that is capable of making modest changes to the evolving representation

• Parallel, interactive, highly context sensitive, probabilistic mechanism which uses all available information to make the best choice at each choice point

• Processor presents the appearance and efficiency of deterministic processing, but is capable of handling the ambiguity which makes truly deterministic processing impossible

66

Double R Process

• Construction Driven Language Processing

– Activation, Selection and Integration of constructions corresponding to the linguistic input

• Lexical items in the input activate constructions

– Activation depends on current input, current context, and prior history of use

– “give” activates ditransitive verb construction

• Most highly activated construction is selected

• Selected construction is integrated with evolving representation

67

Double R Process

• Adhere to well-established cognitive constraints on Human Language Processing

• Don’t use any obviously cognitively implausible mechanisms!

• Adhering to cognitive constraints may actually facilitate the development of functional NLP systems

– Pushes development in directions that are more likely to be successful given inherently human nature of language processing

– You don’t know what you’re giving up when you adopt cognitively implausible mechanisms!

68

ACT-R Cognitive Architecture

• Theory of human cognition based on 40+ years of psychological research (Anderson, 2007)

– Computational implementation since 1993

• Combines a symbolic procedural memory implemented as a production system with a symbolic frame based declarative memory (DM)

• Includes modules for vision, audition, and motor processing

– Supports interaction with external world

http://act-r.psy.cmu.edu/

69


• Procedural memory is the central component

– All modules interface to procedural memory via buffers (e.g. goal buffer, retrieval buffer, visual buffer)

• Productions have “subsymbolic” utilities

– Productions match against buffers of other modules

– Intentional module goal buffer is primary driver of behavior

– Matching production with highest utility is selected for execution

70


• DM contains chunks which are frame based

– Chunk type + slot-value pairs (aka AVMs)

• Chunk types are organized into a single inheritance hierarchy

• Chunks have “subsymbolic” activations based on current input, current context and prior history of use

• Chunks are retrieved from memory by execution of a production which specifies a retrieval template

– DM chunk with highest activation that matches retrieval template is retrieved (soft constraint retrieval)

71


Intentional Module(not identified)

Declarative Module(Temporal/Hippocampus)

Goal Buffer(DLPFC)

Retrieval Buffer(VLPFC)

Matching (Striatum)

Execution (Thalamus)

Selection (Pallidum)

Pro

du

ctio

ns

(Bas

al G

ang

lia)

Visual Module(Occipital/etc)

Manual Module(Motor/Cerebellum)

Visual Buffer(Parietal)

Manual Buffer(Motor)

External World

modules &

buffers

mapped to

brain

regions

72


• Supports timing of cognitive processing

– Production execution takes 50 ms

– DM chunk retrieval time depends on level of activation of retrieved chunk

– Timing of motor events based on Fitts’ Law

– Used for empirical validation of models

• Provides a powerful debugging environment

73

Architectural Constraints

• No language specific module

– although buffers and productions accessing buffers might be viewed as a module

• Forward chaining productions with no backtracking

• Limited pattern matching – not full unification

• Serial bottleneck

– only one production can execute at a time

• Modules interact with production system via buffers

– buffers have limited capacity for storing current context

• Activation spreads in parallel

• Activation and Utility subject to noise

74

Constraints on Human Language Processing

• Visual World Paradigm (Tanenhaus et al. 1995)

– Subjects presented with a visual scene

– Subjects listen to auditory linguistic input describing scene

• Immediate determination of meaning

– Subjects look immediately at referents of linguistic expressions, sometimes before end of expression

• Incremental processing

• Interactive processing (Trueswell et al. 1999)

– Ambiguous expressions are processed consistent with scene

“the green…”

“put the arrow on the paper into the box”

75

• According to Crocker (1999), there are three basic mechanisms for dealing with ambiguity in natural language

– Serial processing with backtracking or reanalysis

– Deterministic processing with lookahead (Marcus 1980)

– Parallel processing with alternative analyses carried forward in parallel (Gibson 1991; MacDonald, Pearlmutter & Seidenberg 1994; Trueswell & Tanenhaus 1994)

• According to Lewis (2000) “…existing evidence is compatible only with probabilistic serial-reanalysis models, or ranked parallel models augmented with a reanalysis component.”

• According to Gibson & Pearlmutter (2000) “noncompetitive ranked parallel models” are most consistent with the empirical evidence


76

• Serial and deterministic with reanalysis for pathological input

– Empirical evidence that we don’t carry forward all representations in parallel – Garden Path Sentences

• “The horse raced past the barn fell” (Bever 1970)

• “The old train the young” (Just & Carpenter, 1987)

– Empirical evidence that we don’t retract previously built representations (Christianson et al. 2001)

• “While Mary dressed the baby sat up on the bed”

– In a post test, a majority of subjects answered yes to the question “Did Mary dress the baby?”

– Processing doesn’t slow down with increasing length of non-pathological input

– Typically only aware of a single interpretation


77

• Parallel and probabilistic with reanalysis for pathological input

– Empirical evidence that we may carry forward multiple representations in parallel – Garden Path Effects can be eliminated with sufficient context

– Empirical evidence that dispreferred representations can affect processing time (Gibson & Pearlmutter 2000)

• It’s extremely difficult to empirically falsify either

– Could be parallel slow down or occasional switch between serial alternatives that causes effect

• Don’t have all the answers, but maybe it’s both!

– A parallel, probabilistic substrate may make a pseudo-deterministic serial processing mechanism possible!


78

Cognitively Implausible Mechanism

• Serial processing with algorithmic backtracking

– Algorithmically simple, but…

• Computationally intractable for NLP which is highly ambiguous

• Context which led to dead end is retracted on backtracking

–Why give up the context?

– How do we know it’s a dead end?

• Practical Consequences

– No hope for on-line processing in real-time in large coverage NLP system

– No hope for integration with speech recognition system

– Performance degrades with length of input

– Can’t easily handle degraded or ungrammatical input

79

Cognitively Implausible Mechanism

• Multiple pass or multi-stage parsing

– Separate passes tokenize and assign part of speech

• Can’t use full context in each pass

• Errors get propagated

– Separate pass builds structure

• Typically limited to using part of speech of words

– Separate pass determines meaning


– Difficult to do on-line processing in real-time

– Can’t easily integrate with speech recognition

– Performance degrades with length of input

– Limited context available to handle ambiguity at each stage

80

Outrageously Implausible Mechanism!

• Parsing input from right to left (Microsoft NLP system)

– May have engineering advantages, but…

• Presumes a staged approach to NLP

• Completely ignores cognitive plausibility

• Practical consequences

• Impossible to do on-line processing in real-time

–Must wait for end of input

• Nearly impossible to integrate with speech recognition

81

Cognitively Plausible Mechanism?

• Deterministic processing with lookahead

– Many ambiguities resolved by looking ahead a few words, but…

• Don’t know how far to look ahead

– Cognitive plausibility improved by limiting amount of lookahead

• 3 constituent lookahead (Marcus 1980)

• 1 word lookahead (Henderson 2004)

• Practical consequences

– Difficult to use with eager algorithms for which there is good empirical evidence (immediate determination of meaning)

– The smaller the lookahead, the less deterministic

82

Cognitively Plausible Mechanism?

• Parallel processing with multiple analyses carried forward

– “Full parallelism – where every analysis is pursued – is not psychologically possible” (Crocker 1999)

– Cognitive plausibility improved by limiting number of analyses carried forward and ranking alternatives (bounded ranked parallelism) and not having analyses compete


– The longer and more ambiguous the input, the less likely to have the correct representation in the parallel spotlight – necessitating a reanalysis mechanism

– Impractical if multiple representations must be built at each choice point as opposed to just being selected

83

Cognitively Plausible Mechanism

• Pseudo-deterministic, serial processing mechanism with context accommodation operating over a parallel, probabilistic substrate

– Parallel, probabilistic substrate proposes best alternative given current context

– Processor proceeds as though it were serial and deterministic, but accommodates the subsequent input as needed

– Integrates the advantages of parallel processing with an essentially serial processing mechanism


– How to accommodate when things go seriously wrong?

– Mechanism is essentially non-monotonic

84

Cognitively Plausible Mechanism

• Serial, Pseudo-deterministic processing and Context Accommodation

– Uses ACT-R’s production system

– Builds structure

– Limited parallelism

• Parallel, Probabilistic processing

– Uses ACT-R’s declarative memory

– Retrieves existing structure from memory

85

• If current input is unexpected given the prior context, then accommodate the input

– Adjust the representation

– Coerce the input into the representation

• The following example demonstrates the context accommodation mechanism

– “no target airspeed or altitude restrictions”

Context Accommodation

86

no

“no” object referring expression

“no” projects obj-refer-expr and functions as specifier

“head-indx” indicates head expected

“bind-indx” provides index for binding

87

no target

“target” head

tree structures generated automatically with dynamic visualization

tool (Heiberg, Harris & Ball 2007) based on phpSyntaxTree

software (Eisenberg & Eisenberg)

integration

88

no target airspeed

“airspeed” head

Accommodation

of second noun via

function shift and overriding

override

function shiftintegration

89

no target airspeed or altitude

“or altitude” conj

Conjunction integrated into

noun

integration

90

no target airspeed or altitude restrictions

“restrictions” head

Appearance of parallel processing!

Accommodation

of new head via

function shift and override

function shift

override

integration

91

• Coercion

– “the hiking of Mt Lemon” – head of nominal

• “hiking” construed objectively, arguments not expressed (“of Mt Lemon” functions as a modifier)

– “a Bin Laden supporter”

• Proper Noun functions as modifier

– “you’re no Jack Kennedy”

• Proper Noun functions as head (following specifier)

– “the newspaper boy porched the newspaper” – nonce expression (H. Clark 1983)

• “porched” construed as transitive action

Types of Accommodation

92

• Override

– Single word vs. Multi-Word Expression (MWE)

• “kicked…” transitive verb

– “kicked the bucket” idiomatic expression

• “take…” transitive verb

– “take a hike” “take five” “take time” “take place” “take out” “take my wife, please” “take a long walk off a short pier” … many idiomatic expressions

• Not possible to carry all forward in parallel

– Morphologically simple vs. complex

• “car…” noun (sing)

– “carpet…” noun (sing)

– “carpets” noun (plur)

– “carpeting” noun (sing) or verb


93

• Grammatical Function Shift

– “he gave it to me”

• direct object (initial preference due to inanimacy)

– “he gave it the ball”

• direct object (initial preference) indirect object

– “he gave her the ball”

• indirect object (initial preference due to animacy)

– “he gave her to the groom”

• indirect object (initial preference) direct object


94

• Nominal Head Override

• “he gave her the dog biscuit” head = her

• “he gave her dog the biscuit” head = dog

• Grammatical Function “Juggling”

– “he gave the…” indirect object

– “he gave the very old bone…” direct object

– “he gave the very old bone collector…” indirect object

– “he gave the very old dog…” indirect object

– “he gave the very old dog collar…” direct object

– “he gave the very old dog to me” direct object


95


– “he said that…”

• In context of “said”, “that” typically functions as a complementizer

– But subsequent context can cause a function shift from

• complementizer

– “he said that she was happy”

• To nominal specifier to

– “he said that book was funny”

• To nominal head

– “he said that.”


96


– “pressure” vs. “pressure valve” vs. “pressure valve adjustment” vs. “pressure valve adjustment screw” vs. “pressure valve adjustment screw fastener” vs. “pressure valve adjustment screw fastener part” vs. “pressure valve adjustment screw fastener part number”

• Serial nouns (and verbs) incrementally shift from head to modifier function as each new head is processed

• Functions like lookahead, but isn’t limited

• Not clear if a bounded ranked parallel mechanism can handle this!

– 2n possibilities if head or modifier at each word


97

• Modulated Projection

– “the rice” vs. “rice”

– “the” projects a nominal and functions as a specifier

– In the context of “the” “rice” is integrated as the head of the nominal

– When there is no specifier, “rice” projects a nominal and functions as the head without separate specification


nominal

spec

the

head

rice

vs. head

rice

nominal

“the rice” “rice”

98

Grammatical Feature Accommodation

99

• Context Accommodation is part and parcel of the pseudo-deterministic processing mechanism

– Not viewed as a repair mechanism (Lewis 1998)

• Processor proceeds as though it were deterministic, but accommodates the input as needed

• Gives the appearance of parallel processing in a serial, deterministic mechanism

Summary of Context Accommodation

100

Combining Serial, Deterministicand Parallel, Probabilistic Mechanisms

Tree Supertagging

Construction Activation

& Selection

Supertag

Stapling

Construction

Integration

Rule ApplicationLexical Rule Selection

Rule

SelectionRule Application

Rule Selection & Application

Parallel Probabilistic

Serial Deterministic

Parallel Distributed Processing

CFG

PCFG

Lexicalized

PCFG

Double R

Probabilistic

LTAG

PDP

Pseudo

Deterministic

Range

Nondeterministic

The parallel probabilistic substrate makes a pseudo-deterministic serial processing mechanism possible!

101

Questions?

double r theory january 2011 jerry ball human effectiveness directorate 711 th human performance...

Documents

double r theory

grounded concepts

actr cognitive architecture

grounded language pilot

journey slide

linguistic representations

concrete concepts good

processing of english