Part 1Finite state modeling of knowledge of language

What are we doing?Language science

Fundamental science

Technology

Clinical Education

Fundamental science of human languageLanguage science

Technology

Clinical Education

What are we doing?

Fundamental science

What are we doing?Fundamental science of human language

Macro-levellanguage change

Usecommunication, social signaling

Processingfull process of encoding, producing, and decoding

Competenceunconscious

“knowledge” of a systematic mapping

between sound/articulation and

meaning

What “knowledge”?

Semantics

Syntax

Morphology

Phonology

Phonetics

What “knowledge”?

Semantics

Syntax

Morphology

Phonology

Phonetics

Delay between articulatory closure and voicing of following vowel for words starting with [p]

English speakers “know”

Spanish speakers “know”

0.000.250.500.751.00

0 25 50 75 100VOT

ndensity

0.000.250.500.751.00

0 25 50 75 100VOT

ndensity

What “knowledge”?

Semantics

Syntax

Morphology

Phonology

Phonetics

English speakers “know”

Turkish speakers “know”

[æp!lætʃ!kol!] (Apalachicola) [æp!l!tʃ!k!l! ] [æpælætʃɪkol!]

OK ill-formed * ill-formed *

[gel + di] (came) [gel + dɨ] [ al + dɨ] (bought)

OK ill-formed *

OK

What “knowledge”?

Semantics

Syntax

Morphology

Phonology

Phonetics

walks walked (*wought) has walkedbrings brought (*bringed/brang) brought (*brung)sings sang sung (*sought)

short shorter shortestconcise more concise most concise

pitsi+mik nigi+vunga (I am eating fish)

pitsi+tu+vunga (I am consuming fish)

(Inuttut)

What “knowledge”?

Semantics

Syntax

Morphology

Phonology

Phonetics

→ What do you know that John saw _? → What do you know if John saw _?

OK*→ NP V-s NP → NP has V-ed/en NP⟺

(John likes cats)(John has liked cats)

→ The old man left → The old old man left → The old old old ... man left

What “knowledge”?

Semantics

Syntax

Morphology

Phonology

Phonetics

→ Allison is taller than every boy. → Allison is a boy.→ Every warrior runs. → Every tall warrior runs.→ Some warriors run. → Some tall warriors run.

→

What are we doing?Chapter 1 of Chomsky 1965

(Aspects of the Theory of Syntax)

Grammar a [purported] description of the ideal speaker-hearer’s intrinsic competenceScope of theory disclaimer

Generative grammar [a grammar that is] perfectly explicit—in other words . . . it does not rely on the intelligence of the understanding reader

Explicitness criterion

Descriptive adequacy must assign to each of an infinite range of sentences a structural description indicating how this sentence is understood by the ideal speaker-hearer

Empirical evaluation

A simple model of syntaxAssigns to every sentence a single bit indicating that this sentence is “understood” by the ideal speaker-hearer; can extract a little more, but not much (e.g., only “dependency” is with the previous word)

Descriptive adequacyEmpirical evaluation

A simple model of syntaxGrammarScope of theory disclaimer

• Can only make binary decisions: does not explain gradient judgments of sentence acceptability

• Cannot explain sentence frequency • Says nothing about the internal

structure of words • Says nothing about the internal

structure of states

A simple model of syntaxGenerative grammarExplicitness criterion

• States. finite set • Arc. incoming, outgoing, arc label • Arc labels. finite set • Inputs. (finite-length) sequences of

elements from the set of arc labels • Output. TRUE for a sequence s iff there is a

path from state 0 to an accepting state where the arc labels match s exactly

Finite state automaton

Empirical problemsCenter embedding

The man comes

Empirical problemsCenter embedding

The man the mouse bites _ comesThe man comes

Empirical problemsCenter embedding

The man the mouse bites _ comesThe man comes

The man the mouse the man sees _ bites _ comes

Empirical problemsCenter embedding

NP NP VP VPNP VP

NP NP NP VP VP VP

Empirical problemsCenter embedding

NP NP VP VPNP VP

NP NP NP VP VP VPNP NP NP VP NP VP VP VP

The man the mouse the man chases the dog sees _ bites _ comesThe man the mouse the man chases him the dog sees _ bites _ comes

Empirical problemsMyhill-Nerode Theorem

Given an arbitrary string w, and the set of acceptable strings L according to a finite-state computation, w will fall into one of a finite set of equivalence classes, defined by what strings v make wv ∈ L.

String (w) Legal completions

Empirical problemsMyhill-Nerode Theorem

Given an arbitrary string w, and the set of acceptable strings L according to a finite-state computation, w will fall into one of a finite set of equivalence classes, defined by what strings v make wv ∈ L.

String (w) Legal completions

The men, The old men, The old old men, ... come

The man, The old man, The old old man, ... comes

Empirical problemsMyhill-Nerode Theorem

Given an arbitrary string w, and the set of acceptable strings L according to a finite-state computation, w will fall into one of a finite set of equivalence classes, defined by what strings v make wv ∈ L.

String (w) Legal completions

The, The old, The old old, ... men come, man comes, old men come, old man comes, old old men come, ...

The men, The old men, The old old men, ... come

The man, The old man, The old old man, ... comes

Empirical problemsMyhill-Nerode Theorem

Given an arbitrary string w, and the set of acceptable strings L according to a finite-state computation, w will fall into one of a finite set of equivalence classes, defined by what strings v make wv ∈ L.

String (w) Legal completions

ε The men come, The man comes, The old men come, The old man comes, ...

The, The old, The old old, ... men come, man comes, old men come, old man comes, old old men come, ...

The men, The old men, The old old men, ... come

The man, The old man, The old old man, ... comes

Empirical problemsMyhill-Nerode Theorem

Given an arbitrary string w, and the set of acceptable strings L according to a finite-state computation, w will fall into one of a finite set of equivalence classes, defined by what strings v make wv ∈ L.

String (w) Legal completions

ε The men come, The man comes, The old men come, The old man comes, ...

The, The old, The old old, ... men come, man comes, old men come, old man comes, old old men come, ...

The men, The old men, The old old men, ... come

The man, The old man, The old old man, ... comes

The man comes, The old man comes, ... ε

Empirical problemsMyhill-Nerode Theorem

Given an arbitrary string w, and the set of acceptable strings L according to a finite-state computation, w will fall into one of a finite set of equivalence classes, defined by what strings v make wv ∈ L.

String (w) Legal completions

ε The men come, The man comes, The old men come, The old man comes, ...

The, The old, The old old, ... men come, man comes, old men come, old man comes, old old men come, ...

The men, The old men, The old old men, ... come

The man, The old man, The old old man, ... comes

The man comes, The old man comes, ... εAll other strings {}

Empirical problemsMyhill-Nerode Theorem

NP NP VP VPNP VP

NP NP NP VP VP VP. . .

String (w) Legal completions

ε

Empirical problemsMyhill-Nerode Theorem

NP NP VP VPNP VP

NP NP NP VP VP VP. . .

String (w) Legal completions

ε NP VP, NP NP VP VP, ...

Empirical problemsMyhill-Nerode Theorem

NP NP VP VPNP VP

NP NP NP VP VP VP. . .

String (w) Legal completions

ε NP VP, NP NP VP VP, ...

NP

Empirical problemsMyhill-Nerode Theorem

NP NP VP VPNP VP

NP NP NP VP VP VP. . .

String (w) Legal completions

ε NP VP, NP NP VP VP, ...

NP VP, NP VP VP, NP NP VP VP VP, ...

VP VP

Empirical problemsMyhill-Nerode Theorem

NP NP VP VPNP VP

NP NP NP VP VP VP. . .

String (w) Legal completions

ε NP VP, NP NP VP VP, ...

NP VP, NP VP VP, NP NP VP VP VP, ...

NP NP VP VP

Empirical problemsMyhill-Nerode Theorem

NP NP VP VPNP VP

NP NP NP VP VP VP. . .

String (w) Legal completions

ε NP VP, NP NP VP VP, ...

NP VP, NP VP VP, NP NP VP VP VP, ...

NP NP VP VP, NP VP VP VP, ...

Empirical problemsMyhill-Nerode Theorem

NP NP VP VPNP VP

NP NP NP VP VP VP. . .

String (w) Legal completions

ε NP VP, NP NP VP VP, ...

NP VP, NP VP VP, NP NP VP VP VP, ...

NP NP VP VP, NP VP VP VP, ...

NP VP

Empirical problemsMyhill-Nerode Theorem

NP NP VP VPNP VP

NP NP NP VP VP VP. . .

String (w) Legal completions

ε NP VP, NP NP VP VP, ...

NP VP, NP VP VP, NP NP VP VP VP, ...

NP NP VP VP, NP VP VP VP, ...

NP VP ε

Empirical problemsMyhill-Nerode Theorem

NP NP VP VPNP VP

NP NP NP VP VP VP. . .

String (w) Legal completions

ε NP VP, NP NP VP VP, ...

NP VP, NP VP VP, NP NP VP VP VP, ...

NP NP VP VP, NP VP VP VP, ...

NP VP ε

VP

Empirical problemsMyhill-Nerode Theorem

NP NP VP VPNP VP

NP NP NP VP VP VP. . .

String (w) Legal completions

ε NP VP, NP NP VP VP, ...

NP VP, NP VP VP, NP NP VP VP VP, ...

NP NP VP VP, NP VP VP VP, ...

NP VP ε

NP NP VP VP

. . .

Empirical problemsMyhill-Nerode Theorem

NP NP VP VPNP VP

NP NP NP VP VP VP. . .

Given an arbitrary string w, and the set of acceptable strings L according to a finite-state computation, w will fall into one of a finite set of equivalence classes, defined by what strings v make wv ∈ L.

Thus, by the Myhill-Nerode theorem, the language above is not the set of acceptable strings according to any finite-state computation.

Limitations of finite-state automata

String (w) Legal completions

ε The men come, The man comes, The old men come, The old man comes, ...

The, The old, The old old, ... men come, man comes, old men come, old man comes, old old men come, ...

The men, The old men, The old old men, ... come

The man, The old man, The old old man, ... comes

The man comes, The old man comes, ... εAll other strings {}

• States. finite set • Arc. incoming, outgoing, arc label • Arc labels. finite set • Inputs. (finite-length) sequences of elements from the

set of arc labels • Output. TRUE for a sequence s iff there is a path from

state 0 to an accepting state where the arc labels match s exactly

243⊥

1

0

Chomsky’s conclusionClaim?

Claim?

A finite state model of English syntax is viable

Finite state computation is sufficient to describe speakers’ knowledge of syntax in any language

Answer: NO

Answer: NO (a fortiori)

“To generate non-finite state languages such as English [syntax] we need fundamentally different methods”

Demos for todayFoma, graphviz

Regular expressions

Writing automata by hand

Nondeterminism

Cross serial dependenciesNested dependencies are very common in the syntax of the worlds’ languages. Another syntactic phenomenon which is clearly not finite

state is found in Swiss German.

Shieber 1985

Cross serial dependencies

Jan säit das mer em Hans es huus hälfed aastriicheJan says that we Hans.DAT house.ACC helped paint“Jan says that we helped Hans paint the house”

Jan säit das mer de Hans es huus lönd aastriicheJan says that we Hans.ACC house.ACC let paint“Jan says that we helped Hans paint the house”

Nested dependencies are very common in the syntax of the worlds’ languages. Another syntactic phenomenon which is clearly not finite

state is found in Swiss German.

Shieber 1985

Cross serial dependencies

Jan säit das mer em Hans es huus hälfed aastriicheJan says that we Hans.DAT house.ACC helped paint“Jan says that we helped Hans paint the house”

Jan säit das mer de Hans es huus lönd aastriicheJan says that we Hans.ACC house.ACC let paint“Jan says that we helped Hans paint the house”

help ~ DATIVElet ~ ACCUSATIVE

Nested dependencies are very common in the syntax of the worlds’ languages. Another syntactic phenomenon which is clearly not finite

state is found in Swiss German.

Shieber 1985

Cross serial dependencies

help ~ DATIVElet ~ ACCUSATIVE

Nested dependencies are very common in the syntax of the worlds’ languages. Another syntactic phenomenon which is clearly not finite

state is found in Swiss German.

Jan says that we Hans.DAT house.ACC helped paintJan says that we Hans.ACC house.ACC let paint

Shieber 1985

Cross serial dependencies

Jan says that we Hans.DAT house.ACC helped paintJan says that we Hans.ACC house.ACC let paint

help ~ DATIVElet ~ ACCUSATIVE

Nested dependencies are very common in the syntax of the worlds’ languages. Another syntactic phenomenon which is clearly not finite

state is found in Swiss German.

Jan säit das mer d’chind em Hans es huus lönd hälfe aastriicheJan says that we children.ACC Hans.DAT house.ACC let help paint“Jan says that we let the children help Hans paint the house”

Shieber 1985

Cross serial dependencies

Jan says that we Hans.DAT house.ACC helped paintJan says that we Hans.ACC house.ACC let paint

help ~ DATIVElet ~ ACCUSATIVE

Nested dependencies are very common in the syntax of the worlds’ languages. Another syntactic phenomenon which is clearly not finite

state is found in Swiss German.

Jan says that we children.ACC Hans.DAT house.ACC let help paint. . .

Shieber 1985