a computational model of staged language acquisition

A ComputationalModel of Staged

Language Acquisition

JACK, KrisDRT/FAR/LIST/DTSI/SRCI/LIC2M

[email protected]



Kris Jack

Introduction

● Children appear to acquire language effortlessly

● They do not, however, do so overnight● Typically, they progress through stages of

linguistic development● Computational modelling can help us to better

understand the language acquisition process by estimating the problem and developing possible solutions

● A computational model that tackles such staged linguistic development is absent from current literature

● Overview of presentation

IntroductionChild LanguageModelsLATTestingResultsDiscussionConclusions



Kris Jack

Stages in Language Acquisition

● How does child language typically develop?

● Language acquisition is consistently described in stages (e.g. Brown, Pinker, Tomasello)

● Five stages from birth to 48 months:

● The Pre-linguistic Stage● The Holophrastic Stage● The Early Multi-word Stage● The Late Multi-word Stage● The Abstract Stage

6 12 18 24 30 360 42 48

Time (months)

Pre-linguisticHolophrastic

Early Multi-wordLate Multi-word

Abstract

IntroductionChild Language➢ Pre-linguistic➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractModelsLATTestingResultsDiscussionConclusions



Kris Jack

The Pre-linguistic Stage (1/2)

Little activity typically characterised as linguistic

Mini-stages including reflexive vocalisations, cooing, vocal play and babbling




Kris Jack

The Pre-linguistic Stage (2/2)

Can differentiate between languages across rhythmic families of stress-timed, syllable-timed or mora-timed (Mehler et al., 1996)

Sensitive to transitional probabilities within syllable sequences (Saffran et al., 1996)




Kris Jack

The Holophrastic Stage (1/2)

Begin when children achieve joint attention (Tomasello, 1995)

First utterances are typically holophrastic

Holistic or atomic units (e.g. “mummy”, “doggy”)

Even seemingly multi-word utterances are holistic (e.g. “I-wanna-do-it” (Pine and Lieven, 1993))




Kris Jack

The Holophrastic Stage (2/2)

Child and adult meanings for a holophrase often differ resulting in:

Underextensions (Reich, 1986)

Overextensions (Barrett, 1978)

Mismatches (Rodgon, 1976)




Kris Jack

The Early Multi-word Stage

Children regularly combine words to produce multi-word utterances

Novel combinations are made (e.g. “allgone sticky” (Braine, 1971))

Many utterances can be described using a pivot grammar (P)ivot (O)pen, O P, O O (Braine, 1963)

E.g. S = P O, where O = “mummy”, “sticky”, “duck”, “red” and P = “allgone”

Children are not sensitive to word-order (Clark, 1975; MacWhinney, 1980; de Villiers and de Villiers, 1978)

Function words and morphological markings tend to be omitted (Hyams, 1986)




Kris Jack

The Late Multi-word Stage

Child language becomes increasingly complex

The emergence of syntactic awareness

In English, word-order can define participant roles e.g. “Make the doggie bite the cat” (de Villiers and de Villiers, 1973)

Children are found to have an irregular, item-based, knowledge of language:

Verb islands (Tomasello, 1992)

Inconsistent use of determiners (Pine and Lieven, 1997)




Kris Jack

The Abstract Stage

Evidence of all of the grammatical machinery found in adults (Pinker, 1994)

The item-specific quality of child language gives way to a more abstract quality (Tomasello, 2003)

Strong generative capacity asserts itself

6 12 18 24 30 360 42 48

Time (months)



Abstract




Kris Jack

Stages Summary

● Pre-linguistic Stage

– little real linguistic activity

● Holophrastic Stage

– first words

● Early Multi-word Stage

– first word combinations

● Late Multi-word Stage

– word combinations with syntax

● Abstract Stage

– strong generative capacity

6 12 18 24 30 360 42 48

Time (months)



Abstract




Kris Jack

Goals

● What triggers the emergence of each stage?

● What accounts for the linguistic shape of each stage?

● We can produce computational models that estimate learning tasks faced by children to help us better understand the problem




Kris Jack

Language Models (1/2)

Many computational models have been produced to study language learning

The Miniature Language Acquisition Paradigm (Feldman et al., 1990)

Place a computational model in an environment with access to visual and acoustic stimuli (simulated or grounded)

Train the model by providing descriptions of visually-based scenes from a miniature language (e.g. “the red square is on top of the green circle”)

The model is said to have acquired the language when it can both comprehend and produce all sentences within the miniature language

IntroductionChild LanguageModels➢ Data Filtering➢ Incremental➢ IteratedLATTestingResultsDiscussionConclusions



Kris Jack

Language Models (2/2)

A computational model that demonstrates stage transitions from the Pre-linguistic Stage to the Abstract Stage is missing from current literature

Some models demonstrate stage-like learning by:

Externally modifying the training data

Modifying the functionality of the model

Internally modifying the training data




Kris Jack

Temporal Data Filtering

Training data are modified during training (e.g. Elman, 1993; Roy, 2002)

Elman (1993) trained a neural network to acquire sentences with both short and long distance dependencies

Success only when the training data were biased towards including more examples of short distance dependencies in early learning and long distance dependencies in late learning

Unrealistic assumption

Although infant direct speech contains different characteristics to adult directed speech, children are still exposed to complex sentences from birth

DataData

DataData

DataData

ModelModelx < t <= y

0 < t <= x

y < t <= z




Kris Jack

Incremental Learning

Functionality is modified during training (Elman, 1993; Dominey and Boucher, 2005)

Elman (1993) trained a neural network to acquire sentences with both short and long distance dependencies

The neural network’s ‘short-term memory’ was incrementally increased during learning, allowing the network to acquire both types of dependencies

Incremental learning has problems:

When should increments be made?

What unit of data should be restricted, syllables

Transitions are not clean and clear

DataData ModelModelt > x

t > 0

t > y

ModuleModule

ModuleModule

ModuleModule




Kris Jack

Iterated Learning

Training data are modified by model

In modelling language evolution, Kirby (2002) shows that learning through cultural transmission can modify the structure of a language

One generation of agents learn a language and then produce a progressively more structured language for teaching to the next generation (Iterated Learning)

However, this is language evolution, not language acquisition

Training data are constant for children

DataData

ModelModel

DataData

ModelModel

DataData

ModelModelModelModel




Kris Jack

Problem

Can we model the stages of language acquisition when:

The functionality of the model is kept constant AND

The training data provided to the model are constant?




Kris Jack

Language Acquisition Toolkit

LAT provides a framework for investigating staged language acquisition

Aim:

Develop a computational model that demonstrates realistic stages of linguistic development

Investigating the problem within a Miniature Language Acquisition Framework where:

The language contains enough complexity to allow the model to demonstrate stages of development

The language is not so complex that it cannot be learned in entirety

Concentrating on comprehension

6 12 18 24 30 360 42 48

Time (months)



Abstract

IntroductionChild LanguageModelsLAT➢ Architecture➢ Learning Env.➢ World Perception➢ Cross-situational➢ Holophrastic➢ Early Multi-word➢ Late Multi-word➢ AbstractTestingResultsDiscussionConclusions



Kris Jack

LAT General Architecture

Environment(real or

simulated)

l1 R esources

renv

r1

r2

.

.

.rx

l2

r1

renv

r2

r1

.

.

.

l3

r3

r2

lx

rx

rx1

c2

r2

c1

r1

c3

r3

cx

rx

.

.

.

Learn ingM odules

C o m prehens ionM odules

WorldPerception Module

renv

Sensory stimuli




Kris Jack

LAT Architecture Instantiated

Environment(real or

simulated)

lcrosssit R esources

renv

rcrosssit

rholophrastic

rearly

rlate

rabstract

lholophrastic

rcrosssit

renv

rholophrastic

rcrosssit

learly

rearly

rholophrastic

cearly

rearly

cholophrastic

rholophrastic

clate

rlate

cabstract

rabstract

Learn ingM odules



renv

Sensory stimuli

llate

rlate

rholophrastic

labstract

rabstract

rlate




Kris Jack

Learning Environment

The model plays the Scene Building Game (Jack, 2005)

Algorithm:

1.The model watches a scene containing a single geometric object

2.Another geometric object is added to the scene and the event is described

3.Return to 1. Notice that the landmark object is described using the

definite article “the” and the new object is described using the indefinite article “a”

a blue circle below the red square




Kris Jack

The World Perception Module

The World Perception Module encodes events (renv

)

Simulated Visual input – detects colour, shape and relative positions

{below(rel), blue(2), circle(2), horizontal_even(rel), red(1), square(1)}

Simulated Acoustic input – event description is perceived as a sequence of syllables

a blue cir cle be low the red square

Joint attention is assumed from the outset

6 12 18 24 30 360 42 48

Time (months)



Abstract




Kris Jack

Cross-situational Learning Module

Aims

Find similarities between observed events

Derive possible form-meaning pairs

Create new resource rcross-sit




Kris Jack

Cross-situational Learning

Method

Form of Cross-situational Analysis (Siskind, 1996)

Words co-occur more often with their intended meanings than with other meanings

Example

Equal string parts are found Equal feature value parts are found New extensions are derived



a green star to the low er right of the blue tri ang gle

{below(rel), blue(1), green(2), right(rel), star(2), triangle(1)}

1)

2)




Kris Jack

Cross-situational Learning

Method

Form of Cross-situational Analysis (Siskind, 1996)

Words co-occur more often with their intended meanings than with other meanings

Example

Equal string parts are found Equal feature value parts are found New extensions are derived



a green star to the low er right of the blue tri ang gle

{below(rel), blue(1), green(2), right(rel), star(2), triangle(1)}

a {below(rel), blue(1)}

a {below(rel), blue(2)}

the {below(rel), blue(1)}

the {below(rel), blue(2)}

blue {below(rel), blue(1)}

blue {below(rel), blue(2)}

1)

2)

rcross-sit




Kris Jack

Holophrastic Learning Module

Aims

Reduce ambiguity by removing homonyms

Reduce ambiguity by removing synonyms

Create new resource rholophrastic

blue {blue(1,2)}red {red(1,2)}green {green(1,2)}square {square(1,2)}cir cle {circle(1,2)}tri ang gle {triangle(1,2)}be low {below(rel), vertical_even(rel)}a bove {above(rel), vertical_even(rel)}blue square {blue(1,2), square(1,2)}blue cir cle {blue(1,2), circle(1,2)}

.

.

.




Kris Jack

Holophrastic Learning Module

In order to remove homonyms and synonyms

1. create abstract extensions

3. keep only the most similar meaning for each form using

3. erase meanings of all extensions that have similarities lower than other extensions with the same meaning, where similarity is

(1) red {red(1)}

is merged with

(2) red {red(2)}

to produce

(3) red {red(1, 2)}

Similarity M i , F j=Frequency F j ,M i

Frequency F j

Similarity F i , M j=Frequency M j , Fi

Frequency M j

rholophrastic




Kris Jack

Holophrastic Comprehension Module

Comprehension:

Given a string to comprehend, the model searches r

holophrastic for extensions that contain the string

From those found, the meaning of the extension that is most similar is returned




Kris Jack

Early Multi-word Learning Module

Aims

To find compositional relationships between form-meaning pairs in r

holophrastic

show no sensitivity to word order nor object roles

Create new resource rearly

blue{blue(1,2)}

cir cle{circle(1,2)}

blue cir cle{blue(1,2), circle(1,2)}

green{green(1,2)}


green cir cle{circle(1,2), green(1,2)}

1) 2)

the{}

red cir cle{circle(1,2), red(1,2)}

the red cir cle{circle(1), red(1)}

3)




Kris Jack


Finding compositionality

an extension is a function of two other extensions when its

form is equal to the concatenation of the forms of the parts (ignoring word order)

meaning is equal to the feature set union of the parts (ignoring object roles)




Kris Jack






do extensions 1) 2) and 3) express a compositional grammar fragment?

blue{blue(1,2)}



1) 2) 3)

blue cir cle = blue + cir cle ?

blue cir cle = cir cle + blue ?




Kris Jack







blue{blue(1,2)}



1) 2) 3)

{blue(1,2), circle(1,2)} = {blue(1,2)} U {circle(1,2)} ?




Kris Jack







blue{blue(1,2)}



1)

2) 3) cir cle{circle(1,2)}

blue{blue(1,2)}


1)

3) 2)

OR




Kris Jack



To reflect child sensitivity during this period, each grammar fragment must have a part that appears in another fragment

This produces a form of pivot grammar where pivot parts can appear with many open parts (Braine, 1963)

blue{blue(1,2)}



green{green(1,2)}



1) 2)

the{}



3)




Kris Jack





blue{blue(1,2)}



green{green(1,2)}



1)

3)

2)

the{}



3)




Kris Jack





blue{blue(1,2)}



green{green(1,2)}



1) 2)

rearly




Kris Jack

Early Multi-word Comprehension Module

Comprehension:

Given a string to comprehend, the model searches rearly

for grammar fragments whose parts can be combined to make the string

For each grammar fragment found, its meanings are combined through union and each result is returned

blue{blue(1,2)}



{blue(1,2)} U {circle(1,2)} = {blue(1,2), circle(1,2)}

comprehend blue cir cle




Kris Jack


Comprehension:




blue{blue(1,2)}



{blue(1,2)} U {circle(1,2)} = {blue(1,2), circle(1,2)}

comprehend cir cle blue

however...




Kris Jack


Comprehension:




the{}



however...

{} U {circle(1,2), red(1,2)} = {circle(1,2), red(1,2)}

comprehend the red cir cle





Kris Jack

Late Multi-word Learning Module

Aims

To find compositional relationships between form-meaning pairs in r

holophrastic

show sensitivity to word order and object roles

Create new resource rlate

blue{blue(1,2)}



((1,2)>(1,2))((1,2)>(1,2))

1)

the{}


the blue cir cle{blue(1), circle(1)}

((1,2)>(1))()

2)

a{}

blue cir cle{circle(1,2), blue(1,2)}

a blue cir cle{blue(2), circle(2)}

((1,2)>(2))()

3)




Kris Jack




form is equal to the concatenation of the forms of the parts (consider word order)

meaning is equal to the feature set union of the parts, after transfomation (consider object roles)




Kris Jack







the{}



1) 2) 3)

blue cir cle = blue + cir cle ?




Kris Jack







{blue(1), circle(1)} = T({}, ()) U T({blue(1,2), circle(1,2)}, ((1,2)>(1))) ?

the{}



1) 2) 3)

i.e. {blue(1), circle(1)} = {} U {blue(1), circle(1)} ?




Kris Jack







the{}



((1,2)>(1))()

1)

2) 3)




Kris Jack



Each grammar fragment must have a part that appears in another fragment

They appears on the same side (same word order)

AND the transformations are the same (same object roles)

1) 2)

the{}



((1,2)>(1))()

the{}

blue square{blue(1,2), square(1,2)}

the blue square{blue(1), square(1)}

((1,2)>(1))()

3)

a{}


a red cir cle{circle(2), red(2)}

((1,2)>(2))()




Kris Jack



Each grammar fragment must have a part that appears in another fragment

They appears on the same side (same word order)

AND the transformations are the same (same object roles)

rlate

1) 2)

the{}



((1,2)>(1))()

the{}


the blue square{blue(1), square(1)}

((1,2)>(1))()




Kris Jack

Late Multi-word Comprehension Module

Comprehension:

Given a string to comprehend, the model searches rlate


For each grammar fragment found, its meanings are mapped and then combined through union and each result is returned

comprehend blue cir cle

blue{blue(1,2)}



((1,2)>(1,2))((1,2)>(1,2))

T({blue(1,2)},((1,2)>(1,2))) UT({circle(1,2)},((1,2)>(1,2))) = {blue(1,2), circle(1,2)}




Kris Jack


Comprehension:




comprehend cir cle blue

blue{blue(1,2)}



((1,2)>(1,2))((1,2)>(1,2))

Meaning not found




Kris Jack


Comprehension:




comprehend a blue cir cle

a{}


a blue cir cle{blue(1,2), circle(1,2)}

((1,2)>(2))()

T({},()) UT({blue(1,2),circle(1,2)},((1,2)>(2))) = {blue(2), circle(2)}





Kris Jack

Abstract Learning Module

Aims

Derive phrasal categories from grammar fragments in r

late

Derive grammar rules that make reference to phrasal categories

Create new resource rabstract

((1)>(1))

((1)>(1), (rel)>(rel))((2)>(2))

((1,2)>(1))

()

((1,2)>(2))

((rel)>(rel))

((1,2)>(1,2))((1,2)>(1,2))((1,2)>(1,2)) ((1,2)>(1,2))

a{}

blue{blue(1,2)}


the{}

red{red(1,2)}

square{square(1,2)}


red square{red(1,2), square(1,2)}

the red square{red(1), square(1)}

a blue cir cle{blue(2), circle(2)}

be low the red square{below(rel), horizontal_even(rel), red(1), square(1)}

a blue cir cle be low the red square{below(rel), blue(2), circle(2), horizontal_even(rel), red(1), square(1)}

be low{below(rel), horizontal_even(rel)}

()




Kris Jack


Aims

Derive phrasal categories from grammar fragments in r

late

Derive grammar rules that make reference to phrasal categories

Create new resource rabstract

NP

((1)>(1))

((1)>(1), (rel)>(rel))((2)>(2))

((1,2)>(1))

()

((1,2)>(2))

((rel)>(rel))

((1,2)>(1,2))((1,2)>(1,2))((1,2)>(1,2)) ((1,2)>(1,2))

DET1 ADJ N DET

2 ADJ N

NP

NP2

NP1 POS

S

REL

()




Kris Jack


Creating phrasal categories:

Phrasal categories can be derived from the grammmar fragments in r

late by assuming that their members share

distributional information

1)

blue{blue(1,2)}



((1,2)>(1,2))((1,2)>(1,2))

2)

blue{blue(1,2)}

square{square(1,2)}


((1,2)>(1,2))((1,2)>(1,2))




Kris Jack


1)

blue{blue(1,2)}



((1,2)>(1,2))((1,2)>(1,2))

2)

blue{blue(1,2)}

square{square(1,2)}


((1,2)>(1,2))((1,2)>(1,2))


square{square(1,2)}

Phrasal category 1:








Kris Jack


1)

blue{blue(1,2)}

Phrasal category 1


((1,2)>(1,2))((1,2)>(1,2))

2)

blue{blue(1,2)}

square{square(1,2)}


((1,2)>(1,2))((1,2)>(1,2))


square{square(1,2)}

Phrasal category 1:








Kris Jack



Phrasal categories often share similar members

Subset categories are replaced by their superset categories


square{square(1,2)}

star{star(1,2)}

tri ang gle{triangle(1,2)}

Phrasal category 2:cir cle

{circle(1,2)}

square{square(1,2)}

Phrasal category 1:

is replaced by




Kris Jack


Creating grammar rules:

Grammar rules are created by linking the grammar fragments that make reference to phrasal categories

((1,2)>(2))

DET1

NP

NP1

()

((1,2)>(1,2)) ((1,2)>(1,2))

ADJ N

NP




Kris Jack


Creating grammar rules:

Grammar rules are created by linking the grammar fragments that make reference to phrasal categories

((1,2)>(2))

DET1

NP

NP1

()

((1,2)>(1,2)) ((1,2)>(1,2))

ADJ N

rabstract




Kris Jack

Abstract Comprehension Module

Comprehension:

Given a string to comprehend, the model searches rabstract

that can be instantiated to make the string

The accompanying meanings is returned

((1,2)>(2))

DET1

NP

NP1

()

((1,2)>(1,2)) ((1,2)>(1,2))

ADJ N

If DET1 = a, ADJ = red, ye low and N = cir cle, heart,

could comprehend a red cir cle, a red heart, a ye low cir cleand a ye low heart




Kris Jack

Summary

Environment(real or

simulated)

lcrosssit R esources

renv

rcrosssit

rholophrastic

rearly

rlate

rabstract

lholophrastic

rcrosssit

renv

rholophrastic

rcrosssit

learly

rearly

rholophrastic

cearly

rearly

cholophrastic

rholophrastic

clate

rlate

cabstract

rabstract

Learn ingM odules



renv

Sensory stimuli

llate

rlate

rholophrastic

labstract

rabstract

rlate




Kris Jack

Testing

Training

The model is trained to learn a miniature language by observing event-description pairs

100 sets of 125 event-description pairs were randomly generated

After each pair is entered, the model is tested for comprehension of a set of strings

The results are used to determine the model's stage of linguistic development

6 12 18 24 30 360 42 48

Time (months)



Abstract

IntroductionChild LanguageModelsLATTesting➢Miniature Lang.➢Templates➢RequirementsResultsDiscussionConclusions



Kris Jack

String Templates for Testing

String Templates

To observe the performance of the model, it is tested for comprehension of a set of strings, shown below by template

String Template Example String Total

Shape cir cle 8

Colour red 8

Position a bove 6

Half Relative Position to the u pper 4

Relative Position a bove the 8

Object red cir cle 82 = 64

Indefinite Object a red cir cle 82 = 64

Definite Object the red cir cle 82 = 64

Object Relative Position a red cir cle above the 83 = 512

Relative Position Object a bove the red cir cle 83 = 512

Event a red cir cle a bove the red square 85 = 32,768

Total No: 34,018




Kris Jack

Judging Linguistic Stage

When the model comprehends strings, its behaviour can be described in terms of stages:

Pre-linguistic – no comprehension

Holophrastic – comprehension of any string

Early – string is comprehended as a composite of its parts

Late – string is comprehended as a composite of its parts that require use of syntactic markings

Abstract – a set of NPs are comprehended, where the set includes all known ADJs and Ns

End point – all sentences are successfully comprehended

6 12 18 24 30 360 42 48

Time (months)



Abstract




Kris Jack

Results

ho lo ph ra st ic

ea rly mu lti w

or d

lat e mult iw

o rd

po st a bs tr ac t

ab st ra ct

0 20 40 60 80 100 120

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Stages of Language Acquisition

Holo

EarlyLate

Abstract

End

No. Events Observed

% o

f Req

uire

men

ts M

et

Onsets: Holo (1); Early (11.9); Late (23.88); Abstract (49.83); End (88.04)Lengths: Holo (10.9); Early (11.98); Late (25.95); Abstract (38.21)

IntroductionChild LanguageModelsLATTestingResults➢Holophrastic➢Early Multi-word➢Late Multi-word➢AbstractDiscussionConclusions



Kris Jack

The Holophrastic Stage

● The majority of strings comprehended (94%) are atomic units in the language

● Word segmentation and association with appropriate meanings

● Discovery of atomic units in the miniature language e.g. cir cle and to the u pper


Object3%

Colour29%

Half-Relative-Position

21%

Relative-Position13%

Definite-Object1% Complete-Event

2%

Shape31%




Kris Jack


● Under-extensions such as cross means {cross(1, 2), above(rel)}

● Over-extensions such as blue cross means {blue(1, 2)}

● Mismatches such as low means {pink(1, 2), below(rel)}


Object3%

Colour29%


21%

Relative-Position13%

Definite-Object1% Complete-Event

2%

Shape31%




Kris Jack


● There is a rise in the comprehension of composite strings

● Strings are comprehended as a composite of their parts e.g. red cir cle is comprehended from the meanings of red and cir cle


Indefinite-Object2%

Definite-Object4%

Object-Relative-Position

1%Complete-Event

2%

Shape24%

Object22%

Relative-Position9%


12%

Colour24%




Kris Jack


No sensitivity to syntactic markings

a red square means the same as the red square and red square

a red square a bove the green cir cle means the same as a green cir cle a bove the red square


Indefinite-Object2%

Definite-Object4%


1%Complete-Event

2%

Shape24%

Object22%

Relative-Position9%


12%

Colour24%




Kris Jack


● There is a rise in the comprehension of composite strings that require sensitivity to syntactic markings

● Strings are comprehended as a composite of their parts e.g. “red cir cle” is comprehended from the meanings of “red” and “cir cle”


Complete-Event8%

Relative-Position-Object

2%

Shape6%

Colour6%


3%

Relative-Position3%


9%

Definite-Object17%

Indefinite-Object16%

Object30%




Kris Jack


Sensitivity to syntactic markings

a red square means a novel red square

the red square means the existing red square

red square means a novel or the existing red square

a red square a bove the green cir cle is differentiated from a green cir cle a bove the red square


Complete-Event8%


2%

Shape6%

Colour6%


3%

Relative-Position3%


9%

Definite-Object17%

Indefinite-Object16%

Object30%




Kris Jack

The Abstract Stage

● The majority of strings comprehended are Complete Events

● Complete Events are comprehended as the composition of multiple atomic units

The Abstract Stage

Object1%

Indefinite-Object1%

Definite-Object1%


3%


3%

Complete-Event91%




Kris Jack

The Abstract Stage

● A form of rote learning is displaced by generative comprehension

● Grammars are derived that allow any string in the miniature language to be comprehended from a relatively small exposure to examples

The Abstract Stage

Object1%

Indefinite-Object1%

Definite-Object1%


3%


3%

Complete-Event91%




Kris Jack

Discussion

Behavioural stages emerge:

● In the same order as found in child language

● At similar time intervals as found in child language

● With similar developmental characteristics as found in child language

What accounts for this similar developmental trajectory bearing in mind that:

● Training data are kept constant?

● The model’s functionality is kept constant?




Kris Jack

Explaining Development

The Modular Architecture

Each module concentrates on performing a different task

Each task requires a different amount of training to produce results

A new behaviour emerges when a learning mechanism solves a task for the first time

Modules depend upon training data which can be internally filtered by other modules




Kris Jack


● The Cross-situational Learning Module receives unaltered training data

● The Holophrastic Modules breaks the language down into atomic units producing holophrastic behaviour

● The Early Multi-word Modules begins to reconstruct the language by discovering compositional relationships

● Both the Holophrastic and Early Multi-word Modules work simultaneously, allowing the model to continue learning words while discovering compositions

● The Late Multi-word Module begins to reconstruct the language by discovering compositionality WITH sensitivity to word order and syntactic markings

● Why is there such a gap between the results produced by the Early and Late Multi-word Comprehension Modules when they perform similar tasks?




Kris Jack


● Why is there such a gap between the results produced by the Early and Late Multi-word Comprehension Modules when they perform similar tasks?

The Late Multi-word Learning Module is performing a more complex task than the Early Multi-word Learning Module

● The Late Multi-word Learning Module has tougher constraints (word-order and transformations must match in constructions). Given the fragments;

● The Early Multi-word Learning Module can keep the fragments but the Late Multi-word Learning Module cannot

((1,2)>(1,2)) ((1,2)>(1,2))

blue{blue(1,2)}



() ((1,2)>(2))

a{}

blue{blue(1,2)}

a blue{blue(2)}




Kris Jack


● The Abstract Modules produce results much earlier than the abstract stage begins

● Much of the generative capacity in the model comes from the Abstract Modules

● The Abstract Comprehension Module accounts for comprehension of novel strings even during the early multi-word stage

● It is inappropriate to think of each module’s contribution to comprehension as being limited to a particular stage

● It is better to think of each stage as being the result of all modules producing the best results that they can given their experience




Kris Jack

Conclusions

A computational model that demonstrates a similar developmental trajectory as found in child language has been produced

There is linguistic maturation without physical maturation

The model is given a realistic exposure to training data

A Modular Structure accounts for much of the developmental shape

The stages emerge on a reasonable timescale

The stages emerge in the same order

Different modules focus upon different problems

Different linguistic behaviours may be the best indicators of underlying learning mechanisms in children

Children may also have a modular framework for learning and comprehending




Kris Jack


Coming soon...

● Language Acquisition Toolkit (LAT) online– Freely available for research

– GNU Licence

– Run language acquisition simulations with your own modules

– Compare results within a common framework

a computational model of staged language acquisition

Technology

multiword holistic

late multiword villiers

multi lat word utterances

multiwordsequences saffran

language acquisition

knowledge of language

prelinguistic vocal

child language adults