1
Functional decomposition of spoken word production and its control
Ardi Roelofs
F.C. Donders Centre for Cognitive Neuroimaging
Max Planck Institutefor Psycholinguistics
Nijmegen Institute for Cognition and Information
Collège de France, Paris, Jeudi 11 Mai 2006
We produce about 2 to 5 words per second.
About 1 error per 1,000 words.
Vocabulary of more than 30,000 words.
A few numbers
With 1 hour talking per day, one produces
about 4.000.000 words per year,
about 200.000.000 words in half a century,
i.e., each word, on average, about 6500 times.
POIRE
2
“I can resist everything except temptation.”
Oscar Wilde
“Une distraction, à l’instant où l’irritationagit, est toujours punie par uneprolongation de durée de l’acte.”
F. C. Donders (1868)
(Archives néerlandaises des sciences exactes et naturelles, 1868)
Subtraction method
“[One] discovers in comparing the incoming and outflowing blood that oxygen has been consumed [by the brain].”
F. C. Donders (1868)
1868 2006
(2004)
Left-lateralized perisylvian network
3
J. Ridley Stroop(1935) CAFÉ
CLÉTANGO
BLEUVERT
ROUGE
Stroop asymmetry
J. Ridley Stroop(1935)
BLEU ROUGE XXXTask: Name ink color!
J. Ridley Stroop(1935)
VERTTask: Read word aloud!
BLEU ROUGE
4
Two accounts of Stroop asymmetry
Pathway strength (GRAIN)1
Functional architecture (WEAVER++)2
Prefrontal-parietal control network
Anterior Cingulate Cortex
Two accounts of ACC function
Conflict detection (GRAIN)1
Top-down regulation (WEAVER++)2
Stroop asymmetry and its time course.
2
Outline of the remainder
Control processes.3
Structure of lexical memory.
1
Stroop asymmetry and its time course.
2
Outline of the remainder
Control processes.3
Structure of lexical memory.
1
5
Slips of the tongue
Rudolf Meringer& Karl MayerVersprechen und Verlesen (1895)
Form
Meaning
“une contusion à la chambre”jambe
“Il faut faire grossir les gros obèses”maigrir
“Franche Dimance”c ch
Slips of the tongue
Syntax“sur la femme de leurs genoux”
les genoux de leur femme
Rossi & Defare (Int. J. Psycholing., 1995)
WEAVER++ Levelt, Roelofs, & Meyer (BBS, 1999)
Meaning
pommier
MEANING
masculine NUMBER singularGENDER
plural
Syntax
Form
Conceptualizing
Lemma retrieval
Form encoding
FORM FORMFORM
<pommier> <-z>
/p/ /o/ /m/ /j/ /e/
1 2 3 4
[pom] [je]
ONSET
<le>
/l/
1 2
/@/
NUCLEUS CODA
5
ONSETNUCLEUS
APPLE TREE(X)
STEM(X)
PLANT(X)
HAS-A
IS-A
Conceptualizing
Lemma retrieval
Form encoding
Articulation
POMMIER
“pommier”
ConceptualizingLemma retrieval
Form encoding
Articulation
POMMIER
“pommier”
Indefrey & Levelt (Cognition, 2004)
6
Stroop asymmetry and its time course.
2
Outline of the remainder
Control processes.3
Structure of lexical memory.
1
Stroop asymmetry and its time course.
2
Outline of the remainder
Control processes.3
Structure of lexical memory.
1
“Hi, I amColin MacLeod”
“Hi, I amColin MacLeod”
Stroop asymmetry
J. M. Cattell (1886)
“Reading is faster than color naming.”
ColorWord
7
-300 -200 -100 0 100 200 300
SOA (in milliseconds)
COLOR FIRST COLOR SECOND
VERT
-300 -200 -100 0 100 200 300
SOA (in milliseconds)
WORD FIRST WORD SECOND
VERT
RT E
FFEC
T (
mse
c)
-100
-50
0
50
100
150
-400
-200
-300
-100
400
1000
300
200
Color naming
SOA (msec)
-100
-50
0
50
100
150Word reading
-400
-200
-300
-100
400
1000
300
200
incongruentcongruent
WORD FIRST WORD SECOND COLOR FIRST COLOR SECOND
Glaser & Glaser (JEP:HPP, 1982)
Melara & Algom (Psych. Rev., 2003)
Melara & Mounts (Mem. & Cogn., 1993)
VERTVERT
(hear: bleu) incongruent
(hear: rouge) congruent
control task: keypress
Roelofs (Mem. & Cogn., 2005)
Repeating is as fast as color naming
400
425
450
475
500
525
550
-300 -200 -100 0 100 200 300
SOA(in milliseconds)
COLOR NAMING LATENCY (in milliseconds)
incongruentcongruent
8
-300 -200 -100 0 100 200 300
300
350
400
450
500
550
600incongruentcongruent
SOA(in milliseconds)
WORD REPETITION LATENCY (in milliseconds)
Two accounts of Stroop asymmetry
Pathway strength (GRAIN)1
Functional architecture (WEAVER++)2
Cohen et al. (1990, Psych. Rev.), Botvinick et al. (2001, Psych. Rev.)
WORD INPUTROUGE VERT
word reading
colornaming
VOCAL RESPONSE“rouge" “vert"
COLOR INPUT
TASK
GRAINincongruentcongruent
GRAIN
Cohen & Huston (1994, Att. & Perf.)
300
350
400
450
500
550
SOA (msec)
-300 -200 -100 0 100 200 300
COLOR FIRST COLOR SECOND
RT
(mse
c)
Word reading
-300 -200 -100 0 100 200 300
WORD FIRST WORD SECOND
500
550
600
650
700
750
SOA (msec)
Color naming
(b)
(a)
colorperception
word-formperception
word-formencoding
lemmaretrieval
conceptualidentification
articulation
COLOR
WORD
OVERT SPEECH
WEAVER++ model of spoken word production
Levelt et al. (BBS, 1999), Roelofs (Cognition, 1992, 1997; Psych. Rev., 2003, 2004)
9
IF task is color namingTHEN select concept for colorand enhance its activation
IF concept is selectedTHEN select lemma
COLOR NAMING(X)
TASKTASK
Roelofs (Psych. Rev., 2003)
COLOR(X)
RED(X)
GREEN(X)
BLACK(X)
WHITE(X)
GRAY(X)
BROWN(X)
YELLOW(X)
ORANGE(X)
PURPLE(X)
PINK(X)
BLUE(X)
OLIVE(X)
VIOLET(X)
couleur
rouge vert
noir blancgris
brun jaune
orangepourpre
rose
bleu
olive
violet
<rouge>
/r/ /u/ /G/
[ruG]
adj.
COLOR INPUT
WORD INPUT
ROUGE
CONCEPTUALSTRATUM
WORD-FORMSTRATUM
SYNTACTICSTRATUM
Colin MacLeod’s (1991) top-16
incongruency, congruency, reverse Stroop, response set, semantic gradient, time course, stimulus, spatial, multiple task, manual, bilingual, training, age,
and pathological effects.
3 free parameters taking 2 values each.r(model-data) = .98
WEAVER++ explains 96 percent of the variance.
250 data points from 16 classic studies
OBSERVED
RT E
FFEC
T (
mse
c)
SOA (msec)
-100
-50
0
50
100
150
-100
-50
0
50
100
150
-400
-200
-300
-100
400
1000
300
200
-400
-200
-300
-100
400
1000
300
200
-100
-50
0
50
100
150
-100
-50
0
50
100
150
-400
-200
-300
-100
400
1000
300
200
-400
-200
-300
-100
400
1000
300
200
Color naming
Word reading
WEAVER++
0
2.0
4.0
6.0
8.0
10.0
0 50 100 150 200 250
ACTI
VATI
ON
TIME (msec)
targetSOA = -100 msec
distractor
SOA = 0 msecdistractor
(b)
(a)
colorperception
word-formperception
word-formencoding
lemmaretrieval
conceptualidentification
articulation
COLOR
WORD
OVERT SPEECH
10
(b)
(a)
colorperception
word-formperception
word-formencoding
lemmaretrieval
conceptualidentification
articulation
COLOR
WORD
OVERT SPEECH
(b)
(a)
colorperception
word-formperception
word-formencoding
lemmaretrieval
conceptualidentification
articulation
COLOR
WORD
OVERT SPEECH
0 100 200
SOA (msec)
-80
-60
-40
-20
0
20
40
LATE
NCY
EFF
ECT
(mse
c)
color namingword reading (after task switch)
mediated (e.g., VESTE)direct (e.g., ROUTE)
VESTE
ROUTE
mediated
direct
Roelofs (Psych. Rev., 2003)
(b)
(a)
object-formperception
word-formperception
word-formencoding
lemmaretrieval
conceptualidentification
articulation
OBJECT
WORD
OVERT SPEECH
-400 -300 -200 -100 0 100 200 300 400
SOA (msec)
CHIEN
OBSERVEDWEAVER++
LATE
NCY
EFFE
CT
(mse
c)
SOA (msec)
-100
-50
0
50
100
150
-100
-50
0
50
100
150
-400
-200
-300
-100
400
1000
300
200
-400
-200
-300
-100
400
1000
300
200
-100
-50
0
50
100
150
-100
-50
0
50
100
150
-400
-200
-300
-100 400
1000
300
200
-400
-200
-300
-100
400
1000
300
200
Word reading
Picture naming
(Glaser & Düngelhoff, JEP:HPP, 1984)
11
LATE
NCY
EFF
ECT
(m
sec)
-100
-50
0
50
100
word categorizing
semantic unrelated
DISTRACTOR
-25
0
25
50
word reading with determiner
OBSERVEDWEAVER++
-25
0
25
word reading
TASK
Picture distractor
CHIEN
Say “chien"
CHIEN
Say “le chien"
Say “animal"
CHIEN
Roelofs (Psych. Rev., 2003)
Stroop asymmetry and its time course.
2
Outline of the remainder
Control processes.3
Structure of lexical memory.
1
Stroop asymmetry and its time course.
2
Outline of the remainder
Control processes.3
Structure of lexical memory.
1 Prefrontal-parietal
control network
Posner & Raichle (Images of Mind, 1994)
anterior cingulate cortex ACC
12
0.0
0.1
0.2
0.3
0.4
SCAN (2.5 sec)
1 2 3 4 5
incongruentcongruent
BOLD RESPONSE (% CHANGE)
MacDonald et al. (Science, 2000)
Two accounts of ACC function
Conflict detection (GRAIN)1
Top-down regulation (WEAVER++)2
Cohen et al. (Psych. Rev., 1990), Botvinick et al. (Psych. Rev., 2001)
WORD INPUTROUGE VERT
word reading
colornaming
VOCAL RESPONSE“rouge" “vert"
conflict detection (ACC)
COLOR INPUT
TASK (DLPF)
GRAIN
Botvinick et al. (Psych. Rev., 2001)
GRAIN
0.0
1.5
0 500
congruentincongruent
250
1.0
0.5
0.0
1.5
0 500
congruentincongruent
250
1.0
0.5
ENERGY
TIME (msec)
art
wfp
executingcontrol
wordplanning
frontallobe
COLORWORD
cpci/lr
wfe
inputcontrol
goalcontrol
centralsulcus
lateralfissure
anteriorcingulate
frontallobe
temporallobe
occipitallobe
VERBALRESPONSE
Roelofs (Psych. Rev., 2003)
WEAVER++
0.0
0.1
0.2
0.3
0.4fMRI observed
(ACC)WEAVER++
1 2 3 4 5
0.0
0.1
0.2
0.3
0.4
1 2 3 4 5
Roelofs & Hagoort(Cogn. Brain Res., 2002)
MacDonald et al. (Science, 2000)
BOLD RESPONSE (% CHANGE)
SCAN (2.5 sec)
incongruentcongruentincongruentcongruent
IF task is color namingTHEN select concept for colorand enhance its activation
13
VERTROUGEXXXX
incongruent
congruent
neutral
Stroop subtractions in neuroimaging
?
GAUCHE
RIGHT(X,(Y))
LEFT(X,(Y))
adj.WORD
CLASS
WORD
CLASSgauche droit
<droit>
/o/ /r//d/
<gauche>
/g/
[drwA][gosh]
1 1 2
on nu
FORMFORM
SENSESENSE
concepts
lemmas
forms
/sh/
2 3
co
IS-NOT
GAUCHE
vocalresponse
middlefinger
indexfinger
manual response
/w/ /A/
3
on1 on2 on3 nu
4
WEAVER++
manual
0
20
40
60
Arrow Word
RT
effe
ct (
mse
c)
Target
0
20
40
60
Arrow Word
RT
effe
ct (
mse
c)
Target
vocal
0
20
40
60
Arrow Word
RT
effe
ct (
mse
c)Target
0
20
40
60
Arrow Word
RT
effe
ct (
mse
c)
Target
observed WEAVER++
Turken & Swick (Nat. Neurosc., 1999)
Roelofs, Van Turennout, & Coles (submitted)
Manually
Respond to the arrow LEFT
LEFT
incongruent
LEFT
LEFT
RIGHT
RIGHT
congruent
RIGHT
RIGHT
neutral
XXXX
RIGHTRespond to the word
Respond to the arrow
incongruent
GAUCHE
incongruent
DROIT
DROIT
congruentcongruent neutral
XXXX
DROIT
XXXX
Respond to the wordGAUCHE
Incongruent vs. congruent in word task
14
Mean RT(ms)
1.2 2.8
arrow task word task475
525
575
625
0.9 4.6 1.2 6.3%E
-1.5-1
-0.50
0.51
1.5
Meanregressioncoefficient
arrow task word task
WEAVER++
475
525
575
625
arrow task word task
-1.5-1
-0.50
0.51
1.5
arrow task word task
congruent
incongruentneutral
Botvinick et al. (Psych. Rev., 2001)
GRAIN
incongruent
congruentneutral
selection
selection
ENER
GY
CYCLE
Summary
Stroop asymmetry and its time course:Strength vs. architecture
2
Control processes:ACC: conflict detection vs. regulation
3
Structure of lexical memory:Concepts, lemmas, forms
1
http://www.nici.ru.nl/~ardiroel/weaver++.htm
http://www.nici.ru.nl/~ardiroel/stroop.htm
Merci beaucoup!