1. introduction 5. exp 1. english: subjecthood vs linear ... · rise in parallel in object...

1
Subjecthood & Linguistic Encoding: Real-time Production of wh-Questions in English and Mandarin Monica Do {[email protected]} Elsi Kaiser {[email protected]} Pengchen Zhao {[email protected]} Department of Linguistics, University of Southern California Language production is multi-stage & incremental [1] The linearization problem: How does linguistic encoding start going from unordered concepts to sequentially produced utterances? Functional Processing (Label-then-Linearize): Lexical representations directly assigned to subject role; subject placed sentence-initially [3] Positional Processing (Linearize-then-Label): Lexical representations assigned to first linear slot in sentence; incidentally, becomes subject [2] How do we tease these apart if subjects are the first elements in sentences? Active vs Passives: Subjecthood, not semantic Agenthood, drives encoding, but subject still linearly-initial in actives and passives [4] Free word order (Russian, Finnish) [5] or Verb-initial (Tzeltal, Tagalog) [6] But, results complicated by discourse and/or morphological factors 1. Introduction 2. Current Study Research Question: Is linguistic encoding driven by functional or positional processing? Investigate: Emergence of syntactic structure effects (functional processing) relative to linear word order effects (positional processing) during linguistic encoding. We use object wh-questions: The subject in these structures is not linearly initial. March 15-17, 201831st Annual CUNY Conference on Human Sentence ProcessingUC Davis References REFERENCES: [1] Levelt, W. (1989). Speaking: From intention to articulation. Bock, K. and Levelt, W. (1994) “Grammatical Encoding.” Handbook of Psycholinguistics. [2] Gleitman,L. et al. (2007). On the give- and-take between event apprehension and utterance formulation. JML.; Brown-Schmidt, S. and Konopka, A. (2008) Little houses and casas pequenas: Message formulation and syntactic form in unscripted speech with speakers of English and Spanish. Cognition. [3-4] Griffin, Z.M. and Bock,K. (2000). What the eyes say about speaking. Psychol. Sci. [5] Myachykov, A. et al. (2011) Visual attention and structural choice in sentence production across languages. Lang Linguist Compass. [6] Norcliffe, E., et al. (2015). Word order affects the timecourse of sentence formulation in Tzeltal. Lang Cogn Neurosci.[7] Prat-Sala, M. and Branigan, H.P. (2000) Discourse Constraints on Syntactic Processing in Language Production: A Cross-linguistic Study in English and Spanish. JML; Ganuschak, L. et al. (2014). What the eyes say about planning of focused referents during sentence formulation: a cross-linguistic investigation. Front. Psychol. ACKNOWLEDGEMENTS: [1] Student Opportunities for Academic Research (USC), Patrice Zhao [2] Russell Endowed Fellowship (USC), Monica Do 3. Experiment Design Participants first saw sentence type cue, then saw image; produced the cued sentence type Statement (S) Object Wh-Question (Q) The nurses tickled the chefs. Which chefs did the nurses tickle? Verbs indicated by instruments (e.g. feather), instrument location indicated subject character To encourage object wh-questions, examples/practice included only object wh- questions; participants told to ask about ‘who the action is happening to’ Visual World Eye-tracking Paradigm: Measured proportion of fixations to subject, object and verb, & Sub-Obj Difference Scores; 33 target items Key time window for Linguistic Encoding: 400-800 ms 4. Hypotheses & Predictions Before linguistic encoding (0-400 ms), speakers (n=30) look to verb first to determine Subj/Obj During linguistic encoding (400-800 ms), differences between Decl & Object wh-Ques emerge Subj-Obj difference scores in declaratives larger than in object wh-questions (|z|= 2.67) Key Pattern: Speakers look to the subject before object in decl & ques (~400ms), but consider the object more in ques than in decl Decl: Rapid rise in looks to subject only ~400ms Ques: Rise in looks to subject & object ~400ms Linguistic encoding may be primarily – but not exclusively – driven by functional processes Speakers juggle functional and positional demands at the same time: Looks to subject and object rise in parallel in object wh-questions Subjecthood assignment is privileged: Looks to subject emerge first and proportion of looks to subject greater than proportion of looks to object 5. Exp 1. English: Subjecthood vs Linear Word Order 6. Exp 2. Mandarin: Word Order vs Information Focus After window of interest, fixations reflect linear word order, as expected Declaratives Questions 0 200 400 600 800 1000 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Time(ms) from Image Onset Proportion of Fixations Subj Obj Shaded areas indicate 95% confidence interval Region Subj Obj Verb Fig1: Eye-Movements Immediately After Image Onset Shaded areas indicate 95% confidence interval Region Subj Obj Verb Fig3: Eye-Movements Immediately After Image Onset 7. Discussion & Conclusion First look at real-time production of questions Differences between declaratives and questions emerge during Linguistic Encoding No differences emerge before encoding (e.g. Message Formulation) To what extent does propositional content of messages in declaratives vs questions differ? Encoding is multi-factorial: Takes into consideration competing structural and positional demands simultaneously Linguistic encoding can start with syntactic roles even when it conflicts with linear word order Open question: Why don’t we see information focus effects during production planning? Information focus (e.g. from prior discourse context) can affect conceptual accessibility [7] Focus also affects speakers’ choice of structure (e.g. free word order languages, It-clefts, etc.), suggests role for focus in positional processing BUT, Mandarin questions have strict SVO order: Effect of information focus may be restricted Research Questions: Wh-words are informationally focused elements. To what extent did information focus drive looks to the object in Exp 1? Exp 2 conducted in Mandarin Chinese (Subject-Verb-Object) Mandarin wh-questions and declaratives have the same linear word order Declarative: Object Wh-Question: 护士 枪毙了 护士 枪毙了 哪个厨? The nurses shot the chefs. The nurses shot which chefs? Eye-movements differences cannot be due to surface word order (1) Message Formulation: Speakers pull together unordered, pre-linguistic concepts they intend to communicate (2) Linguistic Encoding: Pre-linguistic concepts assigned to lexical representations and grammatically encoded (3) Phonological Encoding & (4) Articulation: Sound units assembled & Speech Begins Positional Processing: Lexical representations assigned position/order in the sentence Functional Processing: Lexical representations assigned syntactic role (e.g. subject, object) Declaratives The nurses tickled the chefs. Object wh-questions Which chefs did the nurses tickle? Functionally-driven: Encoding driven by subjecthood Subject Subject Positionally-driven: Encoding driven by linear word order Subject Object Declaratives Questions 1600140012001000 800 600 400 200 0 200 400 600 800 1000 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Time(ms) Aligned To Speech Onset Proportion of Fixations Speech Onset Declaratives Questions 1600140012001000 800 600 400 200 0 200 400 600 800 1000 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Time(ms) Aligned to Speech Onset Proportion of Looks Speech Onset Fig2: Eye-Movements After Encoding Fig4: Eye-Movements After Encoding After window of interest, fixations reflect linear word order, as expected Before linguistic encoding (0-400 ms), speakers (n=35) look to verb first to determine Subj/Obj During linguistic encoding (400-800 ms), we do not detect eye-movement differences between Decl & Object wh-Ques (|z|s < 1.4) Key Pattern: Speakers fixate subject in both decl & ques; do not consider object in either Decl and ques do not differ 200-1000ms after image onset Subj-Obj difference scores do not differ Exp 1 results not confounded by focus: No evidence information focus affects eye-movements Subj Obj

Upload: others

Post on 02-Aug-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 1. Introduction 5. Exp 1. English: Subjecthood vs Linear ... · rise in parallel in object wh-questions • Subjecthoodassignment is privileged: Looks to subject emerge first and

Subjecthood &LinguisticEncoding:Real-timeProductionofwh-QuestionsinEnglishandMandarin

MonicaDo {[email protected]}😺 Elsi Kaiser{[email protected]}😺 Pengchen Zhao{[email protected]}DepartmentofLinguistics,UniversityofSouthernCalifornia

• Languageproductionismulti-stage&incremental[1]

• Thelinearizationproblem:Howdoeslinguisticencodingstartgoingfromunorderedconceptstosequentiallyproducedutterances?• FunctionalProcessing(Label-then-Linearize):Lexicalrepresentationsdirectlyassignedtosubjectrole;subjectplacedsentence-initially[3]

• PositionalProcessing(Linearize-then-Label): Lexicalrepresentationsassignedtofirstlinearslotinsentence;incidentally,becomessubject[2]

• Howdoweteasetheseapartifsubjectsarethefirstelementsinsentences?• ActivevsPassives:Subjecthood,notsemanticAgenthood,drivesencoding,butsubjectstilllinearly-initialinactivesandpassives[4]

• Freewordorder(Russian,Finnish)[5]orVerb-initial(Tzeltal,Tagalog)[6]• But,resultscomplicatedbydiscourseand/ormorphologicalfactors

1.Introduction

2.CurrentStudyResearchQuestion:Islinguisticencodingdrivenbyfunctional

orpositionalprocessing?Investigate:Emergenceofsyntacticstructureeffects

(functionalprocessing)relativetolinearwordordereffects(positionalprocessing)duringlinguisticencoding.

Weuseobjectwh-questions:Thesubjectinthesestructuresisnotlinearlyinitial.

March15-17,2018・31stAnnualCUNYConferenceonHumanSentence Processing・UC Davis

ReferencesREFERENCES:[1] Levelt,W.(1989).Speaking:Fromintentiontoarticulation. Bock,K.andLevelt,W.(1994)“GrammaticalEncoding.”HandbookofPsycholinguistics.[2] Gleitman,L.etal.(2007).Onthegive- and-takebetweeneventapprehensionandutteranceformulation.JML.;Brown-Schmidt,S.andKonopka,A.(2008)Littlehousesandcasaspequenas:MessageformulationandsyntacticforminunscriptedspeechwithspeakersofEnglishandSpanish.Cognition.[3-4] Griffin,Z.M.andBock,K.(2000).Whattheeyessayaboutspeaking.Psychol.Sci. [5]Myachykov,A.etal.(2011)Visualattentionandstructuralchoiceinsentenceproductionacrosslanguages.LangLinguistCompass. [6] Norcliffe,E.,etal.(2015).Wordorderaffectsthetimecourse ofsentenceformulationinTzeltal.LangCogn Neurosci.[7] Prat-Sala,M.andBranigan,H.P.(2000)DiscourseConstraintsonSyntacticProcessinginLanguageProduction:ACross-linguisticStudyinEnglishandSpanish.JML;Ganuschak,L.etal.(2014).Whattheeyessayaboutplanningoffocusedreferentsduringsentenceformulation:across-linguisticinvestigation.Front.Psychol.

ACKNOWLEDGEMENTS:[1] StudentOpportunitiesforAcademicResearch(USC),PatriceZhao[2] RussellEndowedFellowship(USC),MonicaDo

3.Experiment Design• Participantsfirst sawsentencetypecue,then sawimage;producedthecuedsentencetype

Statement(S) ObjectWh-Question(Q)Thenursestickledthechefs.Whichchefsdidthenursestickle?

• Verbsindicatedbyinstruments(e.g.feather),instrumentlocationindicatedsubjectcharacter

• Toencourageobjectwh-questions,examples/practiceincludedonlyobjectwh-questions;participantstoldtoaskabout‘whotheactionishappeningto’

• VisualWorldEye-trackingParadigm:Measuredproportionoffixationstosubject,object and verb,&Sub-Obj DifferenceScores;33targetitems• KeytimewindowforLinguisticEncoding:400-800ms

4.Hypotheses&Predictions

• Beforelinguisticencoding(0-400ms),speakers(n=30)looktoverb firsttodetermineSubj/Obj• Duringlinguisticencoding(400-800ms),differencesbetweenDecl &Objectwh-Quesemerge• Subj-Obj differencescoresindeclarativeslarger thaninobjectwh-questions(|z|=2.67)

• KeyPattern:Speakerslooktothesubject beforeobject indecl &ques(~400ms),butconsidertheobjectmoreinquesthanindecl• Decl:Rapidriseinlookstosubjectonly~400ms• Ques: Riseinlookstosubject&object~400ms

• Linguisticencodingmaybeprimarily– butnotexclusively– drivenbyfunctionalprocesses• Speakersjugglefunctionalandpositionaldemandsatthesametime: Lookstosubjectandobjectriseinparallelinobjectwh-questions

• Subjecthood assignmentisprivileged:Lookstosubjectemergefirstandproportionoflookstosubjectgreaterthanproportionoflookstoobject

5.Exp 1.English:Subjecthood vsLinearWordOrder

6.Exp 2.Mandarin:WordOrdervsInformationFocus

• After windowofinterest, fixationsreflectlinearwordorder,asexpected

Decla

rativ

esQ

uest

ions

0 200 400 600 800 1000

0.00.10.20.30.40.50.60.70.8

0.00.10.20.30.40.50.60.70.8

Time(ms) from Image Onset

Prop

ortio

n of

Fixa

tions

Subj

Obj

Shaded areas indicate 95% confidence interval

Decla

rativ

esQ

uest

ions

0 200 400 600 800 1000

0.00.10.20.30.40.50.60.70.8

0.00.10.20.30.40.50.60.70.8

Time(ms) from Image Onset

Prop

ortio

n of

Fixa

tions

Region Subj Obj VerbFig1:Eye-MovementsImmediatelyAfter ImageOnset

Shaded areas indicate 95% confidence interval

Decla

rativ

esQ

uest

ions

0 200 400 600 800 1000

0.00.10.20.30.40.50.60.70.8

0.00.10.20.30.40.50.60.70.8

Time(ms) from Image Onset

Prop

ortio

n of

Fixa

tions

Region Subj Obj VerbFig3:Eye-MovementsImmediatelyAfterImageOnset

7.Discussion&Conclusion• Firstlookatreal-timeproductionofquestions• DifferencesbetweendeclarativesandquestionsemergeduringLinguisticEncoding• Nodifferencesemergebeforeencoding(e.g.MessageFormulation)• Towhatextentdoespropositionalcontentofmessagesindeclarativesvsquestionsdiffer?

• Encodingismulti-factorial:Takesintoconsiderationcompetingstructuralandpositionaldemandssimultaneously• Linguisticencodingcanstartwithsyntacticrolesevenwhenitconflictswithlinearwordorder

• Openquestion:Whydon’tweseeinformationfocuseffectsduringproductionplanning?• Informationfocus(e.g.frompriordiscoursecontext)canaffectconceptualaccessibility[7]• Focusalsoaffectsspeakers’choiceofstructure(e.g.freewordorderlanguages,It-clefts,etc.),suggestsroleforfocusinpositionalprocessing

• BUT,MandarinquestionshavestrictSVOorder:Effectofinformationfocusmayberestricted

• ResearchQuestions:Wh-wordsareinformationallyfocusedelements.TowhatextentdidinformationfocusdrivelookstotheobjectinExp 1?

• Exp 2conductedinMandarinChinese(Subject-Verb-Object)• Mandarinwh-questionsanddeclarativeshavethesamelinearwordorderDeclarative: ObjectWh-Question:护士们 枪毙了 厨师。 护士们 枪毙了 哪个厨师?Thenurses shot thechefs. Thenurses shotwhichchefs?

• Eye-movementsdifferencescannot beduetosurfacewordorder

(1)MessageFormulation:Speakerspulltogetherunordered,pre-linguisticconceptstheyintendtocommunicate

(2)LinguisticEncoding:Pre-linguisticconceptsassignedtolexicalrepresentationsandgrammaticallyencoded

(3)PhonologicalEncoding&(4)Articulation:Soundunitsassembled&SpeechBegins

PositionalProcessing:Lexicalrepresentationsassignedposition/orderinthesentence

FunctionalProcessing:Lexicalrepresentationsassignedsyntacticrole(e.g.subject,object)

DeclarativesThenursestickledthechefs.

Objectwh-questionsWhichchefsdidthenurses tickle?

Functionally-driven:Encodingdrivenby

subjecthoodSubject Subject

Positionally-driven:Encodingdrivenbylinear

word orderSubject Object

Decla

rativ

esQ

uest

ions

−1600−1400−1200−1000 −800 −600 −400 −200 0 200 400 600 800 1000

0.10.20.30.40.50.60.70.8

0.10.20.30.40.50.60.70.8

Time(ms) Aligned To Speech Onset

Prop

ortio

n of

Fixa

tions

SpeechOnset

Decla

rativ

esQ

uest

ions

−1600−1400−1200−1000 −800 −600 −400 −200 0 200 400 600 800 1000

0.00.10.20.30.40.50.60.70.8

0.00.10.20.30.40.50.60.70.8

Time(ms) Aligned to Speech Onset

Prop

ortio

n of

Loo

ksSpeechOnset

Fig2:Eye-MovementsAfter Encoding

Fig4:Eye-MovementsAfter Encoding

• After windowofinterest, fixationsreflectlinearwordorder,asexpected

• Beforelinguisticencoding(0-400ms),speakers(n=35)looktoverb firsttodetermineSubj/Obj• Duringlinguisticencoding(400-800ms),wedonotdetecteye-movementdifferencesbetweenDecl &Objectwh-Ques(|z|s <1.4)

• KeyPattern:Speakersfixatesubjectinbothdecl &ques;donotconsiderobjectineither• Decl andquesdonotdiffer 200-1000msafterimageonset• Subj-Obj differencescoresdonotdiffer

• Exp 1resultsnotconfoundedbyfocus: Noevidenceinformationfocusaffectseye-movements

Subj

Obj