Real-time interactions between attention and behavior in multimedia learning environments
Post on 11-Feb-2016
Embed Size (px)
DESCRIPTIONReal-time interactions between attention and behavior in multimedia learning environments. Susan Letourneau Postdoctoral Fellow, CREATE Lab NYU & CUNY Graduate Center. LearnLab Summer Workshop August 4, 2012. How can multimedia technology be made more effective for learning?. - PowerPoint PPT Presentation
Real-time interactions between attention and behavior in multimedia learning environmentsSusan LetourneauPostdoctoral Fellow, CREATE LabNYU & CUNY Graduate Center
LearnLab Summer WorkshopAugust 4, 2012
How can multimedia technology be made more effective for learning?CREATE Lab research includes:Systematic investigation of design principles that may support learningIterative development of educational games and simulations
Interactivity and EngagementStudents interact and engage with multimedia materials in different ways:By acting and doingBy looking and thinkingBy reacting and feelingHow can we capture attention, cognition, emotion, in addition to behavioral activity?Multiple measures:Activity logsEye-trackingPhysiological responses
Eye-tracking measures of visual attentionBenefitsRemote, noninvasiveObjectiveContinuous recordingMeasures include:Location of gazeDuration of fixationsFixation Sequences
Integrating Activity Logs & Eye-trackingSynchronized recordings of behavior and attention using common timestampData analysis approaches:Behaviors as individual eventsBehaviors as markers or dividers to parse eye-tracking dataSequences of gaze and behavior over time
Study 1: Gaze and Activity in a Chemistry Simulation26 high school studentsMeasures: Eye-tracking, activity logsPre/post-tests of chemistry knowledge
Gaze transitions between multiple representations correlated with learning outcomes Controllers-Axes: =.54, t(20)=2.88, p=.01, Container-Graph: =.46, t(20)=2.38, p=.02Students often looked to these key areas immediately after changing a variable in the simulation
Study 2: Using visual scaffolds to guide attention
28 high school students, using simulation with or without scaffoldsExamined gaze patterns following interactions with the controllers
- Scanpaths follow the path of the scaffolds. Students with more transitions show higher learning outcomes [Controllers-Axes, r=.56, p
Study 3: Attention during experimentation.32 high school students planned and executed experiments in a chemistry simulationActivity logs used to divide eye-tracking data into three types of activities: Adjusting variables (planning experiment) Watching ongoing experiment Experiment completed
Students directed attention to different parts of the simulation during different activities.Attention to the graph area specifically while students planned an experiment was correlated with post-test scores [=0.49, t(22)=2.51, p=.02].
PlanningWatchingEnd of Experiment
Ongoing work: Physiological measures of cognitive and affective responsesCognition: Eye-trackingEEGEmotion: Skin conductanceHeart rate
Triangulating multiple measuresPhysiological measurements can be synchronized with eye-tracking and behavioral recordings.
Measurements can be time-locked with any channel of information.
Current Research DirectionsControlled comparisons of responses to tasksBehaviorally EngagingCognitively EngagingAffectively Engaging
AcknowledgmentsCREATE LabPIs: Jan Plass, Bruce Homer, Catherine MilneLizzie Hayward, Ruth Schwartz
Institute of Education Sciences, IPORT Fellowship
Measurement of learning outcomes and overt behavior focuses on doing.
*Where students look for the effects of their actions, and how frequently they connect multiple representations can influence what they learn from the simulation.
*Cognitive processes: Mental effort, attention, information processingAffective responses: Curiosity, frustration, boredom, insight (eureka moments)