real-time interactions between attention and behavior in multimedia learning environments

Download Real-time interactions between attention and behavior in multimedia learning environments

Post on 11-Feb-2016




0 download

Embed Size (px)


Real-time interactions between attention and behavior in multimedia learning environments. Susan Letourneau Postdoctoral Fellow, CREATE Lab NYU & CUNY Graduate Center. LearnLab Summer Workshop August 4, 2012. How can multimedia technology be made more effective for learning?. - PowerPoint PPT Presentation


  • Real-time interactions between attention and behavior in multimedia learning environmentsSusan LetourneauPostdoctoral Fellow, CREATE LabNYU & CUNY Graduate Center

    LearnLab Summer WorkshopAugust 4, 2012

  • How can multimedia technology be made more effective for learning?CREATE Lab research includes:Systematic investigation of design principles that may support learningIterative development of educational games and simulations

  • Interactivity and EngagementStudents interact and engage with multimedia materials in different ways:By acting and doingBy looking and thinkingBy reacting and feelingHow can we capture attention, cognition, emotion, in addition to behavioral activity?Multiple measures:Activity logsEye-trackingPhysiological responses

  • Eye-tracking measures of visual attentionBenefitsRemote, noninvasiveObjectiveContinuous recordingMeasures include:Location of gazeDuration of fixationsFixation Sequences

  • Integrating Activity Logs & Eye-trackingSynchronized recordings of behavior and attention using common timestampData analysis approaches:Behaviors as individual eventsBehaviors as markers or dividers to parse eye-tracking dataSequences of gaze and behavior over time

  • Study 1: Gaze and Activity in a Chemistry Simulation26 high school studentsMeasures: Eye-tracking, activity logsPre/post-tests of chemistry knowledge

  • Gaze transitions between multiple representations correlated with learning outcomes Controllers-Axes: =.54, t(20)=2.88, p=.01, Container-Graph: =.46, t(20)=2.38, p=.02Students often looked to these key areas immediately after changing a variable in the simulation

  • Study 2: Using visual scaffolds to guide attention

    28 high school students, using simulation with or without scaffoldsExamined gaze patterns following interactions with the controllers

  • Scanpaths follow the path of the scaffolds. Students with more transitions show higher learning outcomes [Controllers-Axes, r=.56, p
  • Study 3: Attention during experimentation.32 high school students planned and executed experiments in a chemistry simulationActivity logs used to divide eye-tracking data into three types of activities: Adjusting variables (planning experiment) Watching ongoing experiment Experiment completed

  • Students directed attention to different parts of the simulation during different activities.Attention to the graph area specifically while students planned an experiment was correlated with post-test scores [=0.49, t(22)=2.51, p=.02].

  • PlanningWatchingEnd of Experiment

  • Ongoing work: Physiological measures of cognitive and affective responsesCognition: Eye-trackingEEGEmotion: Skin conductanceHeart rate

  • Triangulating multiple measuresPhysiological measurements can be synchronized with eye-tracking and behavioral recordings.

    Measurements can be time-locked with any channel of information.

  • Current Research DirectionsControlled comparisons of responses to tasksBehaviorally EngagingCognitively EngagingAffectively Engaging

  • AcknowledgmentsCREATE LabPIs: Jan Plass, Bruce Homer, Catherine MilneLizzie Hayward, Ruth Schwartz

    Institute of Education Sciences, IPORT Fellowship

    Measurement of learning outcomes and overt behavior focuses on doing.

    *Where students look for the effects of their actions, and how frequently they connect multiple representations can influence what they learn from the simulation.

    *Cognitive processes: Mental effort, attention, information processingAffective responses: Curiosity, frustration, boredom, insight (eureka moments)



View more >