an eye tracking study on expert/novice differences during climate graph … › files ›...

1
Introduction Methods & Tools Eye-Tracking : What is it? Why use it? Non-invasive collects data without interfering with participants’ normal viewing patterns Collects data on attention (where and when one looks and for how long) From eye-tracking data, we can: Create heat maps – show where participant looks and for how long Create gaze plots – show where and when participant looks Statistically analyze when, where and for how long a participant focused attention on any area of interest (AOI) defined by the researcher Tools and Materials Used: Eye-tracker – Tobii TX300 Graph pre-assessment - to determine prior knowledge about graph reading, interpretation and construction Conclusions 1. How do expert and novice eye movements differ? Experts = focus more attention on task-relevant information (i.e. data trends, axes, legend, etc.) Novices = focus more attention on task-irrelevant information (title, question) High-performing novices (top 1/3 of pretest scores) behaved more expert-like 2. How does the amount of time experts and novices spend viewing graphs differ? Experts = spend more time answering main idea and bringing in prior knowledge/extrapolating (Q4) 3. To what extent do expert and novice interpretations differ? Expert responses = longer, more in-depth explanations, include prior knowledge Novices = describe general trends References Environmental Protection Agency (EPA). 2016. Climate change science. Available at www3.epa.gov/climatechange/science/ (accessed 19 February 2016). Freedman, E.G. and Shah, P. 2002. Toward a model of knowledge-based graph comprehension. Diagrammatic Representation and Inference, 2317:18-30. Spence, I. and Lewandowsky, S. 1990. Graphical perception. In J. Fox & J. S. Long, eds., Modern methods of data analysis, p. 13- 57. Wang, Z.H., Wei, S., Ding, W., Chen, X., Wang, X. and Hu, K. 2012. Students’ cognitive reasoning of graphs: Characteristics and progression. International Journal of Science Education, 34:2015-2041. Geocognition and Geoscience Education Research Group An Eye Tracking S tudy on Expert/Novice Differences During Climate Graph Reading Tasks: Implications for Climate Communication The communication of climate change information is often a difficult task due to the interdisciplinary nature of the topic in addition to the challenges of communicating these topics with their intended audiences. In order to present this information effectively, it is important to understand how novices (e.g., the general public) navigate this data differently than experts. In this study, students viewed graphs displaying climate change information and their gaze patterns were collected. Gaze patterns of scientific experts (geoscience graduate students) were used as a comparison to the novices (undergraduate students). Research Questions: 1. How do expert and novice eye movements differ? 2. How does the amount of time experts and novices spend viewing graphs differ? 3. To what extent do expert and novice interpretations differ? Study Design Study Design Participants take pre-assessment Eye-tracking study – view graphs and answer questions View four different graphs with six questions per graph Results GGG SLGT TA48 WWTA Questions Q1: What is the main idea of this graph? Q2: What trends do you see in this graph? Q’s 3a & b: Visual lookup of answer (e.g. What was the approximate Sea Level change in 1950?) Q’s 4a & b: Extrapolation or application of prior knowledge (e.g. What might global temperature be in 2050?) Figure 3 (above). Summary statistics of total fixation durations and differences in fixation durations displayed based on the type of question being asked or the type of graph being displayed. ** indicates p<0.05 * indicates p<0.1 0 5 10 15 20 25 30 35 40 GGG SLGT TA48 WWTA GGG SLGT TA48 WWTA Expert/Novice Pretest Scores Total Fixation Duration (sec) -5 0 5 10 15 20 GGG SLGT TA48 WWTA GGG SLGT TA48 WWTA Expert/Novice Pretest Scores Difference in Fixation Duration (Data-Title/Question) (sec) ** ** ** * * * Experts Novices Figure 2 (left). Fixation duration heat maps showing differences between data fixation durations and question/title fixation durations for experts (top) and novices (bottom). Red indicates locations where more attention is allocated. Relative duration calculation, radius = 205px, opacity = 75%. Figure 1. Graphs used for this study. All graphs contain climate content and were modified from their original EPA (EPA, 2016) publication format for consistency in color, line thickness and overall readability. Rachel Atkins ([email protected]), Karen McNeal ([email protected]) North Carolina State University, Dept. of Marine, Earth and Atmospheric Sciences Figure 4 (below). Construction Integration (CI) model for graph construction and interpretation (Freedman and Shah, 2002) modified for climate content and measured metrics. Our Recommendations for instructors: 1. Provide opportunities for students to increase their knowledge of graph reading and climate content (Freedman and Shah, 2002). 2. Offer scaffolding and training aimed to help students direct attention to data elements (i.e. axes, legends, data trends, etc.) (Wang et al., 2012). 3. Implement changes to graphs that simplify empirical data (Spence and Lewandowsky, 1990) and/or direct viewer attention to data elements either through visual emphasis (i.e. arrows, highlighting, etc.) or adjoining text.

Upload: others

Post on 05-Jul-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: An Eye Tracking Study on Expert/Novice Differences During Climate Graph … › files › earth... · 2018-03-14 · • Create gaze plots – show where and when participantlooks

TemplateID:neonboxesSize:36x48

Introduction

Methods&ToolsEye-Tracking :What is it? Why use it?• Non-invasive – collects data without

interfering with participants’ normal viewingpatterns

• Collects data on attention (where and whenone looks and for how long)

From eye-tracking data, we can:• Create heat maps – show where participant

looks and for how long• Create gaze plots – show where and when

participant looks• Statistically analyze when, where and for

how long a participant focused attention onany area of interest (AOI) defined by theresearcher

Tools and Materials Used:• Eye-tracker – Tobii TX300• Graph pre-assessment - to determine prior

knowledge about graph reading,interpretation and construction

Conclusions1. How do expert and novice eye movements differ?• Experts = focus more attention on task-relevant information (i.e. data trends, axes,

legend, etc.)• Novices = focus more attention on task-irrelevant information (title, question)• High-performing novices (top 1/3 of pretest scores) behaved more expert-like2. How does the amount of time experts and novices spend viewing graphs differ?• Experts = spend more time answering main idea and bringing in prior

knowledge/extrapolating (Q4)3. To what extent do expert and novice interpretations differ?• Expert responses = longer, more in-depth explanations, include prior knowledge• Novices = describe general trends

ReferencesEnvironmental Protection Agency (EPA). 2016.Climate change science. Available atwww3.epa.gov/climatechange/science/(accessed 19 February 2016).

Freedman, E.G. and Shah, P. 2002. Toward amodel of knowledge-based graphcomprehension. Diagrammatic Representationand Inference, 2317:18-30.

Spence, I. and Lewandowsky, S. 1990.Graphical perception. In J. Fox & J. S. Long,eds., Modern methods of data analysis, p. 13-57.

Wang, Z.H., Wei, S., Ding, W., Chen, X., Wang,X. and Hu, K. 2012. Students’ cognitivereasoning of graphs: Characteristics andprogression. International Journal of ScienceEducation, 34:2015-2041.

Geocognition andGeoscienceEducation

ResearchGroup

AnEyeTrackingStudyonExpert/NoviceDifferencesDuringClimateGraphReadingTasks:ImplicationsforClimateCommunication

The communication of climate changeinformation is often a difficult task due to theinterdisciplinary nature of the topic in additionto the challenges of communicating thesetopics with their intended audiences. In orderto present this information effectively, it isimportant to understand how novices (e.g., thegeneral public) navigate this data differentlythan experts. In this study, students viewedgraphs displaying climate change informationand their gaze patterns were collected. Gazepatterns of scientific experts (geosciencegraduate students) were used as a comparisonto the novices (undergraduate students).

Research Questions:1. How do expert and novice eye movements

differ?2. How does the amount of time experts and

novices spend viewing graphs differ?3. To what extent do expert and novice

interpretations differ?

StudyDesignStudyDesign• Participantstakepre-assessment• Eye-trackingstudy– viewgraphsand

answerquestions• Viewfourdifferentgraphswithsix

questionspergraph

Results

GGG SLGT TA48 WWTA

QuestionsQ1:What is themain idea of this graph?Q2:What trends do you see in this graph?Q’s 3a & b: Visual lookup of answer (e.g. What was the approximate Sea Level change in 1950?)Q’s 4a & b: Extrapolation or application of prior knowledge (e.g. What might global temperature be in 2050?)

Figure 3 (above). Summary statistics of totalfixation durations and differences in fixationdurations displayed based on the type ofquestion being asked or the type of graph beingdisplayed.** indicates p<0.05 * indicates p<0.1

0

5

10

15

20

25

30

35

40

GGG SLGT TA48 WWTA GGG SLGT TA48 WWTA

Expert/Novice PretestScores

TotalFixationDu

ratio

n(sec)

-5

0

5

10

15

20

GGG SLGT TA48 WWTA GGG SLGT TA48 WWTA

Expert/Novice PretestScores

Diffe

renceinFixatio

nDu

ratio

n(Data-Title

/Que

stion)(sec)

****

**

** *

Expe

rts

Novices

Figure 2 (left). Fixation duration heat mapsshowing differences between data fixationdurations and question/title fixationdurations for experts (top) and novices(bottom). Red indicates locations wheremore attention is allocated. Relativeduration calculation, radius = 205px,opacity = 75%.

Figure1.Graphsusedforthisstudy.AllgraphscontainclimatecontentandweremodifiedfromtheiroriginalEPA(EPA,2016)publicationformatforconsistencyincolor,linethicknessandoverallreadability.

RachelAtkins([email protected]),KarenMcNeal([email protected])NorthCarolinaStateUniversity,Dept.ofMarine,EarthandAtmosphericSciences

Figure4(below).ConstructionIntegration(CI)modelforgraphconstructionandinterpretation(FreedmanandShah,2002)modifiedforclimatecontentandmeasuredmetrics.

OurRecommendationsforinstructors:1. Provide opportunities for students to increase their

knowledge of graph reading and climate content (Freedmanand Shah, 2002).

2. Offer scaffolding and training aimed to help students directattention to data elements (i.e. axes, legends, data trends,etc.) (Wang et al., 2012).

3. Implement changes to graphs that simplify empirical data(Spence and Lewandowsky, 1990) and/or direct viewerattention to data elements either through visual emphasis(i.e. arrows, highlighting, etc.) or adjoining text.