beatleyweb.simmons.edubeatleyweb.simmons.edu/scholar/files/original/120ec... · web...
TRANSCRIPT
Running head: EXAMINING EYE GAZE IN A DMTS TASK 1
Examining the Behavior of Remembering Utilizing Eye Movements in a
Delayed Match to Sample Task
Elisa Hegg
Simmons College
Author Note
Research conducted by Elisa Hegg, Simmons College, as part of the requirement for the
doctoral program in Behavior Analysis.
The co-authors and the members of the dissertation committee for this study were Dr.
Ron Allen, Department of Behavior Analysis, and Dr. Gretchen Dittrich, Department of
Behavior Analysis, Simmons College, and Dr. Dave Palmer, Department of Psychology, Smith
College.
Special thanks to Dr. Teresa Mitchell, and the Eunice Kennedy Shriver Center –
University of Massachusetts Medical School for her assistance with this study.
Correspondence regarding this study can be sent to: Elisa Hegg, Department of Behavior
Analysis, Simmons College, 300 The Fenway, Boston, MA 02115. Email:
EXAMINING EYE GAZE IN A DMTS TASK 2
Table of Contents
Table of Contents………………………………………………………………………………….2
Abstract……………………………………………………………………………………………5
Introduction………………………………………………………………………………………..6
Cognitive Theories of Memory………………………………………………………………...7
The Visuo-spatial Sketchpad and Eye-Movements……………………………………………8
Behavior Analysis and Memory………………………………………………………………16
Empirical Work on Complex Human Behavior……………………………………………....22
Purpose………………………………………………………………………………………..24
Experiment 1 Method..…………………………………………………………………………..25
Subject Selection……………………………………………………………………………...25
Participants……………………………………………………………………………………26
Setting and Apparatus………………………………………………………………………...26
DMTS Task…………………………………………………………………………………...27
DMTS stimulus sets……………………………………………………………………...27
Stimulus set presentation………………………………………………………………...28
Eye tracking software……………………………………………………………………29
Dependent variables and measurement……………………………………………………….30
Accuracy of match……………………………………………………………………….30
Visual gaze definitions…………………………………………………………………..30
Procedures…………………………………………………………………………………….31
Interobserver Agreement……………………………………………………………………..32
EXAMINING EYE GAZE IN A DMTS TASK 3
Experiment 1 Results...…………………………………………………………………………..32
Accuracy……………………………………………………………………………………..32
Latency……………………………………………………………………………………….33
Observing Responses………………………………………………………………………...34
Delay……………………………………………………………………………………..34
Comparison………………………………………………………………………………36
Experiment 1 Discussion………..……………………………………………………………….39
Accuracy……………………………………………………………………………………..39
Latency……………………………………………………………………………………….40
Observing Responses………………………………………………………………………...40
Delay……………………………………………………………………………………..41
Comparison………………………………………………………………………………44
Experiment 2 Introduction………………………...……………………………………………..47
Experiment 2 Methods…...………………………………………………………………………48
Participants, Setting and Materials…………………………………………………………..48
DMTS Task…………………………………………………………………………………..48
Dependent Variable and Definitions…………………………………………………………49
Procedure…………………………………………………………………………………….49
Experiment 2 Results...…………………………………………………………………………..51
Accuracy……………………………………………………………………………………..51
Latency……………………………………………………………………………………….51
Observing Responses………………………………………………………………………...52
Delay……………………………………………………………………………………..52
EXAMINING EYE GAZE IN A DMTS TASK 4
Comparison………………………………………………………………………………54
Experiment 2 Discussion..……………………………………………………………………….55
Accuracy and Latency………………………………………………………………………..55
Observing Responses………………………………………………………………………...56
Delay……………………………………………………………………………………..56
Comparison………………………………………………………………………………57
General Discussion………………………………………………………………………………58
Utility of Eye Tracking in a DMTS Task……………………………………………………59
Utility of DMTS task…………………………………………………………………….59
Utility of eye gaze measures……………………………………………………………..61
Evaluation of Eye Gaze in the DMTS Task………………………………………………….63
Delay……………………………………………………………………………………..63
Comparison………………………………………………………………………………64
Behavioral Explanations……………………………………………………………………..64
Covert seeing…………………………………………………………………………….65
CMO-T…………………………………………………………………………………...65
Behavioral repetition……………………………………………………………………..66
Future Research………………………………………………………………...……………66
Limitations…………………………………………………………………………………...70
Conclusion…………………………………………………………………………………...72
References……………………………………………………………………………………74
Appendix A.………………………………………………………………………………….81
Appendix B…………………………………………………………………………………..91
EXAMINING EYE GAZE IN A DMTS TASK 5
Abstract
Measures of eye gaze fixations, including the number and duration, were examined during the
delay and comparison in a delayed match-to-sample task. In Experiment 1 participants responded
yes or no using a key press to indicate whether the comparison matched any of the one, two, or
four stimuli from the initial sample. Experiment 2 was developed as pilot study. Participants
responded using a key press to indicate which quadrant of the screen the comparison had been
presented in, moving from two- to four-stimulus initial samples. Across both Experiment 1 and
Experiment 2 the increase in number of stimuli resulted in an increase in complexity, indicated
by a decrease in accuracy across all participants. The average frequency, and total average
duration, of eye movements during the delay increased as the number of stimuli increased. Data
on observing responses from the time that the comparison was presented up to the participant’s
response were more variable. Together, these results extend the research on complex human
behavior by utilizing an objective measure to examine an otherwise largely covert behavior.
Keywords: visual memory, complex human behavior, eye tracking, covert behavior
EXAMINING EYE GAZE IN A DMTS TASK 6
Examining the Behavior of Remembering Utilizing Eye Movements in a Delayed Match to
Sample Task
What memory is, and how it works, has been a topic within the field of psychology since
the field’s inception, and prior to that it was a topic for philosophers. Our ability to analyze and
respond to things that are no longer present is so commonplace within the human experience that
this performance is taken for granted; yet our ability to understand how we do this remains open
to theory and research. Complicating our ability to view and understand memory objectively is
how deeply intermixed the languages of the layman and the scientist are in this area. Both
scientists studying memory, and average people discussing their own experience of memory,
invoke metaphors of storage and retrieval, often without any acknowledgement of the
metaphorical nature of their description. An alternative to a metaphorical understanding of
memory may be to consider remembering as a behavior rather than as a structure. This may
allow for the interpretation and understanding of this everyday phenomenon using only the
principles of behavior that have been uncovered through independent and rigorous
experimentation. While the realm of behavior analytic theory has put forward plausible accounts
of how complex behavior, such as memory, might develop through the application of the
principles of behavior (Donahoe & Palmer, 2004; Skinner, 1953; Skinner, 1974) little empirical
work exists to support these accounts. This gap in behavioral research on complex behavior
leaves the field open to the domain of cognitive psychology, and the continued reliance on
metaphor and explanatory variables that cannot be independently studied.
Cognitive Theories of Memory
An initial way of categorizing cognitive theories of memory is as single-store or dual-store
theories. In single-store models memory is viewed as unitary across time, from seconds to
EXAMINING EYE GAZE IN A DMTS TASK 7
nearly a lifetime, and is influenced by the association with items and their context (Howard &
Kahana, 2002). Dual-store models of memory, on the other hand, theorize a functional
distinction between long-term memory and short-term memory (Atkinson & Shiffrin, 1968).
Following processing in short-term or working memory, some information will enter into long-
term memory, where it may decay over time or it may persist indefinitely, and the size of the
storage capacity is perhaps unlimited. Long-term memories may further be categorized as being
implicit, such as the muscle memory necessary to execute a task, or explicit, such as our ability
to remember and describe events from our childhood (Atkinson & Shiffrin, 1968). Short-term
memory, on the other hand, is conceived as being a system that holds and actively works on
information for a brief span of time prior to it either entering into long-term memory store or
actively being emitted as some form of overt action, or both. Short-term memory lasts only a
matter of seconds, and can hold only a limited amount of information (Atkinson & Shiffrin,
1968). This “modal-model” has been thoroughly detailed by Shiffrin and colleagues in the
1960’s, and this conceptualization has influenced much of the memory theory and research that
has followed.
While the concept of short-term memory does not necessitate or imply active manipulation
for the storage or emission of information, working-memory is a theoretical framework that
describes structures and processes that actively engage with information for both short-term
retention, as well as allowing the use of information for reasoning, comprehension, and eventual
emission of overt behavior. Hitch and Baddeley (1976) provided the initial description for their
multi-component model of working memory (see Baddeley, 1986 for a thorough description of
the early work leading to the development of his influential working memory model).
The initial working memory framework hypothesized a central executive system that
monitors the activity of two slave systems, each responsible for working on a different type of
EXAMINING EYE GAZE IN A DMTS TASK 8
information: verbal and visual (Baddeley, 1986). Later, Baddeley added a third slave system: the
episodic buffer (Baddeley, 2000). Information that enters the system phonologically, either
through hearing or reading language, is worked on by the phonological loop, where information
may be covertly re-articulated as a rehearsal mechanism. The visuo-spatial sketchpad is the
slave system responsible for maintaining and manipulating visually presented information, and
may be further broken down into systems responsible for spatial information, and information on
the shape, color, or other dimensions and aspects of viewed objects or scenes (Baddeley, 1986).
The episodic buffer is responsible for integrating information from each slave system, as well as
maintaining chronological order (Baddeley, 2000). Much of the research on working-memory
has used verbal information to evaluate the utility of the concept of the phonological loop with
regards to maintaining information for immediate use. However, the concept of visual memory
has received more research in recent years.
The Visuo-Spatial Sketchpad and Eye-Movements
The hypothesized role of the visuo-spatial sketchpad is to retain and analyze information
about what we see. The metaphor of a sketchpad was utilized by Baddeley (1986) to suggest a
system that would function in much the same way that a pad of paper might be utilized by
someone trying to work out a geometric problem to temporarily hold and work on spatial
information. This initial system has since been functionally partitioned into object (i.e., the
appearance of individual objects and arrays) and spatial components (Della Sala, Gray,
Baddeley, Allamano, & Wilson, 1999; Logie, 1995), similar to observations that the functional
organization of working memory may mirror the what/where organization of the visual system
(Postle, Idzikowski, Della Sala, Logie, & Baddeley, 2006).
In a series of studies completed in the 1970’s, described briefly in his book on working
memory, Baddeley and colleagues examined the connection between visual working memory
EXAMINING EYE GAZE IN A DMTS TASK 9
and the visual motor system (Baddeley, 1986). The initial studies were then gathered together
and reprinted, along with a more recent study, as a reflection of the impact that the initial line of
research has had on subsequent investigation on visual working memory (Postle et al., 2006).
Together, the experiments examined the importance of eye movements themselves on the
formation and maintenance of spatial information in working memory, the role of voluntary eye
movements on the same visual imagery task, and whether eye movements, or their control
processes, are involved both in the creation and processing of an imagined scene, or if the role of
eye movements is restricted to the processing of the imagined information. Using a task that had
been shown by Brooks (1967) to emphasize either spatial or verbal encoding of information in
working memory, the first two experiments evaluated the effect of voluntary and involuntary eye
movements on recall (Postle et al., 2006). The task presented a script of information to the
participant that either described a path of numbers from 1 to 8 through a 4 X 4 matrix, or
nonsense sentences. Successful recall of the spatial material was considered to be reliant on the
participant’s ability to visualize the spatial relationships, whereas the nonsense sentences are
difficult to visualize, and so were considered reliant on verbal working memory exclusively
(Postle et al., 2006). Neither task presented any visual stimuli as an antecedent. In the initial
experiment to test whether eye movements per se were important for recall of the spatial
information, nystagmus (involuntary movements of the eyes, usually from side to side) was
induced by spinning the participants after they were presented with the to-be remembered
information for 45s. Eye tracking equipment was used to measure the extent of the nystagmus as
a measure of the integrity of the independent variable. The results indicated memory
performance was not disrupted for either the visual or verbal information in either spin or no-
spin conditions (Postle et al., 2006).
EXAMINING EYE GAZE IN A DMTS TASK 10
In the second experiment described by Postle, et al. (2006), voluntary control of eye
movements was disrupted. Eye tracking equipment was again used to measure the integrity of
the independent variable, whether the participants controlled their eye movements as instructed.
During the delay interval participant’s voluntary use of eye movements was controlled by
requiring that they either track a moving object across a stationary visual field, track a moving
object across a moving visual field, or fix on a stationary object in front of a moving visual field.
A free eye-movement control condition was also included. The results demonstrated that recall
for spatial information was most accurate in the free eye-movement condition, slightly lower in
the fixed-eye condition, and markedly lower in the conditions where they were required to move
their eyes. Recall for the verbal information was not disrupted by any of the tasks. These results
were taken by the authors to suggest that while, eye movements per se do not play a role in
working memory for spatial information, the allocation of eye movements to another task
disrupted rehearsal for the spatial information (Postle et al., 2006).
Finally by Baddeley and colleagues (Postle et al., 2006) examined whether the voluntary
use of eye movements was important during the initial encoding of the material. They evaluated
whether; the participants needed to move their eyes along the path through the matrix as is it was
being described to demonstrate successful recall; free eye movements to scan the image already
entered into the working memory store; or both to optimize performance on the recall task. Using
the same task described in Experiment 2, the authors required control of voluntary eye
movements to either a fixed or moving location during presentation of the material, during recall,
and both during presentation and recall. Results indicated that free use of eye movements was
necessary during recall, and might be necessary during encoding. Recall performance was
disrupted to a slight, but significant, degree for both the spatial and verbal material when
voluntary eye-movements were restricted during presentation, indicating that perhaps the
EXAMINING EYE GAZE IN A DMTS TASK 11
disruption in performance was due to interference with a joint system within the model, rather
than interfering with just the spatial working memory system (Postle et al., 2006).
These early studies provide some initial suggestive evidence that recall for material that is
considered based on visual or spatial material may rely to some extent on the use of, or control
of, eye movements. However, while the Brooks (1967) task is considered to be reliant on spatial
memory, it does not make use of visually presented materials. Also, the use of eye-tracking
technology was restricted to ensuring integrity of the independent variable, that is, whether the
participant’s eyes were moving in the manner intended, either as a result of the rotational
nystagmus, or voluntarily using the eyes to fixate or track the specified stimulus. Further, the
eye-tracking equipment from the 1970’s was described as being less sensitive than more modern
technology (Postle et al., 2006).
The final study described in Postle and colleagues (2006) was an extension of a previous
study by Postle, D’Esposito, and Corkin (2005). It utilized more sensitive eye tracking
equipment to ensure that participants were moving their eyes according to the experimental
instructions, while also measuring some aspects of the eye movements, including the number of
saccades (quick movements of both eyes between periods of fixation) and the distance travelled
by the eye during one of the distraction conditions (Postle et al., 2006). The task that the Postle
and colleagues (2006) utilized was also changed from the Brook’s (1967) task to a delayed
recognition task, with a spatial condition and a shape condition. The shapes that were utilized by
Postle and colleagues (2006) were initially developed by Attneave and Arnoult (1956), and then
expanded and modified by Postle and D’Esposito (1999), and had been demonstrated to have a
low likelihood of prior learning history (Vanderplas & Garvin, 1959). The delayed recognition
task was used in part because of the temporal separation between stimulus encoding periods and
response periods, allowing for disruption of the maintenance period in between. There were
EXAMINING EYE GAZE IN A DMTS TASK 12
three disruption conditions tested in both the spatial and the shape main conditions: a) saccadic
distraction, where participants were instructed to move their eyes in any manner they choose
during the delay; b) word-reading distraction, where the participant was instructed to read 12
words per distraction interval; and c) a no distraction control condition. The results of this study
found that when participants were required to move their eyes, using the eye muscles but
supposedly not involving other aspects of attention, recognition for spatial information was
disrupted, but not shape recognition (Postle et al,. 2006). When participants were required to read
words, on the other hand, recognition for shapes was disrupted, but not spatial recognition.
Taken together the authors (Postle et al., 2006) suggested that the results from the four
experiments provided empirical evidence for the theory Baddeley presented in his 1986 book
that, analogous to the importance of sub-vocal rehearsal to the phonological loop, visual
rehearsal may derive from the cognitive resources that support the control of eye movements.
Studies have provided evidence for the hypothesized role of eye movements as the
rehearsal mechanism for the visuo-spatial sketchpad (Bochynska & Laeng, 2015; Della Sala, et
al., 1999; Laeng, Bloem, S’Ascenzo, & Tommasi, 2014; Logie, 1995; Postle et al., 2005).
However, most have relied on a dual-task paradigm, where the concurrent task is assumed to be
selectively disruptive to visual versus verbal memory processes. Rather than using fixed item
presentation and a concurrent task to examine the role of eye movements, Tremblay, Saint-
Aubin, and Jalbert (2006) utilized a serial position task and measured eye movements and
performance data to provide a direct measure of rehearsal based on eye movements. In a
computerized task participants were presented with a sequence of seven dots. All seven dots
were then presented on the screen simultaneously. After a 10 s retention interval participants
were required to point to each dot in the order in which it was originally presented. Rehearsal
was measured as the number of different adjacent locations of dot pairs looked at in the correct
EXAMINING EYE GAZE IN A DMTS TASK 13
order. A pattern of recall emerged that the Tremblay and colleagues (2006) suggested was
similar to what was found in verbal recall studies. Rehearsed pairs from the beginning and end
of the series were recalled with greater accuracy than pairs presented in the middle of the series.
Furthermore, the results indicated that greater numbers of rehearsed pairs during the retention
interval led to better overall recall performance, and that specific rehearsed pairs had a far greater
probability of recall compared with non-rehearsed dot pairs. In a second experiment participants
were presented with the same recall task, but were prevented from moving their eyes during the
retention interval (Tremblay et al., 2006). When rehearsal in the form of eye movements was
prevented, a similar pattern of recall emerged compared with Experiment 1, in terms of primacy
and recency effects; however overall recall accuracy suffered. The authors concluded that overt
rehearsal in the form of eye movements did take place and may aid in the serial recall of spatial
information (Tremblay et al., 2006). However, the authors also noted that while rehearsal in the
form of overt eye movements did lead to better accuracy, overall accuracy was still high even on
trials where there were no rehearsed pairs and when the free use of eye movements was
prevented (Tremblay et al., 2006). For both experiments the to-be-remembered (TBR) stimuli
were kept on the screen during the retention interval (Tremblay et al., 2006). It is unclear what
role eye movements would have had if stimulus cues were not available during the interval.
In contrast to the potential role of overt eye movements as a rehearsal mechanism indicated
in research conducted by Tremblay and colleagues (2006), Godijn and Theeuwes (2012)
provided evidence that suggested that overt rehearsal did not assist in recall of serial information.
In a study by Godijn and Theeuwes (2012) participants were presented with the numbers one
through six in ascending order at various semi-random locations on a computer screen. The
experimental manipulations included requiring that the participants hold their eyes in a fixed
position in the middle of the screen, requiring that they allocate a prescribed number of seconds
EXAMINING EYE GAZE IN A DMTS TASK 14
to numbers at the beginning of the series, requiring that they allocate a prescribed number of
seconds to numbers at the end of the series, and finally, allowing for free eye movements during
the retention interval. The results indicated that when participants allocated attention to the
lower digits their recall for the location of those digits was improved. However, requiring that
saccades be allocate to the higher digits impeded recall for the location of all digits. Further,
there was no difference in recall between the free eye movement and fixed eye conditions.
Godijn and Theeuws (2012) note that during the free eye movement condition, participants were
more likely to allocate saccades to the lower digits, and these digits were the ones that were best
recalled. However, the authors stated that this does not mean that this is due to the superiority of
overt rehearsal over covert rehearsal. Rather they see a role for covert shifts of attention as the
mechanism for maintaining spatial information (Awh, Jonides & Reuter-Lorenz, 1998, Awh &
Jonides 2001), noting that it is plausible that even during fixed or restricted eye movement
conditions that covert attention is allocated to the low digits (Godijn & Theeuws, 2012). Godijn
and Theeuwes (2012) further noted that during the initial observation of the sample, participants
allocated more observing time to the lower digits, suggesting that the allocation of saccades to
these locations during the retention interval may be incidental to the attention being paid to those
locations. Eye movements, in this view, may follow the shifts in attention, but not be a
necessary part of the rehearsal, though hypothesis was not directly tested in this study (Godijn &
Theeuws, 2012).
Logie (1995) has suggested that rather than the eye movements per se being necessary for
spatial memory, it is the active system involved in the planning of movements, including eye
movements, which is responsible for rehearsal of visual information. This alternate theory is
based on demonstrations of disruption for visual information happening with different forms of
task-irrelevant movements, such as finger tapping (Farmer, Berman, & Fletcher, 1986), arm
EXAMINING EYE GAZE IN A DMTS TASK 15
movements (Smyth & Scholey, 1994), as well as eye movements (Lawrence, Myerson, Oonk, &
Abrams, 2001). However, the task-irrelevant movements may have required spatial attention,
and that may have led to the disruptive effects rather than the planning per se (Logie, 1995).
Awh and colleagues (Awh, et al., 1998; Awh & Jonides, 2001) proposed that it was shifts
in spatial selective attention that was responsible for item maintenance in visual memory. Awh,
et al., (1998) conducted three experiments to evaluate their model of visuo-spatial working
memory that utilized shifts in selective attention as a mechanism to promote maintenance. Based
on prior evidence that given a choice task individuals demonstrated greater efficiency in
responding when spatial attention was already directed at the location where the choice stimuli
appear, Awh and colleagues (1998) presented participants with a dual task requiring both spatial
recall and a choice task. Unlike previous studies (Postle et al., 2006; Tremblay et al., 2006;
Godijn & Theeuws, 2012) the choice task that was required during the retention interval was not
intended to disrupt recall on the spatial task, but rather to see if reaction time was faster to the
choice task when the choice stimulus appeared in the same location as the TBR spatial
information. The results of the experiments indicated that reaction time was faster in the choice
task when the to be remembered (TBR) information was spatial, rather than related to item
identity, when participants were required to keep their eyes fixated during the entire trial (Awh et
al., 1998; Awh & Jonides, 2001). This data indicated that eye movements were not necessary for
item maintenance in visuo-spatial memory. The Awh and Jonides (2001) suggest that the eye
movement hypothesis may be reconciled with the spatial-attention hypothesis if one considers
that spatial-attention may be shifted to a location prior to eye movements being directed at that
location.
In summary, the previous research has provided data in support of a modal-model of
memory, with a dedicated visual working memory subsystem (Bochynska & Laeng, 2015; Della
EXAMINING EYE GAZE IN A DMTS TASK 16
Sala, et al., 1999; Laeng, Bloem, S’Ascenzo, & Tommasi, 2014; Logie, 1995; Postle et al., 2005;
Postle et al., 2006; Tremblay et al., 2006). In providing data to support a particular hypothesized
mechanism for item maintenance in visual memory, most of the studies utilized a dual-task
paradigm, where the hypothesized maintenance mechanism was disrupted by a secondary task
that should require the same cognitive resources (Postle et al., 2005; Postle et al., 2006). In most
of the studies, the use of eye-tracking equipment was limited to ensuring procedural integrity,
recording whether participants did or did not move their eyes in accord with the independent
variable (Postle et al., 2005; Postle et al, 2006). Furthermore, much of the aforementioned
research focused on recall for spatial information, rather than item identity. The exception to this
was the work conducted by Postle and colleagues (2005; 2006)
The purpose of the experiments Postle and colleagues (2005; 2006) was in part to compare
spatial recall with recall for a nonsense shape. They found that saccadic eye movements
disrupted performance on the location task, but not for non-spatial information (Postle et al.,
2005; Postle et al., 2006). Further, in reviewing the literature, only one study was found that
tracked eye movements as a dependent variable during a serial recall task (i.e., Tremblay et al.,
1996). In this study, however, the stimulus array was visible for the entire trial. Godijn and
Theeuwes (2012) did measure eye movements during the observation and retention intervals
during a spatial memory task. Their results indicated that overt use of eye movements did not
contribute to recall for the spatial location of presented digits. However, this research evaluated
material presented in a serial order and recall for spatial information (Godijn & Theeuws, 2012).
The stimuli Godijn and Theeuws (2012) used were the digits one through six, which may have
allowed the participant to use the repetition of the numbers as part of their recall strategy. In
reviewing the available literature no studies were found that assessed the role of eye movements
in the recall for visually presented items, rather than spatial information.
EXAMINING EYE GAZE IN A DMTS TASK 17
Behavior analysis and memory
All of the visual memory theories discussed previously in the manuscript have started from
an initial theoretical position that presumes cognitive structures involved in the temporary
maintenance and manipulation of information (Baddeley, 1986; Baddeley & Logie, 1999).
Behavior analysis, on the other hand, seeks to account for the current performance of a behavior
as a function of environmental variables. This poses a problem when it comes to responses that
are traditionally thought of as memory processes, as the environmental stimuli are no longer
present to allow for a simple environmental explanation. For example when asked, “what are you
eating?” while having breakfast the response of “corn flakes” can be accounted for by the
presence of the bowl of corn flakes. When asked later on in the day “what did you eat for
breakfast?” the same response of “corn flakes” requires a different explanation. The corn flakes
are no longer present to act as a discriminative stimulus evoking the tact in the presence of the
question. And the question alone cannot account for the response as the answer may vary day-
to-day depending on what was eaten. That individuals are able to respond about past events is
evident, however an analysis that relies exclusively on observable variables appears to be
insufficient to describe and predict such everyday performances. The difficulty that the field of
behavior analysis has in accounting for such responses allows ample room for cognitively based
philosophical and psychological explanations to step in, and for metaphorical language to take on
the role of scientific explanation.
One difficulty in developing a behavior analytic account of memory is that the level of
control available in the laboratory is not easily achievable for covert activities, thus requiring a
degree of inference in the explanation. Palmer and Donahoe have extensively discussed the role
of interpretation in the science of behavior (Donahoe & Palmer, 2004; Palmer, 1991; Palmer &
Donahoe, 1992). According to Palmer and Donahoe, all sciences have a degree of interpretation,
EXAMINING EYE GAZE IN A DMTS TASK 18
where phenomena that are not directly observable are explained by principles that have been
demonstrated in analogous conditions. A behavior analytic account of complex human behavior,
including memory, is necessarily going to be interpretive for several reasons. For one, the
precise levels of control necessary for empirical research may be difficult to achieve. Further, the
units of behavior are likely to be less discrete. And finally, all parts of the stimulus-response-
stimulus chain may not be independently observable. As in other scientific interpretive
explanations, the description and explanation for the phenomenon of memory should be
constrained by the principles that have been the subject of rigorous experimental study.
Introducing explanatory variables that have not been the subject of independent and objective
observation and manipulation moves the explanation from interpretive to speculative (Donahoe
& Palmer, 2004).
One of the central tenets of behaviorism as a philosophy is that a science of behavior is
possible, and further that it is a natural science (Skinner, 1953; Watson, 1913). As a natural
science the implication is that behaviors are events that are explained by other natural events, and
that the explanations will be sufficient to explain all important phenomena (Baum, 2005). Two
important branches of behaviorist thinking are methodological behaviorism and radical
behaviorism. A primary distinguishing feature between these two views is what is included as
important phenomena in need of an explanation. Methodological behaviorists consider only
observable stimuli and responses within their analysis of behavior (Baum, 2001). Internal, or
cognitive, events are either denied as existing (Watson, 1920), or they are considered to be
unimportant as a subject matter (see Baum, 2011 for a recent example). While this approach to
behaviorism has strength in its rigorous adherence to empiricism, its denial of the existence or
importance of internal events rings as false to the ear of the majority of people who would claim
EXAMINING EYE GAZE IN A DMTS TASK 19
to feel, think, dream, and remember. This leaves those people looking to other disciplines to
answer the questions of how and why these experiences occur.
Radical behaviorism, on the other hand, is often defined by its consideration of what are
termed private events. Private events are the covert stimuli and responses that make up what we
would otherwise describe as our conscious experiences (Day, 1971, in Leigland, 1992).
Advocates of radical behaviorism are careful to distinguish between private events and mental
events. Mental events, such as the account of visual working memory processes described
previously, are events and structures that take place in an inner space and have no necessary
relation to independently verifiable variables found in nature. Mental events are taken as
explanations for the overt events that make up the datum of both cognitive and behavioral
scientists. Private events, in contrast, are not necessary to explain behavior, but rather are
themselves stimuli and behaviors that are subject to the same explanatory variables as overt
stimuli and behaviors (Skinner, 1953).
In Science and Human Behavior (1953) Skinner presented one of the earliest formulations
of a behavior analytic approach to what he termed private events. Rather than viewing the
environment within one’s skin as being different than the environment outside the skin, Skinner
stated that “a private event may be distinguished by its limited accessibility but not, so far as we
know, by any special nature or structure (Skinner, 1953, p. 257). Thus the data that we have
access to by considering our own covert experience is not necessarily wrong, but rather it lacks
independent verifiability. The assumption made by the radical behaviorist is, however, that if we
could independently collect data on private or covert events we would see that these events
respond to behavior principles of respondent and operant conditioning in the same way as the
behaviors that we can independently observe and record (Skinner, 1953).
EXAMINING EYE GAZE IN A DMTS TASK 20
In Skinner’s consideration of perception, he begins by presuming that seeing is a behavior.
And like verbal behavior, the behavior of seeing can be emitted to such a slight degree as to be
undetectable to an outside observer. In the case of verbal behavior we would describe this as
engaging in covert speech or thinking, in the case of perceptual behaviors, such as seeing, we
engage in remembering, imagining, and visualizing (Skinner, 1976). Thus when we “call an
image to mind” we are responding by seeing in the absence of the thing seen because of other
stimuli in the environment that have stimulus control over that response (Skinner, 1976).
Skinner provided two ways that this response might be conditioned using the behavioral
principles that underlie respondent and operant conditioning.
Skinner described respondent seeing as being instances in which we see something that is
not truly there because in the past that stimulus was present at the same time as a stimulus that
we are currently being exposed to (Skinner, 1953). He provides the example of the dinner bell
making us “see” food, as well as salivating. Understanding salivation as a behavior in the
respondent paradigm is well established. Skinner (1953) suggested that we can replace “seeing
food” for salivating in much the same way. Thus, when we are exposed to the smell of freshly
baked cookies, we may engage is several responses that were initially conditioned with this
smell, including seeing, or remembering, a kitchen from our childhood. Because several
responses may be elicited in a cascade of covert perceptual behavior, we might experience the
memory as a reminiscence giving us the feeling that we are reliving our previous experiences
(Donahoe & Palmer, 2004).
Seeing in the absence of the thing itself may also be conditioned by consequences as an
operant. If seeing a stimulus is reinforcing based on its previous conditioning history, then we
may arrange opportunities to be in the presence of that stimulus, or representations of it
(Donahoe & Palmer, 2004). Gazing at the tranquil blue waters off a Caribbean beach may be
EXAMINING EYE GAZE IN A DMTS TASK 21
reinforcing for an individual. In order to contact that reinforcer, they might take a vacation that
will allow them to relax on the beach and gaze at the ocean. Or they might buy a travel
magazine or watch travel shows that feature scenes of blue lagoon and white sandy beaches. Or
they might picture the scene to themselves, evoking remembered sights from the previously
viewed stimuli. Skinner points out that in this type of conditioned seeing, as opposed to the
seeing that has been developed through respondent conditioning, the behavior is controlled by
deprivation and reinforcement (Skinner, 1953). Thus, the longer it has been since a beach
vacation, the more likely it becomes that an individual will engage in behaviors that bring them
into overt or covert contact with the stimuli.
Further in his chapter on the nature of private of events in a natural science Skinner (1953),
described how covert seeing might contact reinforcers beyond the automatic reinforcement of
being in contact with reinforcing stimuli, for example in solving a problem. He provided the
following example:
“Think of a cube, all six surfaces of which are painted red. Divide the cube into twenty-seven equal cubes by making two horizontal cuts and two sets of two vertical cuts each. How many of the resulting cubes will have three faces painted red, how many two, how many one and how many none?” (Skinner, 1953, p 273)
This problem could possibly be worked out through an extensive verbal chain describing
the nature of a cube and its surfaces, but as Skinner (1953) pointed out, it is easier to solve
visually. He further suggested that it can be solved visually without having access to the visual
stimulus by generating a covert image that can be “seen” and manipulated. The degree to which
any given individual is able to use this problem solving technique is likely to be based on their
individual reinforcement history with regards to the stimuli, their reinforcement history with
regards to covert seeing, and the resulting skill at describing the resulting stimulation (Skinner,
1953).
EXAMINING EYE GAZE IN A DMTS TASK 22
Following a radical behavioristic approach to the nature of private events and conditioned
perception, an initial step in attempting an experimental analysis of private events, such as
memory, would be to reframe it as an action rather than a structure. As a behavior, remembering
is likely to be a chain of behaviors leading to a terminal response rather than a single discrete
response (Palmer, 2010). While analyzing every step in the chain may never be possible,
determining relations between behaviors within the chain may further the goal of interpreting
complex human behavior from a behavior analytic perspective. The boundary between what is
private and what is public shifts depending on the technologies available to the observer. For
example, a heartbeat is a private event until the observer uses a stethoscope. Similarly, the
behaviors involved in private events may occur to such a small degree that they are not detectible
to an observer without the use of specialized equipment. The experience of visual memory
would be inferred to involve the same or similar behaviors to what we would observe when
someone is seeing a stimulus that is currently present, but perhaps to a much smaller degree.
These behaviors might include movement of the eyes, perhaps following a similar pattern to the
initial observation of the stimulus. Through the use of eye tracking apparatuses, the behaviors
that happen at a smaller level during the experience of conditioned seeing may become
detectable, allowing for some data to supplement the role of interpretation (Palmer, 2010).
Empirical work on complex human behavior
Complex behavior is often the end result of a chain of very small behavioral events, the
scale of which is so small that it is difficult to bring to bear the experimental methods that
underpin our experimental and applied understanding of behavior (Palmer, 2010). As a result,
behavior analytic research on these areas remains limited, with much of the research that has
been done in the area of language. Experimental procedures often draw from similar areas in
cognitive research, such as the use of priming procedures and electroencephalogram (EEG)
EXAMINING EYE GAZE IN A DMTS TASK 23
readings in language (Barnes-Holmes et al., 2004; Haimson, Wilkinson, Rosenquist, Oimet, &
McIlvane, 2009; Hayes & Bissett, 1998).
In cognitive research the use of response latency has been used to measure the strength of
relationships between words, based on features such as semantic linkages, associational,
meditational and others (Barnes-Holmes et al., 2004). In priming procedures, a word is briefly
presented, followed by a second word. The latency to respond to the second word, for instance
by indicating that it is a word or non-word, is on average shorter when the words are related
(Barnes-Holmes et al., 2004). Response latency is a measure of behavior that has been used to
measure non-behavioral structural accounts of complex human behavior; however, as a measure
it has also been used to evaluate behavior-based accounts of language, such as stimulus
equivalence (Hayes & Bisset, 1998). For example, Hayes and Bisset (1998) used the priming
paradigm to measure responding to word pairs based on pairings derived through stimulus
equivalence. Their results indicated that responding was significantly faster for word pairs
generated through equivalent relations than unrelated word pairs (Hayes & Bisset, 1998). The
use of priming procedures has then been further refined with the use of electroencephalograms
(EEGs) readings measuring the event-related potential (ERP) between the presentation of the
stimulus and electrical activity within the brain (Barnes-Holmes, et al., 2005; Haimson, et al.,
2009). The use of ERPs is considered to possibly be more sensitive than overt responding as a
measure, as it occurs prior to the yes/no judgment, and can allow for measurement of the
emergent relations prior to testing, preventing the problem of learning through testing (Barnes-
Holmes, et al., 2005; Haimson, et al., 2009).
The priming studies previously described utilized the technology of cognitive psychology
in order to examine behavior principles. The eye tracking technology that has been utilized in
visual memory research has also been used to provide a behavioral measure of the primarily
EXAMINING EYE GAZE IN A DMTS TASK 24
covert complex human activity of attention. Dube, Balsamo, Fowler, Dickson, Lombard, and
Gerson (2006) used eye tracking equipment to measure the relation between eye movements and
duration of gaze and accuracy in a delayed matching-to-sample (DMTS) procedure. Participants
were exposed to 2-sample and 4-sample DMTS tasks. All participants had high accuracy scores
for the 2-sample trials. However, when the task became more difficult with the 4-sample
stimulus set, accuracy scores fell for two of the four participants. The data from the eye tracking
equipment indicated that subjects with both high and low accuracy made similar numbers of
observations to the sample stimuli; however, the duration of the observations for those with
higher accuracy scores was longer (Dube et al., 2006). Observation is necessary for stimulus
control to occur (Dinsmoor, 1985); the results of this study indicated that the duration of
observation may be an important variable in establishing accurate performance (Dube et al.,
2006): accuracy for the low-accuracy participants was improved through prompts to engage in
longer duration observations to each stimulus in the sample.
Purpose
Behavior analytic theory has ventured into the realm of complex human behavior, such as
memory and remembering; however, there is little empirical evidence to lend support to these
theories. The topic of visual memory has received attention in the field of cognitive psychology;
however, the methods utilized often include confounding verbal information (see Godijn &
Threeuw, 2012; Postle et al., 2005; Postle et al., 2006), or rely on spatial recall rather than item
recall (Tremblay et al., 2006). Within the field of cognitive psychology the dominant theory of
working memory for visual material incorporates the covert use of a visuo-spatial sketchpad as a
rehearsal and processing mechanism for visually presented stimuli (Baddeley, 1986). While the
field of behavior analysis includes only independently and empirically demonstrated principles
into accounts of behavior, excluding storage accounts and mental sketchpads, we may have
EXAMINING EYE GAZE IN A DMTS TASK 25
explanatory variables that can account for the experience of human memory. By considering
Skinner’s accounts of conditioned perception and covert seeing (Skinner, 1953; Skinner 1974) it
can be hypothesized that the behaviors that occur when the item is first seen are conditioned to
the stimuli present, and that under similar stimulus conditions the same behavior may occur at a
much smaller, or covert, level. Thus, when we are asked to respond to previously viewed stimuli
we may engage in some of the same seeing behaviors that we engaged in when we first viewed
the stimuli. One aspect of seeing that may occur as part of covert seeing may include moving
one’s eyes in a pattern similar to what would be expected if the stimuli were present.
The purpose of the current study was to utilize eye tracking equipment to measure the
relation between the number and duration of eye gaze fixations to the locations of previously
viewed visual stimuli in a delayed match-to-sample task. In order to control for the potential
confound of covert echoic rehearsal contributing to performance on the DMTS task, stimuli
included Mandarin characters, and participants were screened for prior knowledge of Chinese
writing systems. The purpose of Experiment 1 was to evaluate the role of eye gaze in an item
identification task. Experiment 2 was designed as a pilot study to begin to explore the role of eye
movements in a task that required a response based on location.
Experiment 1: Method
In Experiment 1 participants were presented with a DMTS task that began with an initial
sample containing one, two or four stimuli located in the four corners of the screen. Eye tracking
data were gathered during the initial sample observation, and during the 5 sec delay interval up
to and including the response to the comparison. Participants then responded to the comparison
with a ‘Yes’ or ‘No’ key press to indicate whether the comparison matched one of the initial
sample stimuli. Utilizing similar methodology to Dube, et al., 2006 the relations between the
number and duration of eye-gaze fixations and correct responding on the DMTS task were
EXAMINING EYE GAZE IN A DMTS TASK 26
examined. In addition the probability that the last location viewed prior to a correct ‘Yes’
response matching the location of the initial sample was calculated.
Subject Selection
Participants were recruited from a medical school campus. Flyers were posted around the
campus with a brief description of what potential participants would have to do, the requirements
of participating, and the experimenter’s contact information. Participants had to be 18 years or
older and have their own transportation to the study site. Potential participants were informed
that they would be receiving monetary compensation in the amount of $25 upon completion of
the study. Potential participants contacted the experimenter, who provided materials about the
study, including a consent form, as well as a description of the equipment to be used. Potential
participants were informed that they would be able to end experimental sessions at any time and
that they could withdraw from the study at any time without any detrimental effect towards the
participant. Exclusion criteria included self-reported prior knowledge of, or ability to read,
Mandarin characters.
Participants
Participants were four adults (3 males and 1 female), who were enrolled in their first year
of medical school. All were aged 22-25 years old. None of the participants had any known
clinical conditions, and all had vision that was considered adequate to drive, with or without
corrective lenses.
Prior to taking part in the study, participants read and signed a consent form that described
the eye tracking apparatus as “used to measure certain characteristics of your eye (e.g. pupil size)
during computerized learning tasks.” (this was based odd the methods used by Schroeder &
EXAMINING EYE GAZE IN A DMTS TASK 27
Holland, 1968). This deception was included to preserve the natural eye movements that the
individual would typically make during this type of task.
Setting and apparatus
Experimental sessions were conducted in a designated research room in a medical school
campus building. The participant was seated on an adjustable office chair in front of a chin rest.
The chin rest could be raised or lowered by the participant so that it was comfortable for them for
the duration of the session, though they were not required to use the chin rest past calibration of
the equipment. A standard keyboard was placed on the table in front of the participant to record
their responses. The keyboard was marked with four circular stickers to indicate the response
keys, though only two response keys were utilized in Experiment 1. The ‘yes’ response key had a
green sticker placed over top of it and replaced the ‘Z’ key on the standard computer keyboard.
The ‘no’ response key had a red sticker placed over top of it and replaced the ‘/’ key on the
standard keyboard. The spacebar was the only other active key after the initial instruction
screen. A color monitor was used to present sample and comparison stimuli.
On a separate table, located perpendicular to the participant table, were seats for the
experimenter along with two computer monitors and two keyboards. One of the experimenter
computers was used to calibrate the ISCAN © system; following calibration the experimenter
could monitor the eye tracking apparatus, seeing the current point-of-regard for the participant on
this computer. The second experimenter computer was used to start up and follow the stimulus
presentation being viewed by the participant. Neither of the experimenter monitors were visible
to the participant.
The eye-tracking apparatus was an ISCAN © Point-of-Regard system. The ISCAN ©
system shone a laser beam into the participant’s eye and the refraction angle between a point of
light on the participant’s pupil, and the point of light on the participant’s lens was measured to
EXAMINING EYE GAZE IN A DMTS TASK 28
calculate the point-of-regard. A real-time image of what the participant was looking at was
available on the experimenter’s computer, with a cursor to show the precise location of the
participant’s eye gaze. The beam of light was harmless to the eye, and did not disrupt regular
vision.
Delayed Match to Sample Task
DMTS stimulus sets. The stimuli used in the DMTS task were black characters from the
Mandarin alphabet created using Microsoft Word 2007 ®. All characters were in 72pt font, and
presented on a white background. The stimulus set for each trial was drawn at random from a
pool of 210 different characters, without replacement. When all characters had been exhausted
from the pool, and then stimuli were re-drawn from the pool, at random, with the restriction that
correct comparisons from previous trials were not reused.
The stimulus array was initially created using Microsoft PowerPoint ® (2007). Sample
stimulus presentation slides were created containing one-, two-, or four-stimuli per slide, as a
way of increasing the complexity of the DMTS task. Sessions consisted of eight one-stimulus
trials, followed by a block of 24 two-stimulus trials, and a block of 24 four-stimulus trials.
Stimuli were located in the corners of the screen, with the number of corners occupied varying
depending on the number of stimuli presented. The specific corner for each stimulus was semi-
randomly determined using a random number generator, with the criteria that stimuli had to
appear in each location an equal number of times.
Trials were designated as either matched or unmatched using a random number
generator, with a criterion that there were an equal number of each trial type. For the one-
stimulus trials either the same stimulus that was presented as a sample was presented as a
comparison (i.e., matched), or a randomly selected stimulus from the pool was presented (i.e.,
unmatched). For the matched two- and four-stimulus trials one of the stimuli from the initial
EXAMINING EYE GAZE IN A DMTS TASK 29
sample was semi-randomly selected, with the criterion that the match was initially presented in
each of the four screen locations an equal number of times. Unmatched comparisons were
selected in the same manner as one-stimulus trials. The comparison stimulus was presented in
the middle of the screen, equidistant from the top and bottom, and right and left sides.
Stimulus set presentation. The PowerPoint® stimulus presentation slides were converted
into bitmap (BMP) images and entered into Presentation™ software. The Presentation™
software was used to present each slide as a JPEG image according to programmed timing rules.
This software collected data on the latency to respond from the presentation of the comparison
stimulus, as well as accuracy information on whether the response was correct or incorrect as
defined for each trial. Stimulus randomization took place prior to entry into the Presentation™
software, and the order of trials remained fixed for all participants.
Trials were programmed so that each trial started with the initial sample, which remained
on the screen until the participant pressed the spacebar on the computer keyboard. Following the
initial sample presentation the participant viewed a blank white screen for the 5 s delay interval.
Following the delay, the comparison stimulus was presented and remained on the screen until the
participant responded by pressing either of the two response keys. There was a 5 s intertrial
interval between each trial, during which the participant again viewed a blank screen.
Eye tracking software. The eye tracking PRZ™ software collected information on all
locations on the computer screen that the participant viewed during the experiment. Prior to the
experiment, specific areas of the screen were designated as regions of interest (ROI) using the
PRZ software such that data on the number and duration of fixations within these ROI could be
gathered. The initial step in this process was to take the already converted BMP images and
register them using the PRZ software, which converted the image to an IGR file. Once images
EXAMINING EYE GAZE IN A DMTS TASK 30
were saved as registered files, they were then overlaid with the ROI and saved as element (EMT)
files.
A grid with five ROI were used for each slide viewed by the participant during a trial,
including the blank delay screen. The grid had a box for each corner and one box was also placed
in the middle of the screen. In order to register the images with the ROI each registered image
(IGR) file was brought up in the PRZ software, the grid was visible around each stimulus that
was presented on the screen, as well as the remaining locations where stimuli were not currently
located. This screen with the ROI were then saved as an element EMT file.
Dependent variables and measurement
Accuracy of match. Data were collected on the accuracy of the participant’s responses
during the DMTS task. A correct response included instances of the participant pressing the
‘yes’ key on the keyboard when the comparison stimulus matched any one of the initial stimuli
presented in the sample presentation, or pressing the ‘no’ key if the comparison stimulus did not
match any of the initial stimuli. An incorrect response included instances of the participant
responding by pressing ‘yes’ on trials where the comparison did not match any of the initial
sample stimuli, as well as instances of the participant responding by pressing the ‘no’ key when
the comparison stimulus did match one of the initial samples.
Visual gaze definitions. The PRZ software collected data on the number, duration and,
pattern of eye gaze fixations directed at the ROI specified in the element entry grid. A fixation,
or observation, was defined as any instance of the point-of-regard cursor remaining within a
region of interest for at least 100 ms. This definition of fixation was programmed into the PRZ
software, which collected and organized the data. Data on fixations were considered for the
initial sample screen, the delay interval, and for the comparison screen separately. Data on
visual gaze were not collected during the intertrial interval.
EXAMINING EYE GAZE IN A DMTS TASK 31
The number of fixations within one of the ROI was defined as the frequency of observing
responses lasting at least 100 ms within the previously registered element boxes. The number of
observations were recorded during the initial sample viewing, during the delay interval and
during the time the comparison is on the screen, up to the participant’s response.
The duration of fixations were recorded for each of the ROI. Duration was recorded for
fixations lasting at least 100 ms, and ended when the point-of-regard cursor left the region of
interest for at least 100 ms. Duration was aggregated for the total duration that each region was
fixated on during the initial sample presentation, during the delay, and while the comparison was
presented up to the participant’s response.
Procedures
Participants first read and signed a consent form that described the apparatus as “used to
measure certain characteristics of your eyes (e.g. pupil size) during computerized learning trials”
(after Schroeder & Holland, 1968). Any further questions about the apparatus were deferred
until a debriefing following the end of participation in the study.
On entering the eye tracking room, participants took a seat in front of the chin rest to
prepare for calibration. Each session began with a brief calibration routine in which participants
were asked to keep their head still and on the chin rest, and to fixate on targets that appeared in
various locations on the stimulus display monitor. The target stimuli were unrelated to stimuli
used in the experimental procedures. Following calibration, participants were told that they no
longer needed to hold their head still. They were then presented with an instruction screen that
read:
“In this experiment characters will appear on the screen. After you have finished viewing
all of the characters press the spacebar. When you press the spacebar the characters will
disappear. After a few seconds a single character will appear in the middle of the screen.
EXAMINING EYE GAZE IN A DMTS TASK 32
Your task is to decide whether you saw this character on the prior screen. Press the GREEN
button if the second shape matches any of the characters from the previous screen. Press
the RED button if it does not match any of the previous characters. It is most important to
be accurate, but try to work as fast as you can. Press any key to start.”
Participants were asked to read the instructions aloud, and the session started when the subject
pressed any key on the keyboard. After pressing one of the keys, the participant began with the
one-stimulus trials. One-stimulus sessions consisted of eight delayed match to sample trials.
Immediately following the one-stimulus trials were 24 two-stimulus trials, followed by 24
four-stimulus trials. The participant was presented with the initial sample screen which they
could view for as long as they wanted. Pressing the spacebar removed the initial stimulus and
initiated the delay interval which presented a blank white screen for 5 s. Following the delay, the
comparison stimulus appeared on the screen and remained until the participant responded by
pressing either ‘yes’ or ‘no’. After one of these two keys had been pressed the, screen again
went blank for a 5 s intertrial interval. The next trial then automatically began. Participants did
not receive feedback on whether their response was correct or incorrect.
Interobserver agreement
The Presentation software collected data on the accuracy of responses. The PRZ software
collected data on the number and duration of fixations. Because all data were collected by a
computer interobserver, agreement was not calculated.
Results
Accuracy
Overall accuracy on the DMTS task is displayed as a percentage of correct responses over
total responses in the first row of Tables 1, 2, and 3, which show the data for one-, two-, and
four-stimulus trials, respectively. Each column displays the data for one participant. All
EXAMINING EYE GAZE IN A DMTS TASK 33
participants demonstrated a high level of accuracy on the one-stimulus trials, with only
Participant 1 making one error. Overall there was a decrease in accuracy as the task became
more complex. The exceptions were Participant 4 who didn’t make any errors on either one- or
two-stimulus trials, and Participant 2 who had the same overall score of 79.2% accuracy on both
two- and four-stimulus trials.
The second and third rows on Tables 1, 2, and 3 display each participant’s accuracy on
matched and unmatched trials. On the one stimulus trials the, single error made by Participant 1
was on an unmatched trial. On two- and four-stimulus trials, three of the four participants had
better accuracy scores on the unmatched compared with the matched trials. The exception to this
was Participant 4, who had better accuracy on matched than unmatched four-stimulus trials.
Latency
The average latency to respond for each participant is shown in Table 4. Major columns
show the data for each participant, with sub-columns showing data for one-, two-, and four-
stimulus trials. The top row presents the overall latency at each level of complexity, and data on
correct and incorrect as well as matched and unmatched trials follow. The latency to respond to
the comparison increased as the complexity of the task increased for three of the four
participants. The exception is Participant 3, who had a shorter overall latency on the two-
stimulus trials compared with one- and four-stimulus trials. There was minimal difference in
overall latency between one- and four-stimulus trials for this participant.
The data for Participant 1, 2, and 3 indicate that there was a longer latency to respond on
incorrect trials compared with correct trials across all trial types, with the exception of one-
stimulus trials for Participant 1. It should be noted that data presented represents only a single
trial. Participant 4, who only responded incorrectly on four-stimulus trials, had a longer latency
to respond on correct trials compared with incorrect trials.
EXAMINING EYE GAZE IN A DMTS TASK 34
Both across and within participants there was variability on whether a longer or shorter
latency to respond was associated with matched or unmatched trials. Participant 1 had a shorter
average latency to respond on matched trials compared with unmatched on all trial types.
Participant 2 and Participant 3, on the other hand, had a shorter average latency to respond to
unmatched trials across all trial types. Participant 4 showed greater variability in the correlation
between trial type and latency to respond. Participant 4 had a shorter average latency to respond
to unmatched one-stimulus trials, and matched two- stimulus trials. Participants 2, 3, and 4 had
very similar latencies to respond to the comparison on matched and unmatched four-stimulus
trials.
Observing responses
Delay. Data on the average number and duration of fixations made during the delay are
presented in Tables 1, 2, and 3, as well as on Table 5. Tables 1, 2, and 3 display the average
number of observations, or fixations, made to areas of the screen that did and did not display
stimuli during the initial sample presentation, on one-, two-, and four-stimulus trials,
respectively. On one-stimulus trials, all of the participants had a higher average number of
fixations and a longer average duration of observations to locations that contained a sample
during the initial display compared with previously empty locations. The same pattern was
repeated on two-stimulus trials, with all participants having a higher average number of fixations
and longer average duration to the ROI where a stimulus had been presented. On four-stimulus
trials, all ROI displayed stimuli during the sample presentation. For all participants, this
condition had the highest average number of fixations and longest average duration of fixations.
Table 5 shows the average number of observations to ROI that displayed stimuli during the
sample presentation, with average duration shown in brackets. Each panel represents the data for
one participant, with the columns displaying the data per trial type, and the row displaying data
EXAMINING EYE GAZE IN A DMTS TASK 35
for trials that were ultimately correct or incorrect. The (*) symbol is used to indicate when there
was only one trial of a given type. Across participants there is variability in whether a greater
number of fixations was associated with trials that were ultimately correct or incorrect across the
matched and unmatched trials at each level of complexity. Participant 1 had a higher average
number, and longer average duration, of fixations during the delay on trials that were ultimately
correct on one- and two-stimulus trials, though it should be noted that there was only one
incorrect one-stimulus trial. On four-stimulus trials, the average number and average duration of
observations made during the delay were comparable for matched trials that were ultimately
correct and incorrect. On unmatched four-stimulus trials, the trials that were ultimately correct
were correlated with a higher average number and longer average duration of fixations compared
with the trials that were ultimately incorrect. Participant 2 did not have any incorrect responses
on one-stimulus trials. On two- and four-stimulus trials, there is variability on whether having a
higher average number of fixations was correlated with trials that were ultimately correct. On
unmatched two-stimulus trials, there was a slightly higher average frequency of observations
during the delay for correct trials, and on four-stimulus trials there was a higher average number
of observations made on correct matched trials. However, the highest average number and
longest average duration of observations occurred on unmatched trials that were ultimately
incorrect. Both Participant 3 and Participant 4 made very few errors across all levels of
complexity, and each of them had the highest average number and longest average duration of
fixations during the delay. For Participant 3, there was only one error on a matched two-stimulus
trial, and one error each for matched and unmatched four-stimulus trials. The highest number of
fixations to ROI during the delay occurred on an unmatched trial that was ultimately incorrect. In
all other instances, there was a higher number of observations made during the delay on trials
that were ultimately correct. Participant 4 made no errors on one- and two-stimulus trials, and no
EXAMINING EYE GAZE IN A DMTS TASK 36
errors on matched four-stimulus trial. The errors that were made on four-stimulus trials were on
unmatched trials, and while there was a higher average number of observations on the incorrect
unmatched trials compared with correct unmatched trials, the highest average number of
observations made by this participant occurred on matched trials that were ultimately correct.
Comparison. In addition to displaying accuracy and observation data collected during the
delay, Tables 1, 2, and 3 also present the mean frequency and duration of observations made
while the comparison screen was displayed, up to the participant’s response. Data on
observations made to ROI that did and did not initially display a sample, as appropriate, are
shown in the bottom rows below the similar data collected during the delay. Table 1 presents the
data for one-stimulus trials. At this level of complexity all participants had the same, or higher,
average number of fixations to regions that did not display a stimulus during the initial sample.
Participant 1, who had the same average number of fixations to areas that did and did not contain
a sample, had a longer average duration of observations to the locations that did display the
initial sample. For the other three participants, the average duration of observations was also
shorter to locations that displayed the initial sample.
Table 2 shows the data for the two-stimulus trials. Participant 1 made no observations to
the ROI where stimuli were initially presented during the comparison. Participant 2 and 4 both
made a higher average number of fixations, and had a longer average duration of fixations, to
screen locations where stimuli had been presented compared with locations that had been blank.
Participant 3, on the other hand, had only a slightly higher average number and duration of
observations to previously occupied regions compared with unoccupied. Table 3 shows the data
for the four-stimulus trials. As previously mentioned, this Table does not allow comparison
between locations that did and did not present stimuli as all four ROIed held a stimulus on the
EXAMINING EYE GAZE IN A DMTS TASK 37
sample screen. Across all four participants, the highest average number of observations and
longest average durations occurred during the four-stimulus trials.
Similar to Table 5, Table 6 presents the data on observations made during the comparison
as a function of whether the trial was matched or unmatched and correct or incorrect. Each
participant’s data is presented in a separate panel. The columns present the level of complexity,
with sub-columns for matched and unmatched trials. The top row in each panel present data on
correct trials, and the bottom row incorrect trials. The average number of fixations is presented
with the average duration presented in brackets. The (*) symbol is used to indicate when there is
only one trial of a given type. The data for Participant 1 is shown in the top panel. On one
stimulus trials the, only fixations made back to initial sample locations occurred on correct
matched trials. There were no incorrect matched trials, and no observations made back to
occupied ROI on either correct or incorrect unmatched trials. On two-stimulus trials Participant 1
made no observations back to ROI that had initially displayed stimuli. During four-stimulus
trials, Participant 1 had the higher average number of observations on matched correct trials,
compared with unmatched. However, the overall highest average number of observations was
made on incorrect unmatched trials. No observations were made during the comparison on
incorrect matched trials. The data for Participant 2 is shown in the second panel. On one-
stimulus trials, there were no incorrect trials. On matched trials, Participant 2 made no
observations back to locations were stimuli were previously displayed, compared with an
average number of 0.333 observations on correct unmatched trials. On two- and four-stimulus
trials, Participant 2 had fixations back to ROI where stimuli had been presented on all trial types.
On two-stimulus trials, Participant 2 had a higher average number of observations on correct and
incorrect matched trials compared with unmatched trials. The longest average duration of
observations occurred on correct matched trials. Unmatched trials had a far lower average
EXAMINING EYE GAZE IN A DMTS TASK 38
number of observations, though the average duration was comparable. Correct unmatched trials
had a higher average number of observations compared with incorrect unmatched trials. On four-
stimulus trials, Participant 2 had a similar average number of observations to ROI that had
previously displayed stimuli on both correct and incorrect matched trials. On unmatched trials,
Participant 2 had a lower average number of observations on correct trials, and the highest
average number of observations on incorrect trials, as well as the longest average duration.
Participant 3 had a higher average number and longer average duration on correct unmatched
compared with correct matched one-stimulus trials. There were no incorrect trials at this level of
complexity for this participant. On two-stimulus trials, however, there was a higher average
number of observations on correct matched trials compared with correct unmatched trials.
Finally, on four-stimulus trials, Participant 3 had a higher average number of observations on
both correct and incorrect unmatched trials compared with matched. The highest number, and
longest duration, of fixations occurred on an incorrect, unmatched trial, though it should be noted
that there was only one trial of this type. The bottom panel of Table 6 shows the data for
Participant 4. There were no incorrect one- or two-stimulus trials. The only incorrect trials were
unmatched four-stimulus. Across correct trials Participant 4 had similar average number of
observations on both matched and unmatched trials. One one-stimulus trials this number was 0.
On two-stimulus trials, Participant 4 made an average of 1 and 1.167 fixations on matched and
unmatched respectively, with comparable average durations of observations. On four-stimulus
trials, Participant 4 had an average of 2.333 fixations during the comparison on matched trials,
and an average of 2.11 on unmatched trials, again with highly comparable average durations.
Similar to the other three participants, Participant 4 had the highest average number, and highest
average duration, of observations on incorrect unmatched trials.
EXAMINING EYE GAZE IN A DMTS TASK 39
Figure 1 is a bar graph presenting the proportion of matched two- and four-stimulus trials
where the participant made at least one observation back to the region of interest where the
correct sample was located during the comparison slide. The X-axis shows the data for each
participant. The solid bars represent an overall percentage of matched two- and four-stimulus
trials with at least one observation made to the region of interest where the correct sample was
presented. The bar with diagonal lines represents the percentage of correct trials with at least one
observation back to the initial sample location, and the bar with vertical lines represents incorrect
matched trials. Data are presented as a percentage of trials. Participant 1 made no observations to
the location where the correct match had been presented during the comparison. Participant 2
made observations to the region of interest where the matched sample had been presented in 30%
of trials. Of correct trials 40% had at least one fixation to the location of the initial sample, and
on incorrect trials only 10% had at least one fixation back to the location of the initial sample.
Participant 3 had a lower proportion (13%) of matched trials where at least one fixation was
made back to the initial location of the matched stimulus, though all of those fixations were
made on correct trials. Participant 4 made at least one observation back to the region of interest
where the correct match had been presented on 25% of trials. Participant 4 made no errors on
matched trials.
Discussion
Accuracy
Across participants, accuracy on the DMTS task decreased as the number of sample stimuli
increased, indicating that the task was becoming more complex. When considering matched
versus unmatched trials, across all participants and levels of complexity, the accuracy score for
unmatched trials was either equal to (at 100%) or higher than for matched trials. The exceptions
were Participant 1 during the one-stimulus trials, where the single error made was on an
EXAMINING EYE GAZE IN A DMTS TASK 40
unmatched trial, and Participant 4 during the four-stimulus trials. Excluding these exceptions, it
appears that matching identical stimuli was somewhat more challenging for these participants
than responding to a lack of identity. While it is not clear why this would be the case, and it is
outside the scope of this study to evaluate, it does open an area for future research utilizing
similar eye gaze equipment and measures. For example, it may be fruitful to compare the
fixation pattern when perceiving the initial sample to the fixation pattern when considering the
comparison, and evaluating patterns in how the two stimuli are viewed. For instance, it may be
that a high degree of similarity in viewing patterns is correlated with correctly identifying a
match, and deviations in viewing patterns leads to incorrectly identifying a match as unmatched.
Alternatively, for unmatched stimuli it could be informative to consider how early the viewing
patterns, if they exist, can diverge from each other and result in the participant correctly
identifying the two stimuli as being non-identical.
Latency
Along with the increase in complexity there was also an increase in latency to respond to
the comparison as more stimuli were initially presented. The exception was Participant 3, who
had a lower latency on two-stimulus trials compared with one- and four-stimulus trials. This
participant maintained a fairly consistent latency, of about 2.5s, across all trial types. The other
participants all demonstrated an increasing latency as the number of initial samples went up.
However, while there was overall a pattern across participants demonstrating a greater level of
accuracy on unmatched trials, there was not a similar pattern in the latency to respond correlated
with trial type. On one hand, for all participants with the exception of Participant 4, there was on
average a longer latency to respond on incorrect trials compared with correct trials. Participant 4,
who only made incorrect responses on four-stimulus trials, had a longer latency to respond on
correct trials. When considering whether trials were matched or unmatched there was variability
EXAMINING EYE GAZE IN A DMTS TASK 41
across participants. This variability indicates that there are individual differences, and possibly
differences within one individual, depending on characteristics of the stimuli, in the length of
time it takes to determine identity versus non-identity. It is unclear how this might relate to eye
gaze patterns, if at all.
Observing responses
In considering accuracy across the four participants, their performance can be roughly
grouped into those with high and low accuracy on each trial type. With the exception of
Participant 2, who had perfect accuracy on one-stimulus trials, Participant 1 and 2 demonstrated
lower accuracy compared with Participant 3 and 4 across all trial types. Participant 1 and 2 also
had a lower average number and duration of fixations to ROI that previously displayed stimuli
across all trial types compared with Participants 3 and 4. Patterns in observing responses during
the delay and comparison will be considered both within each participant and across the broadly
defined groupings of high and low accuracy responders.
Behavior during the delay interval. Despite the fact that the delay screen was blank,
and offered no further stimulation related to the DMTS task, all participants made eye gaze
fixations back to the locations where the stimuli were initially located during the delay. While
the role of these fixations cannot be definitively determined from the current study, there are
patterns both within and between participants that can be examined.
When considering the one- and two-stimulus trials, the comparison between previously
occupied and previously empty locations is possible. On these trials, all participants had a higher
average number and longer average duration of observations to locations where the stimuli had
initially been presented compared with locations that had been blank. For Participants 1, 3, and
4, the average number of fixations to occupied locations was more than four times the average
number of fixations to previously empty locations. For Participant 2, the difference was less
EXAMINING EYE GAZE IN A DMTS TASK 42
striking, though still more fixations occurred to previously occupied than to empty locations
during the delay. On four-stimulus trials, all locations contained stimuli during the initial sample,
and so the same comparison between occupied and empty regions is not possible. However, the
highest average frequency and longest average duration of fixations to the ROI is seen in this
condition for three out of four participants. Considering the eye gaze data across the increasing
levels of complexity, Participants 2, 3, and 4 displayed an increasing trend in the mean frequency
and duration of fixations as the number of initially presented stimuli increased. The mean
frequency and duration of fixations exhibited by Participant 1, on the other hand, displayed a
decreasing trend, having the highest mean frequency and longest average duration of fixations
during the delay on one-stimulus trials, and the lowest on the four-stimulus trials.
While the data indicate that on the DMTS task participants tended to look back to the
locations where stimuli had been when presented with a blank screen during the delay, the role
of eye gaze cannot be definitively determined. The fixations made during the delay may be
assistive, perhaps covertly re-seeing the stimuli in a manner similar to covertly repeating a list of
items in a verbal recall task. Alternatively the observations made during the delay may be a
corollary behavior, demonstrating a tendency to move one’s eyes, but neither helping nor hurting
accuracy on the matching task. Evidence for the assistive role of eye gaze fixations made during
the delay can be inferred by considering the data across participants, though such a comparison
should be made with caution. Participant 1 and 2 had a lower level of accuracy across all levels
of complexity compared with Participant 3 and 4. Additionally Participant 1 and 2 had a lower
mean frequency and duration of observations to the ROI that displayed an initial sample during
the delay compared with Participant 3 and 4 across all levels of complexity. This may indicate
that making more observations back to the prior locations results in better accuracy. Eye
movements made during the delay may act as a rehearsal mechanism, and individuals who have
EXAMINING EYE GAZE IN A DMTS TASK 43
better accuracy may be better utilizers of this rehearsal strategy. This interpretation would be
consistent with the metaphor of the visuo-spatial sketchpad proposed by Baddeley and
colleagues (Postle et al., 2006; Postle et al., 2005; Baddeley, 1986).
However, when we consider the data within each participant, it is apparent that having a
higher average frequency and/or longer average duration is not necessarily associated with a
greater likelihood of being correct on either matched or unmatched trials. So, while it may be
that individuals with better accuracy are also individuals who happen to fixate more and longer
while engaged in remembering, more and longer fixations do not appear to automatically lead to
better remembering. This is consistent with the results of the study by Godijn and Theeuwes
(2012) who found that, while eye movements did tend to occur during a recall task for a
sequence of visually presented numbers, disrupting the ability for participants to move their eyes
freely did not have a dramatic impact on recall. The interpretation of Awh and colleague was that
it was shifts in attention that aided in memory to visual tasks, and that eye movements were
coincidental with the shifts in attention, rather than being independently necessary (Awh et al.,
1998; Awh & Jonides, 2001).
A final potential interpretation offered here is that it also may also be the case that
individuals simply have a tendency to move their eyes, rather than holding them still. In the
absence of further stimulation during the blank delay screen, fixating back to the location of the
prior samples may be more likely than looking around the room because of the stimulus control
of the computer screen. And a tendency to repeat movements that were recently reinforced may
account for the difference in fixations to locations where stimuli were previously observed,
compared to locations that were blank. In this case, it could be posited that fixating to the
locations of the samples was reinforced by the presence of the samples, and that the behavior of
fixating back to those locations is more likely during the delay as a result. A possible means of
EXAMINING EYE GAZE IN A DMTS TASK 44
evaluating this explanation could be to have a screen briefly presented after the sample screen,
but before the delay, with stimuli that are irrelevant to the task that draw the participant’s
observations, such as colored dots. The patterns of eye movements could then be evaluated to
determine if participants were more likely to look back to the locations of the initially presented
stimuli, or if they were more likely to look back to the locations of the colored dots, the last
places where they may have been reinforced for fixating. No examples of this type of
manipulation were found in the literature reviewed on visual memory.
Future research could also include strategies for interrupting eye movements, such as
requiring participants to fixate on one location during the delay, similar to the procedures
described by Godijn and Theeuwes (2012). Comparisons could be made between free eye
movement conditions and restricted eye movement conditions on the level of accuracy.
Alternatively, low accuracy responders could receive supplemental stimulation to prompt a
greater number and/or duration of eye movements during the delay, and data could be evaluated
to determine whether there is an impact on accuracy. This strategy was utilized by Dube and
colleagues (2006). In their study on eye gaze patterns, during initial observation of a stimulus
array, the authors found that two of their four participants had high accuracy on the DMTS task,
and the other two had lower accuracy, particularly on the four-stimulus trials. Dube and
colleagues found that the high accuracy responders had a more consistent pattern of eye
movements during observation, as well as a longer duration of fixations. As a follow up the
authors used prompting strategies to encourage the low accuracy participants to utilize a similar
pattern of observations, and showed that they were able to increase accuracy following
prompting (Dube et al., 2006). If a similar strategy of prompting eye movements during the delay
back to the regions where stimuli were initially presented resulted in an increase in accuracy, this
may provide additional support to the assistive aspect of eye movements during the delay.
EXAMINING EYE GAZE IN A DMTS TASK 45
Comparison. The pattern of observations made during the comparison demonstrates a less
clear pattern compared with observations made during the delay. While participants did make
observations to the ROI where sample stimuli were initially displayed on two- and four- stimulus
trials, on one-stimulus trials all participants had either the same average number of observations
to regions that did or did not display a stimulus, in the case of Participant 1, or more observations
were allocated on average to the areas of the screen where a stimulus had not been presented, in
the case of the other three participants. The data for Participant 1 on the one-stimulus trials does
indicate that, while the same average number of observations were made to areas that did and did
not display a stimulus, the average duration of fixations was greater for the regions where the
initial sample had been presented. On two-stimulus trials, Participant 1 had no recorded
observations back to any of the ROI during the comparison screen. The other three participants
all made fixations back to the ROI. Participant 2 and 4 both had a higher average frequency to
areas that had presented the initial samples. Participant 2 also had a longer average duration of
observations, whereas Participant 4 displayed a comparable average duration to locations that did
and did not initially display stimuli. Participant 3 had a similar average number and duration of
observations to regions that did and did not display stimuli on the two-stimulus trials. All
participants made the highest average number and longest average duration of fixations on four-
stimulus trials. This is consistent with the increase in latency as the number of sample stimuli
increased.
While the average overall latency to respond increased along with the complexity of the
task for three of the four participants, it was still shorter than the 5s delay interval. The initial
instruction screen indicated for participants to try to be accurate, but also to work quickly. A
future study could place a greater focus on being correct, perhaps by providing feedback on
correct and incorrect responses. With a greater emphasis on accuracy, it may be that participants
EXAMINING EYE GAZE IN A DMTS TASK 46
would exhibit an increased latency to respond. The effect a longer latency might have on the
pattern of eye movements could then be evaluated. To put it in everyday terms, whether a greater
focus on accuracy would lead to participants using more time to “think” about their response, and
whether that “thinking” would include the orderly use of eye gaze fixations prior to making a
selection.
As with the data from the delay, it isn’t possible determine the role of eye movements
during the comparison. Unlike the data from the delay, there isn’t a clear relation between the
average frequency and duration of observation and accuracy. The highest average frequency and
duration of observations made during the comparison were from Participant 2 and Participant 4
on two- and four-stimulus trials. While Participant 1, who had lower accuracy levels on both
two- and four-stimulus trials, had the lowest average number and duration of observations during
the comparison. Participant 3, however, had a high level of accuracy on two-stimulus trials and
the highest level of accuracy on the four-stimulus trials, but demonstrated much lower average
frequency and duration of observations compared with Participant 2 and 4. Considering each
participant’s data individually, they each had the highest average number and longest average
duration of observations on incorrect, unmatched, four-stimulus trials, though in the case of
Participant 3 this includes just one trial.
In considering trials where at least one eye gaze fixation was allocated back to the location
of the correct matched sample (Figure 1), a similar pattern exists across three of the four
participants. Participants 2 and 4 were the most likely to look at least once at the location of the
correct match when viewing the comparison, though neither did so on more that 30% of trials.
Participant 3 looked back at the location where the matched sample had been presented in just
15% of trials, while having some of the highest accuracy scores. For all three of these
participants, they were more likely to look back at the location of the sample on trials that were
EXAMINING EYE GAZE IN A DMTS TASK 47
ultimately correct than incorrect. Participant 1 never looked back to the location of the sample
that matched the comparison.
A confound that may not have been thoroughly controlled for in this study was the use of
verbal behavior. Mandarin characters were selected as stimuli due to their complexity, and it was
presumed that this may impede the tendency to apply labels, or tacts, to the stimuli. However,
participants may still have attempted to apply labels to parts, or the whole, of each stimulus. This
strategy may have played a particular role while viewing the comparison, up to the point of
responding. Future research could utilize both difficult to label stimuli and easy to label stimuli,
such as common objects or written words or numerals, and evaluate the effect that has on eye
movements.
The purpose of Experiment 1 was to evaluate the use of eye movements during the delay in
an increasingly complex object identity DMTS task, and to begin to examine the utility of
utilizing eye gaze measures within a DMTS task to gain access to observable correlates to the
mostly covert behavior of remembering. The increased complexity of the task can be inferred
from the decreasing accuracy as the number of stimuli increased. For three of the four
participants, the latency to respond to the comparison also increased. Across all participants, eye
movements were allocated back to the location where the stimuli were initially located during the
delay, and the average number and duration of eye movements increased as the complexity
increased. Taken together, this provides evidence that the use of eye tracking equipment to
measure eye movements during a DMTS task may provide fruitful data to supplement the
conceptual analysis of a covert behavior, such as remembering. However, the pattern of eye
movements while viewing the comparison did not indicate a strong tendency to use a strategy of
covertly re-seeing stimuli while making a decision on identity. Prior research on visual memory
from the cognitive literature has largely focused on spatial information over object information.
EXAMINING EYE GAZE IN A DMTS TASK 48
Prior to conducting Experiment 1, Experiment 2 was developed as a pilot to gather some
additional data from the already-secured participants.
Experiment 2
Experiment 2 was developed as a pilot study in order to evaluate eye movement patterns
when a different question was asked within an otherwise very similar DMTS task. While the
purpose of Experiment 1 was to extend the research to include a lexical decision making task
based on identity, the purpose of Experiment 2 was to provide a preliminary extension of the
cognitive literature utilizing spatial information by incorporating location information. Similar to
Experiment 1, participants were presented with a DMTS task. However, instead of responding
based on the identity of the comparison to one of the initial samples, participants were asked to
respond with the location of the previously viewed stimulus. Other features of the task, such as
the nature of the stimuli and the timing rules, were kept consistent across Experiments 1 and 2.
Because Experiment 2 was developed as a pilot there were fewer trials both within each trial
type, and overall.
Methods
Participants, Setting, and Materials
The participants, setting, and materials were all the same from Experiment 1 to
experiment 2. As part of gaining consent for Experiment 1, participants were asked if they would
be willing to participate in additional trials. They were informed that their participation in
Experiment 1 did not obligate them to participate in Experiment 2, though all of the participants
did agree to continue. The additional trials were presented immediately after the trials from
Experiment 1 were completed. No additional financial compensation was provided.
The same computer keyboard was utilized, and all four keys that were marked with a
colored sticker were activated. As in Experiment 1, the green and red stickers were placed over
EXAMINING EYE GAZE IN A DMTS TASK 49
the ‘Z’ and ‘/’ keys, though in Experiment 2, they represented the lower left and lower right
quadrants of the screen, respectively. A yellow sticker was placed over the letter ‘W’ to represent
the upper left quadrant, and a blue sticker was placed over the letter ‘P’ for the upper right
quadrant. The spacebar was again used to move the slides forward, and all other keys were made
inactive.
Delayed Match to Sample Task
The same pool of Mandarin characters used for Experiment 1 were utilized in Experiment
2. The stimulus set for each trial was drawn at random from the pool, with the restriction that
stimuli that were matched comparisons in Experiment 1 were excluded. The construction of the
stimulus array was the same in both studies.
In Experiment 2, participants completed 12 two-stimulus trials and 12 four-stimulus trials.
For the two-stimulus trials, each possible combination of corners were utilized, with the correct
location for the comparison occupying each location once. For example, the initial sample screen
presented stimuli in the upper right and upper left corners twice, with the correct comparison
location being the upper right once and upper left once. For the four-stimulus trials, the matched
sample was located in each of the three corners three times. The order of trials was randomized
prior to entry into the Presentation TM software, and remained fixed for all participants.
As was the case in Experiment 1, trials were programmed to start with the initial sample
screen, which remained up until the participant pressed the spacebar on the keyboard. Following
the initial sample, there was a 5 s delay, followed by the comparison screen. The comparison
stimulus was one of the two or four stimuli that had been shown on the sample screen. The
comparison remained on the screen until the participant pressed one of the response keys.
Dependent Variable and Definitions
EXAMINING EYE GAZE IN A DMTS TASK 50
In Experiment 2, correct responding was defined as the participant pressing the color
coded key that corresponded to the location on the sample screen where the comparison was
initially presented. An incorrect response was defined as pressing one of the color-coded keys
that did not correspond with the initial sample location of the comparison. All other visual gaze
definitions were the same from Experiment 1.
Procedure
Following the completion of the experiment 1 trials, participants were offered a break
from the research room before beginning Experiment 2. All of the participants elected to remain
seated, with the eye tracking equipment still calibrated, and immediately moved on to the second
set of trials. The trials for Experiment 2 began with an instruction screen that read:
“In this experiment you will be presented with two or four characters. After
viewing all of the characters press the spacebar. When you press the spacebar
the characters will disappear. After a few seconds a single character will
appear in the middle of the screen. Your task will be to identify the location
where you initially viewed the character. If the character was initially in the
TOP LEFT press the YELLOW button; for the TOP RIGHT press BLUE; for
BOTTOM LEFT press GREEN; and for BOTTOM RIGHT press RED. Press
any key to begin”
Participants were asked to read the instructions aloud, and the session began when the participant
pressed any key on the keyboard.
After pressing one of the keys, the participant began with the two-sample trials. The two-
sample sessions consisted of 12 DMTS trials, immediately followed by 12 four-sample trials.
The participant was presented with the initial sample screen, which they could view for as long
as they wanted. Pressing the spacebar removed the initial stimulus array and initiated the delay
EXAMINING EYE GAZE IN A DMTS TASK 51
interval, which presented a blank screen for 5 s. Following the delay, the comparison stimulus
appeared on the screen and remained on until the participant responded by pressing one of the
colored keys corresponding to the location of the comparison on the initial sample screen. After
one of the keys had been pressed, the screen again went blank for a 5 s intertrial interval. The
next trial then automatically began. Participants did not receive feedback on whether their
response was correct or incorrect.
Results
Because Experiment 2 was developed as a pilot study, there were only 12 presentations of
each trial type. Due to the low number of trials, the calculated means are easily skewed, and all
data should be interpreted with caution.
Accuracy
The accuracy scores on the two- and four-stimulus location DMTS trials is shown in the
first row of Table 7 and 8, respectively. On the two-stimulus trials, Participant 1 made one error,
which translated to 91.7% correct. On the four-stimulus trials, the accuracy for Participant 1
dropped to 66.7%. Participant 2 had the lowest accuracy on the two-stimulus trials at 83.3%, and
the same level of accuracy (83.3%) on the four-stimulus trials. Both Participant 3 and Participant
4 made no errors on the two-stimulus trials, and they each made one error on the four-stimulus
trials.
Latency
Overall latency to respond to the comparison is presented on the second row of Tables 7
and 8 for two-and four-stimulus trials, respectively. Participant 1 responded in 1.876 s to the
comparison on two-stimulus trials and 3.236 s on the four-stimulus trials. Participant 2
responded to the comparison in 1.987s on two-stimulus trials and 2.454 s on four-stimulus trials.
On two-stimulus trials, Participant 3 had a response latency of 1.451 s, and 2.148 s on four-
EXAMINING EYE GAZE IN A DMTS TASK 52
stimulus trials. Finally, Participant 4 had a 2.433 s latency to respond to the comparison on two-
stimulus trials, and 4.640 s latency to respond on the four-stimulus trials.
Latency is further broken down in rows three and four of Tables 7 and 8 to display average
latency on correct and incorrect trials on two- and four-stimulus trials, respectively. On two-
stimulus trials, Participant 1 had a longer latency to respond on correct trials (1.382 s) compared
with incorrect trials (0.622 s). It should be noted that because Participant 1 made only one
incorrect response on the two-stimulus trials, the latency to respond is presented as an average
for correct responses only. Participant 2 had a longer average latency to respond on incorrect
two-stimulus trials (2.928 s) compared with correct trials (1.987 s). Neither Participant 3 nor 4
had any incorrect two-stimulus trials. On four-stimulus trials, Participant 1 responded to the
comparison after an average of 3.501 s for correct trials and 2.706 s for incorrect trials.
Participant 2 responded after an average of 2.339 s for correct trials, compared with 3.028s for
incorrect trials. Participant 3 and 4 each made one error on four-stimulus trials, so the latency for
correct trials is presented as an average; whereas the latency presented for incorrect trials is for
the one trial. On the four stimulus trials, Participant 3 had an average latency of 2.179 s on
correct trials, and took 1.807 s to respond on the incorrect trial. Participant 4 had an average
latency of 4.596 s on correct trials, compared with 5.136 s on the incorrect trial.
Observing Responses
Delay. The data on the mean frequency and duration of observations during the delay on
two-stimulus trials is presented in Table 7. For all participants, there was a much greater average
frequency of fixations to ROI that initially presented a sample compared with the regions that
were blank. Three of the four participants also had a longer mean duration of fixations.
Participant 1 made an average of 8.5 fixations to locations that had presented a sample,
compared with 0.083 to locations that had been blank, with an average duration of 0.434s,
EXAMINING EYE GAZE IN A DMTS TASK 53
compared with 0.003s, respectively. Participant 2 made an average of 11.5 fixations, with an
average duration of 0.527s, to occupied ROI, compared with 1.5 fixations and an average
duration of 0.061s. Participant 3 had the highest mean number of fixations (19.75) during the
delay to ROI where a sample had initially been shown, and the mean duration of fixations was
0.068s. Finally, Participant 4 had an average of 11.917 fixations to the region of interest that
contained a sample, compared with 4.833 fixations to a region that did not have a sample. In
addition to having the smallest difference in mean number of fixations between occupied and
empty corners of the screen, this participant also had a higher average duration of fixations to
previously blank regions compared to regions where a stimulus had been presented.
The data during the delay for four-stimulus trials is shown in Table 8. As with Experiment
1, all ROI presented stimuli, so the same comparison between previously occupied and empty
regions of the screen cannot be made. On the four-stimulus trials, Participant 1 and 2 had a lower
mean frequency of fixations compared with the two-stimulus trials. Participant 1 displayed a
mean frequency of 6.583 fixations, and Participant 2 demonstrated mean frequency of 9.917
observations. Participant 1 had a longer mean duration of observations at 0.313s, compared with
the two-stimulus location trials to locations where the stimuli had been shown. The mean
duration of observations made by Participant 2 was 0.525s on the four stimulus location trials,
which is just over a third of the average duration of observations made by this participant on
similar two-stimulus trials. Participant 3 and 4 both demonstrated a higher mean number of
fixations on the four-stimulus trials compared with the two-stimulus trials. Participant 3 had the
highest mean frequency of fixations at 22.333, though the mean duration of fixations was slightly
shorter than for two-stimulus trials at 1.557s. Participant 4 displayed an average of 17 fixations
to the ROI during the four-stimulus trials, with a mean duration of 1.297s.
EXAMINING EYE GAZE IN A DMTS TASK 54
Table 9 displays the mean frequency of observations for each participant during the delay
and comparison for two- and four-stimulus trials as a function of whether the trial was ultimately
correct or incorrect. The columns present the data for each participant, with sub-columns for
trials that were correct and incorrect. The mean duration of fixations is shown in brackets.
Participant 1 had substantially more fixations to locations where stimuli had been located on
two-stimulus trials that were ultimately correct compared with those that were ultimately
incorrect. On four-stimulus trials, on the other hand, the average number of observations is only
slightly higher for correct than for incorrect trials. Participant 2, on the other hand, had very
similar average numbers of observations on both correct and incorrect two-stimulus trials. On
four-stimulus trials, however, a higher average number and longer average duration of fixations
occurred on incorrect trials. Participant 3 and 4 had no incorrect two-stimulus trials, and one
incorrect trial each on the four-stimulus set. On four-stimulus trials, Participant 3 made on
average almost twice as many fixations to occupied ROI on the incorrect trial compared with the
average number for correct trials. Participant 4 had a much higher average number and duration
of fixations on correct trials compared with the incorrect trial data point.
Comparison. The average frequency and durations of observations made while the
comparison slide was presented, up to the participant’s response is presented in Table 7 and 8 for
two- and four-stimulus trials, respectively. Table 9 also presents data gathered during the
comparison as a function of whether the trial was correct or incorrect. On two-stimulus trials, all
participants made a higher average number and longer average duration of fixations to regions of
the screen where stimuli were initially presented compared with regions that had been blank.
Participant 1 had an average of 0.167 fixations to occupied regions of the screen compared with
0 to previously blank regions. Participant 2 had an average of 1.75 fixations to the occupied ROI
as well as the longest average duration of fixations to those regions at 0.417s. Participant 3 and 4
EXAMINING EYE GAZE IN A DMTS TASK 55
had the highest average frequency of observations to ROI where stimuli had been presented.
Participant 3 made an average of 2.5 fixations to sample presentation regions, compared with
0.167 to blank regions during the comparison slide. Participant 4 had an average of 2.833
fixations to ROI where a sample had been presented, compared with 0.75 to ROI that had been
blank. Both participants also had a longer average duration of fixations to areas where stimuli
had been presented.
Comparison data for four-stimulus trials is shown in Table 8 in the bottom two rows. As
previously mentioned, all regions of the screen displayed stimuli during the sample presentation.
Participant 1 had an identical average number of fixations to the ROI on four-stimulus trials as
on two-stimulus at 0.167. Participant 2 and 3 both had a lower average frequency and duration of
observations on four-stimulus trials compared with the average number of observations to
regions where stimuli were presented on two-stimulus trials. Finally, Participant 4 had a slightly
higher average number of observations, and a slightly shorter average duration of observations,
on four-stimulus trials compared with two-stimulus trials.
Table 9 displays average observation frequency and duration data as a function of whether
the trial was correct or incorrect, with the comparison data presented in the bottom rows for two-
and four-stimulus trials. The average frequency of observations is shown as the top number with
the average duration presented in brackets below. The observations made by Participant 1 during
the comparison on both two- and four-stimulus trials were all made on trials that were correct.
Participant 2 had a higher average number of observations on correct two-stimulus trials
compared with incorrect trials. On four-stimulus trials, however, there was a substantially higher
average number of observations on incorrect trials, though the average duration of observations
was longer on correct trials. Neither Participant 3 nor 4 had incorrect two-stimulus trials. Both
EXAMINING EYE GAZE IN A DMTS TASK 56
had a higher average frequency and duration of observations on correct four-stimulus trials,
though as mentioned previously, there was only one incorrect four-stimulus trial for comparison.
Discussion
The results of Experiment 2 are consistent with the results of Experiment 1 in terms of
accuracy, latency, and observing responses during the delay and comparison. Where differences
emerge is in the numbers for the average frequency and duration of observations made,
especially during the delay, but also the comparison. In considering the observing responses, the
average frequency and duration of fixation made to the locations where stimuli had initially been
presented were higher for some participants compared with the similarly complex trials for
Experiment 1.
Accuracy and Latency
While the nature of the DMTS task differed in that participants were asked to match
location rather than identity, the results of Experiment 2 replicated the results of Experiment 1 in
that the increase in the number of stimuli led to a decrease in the accuracy of responding across
all participants. The change from two-stimulus trials to four-stimulus trials may therefore be
interpreted as being an increase in task complexity. Also consistent between the two studies,
Participant 1 and 2 had lower accuracy scores compared with Participant 3 and 4.
As the task became more complex, a longer average latency to respond was also evidenced
by all four participants. In considering the average latency to respond on trials that were
ultimately correct and incorrect, there was variability. On the two-stimulus location trials,
Participant 1 and 2 made the only errors, though it should be noted that Participant 1 made only a
single error. On that single incorrect trial, Participant 1 had a shorter latency to respond
compared with the average for correct trials. Alternatively, Participant 2 had a longer average
latency to respond on incorrect trials compared with correct trials on both two- and four-stimulus
EXAMINING EYE GAZE IN A DMTS TASK 57
trials. Participants 1, 3, and 4, on the other hand, had a longer average latency to respond on
correct trials compared with incorrect.
Observing responses
Delay. In general, the data on observing responses made during the delay in Experiment
2 are consistent with the results of Experiment 1. In the presence of a blank screen, and therefore
the absence of further stimulation, participants all demonstrated a tendency to fixate back to the
locations where the stimuli were initially presented, rather than holding their eyes still, or
looking at other areas of the screen. On the two-stimulus location trials, all participants had a
substantially higher average number of fixations to locations where the stimuli had been initially
presented compared with the areas that had been blank. The average number of fixations to
locations where samples were presented was higher in the location trials compared with the
identity trials from Experiment 1 for Participants 1, 2, and 4. Participant 4 had a higher average
number of observations on the two-stimulus trials in Experiment 1 compared with Experiment 2.
Also, the difference in the average number of observations to regions with samples, compared to
blank regions, was much greater for Participants 1, 2, and 3 than for the comparable two-
stimulus trials from Experiment 1. Participant 4, on the other hand, had a very similar level of
difference in the average number of fixations to regions where sample had been presented and
regions that were blank across Experiments 1 and 2. On the four-stimulus trials, Participants 1
and 2 had a lower average frequency of observations to the four regions of the screen, all of
which presented stimuli on the sample screen. Participant 2 had a similar average duration of
observations to regions that displayed stimuli, where Participant 1 also had a lower average
duration of fixations. This is in contrast to Participants 3 and 4 who both had a higher average
number and duration of fixation on the four-stimulus trials. It should be noted, however, that the
average frequency of observations made by Participant 4 on the four-stimulus trials is roughly
EXAMINING EYE GAZE IN A DMTS TASK 58
equal to the average frequency of fixations to locations that did and did not display stimuli on the
two-stimulus trials. The higher average frequency of fixations seen in the four-stimulus trials
may, therefore, be somewhat artificial for Participant 4.
Comparison. The data from Experiment 2 on the observing responses during the
comparison are also consistent with the results from Experiment 1. There was a far lower
average number and duration of fixations made during the comparison compared with the delay.
On both the two-stimulus and four-stimulus trials, all of the participants had a higher average
frequency of fixations to the locations where the samples were initially presented on the location
trials compared with the identity trials from Experiment 1. Participants 2, 3, and 4 also had a
greater discrepancy in average frequency between locations that did and did not display an initial
sample in the two-stimulus trials in Experiment 2. At both levels of complexity, Participants 1
and 2 had a lower average frequency and duration of observations during the comparison
compared with Participants 3 and 4, along with lower accuracy.
As with Experiment 1 the role of fixations made during the delay and comparison cannot
be determined from these data. Similar future manipulations may provide greater clarity on the
assistive versus coincidental role of eye movements during a DMTS task. The similarity in
results between Experiments 1 and 2 may call into question the distinction between systems for
spatial versus object memory posited by Baddeley and colleagues (Baddeley, 1986; Postle et al.,
2006). While the average number of observations was higher in most instances in the location
DMTS task compared with the identity task, there were also fewer trials. Further replication with
more trials would allow for an even more direct comparison between the two types of tasks.
General Discussion
Taken together, Experiments 1 and 2 provide some initial suggestions regarding the utility
of using eye gaze as a dependent variable to begin to examine the otherwise covert phenomenon
EXAMINING EYE GAZE IN A DMTS TASK 59
of visual memory. The data from both studies indicate that, even when there is no additional
stimulation to be gained by doing so, individuals tend to allocate eye movements to screen
locations where sample stimuli were initially presented. While the data from these two studies
are indicative of the occurrence of this tendency, no firm conclusions can be drawn about the
specific role, or relative importance, of eye movements in either an identity or location-based
DMTS task. However, the procedures and the measures open avenues for future research
examining behavior analytic accounts of the complex human behavior of remembering.
Utility of Eye Tracking in a DMTS Task
Utility of DMTS task. This study extends previous cognitive research on visual working
memory by utilizing a DMTS task with difficult to name stimuli. While the use of a DMTS task
incorporating nonsense stimuli is common in behavior analytic research, particularly related to
stimulus control, few examples were found in the cognitive literature that utilized a similar
research paradigm. The studies by Postle and colleagues (Postle, et al., 2006; Postle, et al., 2005)
provided the closest comparison in their use of difficult to name stimuli in a delayed recognition
task. The studies by Postle and colleagues utilized shapes that had been shown to have a low
level of prior association. In the object recognition tasks, the authors required a yes/no decision
as to whether a comparison object matched the initial sample after a delay. In contrast to the
current study Postle and colleagues (Postle, et al., 2006; Postle, et al., 2005) presented only one
stimulus as a sample, and the primary dependent variable they measured was accuracy following
different distraction tasks. The distraction tasks were designed to test the presence of different
visual memory sub-systems by selectively interrupting either object or spatial visual working
memory. This was accomplished by introducing either a verbal task to disrupt object recall, or a
movement task to disrupt spatial recall, along with a free eye movement control condition. Eye
movements were tracked in order to ensure the integrity of the movement disruption task, rather
EXAMINING EYE GAZE IN A DMTS TASK 60
than as a dependent variable in its own right (Postle et al., 2006). Also, in contrast to the current
study, Postle and colleagues (2005) changed the nature and complexity of the verbal distraction
task, rather than altering the complexity of the recall task itself, in order to control for the greater
difficulty of the verbal task.
The use of difficult to name stimuli, with which participants have a minimal prior history,
may offer some advantages over the use of other stimulus types, such as numbers (Godijn &
Theeuwes, 2012) or animals (Laeng, et al., 2014). One advantage is that it may reduce a source
of variability across trials, and across participants, allowing for comparisons to be made. When
participants have a prior history with the stimuli being used, performance on the task may be
variable based on that prior history, rather than being under the control of the task itself. Another
advantage to using difficult to name stimuli may be in minimizing the opportunity to utilize
verbal strategies when examining nonverbal remembering. The purpose of the current study was
to begin to examine observable behaviors that may occur in conjunction with covert behaviors
involved with remembering. By minimizing the opportunity to use verbal strategies, including
tacting the initial sample, and then echoing the tacts as a rehearsal strategy, it may be possible to
gain insight into how similar performances happen in our everyday experience.
The use of the DMTS task itself may also offer some advantages over other experimental
manipulations that were found in the literature. For example, the study by Tremblay and
colleagues (2006) utilized eye movements as a dependent variable that they correlated with
correct recollection of the order of presentation of a set of seven dots. However, the dots
remained on the screen throughout the delay. The authors found that recall for dot pairs that were
rehearsed, as indicated by eye movements, was somewhat better than for unrehearsed pairs
(Tremblay et al., 2006). What cannot be determined is what the performance might have been if
there was no further stimulation to be gained by looking at the screen. As with minimizing verbal
EXAMINING EYE GAZE IN A DMTS TASK 61
behavior associated with remembering visually presented information, examining behavior in the
absence of stimuli may provide a better analog to some everyday experiences where we respond
regarding something that is no longer present. The structure of the present DMTS task would
allow for future studies to evaluate differences in performance when stimuli are left up during
the delay, as in the study by Tremblay and colleagues (2006), compared to when the delay
interval is blank. Also, the length of the delay can be easily manipulated to evaluate any
differences between shorter, longer, and even unpredictable delay lengths.
The current study replicated research conducted by Dube and colleagues (2006) by
demonstrating an ability to increase the complexity of a DMTS task by increasing the number of
samples initially presented. Dube and colleagues found that increasing the number of samples led
to a corresponding decrease in accuracy in both Experiment 1 and Experiment 2. An additional
measure that may indicate an increase in task complexity was an increase in latency to respond to
the comparison as the number of initial samples increased, which was demonstrated by three of
the four participants in the current study. The increase in latency may be a more sensitive
measure than accuracy for this type of DMTS task, as it may be less vulnerable to a ceiling
effect. In the current study, increasing the level of complexity allowed for the comparison of eye
gaze measures in increasingly difficult tasks. Another comparison afforded by the current DMTS
task was in the allocation of eye movements to areas of the screen where stimuli had been
initially presented compared with areas that were blank on the one-stimulus and two-stimulus
trials. This type of demonstration was not found in any of the reviewed literature on visual
memory. Future studies could manipulate the complexity level further by including a greater
number of initial samples, or placing the samples in less predictable locations.
Utility of eye tracking measures. The current study also extends the research conducted
by Dube, and colleagues (2006) by providing further demonstration of the utility of eye gaze
EXAMINING EYE GAZE IN A DMTS TASK 62
measures as a dependent variable. In the current study, the dependent variables provided by the
eye tracking equipment included the number and duration of fixations. Future studies could
incorporate other measures, including the pattern of fixations. Much of the research on visual
memory has come out of the domain of cognitive psychology. With few exceptions (Bochynska
& Laeng, 2015; Laeng et al., 2014; Tremblay et al., 2006), the use of eye gaze measures has been
restricted to ensuring the integrity of the independent variable (Godijn & Theeuwes, 2012; Postle
et al., 2006; Postle et al., 2005). In constructing their tasks the authors (Godijn & Theeuwes,
2012; Postle et al., 2006; Postle et al., 2005) started with a hypothesis regarding the nature of
visual working memory, and then sought to validate that hypothesis by providing interfering
tasks and demonstrating a disruption in accuracy. With this framework, the use of eye gaze
measures as a dependent variable is unnecessary. Alternatively, behavior analytic research takes
an inductive, approach to questions about all forms of behavior, including complex processes,
such as remembering. This approach necessitates the use of direct measures of the behavior of
interest, rather than correlates that are hypothesized to demonstrate inner structures and
processes. Thus, in order to develop hypotheses about the role of eye movements in a delayed
recall task, it is necessary to measure the eye movements themselves.
Using eye tracking equipment, it was possible to measure sensitively a behavior that would
otherwise not be available as a dependent variable, and as the technology continues to improve,
the measurement will likely also become more sensitive. This may allow researchers to extend
the conceptual analysis of complex human behavior by demonstrating aspects of complex,
mostly covert, behavior chains that are consistent with the experimentally derived principles of
behavior (Palmer, 2010). While the current study does not allow for a determination to be made
about the role of eye movements, it does demonstrate that such questions may be amendable to
the experimental methods of behavior analysis. As a behavior, eye movement is a free operant,
EXAMINING EYE GAZE IN A DMTS TASK 63
and an individual’s eye gaze behaviors may happen mostly without the interference of rule-
governed behavior. However, it is also a behavior that may be brought under the control of rules,
for instance by requiring that the eyes be held still (Godijn & Theeuwes, 2012), or that they
move in either a random way (Postle et al., 2005), or in a specified way (Postle et al., 2006).
Future studies should alternate the use of free and restricted eye movement conditions in a single
subject research design to gain greater insight on how the use of eye movements might relate
functionally to the behaviors involved in remembering. The functional roles for eye gaze could
include acting as a rehearsal mechanism, consistent with the notion of covert seeing, as was
suggested by Skinner (1953). The same behaviors that occurred as part of seeing are conditioned
to occur at a covert level under circumstances where doing so leads to reinforcement in the form
of improved accuracy to the comparison. Alternatively, it may be that eye movements are
coincidental with whatever covert behaviors are occurring, but that their presence neither assists
nor detracts from the ultimate performance on the DMTS task. For instance, it may be that there
is simply a tendency to repeat behaviors that have recently been reinforced, with the reinforcer in
this case being opportunity to observe the initial samples. The eye movements may then persist
in the absence of further reinforcement, at least until the comparison stimulus is presented. One
way of evaluating this hypothesis may be to look at the pattern of eye movements as the delay
progresses. If eye movements during the delay are the result of reinforcement during the initial
sample, then perhaps the rate at which this behavior occurs might change in an orderly way as
the delay progresses. And finally, it may be that eye gaze fixations during a DMTS task serve
different functions for different individuals, or in different situations.
Evaluation of Eye Gaze in the DMTS Task
Delay. In both Experiments 1 and 2, all participants fixated back to the areas of the
screen where samples had initially been presented during the delay. This is despite the fact that
EXAMINING EYE GAZE IN A DMTS TASK 64
during the delay there was no further stimulation to be gained by looking back to these regions.
On the one-stimulus and two-stimulus identity trials, and on the two-stimulus location trials, it is
possible to compare the number and duration of fixations to locations where samples had initially
been presented compared with the regions that had been blank. On these trials, all participants
made more observations, for a longer total average duration, to the regions where the sample(s)
had been located compared with the number of fixations to the regions that had been blank. A
further comparison can be made in considering the pattern as the complexity increased. Across
Experiments 1 and 2, with few exceptions, the number of fixations increased as the number of
stimuli increased, and the highest average number of fixations occurred during the delay on the
four-stimulus trials. A final comparison, made with caution, is between participants. Two of the
participants consistently had a higher level of accuracy, and two had a lower level of accuracy.
Overall, a higher average frequency and average total duration of observations was displayed by
the high accuracy responders compared with the low accuracy responders.
In contrast to these data trends, however, in both Experiment 1 and Experiment 2, correct
trials were not associated with the highest average frequency or duration of fixations during the
delay compared with incorrect trials. So simply looking back to regions where stimuli were
presented during the delay did not lead in an automatic way to be a greater likelihood of being
correct on any given trial.
Comparison. The data on fixations during the comparison slide demonstrated greater
variability both within and across participants in their tendency to fixate back to regions where
stimuli were initially presented. Comparing the results of Experiment 1 and Experiment 2, there
were a higher number of fixations during the comparison when the task was to indicate the initial
location of a match as opposed to the identity of a match. As with the data from the delay, the
increase in complexity was also associated with an increase in the mean frequency and duration
EXAMINING EYE GAZE IN A DMTS TASK 65
of fixations in most instances, though to a much smaller degree. There was also a greater average
number of fixations demonstrated by the high accuracy responders compared with the low
accuracy responders, though again to a much smaller degree. As with the data from the delay
interval, there was variability on whether a higher frequency of observations during the
comparison was associated with correct or incorrect trials.
Behavioral Explanations
The results of this study do not either rule in or rule out any particular cognitive theory of
visual working memory, or even the presence of a visual working memory system in general.
The purpose of the study was to utilize direct measures of behavior that occurred during a task
that involved responding to a stimulus in its absence, and possible explanations for the data will
largely be restricted to accounts that are consistent with empirically validated principles of
behavior. The principle of determinism applied to behavior indicates that behavior happens for a
reason. Behaviors occur in the presence of discriminative stimuli that indicate the availability of
a reinforcer that is currently motivating, and continue to occur in similar circumstances because
of a history of reinforcement. There are several possible explanations for these data that are
consistent with the principles of behavior, based primarily on differing sources of reinforcement
as well as discriminative stimuli.
Covert seeing. The eye movements may function to assist the individual in responding to a
stimulus that is no longer present as a rehearsal strategy. This would be consistent with a
behavioral interpretation of visual remembering that proposes the possibility of covertly re-
seeing the stimuli, in a manner similar to the covert repetition of verbal stimuli. As opposed to
the cognitive theory of a visuo-spatial sketchpad (Baddeley, 1986), this interpretation does not
require a hypothetical construct that is responsible for the strategy. Instead, seeing is considered
to be a behavior, and as a behavior it may occur to a small enough degree to be considered
EXAMINING EYE GAZE IN A DMTS TASK 66
covert, if engaging in the behavior is reinforced. In this case, the reinforcement would be the
ability to respond to the comparison. There is not, in this behavioral interpretation, a version of
the stimulus that is being viewed and manipulated, but rather, a tendency to engage in similar
behaviors covertly that would lead to reinforcement if done overtly while in the presence of the
stimuli, based on prior learning history
Transitive conditioned motivating operation. A related explanation is that the data may
be consistent with what would be expected if the participants engaged in a behavior chain that
led them to respond to the comparison after the delay. The presentation of the initial sample sets
the occasion for the behavior of remembering that then makes supplemental stimuli
reinforcing. As part of problem solving, individuals may then look for supplemental stimuli,
even in their absence of the stimuli. In this case, the regions of the screen where stimuli were
initially presented may be considered to be a transitive conditioned motivating operation (CMO-
T), where their value as a reinforcer for the behavior of looking is momentarily increased as part
of a behavior chain that terminates with a response to the comparison (Michael, 1993). This does
not necessarily imply that the behaviors are required to reach the terminal response in the chain,
though it also doesn’t rule that out either. That the regions may function as CMO-Ts, and
allocating eye movements to the regions is momentarily more likely because of their place in the
chain, would be consistent with the hypothesis of covert seeing described above, but may also
stand alone as its own behavioral explanation for the data. In this case, the movements may have
been conditioned as part of the larger chain without being necessary to the chain. A disruption in
the chain, by restricting eye movements in some way, may then result in a longer latency to
respond, but not necessarily a disruption in response accuracy.
Behavioral repetition. A final alternative explanation offered here may be that in viewing
the initial sample screen, more stimuli require more eye movements, and it is that tendency that
EXAMINING EYE GAZE IN A DMTS TASK 67
is being repeated, to a smaller extent, during the delay. Eye movements and fixations are
behaviors associated with initially observing the stimuli, and may be reinforced by the
opportunity to observe the sample(s). Behaviors that have been recently reinforced have a
tendency to be repeated. During the delay, it may that the behaviors are simply being repeated
due to their reinforcement history, and due to a more general tendency to engage in some sort of
behavior during a delay. As was suggested earlier in this manuscript one possibility for
evaluating this explanation may be to look at eye movement patterns as the delay progresses.
Further research utilizing eye tracking equipment during a DMTS task is necessary to further
define the role of eye movements during the delay.
Future research
Given that the purpose of the current research was to examine visual remembering by
utilizing eye gaze measures as a dependent variable in a DMTS task, one of the primary
outcomes of the current study is the identification of questions for future research. There are
several possibilities for an extension of the current research to address both limitations of the
current study, and to further explore some of the questions raised. As an initial suggestion, future
studies should look to utilize research designs that allow for functional relations to be drawn.
Many of the suggestions made below may be amendable to the use of some variation of a
reversal or multielement design.
The nature of the DMTS task allows for several manipulations to refine our understanding
of when and how eye gaze behaviors occur during a DMTS task. For instance, the delay interval
could be lengthened to assess what impact that would have on the same eye measures. It may be
that the use of eye gaze fixations increases as the delay increases in order to maintain the
response. A longer delay may also better allow for a closer examination of how eye gaze
behaviors change as the interval progresses. Additionally, in the real world, we often cannot
EXAMINING EYE GAZE IN A DMTS TASK 68
predict how long of an interval there will be between the observation of stimulus and the need to
respond based on that observation. Future studies could examine eye gaze patterns when the
length of the delay is unpredictable.
The current study attempted to control for verbal behavior by utilizing Mandarin language
characters. Mandarin characters were selected due to their relative complexity. The goal of using
complex stimuli was that it would make them difficult to name, thereby reducing the ability to
use verbal strategies during the DMTS task. However, the participants may still have behaved
verbally by assigning labels to parts, or the whole, of each character, and then engaged in covert
verbal rehearsal instead of, or along with, visual strategies. Because of the individual nature of
learning histories, it may not be possible to fully control for the use of the verbal behavior, that
may or may not occur, in this task. Studies from the cognitive domain have utilized different
interfering tasks during the delay in order to confound the use of verbal behavior that may or
may not be emitted during the delay (Postle, et al., 2005; Postle, et al., 2006). In these studies,
the effect of the distraction task was measured indirectly by looking at accuracy, rather than
considering the impact directly on eye movements. A further confound inherent in this
methodology is that the distraction task may simply mask the stimulus control that allows an
individual to respond to a visual stimulus after a delay, rather than interfering with a specific
covert response. An alternative way of evaluating the question may be to compare difficult to
name stimuli with easy to name stimuli and consider any differences in the participant’s use of
eye gaze. The easy to name stimuli could include common objects or written words. Or, in order
to minimize prior learning history, labels could be developed and taught for a subset of the
characters. Then, using either a reversal design between difficult and easy to name stimuli, or a
multielement design between the two stimulus categories, the impact on eye gaze measures could
be evaluated.
EXAMINING EYE GAZE IN A DMTS TASK 69
Another avenue for future research could present a single stimulus, and record the scan
path, along with any other salient patterns, as the individual initially observes the stimulus.
Correspondence between the initial observation patterns and the eye gaze patterns towards a
matched or unmatched comparison could then be evaluated. It could be that similar viewing
patterns are associated with correctly identifying a match, or that a divergence in viewing
patterns results in incorrectly identifying a match as unmatched. This result could lend support to
the covert seeing hypothesis, that the same behaviors that led to successful initial observation are
repeated in order to respond correctly after the delay.
Research on visual memory could be extended in future research using the DMTS
paradigm, by interrupting the participant’s eye movements during the delay. The participant
could be required to hold their eyes fixated on a point presented on the delay screen. This
manipulation could be presented in a reversal design with conditions where the participant can
move their eyes freely. The use of forced movements could also be used instead of forced
fixation, similar to what was demonstrated in th third experimental question in the collection of
studies presented by Postle, and colleagues (2006). Correlations between accuracy and the free
or restricted use of eye movements may add support to the hypothesis that eye movements play
an assistive role in visual memory tasks.
In order to explore the hypothesis that eye movements during the delay are due to repetition
of recently reinforced behavior, rather than being functionally related to accurate responding on
the DMTS task, an additional screen with task irrelevant stimuli, such as simple dots, could be
presented immediately after the sample screen, before the delay, in screen locations that are
randomly assigned. Eye movements could then be evaluated for whether fixations occurs more
frequently, or for a longer duration, toward the locations most recently observed, or towards the
EXAMINING EYE GAZE IN A DMTS TASK 70
locations where the relevant samples had been presented. This information could be correlated
with accuracy similar to the current study.
Not surprisingly, there were differences in accuracy across participants on the DMTS task,
particularly as it became more complex. The differences in accuracy were also associated with
differences in eye gaze fixations, particularly during the delay. An extension of the research
conducted by Dube, and colleagues (2005) could include providing some supplemental
stimulation for low accuracy responders during the delay to prompt their eye movement patterns
to more closely match the patterns displayed by the high accuracy responders. Following
training, the continued use of similar eye gaze patterns in the absence of prompting could be
evaluated, along with its impact on accuracy. Another avenue for changing the performance of
the participants, particularly those with lower accuracy scores, could be to provide feedback on
correct responding, and perhaps to provide additional reinforcement contingent on correct
responding.
Should further research continue to suggest an assistive role for eye movements during the
delay, and while viewing the comparison, this may provide insight on teaching strategies for
individuals who struggle to respond to visually presented material after a delay. For instance,
individuals with a developmental delay or dementia may be able to be taught to utilize shifts in
eye gaze to actively maintain the behavior of looking until the opportunity to respond.
Limitations
There are several limitations to the current study. A primary limitation is the lack of a
strong functional relation demonstrated by the results. This is in part due to the structure of the
current study, which can be addressed in future research, and in part it is due to the nature of the
problem. There may always be a point at which the overt becomes the covert, and is no longer
accessible to measurement. However, this should not discourage future research into complex
EXAMINING EYE GAZE IN A DMTS TASK 71
human behavior such as remembering, as additional evidence will add to our ability to make
interpretations regarding covert behavior based on empirically derived information, rather than
relying on speculation. The current study does not rule out cognitive explanations for
remembering in favor of a behavioral explanation. Nor does it rule out or in any particular theory
of visual memory.
With regards to improving the functional relations demonstrated in future studies, single-
subject research designs, such as reversal and multielement designs, could be utilized to measure
the impact of the chosen independent variable on eye gaze and accuracy in a DMTS task. The
current study replicated the design of Dube, and colleagues (2006), and the purpose of the two
studies were very similar, which was to demonstrate the utility of the eye tracking equipment and
eye gaze measures to begin to examine an otherwise largely covert phenomenon. The internal
validity of the study could be strengthened by conducting systematic replications, using a
research design that allows for greater confidence in the results, as well as considering more
rigorous single-subject designs when examining future research questions. For example,
conditions that allow for, and restrict the use of, eye movements during the delay could be placed
into a reversal design, as could varying delay lengths, or comparing easy and difficult to name
stimuli. Alternatively, these same variables could be compared using a multielement design.
A related limitation of the current study is the small number of trials and participants,
limiting the ability to use the tools of statistics to achieve greater confidence in the results. The
use of single-subject research is one of the hallmarks of behavior analysis, and may continue to
be the most appropriate, given that the function of eye movements during the delay may differ
among participants. However, by including a greater number of trials to allow for the use of
statistics we may gain further insight into the nature of the role of eye movements in a DMTS
task, as well as greater confidence in the robustness of the results.
EXAMINING EYE GAZE IN A DMTS TASK 72
An area for future research that has already been suggested is to evaluate performance in
the presence of easy and difficult to name stimuli in order to control for some of the impact of
verbal behavior. Participant verbal behavior during the task is another limitation of the current
study. That participants may have engaged in some covert verbal behavior while completing the
task is likely, given the learning history that most of us have in using this type of rehearsal
strategy. The tendency to engage in rehearsal behavior increases the challenge of determining the
functional role of eye movements that were measured during the DMTS task.
A final limitation of the current research is the relative sensitivity of the equipment. The
eye tracking equipment used in initial studies by Baddeley and colleagues (reported in Postle et
al., 2006) were described as being of low sensitivity. The fourth experiment reported in the
collection of studies was completed, in part, to utilize more sensitive measures. As is the case
with many other technologies, we should expect that the sensitivity of eye tracking equipment,
and the resulting measures, will become more sensitive over time. It is difficult to predict how
improved sensitivity will change our understanding of the use of eye gaze during a DMTS task;
however this should not prevent future research from seeking out improved equipment and the
data they generate.
Conclusion
In conclusion, the results of this study contribute to the literature on complex human
behavior, and provide some suggestions on how stimulus control might operate over a delay
interval. The results indicated that individuals tend to allocate eye gaze fixations to the locations
where samples were initially presented during the delay, and that the pattern of these fixations
increased in a somewhat orderly way as the number of initial samples increased. These findings
extend the research of Dube, and colleagues (2006) by utilizing eye tracking equipment in a
DMTS task, and examining eye gaze measures during the delay and comparison, rather than the
EXAMINING EYE GAZE IN A DMTS TASK 73
initial observation. Additionally, the present research presented a slightly different DMTS task,
which extended the cognitive literature on visual memory (Postle et al., 2006; Postle et al.,
2005), by requiring the participant to identify whether the comparison matched one of the
samples, as well as identifying the initial location of the comparison. While the results do not
provide definitive answers about the behavior of remembering, they do indicate that this research
paradigm may allow us to further a research agenda into complex, often covert, human behavior.
By doing so we may enhance our understanding of our own behavior, as well as allow us to
develop techniques to help those who may struggle to perform tasks that rely on responding to
visually presented material after a delay.
References
Atkinson, R. C., & Shiffrin, R. M. (1968). The Psychology of Learning and Motivation: II.
Oxford England: Academic Press. doi: 10.1016/S0079-7421(08)60422-3
Attneave, F. & Arnoult, M.D., (1956) The quantitative study of shape and pattern perception.
Psychological Bulletin, 53(6), 452-471.
http://dx.doi.org.ezproxy.simmons.edu:2048/10.1037/h0044049
EXAMINING EYE GAZE IN A DMTS TASK 74
Awh, E., & Jonides, J. (2001). Overlapping mechanisms of attention and spatial working
memory. Trends in Cognitive Sciences, 5(3), 119-126. doi: 10.1016/S1364-6613(00)01593-
X
Awh, E., Jonides, J., & Reuter-Lorenz, P. (1998). Rehearsal in spatial working memory. Journal
of Experimental Psychology: Human Perception and Performance, 24(3), 780-790. doi:
10.1037/0096-1523.24.3.780
Baddeley, A. (1986). Working memory. New York, NY US: Clarendon Press/Oxford University
Press.
Baddeley, A. (2000). The episodic buffer: A new component of working memory? Trends in
Cognitive Sciences, 4(11), 417-423. doi: 10.1016/S1364-6613(00)01538-2
Barnes-Holmes, D., Staunton, C., Whelan, R., Barnes-Holmes, Y., Commins, S., Walsh, D.,
Stewart, I, Smeets, P.M, & Dymond, S. (2005). Derived stimulus relations, semantic
priming, and event-related potentials: Testing a behavioral theory of semantic networks.
Journal of the Experimental Analysis of Behavior, 84(3), 417-433.
doi:10.1901/jeab.2005.78-04
Baum, W. M. (2005). Understanding behaviorism: Behavior, culture, and evolution (2nd ed.).
Malden: Blackwell Publishing.
Baum, W. M. (2011). Behaviorism, private events, and the molar view of behavior. The
Behavior Analyst, 34(2), 185-200.
EXAMINING EYE GAZE IN A DMTS TASK 75
Bochynska, A., & Laeng, B. (2015). Tracking down the path of memory: Eye scanpaths facilitate
retrieval of visuospatial information. Cognitive Processing, 16(1), 159-163.
doi:10.1007/s10339-015-0690-0
Brooks, L. R. (1967). The suppression of visualization by reading. The Quarterly Journal of
Experimental Psychology, 19(4), 289-299. doi: 10.1080/14640746708400105
Della Sala, S., Gray, C., Baddeley, A., Allamano, N., & Wilson, L. (1999). Pattern span: A tool
for unwelding visuo-spatial memory. Neuropsychologia, 37(10), 1189-1199. doi:
10.1016/S0028-3932(98)00159-6
Dinsmoor, J. A. (1985). The role of observing and attention in establishing stimulus control.
Journal of the Experimental Analysis of Behavior, 43(3), 365-381. doi:
10.1901/jeab.1985.43-365
Donahoe, J. W., & Palmer, D. C. (2004). Learning and complex behavior. Needham Heights,
MA US: Allyn & Bacon.
Dube, William V.; Balsamo, Lyn M.; Fowler, Thomas R.; Dickson, Chata A.; Lombard, Kristin
M.; and Tomanari, Gerson Y. (2006) Observing Behavior Topography in Delayed
Matching to Multiple Samples. The Psychological Record, 56(2), 233-244. Available at:
http://opensiuc.lib.siu.edu/tpr/vol56/iss2/6
Farmer, E. W., Berman, J. V., & Fletcher, Y. L. (1986). Evidence for a visuo-spatial scratch-pad
in working memory. The Quarterly Journal of Experimental Psychology: Human
Experimental Psychology, 38, 675-688. doi: 10.1080/14640748608401620
EXAMINING EYE GAZE IN A DMTS TASK 76
Godijn, R., & Theeuweses, J. (2012). Overt is no better than covert when rehearsing visuospatial
information in working memory. Memory & Cognition, 40(1), 52-61. doi: 10.3758/s13421-
011-0132-x
Haimson, B., Wilkinson, K. M., Rosenquist, C., Ouimet, C., & McIlvane, W. J. (2009).
Electrophysiological correlates of stimulus equivalence processes. Journal of the
Experimental Analysis of Behavior, 92(2), 245-256. doi: 10.1901/jeab.2009.92-245
Hayes, S. C., & Bissett, R. T. (1998). Derived stimulus relations produce mediated and episodic
priming. The Psychological Record, 48(4), 617-630.
Hitch, G. J., & Baddeley, A. D. (1976). Verbal reasoning and working memory. The Quarterly
Journal of Experimental Psychology, 28(4), 603-621. doi: 10.1080/14640747608400587
Howard, M. W., & Kahana, M. J. (2002). When does semantic similarity help episodic retrieval?
Journal of Memory and Language, 46(1), 85-98. doi: 10.1006/jmla.2001.2798
Laeng, B., Bloem, I.M., D’ascenzo, S., & Tommasi, L. (2014). Scrutinizing visual images: The
role of gaze in mental imagery and memory. Cognition, 131(2), 263-283. doi:
10.1016/j.cognition.2014.01.003
Lawrence, B. M., Myerson, J., Oonk, H. M., & Abrams, R. A. (2001). The effects of eye and
limb movements on working memory. Memory, 9(4-6), 433-444. doi:
10.1080/09658210143000047
EXAMINING EYE GAZE IN A DMTS TASK 77
Leigland, S. (1992). Radical Behaviorism: Willard Day on Psychology and Philosophy. Reno:
Context Press.
Logie, R. H. (1995). Visuo-spatial working memory. Hillsdale, NJ England: Lawrence Erlbaum
Associates, Inc.
Michael, J. (1993). Establish Operations. The Behavior Analyst 16(2), 191-206.
Palmer, D. C. (1991). A behavioral interpretation of memory. In L. J. Hayes, & P. N. Chase
(Eds.), (pp. 261-279). Reno, NV US: Context Press.
Palmer, D. C. (2010). Behavior under the microscope: Increasing the resolution of our
experimental procedures. Behavior Analyst, 33(1), 37-45.
Palmer, D. C., & Donahoe, J. W. (1992). Essentialism and selectionism in cognitive science and
behavior analysis. American Psychologist, 47(11), 1344-1358. doi: 10.1037/0003-
066X.47.11.1344
Postle, B. R., Idzikowski, C., Sala, S. D., Logie, R. H., & Baddeley, A. D. (2006). The selective
disruption of spatial working memory by eye movements. The Quarterly Journal of
Experimental Psychology, 59(1), 100-120. doi: 10.1080/17470210500151410
Postle, B.R., D’Esposito, M. (1999). ‘What’—then—‘where’ in visual working memory: An
event-related fMRI study. Journal of Cognitive Neuroscience, 11(6), 585-597. doi:
10.1162/089892999563652
EXAMINING EYE GAZE IN A DMTS TASK 78
Postle, B.R., D’Esposito, M., & Corkin, S. (2005). Effects of verbal and nonverbal interference
on spatial and object visual working memory. Memory & Cognition, 33 (2), 203-212.
doi:10.3758/BF03195309
Saint-Aubin, J., Tremblay, S., & Jalbert, A. (2007). Eye movements and serial memory for
visual-spatial information: Does time spent fixating contribute to recall? Experimental
Psychology, 54(4), 264-272. doi: 10.1027/1618-3169.54.4.264
Skinner, B. F. (1953). Science and human behavior. Oxford England: Macmillan.
Skinner, B. F. (1974). About behaviorism. Oxford England: Alfred A. Knopf.
Skinner, B. F. (1969). Contingencies of reinforcement. East Norwalk, CT US: Appleton-Century-Crofts.
Skinner, B. F. (1988). The operational analysis of psychological terms. In A. C. Catania, & S.
Harnad (Eds.), (pp. 150-217). New York, NY US: Cambridge University Press.
Smyth, M. M., & Scholey, K. A. (1994). Interference in immediate spatial memory. Memory &
Cognition, 22(1), 1-13. doi: 10.3758/BF03202756
Tremblay, S., & Saint-Aubin, J. (2009). Evidence of anticipatory eye movements in the spatial-
Hebb repetition effect: Insights for modeling sequence learning. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 35(5), 1256-1265. doi: 10.1037/a0016566
Tremblay, S., Saint-Aubin, J., & Jalbert, A. (2006). Rehearsal in serial memory for visual-spatial
information: Evidence from eye movements. Psychonomic Bulletin & Review, 13(3), 452-
457. doi: 10.3758/BF03193869
EXAMINING EYE GAZE IN A DMTS TASK 79
Vanderplas, J.M., & Garvin, E.A. (1959). The association value of random shapes. Journal of
Experimental Psychology, 57(3), 147-154. doi: 10.1037/h0048723
Watson, J.B. (1913). Psychology as the behaviorist views it. Psychological Review, 20, 158-177.
Watson, J. B. (1920). Is Thinking Merely the Action of Language Mechanisms? British Journal
of Psychology, 11, 87-104. Online: http://psychclassics.yorku.ca/Watson/thinking.htm
Running head: EXAMINING EYE GAZE IN A DMTS TASK
80
Table 1
One-stimulus trial data
Participant 1 2 3 4
Overall accuracy (%) 87.5 100 100 100
Accuracy on matched trials (%) 100 100 100 100
Accuracy on unmatched trials (%) 66.7 100 100 100
Mean frequency of fixations during the delay
ROI that DID contain a sample 4.875 (5.793)
2.875 (2.800)
5.375 (4.534)
4.75 ( 2.712)
ROI that DID NOT contain a sample 1.125 (2.475)
2.25 (2.916)
1.125 ( 1.126)
0.625 ( 1.408)
Mean duration of fixations during the delay
ROI that DID contain a sample 0.249 (0.312)
0.244 (0.261)
0.434 ( 0.214)
0.496 ( 0.177)
ROI that DID NOT contain a sample 0.04 ( 0.088)
0.17 (0.263)
0.054 ( 0.053)
0.263 ( 0.486)
Mean frequency of fixations during the comparison
ROI that DID contain a sample 0.25 (0.463) 0 0.75
(1.035) 0
ROI that DID NOT contain a sample 0.25 (0.707)
1.25 (0.354)
1.375 (1.847)
0.625 (1.06)
Mean duration of fixations during the comparison
ROI that DID contain a sample 0.02 (0.046) 0 0.058
(0.079) 0
ROI that DID NOT contain a sample 0.008 ( 0.025)
0.004 (0.011)
0.07 ( 0.108)
0.076 (0.131)
EXAMINING EYE GAZE IN A DMTS TASK 81
One-stimulus trial data
Participant
1 2 3 4 Overall accuracy (%) 87.5 100 100 100
Accuracy on matched trials (%) 100 100 100 100
Accuracy on unmatched trials (%) 67 100 100 100
Mean frequency of fixations during the delay
ROI that DID contain a sample 4.875 (SD 5.793)
2.875 (SD 2.800)
5.375 (SD 4.534)
4.75 (SD 2.712)
ROI that DID NOT contain a sample 1.125 (SD 2.475)
2.25 (SD 2.916)
1.125 (SD 1.126)
0.625 (SD 1.408))
Mean duration of fixations during the delay
ROI that DID contain a sample 0.249 (SD 0.312)
0.244 (SD 0.261)
0.434 (SD 0.214)
0.496 (SD 0.177)
ROI that DID NOT contain a sample 0.04 (SD 0.088)
0.17 (SD 0.263)
0.054 (SD 0.053)
0.263 (SD 0.486)
Mean frequency of fixations during the comparison
ROI that DID contain a sample 0.25 (SD 0.463) 0 0.75
(SD 1.035) 0
ROI that DID NOT contain a sample 0.25 (SD 0.707)
1.25 (SD 0.354)
1.375 (SD 1.847)
0.625 (SD 1.06)
Mean duration of fixations during the comparison
ROI that DID contain a sample 0.02 (SD 0.046) 0 0.058
(SD 0.079) 0
ROI that DID NOT contain a sample 0.008 (SD 0.025)
0.004 (SD 0.011)
0.07 (SD 0.108)
0.076 (SD 0.131)
Note: The accuracy on identifying a
matched or unmatched comparison, and mean frequency and mean duration of observing responses to ROI (ROI) on one-stimulus
trials. Standard deviation from the mean is presented in brackets.
Table 2
EXAMINING EYE GAZE IN A DMTS TASK 82
Two-stimulus trial data Participant
1 2 3 4 Overall accuracy (%) 87.5 79.2 95.8 100
Accuracy on matched trials (%) 83.3 75.0 91.7 100
Accuracy on unmatched trials (%) 91.7 83.3 100 100
Mean frequency of fixations during the delay
ROI that DID contain a sample 3.792 (3.695)
6.292 (4.750)
9.583 (6.659)
14.667 (9.563)
ROI that DID NOT contain a sample 0.958 (1.922)
1.208 (1.414)
2.625 (6.672)
7.125 (7.870)
Mean duration of fixations during the delay
ROI that DID contain a sample 0.156 (0.166)
0.273 ( 0.233)
0.673 (0.507)
1.143 (0.836)
ROI that DID NOT contain a sample 0.035 (0.071)
0.053 (0.075)
0.093 (0.226)
0.570 (0.678)
Mean frequency of fixations during the comparison
ROI that DID contain a sample 0 1.292 (2.053)
0.375 (1.096)
1.083 (1.586)
ROI that DID NOT contain a sample 0 0.715 (0.994)
0.333 (0.761)
0.958 (1.489)
Mean duration of fixations during the comparison
ROI that DID contain a sample 0 0.052 ( 0.101)
0.017 (0.050)
0.058 (0.085)
ROI that DID NOT contain a sample 0 0.026 ( 0.43)
0.015 (0.041)
0.068 (0.101)
Note: The accuracy on identifying a matched or unmatched comparison, and mean frequency and mean duration of observing
responses to ROI (ROI) on two-stimulus trials. Standard deviation from the mean is presented in brackets.
EXAMINING EYE GAZE IN A DMTS TASK 83
Table 3
EXAMINING EYE GAZE IN A DMTS TASK 84
Four-stimulus trial data Participant
1 2 3 4
Overall accuracy (%) 75 79.2 87.5 83.3
Accuracy on matched trials (%) 66.7 75 83.3 91.7
Accuracy on unmatched trials (%) 83.3 83.3 91.7 75
Mean frequency of fixations during the delay
ROI that DID contain a sample 3.208 (3.695)
10.75 (6.453)
15.208 (8.341)
20.083 (6.743)
Mean duration of fixations during the delay
ROI that DID contain a sample 0.13 ( 0.166)
0.503 (0.330)
0.991 (0.693)
1.375 (0.556)
Mean frequency of fixations during the comparison
ROI that DID contain a sample 0.375 (0.875)
1.917 (2.225)
1.167 (2.120)
2.458 (2.553)
Mean duration of fixations during the comparison
ROI that DID contain a sample 0.017 (0.040)
0.074 (0.086)
0.065 (0.130)
0.218 (0.268)
Note: The accuracy on identifying a matched or unmatched comparison, and mean frequency and mean duration of observing
responses to ROI (ROI) on four-stimulus trials. Standard deviation from the mean is presented in brackets.
Table 4
EXAMINING EYE GAZE IN A DMTS TASK 85
Note: Average latency to respond to the comparison in seconds. Standard deviation from the mean is presented in brackets.
Running head: EXAMINING EYE GAZE IN A DMTS TASK
86
Table 5
Participant 1 fixations during the delay by trial type and accuracy.
One stimulus Two stimuli Four stimuliMatched Unmatched Matched Unmatched Matched Unmatched
Correct trials 7 (0.212)
2(0.105)
3.5(0.149)
4.727(0.242)
2.444(0.092)
4.111(0.173)
Incorrect trials --- 0(0)
0.5(0.015)
3(0.1)
2.667(0.113)
3.333(0.13)
Participant 2 fixations during the delay by type and accuracy.
One stimulus Two stimuli Four stimuliMatched Unmatched Matched Unmatched Matched Unmatched
Correct trials 2.4(0.128)
3.667(0.227) 7.125
(0.31)
4.818(0.193)
7.333(0.323)
12.3(0.602)
Incorrect trials --- --- 11(0.547)
4(0.16)
6.333(0.28)
25(1.155)
Participant 3 fixations during the delay by type and accuracy.
One stimulus Two stimuli Four stimuliMatched Unmatched Matched Unmatched Matched Unmatched
Correct trials 4.6(0.4)
6.667(0.49)
11.7(0.772)
7.846(0.563)
17.182(1.139)
13.273(0.844)
Incorrect trials --- --- 11*(1.03) --- 0*
(0)30*
(1.98)
Participant 4 fixations during the delay by type and accuracy.
One stimulus Two stimuli Four stimuliMatched Unmatched Matched Unmatched Matched Unmatched
Correct trials 5.375(0.542)
3.667(0.423)
16.400(1.314)
13.429(1.02)
23(1.633)
16.222(1.054)
Incorrect trials --- --- --- --- --- 20(1.3)
EXAMINING EYE GAZE IN A DMTS TASK 87
Note: Mean frequency of observations by trial type. Total mean duration is shown in brackets.
Numbers with an (*) symbol represent a single trial. Dashed lines indicate that there are no trials
of that type. SD vaues are presented in Appendix C.
Table 6
Participant 1 fixations during the comparison by trial type and accuracy.
One stimulus Two stimuli Four stimuliMatched Unmatched Matched Unmatched Matched Unmatched
Correct trials .4(0.04)
0(0)
0(0)
0(0)
0.556(0.024)
0.222(0.011)
Incorrect trials --- 0(0)
0(0)
0(0)
0(0))
.667(0.027)
Participant 2 fixations during the comparison by trial type and accuracy.
One stimulus Two stimuli Four stimuliMatched Unmatched Matched Unmatched Matched Unmatched
Correct trials 0(0)
0.333(0.01) 2.125
(0.102)
0.727(0.036)
2.222(0.08)
1(0.045)
Incorrect trials --- --- 1.667(0.038)
0.5(0.025)
2(0.067)
5(0.205)
Participant 3 fixations during the comparison by trial type and accuracy.
One stimulus Two stimuli Four stimuliMatched Unmatched Matched Unmatched Matched Unmatched
Correct trials 0.4(0.046)
1.33(0.077)
0.818(0.125)
0.083(0.003)
.455(0.034)
1.546(0.076)
Incorrect trials --- --- 0*(0) --- 1*
(0.03)5*
(0.33)
Participant 4 fixations during the comparison by trial type and accuracy.
One stimulus Two stimuli Four stimuliMatched Unmatched Matched Unmatched Matched Unmatched
Correct trials 0(0)
0(0)
1(0.053)
1.167(0.063)
2.333(0.203)
2.11(0.208)
Incorrect trials --- --- --- --- --- 4(0.307)
EXAMINING EYE GAZE IN A DMTS TASK 88
Note: Mean frequency of observations by trial type. Total mean duration is shown in brackets.
Numbers with a (*) symbol represent a single trial. Dashed lines indicate that there are no trials
of that type. SD values are presented in Appendix C.
Table 7
Two-stimulus trial data Participant
1 2 3 4
Overall accuracy (%) 91.7 83.3 100 100
Overall latency (s) 1.786 (SD 1.636)
1.987 (SD 1.074)
1.451 (SD 0.671)
2.433 (SD 0.767)
Correct (s) 1.382 (SD 0.892)
1.798 (SD 1.033)
1.451 (SD 0.671)
2.433 (SD 0.767)
Incorrect (s) 0.622 (NA)
2.928 (SD 0.970) --- ---
Mean frequency of fixations during the delay
ROI that DID contain a sample 8.5 (SD 10.238)
11.5 (SD 5.76)
19.75 (SD 9.056)
11.917 (SD 5.435)
ROI that DID NOT contain a sample 0.083 (SD 0.289)
1.5 (SD 1.784)
0.917 (SD 1.443)
4.833 (SD 6.117)
Mean duration of fixations during the delay
ROI that DID contain a sample 0.434 (SD 0.527)
0.527 (SD 0.259)
1.718 (SD 0.844)
0.852 (SD 0.402)
ROI that DID NOT contain a sample 0.003 (SD 0.009)
0.061 (SD 0.072)
0.068 (SD 0.119)
0.282 (SD 0.459)
Mean frequency of fixations during the comparison
ROI that DID contain a sample 0.167 (SD 0.389)
1.75 (SD 2.340)
2.5 (SD 2.431)
2.833 (SD 4.896)
ROI that DID NOT contain a sample 0 (SD 0)
0.073 (SD 0.101)
0.167 (SD 0.389)
0.75 (SD 1.422)
Mean duration of fixations during the comparison
ROI that DID contain a sample 0.008 (SD 0.021)
0.417 (SD 0.669)
0.208 (SD 0.173)
0.295 (SD 0.608)
ROI that DID NOT contain a sample 0 (SD 0)
0.013 (SD 0.022)
0.024 (SD 0.065)
0.037 (SD 0.069)
Note: Accuracy for identifying the location of the comparison as an initial sample on two-
stimulus trials. Overall latency, and latency for correct and incorrect trials, along with observing
response data during the delay and comparison are presented as a mean. Standard deviation from
the mean is presented in brackets.
EXAMINING EYE GAZE IN A DMTS TASK 89
Table 8
Participant
1 2 3 4
Overall accuracy (%) 66.7 83.3 91.7 91.7
Overall latency (s) 3.236 (SD 1.522)
2.454 (SD 1.102)
2.148 (SD 0.961)
4.641 (SD 1.717)
Correct (s) 3.501 (SD 1.488)
2.339 (SD 1.009)
2.179 (SD 1.001)
4.596 (SD 1.793)
Incorrect (s) 2.706 (SD 1.664)
3.028 (SD 1.254)
1.807 * (NA)
5.136 * (NA)
Mean frequency of fixations during the delay
ROI that DID contain a sample 6.583 (SD 4.757)
9.917 (SD 6.374)
22.333 (SD 8.424)
17.000 (SD 6.045)
Mean duration of fixations during the delay
ROI that DID contain a sample 0.313 (SD 0.287)
0.525 (SD 0.302)
1.557 (SD 0.722)
1.297 (SD 0.359)
Mean frequency of fixations during the comparison
ROI that DID contain a sample 0.167 (SD 0.389)
1.718 (SD 1.775)
1.571 (SD 2.023)
2.917 (SD 3.059)
Mean duration of fixations during the comparison
ROI that DID contain a sample 0.005 (SD 0.012)
0.062 (SD 0.076)
0.083 (SD 0.11)
0.272 (SD 0.507)
Note: Accuracy for identifying the location of the comparison as an initial sample on four-
stimulus trials. Overall latency, and latency for correct and incorrect trials, along with observing
response data during the delay and comparison presented as mean values. Standard deviation
from the mean is presented in brackets.
EXAMINING EYE GAZE IN A DMTS TASK 90
Running head: EXAMINING EYE GAZE IN A DMTS TASK
91
Table 9
Participant 1 Participant 2 Participant 3 Participant 4 Correct Incorrect Correct Incorrect Correct Incorrect Correct Incorrect
Delay
Two-stimuli 9.9 (0.454)
3 * (0.220)
11.6 (0.523)
11 (0.530)
11.917 (0.852) - 19.75
(1.718) -
Four-stimuli 6.75 (0.333)
6.25 (0.273)
8.9 (0.479)
15 (0.755)
15.818 (1.230)
30 * (2.030)
23.909 (1.660)
5 * (0.420)
Comparison
Two-stimuli 0.182 (0.009)
0 * (0)
1.909 (0.08)
0 (0)
2.833 (0.295) - 2.5
(0.195) -
Four-stimuli 0.25 (0.006)
0 (0)
1.455 (0.555)
4 (0.33)
3 (0.313)
2 * (0.13)
1.636 (0.091)
0 * (0)
Note: Mean frequency of observations by trial type. Total mean duration is shown in brackets. Numbers with an (*) symbol represent
a single trial. Dashed lines indicate that there are no trials of that type. SD values are presented in Appendix C.
Running head: EXAMINING EYE GAZE IN A DMTS TASK
92
0
5
10
15
20
25
30
35
40
Tota
l
Cor
rect
Inco
rrec
t
Tota
l
Cor
rect
Inco
rrec
t
Tota
l
Cor
rect
Inco
rrec
t
Tota
l
Cor
rect
Inco
rrec
t
Participant 1 Participant 2 Participant 3 Participant 4
Perc
enta
ge o
f tria
ls
Figure 1. Bar graph representing the percentage of matched trials in which the participant looked
at the regions of interest that presented the correct sample during the comparison slide.
EXAMINING EYE GAZE IN A DMTS TASK 93
Appendix B
Sample matched one-stimulus trials
湯
EXAMINING EYE GAZE IN A DMTS TASK 94
Sample unmatched one-stimulus trial
張
EXAMINING EYE GAZE IN A DMTS TASK 95
Sample matched two-stimulus trial
Sample unmatched two-stimulus trial
EXAMINING EYE GAZE IN A DMTS TASK 96
Sample matched four-stimulus trial
EXAMINING EYE GAZE IN A DMTS TASK 97
Sample unmatched four-stimulus trial
EXAMINING EYE GAZE IN A DMTS TASK 98
Sample two-stimulus location trial
EXAMINING EYE GAZE IN A DMTS TASK 99
Sample four-stimulus location trial
EXAMINING EYE GAZE IN A DMTS TASK 100