1September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

2

Stimuli in receptive field of neuron

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

3

Cat V1 (striate cortex)

Orientation preference map

Ocular dominance map

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

4September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

5

Structure of NNs (and some ANNs)• In biological systems, neurons of similar

functionality are usually organized in separate areas (or layers).

• Often, there is a hierarchy of interconnected layers with the lowest layer receiving sensory input and neurons in higher layers computing more complex functions.

• For example, neurons in macaque visual cortex have been identified that are activated only when there is a face (monkey, human, or drawing) in the macaque’s visual field.

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

6

“Data Flow Diagram”of Visual Areas inMacaque Brain

Blue:motion perception pathway

Green:object recognition pathway

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

7

Receptive Fields in Hierarchical Neural Networks

neuron A

receptive field of A

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

8

Receptive Fields in Hierarchical Neural Networks

receptive field of A in input layer

neuron Ain top layer

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

9

Face aftereffect – Thanks to Arash Afraz for the slides!

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

10

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

11

Visual Illusions demonstrate how we perceive an “interpreted version” of the incoming light pattern rather that the exact pattern itself.

Visual Illusions

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

12

Visual Illusions

He we see that the squares A and B from the previous image actually have the same luminance (but in their visual context are interpreted differently).

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

13

Visual Attention• Visual attention is the selective allocation of visual

processing resources.

• For example, we can focus our attention on a particular object of interest in the visual field.

• Visual processing of that object is enhanced while being rather shallow for other objects.

• Also, we can respond more quickly and accurately to changes in an attended region.

• This prioritization is necessary due to our limited processing resources.

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

14

Visual AttentionThe attentional cueing task introduced by Michael Posner gives insight into the dynamics of visual attention.

Subjects are instructed to fixate on the central cross.

One of two boxes (left or right) flashes to capture the subject’s attention (an automatic, involuntary response).

After some a short delay (stimulus onset asynchrony - SOA) an asterisk appears in one of the boxes.

The subject has to report as quickly as possible in which box the asterisk appeared.

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

15

The Posner Attention Task

x

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

16

The Posner Attention Task

x

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

17

The Posner Attention Task

x

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

18

The Posner Attention Task

* x

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

19

The Posner Attention Task

x

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

20

The Posner Attention Task

For short SOAs (< 200 ms), subjects respond faster if flash and asterisk appear on the same side.

Þ Cueing of attention to relevant location allows faster response.

For longer SOAs, subjects respond more slowly if flash and asterisk appear on the same side.

Þ Inhibition-of-Return mechanism makes attention less likely to remain on the side of the flash until the asterisk appears.

Eye Movements

Eye MusclesSeptember 4, 2014 Computer Vision

Lecture 2: Vision, Attention, and Eye Movements

21

Types of Eye Movement

Fixations:

The eye is almost motionless, for example, while reading a single, short word.

The information from the scene is almost entirely acquired during fixation.

Duration varies from 100-1000 ms, typically between 200-600 ms.

Typical fixation frequency is about 3 Hz.

Fixations are interspersed with saccades.

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

22

Types of Eye Movement

Saccades:

Quick “jumps” that connect fixations

Duration is typically between 30 and 120 ms

Very fast (up to 700 degrees/second)

Saccades are ballistic, i.e., the target of a saccade cannot be changed during the movement.

Vision is suppressed during saccades to allow stable perception of surroundings.

Saccades are used to move the fovea to the next object/region of interest.September 4, 2014 Computer Vision

Lecture 2: Vision, Attention, and Eye Movements

23

Types of Eye Movement

Smooth Pursuit Eye Movements:

Smooth movement of the eyes for visually tracking a moving object

Cannot be performed in static scenes (fixation/saccade behavior instead)

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

24

Why Eye-Movement Research?About eye movements and visual attention:

Usually, saccades follow shifts of attention to provide high acuity at the attended position.

It is possible to look at an object without paying attention to it (staring).

It is possible to shift attention without eye movement (covert shifts of attention).

It is impossible to perform a saccade while not shifting attention.

During specific, natural tasks it is reasonable to assume that saccades follow shifts of attention.September 4, 2014 Computer Vision

Lecture 2: Vision, Attention, and Eye Movements

25

Why Eye-Movement Research?The investigation of visual attention, in turn, is at the core of cognitive science.

• Studying visual attention yields insight into general attentional mechanisms.

• It can provide information on a person’s stream of conscious and unconscious processing while solving a task.

• Attention is closely linked to the concept of consciousness.

• Attentional mechanisms could improve artificial vision systems.

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

26

Eye-Movement Studies

Eye movements while watching a girl’s face (early study by Yarbus, 1967)

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

27

Eye-Movement StudiesEye movements as indicators of cognitiveprocesses (Yarbus):

trace 1: examine at will

trace 2: estimate wealth

trace 3: estimate ages

trace 4: guess previous activity

trace 5: remember clothing

trace 6: remember position

trace 7: time since last visit

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

28

Eye-Movement Studies

Visual scan paths on instruments/dashboards – studies for the improvement of human-computer interfaces

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

29

Eye-Movement Studies

Gaze trajectory measurement for the optimization of web page layout

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

30

Eye-Movement Studies

Improving advertisements with eye-movement studiesSeptember 4, 2014 Computer Vision

Lecture 2: Vision, Attention, and Eye Movements

31

Selectivity in Complex Scenes

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

32

Face Recognition

Gaze-contingent window deteriorates face recognition, allows to identify relevant visual information.

September 4, 2014 Computer Vision Lecture 2: Vision, Attention, and Eye

Movements

33