Charles SpenceDepartment of Experimental

Psychology, Oxford University

New Perspectives from the Human Sciences

Consumer Focus Workshop (November, 2001)

The Execution of Lady Jane Grey

• We are aware of only a small amount of the sensory information available at any one time

For example…...

Do people detect changes in ‘real-world’ interactions?

Implications for:

- eye-witness testimony

- head-up displays in cars & planes

Wickens (1980, 1984, 1988, 1992...) Structure of human processing

resources

Same-Side Condition

Different-Sides Condition

Relevant Visual

Irrelevant AuditoryRelevant Auditory

Irrelevant Visual

Relevant Visual

Relevant AuditoryIrrelevant Auditory

Irrelevant Visual

Shadowing Performance

30

40

50

60

70

Chewing-Lips

Speaking-Lips

1) Lip-reading facilitates shadowing

2) Better performance when auditory & visual information from same location

SameSide

DifferentSides

1)

2)

% C

orre

ct

Don’t Dial & Drive?

Spence & Reid (2000)

RS

RS

IS

ShadowSide

ShadowFront

SimulatorScreen

Ergonomics: Design based on Human Capabilities- Multisensory Warning Signals

In the Realm of the Senses

Interactions between smell & vision

• Speeded left/right

response. Menthol

to either nostril.

• Olfactory & visual

stimuli from same

locations.

• Cuing (80% valid,

20% invalid).

Spence, Kettenmann, Kobal, & McGlone (2000, 2001a, b)

120

71 8056

0

50

100

150

Mea

n C

uin

g .

E

ffec

t (m

s)

Tactile Olfactory Visual Olfactory

• Attending to smell affects vision• Implications for: advertising, alerting, &

pain management

Target Modality

Multisensory Perception• Most products are multimodal:

Skincare products have smell, colour, tactile qualities

• Foodstuffs: taste, colour, & smell

• It’s important to know what aspects of a product people attend to, to determine what they will perceive

Multisensory Perception

The McGurk effect

Crossmodal Illusions - McGurk effect

• McGurk effect shows that what we see alters what we hear

• Manipulating information in one modality allows us to change perception in other modalities predictably

Vision can also dominate taste DuBose et al. (1980)

• Cherry-flavoured carbonated drinks perceived as ‘orange-flavoured’ when coloured orange

Sound dominates touch‘Parchment Skin’ Illusion Jousmaki & Hari (1998)

• People estimated skin roughness & wetness, while rubbing hands together

• Sound presented over headphones, & frequency content manipulated

• Changing sound dramatically altered perception of skin roughness

Changing perception of skin through sound?

Dry Hydrated

?

1) Selective use of subjects

2) Results rely on subjective report, & hence open to task demands

Solution: Develop experimental paradigms that rely on more objective measures (eg psychophysics, neuroimaging)

But, there are problems...

Changing perception of products through sound?

Superadditivity: Neurophysiology

Stein & Meredith (1993)

Multisensory Integration“The integration of inputs from different sensory modalities not only transforms some of their individual characteristics, but does so in ways that can enhance the quality of life”

“Integrated sensory inputs produce far richer experiences than would be predicted from their simple coexistence or the linear sum of their individual products”

Stein & Meredith (1993)

Superadditivity: Taste & Smell

(Dalton et al., 2000)

Cognitive Neuroscience

Orbitofrontal Cortex (Reward)

Neutral Touch Pleasant Touch

N1

2µV

-2µV

700msCuedUncued

Improving the quality of life for the physically

challenged / brain damaged

Conclusions• Attention & cognition critically

determine perception & behavior

• A better understanding of multisensory interactions will enhance quality-of-life

• Converging methodologies approach provides powerful & increasingly subtle tools to understand both brain function & behavior (consumer science)