A Dimensional Representation
of Emotion
James A. Russell
Boston College
Affective Computing & Intelligent Interaction
San Antonio, 24-26 Oct 2017.
Affectiva
Website says:
Emotion Recognition: Our technology
analyzes subtle facial expressions to identify
human emotions.
In our products we measure 7 emotion
metrics: anger, contempt, disgust, fear, joy,
sadness, and surprise.
Categories of Creatures
Biological
• Lions
• Tigers
• Bears
• Wolves
Mythical
• Werewolves
• Vampires
• Demons
• Zombies
Two Traditions
in the
Representation
of Emotional State
Category: discrete
A set of events with common properties
Dimension: continuous variation
A property on which events vary
Two Metaphors
Switch Metaphor:
7 patterns triggered:
Categories
Snowflake metaphor:
Infinite Number of
Unique Events:
Dimensions
“there is no limit to the
number of possible different
emotions”
“any classification of the
emotions is seen to be as true
and as 'natural' as any other “
Categories of Emotions?
William James
Outline
Categorical vs. Dimensional Representations of Emotion
Categorical View
Problems
Basic Emotion Theory
Dimensional View
Core Affect
Psychological Construction of Emotional Episodes
Categories can
Oversimplify
Sex: Man vs Woman
Race: Black vs White
Height: Tall vs Short
Shirt size: S M L XL
Color: Black vs Red
Categories
Scientific categories: necessary and
sufficient features
Everyday categories: fuzzy borders and
degree of membership
Categorical Representation
Has Dominated
Aristotle
Darwin
Scientific Version
Basic Emotion Theory
Paul Ekman
Carroll Izard
Aristotle Ekman
Anger: II 2. Anger
Calmness: II 3. Joy
Friendship and enmity: II 4.
Fear: II 5. Fear
Shame and shamelessness: II 6.
Kindness: II 7.
Pity: II 8.
Indignation: II 9.
Envy: II 10.
Emulation: II 11.
Surprise
Sadness
Disgust
Language
No word for “emotion” in many languages.
Some English words for basic emotions – “sadness,” “anger,” “fear,” “disgust” – have no translations in some languages.
Other languages have emotion words lacking in English: amae, fago, liget.
Even where translations exist:Closest Word to “anger” is not a good translation
– Anger for English speakers– Song among Ifaluk– Liget among Ilongot– Ningaq among Utku
Lost in Translation
Which Categories of Positive Emotion?
Darwin (1872)
Astonishment
Contemplation
Determination
Devotion
Happiness
High spirits
Cheerfulness
Joy
Laughter
Love
Maternal love
Pride
Tender (sympathy)
Ekman (1972)
Happiness
Subjective
Feeling
Nonverbal Signale.g., face, voice
Autonomic
Patterne.g., HR, SC
Instrumental
Action
Evente.g., danger
EmotionEvent
Theory of Basic Emotions
Basic Emotion Theory has
Problems
The theory based on categories:
Failure to find ANS patterns
Failure to find facial signals
Failure to find brain localizations
Failure to account for behavior
Lack of correlation among the components
Failure to specify the number of basic emotions
Subjective
Feeling
Nonverbal Signale.g., face, voice
Autonomic
Patterne.g., HR, SC
Instrumental
Action
Evente.g., danger
EmotionEvent
Theory of Basic Emotions
Core Affect:
Two Dimensions
• A neurophysiological state accessible as
the simple feelings of:
• Pleasure-displeasure (valence; hedonic
tone; not moral valence, etc.)
• Activation (subjective sense of energy,
arousal)
High Arousal
Sleepiness
Displeasure Pleasure
Excitement
Serenity
Distress
Depression
Core Affect
• Always in some
state of core affect
• Basis of mood
• Part of, not all of,
emotional episode
High Arousal
Sleepiness
Displeasure Pleasure
Excitement
Serenity
Distress
Depression
Core Affect
• Dimensions are
bipolar
• Axes are
independent
• Intensity of category
is distance from
center
Subjective
Feeling
Nonverbal Signalfacial
expression
Autonomic
Patterne.g., HR, SC
Instrumental
Action
Evente.g., danger
EmotionEvent
Theory of Basic Emotions
Top Science NewsSaturday 21 October, 2017
Fear of Spiders and Snakes is Innate
Snakes and spiders evoke fear in many
people, even in developed countries where
hardly anybody comes into contact with
them.
Stefanie Hoehl, Kahl Hellmer, Maria
Johansson, Gustaf Gredebäck. Itsy Bitsy
Spider…: Infants React with Increased
Arousal to Spiders and Snakes. Frontiers in
Psychology, 2017; 8
DOI: 10.3389/fpsyg.2017.01710
Study6-month-old infants
Shown spiders (and/or flowers) or snakes (and/or
fish)
Measured Pupil Dilation
A measure of arousal (not specific to fear)
So, spiders and snakes may be innately
arousing
Theory of Basic Emotion
Claim: Categorical Differences
happiness sadness anger surprise fear disgust
Affectiva
Website says:
Emotion Recognition: Our technology
analyzes subtle facial expressions to identify
human emotions.
In our products we measure 7 emotion
metrics: anger, contempt, disgust, fear, joy,
sadness, and surprise.
Production:
Do disgusted people produce the nose
scrunch?
Recognition:
Do all human beings recognize disgust from
the nose scrunch?
Does the
Nose Scrunch
express
Disgust?
Production of the “Disgust Face”
Meta-Analysis
Duran, Reisenzein, & Fernandez-Dols (2017)
Found 9 published studies in which a specific basic
emotion was created in the laboratory and the face
was assessed.
By “disgust face” is meant the nose scrunch.
Some studies counted either nose wrinkle or upper
lip raise as a “hit.” Some required both.
Some studies scored a “hit” if nose scrunch
occurred for any one out of a set of stimuli.
Production
Did those who felt disgust produce the “facial
expression of disgust” = Nose Scrunch?
Disgust created by disgust-inducing film, memory of a
disgusting event, or presenting odor of feces. Self-
reported disgust above the middle of scale.
Correlation based on 4 studies (total N = 187): .24.
Proportion based on 5 studies (total N = 279): 32%
68 % of disgusted participants did not show the disgust
expression.
Research Team
Carlos Crivelli, psychologist, De Montfort University, Leicester, UK
Sergio Jarillo, anthropologist, The Metropolitan Museum, NY; American Museum of Natural History, NY
Both speak the vernacular.
Conducted years of ethnographic work prior to experiments.
Know the local customs and live with adoptive families.
“Since these pictures show
universal facial expressions,
the message conveyed by
each face will usually be
quite obvious” (p. 11).
Prior work ConclusionCurrent study Results
Spontaneous Facial Expressions
Studies in Papua New Guinea
1. Free labeling
32 children, aged 14-17 years, 17 male; 15 female
5 Faces from Ekman (1980)
2. Forced Choice with literate respondents
24 Children, aged 12-14 years, 13 male; 11 female
Same 5 faces
5 Ekman labels, plus 2 from free labeling
Results for Disgust Face
Free Labeling
Label as “disgust”: 6%
Label as “gibulwa”: 22%
“Gibulwa” means “wants to avoidsocial interaction”
Forced Choice
“Disgust” : 38%
“Gibulwa”: 33%
General Results
Free Labeling -- Matching Ekman’s prediction
Mean: 8%
Range: 0% - 16%
For no face was predicted label modal
Forced Choice – Matching Ekman’s prediction
Mean: 23%
Range: 13% - 38%
Insight
Even in USA and in Europe, these
spontaneous facial expressions were poorly
recognized (for the emotions predicted by
Ekman).
Most research offered in support of Basic
Emotion Theory used posed facial
expressions.
Study 3: Standardized Faces
Faces: ADFES and Radboud sets
Response Format: Choice from array (5 faces)
Children aged 6-16 years:
Spanish N = 113
Papua New Guinea N = 68
Results for Disgust Face, Study 3
Point to the person who feels disgust
Children in Spain (N = 113)
Disgust Nose Scrunch: 83%
Children in Papua New Guinea (N = 68)
Disgust Nose Scrunch: 25%
Fear Gasping Face: 29% (modal choice)
Study 4
Replication in Mozambique
Control Group in Spain
Same team of researchers
Children (age 6 – 15 years)
Different sets of faces:
Static and Dynamic Faces from ADFES
Other faces we created
Disgust Results
Point to the person who feels disgust
in Mozambique
Static: 37% [32% picked “sad pout”]
Dynamic: 41% [24% picked “fear gasp”]
Study 5
Papua New Guinea and Spain
New Response Formats
Papua New Guinea: 40 children, mean age 12.24 y Spain: 40 children, mean age 12.23 y.
New Faces: 5 faces created in our lab: nose scrunch, plus “sick face” plus happy, angry, and sad.
Study 5
New Response Format
Free labeling: “How does this person feel?”
40 children in Papua New Guinea
Shown the standard disgust face,
“How does this person feel?”
3% said “disgust”
Study 5
Another New Response Format
Give a story: step in feces; finding a worm in
your food
40 children in Papua New Guinea
Pick the face for a story (choice from 5 faces):
27% picked “disgust face”
42% picked “sad face”
Is this person:
Angry
Afraid
Disgusted
Happy
Sad
Surprised
(none of the above)
Typical Method:
the “Disgust Face”
Standard Method
How does she feel?AngryDisgusted Embarrassed HappySadScaredSurprised
Pochedly, Widen, & Russell, 2012
How does she feel?AngryDisgusted Embarrassed HappySadScaredSurprised
3 “anchor” conditions
Anger anchorAnger
omittedSick faceanchor
0102030405060708090
100
Anger Anchor Anger Omitted Sick Face Anchor
% o
f R
esp
on
ses
Anchor Face ChangesInterpretation of “Disgust Face”
Angry DisgustedEmotion Label:
Summary
Basic Emotion Theory has
Problems
Failure to find predicted pattern in self-reported subjective experience
Failure to find facial signals in Papua New Guinea and Mozambique
Production
Recognition
Basic Emotion Theory has
Other Problems
Failure to find ANS patterns
Failure to find brain localizations
Failure to account for behavior
Lack of correlation among the components
Failure to specify the number of basic
emotions
Conclusion
The theory based on categories:
Pop psychology uses categories
No scientific version has been supported by evidence
Dimensional Account
Core Affect (part of, not all of, emotions)
Other Dimensions