alternative theoretical perspectives on emotion representation & modeling
DESCRIPTION
Keynote at EmoSpace2011 WorkshopMarch 2011, Santa Barbara, CA, USTRANSCRIPT
![Page 1: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/1.jpg)
1EmoSPACE2011Hudlicka
Alternative TheoreticalAlternative TheoreticalPerspectives on EmotionPerspectives on Emotion
RepresentationRepresentation & Modeling& ModelingEva Hudlicka
Psychometrix Associates Blacksburg, VA
EmoSPACE2011 Workshop9th FG 2011
March 21, 2011Santa Barbara, CA
![Page 2: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/2.jpg)
2EmoSPACE2011Hudlicka
Outline
• Definition• Three Theoretical Perspectives• Similarities & Differences• Implications for Affective Modeling• Summary & Conclusions
![Page 3: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/3.jpg)
3EmoSPACE2011Hudlicka
Emotions• Evaluative judgments of the:
– World– Others– Self
• … in light of agent’s goals & beliefs
• …motivating & coordinating adaptivebehavior
![Page 4: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/4.jpg)
4EmoSPACE2011Hudlicka
Emotions Are Multimodal
• Manifested across multiple, interactingmodalities:
– Physiological / Somatic (neuroendocrine - e.g., heartrate, GSR)
– Cognitive / Interpretive (“Nothing is good or bad butthinking makes it so…”; appraisal, biasing effects)
– Motor / Behavioral (expressive, action oriented)– Experiential / Subjective (conscious experience)
![Page 5: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/5.jpg)
5EmoSPACE2011Hudlicka
Dominant TheoreticalPerspectives
• Discrete / Categorical (Tomkins, Izard, Ekman, Panskepp…)
• Dimensional (Wundt, Osgood, Lang, Russell, Thayer, Watson,Tellegen, Mehrabian…)
• Componential (focus on cognitive appraisal)(Scherer, Roseman, Reisenzein, Smith, Ellsworth, Frijda,Ortony…)
![Page 6: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/6.jpg)
6EmoSPACE2011Hudlicka
Discrete / Categorical• A small number of ‘hardwired’ basic emotions
– Joy, fear, anger, sadness, disgust, surprise..
• Characterized by distinct patterns of:– Triggering stimuli– Cognitive processing– Behavioral tendencies– Expression
(Cohn, 2006)
![Page 7: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/7.jpg)
7EmoSPACE2011Hudlicka
Dimensional• Emotions defined in terms of 2 or 3 dimensions
– 2D: Pleasure & Arousal– 3D: Pleasure & Arousal & Dominance
• 4th dimension recently proposed:– Unpredictability
(Fontaine, Scherer et al., 2007)
(Breazeal, 2003 adaptedfrom Russell, 1997)
![Page 8: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/8.jpg)
8EmoSPACE2011Hudlicka
Dimensional
from de Groot & Broekens, 2003 – adapted from Mehrabian, 1995
![Page 9: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/9.jpg)
9EmoSPACE2011Hudlicka
2 or 3 Dimensions?
• 2D - cannot distinguish between emotionssharing same P & A values– Negative valence, positive arousal– Anger? Fear?
• Need 3rd dim. to distinguish --> dominance– Anger: - val., + arousal, + dominance+ dominance– Fear: - val., + arousal, - dominance- dominance
![Page 10: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/10.jpg)
10EmoSPACE2011Hudlicka
Componential
• Emotions characterized by synchronizedactivity across multiple modalities– Cognitive, Physiological, Expressive, Behavioral,
Subjective
• Cognitive modality & appraisal characterized byappraisal variables (dimensions)
![Page 11: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/11.jpg)
11EmoSPACE2011Hudlicka
Novelty
Valence
Goal relevance
Certainty
Urgency
Goal congruence
Agency
Coping potential
Norms
StimuliStimuli Relevance Implications Coping Norms
EmotionEmotion
Appraisal Variables
![Page 12: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/12.jpg)
12EmoSPACE2011Hudlicka
STIMULISTIMULI
Novelty
Valence
Goal relevance
Outcomeprobability
Urgency
Goal congruence
Agency
Coping potential
high
high
v. high
low
other
lowlow
low
high
FEARFEAR
![Page 13: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/13.jpg)
13EmoSPACE2011Hudlicka
OCC Theory
• Ortony, Clore, Collins (OCC):Cognitive Structure of Emotions (1988)
• Emotions characterized by abstract evaluativecriteria applied to:– Events (desirable / not…)– Acts by other agents (praiseworthy / not…)– Objects (attractive / not…)
• ~22 emotions defined (including complex/social)
![Page 14: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/14.jpg)
14EmoSPACE2011Hudlicka
Valenced Reactions
Event-basedemotions Attribution
emotionsAttractionemotions
Fortunes-of-selfemotions
Fortunes-of-othersemotions
happy for, pity, gloating..
distress
Prospect-basedemotions
Well-beingemotions
anger
reproachlove,hate
Desirability = low
fear
Praiseworthiness = low
degree of autonomy = highexpectation deviation = high
![Page 15: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/15.jpg)
15EmoSPACE2011Hudlicka
Outline
• Definition• Three Theoretical Perspectives•• Similarities & DifferencesSimilarities & Differences• Implications for Affective Modeling• Summary & Conclusions
![Page 16: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/16.jpg)
16EmoSPACE2011Hudlicka
Semantic Primitives:Atomic Structure of Emotions
Componential
Appraisalvariables (13)
OCC evaluationcriteria (11)
Discrete /Categorical
~6 Basicemotions
Etc.
Dimensional
PA or PADdims.
Becker-Asano, 2005
![Page 17: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/17.jpg)
17EmoSPACE2011Hudlicka
(Size of) Affective Spaces
# of affective states accommodatedsmall large
![Page 18: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/18.jpg)
18EmoSPACE2011Hudlicka
Which Emotions Can be Definedby the Semantic Primitives?
• Discrete / Categorical– Basic emotions (joy, sadness, fear, anger, disgust, surprise)– Complex & social emotions? - Theoretical basis lacking
• Dimensional– Larger set than basic - but not all emotions uniquely defined– Lacks cognitive differentiation
• Componential (cognitive appraisal variables)– Very large space, accommodates many types of affective
states & varying intensities
![Page 19: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/19.jpg)
19EmoSPACE2011Hudlicka
What Do the Different SemanticPrimitives Characterize?
Categorical /Discrete
Basicemotions
Componential
Appraisal variables/ OCC criteria
World &Relationship ofSelf w/ World
Dimensional
PA or PADdims.
Subjective FeltExperience
![Page 20: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/20.jpg)
20EmoSPACE2011Hudlicka
Emotion Modalities Emphasized• Discrete / Categorical
– Multimodal patterns (cognitive, motivational,behavioral,subjective) characterizing distinct basic emotions
• Dimensional– Subjective felt experience -
physiology & subjective experience >> cognition– Characterize ‘core affect’ (Russell)
• Componential (appraisal)– Interpretation of stimuli & the stimuli-agent relationship -
cognitive & interpretive structure >> physiology– Characterize emotions proper
![Page 21: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/21.jpg)
21EmoSPACE2011Hudlicka
Outline
• Definition• Three Theoretical Perspectives• Similarities & Differences•• ImplicationsImplications for Affective Modelingfor Affective Modeling• Summary & Conclusions
![Page 22: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/22.jpg)
22EmoSPACE2011Hudlicka
(Some) Implications for Affective Modeling
• Modeling core affective processes– Modeling emotion generation– Modeling emotion effects
![Page 23: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/23.jpg)
23EmoSPACE2011Hudlicka
Discrete / Categorical
P=xA=yD=z
=A
PD
+
-+
+
Dimensional
OCC (appraisal) Evaluative criteriaDesirabilityPraiseworthinessAttractiveness … etc.
Novelty =xValence =yGoal Congr.=zAgency =wEtc.
=
Componential (appraisal) Appraisal Variables
EMOTION GENERATION MODELSEMOTION GENERATION MODELSDomainDomainStimuliStimuli EmotionsEmotions
![Page 24: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/24.jpg)
24EmoSPACE2011Hudlicka
Modeling Emotion Generation• Emphasis on cognitive modality - cognitive appraisal
(other modalities ignored / minimized)
• Appraisal theories (componential perspective) providebest theoretical support
• Emotions defined in terms of abstract (domain-independent) features:– Vectors of appraisal variables– Vectors of OCC evaluation criteria
• Identifying values of these features may be non trivial
![Page 25: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/25.jpg)
25EmoSPACE2011Hudlicka
MultimodalManifestations
of EmotionsEmotions
Discrete / Categorical
P=xA=yD=z
Dimensional
Novelty =xValence =yGoal Congr.=zAgency =wEtc.
Componential Appraisal Variables
cognition
physiology
speech
expressions& gestures
action
EMOTION EFFECTS MODELSEMOTION EFFECTS MODELS
![Page 26: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/26.jpg)
26EmoSPACE2011Hudlicka
Modeling Emotion Effects• More challenging > emotion generation:
– Cannot easily ignore multiple modalities– Effects on behavior, expression, (less) cognition
• Theoretical support << emotion generation
• PAD representation facilitates modeling of expressivemanifestations
• Supporting data for mapping PAD & appraisalvariables onto effects not always available– More data available at discrete / categorical level
![Page 27: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/27.jpg)
27EmoSPACE2011Hudlicka
Benefits of DimensionalRepresentations
• Model large # of emotions• Facilitate integration of multiple emotions
… but theory still lacking
• Continuous representations facilitate smoothtransitions among emotions & differentintensities– …more realistic affective expression
• Enable parsimonious representation ofshared qualities of different emotions– …high arousal --> rapid movement, high speech pitch
![Page 28: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/28.jpg)
28EmoSPACE2011Hudlicka
Which Perspective & SpaceShould You Choose?
• ….it depends• Which emotions will you need?
• Which affective processes will you model?– Appraisal variables good for emotion generation modeling
(via cognitive appraisal)– PAD dimensions good for dynamics of emotion expression– …
• Are the data available?– For each dimension / variable?– For each modality of interest?
![Page 29: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/29.jpg)
29EmoSPACE2011Hudlicka
Outline
• Definition• Three Theoretical Perspectives• Similarities & Differences• Implications for Affective Modeling•• Summary & ConclusionsSummary & Conclusions
![Page 30: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/30.jpg)
30EmoSPACE2011Hudlicka
Summary of ContinuousCharacterizations of Emotions
• Different spaces defined by different underlyingdimensions (semantic primitives)– PAD vs. appraisal variables (& OCC evaluative criteria)
• Emphasizing different aspects & modalities ofemotions– Felt experience vs. stimulus <--> agent relationship– Physiology vs. cognition
• Different pros/cons for particular affective computingtasks
• “the search for the optimal low-dimensionalrepresentation of the emotion domain remains open”(Fontaine, Scherer, Roesch, Ellsworth, 2007)
![Page 31: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/31.jpg)
31EmoSPACE2011Hudlicka
Conclusions
• Different perspectives / spaces should beviewed as complementary– Decision re: perspective choice guided by specific research
& applied objectives– Multiple perspectives may be used in a single architecture
…supporting different tasks
• It’s too early for a “unified theory” of emotions– Unified theory of emotions may not exist– “Emotion is too broad a class of events to be a single
scientific category, and no one structure suffices.” (Russell &Feldman Barrett, 1999)
![Page 32: Alternative Theoretical Perspectives on Emotion Representation & Modeling](https://reader030.vdocuments.net/reader030/viewer/2022020115/5481c078b4af9f640d8b45df/html5/thumbnails/32.jpg)
32EmoSPACE2011Hudlicka
Thank you
Questions?