issues in emotion-oriented computingreichardt/itemotion/...high-level: emotions in bdi models...
TRANSCRIPT
![Page 1: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/1.jpg)
Issues in Emotion-oriented ComputingTowards a shared understanding
Marc Schröder & Roddy Cowie
http://emotion-research.net
![Page 2: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/2.jpg)
humaine - emotion-research.net 2
HUman-MAchine Interaction Network on Emotions
EU network of excellence 2004-2007scientific and technological groundwork for emotion-oriented computing
interdisciplinary understandingprincipled approaches
![Page 3: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/3.jpg)
humaine - emotion-research.net 3
Sub-areas in Emotion-oriented Computing
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
![Page 4: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/4.jpg)
humaine - emotion-research.net 4
Sub-areas in Emotion-oriented Computing
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Theory ofemotional processes
![Page 5: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/5.jpg)
humaine - emotion-research.net 5
Theory of Emotional Processes
what to study: states that are relevant to emotion-oriented computingterminological clarification: clear definition of “emotions”, “moods”, …practically useful methods for describing emotional states
![Page 6: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/6.jpg)
humaine - emotion-research.net 6
“Less than 10% of human life is truly unemotional”
Cowie et al., QUB:reality TV show “Cast away”ratings of unedited tapes
truly unemotional episodesare rareemotion in the strong senseis rare
need concepts to describe the ~80% “elements of emotion”
![Page 7: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/7.jpg)
humaine - emotion-research.net 7
Aspects of “emotional life”
Cowie, O’Neill et al., QUB
emergent emotion (full-blown)emergent emotion (suppressed)moodpartial emotion (topic shifting)partial emotion (simmering)attitudeestablished emotioninterpersonal stancesaltered states of arousal/control/seriousness
![Page 8: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/8.jpg)
humaine - emotion-research.net 8
Aspects of “emotional life”
Cowie, O’Neill et al., QUB
emergent emotion (full-blown)emergent emotion (suppressed)moodpartial emotion (topic shifting)partial emotion (simmering)attitudeestablished emotioninterpersonal stancesaltered states of arousal/control/seriousness
DefinitionThis refers to states between a mood and an emergent emotion, where a person is in a single kind of emotional state, but it is not fixed on a single thing or issue – instead the emotion attaches to one object, then another, then another, in quite rapid succession. Example“Angry and tired from previous day. Getting annoyed when styling hair because it wouldn’t stay. Angry at dog trying to get out, and things at work.”
![Page 9: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/9.jpg)
humaine - emotion-research.net 9
Labels for “emotional life”
Negative & forceful Anger Annoyance Contempt Disgust Irritation Negative & not in control Anxiety Embarrassment Fear Helplessness
Powerlessness WorryNegative thoughts Doubt Envy Frustration Guilt ShameNegative & passive Boredom Despair Disappointment Hurt Sadness
Agitation Shock Stress TensionPositive & lively Amusement Delight Elation Excitement Happiness
Joy Pleasure
Caring Affection Empathy Friendliness Love Positive thoughts Courage Hope Pride Satisfaction TrustQuiet positive Calm Content Relaxed Relieved Serene
Reactive Interest Politeness Surprise
Cox et al.: List consolidated from Geneva, LIMSI, Belfast sources
![Page 10: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/10.jpg)
humaine - emotion-research.net 10
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Sub-areas in Emotion-oriented Computing
Raw empirical
data
![Page 11: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/11.jpg)
humaine - emotion-research.net 11
Emotions in Empirical Data
Acted
Induced
TV interviews
In action
![Page 12: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/12.jpg)
humaine - emotion-research.net 12
Emotions in Empirical Data
“supportive” datato fill in gaps in theory
“provocative” datato challenge preconceptions
develop suitable labelling schemes
![Page 13: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/13.jpg)
humaine - emotion-research.net 13
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Sub-areas in Emotion-oriented Computing
Emotion detection
![Page 14: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/14.jpg)
humaine - emotion-research.net 14
Emotion Detection: Multimodal Features
![Page 15: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/15.jpg)
humaine - emotion-research.net 15
Emotion Detection: Selected Challenges
Integration of modalitiesdifferent time scales for analysis
face = still picturesvoice = discrete chunks on time scalephysiology = slow, continuous variation over time
Measure of success?same errors as humans?
dialogue system
more accurate than humans?stress detection
![Page 16: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/16.jpg)
humaine - emotion-research.net 16
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Sub-areas in Emotion-oriented Computing
Synthesis of basic signs
![Page 17: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/17.jpg)
humaine - emotion-research.net 17
Greta agentPelachaud et al.,Univ. Paris 8
Emofilt speech synthesis filterBurkhardt, T-Systems
Synthesis of Signs of Emotion:Parametrisable Models of Body, Face and Voice
![Page 18: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/18.jpg)
humaine - emotion-research.net 18
Synthesis of Signs of Emotion
make embodied conversational agents (ECAs) behave emotional
required key capabilities:Coordination of signs in multiple modalities ExpressivityCognitive influences on action (attention model)Creating affective awarenessBackchannel & Feedback behaviour
specify architecture requirementstowards proof-of-concept sub-systems
![Page 19: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/19.jpg)
humaine - emotion-research.net 19
Attention Modelling for ECAs
crucial for initiating and maintaining communicationvisual salience
contrast, color, …facesemotionalexpressions
Peters et al., Univ. Paris 8
![Page 20: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/20.jpg)
humaine - emotion-research.net 20
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Sub-areas in Emotion-oriented Computing
Planning action
![Page 21: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/21.jpg)
humaine - emotion-research.net 21
Emotion in Planning and Action
conceptual and practical work on cognitive architectures with emotion
low-level / sub-symbolic / reactivehigh-level / symbolic / deliberativehybrid: bridging the gap
![Page 22: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/22.jpg)
humaine - emotion-research.net 22
Low-level Perception-action Loop:Emergent “aggression”
robots need two resources to survivewander around avoiding objectsblue robot: bumper sensitivity reduced when deficit too largeresulting behaviour: pushing the otheraway from resourceethological analysis: “aggressive” behaviour
Cañamero et al.,Univ. Hertfordshire
![Page 23: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/23.jpg)
humaine - emotion-research.net 23
Low-level Perception-action Loop:Imprinting a “desired perception”
Imprint visual perception of “optimal” box distanceRobot will follow box in this distance,goal = re-establish desired perceptionK. Lorenz
Blanchard & Cañamero, Univ. Hertfordshire
![Page 24: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/24.jpg)
humaine - emotion-research.net 24
High-level: Emotions in BDI models
Reasoning about emotionsinterpret emotional impactof previous dialogue moveguess possible emotional impactof next dialogue move
Based on an appraisal model (e.g., OCC)link events to emotional meaning
![Page 25: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/25.jpg)
humaine - emotion-research.net 25
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Sub-areas in Emotion-oriented Computing
Emotion in
complex media
![Page 26: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/26.jpg)
humaine - emotion-research.net 26
Emotion in Complex Media
closest to applicationsexploratory work
automatic generation of potentially funny slogansplaying with semantic ambiguities in languagehuman has to decide which slogans are funnyExample: slogan for soft drinks: “Thirst come, thirst served!” (Stock et al., itc-IRST)
persuasive ECA communicationpolitenesslying behaviour
![Page 27: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/27.jpg)
humaine - emotion-research.net 27
Generating Lying Expressions in an ECA
GAMBLE dice gameRehm et al.,Univ. Augsburg
![Page 28: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/28.jpg)
humaine - emotion-research.net 28
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Sub-areas in Emotion-oriented Computing
Usability/ evaluation
![Page 29: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/29.jpg)
humaine - emotion-research.net 29
Usability of Emotion-oriented Systems
existing usability criteria do not applye.g., control, predictability, transparency
Develop criteria and methods to evaluate affective interaction systems
“sensual” feedback methodsWizard-of-Oz studies extended think-aloud protocol
![Page 30: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/30.jpg)
humaine - emotion-research.net 30
“Sensual” Feedback Methods for Usability Testing
SenToy puppetPaiva et al., INESC-ID
Colour and shapeHöök et al., KTH
![Page 31: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/31.jpg)
humaine - emotion-research.net 31
Wizard of Oz Method: ECA-based Dialogue
study user behaviouruser-centered design loop
design->test->redesign
explore affective interactionsocial relationship / empathy
de Rosis et al.,DI-BARI
User interface
Wizard interface
![Page 32: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/32.jpg)
humaine - emotion-research.net 32
Empirical evidenceEmotion
detection
planning
Emotional language
evaluation
Description of
emotion
Ethicsof emotion
Emotion detection
Planning action
Theory ofemotional processes
Raw empirical
data
Emotion incomplexmedia
Usability / evaluation
Synthesis of basicsigns
Application oriented
Core technologies
Psychologically oriented
Sub-areas in Emotion-oriented Computing
Ethics of emotion
![Page 33: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/33.jpg)
humaine - emotion-research.net 33
Ethics in HUMAINE
Data collection: emotional data is sensitivePotential abuse of emotionally competent systems
automatic persuaderssurveillance systems…
Address ethical issues proactively
![Page 34: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/34.jpg)
humaine - emotion-research.net 34
Ethics in HUMAINE
Ethics research is new to this communityunrealistic to formulate a closed set of criteriaapproach: “Principlism” (Beauchamps & Childress, 2001)
Four Level Modelnonmaleficence, autonomy, beneficance, justiceuniversally shared ethical standardsrelies on “seeing things right”
Install ethical committeebecause every case is different
![Page 35: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/35.jpg)
humaine - emotion-research.net 35
Conclusions
Working on these different levelsremoving obstacles in cooperation(“make things fit together”)building basis for future worksuggesting sensible research agendas
Applications Coretechnologies
Psychologicaltheories ofemotion
![Page 36: Issues in Emotion-oriented Computingreichardt/itemotion/...High-level: Emotions in BDI models Reasoning about emotions interpret emotional impact of previous dialogue move guess possible](https://reader034.vdocuments.net/reader034/viewer/2022042612/5f7f577e25a267465c2475bb/html5/thumbnails/36.jpg)
humaine - emotion-research.net 36