making the head smile
DESCRIPTION
Making the head smile. Smile ? -> part of non-verbal communication Functions of non-verbal communication To enhance linguistic information To give new information. Non-verbal communication. Figure by Jari Kätsyri. Voice prosody. Figure by Jari Kätsyri. Head. Figure by Jari Kätsyri. - PowerPoint PPT PresentationTRANSCRIPT
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Making the head smileMaking the head smile
Smile? -> part of non-verbal communication
Functions of non-verbal communication To enhance linguistic information To give new information
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Non-verbal communicationNon-verbal communication
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Voice prosodyVoice prosody
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
HeadHead
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
EyesEyes
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Facial expressionsFacial expressions
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
GesturesGestures
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Conversation regulationConversation regulation
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Conversational signalsConversational signals
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
MaintenanceMaintenance
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
EmotionsEmotions
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Expressing emotionsExpressing emotions
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Conversational signalsConversational signals
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
VoiceVoice
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
HeadHead
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
EyesEyes
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
FaceFace
Figure by Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
ExamplesExamples
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Dimitris Metaxas, Normal Dimitris Metaxas, Normal BadlerBadler Modeling of hand gestures Modeling emotions, personality…
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Facial expressionsFacial expressions
Emotional expressions in face Are expressions showing emotions? How they can be described? What kind of emotions are there?
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Emotion in face: To Emotion in face: To communicate or to express?communicate or to express? Communicating vs. expressing emotions
Intentional communication (Chovil) vs. Involuntary behaviour (Ekman)
Technology viewpoint Emotion = Emotional expression = (Non-
verbal) communication (In most situations emotions
inappropriate)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Emotions and facial Emotions and facial expressions (Ekman)expressions (Ekman) Emotion Facial expression ?
Emotional expressions can be inhibited No evidence for non-expressed emotions
Facial expression Emotion ? Emotional expressions can be faked There are facial muscles with no
voluntary control Facial expression Emotion (?)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Description of expressionDescription of expression
Several descriptions of expressions exist: Facial Action Coding System – FACS
By Eckman and Friesen ”Language of facial expressions”
MPEG-4 Synthetic-Natural hybrid coding (SNHC)
“Technical” description for low-bandwidth transmission
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
FACSFACS
Noticeable changes, not exactly facial muscles Tool for identifying facial actions Not for modeling facial actions Does not describe dynamics
Action units (AUs) AU1: Inner brow raiser … AU23: Lip tightener … AU66: Crosseye
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
AU examples: AU7 (lid AU examples: AU7 (lid tightener)tightener)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
AU examples: AU9 (nose AU examples: AU9 (nose wrinkler)wrinkler)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
AU examples: AU11AU examples: AU11(nasolabial furrow deepener)(nasolabial furrow deepener)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Pure and blended emotionsPure and blended emotions
7 pure emotions Pleasure (pl), surprise (su), fear (f), hate
(h), sorrow (so), disgust (d), (interest) Blended emotions
Combinations of pure emotions Easy: pl+su, h+d, so+f Difficult: su+f, d+pl, h+so, f+pl (Some) additivity
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Possible causes of blended expressions in real world Mixed emotions (sadness+disgust) Consequent emotions
(surprise+happiness) Masking (anger+happiness) …
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Attempts to add emotionsAttempts to add emotions
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
neu
tral
sad
su
rpri
se
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
6 basic expressions6 basic expressions
courtesy Jari Kätsyri
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
courtesy Jari Kätsyri
Blended expressionsBlended expressions
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Textured × non-texturedTextured × non-textured
Identification of expressions from pictures and videos of textured and non-textured version of AP
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
anger disgust fear happin. sadn. surprise
picture
picture+tex
video
video+tex
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Differences among databasesDifferences among databases
Number of correct responses for pictures and videos of different databases
Static
Dynamic5
6
7
8
9
10
CK HUT EF TH
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Neutral expressionNeutral expression
Average evaluations for AP textured and non-textured neutral expressions
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Better implementationBetter implementation
Collecting data
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Collecting data for Collecting data for expressionsexpressions
happy anger
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
HUT facial expression HUT facial expression databasedatabase First Finnish
database of moving facial expressions
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Study:Study:Interaction of blur x type x Interaction of blur x type x emotionemotion New database, 6 actors 3 levels of blur Pictures, short video sequences 6 basic expressions
3 subject groups (17 each) each group – 1 level of blur each actor 6 expressions 3 actors pictures, 3 actors videos
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Interaction of blur x type x Interaction of blur x type x emotionemotion
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
The endThe end
Thank you!
and … a little exercise again