Download - Making the head smile
![Page 1: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/1.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Making the head smileMaking the head smile
Smile? -> part of non-verbal communication
Functions of non-verbal communication To enhance linguistic information To give new information
![Page 2: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/2.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Non-verbal communicationNon-verbal communication
Figure by Jari Kätsyri
![Page 3: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/3.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Voice prosodyVoice prosody
Figure by Jari Kätsyri
![Page 4: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/4.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
HeadHead
Figure by Jari Kätsyri
![Page 5: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/5.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
EyesEyes
Figure by Jari Kätsyri
![Page 6: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/6.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Facial expressionsFacial expressions
Figure by Jari Kätsyri
![Page 7: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/7.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
GesturesGestures
Figure by Jari Kätsyri
![Page 8: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/8.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Conversation regulationConversation regulation
Figure by Jari Kätsyri
![Page 9: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/9.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Conversational signalsConversational signals
Figure by Jari Kätsyri
![Page 10: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/10.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
MaintenanceMaintenance
Figure by Jari Kätsyri
![Page 11: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/11.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
EmotionsEmotions
Figure by Jari Kätsyri
![Page 12: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/12.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Expressing emotionsExpressing emotions
Figure by Jari Kätsyri
![Page 13: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/13.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Conversational signalsConversational signals
Figure by Jari Kätsyri
![Page 14: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/14.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
VoiceVoice
Figure by Jari Kätsyri
![Page 15: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/15.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
HeadHead
Figure by Jari Kätsyri
![Page 16: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/16.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
EyesEyes
Figure by Jari Kätsyri
![Page 17: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/17.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
FaceFace
Figure by Jari Kätsyri
![Page 18: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/18.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
ExamplesExamples
![Page 19: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/19.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Dimitris Metaxas, Normal Dimitris Metaxas, Normal BadlerBadler Modeling of hand gestures Modeling emotions, personality…
![Page 20: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/20.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Facial expressionsFacial expressions
Emotional expressions in face Are expressions showing emotions? How they can be described? What kind of emotions are there?
![Page 21: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/21.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Emotion in face: To Emotion in face: To communicate or to express?communicate or to express? Communicating vs. expressing emotions
Intentional communication (Chovil) vs. Involuntary behaviour (Ekman)
Technology viewpoint Emotion = Emotional expression = (Non-
verbal) communication (In most situations emotions
inappropriate)
![Page 22: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/22.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Emotions and facial Emotions and facial expressions (Ekman)expressions (Ekman) Emotion Facial expression ?
Emotional expressions can be inhibited No evidence for non-expressed emotions
Facial expression Emotion ? Emotional expressions can be faked There are facial muscles with no
voluntary control Facial expression Emotion (?)
![Page 23: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/23.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Description of expressionDescription of expression
Several descriptions of expressions exist: Facial Action Coding System – FACS
By Eckman and Friesen ”Language of facial expressions”
MPEG-4 Synthetic-Natural hybrid coding (SNHC)
“Technical” description for low-bandwidth transmission
![Page 24: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/24.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
FACSFACS
Noticeable changes, not exactly facial muscles Tool for identifying facial actions Not for modeling facial actions Does not describe dynamics
Action units (AUs) AU1: Inner brow raiser … AU23: Lip tightener … AU66: Crosseye
![Page 25: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/25.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
AU examples: AU7 (lid AU examples: AU7 (lid tightener)tightener)
![Page 26: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/26.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
AU examples: AU9 (nose AU examples: AU9 (nose wrinkler)wrinkler)
![Page 27: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/27.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
AU examples: AU11AU examples: AU11(nasolabial furrow deepener)(nasolabial furrow deepener)
![Page 28: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/28.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Pure and blended emotionsPure and blended emotions
7 pure emotions Pleasure (pl), surprise (su), fear (f), hate
(h), sorrow (so), disgust (d), (interest) Blended emotions
Combinations of pure emotions Easy: pl+su, h+d, so+f Difficult: su+f, d+pl, h+so, f+pl (Some) additivity
![Page 29: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/29.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Possible causes of blended expressions in real world Mixed emotions (sadness+disgust) Consequent emotions
(surprise+happiness) Masking (anger+happiness) …
![Page 30: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/30.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Attempts to add emotionsAttempts to add emotions
![Page 31: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/31.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
neu
tral
sad
su
rpri
se
![Page 32: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/32.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
6 basic expressions6 basic expressions
courtesy Jari Kätsyri
![Page 33: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/33.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
courtesy Jari Kätsyri
Blended expressionsBlended expressions
![Page 34: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/34.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Textured × non-texturedTextured × non-textured
Identification of expressions from pictures and videos of textured and non-textured version of AP
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
anger disgust fear happin. sadn. surprise
picture
picture+tex
video
video+tex
![Page 35: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/35.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Differences among databasesDifferences among databases
Number of correct responses for pictures and videos of different databases
Static
Dynamic5
6
7
8
9
10
CK HUT EF TH
![Page 36: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/36.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Neutral expressionNeutral expression
Average evaluations for AP textured and non-textured neutral expressions
![Page 37: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/37.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Better implementationBetter implementation
Collecting data
![Page 38: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/38.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Collecting data for Collecting data for expressionsexpressions
happy anger
![Page 39: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/39.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
HUT facial expression HUT facial expression databasedatabase First Finnish
database of moving facial expressions
![Page 40: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/40.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Study:Study:Interaction of blur x type x Interaction of blur x type x emotionemotion New database, 6 actors 3 levels of blur Pictures, short video sequences 6 basic expressions
3 subject groups (17 each) each group – 1 level of blur each actor 6 expressions 3 actors pictures, 3 actors videos
![Page 41: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/41.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
Interaction of blur x type x Interaction of blur x type x emotionemotion
![Page 42: Making the head smile](https://reader036.vdocuments.net/reader036/viewer/2022081519/56813f92550346895daa821e/html5/thumbnails/42.jpg)
Michael Frydrych, 11.6.2004 Laboratory of Computational Engineering
The endThe end
Thank you!
and … a little exercise again