virtual human research at usc’s human research at usc’s institute for creative technologies...
TRANSCRIPT
![Page 1: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/1.jpg)
![Page 2: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/2.jpg)
Virtual Human Research at USC’s
Institute for Creative Technologies
Jonathan GratchDirector of Virtual Human Research
Professor of Computer Science and Psychology
University of Southern California
The work depicted here was sponsored by the U.S. Army. Statements and opinions expressed do not
necessarily reflect the position or the policy of the United States Government, and no official endorsement
should be inferred.
Images and videos from USC Institute for Creative Technologies
![Page 3: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/3.jpg)
3
Virtual HumansAutonomous virtual characters that have meaningful social
interactions with human users▪ Reason about environment
▪ Understand and express emotion
▪ Communicate through speech & gesture
▪ Play the role of teachers, peers, competitors
![Page 4: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/4.jpg)
4
What’s the point?
![Page 5: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/5.jpg)
5
Computer Science Motivation:Challenge problem to advance artificial social intelligence
Gratch, DeVault, Lucas, Marsella. Negotiation as a Challenge Problem for Virtual Humans. 15th International Conference on Intelligent Virtual Agents. Delft, The Netherlands. 2015
![Page 6: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/6.jpg)
6
▪ User recognition
▪ What is she saying?
▪ What is her expression?
▪ What is her posture?
Computer Science Motivation:Challenge problem to advance artificial social intelligence
Gratch, DeVault, Lucas, Marsella. Negotiation as a Challenge Problem for Virtual Humans. 15th International Conference on Intelligent Virtual Agents. Delft, The Netherlands. 2015
![Page 7: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/7.jpg)
7
▪ User understanding
▪ What does she want?
▪ What does she feel?
▪ Is she lying?
▪ What will she do next?
▪ User influence
▪ How to establish rapport?
▪ How to seem trustworthy?
▪ How to evoke emotion?
▪ How to lie?
▪ How to win?
Computer Science Motivation:Challenge problem to advance artificial social intelligence
Gratch, DeVault, Lucas, Marsella. Negotiation as a Challenge Problem for Virtual Humans. 15th International Conference on Intelligent Virtual Agents. Delft, The Netherlands. 2015
![Page 8: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/8.jpg)
8
de Melo, Carnevale, Read and Gratch. Reading people’s minds from emotion expressions in interdependent decision making. Journal of Personality and
Social Psychology, 106(1), 2014, pp. 73-88
Psychological Motivation:Tools to advance theories of human social cognition
![Page 9: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/9.jpg)
9
Bilge Karacora, Morteza Dehghani, Nicole Krämer and Jonathan Gratch. The Influence of Virtual Agents’ Gender and Rapport on Enhancing Math
Performance. 34th Annual Meeting of the Cognitive Science Society. 2012
Psychological Motivation:Tools to advance theories of human social cognition
![Page 10: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/10.jpg)
10
Celso de Melo, Peter Carnevale and Jonathan Gratch. Using Virtual Confederates to Research Intergroup Bias and Conflict. 74th Annual Meeting of the
Academy of Management. Philadelphia, PA, 2014
Jonathan Gratch. Understanding the mind by simulating the body: virtual humans as a tool for cognitive science research. The Oxford Handbook of
Cognitive Science. Susan Chipman (ed.). Oxford University Press
▪ Virtual Confederates
Allow systematic manipulation of
▪ Race
▪ Gender
▪ Accent
▪ Expressions
▪ Behavior
Psychological Motivation:Tools to advance theories of human social cognition
![Page 11: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/11.jpg)
11
Application Motivation:Unique capabilities to advance education and health
![Page 12: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/12.jpg)
12
David DeVault, Ron Artstein, Grace Benn, Teresa Dey, Alesia Egan, Ed Fast, Kallirroi Georgila, Jonathan Gratch, Arno Hartholt,
Margaux Lhommet, et al. SimSensei: A Virtual Human Interviewer for Healthcare Decision Support. 13th International
Conference on Autonomous Agents and Multiagent Systems. Paris, France 2014
Application Motivation:Unique capabilities to advance education and health
![Page 13: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/13.jpg)
13
How to build it?
![Page 14: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/14.jpg)
14
Learning (LDCR)
PROSODY
GESTURE
EXPRESSION
PROSODY
GESTURE
EXPRESSION
Capture
Data or Theory?
Capture
Prediction Model
e.g., Rapport Agent
Morency, de Kok and Gratch. A Probabilistic Multimodal Approach for Predicting Listener Backchannels. JAAMAS, Springer, 20(1), 2009, 70-84
![Page 15: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/15.jpg)
15
▪ I’m biased towards theory
▪ A neural network is not a theory
▪ Offers limited explanatory value
▪ Challenges in generalizing to other contexts
▪ Theories have certain advantages
▪ Emphasize underlying mechanism
▪ Have good empirical support
▪ Indicate how to bring about change
Data or Theory?
![Page 16: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/16.jpg)
16
▪ Data tells us about the world as it is
▪ Useful if want to recreate reality
▪ e.g., make virtual humans that are identical to real people
▪ I’m more interesting in creating something better than people
▪ Theories can point the way
Data or Theory?
![Page 17: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/17.jpg)
17
Application Motivation:Unique capabilities to advance education and health
Lucas, Gratch, King and Morency. It's Only a Computer: Virtual Humans Increase Willingness to Disclose. Journal of Computers
in Human Behavior. v37. 2014. pp. 94-100
![Page 18: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/18.jpg)
![Page 19: Virtual Human Research at USC’s Human Research at USC’s Institute for Creative Technologies Jonathan Gratch Director of Virtual Human Research Professor of Computer Science and](https://reader030.vdocuments.net/reader030/viewer/2022020412/5ae4297b7f8b9a5d648ef4df/html5/thumbnails/19.jpg)