Incorporating Psychology Theories
into Simulations & Serious Games
SIFT – Smart Information Flow Technologies
Peggy Wu ([email protected]), Tammy Ott, Sonja
Schmer-Galunder, Christopher Miller, Jeff Rye,
July 2015
2
SIFT Introduction
Smart Information Flow Technologies, LLC
(SIFT) is a Consulting Research &
Development company:
Founded in 1999
Headquartered in Minneapolis MN
Offices in Boston, San Diego, Washington
DC
$7.0M+ per year in revenues
35 Full time employees
30+ Advanced Degrees (Psychology,
Computer Science, Engineering)
Advancing research to enhance
information flow in human-computer and
computer mediated human-human
interactions
Technical Personnel
27 July 2015 3
Advanced degrees in:
Artificial Intelligence
Aviation
Autonomous systems
Control Theory
Cognition
Cultural Anthropology
Linguistics
Psychology
NLP
User Experience
27 July 2015
• Intent/Plan Recognition and Task
Tracking
• Intent specification and use
• Context Sensitivity
• Adaptive and Adaptable User
Interfaces and Automation
• Mixed Initiative System Design
• Associate Systems
• Decision Aiding Systems
• RT Planning and Scheduling
• Formal and Testbench security
analysis
• Software Development
• Experimental designs & statistical
analysis
• User Interfaces and Human
Centered Interaction Design
• Anthropological analyses
• Linguistic analysis
• Modeling:
– Human and Human + System
performance
– Tasks
– Functional Representations
– Cognition
– Software Structure
– Information Flow
– Context-to-presentation Matching
– Etiquette/Politeness
– Cultural Perceptions
= Smart Information Flow Technologies
4
Capabilities
5
Customers & Collaborators
Our customers and collaborators include:
BBN, BAE, CMU, Cornell University, GMU, DARPA, Honeywell, Lockheed, NASA,
NIST, Oxford, UofM, UMD, USAF, USC, US Army, US Navy, VA
6
SIFT’s Human Behavior Modeling Work
2007 E4D2
2000 RPA
2008 SUPPORT 2010 EVA Multi Cultural Interactions
2010 GRASP– Keyboard dynamics for cybersecurity
2009+ ELADIS– affective reactions
occur to unconsciously learned
stimuli and increase recognition of
people. Utilized SCR and HR.
2013+ ANSIBLE
2012+ AD ASTRA– non-intrusive assessment of individual and team psycho-social health
2014 CAMO– zero intrusion workload detection via text and Keyboard dynamics
2015 NAPP
Independent
LifeStyle
Assistant
(ILSA)
A NIST ATP Program
2003 ILSA
2007 TLTS
2011 CALM
2009+ ADMIRE– Assessing how people in a social network feel about each other given their politeness behaviors
2006 Phrasebook
2014 R3– Reading, Remembering, Revising 2014 ATHENA– zero
intrusion workload detection
2014 SAGA
7
SIFT’s Human Behavior Modeling Work
2007 E4D2
2000 RPA
2008 SUPPORT 2010 EVA Multi Cultural Interactions
2010 GRASP– Keyboard dynamics for cybersecurity
2009+ ELADIS– affective reactions
occur to unconsciously learned
stimuli and increase recognition of
people. Utilized SCR and HR.
2013+ ANSIBLE
2012+ AD ASTRA– non-intrusive assessment of individual and team psycho-social health
2014 CAMO– zero intrusion workload detection via text and Keyboard dynamics
2015 NAPP
Independent
LifeStyle
Assistant
(ILSA)
A NIST ATP Program
2003 ILSA
2007 TLTS
2011 CALM
2009+ ADMIRE– Assessing how people in a social network feel about each other given their politeness behaviors
2006 Phrasebook
2014 R3– Reading, Remembering, Revising 2014 ATHENA– zero
intrusion workload detection
2014 SAGA
Modeling Humans for
Language & Culture Training
VR for Social &
Psychological Health
Games for Health
Human Workload Detection
Sociolinguistic Modeling
Language & Culture Training
Since 2003, developing Computational Model of Human-Human
Interaction – Etiquette EngineTM
Software Module driving individualized interpretation of politeness
and behavior selection
27 July 2015 8
“swappabl
e cultural
modules”
Visualizati
on of
Etiquette
Validation
of results
with naïve
student
ratings
27 July 2015
Etiquette, Politeness and Compliance
Grice’s sociological concept of Face
Universal, uniquely human
Brown and Levinson– cross-cultural, socio-
linguistic, human-human, politeness model
Positive Face & Negative Face
Each interaction poses “Face Threat” which is a
function of Power, Familiarity and Imposition
Politeness is used to redress Face Threat
We used this concept to create Embodied
Conversational Virtual Agents for Culture &
Language Training
What about other applications?
Natural Language Interfaces for other uses?
Does Politeness affect performance?
11
Do machines have “etiquette”? Should they?
Reeves & Nass (1996)
Humans are equipped with schema for interaction with complex agents
Complex machines activate those schema– unless we fight hard to counteract them.
Parasuraman & Miller (2004)
“Polite” (non-nagging) decision aids improve:
Trust
Perceived workload
Overall performance
Over and above automation reliability
Wu, Ott, & Miller (2009)
“Polite” Automation affects: Compliance
Trust
Perceived workload
Reaction time
0
20
40
60
80
100
HIGH LOW
GOOD COMMUNICATION STYLE
POOR COMMUNICATION STYLE
AUTOMATION RELIABILITY
Earth to Mars
56,005,100 km / 34,800,000 miles, 3min (when its
closest)
402,000,000 km / 250,000,000 miles, 24min
Current Mars DRM: 6 months, 18 months on
surface, 6 months back, plus high crew autonomy!
27 July 2015 17
27 July 2015 21
Motivation
Sensory Monotony, Social Monotony known to cause:
Adjustment disorder, Fatigue, drops in productivity,
anxiety, hostility, risk-taking and rule-breaking behaviors
Virtual Environment as Extension to Real World?
26
Brown and
Levinson (1987)
Politeness Theory
Reeves and Nass
(1996) “The Media
Equation”
Miller et al (2010)
Human Computer
Etiquette
Humans can interact with
Virtual Agents
in socially meaningful ways
Socially appropriate VA
behaviors can affect
Human Performance
Virtual Environment as Extension to Real World?
Yee & Bailenson
(2007)
“The Proteus Effect”
27 July 2015 27
Bandura (1986)
Social Cognitive
Theory “Immersive Entrainment”
Learning is a cognitive process
that occurs in a social context,
and VEs can provide the social
context, so perhaps it’s
possible to learn positive
behaviors in VEs
Virtual
Environment
with social
context
E.g.
Virtual Environment as Extension to Real World?
27 July 2015 28
Hasson (2008) “Neurocinematics”
Konigsberg (2007) Activation of Mirror Neurons
Ramachandran (2012) The Tell-Tale Brain
Iacoboni (2008) Mirroring People
29
Köhler effect: phenomenon that occurs when a person works harder as a member of a group than when working alone.
Industrial psychologist Otto Köhler found members of Berlin Rowing Club worked harder when part of a group vs. as individuals
Virtual Environment as Extension to Real World?
Operationalized Evidence Based Strategies in VE
Combat Sensory Monotony
• Weather, Lighting, Sun Rise/Set
• Virtual Plant life, Nature Scenes
• Virtual spaces for crew discretionary events,
providing different visual stimuli
Combat Social Monotony
• Virtual pets
• Virtual agents as actors
• People watching tied to Earth
• Vary types of interactions e.g. support personal
pursuits and hobbies (3D modeling)
Recall positive memories, memories of gratitude, acts
of kindness (Lyubomirski)
• Game - pictionary for shared memories
• Scavenger Hunt
• Virtual care package
Interpersonal skills training
• Motivational interviewing
• VA to consult and provide guidance on avoiding
global criticisms, contempt, defensiveness,
stonewalling etc.
Continuing rituals (Xygalastas et el., 2011)
• Birthdays, thanksgiving, christmas
Shared experiences, laughter and humor (Cousins,
1976)
• Virtual vacations
• Comedy club
• Turn taking games
Meaningful work: reflecting on past and future,
expectations and responsibilities (Baumeister et al.,
forthcoming, Frankl, 1991)
• Public Interaction area
• Educational outreach
• Music, Creative expressions, personal
pursuits
Mere belonging to increase connectedness
• Sharing random/superficial commonalities
e.g. Sports jersey
Mindfulness and meditation
Review of 80+ publications, seeking evidence based strategies for promoting
psychosocial health, brainstormed implementation ideas for VE+COMM delay
30
“If we knew what we were doing it
wouldn’t be research.”
- Albert Einstein
27 July 2015 31
Is it going to work?
Validation Testing
HISEAS: Hawaii Space Exploration Analog and Simulation
Hawaii actually looks a lot like Mars…
27 July 2015 32
HI-SEAS
Missions 1 & 2: 4 months
Mission 3: 8 months (Oct 2014-June 2015)
Control group for ANSIBLE
Mission 4: 12 months
ANSIBLE group
30min 3x/week
27 July 2015 33
Surveys and Measures (Crew & Family)
Pre and Post:
- STAI (20 Qs)
- Sensory Seeking Survey (40 Qs)
- Outgoing debrief (30min)
In hab (both crew & designated family/friend):
- Modified Circles of Closeness (3x/week 4 Qs ~1min)
- Connectedness & Sensory (3x/week 11 Qs ~2min)
- Journal + 6 questions (3x/week ~25min)
- Perceived Stress Questionnaire (weekly 30 Qs ~8min)
- Modified Social Support (SF) (weekly 18 Qs ~3min)
- Usage (ANSIBLE group only)
- Earth-Space communications in open forum
- PANAS (daily, crew only)
- Sleep (3x week)
- HRV
- Cortisol
34