Incorporating Psychology Theories

into Simulations & Serious Games

SIFT – Smart Information Flow Technologies

Peggy Wu (PWu@siftech.com), Tammy Ott, Sonja

Schmer-Galunder, Christopher Miller, Jeff Rye,

July 2015

2

SIFT Introduction

Smart Information Flow Technologies, LLC

(SIFT) is a Consulting Research &

Development company:

Founded in 1999

Headquartered in Minneapolis MN

Offices in Boston, San Diego, Washington

DC

$7.0M+ per year in revenues

35 Full time employees

30+ Advanced Degrees (Psychology,

Computer Science, Engineering)

Advancing research to enhance

information flow in human-computer and

computer mediated human-human

interactions

Technical Personnel

27 July 2015 3

Advanced degrees in:

Artificial Intelligence

Aviation

Autonomous systems

Control Theory

Cognition

Cultural Anthropology

Linguistics

Psychology

NLP

User Experience

27 July 2015

• Intent/Plan Recognition and Task

Tracking

• Intent specification and use

• Context Sensitivity

• Adaptive and Adaptable User

Interfaces and Automation

• Mixed Initiative System Design

• Associate Systems

• Decision Aiding Systems

• RT Planning and Scheduling

• Formal and Testbench security

analysis

• Software Development

• Experimental designs & statistical

analysis

• User Interfaces and Human

Centered Interaction Design

• Anthropological analyses

• Linguistic analysis

• Modeling:

– Human and Human + System

performance

– Tasks

– Functional Representations

– Cognition

– Software Structure

– Information Flow

– Context-to-presentation Matching

– Etiquette/Politeness

– Cultural Perceptions

= Smart Information Flow Technologies

4

Capabilities

5

Customers & Collaborators

Our customers and collaborators include:

BBN, BAE, CMU, Cornell University, GMU, DARPA, Honeywell, Lockheed, NASA,

NIST, Oxford, UofM, UMD, USAF, USC, US Army, US Navy, VA

6

SIFT’s Human Behavior Modeling Work

2007 E4D2

2000 RPA

2008 SUPPORT 2010 EVA Multi Cultural Interactions

2010 GRASP– Keyboard dynamics for cybersecurity

2009+ ELADIS– affective reactions

occur to unconsciously learned

stimuli and increase recognition of

people. Utilized SCR and HR.

2013+ ANSIBLE

2012+ AD ASTRA– non-intrusive assessment of individual and team psycho-social health

2014 CAMO– zero intrusion workload detection via text and Keyboard dynamics

2015 NAPP

Independent

LifeStyle

Assistant

(ILSA)

A NIST ATP Program

2003 ILSA

2007 TLTS

2011 CALM

2009+ ADMIRE– Assessing how people in a social network feel about each other given their politeness behaviors

2006 Phrasebook

2014 R3– Reading, Remembering, Revising 2014 ATHENA– zero

intrusion workload detection

2014 SAGA

7

SIFT’s Human Behavior Modeling Work

2007 E4D2

2000 RPA

2008 SUPPORT 2010 EVA Multi Cultural Interactions

2010 GRASP– Keyboard dynamics for cybersecurity

2009+ ELADIS– affective reactions

occur to unconsciously learned

stimuli and increase recognition of

people. Utilized SCR and HR.

2013+ ANSIBLE

2012+ AD ASTRA– non-intrusive assessment of individual and team psycho-social health

2014 CAMO– zero intrusion workload detection via text and Keyboard dynamics

2015 NAPP

Independent

LifeStyle

Assistant

(ILSA)

A NIST ATP Program

2003 ILSA

2007 TLTS

2011 CALM

2009+ ADMIRE– Assessing how people in a social network feel about each other given their politeness behaviors

2006 Phrasebook

2014 R3– Reading, Remembering, Revising 2014 ATHENA– zero

intrusion workload detection

2014 SAGA

Modeling Humans for

Language & Culture Training

VR for Social &

Psychological Health

Games for Health

Human Workload Detection

Sociolinguistic Modeling

Language & Culture Training

Since 2003, developing Computational Model of Human-Human

Interaction – Etiquette EngineTM

Software Module driving individualized interpretation of politeness

and behavior selection

27 July 2015 8

“swappabl

e cultural

modules”

Visualizati

on of

Etiquette

Validation

of results

with naïve

student

ratings

27 July 2015 9

27 July 2015

Etiquette, Politeness and Compliance

Grice’s sociological concept of Face

Universal, uniquely human

Brown and Levinson– cross-cultural, socio-

linguistic, human-human, politeness model

Positive Face & Negative Face

Each interaction poses “Face Threat” which is a

function of Power, Familiarity and Imposition

Politeness is used to redress Face Threat

We used this concept to create Embodied

Conversational Virtual Agents for Culture &

Language Training

What about other applications?

Natural Language Interfaces for other uses?

Does Politeness affect performance?

11

Do machines have “etiquette”? Should they?

Reeves & Nass (1996)

Humans are equipped with schema for interaction with complex agents

Complex machines activate those schema– unless we fight hard to counteract them.

Parasuraman & Miller (2004)

“Polite” (non-nagging) decision aids improve:

Trust

Perceived workload

Overall performance

Over and above automation reliability

Wu, Ott, & Miller (2009)

“Polite” Automation affects: Compliance

Trust

Perceived workload

Reaction time

0

20

40

60

80

100

HIGH LOW

GOOD COMMUNICATION STYLE

POOR COMMUNICATION STYLE

AUTOMATION RELIABILITY

NASA’s Vision for Deep Space Exploration

12

Earth to ISS

354 km / 220 miles.

No COMM delay, virtually always connected.

27 July 2015 13

Mission Control flies ISS, not Astronauts

27 July 2015 14

Earth to Moon

383,000 km / 238,000 miles

1.2s one way delay

27 July 2015 15

Earth to NEO

Asteroid 951 Gaspra

1,929,378 km / 1,198,860 miles

6.5s one way delay

27 July 2015 16

Earth to Mars

56,005,100 km / 34,800,000 miles, 3min (when its

closest)

402,000,000 km / 250,000,000 miles, 24min

Current Mars DRM: 6 months, 18 months on

surface, 6 months back, plus high crew autonomy!

27 July 2015 17

Orion Interior?

27 July 2015 18

27 July 2015 19

27 July 2015 20

27 July 2015 21

Motivation

Sensory Monotony, Social Monotony known to cause:

Adjustment disorder, Fatigue, drops in productivity,

anxiety, hostility, risk-taking and rule-breaking behaviors

ANSIBLE

27 July 2015 22

27 July 2015 23

ANSIBLE World Map

ANSIBLE

27 July 2015 24

27 July 2015 25

Virtual Environment as Extension to Real World?

26

Brown and

Levinson (1987)

Politeness Theory

Reeves and Nass

(1996) “The Media

Equation”

Miller et al (2010)

Human Computer

Etiquette

Humans can interact with

Virtual Agents

in socially meaningful ways

Socially appropriate VA

behaviors can affect

Human Performance

Virtual Environment as Extension to Real World?

Yee & Bailenson

(2007)

“The Proteus Effect”

27 July 2015 27

Bandura (1986)

Social Cognitive

Theory “Immersive Entrainment”

Learning is a cognitive process

that occurs in a social context,

and VEs can provide the social

context, so perhaps it’s

possible to learn positive

behaviors in VEs

Virtual

Environment

with social

context

E.g.

Virtual Environment as Extension to Real World?

27 July 2015 28

Hasson (2008) “Neurocinematics”

Konigsberg (2007) Activation of Mirror Neurons

Ramachandran (2012) The Tell-Tale Brain

Iacoboni (2008) Mirroring People

29

Köhler effect: phenomenon that occurs when a person works harder as a member of a group than when working alone.

Industrial psychologist Otto Köhler found members of Berlin Rowing Club worked harder when part of a group vs. as individuals

Virtual Environment as Extension to Real World?

Operationalized Evidence Based Strategies in VE

Combat Sensory Monotony

• Weather, Lighting, Sun Rise/Set

• Virtual Plant life, Nature Scenes

• Virtual spaces for crew discretionary events,

providing different visual stimuli

Combat Social Monotony

• Virtual pets

• Virtual agents as actors

• People watching tied to Earth

• Vary types of interactions e.g. support personal

pursuits and hobbies (3D modeling)

Recall positive memories, memories of gratitude, acts

of kindness (Lyubomirski)

• Game - pictionary for shared memories

• Scavenger Hunt

• Virtual care package

Interpersonal skills training

• Motivational interviewing

• VA to consult and provide guidance on avoiding

global criticisms, contempt, defensiveness,

stonewalling etc.

Continuing rituals (Xygalastas et el., 2011)

• Birthdays, thanksgiving, christmas

Shared experiences, laughter and humor (Cousins,

1976)

• Virtual vacations

• Comedy club

• Turn taking games

Meaningful work: reflecting on past and future,

expectations and responsibilities (Baumeister et al.,

forthcoming, Frankl, 1991)

• Public Interaction area

• Educational outreach

• Music, Creative expressions, personal

pursuits

Mere belonging to increase connectedness

• Sharing random/superficial commonalities

e.g. Sports jersey

Mindfulness and meditation

Review of 80+ publications, seeking evidence based strategies for promoting

psychosocial health, brainstormed implementation ideas for VE+COMM delay

30

“If we knew what we were doing it

wouldn’t be research.”

- Albert Einstein

27 July 2015 31

Is it going to work?

Validation Testing

HISEAS: Hawaii Space Exploration Analog and Simulation

Hawaii actually looks a lot like Mars…

27 July 2015 32

HI-SEAS

Missions 1 & 2: 4 months

Mission 3: 8 months (Oct 2014-June 2015)

Control group for ANSIBLE

Mission 4: 12 months

ANSIBLE group

30min 3x/week

27 July 2015 33

Surveys and Measures (Crew & Family)

Pre and Post:

- STAI (20 Qs)

- Sensory Seeking Survey (40 Qs)

- Outgoing debrief (30min)

In hab (both crew & designated family/friend):

- Modified Circles of Closeness (3x/week 4 Qs ~1min)

- Connectedness & Sensory (3x/week 11 Qs ~2min)

- Journal + 6 questions (3x/week ~25min)

- Perceived Stress Questionnaire (weekly 30 Qs ~8min)

- Modified Social Support (SF) (weekly 18 Qs ~3min)

- Usage (ANSIBLE group only)

- Earth-Space communications in open forum

- PANAS (daily, crew only)

- Sleep (3x week)

- HRV

- Cortisol

34

Experiences or Things you miss most

27 July 2015 35

PWu@Siftech.com

27 July 2015 36