air force research laboratory - homepage - laboratory for

1 DISTRIBUTION STATEMENT A: Approved for Public Release

Air Force Research Laboratory

Integrity Service Excellence

Trust in Automation Research: Current

Directions and Gaps

Date: 5 APR 17

Joseph Lyons, PhD Technical Advisor

Human Trust & Interaction Branch Air Force Research Laboratory


• Personal Background • Trust Background

– Definition & Measures – Trust Literature

• Auto-GCAS Project – What is it? – Survey results – Interview results – Lessons Learned

• Future Directions & Gaps – Shared awareness and shared intent

Outline


• Ph.D. I/O Psych, Minor HF Psych - Wright State University 2005

• Senior Research Psychologist for Air Force Research Laboratory – since 2005

• Research Interests: human-machine trust, interpersonal trust, leadership, Org Science

• 2011-2013 served as Program Officer for Air Force Office of Scientific Research – Chartered the Trust & Influence Portfolio – Air Force lead for Minerva

• 2013 Returned to WPAFB – started the Human Insight and Trust (HIT) Team – Dedicated to understanding human-machine trust for AF

domains – 9 Civilians 1-2 Military, 1 post-doc, approx. 6 FTE contractors

Personal Background


HIT Video


• Prior/Current HIT Research – Automated versus human aids (Lyons & Stokes, 2012) – Transparency and trust (Lyons, 2013; Lyons & Havig, 2014;

Lyons et al. 2016a) – Trust and suspicion processes (Lyons et al., 2011; Bobko et al.,

2014) – Trust of software code (Alarcon et al., in press; Hodgkin et al.,

2016) – Impact of different error types on trust (Guznov et al. 2015;

2016) – Human-Robot Interaction

• Signaling intent – Dialogue, behavior, and rationale

• Social influences (Stokes et al., 2016) • Human-Agent Teaming (Wynne & Lyons, in press)

– Pilot trust of landing aids in Degraded Visual Environments – Trust evaluations of fielded systems (Lyons et al., 2016b; Ho

et al., in press; Kolitai et al., 2014)

HIT Background


• Trust = willingness to be vulnerable w/out the ability or capacity to monitoring (Mayer et al., 1995)

• Necessary Conditions – Human intention – Relational – must have a referent (otherwise its dispositional) – Risk – without it trust is obsolete

• What to Measure? – Intention to be vulnerable – Trustworthiness

• Ability, benevolence, integrity (Mayer et al., 1995) • E.g., Machine characteristics, “Learned trust” (Hoff & Bashir, 2015)

– Reliance behavior • Must be volitional, should be in risky context – more than mere cooperation

– Dispositional trust – Physiology?

Trust Background


• Management, e-commerce, Human Factors, Robotics, Social Psych… trust research is pervasive

• Mayer et al. (1995) • Lee & See (2004) • Parasuraman & Manzey (2010) • Hancock et al (2011) • Chen & Barnes (2014) • Technology Acceptance Model • Merritt et al. (2016) • Hoff & Bashir (2015)

Trust Background… “a small sliver”


• Why is this so complicated??? – Errors for highly automated systems can be

catastrophic (Onnasch et al., 2014) • Asiana flight 214 (2013) – overreliance • Cruise ship Concordia (2012) – under-reliance

– Trust is dynamic & driven by multiple factors • Social, Affective, Cognitive, Dispositional… • Absolute values will change with changing context

– Importance of predictors may also change

– Perceptions of trustworthiness may be inaccurate – Unintended consequences (Parasuraman & Riley, 1997) – Few studies using real operators, real-world

technology & real-world consequences (R3)

The Problem – Inaccurate Trust


• Time-to-collision-based automated system – Components: Collision avoidance Algorithm, Digital

Terrain Elevation Data, interface with flight control computer, HUD interface

• Behavior: wings-roll-to-level maneuver, 5-G pull up • Objectives: do not interfere, do no harm, avoid collision

Automatic Ground Collision Avoidance System


AGCAS Video


• Why is this a good domain to study trust? – Fielded automated DoD system – high level of automation

• Supports AF interests in autonomy & future technologies • Supports pilot safety

– Real-life implications for safety • “Most sophisticated AF safety system ever fielded” • Four “saves” to date since 2014

– Trust-rich context • Potential for false alarm/miss, prior system links, imperfect reliability, novelty

– Good place to test trust antecedents – ecological validity – Engineer/test concerns

• Distrust - Too much trust • Impact of HUD display unknown • Prior warning-based systems distrusted

• Acknowledgment of team: Dr. Nhut Ho, Lauren Hoffmann, Garrett Sadler, Capt

Eric Fergueson, 1Lt Anna Lee Van Abel, Samantha Cals, Kolina Koltai, Maj Casey Richardson, Mark “Tex” Wilkins, Air Vehicles Directorate, Air Force Flight Test Center

Why Auto-GCAS?


• AF Trust research lacks data on operator trust in fielded systems with high levels of autonomy

• Four-year longitudinal study (started FY15) – Examine operational pilot trust evolution in AGCAS

overtime – Identify and document user experience, concerns,

impact, and benefits of technology as they emerge – Obtain operational pilots feedback on AFRL future

automated/autonomous systems • AACAS & Autonomous Wingman

• Uses survey and interview methods – Visited 13 bases (6 were repeat visits in year 2) – Year 1: Survey (N = 142), Interview (N = 168) – Year 2: Survey (N = 100), Interview (N = 131)

AGCAS Field Study


Survey Measures Rated on 7-point Likert Scales

Trust (4 items, α = .95) “I can count on Auto-GCAS to work when needed” Performance (3 items, α = .64) “Auto-GCAS is reliable” Transparency (3 items, α = .84) “I understand how Auto-GCAS works” Benevolence (3 items, α = .74) “I think Auto-GCAS was designed to help me” Benefits (2 items, r = .82) “I benefit from having Auto-GCAS installed on my plane” Confidence (3 items, α = .62) “I feel confident with Auto-GCAS installed on my plane” Aggressive flying (2 items, r = .75) “Auto-GCAS allows me to fly lower to the ground” Automation Schema (Merritt et al., 2016; 3 items, α = .69) “Automated systems rarely make mistakes”


Year 1 Correlations with trust (N = 142)

Predictor r Outcome r

Schema .29** Confidence .68** Performance .63** Aggressive .14 Transparency .23** Benevolence .43** Benefits .55** Note: ** p < .01.


Year 1 Unique Predictors of Trust (N = 142; Lyons et al., 2016c)

Predictor β R2

.47** Schema .13† Performance .39** Transparency .10 Benevolence .06 Benefits .25** Note: † p < .10, ** p < .01.

Consistent with Hoff & Bashir, 2015


Year 2 Correlations with trust

Predictor r Outcome r

Schema .23* Confidence .79** Performance .62** Aggressive .18 Transparency .16 Benevolence .24* Benefits .45** Note: * p < .05, ** p < .01


Year 2 Regression

Predictor β R2

.39** Schema .14 Performance .53** Transparency .05 Benevolence -.05 Benefits .10

Note: ** p < .01

Less uncertainty – diminished impact of Individual differences

Stronger performance perceptions with

“Save” stories pervasive

Likely ceiling effects for transparency, Benevolence, and benefits


Experienced Versus Novice Effects

• How does experience influence trust and aggressive flying? – Operationalized as:

• Experienced > 500 flt hrs., Novice < 500 flt hrs. – Potential for complacency

• Generational differences in automation trust and use – Mostly DoD-irrelevant “older” samples – Focused on cognitive decay – Need research on military-relevant ages


Experienced Versus Novice Effects (Lyons et al., under review)

t(92) = 2.37, p < .05.


• System performance – Operational “Save” videos

• 4 “saves” since 2014! – Perceived nuisance-free operations – Test rides via Pilot Activated Recovery System

• Business case – Need for system – Perceived benevolence via system extremely high

• Error attribution for early errors • Transparency – HUD display

– Increases SA, indicates the system is working • Pedigree of Test Community in the AF • Activations are handled as learning opportunities

Key Drivers of Trust via Interview (Ho et al., in press)


• Development – Transparency is critical – Human-centered design

• Pre-implementation – Think about the business case

• Why is it needed? • Who is advocating for it?

– Study potential barriers • Cultural, dispositional, organizational, process, similar

technologies • Understand the narrative

– Think about evaluation metrics and how you would collect them

– Educate – share performance data

Lessons Learned for Novel Technology Implementation


• Post-implementation – Own the narrative

• Share success stories/videos - seeing is believing! • Share failures

– Use metrics to evaluate success • Establish evaluation processes • Educate based on evidence • Track unintended consequences • Iterate and improve

– Anchor new behaviors in the culture – Understand that indirect knowledge can be a

powerful trust (dis) antecedent

Lessons Learned for Novel Technology Implementation


• Longitudinal research with military operators is challenging! – Access

• SME’s on the team are critical for buy-in – Attrition – Measure appropriateness/timing

• Existing trust models are robust – Performance/error types – Transparency – Individual differences – Perceived benefits – Benevolence – “intent”

Lessons Learned for Trust Research


• More applied trust work – Real operators, real tools, real consequences

• Better rationale for physiological metrics • Need more research on intent-based

transparency for Human-Machine Teams – Methods to signal positive intent – Understand the impact of intent – Methods for developed shared awareness

General Gaps & Future Directions


Transparency

• Human-Robot Transparency (Lyons, 2013)

Intent Environment Task Analytics Teamwork Human State

Social Intent


• Task performance • Predictability & performance are key facets of trust in

automation (Hancock et al., 2011)

• Capabilities and Limitations (Chen & Barnes, 2014) • Error management will be important for human-machine

teams – the ability to learn from mistakes should foster greater trust (Ososky et al., 2013)

• Logic for errors – rationale • Need to make errors predictable/understandable • Understanding analytical underpinnings supports trust

repair (Dzindolet et al., 2003)

• Assumptions • Shared contextual awareness is key for distributed robotics

(Stubbs et al., 2007) • Level of urgency, goal priorities, timeline, level of automation

Shared Awareness – what is it doing, how well, and why


• Social style • Anthropomorphic features impact trust/liking in cars (Waytz et

al., 2014) and trust of decision aids (Pak et al., 2012) • Robotic speech style impacts social distance (Kim et al., 2012) • Dialogue fosters team orientation (Fischer, 2011) • Patient interaction style influences trust & performance

(Parasuraman & Miller, 2004)

• Conferred Benevolence • Movement toward a shared goal • Back up behavior

• Understanding of consequences

• Agent-based emotional displays – appraisal process is key (De Melo et al., 2011)

• Transfer of authority

Intent – what is it going to do, how, and why


Intent – transfer of authority error

Automated Driving Technology:

“I think you should take over now”


Expectations • Appearance affords perceived function among users – may align

or conflict with expectations of users (Wright, 2012) • Spider legs vs. Wheels, Tracks (Sims et al., 2005) • Male vs female robots (Eyssel & Hegel, 2012) • Human-like faces most preferred, more social, perceived to

possess more ability (Broadbent et al., 2013) • Need to avoid too much – “uncanny valley”

• Engagement

• Robot Gaze may signify engagement (Mutlu, 2011) • Attraction

• Use of personal name (Moran et al., 2013) • Liking liked to trust (Merritt, 2011) • Polite robots are liked more (Mutlu, 2011)

Expectations and Engagement


Thanks!

Joseph Lyons, PhD [email protected]

(937) 255-8734

air force research laboratory - homepage - laboratory for

Documents