SENSORY INTEGRATION RESEARCH

Kristen K. Liggett, Ph.D.Crew Systems Engineer

Human Effectiveness Directorate

Air Force Research Laboratory

2

“Say, What’s A Mountain Goat Doing Way Up

Here in the Clouds?”

Spatial Disorientation

3

Class A Mishap Rates (1972-2001)

Total Rate

Operations

Spatial Disorientation0

0.5

1

1.5

2

2.5

3

Year

1972

1973

1974

1975

1976

1977

1978

1979

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

Mis

hap

Rat

e p

er 1

00K

Fly

ing

Ho

urs

SD IS STILL A KILLER!

SD Class A Mishap Rate is Largely Unchanged from 1970s!

4

Spatial Disorientation CounterMeasures Program

• Managed by the Joint Cockpit Office (AFRL/HEM)

• Five Year Effort (FY01 – FY05)

• Budget: $10M

Goal

Reduce the loss of personnel and aircraft

caused by SD of aircrew members

• by a measurable amount within three years

• by 50% by 2010.

5

Spatial Disorientation CounterMeasures Program

• Objective #1 – Training

– Improve training materials and methods

• Objective #2 – Displays

– Improve displays and information transfer

• Objective #3 – Mechanisms

– Improve understanding (modeling) of human perception of orientation

*Transfer research outcomes - www.spatiald.wpafb.af.mil*

6

Sensory Integration Research

7

Sensory Integration Technologies

• Why visual displays?

– Vision DOMINATES spatial orientation

– Visual dominance/vestibular suppression

• Why auditory displays?

– Acoustic attitude indicator

– Enhancement in target acquisition and spatial awareness

• Why tactile displays?

– Sensitivity/discrimination

– Convey directional information faster than audio cues, with some trade off in accuracy

8

TSAS Video

9

• New area of research

– Lessons learned?

• Obtaining tactile display

– TSAS is unavailable in the near-term

– Johnson Kinetics – Electrotactile

• Optimal Integration

– Many options and combinations to try

• Objective measure of SD

– “If you can measure it, you can manage it”

– Directly connecting study results with SD mishaps

Sensory Integration Research

Research Challenges

10

Sensory Integration Research

Study #1• Test bed: Single-Seat Fighter Simulator in CSIL

• Technologies: HMD, non-localized voice commands, and 3-D localized audio cues.

• Task: Unusual attitude recognition and recovery

• Independent Variables: 3 levels of display configuration (visual only, visual + voice commands, and visual + 3-D audio cues); 2 levels of initial pitch condition (nose high and nose low)

The Graveyard Spin

11

Sensory Integration Research

Study #1 (Continued)• Hypotheses:

• The combination of visual and audio cueing will result in faster recognition times than using visual symbology only

• Nose low initial pitch conditions will result in faster recovery times

• Experimental Design: 3 x 2 Within-Subjects Design

• Dependent Variables: time to initial correct stick input, total recovery time, control reversal errors, absolute altitude change

• Subjects: 8 military pilots with HUD experience

• Duration of Study: 1.5 – 2 hours/subject

12

Flight Simulator

13

Unusual Attitude Recovery Experiment

Results

1.276

1.248

1.254

2903.568

4013.208

0

500

1000

1500

2000

2500

3000

3500

4000

4500

Nose High Nose Low

Ab

solu

te A

ltit

ud

e C

han

ge

(Ft)8.239

6.793

0

1

2

3

4

5

6

7

8

9

Nose High Nose Low

To

tal R

eco

very

Tim

e (S

ecs)

1.2961.258 1.262

1

1.05

1.1

1.15

1.2

1.25

1.3

1.35

1.4

1.45

1.5

Visual Only Visual Plus Non-LocalizedVerbal Commands

Visual Plus LocalizedAudio Tones

Tim

e to

Init

ial C

orr

ect

Inp

ut

(Sec

s)

14

Discussion

• Objective Data: No significant difference between attitude symbology sets

• Subjective Data: 7 our of 8 subjects preferred some form of auditory cueing in additional to visual information

• Potential Reason for Lack of Performance Difference:

– Subjects had more experience (on average - 1652 hours) with visual symbology

– Subjects had less experience (on average – 15 minutes) with audio symbology

15

Study #1 Part 2

• Collect data from general aviation pilots.

• Analysis pending.

• Incorporate lessons learned from Study #1 for visual and audio symbology.

• Integrate tactile symbology.

• Conduct research with the combined sensory inputs.

• Transition from UA task to maintaining orientation to address unrecognized SD.

16

DES Centrifuge as Flight Simulator

17

OV-10 Flight Test Vehicle

18

Technology Transfer - JSF

• HMD for attitude information

– Primary Flight Reference ()

– Orientation Symbology (?)

• 3-D Audio

– Targeting ()

– Orientation (?)

• TSAS (?)

– JSF/TSAS Flight Demonstration in Sep 97

– Integrated TSAS, F-22 Cooling Vest, GPS/INS, and UH-60 Helicopter

19

Technology Transfer - AFSOC

• Critical Mission Needs Statement

• Visual is inadequate for hovering

• Tactile necessary (TSAS)

• SDCM lead for Human Factors

• Coordinated with Army to usesimulator

• Flight Test Planned

• Transition to CV-22

20

Technology Transfer - Other

• DAGSI Research (Jul 01 – Jul 03)

• FY03 SBIR

• FY03 IRI with Dr. Sharon Holmes of Qinetiq, UK

• FY04 DUS&T

– General Aviation Interest

• John F. Kennedy Jr. Accident attributed to SD

• Senator Mel Carnahan Accident attributed to SD

• 90% of general aviation accidents attributed to SD are fatal

– Commercial Aviation Interest

• CFIT accidents

21

It Could Be That The Purpose of Your Life Is

Only to Serve As A Warning To Others.