the man-machine integration design & analysis system (midas): recent improvements sandra g. hart...
TRANSCRIPT
The Man-Machine Integration Design & Analysis System (MIDAS):
Recent Improvements
Sandra G. Hart
Brian F. Gore
Peter A. Jarvis
NASA Ames Research Center
Moffett Field, CA 94035
[email protected]/650 604 6072
10/19/04
SGH 10/04-2
Outline
Human Performance Modeling
MIDAS Phase 1: Initial design
Early applications
MIDAS Phase 2: Move from Lisp to C++
Recent applications
MIDAS Phase 3: PC Port/Integrate Apex
SGH 10/04-3
Human Performance Models: Components
Psychological Models
ProceduralModels
Vehicle Models
Equipment Models
Sensory Models
Anthropometric Models
Environment Models
Team/Org Models
Biodynamic Models
Timeline
Task Network
Performance: WL
Performance: Time
Performance: SA
Performance: Errors
Visualization
FoV/Reach Envelope
Model Architecture, Library, Tools
SGH 10/04-4
Human Performance Models: Architectures
Task network: Top-down, based on sequences of human/system tasks (derived from task analysis
MicroSaint WinCrew Crewcut
IPME IMPRINT
ACT-R MIDAS/ AirMIDAS D-OMAR
Soar APEX SAMPLE
Cognitive: Bottom-up, combine theory-based models of memory, decision making, perception, attention, movement, etc
Vision: Computational representation of the way the human visual system processes an image to predict performance given image characteristics
ORACLE NASA Standard Visual Observer
NASA Text Visibility
Optimetrics Visual Perf Model
Georgia Tech Vis Model
Anthropometric/Biodynamic: Physical characteristics of human body; static & dynamic; population characteristics; limitations [RAMSIS, JACK]
Psychological theories, mathematical models, descriptive functions
SGH 10/04-5
Human Performance Models can…
Generate hardware, software, training requirements for tasks that will involve human operators
Depict operators performing tasks in prototype workspaces and/or in remote or risky environments
Perform tradeoff analyses among alternative designs and candidate procedures, saving time and money
Identify general human/system vulnerabilities to estimate overall system performance and reliability
Provide dynamic, animated examples for training and developers
Generate realistic schedules and procedures
SGH 10/04-7
OverviewA comprehensive suite of computational tools - - 3D rapid prototyping, models of perception, cognition, response, real- and fast-time simulation, performance analysis, visualization - - for designing and analyzing human/machine systems was developed primarily in Lisp on a fleet of SGIs
Run Time Visualization
Data AnalysisData Input
SGH 10/04-8
Features Pioneered the development of an engineering design
environment with integrated tools for rapid prototyping, visualization, simulation and analysis
Advanced the capabilities and use of computational representations of human performance in design including a state of the art anthropometric model (Jack®)
Flexible enough to support a range of potential users and target applications
But…. Component models written in Lisp, Fortran, C, C++ Required a suite of SGI machines Modeled a single operator Time based rather than event based; scheduler established
optimal inter-leaving of task components No emergent behaviors
SGH 10/04-9
Richmond, CA Police: 911 Dispatch
Goal: Upgrade the facilities and procedures used in the 911 dispatch facility
Accomplished: Modeled control console
and dispatch activities in MIDAS
Evaluated prototype graphical decision aid
SGH 10/04-10
US Army Air WarriorGoal: Establish baseline performance measures for crews flying Longbow Apache with and without MOPP gear
Accomplished: Modeled copilot/gunner with Jack® (95th male <> 5th female) Rendered cockpit using CAD files from manufacturer Simulated performance of more than 400 activities Measured reach, FoV, workload, timelines
SGH 10/04-11
Short Haul/Civil Tiltrotor
Goal: Evaluate crew performance/workload issues for steep (9º), noise abatement approaches into a vertiport
Accomplished: Constructed MIDAS
models of normal and aborted approaches
Contrasted impact of manual vs automated nacelle control modes
SGH 10/04-12
NASA Shuttle Upgrade
Goal: Support development of an advanced orbiter cockpit with an improved display/control design
Accomplished: Created virtual rendition of current shuttle cockpit Conducted simulation of first 8 min of nominal ascent Provided quantitative measures of workload/SA, timing
SGH 10/04-14
Features
Decreased model development from months to weeks Increased run-time efficiency from 50x RT to near RT Multiple operators Modeled external vision, audition, situation awareness Conditional behaviors emerging from interaction of
top-down goals and environmentally driven contexts Option of non-proprietary “head & hands” model
But… The interface still user un-friendly SGI platform Cognitive models no longer state of the art Performance moderating functions not integrated
SGH 10/04-16
User Interface The interactive graphical user interface is used to create
models, specify and run simulations, and view data. It is organized into a hierarchical series of screens or editors that are navigated with tabs
Different views of the simulation are offered: Structure, Geometry, Outline, Animation, Real-time/post-hoc data
SGH 10/04-17
Vehicle Models A modeled vehicle represents the combination of a guidance/
dynamics model and a visual representation The guidance/dynamics model moves the vehicle along a
prescribed route. MIDAS provides two:NoE helicopter modelSimple point mass model
(used to model arbitrary vehicles in a generic way)
The visual representation is CAD geometry chosen from the MIDAS library or developed by the modeler.
SGH 10/04-18
Environment Model Tools are provided to model the environment outside the crew
station (e.g., terrain, weather, etc) Terrain is modeled as a single object Features are simple objects that have no inherent behavior and do
not move One weather condition may be applied to the environment by
specifying lighting/visibility (these are used by the visual perception model)
SGH 10/04-19
Crew Station/Equipment Models The “crew station” is a collection of equipment with which operators interact Crew station models may
be given a graphical representation for animation
Multiple crew stations per vehicle and multiple operators per crew station possible
QuickTime™ and aVideo decompressor
are needed to see this picture.
SGH 10/04-20
Anthropometric Models
Anthropometric models provide an animated, 3D graphical representation of one or more modeled human operators for visualization
Jack ® (developed at U Penn/distributed by UGS): full-body figure & realistic movements
Head and Hands model: government-developed representation adequate for many purposes for users without a Jack license
SGH 10/04-21
Vision ModelsVisual attention modeled as single “cone”, varying from 3-15º based on task type.
External vision:Peripheral: 160 degreesFoveal: 2.5 degrees
Perceivability: f(visibility, size, distance, local contrast ratio)Perception level: f(dwell time, perceivability)Levels of Perception:
DetectionRecognitionIdentification
Internal vision: Symbolic (check read)Digital (exact value)Text (character string)
SGH 10/04-22
Auditory Model
Only within crew stationExternal sounds are represented only if channeled through equipment
Two Stages of Processing:DetectionComprehension
Content: Verbal stringsSignals
All or none processing (Interruptions disrupt entire message)
SGH 10/04-23
Symbolic Operator Models Significant advance over earlier version, which required
specification of all activities at primitive task level High-level scripting language, Operator Procedure Language
(OPL) serves as front-end to a reactive planning system (RAPS) User-supplied procedures are instructions for accomplishing tasks Manages knowledge and beliefs, integrates human actions with
scenario events
SGH 10/04-24
Simpler model than in MIDAS 1.0 Distinction between long-term/short-term memory was lost Memories are represented as a database of assertions or beliefs
that are symbolic expressions describing the property of objects
Memory can be examined by powerful tools in a querying
language built into OPL
Memory Model
SGH 10/04-25
Attention Model Based on Wicken’s Multiple Resource Model. Acts as a mediator that maintains an account of attention
resources in six different “channels” Necessary attention resources must be available before primitive
tasks are initiated Task onset may be delayed if insufficient resources
MANUAL
VOCAL
ENCODING RESPONDING
CENTRALPROCESSING
VISUAL
AUDITORY
SPATIAL
VERBAL
SPATIAL
VERBAL
CODES
MO
DA
LIT
IES
STAGES
RESPONSES
SGH 10/04-26
Output Behavior Models
If required resources are available an activity that corresponds to a primitive procedure is created
Physical actions and their effects on equipment or environmental objects are modeled, regulated by a motor control process
60+ primitive tasks are available in a Procedure Library with pre-defined load values; easy to add more
Visual Auditory CognitiveSpatial
Cognitive Verbal
Manual Vocal
Estimate Time 0 0 0 2.0 0 0
Visual monitor 5.4 0 6.8 0 0 0
Type (1 hand) 5.9 0 0 5.3 7.0 0
Say message 0 0 0 5.3 1.0 4.5
Move object 5.0 0 1.0 0 2.6 0
SGH 10/04-27
Simulation System Engine/executive (uses discrete-event, fixed-time increment
approach for advancing the simulation) Data collection mechanisms for generating runtime data that is
graphically displayed which the simulation runs and is saved for post-run analyses
Event generation mechanism provides a way for timed events to occur on schedule or with stochastic variance
Provisioning system allows users to change the simulation and re-run without re-loading/re-starting
SGH 10/04-28
Workload & Situation AwarenessWorkload calculations based on McCracken & Aldrich (1988)
Load levels for Visual, Auditory, Cognitive, and Psychomotor dimensions are defined for task primitives on a scale of 1-7
Momentary load based on aggregation
SA calculations based on:Ratio of operator’s relevant knowledge/required knowledgeDistinguishes actual SA from perceived SA
Situational elements can be objects in the crew station or the environment that define a “situation” or are in the operator’s memory and are operationally relevant.
WL and SA values offer a powerful way to simulate realistic errors
QuickTime™ and aVideo decompressor
are needed to see this picture.
SGH 10/04-29
Validation: Search & Rescue Mission
QuickTime™ and aMicrosoft Video 1 decompressorare needed to see this picture.
SGH 10/04-30
Comparison of Models to Simulator Data
Nominal baseline approach/landing and late runway reassignment (sidestep) with and without SVS display
850' Ceiling
1000' Lineup on Final
Parallel Runways
650' Missed Approach
ATC-Commanded Runway Sidestep
Preliminary timeline, SA, attention, wkld, analysis,task execution times error vulnerabilities
ACT-R/PM
U of Illinois
Rice University
Air MIDAS
San Jose State University
A - SA
U of Illinois
IMPRINT/ACT-R
MAAD & Carnegie Mellon
D-OMAR
BBN Technologies
MIDASNASA-ARC/Army
Goodness-of-fit of individual model outputs to empirical data
SGH 10/04-31
Nominal Approach & Landing Simulation PF scanning for TFX, runway PNF monitoring PFD, Nav PF/PNF monitoring radio Flaps 30º/set & confirm PF requests before landing
checklist PNF checks/responds hear
down PF confirms visually/verbally PNF checks/responds flaps 30 PF confirms visually/verbally PNF checks/responds speed
brakes set PF confirms visually/verbally PNF declares checklist
complete PF sets/declares DA at 650 PNF visually confirms DA set Note passing FAF Confirms final descent initiated
QuickTime™ and aVideo decompressor
are needed to see this picture.
SGH 10/04-32
Traffic Call During Approach Final approach checklist
is complete ATC call with traffic
advisory Both pilots scan for
traffic “I don’t see it” Neither pilot notices as
the decision altitude is passed
After the fact, the First Officer notices: “We’re past FAF and not descending”
Crew must decide whether to continue with the approach or abort
QuickTime™ and aVideo decompressor
are needed to see this picture.
SGH 10/04-33
Life Sciences Glove BoxVirtual Glovebox
Onboard KC-135
Life Sciences Glovebox Payload Development Unit received at Ames from the National Aerospace
Development Agency of Japan (NASDA)
MIDAS rendering
Challenges: Astronauts must follow detailed instructions within
strict time constraints; failure to do so introduces risk of science mission failure
Role of Computational ModelingPredict interactive influences of microgravity
(posture, bracing, precise movements, placing, moving, stowing) to develop/evaluate procedures
Watching an animated dry run enables efficient communication among scientists, implementers, astronauts; more effective training
SGH 10/04-34
Life Sciences Glove Box Simulation
Goal: Predict astronauts’ performance of complex experiments designed to answer questions about living organisms’ adaptation to the space environment
Objectives: Evaluate feasibility of following proposed procedures within time/performance constraints; ID factors that will increase risk of mission failure [e.g., waiting too long to photograph slides; interruptions; task requires (unavailable) resource(s)]
The Task: Turn on experimental equipment (monitor, microscope, camera) Measure cell density/viability for each of 6 samples
• Invert sample vial• Place aliquot of sample on slide• Place drop of viability stain in sample• Record time on sample record• Place cover slip on slide• Observe on microscope• Take photographs within specific time window
Dispose of trash, return vials to containers, turn equipment off
SGH 10/04-35
Cell Staining/Photographing Experiment
QuickTime™ and aVideo decompressor
are needed to see this picture.
SGH 10/04-37
MIDAS v3.0 Features
Runs on high-end PC Simple model of microgravity influence on performance Physics model of microgravity impact on objects available Simple within-task fatigue model implemented Fatigue state model (U Penn/Astronaut Scheduling Assistant)
selected Notion of task duration - - how long a task should take as well as
how long it did take Grasping, moving, manipulating objects in workspace Apex will become the heart of the Task Manager and enable multi-
tasking, task prioritization, shedding, deferral, resumption Task primitive definitions include failure modes (time/quality) that
enable the occurrence of emergent behaviors Mission success/performance measures computed: vulnerability
to error, slipped schedules; performance degradation
SGH 10/04-38
MIDAS v3.0 Structure
Workstation Geometry
List of Tasks/Procedures
Mission Environment
Operator Characteristics
Physical SimulationPerceivesAttends
Moves/ActsChanges
Cognitive simulation Behavior modifiers
Situation AwarenessError, Workload
Timeliness
Commands Results
Task Network
Dynamic Animation
Mission success
Task stateOperator state
Task executions
Dynamic models
Timeline
Performance measures
Fit/Reach/Vis envelope
LibraryPrimitive tasksHuman model
Task ManagerPlans
MonitorsRemembers
SensesActuates
SGH 10/04-42
PC Version: Early Simulation
QuickTime™ and aMicrosoft Video 1 decompressorare needed to see this picture.
SGH 10/04-43
Conclusion MIDAS 3.0 now operates on a PC platform and will soon incorporate
significantly enhanced cognitive model (Apex)
MIDAS 3.0 gives users the ability to model the functional and physical aspects of a variety of operators, systems, and environments.
It brings these models together in an interactive, event-filled simulation for quantitative and visual analysis
The interplay between top-down and bottom-up processes and a suite of performance modifying functions enables the emergence of un-forseen, un-scripted behaviors
The government has done what it set out to do - - spur development of human performance modeling tools integrated into a design environment
Our goal is to continue to add functionality with each new application