paper presented at the human factors and ergonomics 47th ...€¦ · paper presented at the human...

Paper presented at the Human Factors and Ergonomics 47th Annual Meeting, October 2003, Denver, Colorado.

SITUATION AWARENESS ORIENTED DESIGN: FROM USER’S COGNITIVE REQUIREMENTS TO

CREATING EFFECTIVE SUPPORTING TECHNOLOGIES

Mica R. Endsley, Cheryl A. Bolstad, Debra G. Jones, and Jennifer M. Riley SA Technologies, Inc.

Situation awareness is a fundamental construct driving human decision making in complex, dynamic environments. By creating designs that enhance an operator’s awareness of what is happening in a given situation, decision making and performance can improve dramatically. The Situation Awareness-Oriented Design process provides a means to improve human decision-making and performance through optimizing situation awareness. This method has been used to develop and evaluate system design concepts in aviation, medical and information intelligence operations. It features three main components: SA Requirements Analysis, SA-Oriented Design Principles, and SA Measurement and Validation. This design process is user-centered, and derived from a detailed analysis of the goals, decisions and situation awareness requirements of the operator derived through a Cognitive Task Analysis methodology called Goal-Directed Task Analysis. The development of tool suites for supporting high levels of situation awareness in military command and control are presented to illustrate the use of the SA-Oriented Design process for translating the results of cognitive task analyses into to user-centered system designs.

INTRODUCTION

Developing and maintaining a highly level of situation

awareness (SA) is the most difficult part of many jobs. It is one of the most critical and challenging tasks in many domains today. A vast portion of the operator’s job is involved in developing SA and keeping it up to date in a rapidly changing environment. This is a task that is not simple in light of the complexity and sheer number of factors that must be taken into account in order to make effective decisions. SA can be thought of as an internalized mental model of the current state of the operator’s environment. All of the incoming data from the many systems, the outside environment, fellow crew members, and others (e.g. (other aircraft and ATC) must all be brought together into an integrated whole. This integrated picture forms the central organizing feature from which all decision making and action takes place.

The key to coping in the "information age" is developing systems that support this process. This is where our current technologies have left human operators the most vulnerable to error. Problems with SA were found to be the leading causal factor in a review of military aviation mishaps (Hartel, Smith, & Prince, 1991) and in a study of accidents among major air carriers, 88% of those involving human error could be attributed to problems with situation awareness (Endsley, 1995b). A similar review of errors in other domains (such as air traffic control or nuclear power) shows that this is not a problem that is limited to aviation, but one we face with many of our complex systems.

The key is in understanding that true situation awareness only exists in the mind of the human operator. Therefore presenting a ton of data will do no good unless it is successfully transmitted, absorbed and assimilated in a timely

manner by the human to form situation awareness. Traditional human factors design methods and principles are insufficient for achieving the situation awareness needed in complex systems in that they primarily address the physical and perceptual characteristics of system components, rather than the way that the integrated system needs to function from a cognitive standpoint. To address this issue, we have developed the SA-Oriented Design process that starts with a cognitive task analysis to uncover the SA requirements involved in a particular domain and translates those requirements into a system design that helps to promote high levels of SA (Figure 1).

Figure 1. SA Oriented Design Process

SA REQUIREMENTS ANALYSIS

The problem of determining what aspects of the situation

are important for a particular operator’s SA has frequently been approached using a form of cognitive task analysis called a goal-directed task analysis. In such analysis, the major goals of a particular job class are identified, along with the major subgoals necessary for meeting each of these goals. Associated with each subgoal, the major decisions that need to be made are then identified. The situation awareness needed for making these decisions and carrying out each sub-goal are identified. These SA requirements focus not only what data the operator needs, but also on how that information is integrated or combined to address each decision. In this analysis process, SA requirements are defined as those

SA MeasurementSA Requirements

AnalysisSA-Oriented

Design Principles


dynamic information needs associated with the major goals or sub-goals of the operator in performing his or her job (as opposed to more static knowledge such as rules, procedures and general system knowledge). This type of analysis is based on goals or objectives, not tasks (as a traditional task analysis might). This is because goals form the basis for decision making in many complex environments. An example of a GDTA goal breakout is shown in Figure 2, and of the delineation of decisions and SA requirements for one goal in Figure 3.

Figure 2. Goal Hierarchy for Army Logistics Officer

Figure 3. GDTA for a Sample Goal

SA-ORIENTED DESIGN PRINCIPLES

Second, the development of a system design for successfully providing the multitude of SA requirements that exist in complex systems is a significant challenge. A set of fifty of design principles have been developed based on a theoretical model of the mechanisms and processes involved

in acquiring and maintaining SA in dynamic complex systems (Endsley, Bolte, & Jones, 2003). These guidelines are focused on a model of human cognition involving dynamic switching between goal-driven and data-driven processing and feature support for limited operator resources. Examples include: 1. Direct presentation of higher level SA needs

(comprehension and projection) is recommended, rather than supplying only low level data that operators must integrate and interpret manually.

2. Goal-oriented information displays should be provided, organized so that the information needed for a particular goal is co-located and directly answers the major decisions associated with the goal.

3. Support for global SA is critical, providing an overview of the situation across the operator’s goals at all times (with detailed information for goals of current interest) and enabling efficient and timely goal switching and projection.

4. Critical cues related to key features of schemata need to be determined and made salient in the interface design. In particular those cues that will indicate the presence of prototypical situations will be of prime importance and will facilitate goal switching in critical conditions.

5. Extraneous information not related to SA needs should be removed (while carefully ensuring that such information is not needed for broader SA needs).

6. Support for parallel processing, such as multi-modal displays should be provided in data rich environments.

In addition to these, other principles are provided for: • Dealing with system complexity, • Conveying levels of confidence and uncertainty

associated with information, • The design of alarm systems, • Design of advanced automation concepts, and • Supporting SA in multi-warfighter and distributed

team environments. The SA-Oriented Design process is applicable to a wide

variety of system designs. It has been successfully applied as a design philosophy for systems involving remote maintenance operations, medical systems, flexible manufacturing cells, and military command and control.

SA DESIGN MEASUREMENT

Many concepts and technologies are currently being developed and touted as enhancing SA. Prototyping and simulation of new technologies, new displays and new automation concepts is extremely important for evaluating the actual effects of proposed concepts with in the context of the task domain and using domain knowledgeable subjects. If SA is to be a design objective, then it is critical that it be specifically evaluated during the design process. Without this it will be impossible to tell if a proposed concept actually helps SA, does not effect it, or inadvertently compromises it in some way.

The Situation Awareness Global Assessment Technique (SAGAT) has been successfully used to provide this

Maintain Unit Readiness –

Keep units 100% supplied

1.0

Project future supply needsof units

2.0

Update/adjust supplyplan

3.0

Coordinate supply schemefor units

3.3

Recommend allocationof land

3.2

Protect resources

3.4

Move resources

3.1

Acquire resources

3.4

Allocate resources

1.0

Project future supply needs

of units

What items will be needed by each unit at

which location over time?

Is the potential COA logistically

supportable?

Projected usage of each item over time

Units in task organization

Type

Size

Equipment

Operation unit is involved in

Mission requirements

Mission timing

Mission tasks

Projected location of unit over time

Past item usage rate for unit and

mission type

Location of projected supply points

Terrain

Distance to supply points

Mobility requirements

Type

Fuel requirements

Logistics supplied organically by

each unit

Projected ability to support units with COA

-Proj. COAs

-Mission requirements of each unit

over time

-Primary and secondary units

-Experience level of units

-Projected enemy unit

size/actions

-Projected impact on mission

-Deviation between items needed

and when we can deliver them

-Proj. usage of each item over time

-Proj. safety of units and logistics

team (3.2)

-Proj. availability of resources

-Proj. time and ability to get items

to units (3.4)


information by directly and objectively measuring operator SA in evaluating avionics concepts, display designs, and interface technologies (Endsley, 1995a). A primary benefit of examining system design from the perspective of operator situation awareness is that the impact of design decisions on situation awareness can be objectively assessed as a measure of quality of the integrated system design when used within the actual challenges of the operational environment.

EXAMPLE OF SAOD IMPLEMENTATION IN

MILITARY COMMAND AND CONTROL We applied the SA-Oriented Design process to the

creation of tools for supporting SA in army command and control operations. Specifically, displays suites were created for the intelligence officer (S2), logistics officer (S4) and operations officer (S3) positions. This effort provides an example to show how SA-oriented design provides a strategy for dealing with potential data overload on the battlefield by creating systems that support warfighter SA.

First a SA requirements analysis was conducted following the goal-directed task analysis methodology described here. The results of these analyses are reported elsewhere (Jones, Bolstad, Riley, Endsley, & Shattuck, 2003). The SA requirements determined from these analyses formed the basis for the development of an integrated set of tools suites for supporting these positions. These tools were designed to provide each officer with the SA needed for his/her job — providing a means of realizing the common relevant operating picture. The SA requirements analysis provided the basis for not only determining what data each position needed, but also how that data needed to be integrated to support each decision and goal in a direct manner. Unneeded searching and performance of mental calculations is eliminated or greatly reduced.

Furthermore, these displays were designed to directly support the presence of team operations in performing these tasks. Many times, several people work together to perform a particular role (in order to provide 24/7 operations, for example), and a good display will help insure that they all derive a consistent and accurate picture of what is happening. In that each officer must routinely also work with officers in different positions, tools for supporting collaborative SA needs between positions were also integrated into the display suite, and consistency of interaction is provided.

As an example, each officer is provided with a map display, Figure 4. This map is easily tailored to each position, through the provision of filters which dictate the types of overlays to be provided on the map. The type of overlays are customized for each position, depending on their SA needs. The fact that the filter buttons are displayed at the bottom of the screen helps to build up team SA —there is less chance of error occurring if one team member does not realize which information is (or is not) being shown. We have made this map easy to navigate and easy to scale up or down with the controls on the left of the screen.

Course of action graphics and history traces showing past unit movements can be displayed, along with higher level

information such as the sensor coverage area or visibility of different units as affected by terrain and weather, or the weapons coverage area of different battlefield elements (Figure 5). These filters directly provide the types of comprehension and projections needed, tailored to each positions’ requirements.

Figure 4. Intelligence analyst map

Figure 5. Map showing weapons coverage areas In addition to this general map, which provides each

officer with the ability to maintain a high level of awareness of current operations, several other screens are provided for specific positions. The intelligence visualization display (shown in Figure 6), for example, provides the intelligence analyst with the detailed information need to deploy, monitor and manage sensors and intelligence assets.

As shown on this display, not only can the analyst track where each sensor is geographically, but he can also directly see the coverage area of that sensor in juxtaposition to friendly forces or reported positions of enemy forces. For any element whose exact location is not being currently tracked by GPS, a timeliness bar is provided under the unit symbol that allows a

History Trace Visibility

Enemy Others COAFriendly

Hydrology Structures Roads

SensorCoverage Weapons

Boundaries Comm

AOI WeatherRulerUnit

Tools

Grid Zone16S GL 08 35

100,000M 10,000M 1,000M

Return to PreviousCenter

Info

Delete

Select

VegetationContours

History Trace Visibility

Enemy Others COA

Structures Roads Boundaries Comm

AOI WeatherRulerUnit

Tools


100,000M 10,000M 1,000M


Info

Delete

Select

HydrologyContours Vegetation

Friendly

XX

FSB

MI

MP

TAC

SensorCoverage Weapons


rapid assessment of the degree of certainty that can be associated with a displayed piece of information. Key details about any element are easily viewed (left side of map) by

rolling the cursor over any element on the map.

Figure 6. Intelligence visualization display (IVD) Other higher level SA information is also provided

directly, such as unit ID, as aggregated from several sources (Figure 7). The danger of fratricide is significantly reduced by directly presenting the probability that a unit may be a friendly(or an unknown), and not just the most likely or highest frequency ID. This supports analyses of SA that show experienced decision makers care very much about how much confidence to place in information.

Figure 7. IVD showing confidence level of unit ID

Maintaining a high level of awareness of the status of

required supplies is a significant job for logistics officers, one that they have had little information to support in past. In the future, however, it will be possible to track whether shipments have gone out as scheduled and when they are expected to arrive at their destinations. The scheduling display (Figure 8) allows logistics officers to not only track shipments, but also to schedule and obtain needed materials and supplies. They can calculate the logistics needs of various units based on the

current course of action, planned usage charts, and tailored to the historical usage differences of those units. They can also schedule delivery of those items via various modes of transportation, whose availability can be assessed, as well as which transportation may be required to meet terrain, weather, and schedule needs. Each of these fairly time consuming calculations is directly supported with easy to use tools.

Figure 8. Schedule display

In addition, the logistics officer is responsible for

assessing the health and status of the unit. Typical (gum-ball or traffic light) iconic representations of green, yellow, red or black associated with the status of troops, equipment, ammo, fuel and mission phase of each operational unit — are not of sufficient detail to meet the planning needs of the logistics officer. A detailed chart is provided of unit status at the level of detail needed, while also providing a high level overview at a glance (Figure 9).

Info Delete New

Ruler UnitUnit

WeatherBoundaries

AOI

StructuresRoads Weapons


100,000M 10,000M 1,000M


Vegetation HydrologyFriendly

TAC

XX

MI

MP

X2/50 ID(M)

Sensor Coverage

IntelAssetsIEnemy

RefugeeHolding Area

N1

N2

Select

FSB

BDA

Info Delete New

Ruler UnitUnit

Enemy

WeatherBoundaries

AOI

StructuresRoads Weapons


100,000M 10,000M 1,000M


Vegetation HydrologyFriendly

TAC

XX

MI

MP

X1/101 ID(M)

X2/50 ID(M)

22

22

22

2

IntelAssets

Sensor Coverage

Placed 08/04/02Company Charlie Last report 08/05/02 11:60Undisturbed

Working status

Select

BDA


Figure 9. Unit status display Other officers (such as the commander), would generally

see this same data at a more summary level. While drawing from a common database, the SA requirements can be used to show how each display needs to be carefully tailored to meet the SA needs of each position. Information needed for direct collaboration, as well as tools for more general collaboration are provided, supporting high levels of team SA.

In summary, the display suites developed provided an integrated interface for supporting high levels of SA in a command and control setting. This approach provided a number of key characteristics important for SA. 1. Provides global overview with detailed information when

needed. This overview is easily tailored to meet individual needs as they vary across time and missions.

2. The use of a standardized interface supports accuracy in sharing & interpretation of information across individuals working a particular position, as well as across the entire command and control structure.

3. Data is organized according to goals. This places all the information needed for addressing particular decisions and goals together, greatly reducing the amount of paging around between windows or displays required.

4. To support goal driven processing and the rapid switching between goals that characterizes cognitive behavior in dynamic and complex operations, we have made it easy to transition between goals. A single button press or glance to an adjacent display allows operators to rapidly move between goals in dynamic operations.

5. The displays supports diagnosis of reports in one place. Officers are not required to pull up different screens to determine the validity of a verbal report or to integrate multiple reports, but can easily make these comparisons in one place. The data behind integrated reports is also easily viewable through a mouse-over, insuring that the subtle cues behind confidence in reports is not lost. Information to support confidence assessments is directly presented to prevent misinterpretations of display data.

6. The displays are specifically built to meet the needs of each officer. The data is integrated in ways that support the level 2 and 3 SA assessments relevant to each

position. By presenting integrated information, rather than data, significant data overload problems can be avoided. Officers at each position are not overwhelmed with data that is too detailed or extraneous to what they need, as is the problem with many common operating picture approaches.

7. The direct support of team operations creates less chance of information falling through the cracks or being missed by others. Off-duty officers can rapidly determine what others did while they were gone. The information needed by other teams is directly communicated through the displays rather than reliant on undependable verbal communications. This also reduces communication loads which are a real problem in this environment.

8. Finally, it is very easy to get needed information. In most cases only a single button click is needed. Extraneous look-ups or mental calculations are minimized or eliminated in most cases. Because the screens are goal-oriented, there is minimal wading through screens to find what is needed.

CONCLUSIONS

The need to process and understand large volumes of data

is critical to many endeavors, from the cockpit to military missions, from power plants to automobiles, and from space stations to day-to-day business operations. The lessons we are learning in advanced systems about the importance of good situation awareness, the challenges that we face in achieving it, and the design principles that are needed to support it, all provide valuable directions for these areas as well. We will not realize the benefits of the information age until we come to grips with the challenges of managing this dynamic information base to provide people with the situation awareness they need on a real-time basis. Doing so is the primary challenge of the next decade of technology.

ACKNOLOWDGEMENTS

Work on this paper was prepared through participation in the Advanced Decision Architectures Collaborative Technology Alliance sponsored by the U.S. Army Research Laboratory (ARL) under Cooperative Agreement DAAD19-01-2-0009. The views and conclusions contained herein, however, are those of the authors and should not be interpreted as representing the official policies, either expressed or implied of the ARL or the U. S. Government.

REFERENCES

Endsley, M. R. (1995a). Measurement of situation awareness in dynamic systems. Human Factors, 37(1), 65-84.

Endsley, M. R. (1995b). A taxonomy of situation awareness errors. In R. Fuller, N. Johnston & N. McDonald (Eds.), Human Factors in Aviation Operations (pp. 287-292). Aldershot, England: Avebury Aviation, Ashgate Publishing Ltd.


Endsley, M. R., Bolte, B., & Jones, D. G. (2003). Designing for situation awareness: An approach to human-centered design. London: Taylor & Francis.

Jones, D. G., Bolstad, C. A., Riley, J. M., Endsley, M. R., & Shattuck, L. (2003). Situation awareness requirements for the future objective force. Proceedings of the ARL

Collaborative Technology Alliances Conference. Adelphi, MD: ARL.

Hartel, C. E., Smith, K., & Prince, C. (1991). Defining aircrew coordination: Searching mishaps for meaning, Proceedings of the Sixth International Symposium on Aviation Psychology. Columbus, OH: Ohio State University

paper presented at the human factors and ergonomics 47th ...€¦ · paper presented at the human...

Documents