automated dynamic symbology for visualization of higher level fusion

Automated Dynamic SymbologyAutomated Dynamic Symbologyfor Visualization of Higher Level Fusionfor Visualization of Higher Level Fusion

T. Kesavadas and Youngseok KimT. Kesavadas and Youngseok Kim

Virtual Reality LaboratoryVirtual Reality LaboratoryCenter for Multisource Information Fusion (CMIF)Center for Multisource Information Fusion (CMIF)

The State University of New York at BuffaloThe State University of New York at Buffalowww.vrlab.buffalo.eduwww.vrlab.buffalo.edu

Preview

1.1. Features of Fusion Visualization (related to COP)Features of Fusion Visualization (related to COP)

2.2. Visualization Tasks for FusionVisualization Tasks for Fusion

3.3. Low Level FusionLow Level Fusion

4.4. High Level FusionHigh Level Fusion

5.5. Future WorkFuture Work

Our Simulation Environment & Goal

• Simulation Environment: Simulation Environment: Natural/man-made post-disaster situation.Natural/man-made post-disaster situation.

• Simulator: Simulator: The Run-Time Interface (RTI) for earthquake in The Run-Time Interface (RTI) for earthquake in Northridge, CA. in 1994. (Northridge, CA. in 1994. (There are several federates in There are several federates in RTI, such as report generator, data fusion, walk-in casualty, RTI, such as report generator, data fusion, walk-in casualty, medical facility, dispatcher-router, and visualization.)medical facility, dispatcher-router, and visualization.)

• The goal of the visualization federate is:The goal of the visualization federate is:to develop effective display and user interface for to develop effective display and user interface for monitoring, situation awareness (SA), and decision monitoring, situation awareness (SA), and decision support for command and control.support for command and control.

Visualization Tasks for Low & High Level Fusion

Position, IdentificationLevel 1

Level 2

Level 3

Situation Awareness

Threat Assessment

GIS Vector Map

Matching UTM Coordinate

Raster Map

Dynamic Symbology

Viz. of Casualty Groups

Tactical & Seamless Graphics

Positioning Symbols

Positioning identifying symbols on the map.

Showing the pattern and its meaning.

Fusion Levels & Goals Visualization Tasks

Post-disaster Visualization

• Earthquake simulation for Northridge area, CAEarthquake simulation for Northridge area, CA

• There are too many cluttered and overlapped icons, There are too many cluttered and overlapped icons, and hard to recognize the situation. and hard to recognize the situation.• Viz. of high level fusion started with “Symbology” Viz. of high level fusion started with “Symbology” study.study.

Visualization for High Level Fusion

Situation Awareness (Level 2)Situation Awareness (Level 2)

• Goal: Display patterns and tactical information.Goal: Display patterns and tactical information.

• Related Tasks:Related Tasks:1.1. Automated Dynamic SymbologyAutomated Dynamic Symbology

• Tactical SymbolsTactical Symbols• Tactical GraphicsTactical Graphics

2. Seamless Transition in Display2. Seamless Transition in Display• Transparency ControlTransparency Control• Application to Casualty GroupingApplication to Casualty Grouping

Early SymbologyEarly Symbology

►Related Research (’60s ~ early ’90s)Related Research (’60s ~ early ’90s)

Mostly implemented by US Army Research Institute for Mostly implemented by US Army Research Institute for the behavioral and social science the behavioral and social science

Difficult to apply to today’s display of variable symbolDifficult to apply to today’s display of variable symbol features (size, color, visual effects, etc.) features (size, color, visual effects, etc.)

Various types of symbol: Various types of symbol: (a)(a) tactical symbol, tactical symbol, (b) image mapped on 3D cubes, (b) image mapped on 3D cubes, (c) detailed icon, (d) blocked (c) detailed icon, (d) blocked outline for larger pixel CRT, outline for larger pixel CRT, (e) filled silhouette, (e) filled silhouette, (f) outlined silhouette, and (f) outlined silhouette, and (g) and detailed 3D model(g) and detailed 3D model

Early SymbologyEarly Symbology

Department of the Army Field Manual. Military symbols. U.S. Army Field Manual (FM 21-30). Headquarters, Department of the Army, 1965.

Michael G. Samet, Ralph E. Geiselman and Betty M. Landee. An experimental evaluation of tactical symbol-design features. Technical Report 498. U.S. Army Research Institute for the Behavioral and Social Science. April, 1980.

Beverly G. Knapp. The Precedence of Global Features in the Perception of Map Symbols. U.S. Army Technical Report 803. ARI Field Unit at Fort Huachuca, Arizona, USA, 1988.

Elizabeth Wheatley. An experiment on coding preference for display symbols. Ergonomics, 20(5), pages 543-552. 1977.

Philip Bersh, Franklin L. Moses and Richard E. Maisano. Investigation of the strength of association between graphic symbology and military information. Technical Report 324. U.S. Army Research Institute for the Behavioral and Social Science. September, 1978.

►References on SymbolsReferences on Symbols

Modern SymbologyModern Symbology

►Recent Research Recent Research

Establish parametric method for remote applications (based Establish parametric method for remote applications (based on on XML type architecture). XML type architecture). MIL-STD-2525B (1999). Common War-fighting SymbologyMIL-STD-2525B (1999). Common War-fighting Symbology Department of Defense Interface Standard. U.S. Department Department of Defense Interface Standard. U.S. Department ofof Defense. Defense.

Two types of symbols are suitable for strategic level Two types of symbols are suitable for strategic level ►Tactical Symbols Tactical Symbols ►Tactical GraphicsTactical Graphics

(a) Tactical Symbols (b) Tactical Graphics(a) Tactical Symbols (b) Tactical Graphics

Our Dynamic SymbologyOur Dynamic Symbology

►Definition Definition

Dynamic Symbology is a parameterized symbol system Dynamic Symbology is a parameterized symbol system in which the values of symbolic entities and graphical in which the values of symbolic entities and graphical properties are generated/calculated in runtime.\ properties are generated/calculated in runtime.\

►Goal: System Automation, Flexibility and RemotenessGoal: System Automation, Flexibility and Remoteness

No need to maintain large number of fixed symbol setsNo need to maintain large number of fixed symbol sets

Dynamic SymbologyDynamic Symbology

► AutomationAutomation Tactical symbols and graphics are generated by Tactical symbols and graphics are generated by composing its components.composing its components.

► FlexibilityFlexibility Symbols/graphics images are not pre-defined, but Symbols/graphics images are not pre-defined, but generated at run-time. Display can be changed generated at run-time. Display can be changed dynamically to fit a user’s need during the simulation.dynamically to fit a user’s need during the simulation.

Tactical Symbol ComponentsTactical Symbol Components Tactical Graphics ComponentsTactical Graphics Components

Dynamic SymbologyDynamic Symbology

► Controlling Visual EffectsControlling Visual Effects

Can be used for various visual effects and human Can be used for various visual effects and human factorsfactors Example: Uncertainty visualization with blurred symbolExample: Uncertainty visualization with blurred symbol

►100 Levels of degraded 100 Levels of degraded images are generated images are generated from only one icon imagefrom only one icon image

►Dynamically generated Dynamically generated at at run time run time

►saves time of manualsaves time of manual image processing image processing

Uncertainty 50 %Uncertainty 50 %

Uncertainty 0 %Uncertainty 0 %

Video Clips for Dynamic Icons (Casualties and Ambulances)Video Clips for Dynamic Icons (Casualties and Ambulances)

Tactical Graphics Tactical Graphics &&

Seamless Transition Seamless Transition

Tactical GraphicsTactical Graphics

►Tactical Graphics for Fusion VisualizationTactical Graphics for Fusion Visualization

“ “Tactical Graphics often produce better results in Tactical Graphics often produce better results in situationsituation understanding, especially for experienced users.” understanding, especially for experienced users.” (Feibush, Gagvani & Williams, (Feibush, Gagvani & Williams, Visualization for situation Visualization for situation awareness.awareness. IEEE Graphics. pp. 38-45. September/October 2000)IEEE Graphics. pp. 38-45. September/October 2000)

Tactical Graphics is useful for visualization of higher Tactical Graphics is useful for visualization of higher level fusion, since it provides strategic view.level fusion, since it provides strategic view.

Automation of tactical graphics can improve situation Automation of tactical graphics can improve situation awareness.awareness.

Can be applied to visualization of a large number of Can be applied to visualization of a large number of casualties.casualties.


►Problem in VisualizationProblem in Visualization

Current Current visualization is for visualization is for location & location & identification of each identification of each casualty from RTI casualty from RTI reports. reports.

However, with a However, with a large number of large number of similar icons, similar icons, information is usually information is usually overloaded with overloaded with cluttered and cluttered and overlapped icons.overlapped icons.


►Proposed SolutionProposed Solution

Current visualization is for location & identification of Current visualization is for location & identification of each casualty from RTI reports. each casualty from RTI reports.

However, with a large number of similar icons, However, with a large number of similar icons, information is usually overloaded with cluttered and information is usually overloaded with cluttered and overlapped icons.overlapped icons.

GroupingGrouping can be an efficient Tactical Graphics can be an efficient Tactical Graphics

IdentificationIdentification(Lower Fusion Level)(Lower Fusion Level)

Situation AwarenessSituation Awareness(Higher Fusion Level)(Higher Fusion Level)

Seamless Seamless TransitionTransition

Seamless TransitionSeamless Transition

► Why Seamless Transition?Why Seamless Transition? Too Much Detail or High Realism Can Distract UserToo Much Detail or High Realism Can Distract User Retains Focus on Objects of InterestRetains Focus on Objects of Interest Saves Time for researching in Different DisplaySaves Time for researching in Different Display Alleviates Work StressAlleviates Work Stress

Level 2

Situation Awareness

Level 0 & 1

Position,Identification

Common Display


► Transparency (Opacity) ControlTransparency (Opacity) Control Transparency blends two or more stateTransparency blends two or more state

The level of blending can be obtained The level of blending can be obtained by interpolation curves, such as Bernstein’s blending by interpolation curves, such as Bernstein’s blending curves curves or any sinusoidal curves which make all sums to 1.0. or any sinusoidal curves which make all sums to 1.0.

NoneBlack & White

0

1.0

Abstractness

Opacity

Color

Realism


► Transition from Raster Map to Analysis SpaceTransition from Raster Map to Analysis Space Visual attributes are manipulated for seamless transitionVisual attributes are manipulated for seamless transition

(Level 0 & 1)

RealismRealism AbstractnessAbstractness

Position Pattern(Level 2 & 3)

Color, B&W, Transparency

Visualization of CasualtiesVisualization of Casualties

► FeaturesFeatures C++ & OpenGL program in 3D environmentC++ & OpenGL program in 3D environment Raster map was made with aerial photos of Northridge Raster map was made with aerial photos of Northridge RegionRegion Used casualty data from RTI reportUsed casualty data from RTI report Groups are sorted by three severity levels (1,2 &3)Groups are sorted by three severity levels (1,2 &3) Outlined groups are shown over the raster mapOutlined groups are shown over the raster map GUI for Seamless TransitionGUI for Seamless Transition

Visualization of Casualty GroupsVisualization of Casualty Groups

Raster map + Casualty groups

Severity of Severity of CasualtiesCasualties

Level 1Level 1

Level 2Level 2

Level 3Level 3


More opacity added


Removed the realism of raster map


Level 3 groups (in red) added

The overlapped regions in purple can be regarded as

dense areas.


Added the context of raster map

Future WorksFuture Works

► Continue Situation Awareness DevelopmentContinue Situation Awareness Development Casualty GroupsCasualty Groups

► Real Data from Fusion FederateReal Data from Fusion Federate Interface Design for Fast Identification Interface Design for Fast Identification Visualization of Statistical DataVisualization of Statistical Data

► 3D Urban City Modeling and VR Simulation of 3D Urban City Modeling and VR Simulation of Earthquake Impact Earthquake Impact

Fusion representationFusion representation Damage and Fire SimulationDamage and Fire Simulation

► Time-aggregation data (statistical information)Time-aggregation data (statistical information)

► Performance Evaluation of the Viz. & UI systemPerformance Evaluation of the Viz. & UI system

AcknowledgementAcknowledgement

This work was supported by the AFOSR under award This work was supported by the AFOSR under award F49620-01-1-0371. The authors gratefully acknowledge F49620-01-1-0371. The authors gratefully acknowledge valuable input from Dr. Peter D. Scott, Dr. James Llinas valuable input from Dr. Peter D. Scott, Dr. James Llinas

and Dr. Ann M. Bisantz.and Dr. Ann M. Bisantz.

ContactContact

Dr. T. KesavadasDr. T. [email protected]@eng.buffalo.edu

Or visit us online: Virtual Reality Laboratory Or visit us online: Virtual Reality Laboratory www.vrlab.buffalo.eduwww.vrlab.buffalo.edu

automated dynamic symbology for visualization of higher level fusion

Documents

systeminterface level

human factorcognitive

data fusion

fusion outputs

low high level fusionposition

tactical information

seamless display

display patterns