display techniques in information-rich virtual environments

Display Techniques in Information-Rich Virtual Environments

Display Techniques in Information-Rich Virtual

Environments Nicholas F. Polys

PhD Research ProposalAugust 18, 2004

Committee:

Dr. Doug Bowman, VT

Dr. Chris North, VT

Dr. Scott McCrickard, VT

Dr. Ken Livingston, Vassar College

Dr. Don Brutzman, Naval Postgraduate School


Proposal Outline

• Problem scope & Statement

• Background

• Research Questions & Goals

• Information Design Dimensions

• Approach, Method, Measures

• Experimental Program

• Significance


• Complex systems typically span multiple scales and involve heterogeneous data types (objects, spatial relations, attributes)

• Engineers, researchers, and analysts need to access, manage, and understand a wide variety of information and inter-relationships

General Problem: Integrated Information Spaces


Fundamental Data Types

• Spatial / perceptual data:

geometry, colors, textures, lighting

• Abstract data / world & object attributes: nominal, ordinal, quantitative

• Temporal data / behaviors:

states, dynamics


Information-Rich Domains

Simulation and design applications require spatial/perceptual fidelity and information enhancement:

• Engineering / CAD• Construction / architecture • Medicine / biology • Science / research• Education / training• …


User Problem: Integrated Information Spaces

• Multiple applications and fragmented views make it difficult to understand the relationships between information types

• Next generation information interfaces must unify display and interaction spaces for:– Exploration– Search– Comparison and Pattern recognition


Problem Statement:We lack precise definitions, development tools, and

systematic research as to how perception and cognition operate in information-rich interfaces and environments:

• Combining Virtual Environments and Information Visualizations is currently ad hoc and application-specific

• There are competing models of Vision and Working Memory that may apply

• An experimental methodology and theory is required to assess, design, and deliver ‘appropriate’ displays


Background:Information Psychophysics

• Edward Tufte, Envisioning Information

(1983, 1990)• Jaques Bertin, Semiology of Graphics

(1983)• Donald Norman, Cognitive Engineering

(1986)• Joseph Goguen, Semiotic Morphisms

(2000)• Colin Ware, Perception for Design

(2003)


Background:Information Visualization

Visual display of abstract information• Visual Markers

(Cleveland & McGill, 1984; Mackinlay, 1986; Card et al, 1999)

• Multiple Views (North, 2001; North et al, 2002; Convertino et al, 2003)

• Zoom-able Interfaces (Bederson et al, 1996, 2000; Woodruff et al, 1998a-c)


Background:Virtual Environments

Visual display of spatial and

perceptual data• Immersive and Desktop Platforms• Conceptual Learning (Salzman et al, 1999)

• Navigating space (Darken et al, 1996, 2002)

• Naturalism & Performance (Bowman, 2002; Bowman et al, 2004)

• Image Plane (Pierce et al, 1997)


Background:Augmented Reality

Enhancing perceptual scenes with additional abstract information

• Feiner et al. Windows on the World (1993)• Bell et al. Dynamic Space Management,

View Management (2000, 2001)


Background:Multimedia & Comprehension

• Co-references between text and images(Chandler & Sweller, 1990; Faraday & Sutcliffe, 1997, 1998)

• Task Knowledge Structure (Sutcliffe & Faraday, 1994; Sutcliffe 2003)

• Meaningful Learning: troubleshooting, redesigning, deriving principles (Mayer, 2002)


Background:Architecture of WM

• Componentized WM (Baddeley, 2003) – phonological loop, visuospatial sketchpad,

episodic buffer, central executive

• Individual differences in WM Capacity (Just & Carpenter: 1996)

• Short and Long term WM (Ericsson & Kintsch: 1995)


Working Memory (Baddeley, 2003)

Central Executive

PhonologicalLoop

Episodic bufferVisuospatial sketchpad

Visual semantics Episodic LTM Language

Fluid system

Crystallized system


Background: Visuospatial WM

• Capacity 3-5 ‘items’ • Functional units & chunking

– Objects & features (Vogel et al, 2001)

– Visual indices & dynamic feature binding (Saiki, 2003)

• Subsystems: form & color, space & movement (Logie 1995)

• Relation to Central Executive(Miyake et al. 2001)


Background: Interpreting Linegraphs

• Computational model: Understanding Cognitive Information Engineering [UCIE] (Lohse, 1991) – similar to GOMS (Card, Moran, & Newell, 1983), and ACT*

(Anderson, 1983)

• Processing – encoding visual patterns– inferring and retrieving functional relations via

graph schema– associating / labeling referents


Proposal: Information-Rich Virtual Environments (IRVEs)

We need to understand:

• How spatial/perceptual information and abstract information can be combined and displayed

• What makes the combinations effective• What makes them usable and • How users think and act when using them


Research Question for IRVE Information Design:

Where and How should enhancing abstract information be displayed

relative to its perceptual referent

so that

the respective information can be understood together and separately?


Research Goals:

• To understand how the respective design techniques of Virtual Environments and Information Visualization can be combined and balanced

• To enable stronger mental associations between spatial and abstract information while preserving the models of each type of information.


Research Goal 1:

• Define a theoretical framework for Information-Rich Virtual Environments (IRVEs) as the solution to the problem of integrated information spaces

• Results: – Terminology– Definitions– Bowman et al, 2003


Research Goal 2:

• Enumerate the design space for IRVE tasks and display techniques through Usability Engineering process and literature review

• Results:– IRVE Task Space, Information Design Space,

Interaction Design Space– Polys & Bowman, 2004; Polys et al, 2004c


Research Goal 3:

• Prototype information-rich application interfaces to identify problems and generate hypotheses regarding optimal IRVE information design per task

• Results:– IRVE Display Prototypes– Claims Analysis 1 (heuristic evaluation, informal user

studies, pluralistic walkthroughs)– Polys, 2003; Polys et al, 2004a, 2004b


Research Goal 4:

• Describe IRVE display configurations in a concise XML DTD and Schema and use this display description to generate runtime components

• Results [pending]:– IRVE Testbed


Research Goal 5:

• Provide a quantitative basis by which to characterize the density and distribution of information in an IRVE

• Results [pending]:– Assessment method for IRVE information sets


Research Goal 6:

• Identify tradeoffs and guidelines for the IRVE display design space using

– Empirical usability evaluations and

– Metrics for individual cognitive differences

• Results [pending]:– Empirical data relating design techniques,

information sets, and user performance


Research Question re-phrased:

For the combination of abstract and perceptual visualizations in IRVEs, association should be maximized and interference (such as occlusion and crowding by layouts) should be minimized…

What are the best ways to manage layout space and association cues so that perceptual and

abstract information can be understood together and separately?


Goals & Results Summary

• Survey IRVE information design space

• Prototype IRVE display techniques and applications

• Understand and quantify how IRVE information sets vary

• Enumerate information design heuristics and guidelines for IRVEs


IRVE Information Design Challenges

• Visibility

• Legibility

• Association

• Occlusion

• Aggregation


Abstract information design parameter

Psychological process

Usability impact

Visual attributes:- color- fonts- size- background- transparency

Perception - Legibility - Readability

- Occlusion

Layout attributes:- location- association - density

Interpretation - Relating abstract and perceptual information- Conceptual categories & abstractions- Occlusion

Aggregation:- level of information encoding - type of visualization

Making Sense - Comparison & Pattern Recognition- Effectiveness - Satisfaction

IRVE Information Design Dimensions


Layout Space (Locations)

The layout space of abstract information in IRVEs is described by the coordinate system it is resident in:

• Object• World• User• Viewport• Display


Object Space

Object space is relative to an object’s location in

the environment (e.g. Semantic Objects).


World SpaceWorld space is

relative to an area, region, or location in the

environment.


User Space

User space is relative to the user’s location but not their viewing angle.


Viewport Space

Viewport space-is the image plane where HUDs or overlays may be located.


Display Space

Display layout space where abstract visualizations are located outside the rendered view in some additional screen area.


Association

The association dimension of IRVE information design is delineated by the Gestalt principles:


Approach (Goals 4-6)

• Develop methods to describe and generate IRVE display components

• Develop quantitative methods to characterize IRVE datasets

• Use the above to control testbed environment and stimuli and run empirical usability evaluations


IRVE Testbed Configuration Syntax

• The composition of an IRVE display will be defined under an XML DTD and Schema. The DTD and Schema will provide syntactic and semantic production rules for IRVE display spaces.

• In order to instantiate an IRVE display, the testbed will read the information mapping configuration from the XML and generate the X3D code for that

IRVE.


IRVE Stimuli & Metrics

Exocentric vs. Egocentric:

Exocentric metrics have an advantage in that they may be employed on a data set independent of the user or rendering; egocentric metrics may be more advantageous to find dynamic, perspective-specific layouts.

This research proposal will investigate exocentric metrics for IRVE data characteristics such as the quantity, density, and distribution of abstract information in the VE.


IRVE Layout & Association Dimensions

Common Region

Proximity Connected ness

Similarity Common Fate

Object x x x x x

World x x x x x

User x x x x x

Viewport x x x x x

Display x x x x x

Experiment 1 Experiments 2 & 3


Evaluation Method

• The usability experiments will have a factorial, within-subjects, counterbalanced design

• Cognitive battery test will be used control between-subject variance and may provide insight into how display techniques work with different visuospatial abilities.

These experiments will allow us to explore tradeoffs in IRVE information design and identify guidelines and design patterns for integrated information spaces.


Procedure

• Subjects from VT population

• Demographic questionnaire &

Cognitive battery tests

• Training session for spatial navigations

• Experimental Protocol &

dependent measures


Measures

• Time to Completion

• Correctness

• Satisfaction / Ease of Use

• User strategy

• Cognitive Battery


Perceptual Speed / Closure Flexibility

Spatial Visualization

Spatial Orientation Cognitive

Battery


IRVE TasksIRVE Search Tasks require subjects to:• find a piece of abstract information based on some

perceptual criteria• find a piece of perceptual information based on

some abstract criteria. IRVE Comparison Tasks require subjects to:• compare perceptual attributes of two items with a

given abstract criteria• compare abstract attributes of two items with a

given perceptual criteria


Experiment 1:

Object space layouts can be drawn at a fixed orientation or billboarded to always face the user. Is one technique better than the other for common tasks and navigation modes?

Here we will investigate Search and Comparison tasks combined with flying or terrain-following navigation.


Object space:Billboard vs. Fixed

• Search & Comparison Tasks• Terrain following and Flying navigation• Fixed orientations provide additional

wayfinding cues• Billboarded orientations require less spatial

navigation for legibility• In these circumstances, is one display

technique better that the other?


Experimental Unit Human Subject (n = 8)

Tests: Flexibility of Closure, Perceptual Speed, Spatial Orientation,

Factor 1 Object space layout

Level 1 Billboarded

Level 2 Fixed

Factor 2 Navigation type

Level 1 Walk

Level 2 Fly

Factor 3 Task: Search

Level 1 Spatial -> Abstract

Level 2 Abstract -> Spatial

Factor 4 Task: Comparison

Level 1 2 Spatial items

Level 2 2 Abstract items

Response Variables Performance: Time to Completion, CorrectnessSatisfaction: Ease of Use

16 conditions


Experiment 2:

Abstract information is associated to perceptual information through different cues in Object and Viewport spaces. How do the Gestalt principles rank in IRVEs?

We will examine connectedness, proximity, and common region for Search tasks between these two layout spaces.


Association in Object vs. Viewport spaces

• Search Task

• Relating abstract and perceptual information between layout spaces and association parameters

• Do depth cues of object space enable a tighter coupling of information types or is the occlusion unneccessary?



Tests: Flexibility of Closure, Perceptual Speed, Spatial Orientation,

Factor 1 Layout space

Level 1 Object

Level 2 Viewport

Factor 2 Gestalt principle

Level 1 Connectedness

Level 2 Proximity

Level 3 Common Region

Factor 3 Task: Search

Level 1 Spatial -> Abstract

Level 2 Abstract -> Spatial


12 conditions


Experiment 3:

The theory behind IRVEs must account for a wide range of content. How does content determine an appropriate layout space if at all? We will examine Object and Viewport spaces for Comparison tasks in different environments.

We will factorially compare environments of high and low density and distribution.


Density and Distribution thresholds for

Object vs. Viewport spaces

• Comparison Task

• Can environment characteristics predict advantageous display techniques and ground design guidelines?



Tests: Flexibility of Closure, Perceptual Speed, Spatial Orientation, Digit Span

Factor 1 Layout Space

Level 1 Object

Level 2 Viewport

Factor 2 Task: Comparison

Level 1 2 Spatial items

Level 2 2 Abstract items

Factor 3 Information Density

Level 1 Low

Level 2 High

Factor 4 Information Distribution

Level 1 Homogenous

Level 2 Clustered


12 conditions


Significance

• Establishes a research area combining Information Visualization and Virtual Environments

• Investigates how search and comparison tasks can be supported with IRVE display techniques

• Relates display techniques to models of perception and working memory


Significance (cont’d)

• Supports and extends international standards technology (e.g. X3D, XML)

• Provides tools for IRVE developers

• Provides guidelines for IRVE designers

• Environment characteristics and display guidelines may also transfer to Augmented Reality


Timeline & MilestonesSeptember 2004 IRVE Task Space

EnumerationIRVE Data Metrics

October 2004 IRVE Display LanguageTestbed completed

November 2004 Experiment 1

January 2004 Experiment 2

March 2005 Experiment 3

Fall 2005 Research Defense

Spring 2006 Dissertation Defense


Thank YouQuestions & Discussion

[email protected]

http://www.virtuworlds.com/X3D/npolys_PhD_Proposal.doc

http://www.virtuworlds.com/media_02/index.html#wares

display techniques in information-rich virtual environments

Documents