© 2003 microsoft corporation. all rights reserved. large display user interaction mary czerwinski,...

© 2003 Microsoft Corporation. All rights reserved.

Large Display User Interaction

Mary Czerwinski, Senior ResearcherMicrosoft Research


Thanks to my Colleagues

• Desney Tan

• George Robertson

• Greg Smith

• Patrick Baudisch

• Brian Meyers

• Gary Starkweather


Ignore Science Fiction at Our Peril

Workstation in the world of the Matrix


Large Display Surfaces are Here

Workstation in the real world


Why A Larger Display Surface?

• Productivity benefits 15-30% (despite OS issues)

• Users prefer more display surface

• Prices dropping fast

• Footprints getting smaller

Projected LCD Pricing 2002-2005

$437$378 $327 $283

$699$597

$510$436

$1,089

$905

$752$625

$0

$200

$400

$600

$800

$1,000

$1,200

2002 2003 2004 2005$

US

15" -13.5%

17" -14.6%

18" -16.9%


Multimon Trend is Growing

(Jon Peddie Research

Dec, 2002 N=6652)

No Multimon30%

Plan to Use Multimon

38%

Use Multimon32%


2004 Large Monitor ASP Projections

Single

20”Diagonal

2 x 17” (30” Diagonal)

2 x 15” (26” Diagonal)

17”Diagonal

15”Diagonal

16:9 x 22” Diagonal

Note: All Prices are for Liquid Crystal DisplaysSource for Single Panel Pricing: IDC and Display Search

Rel

ativ

e P

ricin

g

Multiple Wide

$1000


Large Display User Experience, MSR

• Large display surfaces fundamentally change user interaction– Visualization, input techniques, work

management, …

• Large display surfaces provide non-linear productivity increases– Additional space has different utility– E.g. Focal/peripheral displays provide

different cues


Windows and Task Management Issues Emerge

• Larger displays = more open windows

• Multimon users arrange windows spatially

• TaskBar does not scale:

– Aggregation model not task-based

– Users can’t operate on groups of related windows

Relationship between # of Monitors and # of Windows Left Open

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

16.00

18.00

No. of Monitors

Avg

. #

of

Win

do

ws

Lef

t O

pen

Single Monitor

DualMon

TripleMon


DEMOS: High Density Cursor

fill-in cursors

current framefill-in cursors

previous frame

mouse

motion

mouse

motion

solution• high-density cursor inserts additional

cursor images between actual cursor positions

• the mouse cursor appear more continuous

problem• at high mouse speeds, the mouse cursor

seems to jump from one position to the next

the windows mouse trail…• makes mouse trail last longer• drawback: cursor images lag behind

...is not high-density cursor• hd cursor makes mouse trail denser• lag-free: mouse stops=>cursor stops

regular mouse cursor

high-density cursor

Windows mouse trail

high-density cursor


INPUT: Drag ‘n Pop

• Problems:– Large displays

create long distance mouse movement

– Drag ‘n Pop brings proxies of targets to the user from across display surfaces


DEMO: GroupBar

• GroupBar joins related items in the taskbar and remembers spatial layouts of tasks

• To help solve this problem, Shell needs to remember Windows’ states between sessions

• Download: \\gregsmi2\dowloads\groupbar.exe


DEMO: GroupBar Layouts

• Windows management time-consuming– Screen real estate often gets wasted during layout

• Solution—Smart Windows Layout Options


Advanced UI: Scalable Fabric

• Configurable central focus + peripheral context areas

• Easy task switch from periphery to focus area

• Personally meaningful layouts are preserved by system

• Leverages human spatial memory

• Proximity determines group membership


Women Take a Wider View (CHI 2002)

• Grew from work designing and evaluating 3D virtual navigation techniques

• On regular desktop display:– Men performed significantly better than women

• On exploratory widescreen display:– Overall improvement for all users

• Surprising finding: – Gender gap disappeared - Males and females

performed equally on widescreen display


Related Work

• Formation of cognitive maps while navigating 3D virtual worlds– Spatial abilities – Artifacts (maps, landmarks,…)

• Gender differences in spatial ability and navigation strategies– Most report male advantages, especially in

virtual environments


Related Work: Optical Flow

• Changing retinal image as we move through the environment– Aids perception of environmental

structure


What we know about Optical Flow

• Optical flow benefits heading perception in active navigation– Shown for fields of view up to 90 degrees

• Hypothesized that effectiveness of optical flow depends on spatial ability– [Cutmore et al. 2000]– Gender unexplored


Our Hypotheses

1. Optical flow cues help all users form better cognitive maps when navigating 3D virtual environments

2. Better optical flow cues help women more than men in cognitive map formation

3. Wider displays offer even better optical flow cues


Dsharp Display

43"

11"


Task: General Description

• Learning: User controls movement along path through virtual 3D maze

• Testing: Remember path traveled


Virtual Maps

• 14 rooms (6 straight ahead, 8 turns)

• Some paths go through same room twice

For example:


Cognitive Map Learning

• Use arrow keys to go through green door

• Determine if path crosses itself

• Remember full path


Cognitive Map Memory Test

• Tested on memory for maze– Forward test and backward test– Measured task time & number of correct

doors opened on first attempt

• Same controls as in learning phase, but without green door guides

• Given feedback


Experimental Design

Large FOV: 120 degreesSmall FOV: 100 degrees

Optical FlowPresent

Optical FlowAbsent

Male

Female


Experimental Procedure

• Paper folding test of spatial ability

• 1 practice trial + 4 test trials

• Satisfaction questionnaire


Benefits of Optical Flow

Benefits of Optical Flow Cues

0102030405060708090

100

Display Condition

Ave

rag

e T

ask

Tim

e (S

ecs)

No Optical Flow Cues

Optical Flow Cues


Optical Flow Helps All Users in Forward Test

Direction x Optical Flow

0102030405060708090

100

Display Condition

Ave

rag

e T

ask

Tim

e (S

ecs)

Forward-No Flow

Forward-Flow

Backward-No Flow

Backward-Flow

Forward Backward


Optical Flow Benefits Females More in the Forward Test

Gender x Optical Flow: Forward Trials

0102030405060708090

100

Display Condition

Ave

rag

e T

ask

Tim

e (S

ecs)

Females-No Flow

Females-Flow

Males-No Flow

Males-Flow

Females Males


Other Results

• No effects for field of view

• No effects for spatial ability measure

• Satisfaction ratings matched performance results


Conclusion1. Optical flow cues help all users form

better cognitive maps when navigating 3D virtual environments

2. Better optical flow cues help women more than men in cognitive map formation• Unexplained by biases in spatial ability

3. Wider displays offer even better optical flow cues• 100 degree field of view seems sufficient


Information Voyeurism: Social Impact of Large Displays

• Exploit social cues induced by physical size:– Help people communicate– Increase productivity on individual tasks

• Must quantify in order to exploit

• Information on large displays more public– Ask user? Cannot guarantee accuracy– Video? Cannot disambiguate glance from

reading


Measuring ‘Peeking’

• Implicit memory priming paradigm– Expose user to stimulus– Test user implicitly on how much they’ve

processed stimulus• Word stem completion• Eg. Mon_____

– Priming measured by faster response or higher frequency of stimulus

• Monkey, Money, Monster, Monday, Monopoly, …


Experiment Materials

• Stimulus: 30 words embedded in:– 7 e-mail subject lines– 2 e-mail messages

• Place e-mail where it can be seen by user

• Priming test to see if they’ve read it– Word stem completion


Experimental Setup

156″

38″

16″27.5″

66″114″

Experimenter

Participant

Large ProjectionScreen

Small Desktop Monitor


Implicit Memory Results

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

1 2Small Display Large Display

Avera

ge #

of

Targ

et

Word

s

N=12

N=12


Other Converging Data

• More users admitted reading text on:– Large Screen (7/12) vs.

Small Screen (3/12)

• Comments indicated reading someone else’s e-mail more acceptable on large screen

• Video shows users glanced more at:– Large Screen (M=19 seconds) vs.

Small Screen (M=14 seconds)


Future Work• Understand why large displays more public

– Physical proximity of information to owner?– Wall-mounted nature of large display?

• Protect private information from prying eyes– Private information never placed on public

screens– Interface conventions that convey level of privacy

• Facilitate ad hoc collaboration– Display systems that make people interact more


Summary

• Novel application of implicit memory priming paradigm for measuring if someone has peeked at information

• People are more willing to read someone else’s e-mail on large wall-sized displays than on smaller displays– Even with identical visual angles and legibility

• Future Work


Future Research Directions

• Continue researching benefits of and methods for testing large displays

• Task management for information workers• Reconfigurable information display

surfaces– Heterogeneous devices

• Social impact, collaboration and communication scenarios

• Gestures, vision and tracking• Novel UI and interaction techniques


Thank you…


Large Display Surfaces are Here

Workstation in the real world (InfoCockpit--CMU)


Optical Flow Benefits All Users Equally in the Backward Test

Gender x Optical Flow: Backward Trials

0102030405060708090

100

Display Condition

Ave

rag

e T

ask

Tim

e (S

ecs)

Females-No Flow

Females-Flow

Males-No Flow

Males-Flow


User Views of Maze

Narrow field of view (100 degrees)

Wide field of view (120 degrees)

© 2003 microsoft corporation. all rights reserved. large display user interaction mary czerwinski,...

Documents

microsoft corporation

surfaces slide

smaller slide

matrix slide

real world slide

group membership slide

large displays

large display user experience