a toolkit for managing user attention in peripheral displays tara matthews, anind k. dey*٭,...
Post on 17-Dec-2015
216 Views
Preview:
TRANSCRIPT
A Toolkit for Managing User Attention in Peripheral
DisplaysTara Matthews, Anind K. Dey*٭, Jennifer Mankoff٭, Scott Carter, Tye Rattenbury
EECS Department, UC Berkeley*Intel-Berkeley Research
currently of HCII, Carnegie Mellon٭
10/27/2004 Peripheral Display Toolkit 2
Problematic Scenario• Bus schedule
– Requires a task switch to use– Doesn’t afford continuous monitoring
RTE LAWR GRIZ EUCL CNTRNUM HALL SR. GRIZ SHAT65 605a 616a 630a 65 635a 646a 700a 65 703a 716a 730a 65 735a 746a 800a 65 803a 816a 830a 65 835a 846a 900a 65 903a 916a 930a 65 935a 946a 1000a 65 1003a 1016a 1030a 65 1035a 1046a 1100a 65 1103a 1116a 1130a 65 1135a 1146a 1200p 65 1203p 1216p 1230p 65 1235p 1246p 100p 65 103p 116p 130p 65 135p 146p 200p 65 203p 216p 230p 65 235p 246p 300p65 303p 316p 330p 65 335p 346p 400p
10/27/2004 Peripheral Display Toolkit 3
Solution: Peripheral Displays• Provide awareness with min attention
• Separate from primary task
• Bus LED Display+ Get bus info at a glance+ Dynamically manages attention it attracts– Hard to accomplish
10/27/2004 Peripheral Display Toolkit 4
Why is creating PDs hard?• Need to abstract info to be glance-able
• Need mechanisms for dynamically managing attention PDs attract:– Deciding attention levels to attract
(notification levels)– Displaying info appropriately (transitions)
• Our contribution:– Peripheral Display Toolkit (PTK) supports
these key issues in PD creation
10/27/2004 Peripheral Display Toolkit 5
Example PTK Applications• Remote Activity
– Social Guitar– Audio Monitor– Motion Monitor– Remote Awareness Display
• Bus Displays– Bus Mobile– Bus LED
• Instant Messenger Status
IM Picture FrameSocial Guitar Bus LED BusMobile
Orb showing remote activity
10/27/2004 Peripheral Display Toolkit 6
Example PTK Applications• Remote Activity
– Social Guitar– Audio Monitor– Motion Monitor– Remote Awareness Display
• Bus Displays– Bus Mobile– Bus LED
• Instant Messenger Status
IM Picture FrameSocial Guitar Bus LED BusMobile
Orb showing remote activity
10/27/2004 Peripheral Display Toolkit 7
Example PTK Applications• Remote Activity
– Social Guitar– Audio Monitor– Motion Monitor– Remote Awareness Display
• Bus Displays– Bus Mobile– Bus LED
• Instant Messenger Status
IM Picture FrameSocial Guitar Bus LED BusMobile
Orb showing remote activity
10/27/2004 Peripheral Display Toolkit 8
Example PTK Applications• Remote Activity
– Social Guitar– Audio Monitor– Motion Monitor– Remote Awareness Display
• Bus Displays– Bus Mobile– Bus LED
• Instant Messenger Status
IM Picture FrameSocial Guitar Bus LED BusMobile
Orb showing remote activity
10/27/2004 Peripheral Display Toolkit 9
Example PTK Applications+ Manage attention with
abstraction, notification levels, & transitions
+ Glance-able+ Few lines of code+ Simplified code design
& code re-use
IM Picture FrameSocial Guitar Bus LED BusMobile
Orb showing remote activity
10/27/2004 Peripheral Display Toolkit 10
Simple Example: Motion Monitor
no/low motion
medium motion
high motion
10/27/2004 Peripheral Display Toolkit 11
Outline• PTK: Motivation & Examples
• Managing attention– Abstraction– Notification levels– Transitions
• Toolkit Architecture & Library
• Conclusion
10/27/2004 Peripheral Display Toolkit 12
Managing User Attention
• Attention is a crucial issue in design– Significant impact on success– More info can be monitored w/o overburdening
• Attention management is difficult– Used to designing apps that attract full attention– Attention attracted should match info urgency
• Literature surveys led to key characteristics of attention management
info urgency user attention
Attentionnone focal
10/27/2004 Peripheral Display Toolkit 13
Attention Management Characteristics
info urgency user attention
From survey of peripheral displays:1. Abstraction
• enables lower-attention info consumption
2. Notification levels• help determine how much attention to
attract based on urgency of info
3. Transitions• enable display of info to match urgency
10/27/2004 Peripheral Display Toolkit 14
Abstraction• Enables lower-attention consumption of info• Extracting features or reducing fidelity of info• Motion Monitor: replaces video images with
general motion amount (high, medium, none)
video abstraction amount of
images motion
• high motion
• medium motion
• no/low motion
10/27/2004 Peripheral Display Toolkit 15
Notification Levels• Indicate the urgency of info
– Low levels = non-urgent info– High levels = more urgent info
• Motion Monitor: doesn’t show urgent info, so uses low notification levels
10/27/2004 Peripheral Display Toolkit 16
More on Notification Levels
• Attention is (roughly) scalar – PD designers must decide where on the scale displayed info should fall
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 17
More on Notification Levels
Attention Type1. None
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 18
More on Notification Levels
Attention Type1. None
Notification LevelIgnore
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 19
More on Notification Levels
Attention Type1. None2. Inattention
Notification LevelIgnore
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 20
More on Notification Levels
Attention Type1. None2. Inattention
Notification LevelIgnoreChange blind
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 21
More on Notification Levels
Attention Type1. None2. Inattention3. Divided
Notification LevelIgnoreChange blind
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 22
More on Notification Levels
Attention Type1. None2. Inattention3. Divided
Notification LevelIgnoreChange blindMake aware
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 23
More on Notification Levels
Attention Type1. None2. Inattention3. Divided4. Focused
Notification LevelIgnoreChange blindMake aware
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 24
More on Notification Levels
Notification LevelIgnoreChange blindMake awareInterrupt
Attention Type1. None2. Inattention3. Divided4. Focused
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 25
More on Notification Levels
Notification LevelIgnoreChange blindMake awareInterruptDemand action
Attention Type1. None2. Inattention3. Divided4. Focused5. Focused + interaction
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 26
Transitions
• Enable display of info to match urgency
• Depend on notification level– Ignore: no changes– Change blind: fading, slow motion, tiny
updates– Make aware: discrete, abrupt updates– Interrupt: flashing, beeping, vibration– Demand action: interrupt until user does action
• Motion Monitor: varies pulse speeds for different notification levels
10/27/2004 Peripheral Display Toolkit 27
Outline• PTK: Motivation & Examples
• Managing attention– Abstraction– Notification levels– Transitions
• Toolkit Architecture & Library
• Conclusion
10/27/2004 Peripheral Display Toolkit 28
PTK Architecture1. Support for managing impact on human
attention using abstraction, notification levels, and transitions
2. Simplified code design and code re-use
3. Library of common PD components
Output
Transition
Input
NotificationMap
Abstractor
10/27/2004 Peripheral Display Toolkit 29
Motion Monitor Architecture
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidget
Trans
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidget
Trans OrbWidget
Trans
10/27/2004 Peripheral Display Toolkit 30
Motion Monitor Architecture
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidget
Trans
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidget
Trans OrbWidget
Trans
10/27/2004 Peripheral Display Toolkit 31
Motion Monitor Architecture
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidget
Trans
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidget
Trans OrbWidget
Trans
motion = medium
10/27/2004 Peripheral Display Toolkit 32
Motion Monitor Architecture
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidget
Trans
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidget
Trans OrbWidget
Trans
motion = medium
notification level = change blind
10/27/2004 Peripheral Display Toolkit 33
Motion Monitor Architecture
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidget
Trans
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidget
Trans OrbWidget
Trans
motion = medium
notification level = change blind
10/27/2004 Peripheral Display Toolkit 34
Motion Monitor Architecture
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidget
Trans
Input-Side Output-Side
Motion MonitorPTK
DiscoveryServer
DiscoveryServer
CameraInput
MotionAbstractor
NotificationMap
OrbWidget
Trans OrbWidget
Trans
5 lines of logic code
10/27/2004 Peripheral Display Toolkit 35
Remote Awareness Display
Count # people,How much motion?
Phone ring?Voices?
Add notification level
10/27/2004 Peripheral Display Toolkit 36
Motion Monitor Remote Awareness Display
Input-Side Output-Side
Motion Monitor
DiscoveryServer
CameraInput
PTKDiscovery
Server
MotionAbstractor
Exact Match(Motion) Orb
WidgetTrans
Input-Side Output-Side
Motion Monitor
DiscoveryServer
CameraInput
PTKDiscovery
Server
MotionAbstractor
Exact Match(Motion) Orb
WidgetTrans
10/27/2004 Peripheral Display Toolkit 37
Motion Monitor Remote Awareness Display
Input-Side Output-SideDiscoveryServer
CameraInput
PTKDiscovery
Server
MotionAbstractor
Exact Match(Motion) Orb
WidgetTrans
Motion Monitor
Input-Side Output-SideDiscoveryServer
CameraInput
PTKDiscovery
Server
MotionAbstractor
Exact Match(Motion) Orb
WidgetTrans Orb
WidgetTrans
Motion Monitor
10/27/2004 Peripheral Display Toolkit 38
Motion Monitor Remote Awareness Display
Input-Side Output-SideDiscoveryServer
CameraInput
PTKDiscovery
Server
MotionAbstractor
# PeopleAbstractor
Exact Match(Motion) Orb
WidgetTrans
Motion Monitor
Input-Side Output-SideDiscoveryServer
CameraInput
PTKDiscovery
Server
MotionAbstractor
# PeopleAbstractor
Exact Match(Motion) Orb
WidgetTrans Orb
WidgetTrans
Motion Monitor
10/27/2004 Peripheral Display Toolkit 39
Motion Monitor Remote Awareness Display
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
PTKDiscovery
Server
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion) Orb
WidgetTrans
Motion Monitor
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
PTKDiscovery
Server
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion) Orb
WidgetTrans Orb
WidgetTrans
Motion Monitor
10/27/2004 Peripheral Display Toolkit 40
Motion Monitor Remote Awareness Display
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
PTKDiscovery
Server
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion)
Exact Match(Audio)
Threshold(# People)
OrbWidget
Trans
Motion Monitor
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
PTKDiscovery
Server
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion)
Exact Match(Audio)
Threshold(# People)
OrbWidget
Trans OrbWidget
Trans
Motion Monitor
10/27/2004 Peripheral Display Toolkit 41
Motion Monitor Remote Awareness Display
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
PTKDiscovery
Server
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion)
Exact Match(Audio)
Threshold(# People)
OrbWidget
Trans
Orb Wrapper
Motion Monitor
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
PTKDiscovery
Server
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion)
Exact Match(Audio)
Threshold(# People)
OrbWidget
Trans OrbWidget
Trans
Orb Wrapper
Motion Monitor
10/27/2004 Peripheral Display Toolkit 42
Remote Awareness Display
54 – total lines of code
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion)
Exact Match(Audio)
Threshold(# People)
OrbWidget
Trans
Orb Wrapper
TickerWidget
Trans
Tikr. Wrapper
PTKDiscovery
Server
Remote Awareness Display
Input-Side Output-SideDiscoveryServer
CameraInput
AudioInput
MotionAbstractor
# PeopleAbstractor
PhoneAbstractor
VoiceAbstractor
Exact Match(Motion)
Exact Match(Audio)
Threshold(# People)
OrbWidget
Trans OrbWidget
Trans
Orb Wrapper
TickerWidget
Trans TickerWidget
Trans
Tikr. Wrapper
PTKDiscovery
Server
Remote Awareness Display
10/27/2004 Peripheral Display Toolkit 43
General Architecture Diagram
Input-Side DiscoveryServer
Output-Side
Peripheral Display 1
(Abstractors) (NotificationMaps)
(Output Widgets w/ optional Transition)
Peripheral Display 2
Input 1
Input 2
Input N
Abs. 1
Abs. NPTK
DiscoveryServer
Abs. 2
Abs. N
Abs. 1 N.M. 1
N.M. 2
N.M. N
Out 1Trans
Out 2
Out N
.
.
. …
Abs. 2
Abs. N
Abs. 1 N.M. 1
N.M. 2
N.M. N
Out 2Trans
Out 1
Out N
…
Input-Side DiscoveryServer
Output-Side
Peripheral Display 1
(Abstractors) (NotificationMaps)
(Output Widgets w/ optional Transition)
Peripheral Display 2
Input 1
Input 2
Input N
Abs. 1
Abs. NPTK
DiscoveryServer
Abs. 2
Abs. N
Abs. 1 N.M. 1
N.M. 2
N.M. N
Out 1Trans Out 1Trans
Out 2
Out N
.
.
. …
Abs. 2
Abs. N
Abs. 1 N.M. 1
N.M. 2
N.M. N
Out 2Trans Out 2Trans
Out 1
Out N
…
10/27/2004 Peripheral Display Toolkit 44
Library Components• Input
– audio, camera, Phidgets, Context Toolkit, online calendars, news, stocks, weather,Web page parser, serial port communication
10/27/2004 Peripheral Display Toolkit 45
Library Components• Input
– audio, camera, Phidgets, Context Toolkit, online calendars, news, stocks, weather,Web page parser, serial port communication
• Output– ticker text, Ambient Orb, Phidgets
10/27/2004 Peripheral Display Toolkit 46
Library Components• Input
– audio, camera, Phidgets, Context Toolkit, online calendars, news, stocks, weather,Web page parser, serial port communication
• Output– ticker text, Ambient Orb, Phidgets
• Abstractors– motion, people counting, voices, phone ringing
10/27/2004 Peripheral Display Toolkit 47
Library Components• Input
– audio, camera, Phidgets, Context Toolkit, online calendars, news, stocks, weather,Web page parser, serial port communication
• Output– ticker text, Ambient Orb, Phidgets
• Abstractors– motion, people counting, voices, phone ringing
• Notification– exact match, threshold, contains,
degree of change
10/27/2004 Peripheral Display Toolkit 48
Transitions• PTK provides architectural support
for implementing as animations• Timing• Number of steps• Flashing (single, repeated)
• Tied to output– Depend on output modality– Example transitions: color changing and
pulsing (orb); fading, motion, & flashing (ticker text)
10/27/2004 Peripheral Display Toolkit 49
Future Work & Conclusion• PTK supports the creation of PDs• Focuses on managing user attention• To do this, uses abstraction,
notification levels, transitions• Validated with 7 example applications
– Supports attention management– Facilitates code re-use
• Future work– Incorporate interruptibility and
local context sensing– Cross-modality animations– PD evaluation support
10/27/2004 Peripheral Display Toolkit 50
Questions?
• For more information & downloads:
www.cs.berkeley.edu/projects/io/ptk/
tmatthew@cs.berkeley.edu
Thanks to…
NSF, Ed De Guzman, Jeff Heer,Scott Klemmer, Berkeley & CMU
10/27/2004 Peripheral Display Toolkit 51
What about user context?• PDs must 2 sets of attention issues:
– User context• interruptibility, primary activity, focus of attention
– Attention management• balancing info urgency and user attention
• PTK focuses on attention management– User context invasive to sense, not practical to
assume PDs have access– Management issues are independent of user
context
• Context input can be considered on a library level (in notification maps)
10/27/2004 Peripheral Display Toolkit 52
Related Work• AROMA: architectural support for abstraction
• Context Toolkit: infrastructure for context input and some abstraction
• Phidgets: library of hardware widgets for input and output
• Real World Interfaces Toolkit: library of X10 output widgets
• iStuff: infrastructure for distributed inputs and outputs
10/27/2004 Peripheral Display Toolkit 53
Focus Attention
• Peripheral display + attracting focused attention = contradiction? Nope!– Notifications enable displays to be peripheral
when not displaying urgent info– Attention is scalar: PDs should cover entire range
0% 100%attention
awareness
unconscious
10/27/2004 Peripheral Display Toolkit 54
PTK ValidationCreated 7 example applications that illustrate:• Supports attention management
• 3 used abstraction
• 5 used notification levels and transitions
• Easier to create• 5-55 lines of logic code
• Facilitates code re-use• used same input for 4 displays
IM Picture FrameSocial Guitar Bus LED BusMobile
Orb showing remote activity
10/27/2004 Peripheral Display Toolkit 55
Notification Maps
If event value equal to: Set notification level to:
no motion ignore
med motion change blind
high motion make aware
(null) interrupt
(null) demand action
Exact Match Notification Map
10/27/2004 Peripheral Display Toolkit 56
Notification Maps• Exact match
• checks event for equality w/specified values
• Threshold• determines if event within specified ranges
• Contains• checks if event contains specified strings
• Degree of change• determines if event has changed by a
specified amount since last event
10/27/2004 Peripheral Display Toolkit 57
Introduction• Peripheral displays (PDs) enable you
to monitor many info sources while maintaining a calm environment– But, only calm if designed to manage the
attention they attract
• PDs are hard to create– No existing tools or guidelines for
managing human attention
• Peripheral Display Toolkit (PTK):– Enables easier creation of PDs that
manage human attention
10/27/2004 Peripheral Display Toolkit 58
Why Peripheral Displays?• Provide awareness w/ minimal attention
• Separate from user’s primary activity
• Non-focal display not used unless providing peripheral info
• Important to ubicomp vision of many devices to one user
10/27/2004 Peripheral Display Toolkit 59
Peripheral Display Toolkit (PTK)
• Supports the creation of PDs1. Focuses on managing user attention
2. Simplifies code
3. Provides library of common components
• Managers user attention with– Abstraction– Notification levels– Transitions
10/27/2004 Peripheral Display Toolkit 60
Characteristics of Attention Management• Abstraction of info to be glance-able
• Dynamically managed attention– Notification levels for deciding how
much attention to attract– Transitions for displaying info
appropriately
10/27/2004 Peripheral Display Toolkit 61
Peripheral Display Toolkit (PTK)
• Supports the creation of PDs
• Focuses on managing user attention– Balance info urgency and user attention
• To do this, uses– Abstraction– Notification levels– Transitions
10/27/2004 Peripheral Display Toolkit 62
More on Notification Levels
x = attention 100% 0%
y = awareness indirect cognitive
direct cognitive
influence
divided attention
focused attention
pre -
attention
setting (layout, color, objects, names)
inattention
conscious perception (x > 0)
change blind
ambient
alerting
influence
top related