design visual thinking tools for mixed-initiative systems
TRANSCRIPT
![Page 1: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/1.jpg)
Design visual thinking tools for mixed-initiative systems
Pearl PuDenis Lalanne
Ergonomics of Intelligent Systems & DesignSwiss Institute of Technology Lausanne
Monday, December 17, 2001
![Page 2: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/2.jpg)
IUI 2
002,
San
Fra
ncis
co
An example of visual thinking tool
• Input, output• Calculates (add)• Transforms cognitive
task into motor and reading task
• = constrains and guides computation
![Page 3: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/3.jpg)
IUI 2
002,
San
Fra
ncis
co
External representation work (Zhang & Norman)
• External representation is more than inputs, stimuli, memory aids
• Externalization is intrisinc components of cognitive tasks
Can we turn cognitive tasks into motor or perception-based tasks?
![Page 4: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/4.jpg)
IUI 2
002,
San
Fra
ncis
co
Perception-based computation -information visualization
• Humans posses highly parallel perceptual processing power
• Visual affordance – how to react to an object by its appearance
• Most commonly infovis is about using images to – Reveal evidence, patterns, and trends, and involve users
in the decision process
• Combining AI and Information visualization seems to be reasonable starting for visual thinking tools
Executive notebook:
But the challenge is to ….
Executive notebook:
But the challenge is to ….
![Page 5: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/5.jpg)
IUI 2
002,
San
Fra
ncis
co
Emphasis Techniques
• Use preattentive perception to allow users to “see” rather than “read” relationships
• Created by applying a transformation function to a visual scale
![Page 6: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/6.jpg)
IUI 2
002,
San
Fra
ncis
co
Outline of the talk
• Describe configuration tasks• Design of visual cues in a visualization-
enabled MI system (COMIND)– Kaleidoscope (search)– Tradeoff Map (select optimal solution)– Conflict lattice (detect formulation conflict,
redefine problem, show unseen path)• User study• Conclusion -- visual thinking tools
![Page 7: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/7.jpg)
IUI 2
002,
San
Fra
ncis
co
Configuration tasks
• PCs, automobiles, travel planning, etc.• Configure a set of objects so that the
interrelationships of those objects respect a certain constraints
• Hard for both humans and machines• Constraint problem solving (CSP) techniques are
often used to solve configuraiton problems • Constraints can be add to suit different user profiles
![Page 8: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/8.jpg)
City planning
![Page 9: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/9.jpg)
IUI 2
002,
San
Fra
ncis
co
Land characteristics
• Lot3, lot5, lot7 and lot9 are all relatively flat sites with fairly good soil conditions.
• Lot10 and lot12 are moderately sloped sites in a nice wooded location, but have poor soil conditions.
• Lot17 is a very steep site. • Lot11 ant lot17 are elevated sites facing
southwest and down into a valley that has a lake and some wooded area.
![Page 10: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/10.jpg)
1. The dumpsite and the cemetery should not be visible from either of the dwellings.
2. Steep slopes are to be avoided for building.3. Poor soil should be avoided for those land uses that
involve construction. 4. The recreational area has to be near the lake. 5. The highway is noisy and ugly and should be avoided
when locating the apartments, the single-family housing complex and the recreational areas.
6. The supermarket cannot be in front of the single-family houses, the dumpsite, and the cemetery, mainly for esthetical reason.
Criteria and constraints
![Page 11: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/11.jpg)
IUI 2
002,
San
Fra
ncis
co
Solving configuration problem
• The problem-solver's task is to come up with assignments of land uses to sites.
• A complete design is one in which each land use has been assigned to a lot.
• The final design should be one which complies with a given set of criteria.
![Page 12: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/12.jpg)
Search algorithms for CSP
• simple backtracking• pre-processing and SB• Monte Carlo method by Knuth• algorithms are np-complete in general
![Page 13: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/13.jpg)
Kaleidoscope - for SB
![Page 14: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/14.jpg)
Simple backtracking
Knuth algorithm
![Page 15: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/15.jpg)
IUI 2
002,
San
Fra
ncis
co
Discovery with Kaleidoscope
• does thrashing occur, frequently?• Are solutions diversified or concentrated in
clusters?• Are solutions abundant or futile• if variables re-ordered, does solution
generation become faster?
![Page 16: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/16.jpg)
IUI 2
002,
San
Fra
ncis
co
TradeoffMap –under-constrained space
![Page 17: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/17.jpg)
MAP: multiple attribute pareto
![Page 18: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/18.jpg)
![Page 19: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/19.jpg)
IUI 2
002,
San
Fra
ncis
co
Decision making using MAP
• Is there a dominant solution?• Are there numerous or few non-dominant
solutions?• Should additional criteria be defined in order
to push out dominant solutions?• Are solutions cluttered around a certain
area, or more spread out in the MAP?
![Page 20: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/20.jpg)
IUI 2
002,
San
Fra
ncis
co
• one or several sets of constraints contain no solution
• diagnosing them is hard without visualization
• Lattice visualization
Search in over-constrained space
![Page 21: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/21.jpg)
Lattice explained
• Each set is a square• Sets are smaller in
size than the ones below
• Two cases of conflicts– Inconsistencies– Overconstrained
![Page 22: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/22.jpg)
Kaleidoscope, MAP, and Lattice
![Page 23: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/23.jpg)
IUI 2
002,
San
Fra
ncis
co
Resolve conflicts
• Is there a single or several minimal conflicts (black squares) in the lattice?
• Which one of the conflict sets to relax? • If certain conflicts are removed, which
potential solutions are ideal? (use visualization of MAP)
![Page 24: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/24.jpg)
IUI 2
002,
San
Fra
ncis
co
Related works
• Mixed initiative system - general principles (Hovitz)
• HCI principles for interactive search -specific principles
• Human-guided search – machines find local minima, humans pin point search space
![Page 25: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/25.jpg)
IUI 2
002,
San
Fra
ncis
co
What is mixed-initiative systems --Maes v.s. Shneiderman
Autonomousagents
Directmanipulation
Proactive Strictlytools
Infering Users' goals Users are
in control
MixedInitiativesystems
Executive notebook:
Future style of interaction
Executive notebook:
Future style of interaction
![Page 26: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/26.jpg)
IUI 2
002,
San
Fra
ncis
co
Our work in comparison with other mixed-initiative systems
• Dialog-based• Direct manipulation-based• Differentiating factor
– Visualization-enabled MI interface– User task and context centered
Executive notebook:
Based on visual metaphors, these systems only recently begin to appear.
Executive notebook:
Based on visual metaphors, these systems only recently begin to appear.
![Page 27: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/27.jpg)
IUI 2
002,
San
Fra
ncis
co
Usability study – what to test
• Hypotheses tested– Can humans solve the problems without
COMIND? – Designers can perform better with visualization -
yes– Help designers discover new solutions – yes
• Observations– More lateral behavior (users do not follow
routines)– More eye movements
![Page 28: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/28.jpg)
IUI 2
002,
San
Fra
ncis
co
Requirements for visual thinking tools - conclusion
• Valued-added automation in the form of how results contribute to tasks
• Visual reification of task• Suggest appropriate actions with visual affordances
– Constrained set of operators (reduce errors, direct manipulation)
– Evaluation of solution path by giving assessment between goals and current state (mirrors your strategy, progress gauge)
– point out unseen paths
Turn cogntive tasks into perception-based and decision tasks
![Page 29: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/29.jpg)
End
![Page 30: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/30.jpg)
IUI 2
002,
San
Fra
ncis
co
Our objectives
• Does MIUI stands a chance to be the next generation interaction style?
• Designing Visual thinking tools Executive notebook:
A central concern to IUI
Executive notebook:
A central concern to IUI
Executive notebook:
Executive notebook:
![Page 31: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/31.jpg)
IUI 2
002,
San
Fra
ncis
co
An example
• Tower of hanoi• Rings, coffee, search
space
![Page 32: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/32.jpg)
IUI 2
002,
San
Fra
ncis
co
Modification to papers
• Keiladoscope, tradeoff map, and conflict resolution alone would not be sufficient, but the original task map linked with these tools make up the reification task.
• Incoporate Zhang and Norman's work. It's not enough for external representation, but abstraction process must be reified. Hanoi example.
![Page 33: Design visual thinking tools for mixed-initiative systems](https://reader031.vdocuments.net/reader031/viewer/2022020704/61fb523d2e268c58cd5cc9b2/html5/thumbnails/33.jpg)
IUI 2
002,
San
Fra
ncis
co
Some more images as backup slidesOtherwise this presentation is done