dialog design - gesture & pen interfaces, mobile devices cs / psych 6750 1 this material has...
TRANSCRIPT
Dialog Design - Gesture & Pen Interfaces, Mobile Devices
CS / Psych 6750 1
This material has been developed by Georgia Tech HCI faculty, and continues to evolve. Contributors include Gregory Abowd, Jim Foley, Elizabeth Mynatt, Jeff Pierce, Colin Potts, Chris Shaw, John Stasko, Bruce Walker, and Melody Moore Jackson. Comments directed to [email protected] are encouraged. Permission is granted to use with acknowledgement for non-profit purposes. Last revision: October 2007.
Dialog Styles
1. Command languages 2. WIMP - Window, Icon, Menu, Pointer3. Direct manipulation4. Speech/natural language5. Gesture & pen
CS / Psych 6750 2
Agenda
PDA overviewPen input styles
CS / Psych 6750 3
How to use a PDA
CS / Psych 6750 4
Personal Digital Asst. (PDA)
CS / Psych 6750 5
Palm Treo
Apple Newton (1993) Dell / Asus A639 GPS PDA
Apple iPhone
Blackberry Curve
PDAs
Now ubiquitousSmall displaysOften touch and pen interfacesSmall thumb-based keyboardsRecent Improvements
Wi-Fi, GPS, more memory, better CPU, better OS, BlueTooth
CS / Psych 6750 6
Is it a PDA? Phone? GPS? Camera? Computer?
Line between devices is blurred today
Apple iPhone – phone, MP3 player, PDA, camera
Palm Treo 700w – phone, Windows computer, PDA, camera
Asus MyPal – Windows computer, GPS device
CS / Psych 6750 7
No Shredder…
CS / Psych 6750 8
Input Options
Pen / Stylus is dominant form Main techniques
Free-form ink Soft keyboard Numeric keyboard => text Stroke recognition - strokes not in the shape of characters Hand printing / writing recognition
Sometimes have or can connect keyboard
CS / Psych 6750 9
Free-form Ink
Ink is the data, take as is
Human is responsible forunderstanding andinterpretation
Like a sketch pad
CS / Psych 6750 10
Soft Keyboards
Common on PDAs and mobile devices
CS / Psych 6750 11
Soft Keyboard
Presents a small diagram of keyboard
You click on buttons/keys with pen or finger
QWERTY vs. alphabetical Tradeoffs? Alternatives?
12CS / Psych 6750
Apple iPhone soft keyboard
Numeric Keypad -T9
Tegic Communications developedYou press out letters of your word, it matches
the most likely word, then gives optional choices
Faster than multiple presses per keyUsed in mobile phoneswww.tegic.com/t9
CS / Psych 6750 13
Stroke Recognition - Graffiti
Graffiti - Unistroke alphabet on Palm PDAWhat are your
experienceswith Graffiti?
Graffiti demo:mms://199.77.128.107/pub/hcirep/demos/
DEMO-palm.wmv
CS / Psych 6750 14
Stroke Recognition - Cirrin
Developed by Jen Mankoff (GT -> Berkeley CS Faculty)
Word-level unistroke techniqueUIST ‘98 paperUse stylus to go
from one letterto the next ->
CS / Psych 6750 15
Hand Printing & Hand Writing Recognition
Recognizing letters and numbers and special symbols Lots of commercial systems English, kanji, etc. Not perfect, but people aren’t either!
People - 96% handprinted single characters Computer - >97% is really good
OCR (Optical Character Recognition)
CS / Psych 6750 16
Recognition Issues Off-line vs. On-line
Off-line: After all writing is done, speed not an issue, only quality.Work with either a bit map or vector sequence
On-line: Must respond in real-time - but have richer set of features - acceleration, velocity, pressure
Use best-guess pattern matching, including digram, trigram probabilities and word lists to remove ambiguity 1 I l
CS / Psych 6750 17
More Issues
Boxed vs. Free-Form inputSometimes encounter boxes on forms
Printed vs. CursiveCursive is much more difficult to impossible
Letters vs. WordsCursive is easier to do in words vs individual letters,
as words create more context
CS / Psych 6750 18
Pen Gesture Commands
CS / Psych 6750 19
-Might mean delete
-Insert
-Paragraph
Define a series of (hopefully) simple drawing gesturesthat mean different commands in a system
Pen Use Modes
Often, want a mix of free-form drawing and special commands
How does user switch modes?Mode icon on screenButton on penButton on device
CS / Psych 6750 20
Error Correction
Having to correct errors can slow input tremendously
StrategiesErase and try againWhen uncertain system shows list of best guesses...
CS / Psych 6750 21
A Different Application
Signature verification
But not with a mouse :)
CS / Psych 6750 22
Multi-touch interfaces
Apple iPhone
CS / Psych 6750 23
Capacitive touchscreen:http://www.apple.com/iphone/features/index.html#touch
Gestures:flick, tap, pinch, un-pinch
http://www.apple.com/iphone/features/index.html#map