advanced interaction techniques...history 1983 – videoplace • myron krueger • real time image...
TRANSCRIPT
![Page 1: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/1.jpg)
Advanced Interaction Techniques
Carlos Duarte 2015/2016
Gestures
![Page 2: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/2.jpg)
What is a gesture?
In this context, any physical movement that can be sensed and responded by a digital system without the aid of a traditional input device such as keyboard or mouse.
History• Sketchpad used light-pen gestures
(1963) • Teitelman in 1964 developed the first
trainable gesture recogniser • A very early demonstration of gesture
recognition was Tom Ellis' GRAIL system on the RAND tablet (1964, ARPA funded).
• A gesture-based text editor using proof-reading symbols was developed at CMU by Michael Coleman in 1969.
• 1972 – PLATO IV Touch Screen Terminal. Computer-based Education Research Laboratory, University of Illinois. 16-by-16 grid infrared touch panel
• Gesture recognition has been used in commercial CAD systems since the 1970s, and came to universal notice with the Apple Newton in 1992
![Page 3: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/3.jpg)
History1983 – VIDEOPLACE
• Myron Krueger
• Real time image processing
• Background subtraction and edge detection to create a silhouette of the user
• Introduce Pinch gesture
History1991 – Digital Desk
• Pierre Wellner
• Project a digital surface onto a physical desk
• Use optical and acoustic techniques to sense hands/fingers
• Demonstrate multi-touch concepts such as two finger scaling and translation
![Page 4: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/4.jpg)
History
1992 – Simon
• First smart phone
• Touch-screen
History2006 – Jeff Han
• Frustrated Total Internal Reflection
• Multi-touch touch-screen
• Created Perceptive Pixel
• http://www.youtube.com/watch?v=5JcSu7h-I40
![Page 5: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/5.jpg)
History
2007 – iPhone
• Soft touch-based interface
• Very smooth interaction
• Multi-touch capability to a limited degree
History
2007 – Microsoft Surface
• Interactive table surface
• Capable of sensing multiple fingers and hands
• Capable of identifying various objects and their position on the surface
![Page 6: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/6.jpg)
Characteristics
Types of gestures• Touch
• Single touch
• Multi touch
• Free-form
• Body (hand, head, …) gesture
• Mouse, glove, …
![Page 7: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/7.jpg)
Styles of gestures• Deictic - the gestures of pointing to objects in the
environment
• Manipulative - direct relationship between the movement of the hand and the object to be manipulated
• Symbolic - system of gestures that use a dynamic or static dictionary
• Gesticulation - most natural ways of communicating, frequently used in multimodal interfaces
Gestural interfaces• Pros
• More flexible
• Less visible hardware
• More “natural”
• More fun
• Cons
• Heavy data input
• Relies heavily on the visual
• Can be inappropriate for context
• More physically demanding
![Page 8: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/8.jpg)
Good characteristics• Discoverable (visibility
and affordable)
• Trustworthy
• Responsive
• Appropriate
• Meaningful / Adequate
• Smart
• Clever
• Playful
• Pleasurable
• Good
Attributes• Presence
• Position / Locale
• Duration
• Motion / Speed
• Pressure (if touch)
• Size / Form
• Orientation
• Objects / Context
• Number of touch /recognised points
• Participants
• Sequence
![Page 9: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/9.jpg)
Technology
Sensors• Pressure
• Light
• Proximity
• Acoustic
• Tilt
• Motion
• Orientation
![Page 10: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/10.jpg)
The (really) free
The (not so) free
![Page 11: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/11.jpg)
TouchscreensResistive
• Change in electrical field is noted and the coordinates of the point of contact are calculated
TouchscreensCapacitive
• A layer that stores electrical charge is placed in the monitor
• Some charge is transferred to the user
![Page 12: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/12.jpg)
TouchscreensSurface acoustic wave systems
• Ultrasonic waves pass over the touchscreen panel
• When the panel is touched, a portion of wave is absorbed
TouchscreensInfrared grid
• Finger interrupts infrared light grid • FTIR • RDI • LLP • DSI
![Page 13: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/13.jpg)
TouchscreensInfrared grid
• FTIR - Frustrated Total Internal Reflection • Infrared light is shined into the
side of an acrylic panel • Light is trapped inside the
acrylic by internal reflection • When a finger touches the
acrylic surface this light is “frustrated”
• Picked up by an infrared camera
TouchscreensInfrared grid
• RDI - Rear Diffused Illumination • Infrared light is shined at the
screen from below the touch surface
• Diffuser is placed on top or bottom of the touch surface
• When a finger touches the surface it reflects more light than diffuser
• Picked up by an infrared camera
![Page 14: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/14.jpg)
TouchscreensInfrared grid
• LLP - Laser Light Plane • Infrared light from a laser(s) is
shined just above the surface • Laser plane of light is about
1mm tick • When a finger touches it, it will
hit the tip of the finger which will register as a IR blob
• Picked up by an infrared camera
TouchscreensInfrared grid
• DSI - Diffused Surface Illumination • Uses a special acrylic to
distribute the IR evenly across the surface
• This acrylic uses small particles that are inside the material, acting like thousands of small mirrors
• When a finger touches the surface it reflects more light
• Picked up by an infrared camera
![Page 15: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/15.jpg)
Humans
Ergonomics of motion• Avoid hyperextension or extreme stretches
• Avoid repetition
• Utilize relaxed, neutral positions
• Avoid staying in a static position
• No “Gorilla Arm”
![Page 16: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/16.jpg)
Arms, hands and fingers
Hands and fingers• Fingernails: Blessing and curse • Fake fingernails: evil • Finger oil • Fingerprints • (Left) Handedness • Wrist support • Gloves • Inaccurate (when compared to a cursor) • Attached to a hand (Screen Coverage)
![Page 17: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/17.jpg)
Screen Coverage• Avoid putting essential features or information like a
label below an interface element that can be touched, as it may become hidden by the user’s own hand
Screen Coverage• Avoid putting essential features or information like a
label below an interface element that can be touched, as it may become hidden by the user’s own hand
![Page 18: Advanced Interaction Techniques...History 1983 – VIDEOPLACE • Myron Krueger • Real time image processing • Background subtraction and edge detection to create a silhouette](https://reader033.vdocuments.net/reader033/viewer/2022050404/5f814ec060bc0d004c73e93a/html5/thumbnails/18.jpg)
Screen Coverage• Activity Zones for Touchscreen Tablets and Phone
• Dan Saffer • http://www.kickerstudio.com/blog/2011/01/
activity-zones-for-touchscreen-tablets-and-phones/
Touch Target Tips
Iceberg tips Adaptive targets