tablet+stylus - stanford university › courses › cs376 › 2016 › slides › cs376-10-in… ·...
TRANSCRIPT
![Page 1: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/1.jpg)
Suggestions from Sean� Tablet+Stylus� https://www.youtube.com/watch?v=9dgHgHQSuuY� http://research.microsoft.com/en-us/um/people/kenh/All-Published-
Papers/Pre-Touch-CHI-2016.pdf� Soli: https://atap.google.com/soli/
1
![Page 2: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/2.jpg)
Input and interaction
MICHAEL BERNSTEINCS 376
![Page 3: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/3.jpg)
Recall: Skinput
![Page 4: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/4.jpg)
Recall: Omnitouch
![Page 5: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/5.jpg)
Recall: SenseCam
![Page 6: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/6.jpg)
Input and interaction research� How can the user interact fluidly with the world around them?� New input modalities: e.g., depth cameras� New output modalities: e.g., pico projectors and fabrication� New user vocabulary: e.g., gestures
� This research is often driven by, or involves the creation of, new hardware
6
![Page 7: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/7.jpg)
Foundations
![Page 8: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/8.jpg)
Bolt. “Put-that-there”: voice and gesture at the graphics interface. SIGGRAPH ’80.
![Page 9: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/9.jpg)
Put That There� Contribution: combined gesture and voice input� In a closed world� With a toy goal� Using simple manipulation operations� Using a laser attached to the wrist
� In many ways, our goal since 1980 has been to relax those assumptions
9
![Page 10: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/10.jpg)
looks a bit like harry potter...
Wellner. Interacting with paper on the DigitalDesk. CACM ‘93.
![Page 11: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/11.jpg)
DigitalDesk� Contribution: fluid boundaries between digital and physical
objects� In a constrained space� On a small set of tasks� With predefined behaviors
� Again, we work to relax these assumptions
11
![Page 12: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/12.jpg)
Input technologies
![Page 13: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/13.jpg)
Tabletop
13
Dietz and Leigh. DiamondTouch: a multi-user touch technology. UIST’ 01.
![Page 14: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/14.jpg)
Tabletop
14
Han. Low-cost multi-touch sensing through frustrated total internal reflection. UIST ’05.
![Page 15: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/15.jpg)
Pen
15Hinckley et al. Pen + touch = new tools. UIST ’10.
![Page 16: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/16.jpg)
Depth sensing
16Izadi et al. KinectFusion. UIST ’11.
![Page 17: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/17.jpg)
Sensing biosignals
17
Saponas et al. Enabling Always-Available Input with Muscle-Computer Interfaces. UIST ’09.
![Page 18: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/18.jpg)
Machine learning model
18
30 millisecond sample
Root Mean Square (RMS) ratios between channels
Frequency Energy10 Hz bands
Phase Coherence ratios between channels
x6 Sensors
Support VectorMachine
user specific model
gesture classification
![Page 19: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/19.jpg)
Output technologies
![Page 20: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/20.jpg)
pico projector
20
Handheld projectors
Willis et al. SideBySide: ad-hoc multi-user interaction with handheld projectors. UIST ’11.
![Page 21: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/21.jpg)
Physically actuated material
21
Follmer et al. Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices. UIST ’12.
![Page 22: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/22.jpg)
jamming uis� \
22
Magnetic levitation
Lee, Post and Ishii. ZeroN: mid-air tangible interaction enabled by computer controlled magnetic levitation. UIST ’11.
![Page 23: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/23.jpg)
23
Projectors
Wilson et al. Steerable augmented reality with the Beamatron. UIST ’12.
![Page 24: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/24.jpg)
24
3D printing
Willis et al. Printed Optics: 3D Printing of Embedded Optical Elements for Interactive Devices. UIST ’12.
![Page 25: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/25.jpg)
Skill sets for input research� Learn “enough to get by” in...� Electrical engineering� Mechanical engineering� Computer graphics
� Known techniques for research in these domains often have direct mappings onto open questions in input
25
![Page 26: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/26.jpg)
Today’s papers
![Page 27: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/27.jpg)
Tangible Bits� What?� An extremely influential vision of making computing something
physically perceivable and manipulable� Why?� Ishii’s Tangible Bits projects laid the foundations
27
![Page 28: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/28.jpg)
Touché� What?� Swept-frequency capacitive sensing to identify user interactions with
the environment� Why?� A great example of a creative use of sensing: using “new parts of the
spectrum” for HCI
28
![Page 29: Tablet+Stylus - Stanford University › courses › cs376 › 2016 › slides › cs376-10-in… · Recall: Skinput. Recall: Omnitouch. Recall: SenseCam. Input and interaction research](https://reader033.vdocuments.net/reader033/viewer/2022042310/5ed8230e0fa3e705ec0de71e/html5/thumbnails/29.jpg)
Discussion rooms
29
Rotation Littlefield 107 Littlefield 104
a 12 34
b 24 13
c 14 23
d 34 12
e 13 24
f 23 14