the glass class lecture 7: future research
DESCRIPTION
Lecture 7 in the Glass Class course. Presented on February 21st 2014 by Mark Billinghurst. This lecture discusses directions for future research using Google Glass.TRANSCRIPT
![Page 1: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/1.jpg)
The Glass Class: Lecture 7 – Future Research
Feb 17th – 21st 2014
Mark Billinghurst, Gun Lee HIT Lab NZ
University of Canterbury
![Page 2: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/2.jpg)
THE GLASS CLASS
![Page 3: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/3.jpg)
THE GLASS CLASS
“The best way to Predict the future is to Invent it.” Alan Kay Computer Scien3st (1940-‐ )
![Page 4: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/4.jpg)
THE GLASS CLASS
Directions for Research New devices Input methods User experience Scaling up Social Consequences
![Page 5: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/5.jpg)
New Devices
![Page 6: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/6.jpg)
THE GLASS CLASS
Kopin Pupil
Eye-Glass display 428x240 resolution Voice interactivity
![Page 7: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/7.jpg)
THE GLASS CLASS
GlassUp - http://www.glassup.net/
Glasses form factor – 320x240 pixel resolution Secondary mobile display
![Page 8: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/8.jpg)
THE GLASS CLASS
Telepathy One -http://tele-pathy.org/
Minimal display
![Page 9: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/9.jpg)
THE GLASS CLASS
![Page 10: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/10.jpg)
THE GLASS CLASS
Samsung Galaxy Gear
Watch based wearable
![Page 11: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/11.jpg)
THE GLASS CLASS
Samsung Galaxy Gear
![Page 12: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/12.jpg)
THE GLASS CLASS
Nike Fuelband
Activity/sleep tracking
![Page 13: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/13.jpg)
THE GLASS CLASS
Device Ecosystem
![Page 14: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/14.jpg)
THE GLASS CLASS
Wearable Attributes
fafds
![Page 15: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/15.jpg)
Input Techniques/User Experience
![Page 16: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/16.jpg)
THE GLASS CLASS The Vision of AR
![Page 17: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/17.jpg)
THE GLASS CLASS
To Make the Vision Real.. Hardware/software requirements
Intelligent systems Contact lens displays Free space hand/body tracking Speech/gesture recognition Etc..
Most importantly Usability
![Page 18: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/18.jpg)
THE GLASS CLASS
Environment Sensing
Create virtual mesh over real world
Update at 10 fps – can move real objects
Use by physics engine for collision detection (virtual/real)
Use by OpenScenegraph for occlusion and shadows
![Page 19: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/19.jpg)
THE GLASS CLASS
Natural Hand Interaction
Using bare hands to interact with AR content MS Kinect depth sensing Real time hand tracking Physics based simulation model
![Page 20: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/20.jpg)
THE GLASS CLASS
Meta Gesture Interaction
Depth sensor + Stereo see-through
![Page 21: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/21.jpg)
THE GLASS CLASS
Meta Video
![Page 22: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/22.jpg)
THE GLASS CLASS
Gesture Based Interaction
3 Gear Systems Kinect/Primesense Sensor Two hand tracking http://www.threegear.com
![Page 23: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/23.jpg)
THE GLASS CLASS
Gesture Interaction + AR
HMD AR View Viewpoint tracking
Two hand input Skeleton interaction, occlusion
![Page 24: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/24.jpg)
THE GLASS CLASS
Multimodal Interaction
Combined speech and Gesture Input Free-hand gesture tracking Semantic fusion engine (speech + gesture input history)
![Page 25: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/25.jpg)
THE GLASS CLASS
User Evaluation
Change object shape, colour and position Results
MMI signif. faster (11.8s) than gesture alone (12.4s) 70% users preferred MMI (vs. 25% speech only)
Billinghurst, M., & Lee, M. (2012). Multimodal Interfaces for Augmented Reality. In Expanding the Frontiers of Visual Analytics and Visualization (pp. 449-465). Springer London.
![Page 26: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/26.jpg)
THE GLASS CLASS
Contact Lens Display Babak Parviz
University Washington MEMS components
Transparent elements Micro-sensors
Challenges Miniaturization Assembly Eye-safe
![Page 27: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/27.jpg)
THE GLASS CLASS
Contact Lens Prototype
![Page 28: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/28.jpg)
THE GLASS CLASS
Intelligent Feedback
Actively monitors user behaviour Implicit vs. explicit interaction
Provides corrective feedback
![Page 29: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/29.jpg)
Scaling Up
![Page 30: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/30.jpg)
THE GLASS CLASS
Ego-Vision Collaboration
Google Glass camera + processing + display + connectivity
![Page 31: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/31.jpg)
THE GLASS CLASS
Ego-Vision Research System
How do you capture the user's environment? How do you provide good quality of service?
Interface What visual and audio cues provide best experience? How do you interact with the remote user?
Evaluation How do you measure the quality of collaboration?
![Page 32: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/32.jpg)
THE GLASS CLASS
AR + Human Computation Human Computation
Real people solving problems difficult for computers
Web-based, non real time Little work on AR + HC
AR attributes Shared point of view Real world overlay Location sensing
What does this say?
![Page 33: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/33.jpg)
THE GLASS CLASS
Human Computation Architecture
Add AR front end to typical HC platform
![Page 34: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/34.jpg)
THE GLASS CLASS
AR + HC Research Questions System
What architecture provides best performance? What data is needed to be shared?
Interface What cues are needed by the human computers? What benefits does AR provide cf. web systems?
Evaluation How can the system be evaluated?
![Page 35: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/35.jpg)
THE GLASS CLASS
Scaling Up
Seeing actions of millions of users in the world Augmentation on city/country level
![Page 36: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/36.jpg)
THE GLASS CLASS
AR + Smart Sensors + Social Networks
Track population at city scale (mobile networks) Match population data to external sensor data
medical, environmental, etc
Mine data to improve social services
![Page 37: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/37.jpg)
THE GLASS CLASS
![Page 38: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/38.jpg)
THE GLASS CLASS
![Page 39: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/39.jpg)
THE GLASS CLASS
Orange Data for Development
Orange made available 2.5 billion phone records 5 months calls from Ivory Coast
> 80 sample projects using data eg: Monitoring human mobility for disease modeling
![Page 40: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/40.jpg)
THE GLASS CLASS
Research Questions System
How can you capture the data reliably? How can you aggregate and correlate the information?
Interface What data provides the most values? How can you visualize the information?
Evaluation How do you measure the accuracy of the model?
![Page 41: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/41.jpg)
Social Consequences
![Page 42: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/42.jpg)
THE GLASS CLASS
The Future of Wearables
![Page 43: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/43.jpg)
THE GLASS CLASS
Sight Video Demo
![Page 44: The Glass Class Lecture 7: Future Research](https://reader033.vdocuments.net/reader033/viewer/2022060110/555ae5f1d8b42abe058b45b5/html5/thumbnails/44.jpg)
THE GLASS CLASS
More Information Mark Billinghurst
Email: [email protected] Twitter: @marknb00
HIT Lab NZ http://www.hitlabnz.org/