Download - COSC 426 Lect. 8: AR Research Directions
![Page 1: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/1.jpg)
Lecture 8: Research Directions
Mark BillinghurstgHIT Lab NZ
University of Canterbury University of Canterbury
![Page 2: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/2.jpg)
Looking to the Future
![Page 3: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/3.jpg)
The Future is with usIt takes at least 20 years for new technologies to go It takes at least 20 years for new technologies to go
from the lab to the lounge..“The technologies that will significantly affect our lives
over the next 10 years have been around for a decade.
The future is with us The trick is learning how to spot it The future is with us. The trick is learning how to spot it. The commercialization of research, in other words, is far more about prospecting than alchemy.”
Bill Buxton Oct 11th 2004
![Page 4: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/4.jpg)
R h Di tiResearch Directions
experiencesUsability
applications Interaction
tools Authoringtools Authoring
components Tracking, Display
Sony CSL © 2004
![Page 5: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/5.jpg)
Research DirectionsCComponents
Markerless tracking, hybrid trackingDi l i t d iDisplays, input devices
ToolsAuthoring tools user generated contentAuthoring tools, user generated content
ApplicationsInteraction techniques/metaphorsInteraction techniques/metaphors
ExperiencesUser evaluation novel AR/MR experiencesUser evaluation, novel AR/MR experiences
![Page 6: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/6.jpg)
HMD DesignHMD Design
![Page 7: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/7.jpg)
Occlusion with See-through HMDTh P blThe Problem
Occluding real objects with virtualOccluding virtual objects with realOccluding virtual objects with real
Real Scene Current See-through HMD
![Page 8: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/8.jpg)
ELMO (Kiyokawa 2001)
Occlusive see-through HMDMasking LCDReal time range finding
![Page 9: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/9.jpg)
ELMO Demo
![Page 10: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/10.jpg)
ELMO DesignVirtual imagesfrom LCD
LCD MaskDepth Sensing
RealWorld
LCD Maskg
WorldOpticalCombiner
Use LCD mask to block real world
Combiner
Use LCD mask to block real worldDepth sensing for occluding virtual images
![Page 11: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/11.jpg)
ELMO Results
![Page 12: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/12.jpg)
ToolsTools
![Page 13: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/13.jpg)
BuildAR
http://www.hitlabnz.org/wiki/BuildARStand alone applicationStand alone applicationVisual interface for AR model viewing application Enables non-programmers to build AR scenesp g
![Page 14: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/14.jpg)
Mobile BuildARId l h lIdeal authoring tool
Develop on PC, deploy on handheld
AR Scene
AR PlayerPC
PC
EdgelibBuildAR XML
S bi /WM
Mobile Phone stbES
Symbian/WM
Python
![Page 15: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/15.jpg)
Desktop PC authoring tool
Desktop PC
Mobile Phone
![Page 16: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/16.jpg)
ApplicationsApplications
![Page 17: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/17.jpg)
Future DirectionsSLIDE 17
Interaction TechniquesInput techniques
3D vs. 2D inputP /b /Pen/buttons/gestures
Collaboration techniquesSimultaneous access to AR contentSimultaneous access to AR content
User studies…
![Page 18: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/18.jpg)
Flexible DisplaysFlexible Lens Surface
Bimanual interactionDigital paper analogyDigital paper analogy
Red Planet, 2000Red Planet, 2000
![Page 19: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/19.jpg)
Sony CSL © 2004
![Page 20: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/20.jpg)
Sony CSL © 2004
![Page 21: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/21.jpg)
Tangible User Interfaces (TUIs)GUMMI bendable display prototypeReproduced by permission of Sony CSL
![Page 22: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/22.jpg)
Sony CSL © 2004
![Page 23: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/23.jpg)
Sony CSL © 2004
![Page 24: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/24.jpg)
Lucid TouchM f R h & M b h El R h L bMicrosoft Research & Mitsubishi Electric Research LabsWigdor, D., Forlines, C., Baudisch, P., Barnwell, J., Shen, C.LucidTouch: A See-Through Mobile DeviceIn Proceedings of UIST 2007, Newport, Rhode Island, October 7-10, 2007, pp. 269–278.
![Page 25: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/25.jpg)
![Page 26: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/26.jpg)
Auditory Modalities
Auditoryauditory iconsauditory iconsearcons speech synthesis/recognition
Nomadic Radio (Sawhney) bi ti li d di- combines spatialized audio
- auditory cues- speech synthesis/recognition
![Page 27: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/27.jpg)
Gestural interfaces1. Micro-gestures
(unistroke, smartPad)
2 D i b d 2. Device-based gestures(tilt based examples)
3 Embodied interaction3. Embodied interaction(eye toy)
![Page 28: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/28.jpg)
Haptic ModalitiesHaptic interfaces
Simple uses in mobiles? (vibration instead of ringtone)Sony’s Touchengine
- physiological experiments show you can perceive two stimulus 5ms apart, and spaced as low as 0.2 microns
n層28 μm
n層
4 μm
n層
VV
![Page 29: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/29.jpg)
Haptic Input
AR Haptic WorkbenchCSIRO 2003 – Adcock et al CSIRO 2003 – Adcock et. al.
![Page 30: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/30.jpg)
AR Haptic Interface
Phantom, ARToolKit, Magellan
![Page 31: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/31.jpg)
Multimodal InputCombining speech and gesture builds on the strength of each
Speech – mode selection, group selectionp g pGesture – direct manipulation
Key problemCommand disambiguation
- “Move that chair” - which chair?
Use statistical methods for disambiguationUse statistical methods for disambiguationSpeech and gesture recognition provide multiple possibilities –need to look for most probableSenseShapes detect object of interest (Olwal 2003)
Olwal 2003
![Page 32: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/32.jpg)
ExperiencesExperiences
![Page 33: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/33.jpg)
ExperiencesCrossing Boundaries
Ubiquitous VR/AR
Massive ARAR + Social Networking
Usability
![Page 34: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/34.jpg)
Crossing Boundaries
Jun Rekimoto, Sony CSL
![Page 35: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/35.jpg)
Invisible Interfaces
Jun Rekimoto, Sony CSL
![Page 36: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/36.jpg)
Milgram’s Reality-Virtuality continuum
Mixed Reality
Real Augmented Augmented VirtualEnvironment Reality (AR) Virtuality (AV) Environment
Reality - Virtuality (RV) Continuum
![Page 37: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/37.jpg)
The MagicBook
Reality VirtualityAugmented Reality (AR)
Augmented Virtuality (AV)Reality (AR) Virtuality (AV)
![Page 38: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/38.jpg)
Invisible Interfaces
Jun Rekimoto, Sony CSL
![Page 39: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/39.jpg)
Example: Visualizing Sensor NetworksRauhala et. al. 2007 (Linkoping)Network of Humidity SensorsNetwork of Humidity Sensors
ZigBee wireless communication
Use Mobile AR to Visualize HumidityUse Mobile AR to Visualize Humidity
![Page 40: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/40.jpg)
![Page 41: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/41.jpg)
![Page 42: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/42.jpg)
Example: Sensor Input for AR InteractionUbiComp sensor
Light, temp, motion, soundRF connection
AR software plug-inSensor input interacting with AR applications
![Page 43: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/43.jpg)
uPartuPart USB BridgeUSB Bridge
ParticleParticle
http://particle.teco.eduhttp://particle.teco.edu idle: 16 Houridle: 16 Hour
![Page 44: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/44.jpg)
AR response to change in light levelsp g g
![Page 45: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/45.jpg)
Invisible Interfaces
Jun Rekimoto, Sony CSL
![Page 46: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/46.jpg)
UbiVR – CAMAR
CAMAR ControllerCAMAR Controller
CAMAR ViewerCAMAR Viewer
CAMAR CompanionCAMAR CompanionGIST - KoreaGIST - Korea
![Page 47: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/47.jpg)
ubiHome @ GISTMedia services Light service MR window
ubiTrackubiTrack
Where/When
ubiKeyTag-it
©ubiHome
What/When/HowWho/What/When/How
Couch SensorPDA Door SensorPDA Door Sensor
Wh /HWh /HWh /Wh t/Wh /H When/HowWhen/HowWho/What/When/How
![Page 48: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/48.jpg)
CAMAR - GIST
(CAMAR: Context-Aware Mobile Augmented Reality)y)
![Page 49: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/49.jpg)
UCAM: ArchitectureLight
<Service> IR receiver
wearServicewear-UCAM
Sensor
Content
<Service>
MRWindow<Service>
UserProfileManager< wearService >
UserConditionalContext #2
UserConditionalContext #3
ubiTrack
BioSensorIR receiver
wearSensor
Sensor Service(Integrator,Manager,
Interpreter,ServiceProvider) Media services Light service
ubiTV<Service>
UserConditionalContext #1
Context Interface
Couch Sensor
Tag-it
Door Sensor
ubiTrack
What/When/How Where/When
PDA
BAN/PAN TCP/IP
Network Interface When/HowWhen/How
PDA
Who/What/When/How
ubi-UCAM
O ti S t
BAN/PAN(BT)
TCP/IP(Discovery,Control,Event)
Operating Systemvr-UCAM
![Page 50: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/50.jpg)
UbiquitousUbiComp
Ubi AR
Ubi VR
Weiser Mobile AR
Desktop AR VR
Reality Virtual Reality
Terminal
Reality Virtual Reality
Milgram
From: Joe Newman
![Page 51: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/51.jpg)
Future DirectionsSLIDE 51
Massive MultiuserHandheld AR for the first time allows extremely high numbers of AR usersR iRequires
New types of applications/gamesNew infrastructure (server/client/peer to peer)New infrastructure (server/client/peer-to-peer)Content distribution…
![Page 52: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/52.jpg)
Massive Multi User
Ubiquitous
TerminalSingle User
Reality
Single User
VRVR
![Page 53: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/53.jpg)
Massive MultiUser2D Applications
MSN – 29 millionSkype – 10 millionSkype 10 millionFacebook – up to 70m
3D/VR ApplicationsSecondLife > 50KStereo projection - <500
A mented RealitAugmented RealityShared Space (1999) - 4Invisible Train (2004) - 8( )
![Page 54: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/54.jpg)
BASIC VIEW
![Page 55: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/55.jpg)
PERSONAL VIEW
![Page 56: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/56.jpg)
Augmented Reality 2.0 Infrastructure
![Page 57: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/57.jpg)
L i W b 2 0Leveraging Web 2.0Content retrieval using HTTPgXML encoded meta information
KML placemarks + extensionsQueries Queries
Based on location (from GPS, image recognition)Based on situation (barcode markers)
Q i l d li ki f d bQueries also deliver tracking feature databasesEverybody can set up an AR 2.0 serverSyndication: y
Community servers for end-user contentTagging
AR client subscribes to arbitrary number of feedsAR client subscribes to arbitrary number of feeds
![Page 58: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/58.jpg)
CContentContent creation and deliveryContent creation and delivery
Content creation pipelineDelivering previously unknown contentDelivering previously unknown content
Streaming ofData (objects multi media)Data (objects, multi-media)Applications
DistributionDistributionHow do users learn about all that content?H d th it?How do they access it?
![Page 59: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/59.jpg)
Twitter 360
Twitter 360 http://www.twitter-360.compAR to geo-locate Tweets around you
Better than Google maps?g p
![Page 60: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/60.jpg)
Scaling Up
AR on a City ScaleyUsing mobile phone as ubiquitous sensorMIT Senseable City LabMIT Senseable City Lab
http://senseable.mit.edu/
![Page 61: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/61.jpg)
![Page 62: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/62.jpg)
WikiCity Rome (Senseable City Lab MIT)
![Page 63: COSC 426 Lect. 8: AR Research Directions](https://reader034.vdocuments.net/reader034/viewer/2022042607/555ae621d8b42abe058b45cb/html5/thumbnails/63.jpg)
More Information• Mark Billinghurst• Mark Billinghurst
• Websiteshttp://www hitlabnz org/ – http://www.hitlabnz.org/
– http://artoolkit.sourceforge.net/http://www osgart org/– http://www.osgart.org/
– http://www.hitlabnz.org/wiki/buildAR/