cosc 426 lect 2. - ar technology
DESCRIPTION
Lecture 2 in the COSC 426 class on Augmented Reality. Taught by Mark BillinghurstTRANSCRIPT
![Page 1: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/1.jpg)
L 2 AR T h lLecture 2: AR Technology
Mark Billinghurst
July 2011
COSC 426: Augmented RealityCOSC 426: Augmented Reality
![Page 2: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/2.jpg)
Key Points from Lecture 1
![Page 3: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/3.jpg)
Augmented Reality DefinitionDefining Characteristics [Azuma 97]
Combines Real and Virtual Imagesg- Both can be seen at the same time
Interactive in real-time- Virtual content can be interacted with
Registered in 3Dg- Virtual objects appear fixed in space
![Page 4: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/4.jpg)
What is not Augmented Reality?
Location-based servicesB d d i (QR d )Barcode detection (QR-codes)Augmenting still imagesg g gSpecial effects in movies…… but they can be combined with AR!… but they can be combined with AR!
![Page 5: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/5.jpg)
Milgram’s Reality-Virtuality Continuum
Mixed Reality
Real Augmented Augmented VirtualEnvironment Reality (AR) Virtuality (AV) Environment
Reality - Virtuality (RV) Continuum
![Page 6: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/6.jpg)
Metaverse
![Page 7: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/7.jpg)
AR History Summary
1960’s – 80’s: Early Experimentation1980’s – 90’s: Basic Research1980 s 90 s: Basic Research
Tracking, displays
1995 – 2005: Tools/ApplicationsInteraction, usability, theoryy y
2005 - : Commercial ApplicationsG M di l I dGames, Medical, Industry
![Page 8: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/8.jpg)
Applications
MedicineManufacturingInformation overlayInformation overlayArchitectureMuseumMarketingMarketingGaming
![Page 9: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/9.jpg)
Interaction Design Process
![Page 10: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/10.jpg)
Interaction Design is All About You
Users should be involved throughoutinvolved throughout the Design ProcessConsider all the needs Co s de a t e eedsof the user
![Page 11: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/11.jpg)
B ildi C lli AR E iBuilding Compelling AR Experiences
experiencesUsability
applications Interaction
tools Authoringtools Authoring
components Tracking, Display
![Page 12: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/12.jpg)
AR Technology
![Page 13: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/13.jpg)
Building Compelling AR ExperiencesBuilding Compelling AR Experiences
experiences
applications
toolstools
components Display, Tracking
Sony CSL © 2004
![Page 14: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/14.jpg)
AR TechnologyKey Technologies
Displayp yTrackingInput
DisplayTracking
InputProcessing Processing
Input
![Page 15: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/15.jpg)
AR Displays
![Page 16: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/16.jpg)
AR Displays
AR
Primarily IndoorEnvironments
Primarily Outdoor(Daylight) Environments
Visual Displays
Virtual Images Projection CRT Display
Not Head-Mounted
Liquid Crystal
Head-MountedDisplay (HMD)
Cathode Ray Tube (CRT)or Virtual Retinal Display (VRD)
Head-MountedDisplay (HMD)
Projection DisplayNavigational Aids in Cars
Not Head Mounted(e.g. vehicle mounted)
e.g. windowreflections
seen off windows
e.g. Reach-In
using beamsplitter
e.g. Shared SpaceMagic Book
Displays LCDs
e.g. WLVAand IVRD
p y ( )Many Military Applications& Assistive Technologies
e.g. Head-UpDisplay (HUD)
gMilitary Airborne Applications
![Page 17: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/17.jpg)
Head Mounted Displays
![Page 18: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/18.jpg)
Head Mounted Displays (HMD)- Display and Optics mounted on Head- May or may not fully occlude real world- Provide full-color images- Considerations
• Cumbersome to wear• Brightness• Low power consumption• Low power consumption• Resolution limited• Cost is high?g
![Page 19: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/19.jpg)
Types of Head Mounted Displays
OccludedSee-thru
Multiplexed
![Page 20: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/20.jpg)
Immersive VR Architecture
head position/orientation
VirtualWorld
HeadTracker
head position/orientationNon see-thruImage source
& optics
Host P
Data BaseM d l
RenderingE i FrameProcessor Model Engine Buffer
to networkvirtualobject
DisplayDriver
![Page 21: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/21.jpg)
See-thru AR Architecture
head position/orientationHead
Tracker
head position/orientationsee-thru
combinerreal world
Host P
Data BaseM d l
RenderingE i FrameProcessor Model Engine Buffer
to network Virtual Image DisplayDriver
superimposedover real world object
Image sourceg
![Page 22: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/22.jpg)
Optical see-through head-mounted display
Virtual imagesfrom monitorsfrom monitors
RealWorld
OpticalCombiners
![Page 23: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/23.jpg)
Optical See-Through HMD
![Page 24: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/24.jpg)
Optical see-through HMDsVirtual Vision VCAP
Sony GlasstronSony Glasstron
![Page 25: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/25.jpg)
DigiLensC HOECompact HOE
Solid state opticsSwitchable Bragg GratingStacked SBGFast switchingUltra compact
www.digilens.com
![Page 26: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/26.jpg)
The Virtual Retinal Display
Image scanned onto retinaage sca e o to et aCommercialized through Microvision
Nomad System - www.mvis.comNomad System www.mvis.com
![Page 27: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/27.jpg)
Strengths of optical ARSimpler (cheaper)Di i f l ldDirect view of real world
Full resolution, no time delay (for real world)SafetyLower distortion Lower distortion
No eye displacementbut COASTAR video see-through avoids this
![Page 28: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/28.jpg)
Video AR Architecture
head position/orientation
Head-mounted camera aligned to
display opticsHead
Tracker
head position/orientation display optics
Video
Video image of real world
Host P
Graphicsd
DigitalMi Frame
Processor
Processor renderer Mixer Buffer
to networkDisplayDriver
Non see-thru
Virtual image inset into video of realImage source
& optics
video of real world
![Page 29: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/29.jpg)
Video see-through HMDVideocameras Video
Graphics
Monitors CombinerCombiner
![Page 30: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/30.jpg)
Video See-Through HMD
![Page 31: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/31.jpg)
Video see-through HMD
’ COAS A MR Laboratory’s COASTAR HMD(Co-Optical Axis See-Through Augmented Reality)Parallax free video see through HMDParallax-free video see-through HMD
![Page 32: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/32.jpg)
TriVisiowww.trivisio.comStereo video inputp
PAL resolution cameras
2 x SVGA displays2 x SVGA displays30 degree FOVUser adjustable convergenceUser adjustable convergence
$6,000 USD
![Page 33: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/33.jpg)
Vuzix Display
www.vuzix.comWrap 920$350 USD$350 USDTwin 640 x 480 LCD displays 31 degree diagonal field of view Weighs less than three ounces Weighs less than three ounces
![Page 34: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/34.jpg)
Strengths of Video ARTrue occlusion
Kiyokawa optical display that supports occlusiony p p y pp
Digitized image of real worldFl b l Flexibility in compositionMatchable time delaysMore registration, calibration strategies
Wide FOV is easier to supportWide FOV is easier to support
![Page 35: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/35.jpg)
Optical vs. Video AR SummaryBoth have proponentsVideo is more popular today?
Likely because lack of available optical productsy p p
Depends on application?M f i i l i hManufacturing: optical is cheaperMedical: video for calibration strategies
![Page 36: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/36.jpg)
Eye multiplexed AR Architecture
head position/orientationHead
Tracker
head position/orientation
real world
Host P
Data BaseM d l
RenderingE i FrameProcessor Model Engine Buffer
to networkDisplayDriver
Virtual Image inset intoreal world scene
OpaqueImage source
![Page 37: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/37.jpg)
Virtual Image ‘inset’ into real
![Page 38: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/38.jpg)
Virtual Vision Personal Eyewear
![Page 39: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/39.jpg)
Virtual image inset into real world
![Page 40: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/40.jpg)
Spatial/Projected AR
![Page 41: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/41.jpg)
Spatial Augmented Reality
Project onto irregular surfacesGeometric RegistrationProjector blending, High dynamic range
Book: Bimber, Rasker “Spatial Augmented Reality”p g y
![Page 42: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/42.jpg)
Projector-based ARUser (possiblyhead-tracked)
Projector
Examples:Raskar MIT Media LabReal objects Raskar, MIT Media LabInami, Tachi Lab, U. Tokyowith retroreflective
covering
![Page 43: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/43.jpg)
Example of projector-based AR
Ramesh Raskar, UNC, MERL, ,
![Page 44: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/44.jpg)
Example of projector-based AR
Ramesh Raskar, UNC Chapel Hill
![Page 45: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/45.jpg)
The I/O Bulb
P j + CProjector + CameraJohn Underkoffler, Hiroshi Ishii MIT Media Lab
![Page 46: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/46.jpg)
Head Mounted Projector
Head Mounted ProjectorJannick Rolland (UCF)J ( )
Retro-reflective MaterialPotentially portablePotentially portable
![Page 47: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/47.jpg)
Head Mounted Projector
NVIS P 50 HMPDNVIS P-50 HMPD1280x1024/eyeStere sc icStereoscopic50 degree FOV
iwww.nvis.com
![Page 48: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/48.jpg)
HMD vs. HMPD
Head Mounted Display Head Mounted Projected Display
![Page 49: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/49.jpg)
Pico Projectors
Microvision - www.mvis.com3M S Phili t3M, Samsung, Philips, etc
![Page 50: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/50.jpg)
MIT Sixth Sense
Body worn camera and projectorhttp://www.pranavmistry.com/projects/sixthsense/p p y p j
![Page 51: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/51.jpg)
Other AR Displays
![Page 52: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/52.jpg)
Video Monitor AR
Videocameras Monitor
Stereoglassescameras g
Video
Graphics Combiner
![Page 53: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/53.jpg)
Virtual Showcase
Mirrors on a projection tableH d k d Head tracked stereoUp to 4 usersM hi d l bjMerges graphic and real objectsExhibit/museum applications
Fraunhofer Institute (2001)Bimber, Frohlich
![Page 54: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/54.jpg)
Augmented Paleontology
Bimber et. al. IEEE Computer Sept. 2002
![Page 55: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/55.jpg)
Alternate Displays
LCD Panel Laptop PDA
![Page 56: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/56.jpg)
Handheld Displays
Mobile PhonesCameraDisplay InputInput
![Page 57: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/57.jpg)
Other Types of AR DisplayAudio
spatial soundpambient audio
T til Tactile physical sensation
Hapticvirtual touchvirtual touch
![Page 58: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/58.jpg)
Haptic Input
AR Haptic WorkbenchCSIRO 2003 Adcock et al CSIRO 2003 – Adcock et. al.
![Page 59: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/59.jpg)
Phantom
Sensable Technologies (www.sensable.com)6 DOF Force Feedback Device
![Page 60: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/60.jpg)
AR Haptic Interface
Phantom, ARToolKit, Magellang
![Page 61: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/61.jpg)
AR Tracking and Registration
![Page 62: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/62.jpg)
TrackingLocating the users viewpointg pPosition (x,y,z)Orientation (r p y)Orientation (r,p,y)
RegistrationPositioning virtual object wrt real world
![Page 63: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/63.jpg)
Tracking Requirements
Head Stabilized Body Stabilized World Stabilized
Augmented Reality Information Display World StabilizedBody StabilizedHead Stabilized
Increasing Tracking Requirements
![Page 64: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/64.jpg)
Tracking Technologies
• Mechanical• Electromagneticg• Optical• Acoustic
I ti l d d d k i• Inertial and dead reckoning• GPS• Hybrid• Hybrid
![Page 65: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/65.jpg)
AR Tracking Taxonomy
ARTRACKING
IndoorEnvironment
OutdoorEnvironment
TRACKING
Limited Range Extended Range Low Accuracy &Not Robust
High Accuracy& Robust
Low Accuracyat 15-60 Hz
High Accuracy& High Speed
H b id
Many Fiducialsin space/time
b t
Not HybridizedGPS or
C
Hybrid TrackingGPS and
C d
e g AR Toolkit e g IVRD
HybridTracking
e g HiBall
butno GPS
e g WLVA
Camera orCompass
e g BARS
Camera andCompass
e.g. AR Toolkit e.g. IVRD e.g. HiBall e.g. WLVA e.g. BARS
![Page 66: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/66.jpg)
Tracking Types
Magnetic T k
Inertial T k
Ultrasonic T k
Optical T k
Mechanical TrackerTracker Tracker Tracker Tracker
Marker Based Markerless Specialized
Tracker
Marker-Based Tracking
Markerless Tracking
Specialized Tracking
Edge-Based Tracking
Template-Based Tracking
Interest Point Trackingg g g
![Page 67: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/67.jpg)
Tracking Systems Mechanical Tracker
Magnetic TrackerMagnetic TrackerUltrasonic TrackerInertial TrackerVision (Optical Tracking)Vision (Optical Tracking)
Specialized (Infrared, Retro-Reflective)M l (DVC W b )Monocular (DVCam, Webcam)
![Page 68: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/68.jpg)
Mechanical TrackerId h l h Idea: mechanical arms with joint sensors
++: high accuracy haptic feedback Microscribe
++: high accuracy, haptic feedback -- : cumbersome, expensive
![Page 69: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/69.jpg)
Magnetic TrackerIdea: difference between a magnetic transmitter Idea: difference between a magnetic transmitter and a receiver
6DOF b
Flock of Birds (Ascension)
++: 6DOF, robust -- : wired, sensible to metal, noisy, expensivey p
![Page 70: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/70.jpg)
I l T kInertial TrackerIdea: measuring linear and angular orientation rates (accelerometer/gyroscope)
++: no transmitter cheap small high frequency wireless
IS300 (Intersense) Wii Remote
++: no transmitter, cheap, small, high frequency, wireless -- : drift, hysteris only 3DOF
![Page 71: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/71.jpg)
Ultrasonics TrackerIdea: Time of Flight or Phase Coherence Sound WavesIdea: Time of Flight or Phase-Coherence Sound Waves
Ultrasonic Logitech
++: Small, Cheap
Logitech IS600
-- : 3DOF, Line of Sight, Low resolution, Affected Environment Conditon (pressure, temperature)
![Page 72: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/72.jpg)
Global Positioning System (GPS)
Created by US in 1978Currently 29 satellites
Satellites send position + time GPS Receiver positioningp g
4 satellites need to be visibleDifferential time of arrivalTriangulation
Accuracyy5-30m+, blocked by weather, buildings etc
![Page 73: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/73.jpg)
Problems with GPSTakes time to get satellite fixTakes time to get satellite fix
Satellites moving around
Earths atmosphere affects signalp gAssumes consistent speed (the speed of light). Delay depends where you are on EarthWeather effects
Signal reflectionMulti-path reflection off buildings
Signal blockingT b ildi t iTrees, buildings, mountains
Satellites send out bad dataMisreport their own positionMisreport their own position
![Page 74: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/74.jpg)
Accurate to < 5cm close to base station (22m/100 km)Accurate to 5cm close to base station (22m/100 km)Expensive - $20-40,000 USD
![Page 75: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/75.jpg)
Optical Tracking
![Page 76: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/76.jpg)
Optical TrackerIdea: Image Processing and Computer Visiong g pSpecialized
f f SInfrared, Retro-Reflective, Stereoscopic
ART Hi-Ball
Monocular Based Vision Trackingg
![Page 77: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/77.jpg)
Outside-In vs. Inside-Out Tracking
![Page 78: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/78.jpg)
Optical Tracking Technologies
Scalable active trackersScalable active trackersInterSense IS-900, 3rd Tech HiBall
3 d T h IPassive optical computer visionLine of sight may require landmarks
3rd Tech, Inc.
Line of sight, may require landmarks Can be brittle.C i i i i ll i iComputer vision is computationally-intensive
![Page 79: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/79.jpg)
HiBall Tracking System (3rd Tech)O Inside-Out Tracker
$50K USD
Scalable over large areaF t d t (2000H ) Fast update (2000Hz) Latency Less than 1 ms.
AccuratePosition 0 4mm RMSPosition 0.4mm RMSOrientation 0.02° RMS
![Page 80: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/80.jpg)
![Page 81: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/81.jpg)
Starting simple: Marker trackingHas been done for more than 10 yearsS l l i iSeveral open source solutions existFairly simple to implementy p p
Standard computer vision methods
A l k id 4 iA rectangular marker provides 4 corner pointsEnough for pose estimation!
![Page 82: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/82.jpg)
Marker Based Tracking: ARToolKit
http://artoolkit.sourceforge.net/
![Page 83: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/83.jpg)
C d SCoordinate Systems
![Page 84: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/84.jpg)
M k T k OMarker Tracking – Overview
![Page 85: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/85.jpg)
Marker Tracking – Fiducial DetectionThreshold the whole image to black and whiteThreshold the whole image to black and whiteSearch scanline by scanline for edges (white to black)Follow edge until either
Back to starting pixelImage border
Check for sizeReject fiducials early that are too small (or too large)
![Page 86: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/86.jpg)
Marker Tracking – Rectangle FittingS i h bi i “ ” h Start with an arbitrary point “x” on the contourThe point with maximum distance must be a corner c0C d l h h h Create a diagonal through the centerFind points c1 & c2 with maximum distance left and right of diag.New diagonal from c1 to c2Find point c3 right of diagonal with maximum distance
![Page 87: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/87.jpg)
Marker Tracking – Pattern checkingCalculate homography using the 4 corner pointsCalculate homography using the 4 corner points
“Direct Linear Transform” algorithmMaps normalized coordinates to marker coordinatesp(simple perspective projection, no camera model)
Extract pattern by samplingCheck pattern
Id (implicit encoding)T l ( li d l i )Template (normalized cross correlation)
![Page 88: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/88.jpg)
Marker Tracking – Corner refinementRefine corner coordinatesRefine corner coordinates
Critical for high quality trackingRemember: 4 points is the bare minimum!So these 4 points should better be accurate…
Detect sub-pixel coordinatesE.g. Harris corner detectorg- Specialized methods can be faster and more accurateStrongly reduces jitter!g y j
Undistort corner coordinatesR di l di t ti f lRemove radial distortion from lens
![Page 89: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/89.jpg)
M k k P Marker tracking – Pose estimationC Calculates marker position and rotation relative to the cameraInitial estimation directly from homography
Very fast but coarseVery fast, but coarseJitters a lot…
R fi i G N i iRefinement via Gauss-Newton iteration6 parameters (3 for position, 3 for rotation) to refineAt each iteration we optimize on the reprojection error
![Page 90: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/90.jpg)
Coordinates for Marker Tracking
![Page 91: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/91.jpg)
Coordinates for Marker Tracking
Marker Camera•GoalR t ti & T l ti
•Camera Ideal Screen•Perspective model•Obtained from Camera Calibration
•Ideal Screen Observed Screen•Nonlinear function (barrel shape)•Marker Observed Screen•Correspondence of 4 vertices
•Rotation & Translation•Obtained from Camera Calibration•Obtained from Camera Calibration•Real time image processing
![Page 92: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/92.jpg)
F M k T CFrom Marker To CameraRotation & Translation
TCM : 4x4 transformation matrixfrom marker coord. to camera coord.
![Page 93: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/93.jpg)
Tracking challenges in ARToolKit
Unfocused camera, motion blur
Dark/unevenly lit scene, vignetting
Jittering (Photoshop illustration)
Occlusion(image by M. Fiala)
False positives and inter-marker confusion(image by M. Fiala)
Image noise(e.g. poor lens, block coding /
compression, neon tube)
![Page 94: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/94.jpg)
Tracking, Tracking, Tracking
![Page 95: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/95.jpg)
Other Marker Tracking LibrariesTarTaghttp://www.artag.net/
ARToolKitPlus [Discontinued]http://studierstube.icg.tu-graz ac at/handheld ar/artoolkitplus phpgraz.ac.at/handheld_ar/artoolkitplus.php
stbTrackerhtt // t di t b i thttp://studierstube.icg.tu-graz.ac.at/handheld_ar/stbtracker.php
MXRToolKitMXRToolKithttp://sourceforge.net/projects/mxrtoolkit/
![Page 96: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/96.jpg)
Markerless Tracking
![Page 97: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/97.jpg)
![Page 98: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/98.jpg)
Markerless Tracking
M i T k I i l Ul i O i l
No more Markers! Markerless Tracking
Magnetic Tracker Inertial Tracker
Ultrasonic Tracker
Optical Tracker
Marker-Based Tracking
Markerless Tracking
Specialized Tracking
Edge-Based Template-Based Interest Point Tracking Tracking Tracking
![Page 99: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/99.jpg)
Natural feature trackingTracking from features of the surrounding environmentenvironment
Corners, edges, blobs, ...
G ll diffi l h k kiGenerally more difficult than marker trackingMarkers are designed for their purposeg p pThe natural environment is not…
Less well established methodsLess well-established methodsUsually much slower than marker tracking
![Page 100: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/100.jpg)
Natural Feature TrackingFeatures Points
Use Natural Cues of Real ElementsEdges
Contours
Features Points
Surface Texture Interest Points
Model or Model-Free++: no visual pollution
SurfacesSurfaces
![Page 101: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/101.jpg)
Texture Tracking
![Page 102: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/102.jpg)
T k b dTracking by detectionThis is what most trackers do Camera ImageThis is what most trackers do…Targets are detected every frame
Keypoint detection
g
Popular becausetracking and detection
yp
Descriptor creationd t hiare solved simultaneously and matching
Outlier RemovalOutlier Removal
Pose estimationand refinementand refinement
Pose
![Page 103: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/103.jpg)
Natural feature tracking – What is a keypoint?
It depends on the detector you use!For high performance use the FAST corner For high performance use the FAST corner detector
A l FAST t ll i l f iApply FAST to all pixels of your imageObtain a set of keypoints for your image
R d h f - Reduce the amount of corners using non-maximum suppression
Describe the keypoints
E. Rosten and T. Drummond (May 2006). "Machine learning for high‐speed corner detection".
![Page 104: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/104.jpg)
Corner keypoint
![Page 105: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/105.jpg)
Natural feature tracking – DescriptorsAgain depends on your choice of a descriptor!Again depends on your choice of a descriptor!Can use SIFT
E i h d i k iEstimate the dominant keypointorientation using gradientsC m ensate f r detectedCompensate for detectedorientationDescribe the keypoints in termsDescribe the keypoints in termsof the gradients surrounding it
Wagner D., Reitmayr G., Mulloni A., Drummond T., Schmalstieg D., Real‐Time Detection and Tracking for Augmented Reality on Mobile Phones.IEEE Transactions on Visualization and Computer Graphics, May/June, 2010
![Page 106: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/106.jpg)
NFT D b NFT – Database creationOffline steppSearching for corners in a static imageFor robustness look at corners on multiple scalesFor robustness look at corners on multiple scales
Some corners are more descriptive at larger or smaller scalesW d ’t k h f ill b f iWe don’t know how far users will be from our image
Build a database file with all descriptors and their i i h i i l iposition on the original image
![Page 107: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/107.jpg)
NFT R l kNFT – Real-time trackingSearch for keypoints Camera ImageSearch for keypointsin the video imageC t th d i t Keypoint detection
Camera Image
Create the descriptorsMatch the descriptors from the
Keypoint detection
Descriptor creation
live video against thosein the database
pand matching
O tli R lBrute force is not an optionNeed the speed-up of special
Outlier Removal
Pose estimationand refinementdata structures
- E.g., we use multiple spill trees
and refinement
PosePose
![Page 108: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/108.jpg)
NFT O l lNFT – Outlier removalCascade of removal techniquesCascade of removal techniquesStart with cheapest, finish with most expensive…
First simple geometric testsFirst simple geometric tests- E.g., line tests
• Select 2 points to form a line• Check all other points being on correct side of line
Then, homography-based tests
![Page 109: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/109.jpg)
NFT P fNFT – Pose refinementPose from homography makes good starting pointBased on Gauss-Newton iteration
Try to minimize the re-projection error of the keypoints
Part of tracking pipeline that mostly benefits Part of tracking pipeline that mostly benefits from floating point usageCan still be implemented effectively in fixed pointCan still be implemented effectively in fixed pointTypically 2-4 iterations are enough…
![Page 110: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/110.jpg)
NFT R l kNFT – Real-time trackingSearch for keypoints
Camera Image
in the video imageCreate the descriptors Keypoint detection
Camera Image
pMatch the descriptors from thelive video against those
Keypoint detection
Descriptor creationlive video against thosein the databaseRemove the keypoints that
and matching
Outlier RemovalRemove the keypoints thatare outliersU h i i k i
Outlier Removal
Pose estimationand refinementUse the remaining keypoints
to calculate the pose f h
and refinement
Poseof the camera
![Page 111: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/111.jpg)
NFT R lNFT – Results
Wagner D., Reitmayr G., Mulloni A., Drummond T., Schmalstieg D., Real‐Time Detection and Tracking for Augmented Reality on Mobile Phones.IEEE Transactions on Visualization and Computer Graphics, May/June, 2010
![Page 112: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/112.jpg)
Edge Based TrackingRAPiD [Drummond et al. 02]
Initialization, Control Points, Pose Prediction (Global Method)
![Page 113: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/113.jpg)
Line Based TrackingVisual Servoing [Comport et al. 2004]
![Page 114: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/114.jpg)
Model Based TrackingOpenTL - www.opentl.org
General purpose library for model based visual tracking
![Page 115: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/115.jpg)
OpenTL Features
![Page 116: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/116.jpg)
![Page 117: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/117.jpg)
Visual Modalities Used For Tracking
![Page 118: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/118.jpg)
The Tracking Pipeline
![Page 119: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/119.jpg)
Marker vs. natural feature trackingMarker tracking
Usually requires no database to be storedMarkers can be an eye-catcherTracking is less demandingg gThe environment must be instrumented with markersMarkers usually work only when fully in viewy y y
Natural feature trackingA database of keypoints must be stored/downloadedA database of keypoints must be stored/downloadedNatural feature targets might catch the attention lessN t l f t t t t ti ll hNatural feature targets are potentially everywhereNatural feature targets work also if partially in view
![Page 120: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/120.jpg)
Hybrid Tracking
![Page 121: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/121.jpg)
Example: Outdoor Hybrid Tracking
Combinescomputer visioncomputer vision
- natural feature trackinginertial gyroscope sensorsinertial gyroscope sensors
Both correct for each otherInertial gyro - provides frame to frameInertial gyro provides frame to frame prediction of camera orientationComputer vision - correct for gyro driftComputer vision correct for gyro drift
![Page 122: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/122.jpg)
Outdoor AR Tracking System
You, Neumann, Azuma outdoor AR system (1999)
![Page 123: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/123.jpg)
Robust Outdoor Tracking
H b id T kiHybrid TrackingComputer Vision, GPS, inertial
Going OutReitmayer & Drummond (Univ. Cambridge)Reitmayer & Drummond (Univ. Cambridge)
![Page 124: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/124.jpg)
Handheld Display
![Page 125: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/125.jpg)
Registration
![Page 126: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/126.jpg)
The Registration ProblemVirtual and Real must stay properly alignedIf tIf not:
Breaks the illusion that the two coexistPrevents acceptance of many serious applications
![Page 127: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/127.jpg)
Sources of registration errorsS Static errors
Optical distortionsMechanical misalignmentsTracker errorsIncorrect viewing parameters
Dynamic errorsDynamic errorsSystem delays (largest source of error)
1 d l 1/3 i t ti - 1 ms delay = 1/3 mm registration error
![Page 128: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/128.jpg)
Reducing static errorsDistortion compensationManual adjustmentsManual adjustmentsView-based or direct measurements
[Azuma94] [Caudell92] [Janin93] etc.
Camera calibration (video)Camera calibration (video)[ARGOS94] [Bajura93] [Tuceryan95] etc.
![Page 129: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/129.jpg)
View Based Calibration (Azuma 94)
![Page 130: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/130.jpg)
Dynamic errorsApplication Loop
Tracking Calculate ViewpointSimulation
Render Scene
Draw to Display
x,y,zr,p,y
Simulation
20 Hz = 50ms 500 Hz = 2ms 30 Hz = 33ms 60 Hz = 17ms
Total Delay = 50 + 2 + 33 + 17 = 102 ms1 ms delay = 1/3 mm = 33mm error1 ms delay 1/3 mm 33mm error
![Page 131: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/131.jpg)
Reducing dynamic errors (1)
Reduce system lag[Olano95] [Wloka95a] [Regan SIGGRAPH99]
Reduce apparent lagReduce apparent lagImage deflection [Burbidge89] [Regan94] [So92] [Kiji ISMR 2001][Kijima ISMR 2001]Image warping [Mark 3DI 97]
![Page 132: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/132.jpg)
Reducing System Lag
Application Loop
Tracking Calculate Viewpoint
Render Scene
Draw to Display
x,y,zr,p,y
ViewpointSimulation
Scene Display
Faster Tracker Faster CPU Faster GPU Faster DisplayFaster Tracker Faster CPU Faster GPU Faster Display
![Page 133: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/133.jpg)
Reducing Apparent Lag
x,y,zVirtual Display
Ph i l
Virtual Display
Physical
Tracking
yr,p,yPhysical
Display(640x480)
Physical Display
(640x480)TrackingUpdate1280 x 960
Last known position
1280 x 960
Latest positionLast known position Latest position
Application Loop
Tracking Calculate Viewpoint
Render Scene
Draw to Display
x,y,zr,p,y
pSimulation
p y
![Page 134: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/134.jpg)
Reducing dynamic errors (2)Match input streams (video)
Delay video of real world to match system lagDelay video of real world to match system lag
Predictive Tracking[Azuma94] [Emura94] Inertial sensors helpful
Azuma / Bishop 1994u a / s op 99
![Page 135: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/135.jpg)
Predictive Tracking
Pos Nowition
TimePast Future
Time
Can predict up to 80 ms in future (Holloway)
![Page 136: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/136.jpg)
Predictive Tracking (Azuma 94)
![Page 137: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/137.jpg)
Wrap-upTracking and Registration are key problemsRegistration errorRegistration error
Measures against static errorM i d i Measures against dynamic error
AR typically requires multiple tracking technologiesyp y q p g gResearch Areas: Hybrid Markerless Techniques, Deformable Surface Mobile Outdoors Deformable Surface, Mobile, Outdoors
![Page 138: COSC 426 Lect 2. - AR Technology](https://reader035.vdocuments.net/reader035/viewer/2022062513/5558e63cd8b42ab8168b4dba/html5/thumbnails/138.jpg)
More Information
M k Billi h t• Mark Billinghurst– [email protected]
• Websiteshi l b– www.hitlabnz.org