3d vision the university of oxford andrew glennerster physiology andrew fitzgibbon engineering
TRANSCRIPT
3D Vision
The University of Oxford
Andrew Glennerster Physiology
Andrew Fitzgibbon Engineering
Vision
A fundamental biological system
An attractive engineering problem
How are these related?
A 3D Visual Task
TOC
• Computer vision state of the art• Biology state of the art
– an implausible model?
• Open questions
Artificial 3D Vision
Measurements
3D Structure
Application: 3D Video Editing
Application: 3D Video Editing
“Computer Vision” Summary
• Single representation suits all– pointing– navigation
• Built on– algebra– geometry– statistics
• Really works on real data
Biological systems can perform 3D tasksusing visual information
• Pointing to remembered objects• Ants homing• Spiders turning to view prey
Computer vision is an unlikely model for animal vision
• A “likely” model needs topredict experimental databe neurally plausible
• We don’t know how to multiply matrices in neurons
• CV is open loop– animate systems characterized by feedback
• 3D not necessary for many tasks
What is the incumbent model?
Posterior parietalcortex
Eye-centredcoordinates
Head-centredcoordinates
Body-centredcoordinates
World-centredcoordinates
Eyeposition
Audition
Neck proprioception
Vestibularinformation
Visualinformation
Adapted from figure 3 from Andersen et al (1997)
An Alternative
• In simpler animals, the requirement for a map is questioned.
• Many tasks can be performed by local, low-level, reactive rules
(Franz and Mallot, 2000)
“Biomimetic” Navigation
(Franz and Mallot, 2000)
“Biomimetic” Navigation
“Biomimetic” Navigation
• Neurally plausible
• Real robot implementations
• Cannot model 3D pointing yet
• No complete model of ant, for example
“computer vision hitting a computational wall”
• Need infinite-time, infinite-space operation
• Computers are getting faster...
• When will O(2t) beat O(n3)?– n is number of features– t is years from now
Dynamic Scenes
Open Problems
• No complete model of ant navigation– Photons to pointing
• No plausible model of 3D vision in higher animals– Coordinate system: Is there one?– One representation or many?
• Computer vision’s computational wall– One solution will be better engineering– Another may be “parallel slow neurons”
• Dynamic scenes
Thanks To
Andrew GlennersterAndrew ParkerAndrew Blake
Andrew ZissermanAndrew DavisonAndrew Stoddart
...and a few non-andrews
Questions
• How do biological systems perform such tasks?– We want an explanation “from
photons to pointing”
• How might artificial systems perform such tasks?– Artificial systems are “easy” to analyse
Status
• We have models for some of the biology
• We have artificial systems which work in useful but limited domains