eics '11: estimating scale using depth from focus for mobile augmented reality
DESCRIPTION
Whilst there has been a considerable progress in augmented reality (AR) over recent years, it has principally been related to either marker based or apriori mapped systems, which limits its opportunity for wide scale deployment. Recent advances in marker-less systems that have no apriori information, using techniques borrowed from robotic vision, are now finding their way into mobile augmented reality and are producing exciting results. However, unlike marker based and apriori tracking systems these techniques are independent of scale which is a vital component in ensuring that augmented objects are contextually sensitive to the environment they are projected upon. In this paper we address the problem of scale by adapting a Depth From Focus (DFF) technique, which has previously been limited to high-end cameras to a commercial mobile phone. The results clearly show that the technique is viable and adds considerably to the enhancement of mobile augmented reality. As the solution only requires an auto-focusing camera, it is also applicable to other AR platforms. Full Paper: http://dl.acm.org/citation.cfm?id=1996461.1996531TRANSCRIPT
Estimating Scale using Depth From Focus for Mobile Augmented Reality
Klen Čopič PuciharDr. Paul Coulton
Overview
• Introduction to vision based Augmented Reality;
• Problem identification;• Solving scale ambiguity;• Results of empirical study;• Implications;• Conclusions with future work
suggestions;
Introduction to vision based Augmented Reality
Where is the problem?
Marker based AR (AR Toolkit) Markerless AR (PTAM on iPhone)
Informed Un-informed
Problem of monocular vision
“It is impossible to determine the scale of the scene from a sequence of images alone.”
Hartley, R.I. and Zisserman, A.
Let’s have another look.
Marker based AR (AR Toolkit) Markerless AR (PTAM on iPhone)
Informed Un-informed
“It is impossible to determine the scale of the scene from a sequence of images alone.”
Scale known
Scale might be known Scale unknown
Solving scale ambiguity
• Till now, the only alternative for scale initialization was based on asking the user to subjectively introduce additional information to the system.
What solution do we propose?
Introducing scale using Depth From Focus (DFF)
Where is the novelty?
Image formation in a convex lens
Novelty
• The Depth From Focus method has been previously limited to high end cameras and was never before implemented and tested on a mobile phone camera.
• Innovative use of Depth From Focus in context of mobile augmented reality.
Quick view at results• 4 Nokia N900 Phones.• Data was captured in the range of 7-50 cm with
20 measurements at each given distance.• Data was aggregated based on each phone and
auto-focusing method.• Three different modes of operation were tested.
What are the implications?Experiment photo
• The identified useful range is between 7 and 30 cm.
• Maximal relative depth error was between 6.6 and 12.2 %.
Implications
• Measuring applications;• Simplified initialization of Public ‘You are Here’
maps for navigation system on mobile devices;• Introducing scale to markerless mobile augmented reality;
Screenshot of Meter application.
Why is scale so important in AR?
Why is scale so important?
Picture of a miniature garden scene.
Conclusion with future work suggestions
• Future work:– Solution needs to be tested on a fully functional marker-
less AR system.– Detailed feasibility study on other AR platforms should
be performed. – Accuracy of the system needs testing on horizontal
surfaces, as well as in cases where camera image plan and the scene plane are not parallel.
• Uninformed markerless AR is the most appropriate vision based AR system for wide scale deployment.
• The limitation of unknown scale has been solved by the proposed DFF scale initialization.
• The solution is applicable to other platforms that have auto-focusing camera with access to the camera driver.
Time for questions.
Thank you very much for your attention.