mobile feature-cloud panorama construction for image recognition applications

MACHINE VISION GROUP

MOBILE FEATURE-CLOUD PANORAMA CONSTRUCTION FOR IMAGE

RECOGNITION APPLICATIONS

Miguel Bordallo, Jari Hannuksela, Olli silvénMachine Vision Group

University of Oulu

MACHINE VISION GROUP

Contents

• Introduction• Image recognition applications

– Comparison of image-based context retrieval methods

• Context retrieval from video analysis• System design

– Application flow– Automatic start– Image registration– Moving-objects detection– Quality assesment

• Performance analysis• Conclusions

MACHINE VISION GROUP

Image-based context retrieval applications

•Point Your Camera to an object (landmark, poster)•Take a Picture•Get context information and display it

MACHINE VISION GROUP

Mobile context retrieval applications

Google googles

Snaptell

Kooaba

Nokia Point & Find

MACHINE VISION GROUP

Image recognition approaches

• Videos contain lots of information– Most of it redundant

• Image registration is easy – Smaller motions between frames– some frames can be discarded without

losing information

• Videos can capture wide angle scenes. – 3D world is better represented

• Transmission of compressed still image• Needs lots of storage in server• Image size implies large amount of data transmitted• Compression artifacts diminish quality

• Features extracted from still images• Amount of features needed not know beforehand• No feedback. Re-takes needed often• Two dimensional representation

• Feature-cloud extracted from video frames

MACHINE VISION GROUP

Still image vs. Video based

MACHINE VISION GROUP

Constructing a feature-cloud

Frame #1 Frame #16 Frame #31 Frame #46 Frame #61

MACHINE VISION GROUP

System design

MACHINE VISION GROUP

Application flow (client side)

MACHINE VISION GROUP

Automatic start of the application

•Recognizing characteristic motion patterns • Holding phone like a camera• Panning back and forth

•Reduces perceived latencies

MACHINE VISION GROUP

Interactive capture

VGA video analysis

Motion estimation system calculates shift, rotation and scale in real time

When frame is suitable for recognition (high quality), the user receives feedback and instructions

MACHINE VISION GROUP

Feature extraction & Image registration

• Feature extraction based on CHoG features• Compressed Histogram of Gradients

• Block Matching

• Best Linear Unbiased Estimator

• Compute registration parameters in real time to send to the server:

•Shift, rotation and change of scale

MACHINE VISION GROUP

Moving-objects detection

Object-detection ON Object-detection OFF

MACHINE VISION GROUP

Moving-objects detection

The features corresponding to a moving object are not sent to the server

Not-valid features are transmitted tothe server

Object-detection ON Object-detection OFF

MACHINE VISION GROUP

Quality assesment

Server receives only the features corresponding to high quality frames

MACHINE VISION GROUP

Performance comparison

MACHINE VISION GROUP

Summary

• Improve results in 3 dimensional environments• Interactivity• Detection of moving objects• Image quality assesment• Bigger field of view

• Reduce the communications need between clients and server

•Bandwidth reduction

• Reduce the workload of the servers