+Google Goggles

By Sayantan Ghosh

+What is Google Goggles ? Google Goggles is a visual search application that lets a user submit a search query by taking a picture. It uses computer vision techniques to recognize objects in a image and then return information on it or do some further processing on it.

+What all can it currently do?

User case demo

I know what this isbring it into my phone and tell me more

I dont know what this istell me

Barcodes

Landmarks

Text

Books

Art Work

Wine

Logos

Contact information

SUDOKU

+How does it do it?

n

n

nIt uses multiple techniques to recognize the image.

First it tries to identify the object by some object recognitionalgorithms and compares it against Google's already quitelarge database of images on almost everything.

To assist the search it tries to find text in the image usingoptical character recognition to get a better idea of what theobject might be.

It also uses GPS to figure out where you are to filter resultsreceived for landmarks to those that are relevant to yourlocation.

+Object Recognition and SearchThere are many algorithms out there for Object Recognition

Edge Detection

Scale Invariant Feature Transform For Text

OCR (Optical Character Recognition)For Images

Feature Detection (SIFT, SURF, PCA-SIFT) Search By color Visual Similarity Search

Life of a goggles queryHighly schematic Highly parallel

+Issuesn

n

n

nGoogle goggles was initially so sensitive that it could in manycases find a person through an image and return a link to ablog leading up to him.Google realized the privacy issues inthis and now controls the results so as to not recognizepeople.

Whether it still keeps that information for itself or not ?Whoknows ?

Google goggles is still in its experimental phase and not allresults are accurate.

Google Goggles doesnt work well with pets,furniture andclothing.

Barriers

The faster Goggles responds, the more usage we haveIts good to respond fast even (especially) when we can'tgive the right answer

Delays

Client delays

Camera focusing: 1.5 s Picture acquisition: 2 s Image (re) encoding.

Network delays Goals at odds with Universal visual search as Barcode and place recognition very different

Drawbacks

Supports only android and IOS mobiles

Pictures captured in the frame must be clear

Search results may not be precise

What is coming next in Google goggles?

Expanding the corpus of recognized objects

Using search quality techniques to improve relevancy

Increasing the Goggles footprint

Enabling 3rd party developers to create interesting applications based on computer vision

+Future

Google is working to provide the ability to search for leavesand plants which will also give the ability to tell weather aplant/leaf is poisonous.

More augmented reality like providing a 3D rendering of theobject in the image youre shooting.There are other Appsthat do provide this functionality.

More accuracy and better search results and moreapplications of this technology.

Use a front facing camera to detect if the user is happy

DemoQuesto un messaggio di prova

References

Google Goggles Main Page

Google Goggles by Symbioun Technologies Inc

Presentation on Google Goggles by Sahil Handa

Search by Sight: Google Goggles by David Petrou

Thank you.