propagation ofpropagation of geotags based onbased on object … · 2019-01-22 · 3 over the last...

1

Propagation of geotags based onPropagation of geotags based onobject duplicate detection

Peter Vajda, Ivan Ivanov, Jong-Seok Lee,Lutz Goldmann, Touradj Ebrahimi

Multimedia Signal Processing GroupEcole Polytechnique Federale de Lausanne, Switzerland

Multimedia Signal Processing GroupSwiss Federal Institute of Technology

2Outline

Moti ation Motivation System overview

Obj t d li t d t ti Object duplicate detection GeoTag propagation

D t t Dataset Results Conclusions


3

Over the last few years social network systems and

Motivation

Over the last few years, social network systems and photo sharing web sites have greatly increased their popularity

To manage a large number of photos tagging is one of To manage a large number of photos, tagging is one of the popular methods, which enables us to search our photo collection with keywords


4

A GPS sensor provides only the location of the photographer instead

Motivation

A GPS sensor provides only the location of the photographer instead of that of the captured landmark GPS alone may not be enough to distinguish between two landmarks within a

cityGPS i t il bl f l i GPS is not available for several images GeoTag propagation can solve this problem and make available several GPS

based application GPS coordinates are not precise every timep y

Large error distance, sometimes several 100m IPTC schema is considered:

City, Sublocation (area or name of the landmark) Examples: Berlin (Reichstag), San Francisco (Golden Gate Bridge)

Build up an automatic tag propagation system in order to decrease the annotation time and increase the accuracy of the system


5

GeoTag propagation can improve several applications

Motivation

GeoTag propagation can improve several applications


6System overview

Evaluating uploaded Evaluating uploaded images

The system propose Landmark namesLandmark names.

Several application benefit from

ibl GPSaccessible GPS or landmark information


7Object duplicate detectionFeature extraction

Interest region detector (HessAff)

Region descriptor (SIFT)Image Features

Trainingg

Hierarchical clustering

Spatial neighborhood

Feature extraction

Training Images

M d l

Level 1

Level 2

graphModel

Testing

Feature FeatureT t

Level 3

Feature extraction

Feature matching

Non-maximum suppression

Bounding box merging based on Graph repr.

Bounding boxes

Test images

Graph matching

General Hough transform


8

Object duplicate detection module provides a matching score

Tag Propagation

Object duplicate detection module provides a matching score matrix

Two applications scenarios:Cl d t bl

, 1 , 1 i , jS i ,M j ,N

Closed set problem:

11 if

0 otherwise

i , j i , ji ,Mi , j

, S max SC

Open set problem:

0 otherwise ,

11 if

0 otherwise

i , j i , j i , ji ,Mi , j

ˆ, S max S S SO

,


9

1320 images 22 cities 3 sublocations for each city

Dataset

1320 images ‒ 22 cities, 3 sublocations for each city, 20 sample images for each sublocation


10

The recognition rate for all landmarks

Results

The recognition rate for all landmarks


11

The recognition rate across the different locations groups

Results

The recognition rate across the different locations groups (bars) and the recognition rate of all locations (dashed line)


12Conclusions

An efficient system for automaticAn efficient system for automatic geotag propagation by associating locations with distinctive landmarks and using object duplicateand using object duplicate detection for tag propagation

Several application benefit from ppaccessible GPS or landmark information Augmented reality Augmented reality Landmark search Tourist information

Saves several hour of tagging!Multimedia Signal Processing GroupSwiss Federal Institute of Technology

Saves several hour of tagging!

13

Questions?


propagation ofpropagation of geotags based onbased on object … · 2019-01-22 · 3 over the last...

Documents