SocialSensor at MediaEval Placing Task 2014

Giorgos Kordopatis-Zilos, Giorgos Orfanidis,

Symeon Papadopoulos and Yiannis Kompatsiaris

Information Technologies Institute, CERTH, Thessaloniki, Greece

Summary of our participation

• Tag-based location estimation (3 runs)– Using the language model-based scheme of (A. Popescu, MediaEval ‘13)

as basis;– Extending it with the use of the Similarity Search method, an internal grid

technique and a Gaussian distribution model based on the spatial entropy of tags.

• Visual-based location estimation (2 runs)– Extract SURF+VLAD and CS-LBP+VLAD features.– Training of linear SVM for clustering the samples.

2

Tag-based location estimation

3

Baseline Approach

Pre-Processing & Filtering

Grid Of Cells & Language Model

Assignment in Cells

Baseline Approach (1/3)

4

Baseline Approach

Pre-Processing & Filtering

Grid Of Cells & Language Model

Assignment in Cells

• Remove all punctuation and symbols, e.g. “.%!&”

• Transform all characters to lower case

• Remove images with empty tags and title

Baseline Approach (2/3)

5

Baseline Approach

Pre-Processing & Filtering

Grid Of Cells & Language Model

Assignment in Cells

Baseline Approach (3/3)

6

Baseline Approach

Pre-Processing & Filtering

Grid Of Cells & Language Model

Assignment in Cells

Extensions

7

Baseline Approach

Pre-Procession & Filtering

Grid Of Cells & Language Model

Assignment in Cells

Extensions

8

Extensions(1/3) - Similarity Search

9

Extensions(2/3) – Internal Grid

10

Extensions(3/3) – Spatial Entropy

11

Visual-based location estimation Model building• Extract two features, SURF+VLAD and CS-LBP+VLAD• Training of linear SVM in a predefined number of spatial clusters and subclusters

– 50 clusters and 50 subclusters corresponding to each cluster

Location Estimation• Decision of cluster membership:

– Estimation scores provided by the cluster weight vectors – Scores corresponding to the best subcluster weight score in

each cluster.•Variants for the final estimation:

– Assign each cluster with the coordinates of the most possible subcluster

– Similarity Search with 1000 samples of the selected subcluster

Runs and Results

12

• Tag-based runs: Run 1, Run 4 and Run 5• Visual runs: Run 2 and Run 3• The full test set used for all runs except Run 3• For Run 3 a subset of 25,500 images was used

The end

This work was supported by the SocialSensor FP7 project

13