visual analysis of image collections

Visual Analysis ofImage Collections

Danilo Medeiros Eler

SP-ASC – July, 2010

Visual Analysis ofImage Collections

Danilo Medeiros ElerMarcel Yugo Nakazaki

Fernando Vieira PaulovichDavi Pereira Santos

Gabriel AnderyBruno Brandoli

Maria Cristina Ferreira de OliveiraJoão do Espírito Santo Batista Neto

Rosane Minghim

SP-ASC – July, 2010

3

Contents

Exploration of image collections Approach to compare

Distance metrics Feature vectors

New approach to feature space definition

4

Least Squares Projection (LSP)

(Paulovich et al, 2008)

5

Neighbor-Joining (NJ) Similarity Tree

(Cuadros et al, 2007)

6

Projection Explorer (PEx) Framework

(Paulovich et al, 2007)

7

Projection Explorer for Images(PEx-Image)

(Eler et al, 2009)

8

PEx-Image – Sample Content

9

Detailed Inspection

537 X-Ray images112 classes

(ImageCLEF 2006)Wavelet Features

10

Detailed Inspection

537 X-Ray images112 classes

(ImageCLEF 2006)Wavelet Features

11

Detailed Inspection (zoom in)

537 X-Ray images112 classes

(ImageCLEF 2006)Wavelet Features

12

PEx-Image – Group Content

13

PEx-Image – Image as Visual Mark

537 X-Ray images112 classes

(ImageCLEF 2006)Wavelet Features

video_Interaction.avi

15

ImageCLEF Training Data Set (1)

Wavelet Features

9000 X-Ray images116 classes

(ImageCLEF 2006)

Wavelet Features

9000 X-Ray images116 classes

(ImageCLEF 2006)

16

ImageCLEF Training Data Set (1)

Wavelet Features

9000 X-Ray images116 classes

(ImageCLEF 2006)

17

ImageCLEF Training Data Set (2)

Class 108 Class 111

Wavelet Features

9000 X-Ray images116 classes

(ImageCLEF 2006)

18

Images Without Class Information

537 X-Ray images112 classes

(ImageCLEF 2006)Wavelet Features

19

Images Without Class Information 537 X-Ray images

112 classes(ImageCLEF 2006)Wavelet Features

20

Colors from NN Classifier

Training Data Set

Neural Network

Neural Network

Classifier

Neural Network

Classifier

Image Data set

Labeled Images

Labeled Images

21

Colors from NN Classifier (1)

Class Information NN Information

537 X-Ray images112 classes

(ImageCLEF 2006)Wavelet Features

22

Class Information NN Information

Colors from NN Classifier (1)

23

Class Information NN Information

Colors from NN Classifier (1)

24

PEx-Image – Coordination

25

PEx-Image – Coordination

26

Comparison of Distance Metrics

Euclidean City Block Cosine

512 MRI medical images12 classes

27

Comparison of Distance Metrics

Euclidean City Block Cosine

512 MRI medical images12 classes

28

Comparison of Feature Space (1)

16 GaborFilters

Fourier, Meanand Deviation

72 co-ocurrencematrices All combined

512 MRI medical images12 classes

29

Comparison of Feature Space (1)

16 GaborFilters

Fourier, Meanand Deviation

72 co-ocurrencematrices All combined

512 MRI medical images12 classes

30

Comparison of Feature Space (2)

All combined

1000 X-Ray images from ImageCLEF116 classes

Wavelet Features

31

Comparison of Feature Space (2)

All combined

1000 X-Ray images from ImageCLEF116 classes

Wavelet Features

32

Recent Approach (Brandoli et al, 2010) Main Goals

Visual framework which help users to better “understand” different sets of features

A method to objectively evaluate the quality of projections

33

Recent Approach (Brandoli et al, 2010)

(Brandoli et al, 2010)

The silhouette can vary between -1 <= S <= 1Larger values indicate better cohesion and separation between clusters

34

Recent Approach (Brandoli et al, 2010)

Dataset: 70 texture images from BrodatzFeatures: Gabor filters (4 orientations and 4 scales)

Silhouette: 0.676

35

Recent Approach (Brandoli et al, 2010)

Dataset: 100 texture images from BrodatzFeatures: Gabor filters (4 orientations and 4 scales)

Silhouette: 0.429

36

Recent Approach (Brandoli et al, 2010)

Zoom in

37

Recent Approach (Brandoli et al, 2010)

Dataset: 70 texture images from BrodatzFeatures: Gabor filters (90o orientation and 4 scales)

Silhouette: 0.474

38

Recent Approach (Brandoli et al, 2010)

Dataset: 70 texture images from BrodatzFeatures: Gabor filters (90o orientation and 4 scales)

Silhouette: 0.474

39

Recent Approach (Brandoli et al, 2010)

Silhouette: 0.583

Dataset: 70 texture images from BrodatzFeatures: Co-occurrence Matrix(5 measures, 5 distances and 4 directions)

40

Recent Example KTH-TIPS database

10 colorful texture classes 9 different scales

3 illumination directions and 3 poses 9 images per scale

Texture methods Gabor Filtes Co-Occurrence Matrix

Color methods Color Moment Invariants RGB Histogram SIFT

41

Texture Methods – KTH-TIPS database (Colored Texture)

Feature: GaborSilhouette: -0.2535K-NN: 83%

Feature: Co-occurrence MatrixSilhouette: -0.3727K-NN: 70%

Feature: GaborSilhouette: -0.2535K-NN: 83%

42

Color Methods – KTH-TIPS database (Colored Texture)

Feature: Color Moment InvariantsSilhouette: -0.2835K-NN: 78%

Feature: RGB HistogramSilhouette: -0.1845K-NN: 91%

43

Color Methods – KTH-TIPS database (Colored Texture)

Feature: SIFTSilhouette: -0.1025K-NN: 92%

44

Color Methods – KTH-TIPS database (Colored Texture)

Feature: All Previous CombinedSilhouette: -0.2547K-NN: 84%

Feature: PCA Reduction to 10 dimensionsSilhouette: 0.1290K-NN: 98%

45

46

Conclusions

PEx-Image: a set of tools and a novel approach to Map an image data set onto 2D space Make data analysis and exploration more effective

Provide evaluation of Similarity measures Feature spaces Feature selection strategies

Recent Approach (Brandoli et al, 2010) Guidance to understand and define a set of

features that properly represents an image dataset

47

Thank you

More information: http://infoserver.lcad.icmc.usp.br danilome@gmail.com

48

References Eler, D.; Nakazaki, M.; Paulovich, F.; Santos, D.; Andery, G.; Oliveira, M.;

Batista, J.; Minghim, R. Visual analysis of image collections. The Visual Computer, v. 25, n. 10, p. 923–937, 2009.

Eler, D. M.; Nakazaki, M. Y.; Paulovich, F. V.; Santos, D. P.; Oliveira, M. C. F.; Batista, J.; Minghim, R. Multidimensional visualization to support analysis of image collections. In: Proceedings of the XXI Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 2008), Campo Grande, Brazil: IEEE Computer Society, 2008, p. 289–296.

Eler, D. M.; Paulovich, F. V.; Oliveira, M. C. F. d.; Minghim, R. Coordinated and multiple views for visualizing text collections. In: IV ’08: Proceedings of the 12th International Conference Information Visualisation, Washington, DC, USA: IEEE Computer Society, 2008, p. 246–251.

Eler, D. M.; Paulovich, F. V.; Oliveira, M. C. F. d.; Minghim, R. Topic-based coordination for visual analysis of evolving document collections. In: IV ’09: Proceedings of the 13th International Conference Information Visualisation, Washington, DC, USA: IEEE Computer Society, 2009, p. 149–155.

Paulovich, F. V.; Eler, D. M.; Poco, J.; Nonato, L. G.; Botha, C. P.; Minghim, R. A fast projection technique and its applications to visualization of large data sets. Technical Report 349, Instituto de Ciências Matemáticas e de Computação – Universidade de São Paulo, 2010.

49

References PAULOVICH, F. V.; OLIVEIRA, M. C. F.; MINGHIM, R. The Projection Explorer:

A flexible tool for projection-based multidimensional visualization. In: Proceedings of the XX Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI ’07), Washington, DC, USA: IEEE Computer Society, 2007, p. 27–36

CUADROS, A. M.; PAULOVICH, F. V.; MINGHIM, R.; TELLES, G. P. Point placement by phylogenetic trees and its application for visual analysis of document collections. In: IEEE Symposium on Visual Analytics Science and Technology 2007, Sacramento, CA, USA, 2007, p. 99–106

Brandoli, B.; Eler, D. M.; Paulovich, F. V.; Minghim, R.; Batista, J. Visual Data Exploration to Feature Space Definition. In: Proceedings of the XXIII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 2010) – To Appear – Gramado, Brazil: IEEE Computer Society, 2010