visual analysis of image collections
DESCRIPTION
SP-ASC – July, 2010. Visual Analysis of Image Collections. Danilo Medeiros Eler. SP-ASC – July, 2010. Visual Analysis of Image Collections. Danilo Medeiros Eler Marcel Yugo Nakazaki Fernando Vieira Paulovich Davi Pereira Santos Gabriel Andery Bruno Brandoli - PowerPoint PPT PresentationTRANSCRIPT
Visual Analysis ofImage Collections
Danilo Medeiros Eler
SP-ASC – July, 2010
Visual Analysis ofImage Collections
Danilo Medeiros ElerMarcel Yugo Nakazaki
Fernando Vieira PaulovichDavi Pereira Santos
Gabriel AnderyBruno Brandoli
Maria Cristina Ferreira de OliveiraJoão do Espírito Santo Batista Neto
Rosane Minghim
SP-ASC – July, 2010
3
Contents
Exploration of image collections Approach to compare
Distance metrics Feature vectors
New approach to feature space definition
4
Least Squares Projection (LSP)
(Paulovich et al, 2008)
5
Neighbor-Joining (NJ) Similarity Tree
(Cuadros et al, 2007)
6
Projection Explorer (PEx) Framework
(Paulovich et al, 2007)
7
Projection Explorer for Images(PEx-Image)
(Eler et al, 2009)
8
PEx-Image – Sample Content
9
Detailed Inspection
537 X-Ray images112 classes
(ImageCLEF 2006)Wavelet Features
10
Detailed Inspection
537 X-Ray images112 classes
(ImageCLEF 2006)Wavelet Features
11
Detailed Inspection (zoom in)
537 X-Ray images112 classes
(ImageCLEF 2006)Wavelet Features
12
PEx-Image – Group Content
13
PEx-Image – Image as Visual Mark
537 X-Ray images112 classes
(ImageCLEF 2006)Wavelet Features
video_Interaction.avi
15
ImageCLEF Training Data Set (1)
Wavelet Features
9000 X-Ray images116 classes
(ImageCLEF 2006)
Wavelet Features
9000 X-Ray images116 classes
(ImageCLEF 2006)
16
ImageCLEF Training Data Set (1)
Wavelet Features
9000 X-Ray images116 classes
(ImageCLEF 2006)
17
ImageCLEF Training Data Set (2)
Class 108 Class 111
Wavelet Features
9000 X-Ray images116 classes
(ImageCLEF 2006)
18
Images Without Class Information
537 X-Ray images112 classes
(ImageCLEF 2006)Wavelet Features
19
Images Without Class Information 537 X-Ray images
112 classes(ImageCLEF 2006)Wavelet Features
20
Colors from NN Classifier
Training Data Set
Neural Network
Neural Network
Classifier
Neural Network
Classifier
Image Data set
Labeled Images
Labeled Images
21
Colors from NN Classifier (1)
Class Information NN Information
537 X-Ray images112 classes
(ImageCLEF 2006)Wavelet Features
22
Class Information NN Information
Colors from NN Classifier (1)
23
Class Information NN Information
Colors from NN Classifier (1)
24
PEx-Image – Coordination
25
PEx-Image – Coordination
26
Comparison of Distance Metrics
Euclidean City Block Cosine
512 MRI medical images12 classes
27
Comparison of Distance Metrics
Euclidean City Block Cosine
512 MRI medical images12 classes
28
Comparison of Feature Space (1)
16 GaborFilters
Fourier, Meanand Deviation
72 co-ocurrencematrices All combined
512 MRI medical images12 classes
29
Comparison of Feature Space (1)
16 GaborFilters
Fourier, Meanand Deviation
72 co-ocurrencematrices All combined
512 MRI medical images12 classes
30
Comparison of Feature Space (2)
All combined
1000 X-Ray images from ImageCLEF116 classes
Wavelet Features
31
Comparison of Feature Space (2)
All combined
1000 X-Ray images from ImageCLEF116 classes
Wavelet Features
32
Recent Approach (Brandoli et al, 2010) Main Goals
Visual framework which help users to better “understand” different sets of features
A method to objectively evaluate the quality of projections
33
Recent Approach (Brandoli et al, 2010)
(Brandoli et al, 2010)
The silhouette can vary between -1 <= S <= 1Larger values indicate better cohesion and separation between clusters
34
Recent Approach (Brandoli et al, 2010)
Dataset: 70 texture images from BrodatzFeatures: Gabor filters (4 orientations and 4 scales)
Silhouette: 0.676
35
Recent Approach (Brandoli et al, 2010)
Dataset: 100 texture images from BrodatzFeatures: Gabor filters (4 orientations and 4 scales)
Silhouette: 0.429
36
Recent Approach (Brandoli et al, 2010)
Zoom in
37
Recent Approach (Brandoli et al, 2010)
Dataset: 70 texture images from BrodatzFeatures: Gabor filters (90o orientation and 4 scales)
Silhouette: 0.474
38
Recent Approach (Brandoli et al, 2010)
Dataset: 70 texture images from BrodatzFeatures: Gabor filters (90o orientation and 4 scales)
Silhouette: 0.474
39
Recent Approach (Brandoli et al, 2010)
Silhouette: 0.583
Dataset: 70 texture images from BrodatzFeatures: Co-occurrence Matrix(5 measures, 5 distances and 4 directions)
40
Recent Example KTH-TIPS database
10 colorful texture classes 9 different scales
3 illumination directions and 3 poses 9 images per scale
Texture methods Gabor Filtes Co-Occurrence Matrix
Color methods Color Moment Invariants RGB Histogram SIFT
41
Texture Methods – KTH-TIPS database (Colored Texture)
Feature: GaborSilhouette: -0.2535K-NN: 83%
Feature: Co-occurrence MatrixSilhouette: -0.3727K-NN: 70%
Feature: GaborSilhouette: -0.2535K-NN: 83%
42
Color Methods – KTH-TIPS database (Colored Texture)
Feature: Color Moment InvariantsSilhouette: -0.2835K-NN: 78%
Feature: RGB HistogramSilhouette: -0.1845K-NN: 91%
43
Color Methods – KTH-TIPS database (Colored Texture)
Feature: SIFTSilhouette: -0.1025K-NN: 92%
44
Color Methods – KTH-TIPS database (Colored Texture)
Feature: All Previous CombinedSilhouette: -0.2547K-NN: 84%
Feature: PCA Reduction to 10 dimensionsSilhouette: 0.1290K-NN: 98%
45
46
Conclusions
PEx-Image: a set of tools and a novel approach to Map an image data set onto 2D space Make data analysis and exploration more effective
Provide evaluation of Similarity measures Feature spaces Feature selection strategies
Recent Approach (Brandoli et al, 2010) Guidance to understand and define a set of
features that properly represents an image dataset
48
References Eler, D.; Nakazaki, M.; Paulovich, F.; Santos, D.; Andery, G.; Oliveira, M.;
Batista, J.; Minghim, R. Visual analysis of image collections. The Visual Computer, v. 25, n. 10, p. 923–937, 2009.
Eler, D. M.; Nakazaki, M. Y.; Paulovich, F. V.; Santos, D. P.; Oliveira, M. C. F.; Batista, J.; Minghim, R. Multidimensional visualization to support analysis of image collections. In: Proceedings of the XXI Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 2008), Campo Grande, Brazil: IEEE Computer Society, 2008, p. 289–296.
Eler, D. M.; Paulovich, F. V.; Oliveira, M. C. F. d.; Minghim, R. Coordinated and multiple views for visualizing text collections. In: IV ’08: Proceedings of the 12th International Conference Information Visualisation, Washington, DC, USA: IEEE Computer Society, 2008, p. 246–251.
Eler, D. M.; Paulovich, F. V.; Oliveira, M. C. F. d.; Minghim, R. Topic-based coordination for visual analysis of evolving document collections. In: IV ’09: Proceedings of the 13th International Conference Information Visualisation, Washington, DC, USA: IEEE Computer Society, 2009, p. 149–155.
Paulovich, F. V.; Eler, D. M.; Poco, J.; Nonato, L. G.; Botha, C. P.; Minghim, R. A fast projection technique and its applications to visualization of large data sets. Technical Report 349, Instituto de Ciências Matemáticas e de Computação – Universidade de São Paulo, 2010.
49
References PAULOVICH, F. V.; OLIVEIRA, M. C. F.; MINGHIM, R. The Projection Explorer:
A flexible tool for projection-based multidimensional visualization. In: Proceedings of the XX Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI ’07), Washington, DC, USA: IEEE Computer Society, 2007, p. 27–36
CUADROS, A. M.; PAULOVICH, F. V.; MINGHIM, R.; TELLES, G. P. Point placement by phylogenetic trees and its application for visual analysis of document collections. In: IEEE Symposium on Visual Analytics Science and Technology 2007, Sacramento, CA, USA, 2007, p. 99–106
Brandoli, B.; Eler, D. M.; Paulovich, F. V.; Minghim, R.; Batista, J. Visual Data Exploration to Feature Space Definition. In: Proceedings of the XXIII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 2010) – To Appear – Gramado, Brazil: IEEE Computer Society, 2010