Download - Visual Analysis of Image Collections
![Page 1: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/1.jpg)
Visual Analysis ofImage Collections
Danilo Medeiros Eler
SP-ASC – July, 2010
![Page 2: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/2.jpg)
Visual Analysis ofImage Collections
Danilo Medeiros ElerMarcel Yugo Nakazaki
Fernando Vieira PaulovichDavi Pereira Santos
Gabriel AnderyBruno Brandoli
Maria Cristina Ferreira de OliveiraJoão do Espírito Santo Batista Neto
Rosane Minghim
SP-ASC – July, 2010
![Page 3: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/3.jpg)
3
Contents
Exploration of image collections Approach to compare
Distance metrics Feature vectors
New approach to feature space definition
![Page 4: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/4.jpg)
4
Least Squares Projection (LSP)
(Paulovich et al, 2008)
![Page 5: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/5.jpg)
5
Neighbor-Joining (NJ) Similarity Tree
(Cuadros et al, 2007)
![Page 6: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/6.jpg)
6
Projection Explorer (PEx) Framework
(Paulovich et al, 2007)
![Page 7: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/7.jpg)
7
Projection Explorer for Images(PEx-Image)
(Eler et al, 2009)
![Page 8: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/8.jpg)
8
PEx-Image – Sample Content
![Page 9: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/9.jpg)
9
Detailed Inspection
537 X-Ray images112 classes
(ImageCLEF 2006)Wavelet Features
![Page 10: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/10.jpg)
10
Detailed Inspection
537 X-Ray images112 classes
(ImageCLEF 2006)Wavelet Features
![Page 11: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/11.jpg)
11
Detailed Inspection (zoom in)
537 X-Ray images112 classes
(ImageCLEF 2006)Wavelet Features
![Page 12: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/12.jpg)
12
PEx-Image – Group Content
![Page 13: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/13.jpg)
13
PEx-Image – Image as Visual Mark
537 X-Ray images112 classes
(ImageCLEF 2006)Wavelet Features
![Page 14: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/14.jpg)
video_Interaction.avi
![Page 15: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/15.jpg)
15
ImageCLEF Training Data Set (1)
Wavelet Features
9000 X-Ray images116 classes
(ImageCLEF 2006)
Wavelet Features
9000 X-Ray images116 classes
(ImageCLEF 2006)
![Page 16: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/16.jpg)
16
ImageCLEF Training Data Set (1)
Wavelet Features
9000 X-Ray images116 classes
(ImageCLEF 2006)
![Page 17: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/17.jpg)
17
ImageCLEF Training Data Set (2)
Class 108 Class 111
Wavelet Features
9000 X-Ray images116 classes
(ImageCLEF 2006)
![Page 18: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/18.jpg)
18
Images Without Class Information
537 X-Ray images112 classes
(ImageCLEF 2006)Wavelet Features
![Page 19: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/19.jpg)
19
Images Without Class Information 537 X-Ray images
112 classes(ImageCLEF 2006)Wavelet Features
![Page 20: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/20.jpg)
20
Colors from NN Classifier
Training Data Set
Neural Network
Neural Network
Classifier
Neural Network
Classifier
Image Data set
Labeled Images
Labeled Images
![Page 21: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/21.jpg)
21
Colors from NN Classifier (1)
Class Information NN Information
537 X-Ray images112 classes
(ImageCLEF 2006)Wavelet Features
![Page 22: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/22.jpg)
22
Class Information NN Information
Colors from NN Classifier (1)
![Page 23: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/23.jpg)
23
Class Information NN Information
Colors from NN Classifier (1)
![Page 24: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/24.jpg)
24
PEx-Image – Coordination
![Page 25: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/25.jpg)
25
PEx-Image – Coordination
![Page 26: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/26.jpg)
26
Comparison of Distance Metrics
Euclidean City Block Cosine
512 MRI medical images12 classes
![Page 27: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/27.jpg)
27
Comparison of Distance Metrics
Euclidean City Block Cosine
512 MRI medical images12 classes
![Page 28: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/28.jpg)
28
Comparison of Feature Space (1)
16 GaborFilters
Fourier, Meanand Deviation
72 co-ocurrencematrices All combined
512 MRI medical images12 classes
![Page 29: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/29.jpg)
29
Comparison of Feature Space (1)
16 GaborFilters
Fourier, Meanand Deviation
72 co-ocurrencematrices All combined
512 MRI medical images12 classes
![Page 30: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/30.jpg)
30
Comparison of Feature Space (2)
All combined
1000 X-Ray images from ImageCLEF116 classes
Wavelet Features
![Page 31: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/31.jpg)
31
Comparison of Feature Space (2)
All combined
1000 X-Ray images from ImageCLEF116 classes
Wavelet Features
![Page 32: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/32.jpg)
32
Recent Approach (Brandoli et al, 2010) Main Goals
Visual framework which help users to better “understand” different sets of features
A method to objectively evaluate the quality of projections
![Page 33: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/33.jpg)
33
Recent Approach (Brandoli et al, 2010)
(Brandoli et al, 2010)
The silhouette can vary between -1 <= S <= 1Larger values indicate better cohesion and separation between clusters
![Page 34: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/34.jpg)
34
Recent Approach (Brandoli et al, 2010)
Dataset: 70 texture images from BrodatzFeatures: Gabor filters (4 orientations and 4 scales)
Silhouette: 0.676
![Page 35: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/35.jpg)
35
Recent Approach (Brandoli et al, 2010)
Dataset: 100 texture images from BrodatzFeatures: Gabor filters (4 orientations and 4 scales)
Silhouette: 0.429
![Page 36: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/36.jpg)
36
Recent Approach (Brandoli et al, 2010)
Zoom in
![Page 37: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/37.jpg)
37
Recent Approach (Brandoli et al, 2010)
Dataset: 70 texture images from BrodatzFeatures: Gabor filters (90o orientation and 4 scales)
Silhouette: 0.474
![Page 38: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/38.jpg)
38
Recent Approach (Brandoli et al, 2010)
Dataset: 70 texture images from BrodatzFeatures: Gabor filters (90o orientation and 4 scales)
Silhouette: 0.474
![Page 39: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/39.jpg)
39
Recent Approach (Brandoli et al, 2010)
Silhouette: 0.583
Dataset: 70 texture images from BrodatzFeatures: Co-occurrence Matrix(5 measures, 5 distances and 4 directions)
![Page 40: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/40.jpg)
40
Recent Example KTH-TIPS database
10 colorful texture classes 9 different scales
3 illumination directions and 3 poses 9 images per scale
Texture methods Gabor Filtes Co-Occurrence Matrix
Color methods Color Moment Invariants RGB Histogram SIFT
![Page 41: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/41.jpg)
41
Texture Methods – KTH-TIPS database (Colored Texture)
Feature: GaborSilhouette: -0.2535K-NN: 83%
Feature: Co-occurrence MatrixSilhouette: -0.3727K-NN: 70%
Feature: GaborSilhouette: -0.2535K-NN: 83%
![Page 42: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/42.jpg)
42
Color Methods – KTH-TIPS database (Colored Texture)
Feature: Color Moment InvariantsSilhouette: -0.2835K-NN: 78%
Feature: RGB HistogramSilhouette: -0.1845K-NN: 91%
![Page 43: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/43.jpg)
43
Color Methods – KTH-TIPS database (Colored Texture)
Feature: SIFTSilhouette: -0.1025K-NN: 92%
![Page 44: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/44.jpg)
44
Color Methods – KTH-TIPS database (Colored Texture)
Feature: All Previous CombinedSilhouette: -0.2547K-NN: 84%
Feature: PCA Reduction to 10 dimensionsSilhouette: 0.1290K-NN: 98%
![Page 45: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/45.jpg)
45
![Page 46: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/46.jpg)
46
Conclusions
PEx-Image: a set of tools and a novel approach to Map an image data set onto 2D space Make data analysis and exploration more effective
Provide evaluation of Similarity measures Feature spaces Feature selection strategies
Recent Approach (Brandoli et al, 2010) Guidance to understand and define a set of
features that properly represents an image dataset
![Page 48: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/48.jpg)
48
References Eler, D.; Nakazaki, M.; Paulovich, F.; Santos, D.; Andery, G.; Oliveira, M.;
Batista, J.; Minghim, R. Visual analysis of image collections. The Visual Computer, v. 25, n. 10, p. 923–937, 2009.
Eler, D. M.; Nakazaki, M. Y.; Paulovich, F. V.; Santos, D. P.; Oliveira, M. C. F.; Batista, J.; Minghim, R. Multidimensional visualization to support analysis of image collections. In: Proceedings of the XXI Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 2008), Campo Grande, Brazil: IEEE Computer Society, 2008, p. 289–296.
Eler, D. M.; Paulovich, F. V.; Oliveira, M. C. F. d.; Minghim, R. Coordinated and multiple views for visualizing text collections. In: IV ’08: Proceedings of the 12th International Conference Information Visualisation, Washington, DC, USA: IEEE Computer Society, 2008, p. 246–251.
Eler, D. M.; Paulovich, F. V.; Oliveira, M. C. F. d.; Minghim, R. Topic-based coordination for visual analysis of evolving document collections. In: IV ’09: Proceedings of the 13th International Conference Information Visualisation, Washington, DC, USA: IEEE Computer Society, 2009, p. 149–155.
Paulovich, F. V.; Eler, D. M.; Poco, J.; Nonato, L. G.; Botha, C. P.; Minghim, R. A fast projection technique and its applications to visualization of large data sets. Technical Report 349, Instituto de Ciências Matemáticas e de Computação – Universidade de São Paulo, 2010.
![Page 49: Visual Analysis of Image Collections](https://reader036.vdocuments.net/reader036/viewer/2022062520/5681593b550346895dc677e9/html5/thumbnails/49.jpg)
49
References PAULOVICH, F. V.; OLIVEIRA, M. C. F.; MINGHIM, R. The Projection Explorer:
A flexible tool for projection-based multidimensional visualization. In: Proceedings of the XX Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI ’07), Washington, DC, USA: IEEE Computer Society, 2007, p. 27–36
CUADROS, A. M.; PAULOVICH, F. V.; MINGHIM, R.; TELLES, G. P. Point placement by phylogenetic trees and its application for visual analysis of document collections. In: IEEE Symposium on Visual Analytics Science and Technology 2007, Sacramento, CA, USA, 2007, p. 99–106
Brandoli, B.; Eler, D. M.; Paulovich, F. V.; Minghim, R.; Batista, J. Visual Data Exploration to Feature Space Definition. In: Proceedings of the XXIII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 2010) – To Appear – Gramado, Brazil: IEEE Computer Society, 2010