hierarchical matching with side information for image classification authors qiang chen et al....
TRANSCRIPT
![Page 1: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/1.jpg)
Hierarchical Matching with Side Information for Image Classification
Authors Qiang Chen et al.
Presenter Hyung-seok Lee (이형석 )
CVPR 2012
![Page 2: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/2.jpg)
2
Motivation (1/2)• Bag-of-Words (BoW)• Frequency histogram of visual words• Not emphasize any elements with regard to image layout
![Page 3: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/3.jpg)
3
Motivation (2/2)• Spatial Pyramid Matching (SPM)• Partitioning the image plane into fine sub-regions• Not optimum for object-centered recognition problem
Beyond Bags of Features : Spatial Pyramid Matching for Recognizing Natural Scene Categories, Svetlana Lazebnik et al.
![Page 4: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/4.jpg)
4
Main Idea• Generalized Hierarchical Matching (GHM)• Extends the popular matching work• Integrate prior knowledge for enhancing feature matching
side information
![Page 5: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/5.jpg)
5
Overview
Im-age
Hierarchi-cal cluster-
ing
Hierarchical structure
representation
Matching over each cluster
Local features &
side informa-tion
![Page 6: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/6.jpg)
6
Side Information (1/4)• Object Confidence Map• Object position should be extremely beneficial• Shape model / Appearance model
![Page 7: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/7.jpg)
7
Side Information (2/4)• Shape-based object detection : HOG features
• Appearance-based object detection : BoW features
Object Confidence Map
![Page 8: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/8.jpg)
8
Side Information (3/4)• Visual Saliency Map• Aim to detect the important content in images• Explain human visual search strategies
SUN : A Bayesian Framework for Saliency Using Natural Statistics, Lingyun Zhang et el.
![Page 9: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/9.jpg)
9
Side Information (4/4)• Use the saliency model SUN (Saliency Using Natural statistics)• Based on natural image statistics
![Page 10: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/10.jpg)
10
Side Information Combination• Combine object confidence map & visual saliency map• Design a 3 level hierarchical structure
- Object confidence map is used in level 2
- Background area will be further utilized in level 3
![Page 11: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/11.jpg)
11
Encoding & Matching (1/2)
![Page 12: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/12.jpg)
12
Encoding & Matching (2/2)• Encoding is operated on each cluster• Matching over each corresponding cluster
![Page 13: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/13.jpg)
13
Experiments (1/2)• Caltech-UCSD Birds 200 & Oxford Flowers 17 and 102
![Page 14: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/14.jpg)
14
Experiments (2/2)• VOC 2007
• VOC 2010
![Page 15: Hierarchical Matching with Side Information for Image Classification Authors Qiang Chen et al. Presenter Hyung-seok Lee ( 이형석 ) CVPR 2012](https://reader035.vdocuments.net/reader035/viewer/2022062804/5697bf881a28abf838c8989d/html5/thumbnails/15.jpg)
15
Conclusion• Propose the Generalized Hierarchical Matching framework
- Extends the popular pyramid matching
• Two novel kinds of side information are introduced to enhance object-oriented image classification tasks
- Object confidence map- Visual saliency amp