fine-grained fine-grained recognition( 细粒度分类 ) 沈志强
DESCRIPTION
Methods feature extraction + classification global feature extraction + part feature representationsTRANSCRIPT
![Page 1: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/1.jpg)
Fine-grained Recognition(细粒度分类 )
沈志强
![Page 2: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/2.jpg)
Datasets -- Caltech-UCSD Bird-200-2011
Number of categories: 200Number of images: 11,788Annotations per image: 15 Part Locations, 1 Bounding
Box
![Page 3: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/3.jpg)
Methods
feature extraction + classification
global feature extraction + part feature representations
![Page 4: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/4.jpg)
Object hypothesis[1]
• Multiscale model: the resolution of part filters is twice the resolution of the root
![Page 5: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/5.jpg)
Scoring an object hypothesis• The score of a hypothesis is the sum of filter
scores minus the sum of deformation costs
),,,()(),...,( 22
0 10 ii
n
i
n
iiiiiin dydxdydxDpHFppscore
Filters
Subwindow features
Deformation weights
Displacements
![Page 6: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/6.jpg)
Scoring an object hypothesis• The score of a hypothesis is the sum of filter
scores minus the sum of deformation costs
)()( zHwzscore
Concatenation of filter and
deformation weights
Concatenation of subwindow features and displacements
),,,()(),...,( 22
0 10 ii
n
i
n
iiiiiin dydxdydxDpHFppscore
Filters
Subwindow features
Deformation weights
Displacements
![Page 7: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/7.jpg)
Training• Our classifier has the form
• w are model parameters, z are latent hypotheses
• Latent SVM training:• Initialize w and iterate:• Fix w and find the best z for each training example
(detection)• Fix z and solve for w (standard SVM training)
• Issue: too many negative examples• Do “data mining” to find “hard” negatives
),(max)( zxHwxf z
![Page 8: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/8.jpg)
Deformable Part Descriptors (DPDs) - ICCV2013[4]
Strongly-supervised DPD Weakly-supervised DPD
![Page 9: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/9.jpg)
Pose-normalization
Strongly-supervised DPD
is the pooled image feature for semantic region rl figure out a mapping S(j) :
![Page 10: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/10.jpg)
Pose-normalization
Weakly-supervised DPD
![Page 11: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/11.jpg)
Detection results
![Page 12: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/12.jpg)
![Page 13: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/13.jpg)
Nonparametric Part Transfer for Fine-grained Recognition(CVPR 2014) [3]
![Page 14: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/14.jpg)
Nonparametric Part Transfer for Fine-grained Recognition(CVPR 2014)
![Page 15: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/15.jpg)
Nonparametric Part Transfer for Fine-grained Recognition(CVPR 2014) The distribution is clearly non-Gaussian,
therefore, a single DPM model would not be able to model the variation present in the training dataset.
![Page 16: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/16.jpg)
Nonparametric Part Transfer for Fine-grained Recognition(CVPR 2014)
![Page 17: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/17.jpg)
Example detections
![Page 18: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/18.jpg)
Part-based R-CNNs for Fine-grained Category Detection(ECCV 2014 oral) [2]
![Page 19: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/19.jpg)
Part-based R-CNNs for Fine-grained Category Detection(ECCV 2014 oral) Geometric constraints Let X = {x0 , x1 ,..., xn} denote the locations (bounding
boxes) of object p0 and n parts {pi}.
where σ(·) is the sigmoid function and φ(x) is the CNN feature descriptor extracted at location x.
where ∆(X) defines a scoring function over the joint configuration of the object and root bounding box.
![Page 20: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/20.jpg)
Part-based R-CNNs for Fine-grained Category Detection(ECCV 2014 oral) Box constraints
![Page 21: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/21.jpg)
Part-based R-CNNs for Fine-grained Category Detection(ECCV 2014 oral) Geometric constraints
where δi is a scoring function for the position of the part pi given the training data.
![Page 22: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/22.jpg)
Illustration of geometric constant
![Page 23: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/23.jpg)
![Page 24: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/24.jpg)
Recall
![Page 25: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/25.jpg)
Results
![Page 26: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/26.jpg)
Conclusionfeature extraction + classification
global feature extraction and part feature representations
Part localization is a crucial step .
![Page 27: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/27.jpg)
References[1] Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D. Object detection with discriminatively trained part based models. IEEE Transactions on Pattern Analysis and Machine Intelligence (2010) [2] Ning Zhang, Jeff Donahue, Ross Girshick, Trevor Darrell.Part-based R-CNNs for Fine-grained Category Detection. ECCV 2014.[3] Christoph Goring, Erik Rodner, Alexander Freytag, and Joachim Denzler∗. Nonparametric Part Transfer for Fine-grained Recognition. CVPR 2014[4] N. Zhang, R. Farrell, F. Iandola, and T. Darrell. Deformable part descriptors for fine-grained recognition and attribute prediction. In ICCV, 2013.
![Page 28: Fine-grained Fine-grained Recognition( 细粒度分类 ) 沈志强](https://reader033.vdocuments.net/reader033/viewer/2022061410/5a4d1c0c7f8b9ab0599f44b2/html5/thumbnails/28.jpg)
Thanks & Questions