An Exemplar Model for Learning Object Classes

Authors: Ondrej Chum Andrew Zisserman@University of Oxford

Presenter: Shao-Chuan Wang

An Exemplar Model for Learning Object Classes

• Objective:– Give training images known to contain instances of an

object class, without specifying locations and scales.– Detect and localize object

• Kea Ideas: – Learn region of interest (ROI) around class instance in

weakly supervised training data.– Based on discriminative features to initialize ROI for

the optimization problem

An Exemplar Model for Learning Object Classes

• Exemplar model:

• Detection (cost function):

X

eewwD

AYXdYXdC

2

2)()),((),(

X: exemplar setX^w: PHOW descriptorX^e: PHOG descriptorA: aspect ratio of target region

XY

d: distance function/mu: mean of exemplars’ aspect ratio/sigma: std of exemplars’ aspect ratio/alpha, /beta: weighting to be tuned/learned

ii

ii

yx

yxyxyxyxd

2222 )(

),(;),(),(

An Exemplar Model for Learning Object Classes

• Learning the exemplar model:– Learn the regions in all images simultaneously.

• How to Determine initial ROI?– > By discriminative features

X

ee

Y

wwL

AYXdYXdC

2

2)()),((),(

Discriminative features

• Definition:

w

wwD

containingdatabaseinimage#

containingimageslabelledclass#~)(

Top 10 most discriminative visual words

Constructing ROI exemplars: Algorithm

Constructing ROI exemplars: Algorithm

1. Initialization– Calculate discriminability of visual words– Initialize the ROI in each training image by a bounding box of the

64 most discriminative features

2. Optimization of cost function– Find the ROI to minimize the cost function with \beta = 0– Re-initialization by detection

3. Refinement– Enlarge the ROI in the training images by 10%– Calculate discriminability of visual words using only the features

inside the ROI– Optimization of cost function (goto 2.)

Constructing ROI exemplars: Algorithm

1. Initialization– Calculate discriminability of visual words– Initialize the ROI in each training image by a bounding box of the

64 most discriminative features

2. Optimization of cost function– Find the ROI to minimize the cost function with \beta = 0– Re-initialization by detection

3. Refinement– Enlarge the ROI in the training images by 10%– Calculate discriminability of visual words using only the features

inside the ROI– Optimization of cost function (goto 2.)

Constructing ROI exemplars: Algorithm

1. Initialization– Calculate discriminability of visual words– Initialize the ROI in each training image by a bounding box of the

64 most discriminative features

2. Optimization of cost function– Find the ROI to minimize the cost function with \beta = 0– Re-initialization by detection

3. Refinement– Enlarge the ROI in the training images by 10%– Calculate discriminability of visual words using only the features

inside the ROI– Optimization of cost function (goto 2.)

X

ee

Y

wwL

AYXdYXdC

2

2)()),((),(

Constructing ROI exemplars: Algorithm

1. Initialization– Calculate discriminability of visual words– Initialize the ROI in each training image by a bounding box of the

64 most discriminative features

2. Optimization of cost function– Find the ROI to minimize the cost function with \beta = 0– Re-initialization by detection.

3. Refinement– Enlarge the ROI in the training images by 10%– Calculate discriminability of visual words using only the features

inside the ROI– Optimization of cost function (goto 2.)

Constructing ROI exemplars: Algorithm

1. Initialization– Calculate discriminability of visual words– Initialize the ROI in each training image by a bounding box of the

64 most discriminative features

2. Optimization of cost function– Find the ROI to minimize the cost function with \beta = 0– Re-initialization by detection.

3. Refinement– Enlarge the ROI in the training images by 10%– Calculate discriminability of visual words using only the features

inside the ROI– Optimization of cost function (goto 2.)

Constructing ROI exemplars: Algorithm

1. Initialization– Calculate discriminability of visual words– Initialize the ROI in each training image by a bounding box of the

64 most discriminative features

2. Optimization of cost function– Find the ROI to minimize the cost function with \beta = 0– Re-initialization by detection.

3. Refinement– Enlarge the ROI in the training images by 10%– Calculate discriminability of visual words using only the features

inside the ROI– Optimization of cost function (goto 2.)

Constructing ROI exemplars: Algorithm

• Three stages of the optimization process

Initialization

Optimization

Re-initializationviadetection

Using the exemplar model

• Object Detection

X

eewwD

AYXdYXdC

2

2)()),((),(

),( iRwHypothesis

Clustering

w

nwDRwS Rw

#)(),( ),(

Score of a hypothesis

n_(w,R): the number of exemplar Images consistent with the hypothesis

#w: the number of appearances of the visual word w in the exemplar images

20 strongest hypotheses are tested on each test image

Using other models

• Training:– Train an SVM, using features within ROI by

exemplar models• Object detection– Scores are ranked by SVM score

Results

Conclusion

• When constructing exemplars’ ROI, they use discriminability to initialize bounding box

• In detection, they used relative position of bounding boxes and visual words to try the most probable hypotheses.

• It may failed to detect when significant class variability in the exemplars, such as people class.