2009. 8. 21 (fri) young ki baik computer vision lab

2009. 8. 21 (Fri)

Young Ki Baik

Computer Vision Lab.

References

Compact Signatures for High-speed Interest Point Description and Matching Michael Calonder, Vincent Lepetit, Pascal Fua et.al. (ICCV 2009 oral)

Keypoint signatures for fast learning and recognition M. Calonder, V. Lepetit, P. Fua (ECCV 2008)

Fast Keypoint Recognition using Random Ferns M. Ozuysal, M. Calonder, V. Lepetit, P. Fua (PAMI 2009)

Keypoint recognition using randomized trees V. Lepetit, P. Fua (PAMI 2006)


ClassificationClassification

DescriptorDescriptor

Outline

Previous worksRandomized treeFern classifiersSignatures

Proposed algorithmCompact signatures

Experimental results

Conclusion


Problem statement

Classification (or Recognition)Assumption

• The number of class = N• Appearance of an image patch = p

• Obtain new patches from the input image


What is class of p?What is class of p?

Problem statement

Classification (or Recognition)Conventional approaches

• Compute all possible distance of p • Select NN class to the solution.


∑ ( - )2 = 1

∑ ( - )2 =10

∑ ( - )2 =11

=

Nerest neighbor

Problem statement

Classification (or Recognition)Advanced approaches (SIFT, …)

• Compute descriptor and match…• View (rotation, scale, affine) and illumination changes.


∑ ( - )2 =?

SIFTSURF …Bottleneck!!Bottleneck!!

!!

When we have to make descriptors for many input patches, bottleneck problem is occurred.

Randomized tree


Randomized tree

A simple classifier fRandomly select pixel position a and b in the image

patchSimple binary test

• The tests compare the intensities of two pixels around the keypoint:

I(*) : Intensity of pixel position *

Invariant to light change by any raising function


a

b

Randomized tree

Initialization of RT with classifier fDefine depth of tree d and establish binary tree…Each leaf has a N dimensional vector, which denotes the

probability.


f0

f1 f2

f3 f4 f5 f6

depth 1

leaf

1

1 0

0

01

depth 2

depth 3

NNumber of leaves = 2d , where d = total depth

Randomized tree

Training RTGenerate patches to cover image variations (scale, rotation, affine transform, …)


Randomized tree

Training RT Implement training for all generating patches…Update probabilities of leaves…


f0

f1 f2

depth 1

leaf

1

1 0

0

01

depth 2

N

Randomized tree

Training RT Implement training for all generating patches…Update probabilities of leaves…


f0

f1 f2

depth 1

leaf

1

1 0

0

01

depth 2

N N

Randomized tree

Classification with trained RT Implement classification for input patches…Confirm probability when a patch reaches a leaf…


f0

f1 f2

depth 1

leaf

1

1 0

0

01

depth 2

N N

Randomized tree

Random forestMultiple RTs (or RF) are used for robustness .


Randomized tree

Random forest Final probability is summed value of probability of each RT.


+

Final probability

Randomized tree

Pros.Easily handle multi-class problems.Easily cover large perspective and scale variations.Classifier training is time consuming, but recognition is

very fast and robust.

Cons.Memory requirement is high.


Fern classifier


Fern classifier

Randomized tree


f0

f1 f2

f3 f4 f5 f6

1

1 0

0

01

depth 1

depth 2

depth 3

Fern classifier

Modified randomized tree In same depth, same classifier f is used


f0

f1 f1

f2 f2 f2 f2

1

1 0

0

01

depth 1

depth 2

depth 3

Fern classifier

Fern classifier (Randomized list)Modified RT and RL(Fern) are identical…


f0

f1

f2

depth 1

depth 2

depth 3

N classes

2d

Fern classifier

Fern classifier (Randomized list)


Tree List

Fern classifier

Multiple fern classifier (training) Initialize each fern classifier with depth d and class N


Multiple fern classifier (training)Update probabilities of fern classifiers for all reference

image…

Fern classifier


0

1

1

1

1

1

0

0

1

6

4

7

Fern classifier

Multiple fern classifier (training)Establish trained fern classifiers…


Multiple fern classifier (Recognition) Just apply new patch image and obtain final probability.

Fern classifier


+

+

Fern classifier

Pros.Same as pros. of randomized tree.A small number of f classifiers are used.Fast, easy to implement relative to RT.

Cons.Still takes a lot of memory…


Signature


Signature

AssumptionDefinition

• Classifier C with patch p


Signature

AssumptionDefinition

• If the classifier C has been trained well, then result of classifier C for the deformed patch is also same as original one.


Signature

Assumption Input patch q is not member of class.Definition of signature …

• Signature of q is the result of classifier C.


Signature

Assumption If input patch q’ is same member of q, then signatures of

q and q’ are almost same...

C() can make the signature of q.Signature of q (= C(q)) can be descriptor.


Signature

Sparse signature q is not a member of K. Response of C(q) can not be an exact probability of S(q). Change the signature value by using user defined threshold.

Fern classifiers is used for descriptor maker (=SIFT). A result of sparse signature(q) = descriptor of q.


Compact Signature


Compact Signature

PurposeThere is no loss of good matching rate while reduced

memory size of classifiers!


FF11 FF22 FFJJ

+

+

2d

N

th()

Signature generation

Compact Signature

Approach #1 (Dimension reduction)To reduce the size of memory…Random Ortho-Projection(ROP) matrix

Definition of compact signature• A ROP is applied to all probabilities…


FF11

2d

N

FF11

M

(N >>M)

Compact Signature

Approach #1 (Dimension reduction)No loss of good matching rate, while reduced memory

size of classifiers! (N >>M)


FF11 FF22 FFJJ

+

+

2d

M

Compact Signature generation

Compact Signature

Approach #2 (Quantization) The computation can be further streamlined by quantizing the

compressed leaf vectors. Using 1 byte per element instead of 4 bytes required by floats.


FF11

2d

M

Pfloat

Pbyte

Quantization

Compact Signature Total complexity of compact signature

classifier (or descriptor maker)


50 times less memory


Comparison4 image database

• Wall, light, jpg and fountain • http://www.robots.ox.ac.uk/~vgg/research/affine

Descriptors• SURF-64 (Speed Up Robust Feature)• Sparse signatures• Compact signatures-176 • Compact signatures-88


Class N = 500

http://www.robots.ox.ac.uk/~vgg/research/affine


Matching performance


m|n

test image

reference image


CPU time and Memory Consumption


Conclusion

ContributionA Descriptor maker is proposed by using novel classifier

which is well-known Fern-classifier.• Descriptors can be comuted rapidly.

Classifier size is extremly reduced • Almost 50 times less memory than sparse ones…• by using dimesion reduction (ROP) and simple

quantization technique

DiscussionWell trained classifier is difficult to obtain in in normal

situation.


Q & A


2009. 8. 21 (fri) young ki baik computer vision lab

Documents