![Page 1: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/1.jpg)
Interest Point Detection
Lecture-4
![Page 2: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/2.jpg)
Local features: main components1) Detection: Identify the
interest points
2) Description :Extract feature vector descriptor surrounding each interest point.
3) Matching: Determine correspondence between descriptors in two views
],,[ )1()1(11 dxx x
],,[ )2()2(12 dxx x
Kristen Grauman
![Page 3: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/3.jpg)
Where can we use it? Automate object tracking Point matching for computing disparity Stereo calibration
Estimation of fundamental matrix Motion based segmentation Recognition 3D object reconstruction Robot navigation Image retrieval and indexing
![Page 4: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/4.jpg)
Goal: interest operator repeatability
• We want to detect (at least some of) the same points in both images.
• Yet we have to be able to run the detection procedure independently per image.
No chance to find true matches!
Kristen Grauman
![Page 5: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/5.jpg)
Goal: descriptor distinctiveness
• We want to be able to reliably determine which point goes with which.
• Must provide some invariance to geometricand photometric differences between the two views.
?
Kristen Grauman
![Page 6: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/6.jpg)
Some patches can be localizedor matched with higher accuracy than
others.
![Page 7: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/7.jpg)
Local features: main components1) Detection: Identify the
interest points
2) Description:Extract vector feature descriptor surrounding each interest point.
3) Matching: Determine correspondence between descriptors in two views
Kristen Grauman
![Page 8: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/8.jpg)
What is an interest point Expressive texture
The point at which the direction of the boundary of object changes abruptly
Intersection point between two or more edge segments
![Page 9: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/9.jpg)
Synthetic & Real Interest Points
Corners are indicated in red
![Page 10: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/10.jpg)
Properties of Interest Point Detectors Detect all (or most) true interest points No false interest points Well localized. Robust with respect to noise. Efficient detection
![Page 11: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/11.jpg)
Possible Approaches to Corner Detection Based on brightness of images
Usually image derivatives Based on boundary extraction
First step edge detection Curvature analysis of edges
![Page 12: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/12.jpg)
Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give
a large change in intensity
C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988
![Page 13: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/13.jpg)
Basic Idea
“flat” region:no change in all directions
“edge”:no change along the edge direction
“corner”:significant change in all directions
![Page 14: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/14.jpg)
Aperture Problem
![Page 15: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/15.jpg)
Correlation
k lljkihlkfhf ,,
KernelImage
hf
h1 h2 h3
h4 h5 h6
h7 h8 h9
hf1 f2 f3
f4 f5 f6
f7 f8 f9
998877
665544
332211
*
hfhfhfhfhfhf
hfhfhfhf
f
![Page 16: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/16.jpg)
Correlation
k lljkihlkfhf ,, Cross correlation
k l
ljkiflkfff ,, Auto correlation
![Page 17: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/17.jpg)
Correlation Vs SSD 2,,
k l
ljkihlkfSSD Sum of Squares Difference
k l
ljkiflkfff ,,
k l
ljkihlkfljkihlkfSSD 22 ,,,2,
minimize
minimize
maximize
k l
lkfljkihSSD ,,2
k l
lkfljkihnCorrelatio ,,maximize
k l
lkfljkihSSD ,,2
![Page 18: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/18.jpg)
Mathematics of Harris Detector Change of intensity for the shift (u,v)
yx
yxIvyuxIyxwvuE,
2
intensityintensity shiftedfunctionwindow
]),(),([),(),(
Window functions
Auto-correlation
UNIFORM GAUSSIAN
![Page 19: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/19.jpg)
Auto-Correlation
![Page 20: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/20.jpg)
Taylor Series
ƒ(x) Can be represented at point a in terms of its derivatives
![Page 21: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/21.jpg)
Mathematics of Harris Detector
yx
yx vIuIyxwvuE,
2])[,(),(
yx y
x
II
vuyxwvuE,
2
),(),(
yxyx
y
x
vu
IIII
vuyxwvuE,
),(),(
v
uII
II
yxwvuvuEyx
yxy
x
,),(),(
yx
yxIvyuxIyxwvuE,
2
intensityintensity shiftedfunctionwindow
]),(),([),(),(
yx
yx yxIvIuIyxIyxwvuE,
2
intensityintensity shiftedfunction window
]),(),([),(),(
Taylor Series
vu
MvuvuE ),(
![Page 22: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/22.jpg)
Mathematics of Harris Detector
E(u,v) is an equation of an ellipse, where M is the covariance
Let 1 and 2 be eigenvalues of M
vu
MvuvuE ),(
yx yyyx
yxxx
IIIIIIII
yxwM,
),(
direction of the slowest change
direction of the fastest change
(max)-1/2
(min)-1/2
![Page 23: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/23.jpg)
Eigen Vectors and Eigen ValuesThe eigen vector, x, of a matrix A is a special vector, with the following property
Where is called eigen value
To find eigen values of a matrix A first find the roots of:
Then solve the following linear system for each eigen value to find corresponding eigen vector
![Page 24: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/24.jpg)
Example
Eigen Values
Eigen Vectors
![Page 25: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/25.jpg)
Eigen Values
![Page 26: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/26.jpg)
Eigen Vectors
![Page 27: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/27.jpg)
Mathematics of Harris Detector
1
2
“Corner”1 and 2 are large,1 ~ 2;E increases in all directions
1 and 2 are small;E is almost constant in all directions
“Edge” 1 >> 2
“Edge” 2 >> 1
“Flat” region
Classification of image points using eigenvalues of M:
![Page 28: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/28.jpg)
Mathematics of Harris Detector Measure of cornerness in terms of 1, 2
2det traceMkMR 22121 kR
2
1
00
M
![Page 29: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/29.jpg)
Mathematics of Harris Detector
1
2 “Corner”
“Edge”
“Edge”
“Flat”
• R depends only on eigenvalues of M
• R is large for a corner
• R is negative with large magnitude for an edge
• |R| is small for a flatregion
R > 0
R < 0
R < 0|R| small
![Page 30: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/30.jpg)
![Page 31: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/31.jpg)
Compute corner response
![Page 32: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/32.jpg)
Find points with large corner response: R> thresholdFind points with large corner response: R> threshold
![Page 33: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/33.jpg)
Take only the points of local maxima of R
If pixel value is greater than its neighbors then it is a local maxima.
![Page 34: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/34.jpg)
![Page 35: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/35.jpg)
Other Version of Harris Detectors
21 R
21
21
1)()det(
Mtrace
MR
Triggs
Szeliski (Harmonic mean)
1R Shi-Tomasi
![Page 36: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/36.jpg)
1
2
![Page 37: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/37.jpg)
Algorithm
![Page 38: Interest Point Detection - WordPress.com · Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give a large change in intensity](https://reader034.vdocuments.net/reader034/viewer/2022050420/5f8f9214a130141da1220154/html5/thumbnails/38.jpg)
Reading Material Section 4.1.1 Feature Detectors
Richard Szeliski, "Computer Vision: Algorithms and Application“, Springer.