![Page 1: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/1.jpg)
Detecting corners
16-385 Computer VisionSpring 2020, Lecture 5http://www.cs.cmu.edu/~16385/
![Page 2: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/2.jpg)
Course announcements
• Homework 1 posted on course website.- Due on February 5th at 23:59.- This homework is in Matlab.- How many of you have looked at/started/finished homework 1?
• First theory quiz on course website and due on February 3rd, at 23:59.
• From here on, all office hours will be at Smith Hall 200.- Conference room next to the second floor restrooms.
![Page 3: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/3.jpg)
Leftover from Lecture 4:
• More on Hough lines.
• Hough circles.
New in lecture 5:
• Why detect corners?
• Visualizing quadratics.
• Harris corner detector.
• Multi-scale detection.
• Multi-scale blob detection.
Overview of today’s lecture
![Page 4: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/4.jpg)
Slide credits
Most of these slides were adapted from:
• Kris Kitani (15-463, Fall 2016).
Some slides were inspired or taken from:
• Fredo Durand (MIT).• James Hays (Georgia Tech).
![Page 5: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/5.jpg)
Why detect corners?
![Page 6: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/6.jpg)
Why detect corners?
Image alignment (homography, fundamental matrix)
3D reconstruction
Motion tracking
Object recognition
Indexing and database retrieval
Robot navigation
![Page 7: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/7.jpg)
Planar object instance recognition
Database of planar objects Instance recognition
![Page 8: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/8.jpg)
3D object recognitionDatabase of 3D objects 3D objects recognition
![Page 9: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/9.jpg)
Recognition under occlusion
![Page 10: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/10.jpg)
Location Recognition
![Page 11: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/11.jpg)
Robot Localization
![Page 12: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/12.jpg)
Image matching
![Page 13: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/13.jpg)
NASA Mars Rover images
Where are the corresponding points?
![Page 14: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/14.jpg)
![Page 15: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/15.jpg)
Pick a point in the image.
Find it again in the next image.
What type of feature would you select?
![Page 16: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/16.jpg)
Pick a point in the image.
Find it again in the next image.
What type of feature would you select?
![Page 17: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/17.jpg)
Pick a point in the image.
Find it again in the next image.
What type of feature would you select?
a corner
![Page 18: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/18.jpg)
Visualizing quadratics
![Page 19: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/19.jpg)
Equation of a circle
Equation of a ‘bowl’ (paraboloid)
If you slice the bowl at
what do you get?
![Page 20: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/20.jpg)
Equation of a circle
Equation of a ‘bowl’ (paraboloid)
If you slice the bowl at
what do you get?
![Page 21: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/21.jpg)
can be written in matrix form like this…
![Page 22: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/22.jpg)
‘sliced at 1’
![Page 23: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/23.jpg)
What happens if you increase
coefficient on x?
and slice at 1
![Page 24: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/24.jpg)
What happens if you increase
coefficient on x?
and slice at 1
decrease width in x!
![Page 25: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/25.jpg)
What happens if you increase
coefficient on y?
and slice at 1
![Page 26: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/26.jpg)
What happens if you increase
coefficient on y?
and slice at 1
decrease width in y
![Page 27: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/27.jpg)
can be written in matrix form like this…
What’s the shape?
What are the eigenvectors?
What are the eigenvalues?
![Page 28: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/28.jpg)
can be written in matrix form like this…
eigenvalues
along diagonaleigenvectors
Result of Singular Value Decomposition (SVD)
axis of the ‘ellipse
slice’
Inverse sqr of
length of the
quadratic along
the axis
![Page 29: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/29.jpg)
EigenvaluesEigenvectors
Eig
en
ve
cto
rs
Eigenvector
Eig
en
ve
cto
r
Inverse sqr of the size of the axis
![Page 30: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/30.jpg)
you can smash this bowl in the y direction
you can smash this bowl in the x direction
Recall:
![Page 31: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/31.jpg)
Eigenvalues
Eigenvectors Eigenvectors
Eigenvector
Eig
en
ve
cto
r
Inverse sqr of length of axis
![Page 32: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/32.jpg)
Eigenvalues
Eigenvectors Eigenvectors
![Page 33: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/33.jpg)
Eigenvalues
Eigenvectors Eigenvectors
![Page 34: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/34.jpg)
Error function for Harris Corners
The surface E(u,v) is locally approximated by a quadratic form
We will need this to understand the…
![Page 35: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/35.jpg)
Harris corner detector
![Page 36: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/36.jpg)
How do you find a corner?
![Page 37: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/37.jpg)
How do you find a corner?
Easily recognized by looking through a small window
Shifting the window should give large change in intensity
[Moravec 1980]
![Page 38: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/38.jpg)
“edge”:
no change along the edge
direction
“corner”:
significant change in all
directions
“flat” region:
no change in all
directions
Easily recognized by looking through a small window
Shifting the window should give large change in intensity
[Moravec 1980]
![Page 39: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/39.jpg)
Design a program to detect corners(hint: use image gradients)
![Page 40: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/40.jpg)
Finding corners(a.k.a. PCA)
1.Compute image gradients over
small region
2.Subtract mean from each image
gradient
3.Compute the covariance matrix
4.Compute eigenvectors and
eigenvalues
5.Use threshold on eigenvalues to
detect corners
![Page 41: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/41.jpg)
1. Compute image gradients over a small region(not just a single pixel)
![Page 42: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/42.jpg)
1. Compute image gradients over a small region(not just a single pixel)
array of x gradients
array of y gradients
![Page 43: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/43.jpg)
visualization of gradients
image
X derivative
Y derivative
![Page 44: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/44.jpg)
What does the distribution tell you about the region?
![Page 45: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/45.jpg)
distribution reveals edge orientation and magnitude
![Page 46: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/46.jpg)
How do you quantify orientation and magnitude?
![Page 47: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/47.jpg)
2. Subtract the mean from each image gradient
![Page 48: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/48.jpg)
2. Subtract the mean from each image gradient
plot intensities
constant intensity
gradient
intensities along the line
![Page 49: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/49.jpg)
2. Subtract the mean from each image gradient
plot intensities
constant intensity
gradient
intensities along the line
plot of image gradients
subtract mean
![Page 50: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/50.jpg)
2. Subtract the mean from each image gradient
plot intensities
constant intensity
gradient
intensities along the line
plot of image gradientsdata is centered
(‘DC’ offset is removed)
subtract mean
![Page 51: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/51.jpg)
3. Compute the covariance matrix
![Page 52: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/52.jpg)
3. Compute the covariance matrix
Where does this covariance matrix come from?
array of x gradients array of y gradients
.*=sum( )
![Page 53: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/53.jpg)
“edge”:
no change along the edge
direction
“corner”:
significant change in all
directions
“flat” region:
no change in all
directions
Easily recognized by looking through a small window
Shifting the window should give large change in intensity
[Moravec 1980]
![Page 54: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/54.jpg)
Error function
Change of intensity for the shift [u,v]:
IntensityShifted
intensity
Window
function
orWindow function w(x,y) =
Gaussian1 in window, 0 outside
Error
function
Some mathematical background…
![Page 55: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/55.jpg)
Error function approximation
Change of intensity for the shift [u,v]:
First-order Taylor expansion of I(x,y) about (0,0)
(bilinear approximation for small shifts)
![Page 56: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/56.jpg)
Bilinear approximation
For small shifts [u,v] we have a ‘bilinear approximation’:
where M is a 2×2 matrix computed from image derivatives:
Change in
appearance for a
shift [u,v]
M
‘second moment’ matrix
‘structure tensor’
![Page 57: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/57.jpg)
By computing the gradient covariance matrix…
we are fitting a quadratic to the gradients over a
small image region
![Page 58: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/58.jpg)
Visualization of a quadratic
The surface E(u,v) is locally approximated by a quadratic form
![Page 59: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/59.jpg)
Which error surface indicates a good image feature?
What kind of image patch do these surfaces represent?
![Page 60: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/60.jpg)
Which error surface indicates a good image feature?
flat
![Page 61: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/61.jpg)
Which error surface indicates a good image feature?
flat edge
‘line’
![Page 62: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/62.jpg)
flat edge
‘line’
corner
‘dot’
Which error surface indicates a good image feature?
![Page 63: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/63.jpg)
4. Compute eigenvalues and eigenvectors
![Page 64: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/64.jpg)
4. Compute eigenvalues and eigenvectors
eigenvector
eigenvalue
![Page 65: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/65.jpg)
1. Compute the determinant of(returns a polynomial)
eigenvector
eigenvalue
4. Compute eigenvalues and eigenvectors
![Page 66: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/66.jpg)
1. Compute the determinant of(returns a polynomial)
eigenvector
eigenvalue
2. Find the roots of polynomial(returns eigenvalues)
4. Compute eigenvalues and eigenvectors
![Page 67: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/67.jpg)
1. Compute the determinant of(returns a polynomial)
eigenvector
eigenvalue
2. Find the roots of polynomial(returns eigenvalues)
3. For each eigenvalue, solve(returns eigenvectors)
4. Compute eigenvalues and eigenvectors
![Page 68: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/68.jpg)
eig(M)
![Page 69: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/69.jpg)
Visualization as an ellipse
Since M is symmetric, we have
We can visualize M as an ellipse with axis lengths determined by the
eigenvalues and orientation determined by R
direction of the
slowest change
direction of the fastest
change
(lmax)-1/2
(lmin)-1/2
Ellipse equation:
![Page 70: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/70.jpg)
interpreting eigenvalues
l1
l2
l2 >> l1
l1 >> l2
What kind of image patch does
each region represent?
![Page 71: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/71.jpg)
interpreting eigenvalues
‘horizontal’
edge
‘vertical’
edge
flat
corner
l1
l2
l2 >> l1
l1 >> l2
l1 ~ l2
![Page 72: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/72.jpg)
interpreting eigenvalues
‘horizontal’
edge
‘vertical’
edge
flat
corner
l1
l2
l2 >> l1
l1 >> l2
l1 ~ l2
![Page 73: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/73.jpg)
interpreting eigenvalues
‘horizontal’
edge
‘vertical’
edge
flat
corner
l1
l2
l2 >> l1
l1 >> l2
l1 ~ l2
![Page 74: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/74.jpg)
5. Use threshold on eigenvalues to detect corners
![Page 75: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/75.jpg)
flat
l1
l2
5. Use threshold on eigenvalues to detect corners
Think of a function to
score ‘cornerness’
![Page 76: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/76.jpg)
flat
l1
l2
5. Use threshold on eigenvalues to detect corners
Think of a function to
score ‘cornerness’
strong corner
![Page 77: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/77.jpg)
flat
corner
l1
l2
5. Use threshold on eigenvalues to detect corners
(a function of )^
Use the smallest eigenvalue as
the response function
![Page 78: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/78.jpg)
flat
corner
l1
l2
5. Use threshold on eigenvalues to detect corners
(a function of )^
Eigenvalues need to be
bigger than one.
Can compute this more efficiently…
![Page 79: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/79.jpg)
flat
corner
l1
l2
R < 0 R > 0
R < 0
5. Use threshold on eigenvalues to detect corners
(a function of )^
![Page 80: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/80.jpg)
Harris & Stephens (1988)
Kanade & Tomasi (1994)
Nobel (1998)
![Page 81: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/81.jpg)
1. Compute x and y derivatives of image
2. Compute products of derivatives at every pixel
3. Compute the sums of the products of derivatives at
each pixel
Harris DetectorC.Harris and M.Stephens. “A Combined Corner and Edge Detector.”1988.
![Page 82: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/82.jpg)
Harris DetectorC.Harris and M.Stephens. “A Combined Corner and Edge Detector.”1988.
4. Define the matrix at each pixel
5. Compute the response of the detector at each pixel
6. Threshold on value of R; compute non-max suppression.
![Page 83: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/83.jpg)
Yet another option:
How do you write this equivalently
using determinant and trace?
![Page 84: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/84.jpg)
Yet another option:
![Page 85: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/85.jpg)
Harris criterion
Different criteria
![Page 86: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/86.jpg)
![Page 87: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/87.jpg)
Corner response
![Page 88: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/88.jpg)
![Page 89: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/89.jpg)
Thresholded corner response
![Page 90: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/90.jpg)
Non-maximal suppression
![Page 91: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/91.jpg)
![Page 92: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/92.jpg)
Harris corner response is
invariant to rotation
Ellipse rotates but its shape
(eigenvalues) remains the same
Corner response R is invariant to image rotation
![Page 93: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/93.jpg)
Partial invariance to affine intensity change
Only derivatives are used => invariance to intensity
shift I → I + b
Intensity scale: I → a I
R
x (image coordinate)
threshold
R
x (image coordinate)
Harris corner response is
invariant to intensity changes
![Page 94: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/94.jpg)
The Harris detector is not invariant to changes in …
![Page 95: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/95.jpg)
The Harris corner detector is not
invariant to scale
edge!corner!
![Page 96: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/96.jpg)
Multi-scale detection
![Page 97: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/97.jpg)
How can we make a feature detector scale-invariant?
![Page 98: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/98.jpg)
How can we automatically select the scale?
![Page 99: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/99.jpg)
Multi-scale blob detection
![Page 100: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/100.jpg)
![Page 101: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/101.jpg)
Find local maxima in both position and scale
f
region size
Image 1f
region size
Image 2
s1 s2
Intuitively…
![Page 102: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/102.jpg)
Highest response when the signal has the
same characteristic scale as the filter
Laplacian filter
Formally…
![Page 103: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/103.jpg)
characteristic scale
characteristic scale - the scale that
produces peak filter response
we need to search over characteristic scales
![Page 104: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/104.jpg)
Full size 3/4 size
What happens if you apply different Laplacian filters?
![Page 105: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/105.jpg)
jet color scale
blue: low, red: high
![Page 106: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/106.jpg)
![Page 107: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/107.jpg)
![Page 108: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/108.jpg)
![Page 109: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/109.jpg)
![Page 110: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/110.jpg)
![Page 111: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/111.jpg)
Full size 3/4 size
What happened when you applied different Laplacian filters?
![Page 112: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/112.jpg)
![Page 113: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/113.jpg)
![Page 114: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/114.jpg)
peak!
![Page 115: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/115.jpg)
peak!
![Page 116: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/116.jpg)
![Page 117: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/117.jpg)
![Page 118: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/118.jpg)
Full size 3/4 size
What happened when you applied different Laplacian filters?
![Page 119: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/119.jpg)
2.1 4.2 6.0
9.8 15.5 17.0
peak!
![Page 120: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/120.jpg)
2.1 4.2 6.0
9.8 15.5 17.0
maximum response
![Page 121: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/121.jpg)
optimal scale2.1 4.2 6.0 9.8 15.5 17.0
Full size image
2.1 4.2 6.0 9.8 15.5 17.0
3/4 size image
![Page 122: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/122.jpg)
optimal scale2.1 4.2 6.0 9.8 15.5 17.0
Full size image
2.1 4.2 6.0 9.8 15.5 17.0
3/4 size image
maximum
response
maximum
response
![Page 123: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/123.jpg)
cross-scale maximum
local maximum
local maximum
local maximum
4.2
6.0
9.8
![Page 124: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/124.jpg)
How would you implement
scale selection?
![Page 125: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/125.jpg)
implementation
For each level of the Gaussian pyramid
compute feature response (e.g. Harris, Laplacian)
For each level of the Gaussian pyramid
if local maximum and cross-scale
save scale and location of feature
![Page 126: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/126.jpg)
![Page 127: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/127.jpg)
![Page 128: Detecting corners16385/lectures/lecture5.pdf · Finding corners (a.k.a. PCA) 1.Compute image gradients over small region 2.Subtract mean from each image gradient 3.Compute the covariance](https://reader036.vdocuments.net/reader036/viewer/2022071016/5fcf8104ade15a7a6c2a6a86/html5/thumbnails/128.jpg)
Basic reading:• Szeliski textbook, Sections 4.1.
References