cs376 computer vision lecture 3: linear filtershuangqx/cs376_lecture_3.pdf · 2019. 1. 30. ·...
TRANSCRIPT
![Page 1: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/1.jpg)
CS376 Computer VisionLecture 3: Linear Filters
Qixing Huang
January 30th 2019
![Page 2: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/2.jpg)
Last Lecture
• Image Noise
• Image Filtering
– Denoising
![Page 3: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/3.jpg)
This Lecture
• Edge Detection
• Template Matching
![Page 4: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/4.jpg)
Edge detection
• Goal: map image from 2d array of pixels to a set of curves or line segments or contours.
Source: https://en.wikipedia.org/wiki/Edge_detection#/media/File:%C3%84%C3%A4retuvastuse_n%C3%A4ide.png
Edge detection is a fundamental tool in image processing, machine vision and computer vision, particularly in the areas of feature detection and feature extraction.
![Page 5: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/5.jpg)
Edge detection
Besides the textbook, I also recommend Wikipedia pages (Up to date updates and fairly accurate)
![Page 6: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/6.jpg)
What causes an edge?
discontinuities in depth
discontinuities in surface orientation
variations in scene illumination/Shadow
changes in material properties and
Slide credit:
Kristen GraumanMore from https://en.wikipedia.org/wiki/Edge_detection
![Page 7: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/7.jpg)
Edges/gradients and invariance
Slide credit:
Kristen Grauman
![Page 8: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/8.jpg)
Derivatives and edges
imageintensity function
(along horizontal scanline) first derivative
edges correspond to
extrema of derivative
Source: L. Lazebnik
An edge is a place of rapid change in the
image intensity function.
![Page 9: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/9.jpg)
Partial derivatives of an image
Which shows changes with respect to x?
-1
1
1
-1or
?-1 1
x
yxf
),(
y
yxf
),(
(showing filters for correlation)Slide credit:
Kristen Grauman
![Page 10: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/10.jpg)
Image gradientThe gradient of an image:
The gradient points in the direction of most rapid change in intensity
The gradient direction (orientation of edge normal) is given by:
The edge strength is given by the gradient magnitude
Slide credit Steve Seitz
![Page 11: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/11.jpg)
Effects of noiseConsider a single row or column of the image
– Plotting intensity as a function of position gives a signal
Where is the edge?
Slide credit Steve Seitz
![Page 12: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/12.jpg)
Where is the edge?
Solution: smooth first
Look for peaks in
![Page 13: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/13.jpg)
Derivative theorem of convolution
Differentiation property of convolution.
Slide credit Steve Seitz
![Page 14: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/14.jpg)
11 − 0.0030 0.0133 0.0219 0.0133 0.0030
0.0133 0.0596 0.0983 0.0596 0.0133
0.0219 0.0983 0.1621 0.0983 0.0219
0.0133 0.0596 0.0983 0.0596 0.0133
0.0030 0.0133 0.0219 0.0133 0.0030
)()( hgIhgI =
Derivative of Gaussian filters
![Page 15: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/15.jpg)
Derivative of Gaussian filters
x-direction y-direction
Source: L. Lazebnik
![Page 16: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/16.jpg)
Laplacian of GaussianConsider
Laplacian of Gaussian
operator
Where is the edge? Zero-crossings of bottom graphSlide credit: Steve Seitz
![Page 17: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/17.jpg)
2D edge detection filters
• is the Laplacian operator:
Laplacian of Gaussian
Gaussian derivative of Gaussian
Slide credit: Steve Seitz
![Page 18: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/18.jpg)
Smoothing with a Gaussian
Recall: parameter σ is the “scale” / “width” / “spread” of the
Gaussian kernel, and controls the amount of smoothing.
…
Slide credit:
Kristen Grauman
![Page 19: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/19.jpg)
Effect of σ on derivatives
The apparent structures differ depending on
Gaussian’s scale parameter.
Larger values: larger scale edges detected
Smaller values: finer features detected
σ = 1 pixel σ = 3 pixels
Slide credit:
Kristen Grauman
![Page 20: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/20.jpg)
Natural distribution of image gradients
• An effective prior (heavy tailed) in image restoration, e.g., Image deblurring
Blurry input Ground truth Without prior With prior [Shan et al. 08]
![Page 21: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/21.jpg)
Seam Carving
![Page 22: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/22.jpg)
Seam carving: main idea
[Shai & Avidan, SIGGRAPH 2007]
![Page 23: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/23.jpg)
Content-aware resizing
Traditional resizing
Seam carving: main idea
[Shai & Avidan, SIGGRAPH 2007]
![Page 24: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/24.jpg)
Seam carving: main idea
![Page 25: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/25.jpg)
Real image example
Slide credit:
Kristen Grauman
![Page 26: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/26.jpg)
Content-aware resizing
Seam carving: main idea
Intuition:
• Preserve the most “interesting” content
→ Prefer to remove pixels with low gradient energy
• To reduce or increase size in one dimension, remove irregularly shaped “seams”
→ Optimal solution via dynamic programming.
Slide credit:
Kristen Grauman
![Page 27: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/27.jpg)
• Want to remove seams where they won’t be very noticeable:– Measure “energy” as gradient magnitude
• Choose seam based on minimum total energy path across image, subject to 8-connectedness.
Seam carving: main idea
=)( fEnergy
Slide credit:
Kristen Grauman
![Page 28: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/28.jpg)
Let a vertical seam s consist of h positions that form an 8-connected path.
Let the cost of a seam be:
Optimal seam minimizes this cost:
Compute it efficiently with dynamic programming.
Seam carving: algorithm
=
=h
i
isfEnergyCost1
))(()(s
)(min* sss
Cost=
s1
s2
s3
s4
s5
=)( fEnergy
Slide credit:
Kristen Grauman
![Page 29: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/29.jpg)
How to identify the minimum cost seam?
• First, consider a greedy approach:
625982031
Energy matrix(gradient magnitude)
Slide credit:
Kristen Grauman
![Page 30: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/30.jpg)
row i-1
Seam carving: algorithm
• Compute the cumulative minimum energy for all possible connected seams at each entry (i,j):
• Then, min value in last row of M indicates end of the minimal connected vertical seam.
• Backtrack up from there, selecting min of 3 above in M.
( ))1,1(),,1(),1,1(min),(),( +−−−−+= jijijijiEnergyji MMMM
j-1
jrow i
M matrix: cumulative min energy
(for vertical seams)
Energy matrix(gradient magnitude)
j j+1
Slide credit:
Kristen Grauman
![Page 31: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/31.jpg)
Example
625982031
Energy matrix(gradient magnitude)
M matrix(for vertical seams)
1458
983
031
( ))1,1(),,1(),1,1(min),(),( +−−−−+= jijijijiEnergyji MMMM
Slide credit:
Kristen Grauman
![Page 32: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/32.jpg)
Example
625982031
Energy matrix(gradient magnitude)
M matrix(for vertical seams)
1458
983
031
( ))1,1(),,1(),1,1(min),(),( +−−−−+= jijijijiEnergyji MMMM
Slide credit:
Kristen Grauman
![Page 33: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/33.jpg)
Real image example
Original Image Energy Map
Blue = low energy
Red = high energySlide credit:
Kristen Grauman
![Page 34: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/34.jpg)
Real image example
Slide credit:
Kristen Grauman
![Page 35: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/35.jpg)
Seam Carving
![Page 36: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/36.jpg)
Extensions of Seam Carving
[Rubinstein et al. 08]
![Page 37: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/37.jpg)
Image Deformation
[Wang et al. 08]
![Page 38: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/38.jpg)
Edge Detection
![Page 39: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/39.jpg)
Gradients -> edges
Primary edge detection steps:
1. Smoothing: suppress noise
2. Edge enhancement: filter for contrast
3. Edge localization
Determine which local maxima from filter output are actually edges vs. noise
• Threshold, Thin
Slide Credit: Kristen Grauman
![Page 40: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/40.jpg)
Original image
Slide Credit: Kristen Grauman
![Page 41: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/41.jpg)
Gradient magnitude image
Slide Credit: Kristen Grauman
![Page 42: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/42.jpg)
Thresholding gradient with a lower threshold
Slide Credit: Kristen Grauman
![Page 43: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/43.jpg)
Thresholding gradient with a higher threshold
Slide Credit: Kristen Grauman
![Page 44: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/44.jpg)
Canny Edge Detector
• Filter image with derivative of Gaussian and find magnitude and orientation of gradient
• Non-maximum suppression
– Thin wide “ridges” down to single pixel with
• Linking and thresholding (hysteresis):
– Define two thresholds: low and high
– Use the high threshold to start edge curves and the low threshold to continue them
Slide Credit: Kristen Grauman
![Page 45: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/45.jpg)
The Canny edge detector
original image (Lena)
Slide credit: Steve Seitz
![Page 46: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/46.jpg)
The Canny edge detector
norm of the gradient
![Page 47: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/47.jpg)
The Canny edge detector
thresholding
![Page 48: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/48.jpg)
The Canny edge detector
thresholding
How to turn
these thick
regions of the
gradient into
curves?
![Page 49: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/49.jpg)
Non-maximum suppression
Check if pixel is local maximum along gradient direction, select single max across width of the edge– requires checking interpolated pixels p and r
![Page 50: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/50.jpg)
The Canny edge detector
thinning
(non-maximum suppression)
Problem:
pixels along
this edge
didn’t
survive the
thresholding
![Page 51: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/51.jpg)
Credit: James Hays
![Page 52: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/52.jpg)
Hysteresis thresholding
• Use a high threshold to start edge curves, and a low threshold to continue them.
Source: Steve Seitz
![Page 53: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/53.jpg)
Credit: James Hays
![Page 54: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/54.jpg)
Recap: Canny Edge Detector
• A typical example of successful hand-crafted feature design– Derivatives as a starting point
– Non-maximum suppressing to handle fuzzy edges
– Linking and thresholding (hysteresis)
– Each (simple) heuristic addresses one importantissue
– Has been applied in other settings
– How to improve Canny Edge Detector?
![Page 55: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/55.jpg)
Edge detection on 3D shapes
[Pauly et al. 03]
![Page 56: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/56.jpg)
How to Evaluate is Very Importantin Computer Vision Research
Labeled data
Evaluation Protocol
![Page 57: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/57.jpg)
Background Texture Shadows
Low-level edges vs. perceived contours
Slide credit: Kristen Graumann
![Page 58: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/58.jpg)
Low-level edges vs. perceived contours
• Berkeley segmentation database:http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/
image human segmentation gradient magnitude
Source: L. Lazebnik
![Page 59: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/59.jpg)
Credit: David MartinBerkeley Segmentation Data SetDavid Martin, Charless Fowlkes, Doron Tal, Jitendra Malik
![Page 60: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/60.jpg)
![Page 61: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/61.jpg)
![Page 62: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/62.jpg)
[D. Martin et al. PAMI 2004] Human-marked segment boundaries
Learn from humans which combination of features is most indicative of a “good” contour?
![Page 63: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/63.jpg)
![Page 64: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/64.jpg)
What features are responsible for perceived edges?
![Page 65: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/65.jpg)
[D. Martin et al. PAMI 2004]
What features are responsible for perceived edges?
Feature profiles (oriented energy, brightness, color, and texture gradients) along the patch’s horizontal diameter
Slide Credit: Kristen Grauman
![Page 66: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/66.jpg)
[D. Martin et al. PAMI 2004]
What features are responsible for perceived edges?
Feature profiles (oriented energy, brightness, color, and texture gradients) along the patch’s horizontal diameter
Kristen Grauman, UT-Austin
![Page 67: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/67.jpg)
Credit: David Martin
![Page 68: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/68.jpg)
[D. Martin et al. PAMI 2004] Kristen Grauman, UT-Austin
![Page 69: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/69.jpg)
Contour Detection
Source: Jitendra Malik:
http://www.cs.berkeley.edu/~malik/malik-talks-ptrs.html
Prewitt, Sobel, Roberts
Canny
Canny+optthresholds
Learned with combined features
Human agreement
![Page 70: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/70.jpg)
Template Matching
![Page 71: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/71.jpg)
Another Application: Template Matching
• A building block of neural networks is called a filter
– Map raw pixels to an intermediate (feature) representation
– Neural networks utilize filters in a hierarchical manner
Image credit: Yann Lecun
![Page 72: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/72.jpg)
Template matching
• Filters as templates
• Note that filters look like the effects they are intended to find--- “matched filters”
• Use normalized cross-correlation score to find a given pattern(template) in the image
• Normalization needed to control for relative brightnesses
![Page 73: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/73.jpg)
Template matching
A toy example
![Page 74: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/74.jpg)
Template matching
Detected template
![Page 75: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/75.jpg)
Template matching
Correlation map
Peak
Detected template
![Page 76: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/76.jpg)
Where’s Lion Messi
Scene
Template
![Page 77: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/77.jpg)
Where’s Lion Messi
Correlation Map
Template
Try multiple scales
![Page 78: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/78.jpg)
Where’s Lion Messi
Scene
Template
![Page 79: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/79.jpg)
Template matching
What if the template is not identical to
some subimage in the scene?
Scene
Template
Slide credit:
Kristen Grauman
![Page 80: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/80.jpg)
Template matching
Detected template
Template
Match can be meaningful, if scale, orientation,
and general appearance is right.
Slide credit:
Kristen Grauman
![Page 81: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/81.jpg)
How about human?
• Deformable part model (deforming template) [Felzenszwalb et al. 10]
• Multilayer-neural network [He et al. 16]
– The deformation in eachlayer is close to identity
![Page 82: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/82.jpg)
Recap: Linear Filters
• Smoothing– Values positive
– Sum to 1 → constant regions same as input
– Amount of smoothing proportional to mask size
– Remove “high-frequency” components; “low-pass” filter
• Derivatives– Opposite signs used to get high response in regions of high contrast
– Sum to 0 → no response in constant regions
– High absolute value at points of high contrast
• Filters act as templates– Highest response for regions that “look the most like the filter”
– Dot product as correlation
![Page 83: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/83.jpg)
Summary
• Image gradients
• Seam carving -> gradients as “energy”
• Gradients -> edges and contours
• Template matching– Image patch as a filter
![Page 84: CS376 Computer Vision Lecture 3: Linear Filtershuangqx/CS376_Lecture_3.pdf · 2019. 1. 30. · CS376 Computer Vision Lecture 3: Linear Filters Qixing Huang January 30th 2019. Last](https://reader035.vdocuments.net/reader035/viewer/2022081410/609f617bca84e0063226b130/html5/thumbnails/84.jpg)
Coming up
• A1 out tonight (11.59 pm), due in 2 weeks