cs 1699: intro to computer vision edges and binary images prof. adriana kovashka university of...
TRANSCRIPT
![Page 1: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/1.jpg)
CS 1699: Intro to Computer Vision
Edges and Binary Images
Prof. Adriana KovashkaUniversity of Pittsburgh
September 15, 2015
![Page 2: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/2.jpg)
Plan for today
• Edge detection• Binary image analysis
![Page 3: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/3.jpg)
Homework 1
• Due on 9/22, 11:59pm
• Review slides from 9/03 regarding how to do Part IV
• There are different ways to compute image gradients
• Office hours for 9/17 will be at 11am-12pm on 9/18
![Page 4: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/4.jpg)
Edge detection
• Goal: map image from 2d array of pixels to a set of curves or line segments or contours.
• Why?
• Main idea: look for strong gradients, post-process
Figure from J. Shotton et al., PAMI 2007
Source: K. Grauman
![Page 5: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/5.jpg)
Origin of Edges
Edges are caused by a variety of factors
depth discontinuity
surface color discontinuity
illumination discontinuity
surface normal discontinuity
Source: S. Seitz
![Page 6: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/6.jpg)
What causes an edge?
Depth discontinuity: object boundary
Change in surface orientation: shape
Cast shadows
Reflectance change: appearance information, texture
Source: K. Grauman
![Page 7: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/7.jpg)
Edges/gradients and invariance
Source: K. Grauman
![Page 8: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/8.jpg)
Closeup of edges
Source: D. Hoiem
![Page 9: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/9.jpg)
Closeup of edges
Source: D. Hoiem
![Page 10: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/10.jpg)
Characterizing edges• An edge is a place of rapid change in the image
intensity function
imageintensity function
(along horizontal scanline) first derivative
edges correspond toextrema of derivative
Source: L. Lazebnik
![Page 11: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/11.jpg)
Intensity profile Intensity
Gradient
Source: D. Hoiem
![Page 12: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/12.jpg)
With a little Gaussian noise
Gradient
Source: D. Hoiem
![Page 13: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/13.jpg)
Effects of noise• Consider a single row or column of the image
– Plotting intensity as a function of position gives a signal
Where is the edge?Source: S. Seitz
![Page 14: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/14.jpg)
Effects of noise• Difference filters respond strongly to noise
– Image noise results in pixels that look very different from their neighbors
– Generally, the larger the noise the stronger the response• What can we do about it?
Source: D. Forsyth
![Page 15: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/15.jpg)
Solution: smooth first
• To find edges, look for peaks in )( gfdx
d
f
g
f * g
)( gfdx
d
Source: S. Seitz
![Page 16: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/16.jpg)
• Differentiation is convolution, and convolution is associative:
• This saves us one operation:
gdx
dfgf
dx
d )(
Derivative theorem of convolution
gdx
df
f
gdx
d
Source: S. Seitz
![Page 17: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/17.jpg)
Derivative of Gaussian filter
• Is this filter separable?
* [1 0 -1] =
Source: D. Hoiem
![Page 18: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/18.jpg)
• Smoothed derivative removes noise, but blurs edge. Also finds edges at different “scales”.
1 pixel 3 pixels 7 pixels
Tradeoff between smoothing and localization
Source: D. Forsyth
![Page 19: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/19.jpg)
Designing an edge detector
• Criteria for a good edge detector:– Good detection: find all real edges, ignoring noise or other
artifacts– Good localization
• detect edges as close as possible to the true edges• return one point only for each true edge point
• Cues of edge detection– Differences in color, intensity, or texture across the boundary– Continuity and closure– High-level knowledge
Source: L. Fei-Fei
![Page 20: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/20.jpg)
Gradients -> edges
Primary edge detection steps:
1. Smoothing: suppress noise
2. Edge enhancement: filter for contrast
3. Edge localization
Determine which local maxima from filter output are actually edges vs. noise
• Threshold, Thin
Source: K. Grauman
![Page 21: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/21.jpg)
Thresholding
• Choose a threshold value t• Set any pixels less than t to zero (off)• Set any pixels greater than or equal to t to one
(on)
Source: K. Grauman
![Page 22: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/22.jpg)
Original image
Source: K. Grauman
![Page 23: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/23.jpg)
Gradient magnitude image
Source: K. Grauman
![Page 24: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/24.jpg)
Thresholding gradient with a lower threshold
Source: K. Grauman
![Page 25: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/25.jpg)
Thresholding gradient with a higher threshold
Source: K. Grauman
![Page 26: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/26.jpg)
Canny edge detector• This is probably the most widely used edge
detector in computer vision• Theoretical model: step-edges corrupted by
additive Gaussian noise• Canny has shown that the first derivative of the
Gaussian closely approximates the operator that optimizes the product of signal-to-noise ratio and localization
J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8:679-714, 1986.
Source: L. Fei-Fei
![Page 27: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/27.jpg)
Canny edge detector• Filter image with derivative of Gaussian
• Find magnitude and orientation of gradient
• Non-maximum suppression:
– Thin wide “ridges” down to single pixel width
• Linking and thresholding (hysteresis):
– Define two thresholds: low and high
– Use the high threshold to start edge curves and the low threshold to continue them
• MATLAB: edge(image, ‘canny’);
• >>help edgeSource: D. Lowe, L. Fei-Fei
![Page 28: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/28.jpg)
Example
input image (“Lena”)
![Page 29: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/29.jpg)
Derivative of Gaussian filter
x-direction y-direction
Source: L. Lazebnik
![Page 30: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/30.jpg)
Compute Gradients (DoG)
X-Derivative of Gaussian Y-Derivative of Gaussian Gradient Magnitude
Source: D. Hoiem
![Page 31: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/31.jpg)
Get Orientation at Each Pixel• Threshold at minimum level• Get orientation
theta = atan2(-gy, gx)
Source: D. Hoiem
![Page 32: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/32.jpg)
The Canny edge detector
norm of the gradient
Source: K. Grauman
![Page 33: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/33.jpg)
The Canny edge detector
thresholding
Source: K. Grauman
![Page 34: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/34.jpg)
The Canny edge detector
thresholding
How to turn these thick regions of the gradient into curves?
Source: K. Grauman
![Page 35: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/35.jpg)
Non-maximum suppression
Check if pixel is local maximum along gradient direction, select single max across width of the edge• requires checking interpolated pixels p and r
Source: K. Grauman
![Page 36: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/36.jpg)
Bilinear Interpolation
http://en.wikipedia.org/wiki/Bilinear_interpolation
![Page 37: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/37.jpg)
The Canny edge detector
thinning
(non-maximum suppression)
Problem: pixels along this edge didn’t survive the thresholding
Source: K. Grauman
![Page 38: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/38.jpg)
Hysteresis thresholding
• Use a high threshold to start edge curves, and a low threshold to continue them.
Source: S. Seitz
![Page 39: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/39.jpg)
Hysteresis thresholding
original image
high threshold(strong edges)
low threshold(weak edges)
hysteresis threshold
Source: L. Fei-Fei
![Page 40: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/40.jpg)
Hysteresis thresholding
original image
high threshold(strong edges)
low threshold(weak edges)
hysteresis threshold
Source: L. Fei-Fei
![Page 41: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/41.jpg)
Recap: Canny edge detector• Filter image with derivative of Gaussian
• Find magnitude and orientation of gradient
• Non-maximum suppression:
– Thin wide “ridges” down to single pixel width
• Linking and thresholding (hysteresis):
– Define two thresholds: low and high
– Use the high threshold to start edge curves and the low threshold to continue them
• MATLAB: edge(image, ‘canny’);
• >>help edgeSource: D. Lowe, L. Fei-Fei
![Page 42: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/42.jpg)
Effect of (Gaussian kernel spread/size)
Canny with Canny with original
The choice of depends on desired behavior• large detects large scale edges• small detects fine features
Source: S. Seitz
![Page 43: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/43.jpg)
Background Texture Shadows
Low-level edges vs. perceived contours
Source: K. Grauman
![Page 44: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/44.jpg)
Low-level edges vs. perceived contours
Berkeley segmentation database:http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/
image human segmentation gradient magnitude
Source: L. Lazebnik
![Page 45: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/45.jpg)
pB Boundary Detector
Martin, Fowlkes, Malik 2004: Learning to Detection Natural Boundaries…http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/papers/mfm-pami-boundary.pdf
Figure from Fowlkes
![Page 46: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/46.jpg)
[D. Martin et al. PAMI 2004] Human-marked segment boundaries
Learn from humans which combination of features is most indicative of a “good” contour?
![Page 47: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/47.jpg)
pB Boundary Detector
Figure from Fowlkes
![Page 48: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/48.jpg)
Brightness
Color
Texture
Combined
Human
![Page 49: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/49.jpg)
Results
Human (0.95)
Pb (0.88)
Source: D. Hoiem
![Page 50: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/50.jpg)
Results
Human
Pb
Human (0.96)
Global PbPb (0.88)
Source: D. Hoiem
![Page 51: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/51.jpg)
Human (0.95)
Pb (0.63)
Source: D. Hoiem
![Page 52: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/52.jpg)
Human (0.90)
Pb (0.35)
For more: http://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/bench/html/108082-color.html
![Page 53: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/53.jpg)
Plan for today• Edge detection• Binary image analysis
![Page 54: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/54.jpg)
Binary images
Source: K. Grauman
![Page 55: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/55.jpg)
Binary image analysis: basic steps
• Convert the image into binary form – Thresholding
• Clean up the thresholded image– Morphological operators
• Extract separate blobs– Connected components
• Describe the blobs with region properties
Source: K. Grauman
![Page 56: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/56.jpg)
Binary images
• Two pixel values– Foreground and background– Mark region(s) of interest
Source: K. Grauman
![Page 57: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/57.jpg)
Thresholding
• Grayscale -> binary mask• Useful if object of interest’s intensity distribution
is distinct from background
• Example http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FITZGIBBON/simplebinary.html
Source: K. Grauman
![Page 58: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/58.jpg)
Thresholding• Given a grayscale image or an intermediate matrix
threshold to create a binary output.
Gradient magnitude
Looking for pixels where gradient is strong.
fg_pix = find(gradient_mag > t);
Example: edge detection
Source: K. Grauman
![Page 59: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/59.jpg)
=-
Thresholding• Given a grayscale image or an intermediate matrix
threshold to create a binary output.
Example: background subtraction
Looking for pixels that differ significantly from the “empty” background.
fg_pix = find(diff > t);Source: K. Grauman
![Page 60: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/60.jpg)
Thresholding• Given a grayscale image or an intermediate matrix
threshold to create a binary output.
Example: intensity-based detection
Looking for dark pixels
fg_pix = find(im < 65);
Source: K. Grauman
![Page 61: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/61.jpg)
Thresholding• Given a grayscale image or an intermediate matrix
threshold to create a binary output.
Example: color-based detection
Looking for pixels within a certain hue range.
fg_pix = find(hue > t1 & hue < t2);
Source: K. Grauman
![Page 62: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/62.jpg)
A nice case: bimodal intensity histograms
Ideal histogram, light object on dark background
Actual observed histogram with noise
Images: http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/OWENS/LECT2/node3.html
Source: K. Grauman
![Page 63: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/63.jpg)
Issues
• What to do with “noisy” binary outputs?– Holes– Extra small fragments
• How to demarcate multiple regions of interest? – Count objects– Compute further features per
object
Source: K. Grauman
![Page 64: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/64.jpg)
Morphological operators
• Change the shape of the foreground regions via intersection/union operations between a scanning structuring element and binary image.
• Useful to clean up result from thresholding
• Basic operators are:– Dilation– Erosion
Source: K. Grauman
![Page 65: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/65.jpg)
Dilation• Expands connected components• Grow features• Fill holes
Before dilation After dilation
Source: K. Grauman
![Page 66: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/66.jpg)
Erosion• Erode connected components• Shrink features• Remove bridges, branches, noise
Before erosion After erosion
Source: K. Grauman
![Page 67: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/67.jpg)
Structuring elements
• Masks of varying shapes and sizes used to perform morphology, for example:
• Scan mask across foreground pixels to transform the binary image
>> help strel
Source: K. Grauman
![Page 68: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/68.jpg)
Dilation vs. Erosion
At each position:• Dilation: if current pixel is foreground, OR the
structuring element with the input image.
Source: K. Grauman
![Page 69: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/69.jpg)
Example for Dilation (1D)
SExfxg )()(
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1Output Image
111
Adapted from T. Moeslund Source: K. Grauman
![Page 70: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/70.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1Output Image
111
Source: K. Grauman
![Page 71: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/71.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1 0Output Image
111
Source: K. Grauman
![Page 72: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/72.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1 0 0Output Image
111
Source: K. Grauman
![Page 73: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/73.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1 0 1 1 1Output Image
111
Source: K. Grauman
![Page 74: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/74.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1 0 1 1 1 1Output Image
111
Source: K. Grauman
![Page 75: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/75.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1 0 1 1 1 1 1Output Image
111
Source: K. Grauman
![Page 76: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/76.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1 0 1 1 1 1 1Output Image
111
Source: K. Grauman
![Page 77: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/77.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1 0 1 1 1 1 1 1 1Output Image
111
Note that the object gets bigger and holes are filled.>> help imdilate
Source: K. Grauman
![Page 78: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/78.jpg)
2D example for dilation
Source: Shapiro and Stockman
![Page 79: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/79.jpg)
Dilation vs. Erosion
At each position:• Dilation: if current pixel is foreground, OR
the structuring element with the input image.• Erosion: if every pixel under the structuring
element’s nonzero entries is foreground, OR the current pixel with S.
Source: K. Grauman
![Page 80: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/80.jpg)
Example for Erosion (1D)
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0Output Image
111
SExfxg O)()( _
Source: K. Grauman
![Page 81: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/81.jpg)
Example for Erosion (1D)
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0Output Image
111
SExfxg O)()( _
Source: K. Grauman
![Page 82: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/82.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0Output Image
111
Source: K. Grauman
![Page 83: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/83.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0 0Output Image
111
Source: K. Grauman
![Page 84: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/84.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0 0 0Output Image
111
Source: K. Grauman
![Page 85: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/85.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0 0 0 1Output Image
111
Source: K. Grauman
![Page 86: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/86.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0 0 0 1 0Output Image
111
Source: K. Grauman
![Page 87: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/87.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0 0 0 1 0 0Output Image
111
Source: K. Grauman
![Page 88: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/88.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0 0 0 1 0 0 0Output Image
111
Source: K. Grauman
![Page 89: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/89.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0 0 0 1 0 0 0 1Output Image
111
Note that the object gets smaller>> help imerode Source: K. Grauman
![Page 90: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/90.jpg)
2D example for erosion
Source: Shapiro and Stockman
![Page 91: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/91.jpg)
Opening• Erode, then dilate• Remove small objects, keep original shape
Before opening After opening
Source: K. Grauman
![Page 92: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/92.jpg)
Closing• Dilate, then erode • Fill holes, but keep original shape
Before closing After closing
Applet: http://bigwww.epfl.ch/demo/jmorpho/start.phpSource: K. Grauman
![Page 93: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/93.jpg)
Summary
• Can use filters to process and describe local neighborhood
– Derivatives to locate gradients– Convolution properties will influence efficiency
• Edge detection processes the image gradient to find curves
• Clean up thresholding outputs with binary image morphology operations
Adapted from K. Grauman
![Page 94: CS 1699: Intro to Computer Vision Edges and Binary Images Prof. Adriana Kovashka University of Pittsburgh September 15, 2015](https://reader036.vdocuments.net/reader036/viewer/2022062517/56649ef25503460f94c04569/html5/thumbnails/94.jpg)
Next time
• Computing image features: Detecting and describing keypoints