edges and binary image analysis - university of california ...yjlee/teaching/ecs189g...edge...
TRANSCRIPT
![Page 1: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/1.jpg)
Edges and Binary Image AnalysisApril 9th, 2015
Yong Jae LeeUC Davis
![Page 2: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/2.jpg)
Previously• Filters allow local image neighborhood to
influence our description and features– Smoothing to reduce noise
– Derivatives to locate contrast, gradient
• Seam carving application: – use image gradients to measure “interestingness” or
“energy”
– remove 8-connected seams so as to preserve image’s energy
Slide credit: Kristen Grauman
2
![Page 3: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/3.jpg)
Today
• Edge detection and matching– process the image gradient to find curves/contours– comparing contours
• Binary image analysis– blobs and regions
Slide credit: Kristen Grauman
3
![Page 4: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/4.jpg)
Edge detection• Goal: map image from 2d array of pixels to a set of
curves or line segments or contours.• Why?
• Main idea: look for strong gradients, post-process
Figure from J. Shotton et al., PAMI 2007
Figure from D. Lowe
Slide credit: Kristen Grauman
4
![Page 5: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/5.jpg)
Gradients -> edges
Primary edge detection steps:1. Smoothing: suppress noise2. Edge enhancement: filter for contrast3. Edge localization
Determine which local maxima from filter output are actually edges vs. noise
• Threshold, Thin
Slide credit: Kristen Grauman
5
![Page 6: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/6.jpg)
Thresholding
• Choose a threshold value t• Set any pixels less than t to zero (off)• Set any pixels greater than or equal to t to one
(on)
Slide credit: Kristen Grauman
6
![Page 7: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/7.jpg)
Original image
Slide credit: Kristen Grauman
7
![Page 8: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/8.jpg)
Gradient magnitude image
8
![Page 9: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/9.jpg)
Thresholding gradient with a lower threshold
Slide credit: Kristen Grauman
9
![Page 10: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/10.jpg)
Thresholding gradient with a higher threshold
Slide credit: Kristen Grauman
10
![Page 11: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/11.jpg)
Canny edge detector• Filter image with derivative of Gaussian
• Find magnitude and orientation of gradient
• Non-maximum suppression:
– Thin wide “ridges” down to single pixel width
• Linking and thresholding (hysteresis):
– Define two thresholds: low and high
– Use the high threshold to start edge curves and the low threshold to continue them
• MATLAB: edge(image, ‘canny’);
• >>help edgeSlide credit: David Lowe, Fei-Fei Li
11
![Page 12: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/12.jpg)
The Canny edge detector
original image (Lena)Slide credit: Steve Seitz
12
![Page 13: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/13.jpg)
The Canny edge detector
gradient magnitude
Slide credit: Kristen Grauman
13
![Page 14: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/14.jpg)
Compute Gradients (DoG)
X-Derivative of Gaussian Y-Derivative of Gaussian Gradient Magnitude
Slide credit: Svetlana Lazebnik
![Page 15: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/15.jpg)
The Canny edge detector
gradient magnitude
Slide credit: Kristen Grauman
15
![Page 16: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/16.jpg)
The Canny edge detector
thresholding
Slide credit: Kristen Grauman
16
![Page 17: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/17.jpg)
The Canny edge detector
thresholding
How to turn these thick regions of the gradient into curves?
Slide credit: Kristen Grauman
17
![Page 18: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/18.jpg)
Non-maximum suppression
Check if pixel is local maximum along gradient directionSelect single max across width of the edgeRequires checking interpolated pixels p and r
Slide credit: Kristen Grauman
18
![Page 19: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/19.jpg)
The Canny edge detector
thinning(non-maximum suppression)
Problem: pixels along this edge didn’t survive the thresholding
Slide credit: Kristen Grauman
19
![Page 20: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/20.jpg)
Hysteresis thresholding
• Use a high threshold to start edge curves, and a low threshold to continue them.
Slide credit: Steve Seitz
20
![Page 21: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/21.jpg)
Hysteresis thresholding
original image
high threshold(strong edges)
low threshold(weak edges)
hysteresis threshold
Slide credit: Fei-Fei Li
21
![Page 22: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/22.jpg)
Hysteresis thresholding
http://users.ecs.soton.ac.uk/msn/book/new_demo/thresholding/
Slide credit: Kristen Grauman
22
![Page 23: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/23.jpg)
Recap: Canny edge detector• Filter image with derivative of Gaussian
• Find magnitude and orientation of gradient
• Non-maximum suppression:
– Thin wide “ridges” down to single pixel width
• Linking and thresholding (hysteresis):
– Define two thresholds: low and high
– Use the high threshold to start edge curves and the low threshold to continue them
• MATLAB: edge(image, ‘canny’);
• >>help edgeSlide credit: David Lowe, Fei-Fei Li
23
![Page 24: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/24.jpg)
Background Texture Shadows
Low-level edges vs. perceived contours
Slide credit: Kristen Grauman 24
![Page 25: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/25.jpg)
Low-level edges vs. perceived contours
• Berkeley segmentation database:http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/
image human segmentation gradient magnitude
Slide credit: Svetlana Lazebnik
25
![Page 26: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/26.jpg)
[D. Martin et al. PAMI 2004]
Human-marked segment boundaries
Learn from humans which combination of features is most indicative of a “good” contour?
Slide credit: Kristen Grauman26
![Page 27: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/27.jpg)
pB boundary detector
Figure from Fowlkes
Martin, Fowlkes, Malik 2004: Learning to Detection Natural Boundaries…http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/papers/mfm-pami-boundary.pdf
![Page 28: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/28.jpg)
pB Boundary Detector
Figure from Fowlkes
![Page 29: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/29.jpg)
[D. Martin et al. PAMI 2004]Slide credit: Kristen Grauman 29
![Page 30: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/30.jpg)
State-of-the-Art in Contour Detection
Source: Jitendra Malik: http://www.cs.berkeley.edu/~malik/malik-talks-ptrs.html
Prewitt, Sobel, Roberts
Canny
Canny+optthresholds
Learned with combined features
Human agreement
Slide credit: Adapted from Kristen Grauman
30
![Page 31: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/31.jpg)
Global pB boundary detector
Figure from Fowlkes
![Page 32: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/32.jpg)
Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013)
• Goal: quickly predict whether each pixel is an edge
• Insights– Predictions can be learned from training
data– Predictions for nearby pixels should not be
independent
• Solution– Train structured random forests to split
data into patches with similar boundaries based on features
– Predict boundaries at patch level, rather than pixel level, and aggregate (average votes)
http://research.microsoft.com/pubs/202540/DollarICCV13edges.pdf
Boundaries in patch
Slide credit: Derek Hoiem
![Page 33: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/33.jpg)
Edge Detection with Structured Random Forests
P. Dollar and L. Zitnick, Structured forests for fast edge detection, ICCV 2013
Training dataInput images
Ground truth
Output
Slide credit: Svetlana Lazebnik
![Page 34: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/34.jpg)
State of edge detection
• Local edge detection is mostly solved– Intensity gradient, color, texture
• Work on RGB-D edge detection is currently more active
• Often used in combination with object detectors or region classifiers
Slide credit: Derek Hoiem
![Page 35: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/35.jpg)
Today
• Edge detection and matching– process the image gradient to find curves/contours– comparing contours
• Binary image analysis– blobs and regions
Slide credit: Kristen Grauman
35
![Page 36: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/36.jpg)
Figure from Belongie et al.Slide credit: Kristen Grauman
36
![Page 37: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/37.jpg)
Chamfer distance• Average distance to nearest feature
=I
=T
Set of points in image
Set of points on (shifted) template
=)(tdIMinimum distance between point t and some point in I
Slide credit: Kristen Grauman
37
![Page 38: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/38.jpg)
Chamfer distance
Slide credit: Kristen Grauman
38
![Page 39: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/39.jpg)
Chamfer distance• Average distance to nearest feature
Edge image
How is the measure different than just filtering with a mask having the shape points?
How expensive is a naïve implementation?
Slide credit: Kristen Grauman
39
![Page 40: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/40.jpg)
34
23
23
5 4 4
223
112
2 1 1 2 11 0 0 1 2 1
0001
2321011 0 1 2 3 3 2
101110 1
2
1 0 1 2 3 4 3 210122
Distance TransformImage features (2D)
Distance Transform is a function that for each image pixel p assigns a non-negative number corresponding to
distance from p to the nearest feature in the image I
)(⋅D)( pD
Features could be edge points, foreground points,…
Distance transform
Slide credit: Yuri Boykov
40
![Page 41: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/41.jpg)
Distance transform
original distance transformedges
Value at (x,y) tells how far that position is from the nearest edge point (or other binary mage structure) >> help bwdist
Slide credit: Kristen Grauman
41
![Page 42: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/42.jpg)
Distance transform (1D)
// 0 if j is in P, infinity otherwise
Slide adapted from Dan Huttonlocher, Kristen Grauman
0 1 0 1 0 0 0 1 0Image features (edges)
Distance transform
42
![Page 43: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/43.jpg)
Distance Transform (2D)
Slide credit: Adapted by Kristen Grauman from Dan Huttonlocher
43
![Page 44: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/44.jpg)
Chamfer distance• Average distance to nearest feature
Edge image Distance transform image
Slide credit: Kristen Grauman
44
![Page 45: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/45.jpg)
Chamfer distance
Fig from D. Gavrila, DAGM 1999
Edge image Distance transform image
Slide credit: Kristen Grauman
45
![Page 46: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/46.jpg)
Chamfer distance: properties
• Sensitive to scale and rotation• Tolerant of small shape changes, clutter• Need large number of template shapes• Inexpensive way to match shapes
Slide credit: Kristen Grauman
46
![Page 47: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/47.jpg)
Today
• Edge detection and matching– process the image gradient to find curves/contours– comparing contours
• Binary image analysis– blobs and regions
Slide credit: Kristen Grauman
47
![Page 48: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/48.jpg)
Binary images
Slide credit: Kristen Grauman
48
![Page 49: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/49.jpg)
Binary image analysis: basic steps
• Convert the image into binary form – Thresholding
• Clean up the thresholded image– Morphological operators
• Extract separate blobs– Connected components
• Describe the blobs with region properties
Slide credit: Kristen Grauman
49
![Page 50: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/50.jpg)
Binary images
• Two pixel values– Foreground and background– Mark region(s) of interest
Slide credit: Kristen Grauman
50
![Page 51: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/51.jpg)
Thresholding
• Grayscale -> binary mask• Useful if object of interest’s intensity distribution
is distinct from background
• Examplehttp://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FITZGIBBON/simplebinary.html
Slide credit: Kristen Grauman
51
![Page 52: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/52.jpg)
Thresholding• Given a grayscale image or an intermediate matrix
threshold to create a binary output.
Gradient magnitude
Looking for pixels where gradient is strong.
fg_pix = find(gradient_mag > t);
Example: edge detection
Slide adapted from Kristen Grauman
52
![Page 53: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/53.jpg)
=-
Thresholding• Given a grayscale image or an intermediate matrix
threshold to create a binary output.
Example: background subtraction
Looking for pixels that differ significantly from the “empty” background.
fg_pix = find(diff > t);Slide credit: Kristen Grauman 53
![Page 54: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/54.jpg)
Thresholding• Given a grayscale image or an intermediate matrix
threshold to create a binary output.
Example: intensity-based detection
Looking for dark pixelsfg_pix = find(im < 65);
Slide credit: Kristen Grauman
54
![Page 55: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/55.jpg)
Thresholding• Given a grayscale image or an intermediate matrix
threshold to create a binary output.
Example: color-based detection
Looking for pixels within a certain hue range.
fg_pix = find(hue > t1 & hue < t2);
Slide credit: Kristen Grauman
55
![Page 56: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/56.jpg)
A nice case: bimodal intensity histograms
Ideal histogram, light object on dark background
Actual observed histogram with noise
Slide credit: Kristen GraumanImages: http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/OWENS/LECT2/node3.html
56
![Page 57: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/57.jpg)
Not so nice cases
Slide credit: Shapiro and Stockman57
![Page 58: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/58.jpg)
Issues
• What to do with “noisy” binary outputs?– Holes– Extra small fragments
• How to demarcate multiple regions of interest? – Count objects– Compute further features per
object
Slide credit: Kristen Grauman
58
![Page 59: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/59.jpg)
Morphological operators
• Change the shape of the foreground regions via intersection/union operations between a scanning structuring element and binary image
• Useful to clean up result from thresholding
• Basic operators are:– Dilation– Erosion
Slide credit: Kristen Grauman
59
![Page 60: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/60.jpg)
Dilation• Expands connected components• Grow features• Fill holes
Before dilation After dilation
Slide credit: Kristen Grauman
60
![Page 61: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/61.jpg)
Erosion• Erode connected components• Shrink features• Remove bridges, branches, noise
Before erosion After erosion
Slide credit: Kristen Grauman
61
![Page 62: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/62.jpg)
Structuring elements• Masks of varying shapes and sizes used to
perform morphology, for example:
• Scan mask across foreground pixels to transform the binary image
>> help strel
Slide credit: Kristen Grauman
62
![Page 63: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/63.jpg)
Dilation vs. ErosionAt each position:• Dilation: if current pixel is 1, then set all the
output pixels corresponding to structuring element to 1.
63
![Page 64: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/64.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1Output Image
111
Slide credit: Adapted by Kristen Grauman from T. Moeslund
64
![Page 65: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/65.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1Output Image
111
Slide credit: Kristen Grauman
65
![Page 66: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/66.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1 0Output Image
111
Slide credit: Kristen Grauman
66
![Page 67: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/67.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1 0 0Output Image
111
Slide credit: Kristen Grauman
67
![Page 68: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/68.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1 0 1 1 1Output Image
111
Slide credit: Kristen Grauman
68
![Page 69: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/69.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1 0 1 1 1 1Output Image
111
Slide credit: Kristen Grauman
69
![Page 70: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/70.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1 0 1 1 1 1 1Output Image
111
Slide credit: Kristen Grauman
70
![Page 71: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/71.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1 0 1 1 1 1 1Output Image
111
Slide credit: Kristen Grauman
71
![Page 72: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/72.jpg)
Example for Dilation
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
1 1 0 1 1 1 1 1 1 1Output Image
111
Note that the object gets bigger and holes are filled.>> help imdilate
Slide credit: Kristen Grauman
72
![Page 73: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/73.jpg)
2D example for dilation
Slide credit: Shapiro & Stockman73
![Page 74: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/74.jpg)
Dilation vs. ErosionAt each position:• Dilation: if current pixel is 1, then set all the
output pixels corresponding to structuring element to 1.
• Erosion: if every pixel under the structuring element is 1, then set the output pixel corresponding to the current pixel to 1.
74
![Page 75: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/75.jpg)
Example for Erosion (1D)
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0Output Image
111
Slide credit: Kristen Grauman
75
![Page 76: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/76.jpg)
Example for Erosion (1D)
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0Output Image
111
Slide credit: Kristen Grauman
76
![Page 77: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/77.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0Output Image
111
Slide credit: Kristen Grauman
77
![Page 78: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/78.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0 0Output Image
111
Slide credit: Kristen Grauman
78
![Page 79: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/79.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0 0 0Output Image
111
Slide credit: Kristen Grauman
79
![Page 80: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/80.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0 0 0 1Output Image
111
Slide credit: Kristen Grauman
80
![Page 81: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/81.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0 0 0 1 0Output Image
111
Slide credit: Kristen Grauman
81
![Page 82: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/82.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0 0 0 1 0 0Output Image
111
Slide credit: Kristen Grauman
82
![Page 83: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/83.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0 0 0 1 0 0 0Output Image
111
Slide credit: Kristen Grauman
83
![Page 84: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/84.jpg)
Example for Erosion
1 0 0 0 1 1 1 0 1 1Input image
Structuring Element
0 0 0 0 0 1 0 0 0 1Output Image
111
Note that the object gets smaller>> help imerode
Slide credit: Kristen Grauman
84
![Page 85: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/85.jpg)
2D example for erosion
Slide credit: Shapiro & Stockman85
![Page 86: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/86.jpg)
Opening• Erode, then dilate• Remove small objects, keep original shape
Before opening After opening
Slide credit: Kristen Grauman
86
![Page 87: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/87.jpg)
Closing• Dilate, then erode • Fill holes, but keep original shape
Before closing After closing
Applet: http://bigwww.epfl.ch/demo/jmorpho/start.phpSlide credit: Kristen Grauman
87
![Page 88: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/88.jpg)
Morphology operators on grayscale images
• Dilation and erosion typically performed on binary images.
• If image is grayscale: for dilation take the neighborhood max, for erosion take the min.
original dilated eroded
Slide credit: Kristen Grauman
88
![Page 89: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/89.jpg)
Issues
• What to do with “noisy” binary outputs?– Holes– Extra small fragments
• How to demarcate multiple regions of interest? – Count objects– Compute further features per
object
Slide credit: Kristen Grauman
89
![Page 90: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/90.jpg)
Connected components• Identify distinct regions of “connected pixels”
Slide credit: Shapiro and Stockman
90
![Page 91: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/91.jpg)
Connectedness• Defining which pixels are considered neighbors
4-connected 8-connected
Slide credit: Chaitanya Chandra91
![Page 92: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/92.jpg)
Connected components
• We’ll consider a sequential algorithm that requires only 2 passes over the image.
• Input: binary image• Output: “label” image,
where pixels are numbered per their component
• Note: foreground here is denoted with black pixels.
Slide credit: Kristen Grauman
92
![Page 93: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/93.jpg)
Sequential connected components
Slide credit: Adapted by Kristen Grauman from J. Neira
93
![Page 94: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/94.jpg)
Sequential connected components
Slide credit: Kristen Grauman
94
![Page 95: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/95.jpg)
Sequential connected components
Slide credit: Kristen Grauman
95
![Page 96: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/96.jpg)
Sequential connected components
Slide credit: Kristen Grauman
96
![Page 97: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/97.jpg)
Connected components
Slide credit: Pinar Duygulu
97
![Page 98: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/98.jpg)
Region properties• Given connected components, can compute
simple features per blob, such as:– Area (num pixels in the region)– Centroid (average x and y position of pixels in the region)– Bounding box (min and max coordinates)– Circularity (ratio of mean dist. to centroid over std)
A1=200A2=170
Slide credit: Kristen Grauman
98
![Page 99: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/99.jpg)
Binary image analysis: basic steps (recap)
• Convert the image into binary form – Thresholding
• Clean up the thresholded image– Morphological operators
• Extract separate blobs– Connected components
• Describe the blobs with region properties
Slide credit: Kristen Grauman
100
![Page 100: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/100.jpg)
Matlab• N = hist(Y,M)• L = bwlabel (BW,N);• STATS = regionprops(L,PROPERTIES) ;
– 'Area'– 'Centroid' – 'BoundingBox' – 'Orientation‘, …
• IM2 = imerode(IM,SE);• IM2 = imdilate(IM,SE);• IM2 = imclose(IM, SE);• IM2 = imopen(IM, SE);
Slide credit: Kristen Grauman
101
![Page 101: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/101.jpg)
Example using binary image analysis: OCR
[Luis von Ahn et al. http://recaptcha.net/learnmore.html]Slide credit: Kristen Grauman 102
![Page 102: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/102.jpg)
Example using binary image analysis: segmentation of a liver
Slide credit: Li Shen
103
![Page 103: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/103.jpg)
Binary images• Pros
– Can be fast to compute, easy to store– Simple processing techniques available– Lead to some useful compact shape descriptors
• Cons– Hard to get “clean” silhouettes– Noise common in realistic scenarios– Can be too coarse of a representation– Not 3d
Slide credit: Kristen Grauman
104
![Page 104: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/104.jpg)
Summary• Operations, tools
• Features, representations
Edges, gradientsBlobs/regionsLocal patternsTextures (next)Color distributions
Derivative filtersSmoothing, morphologyThresholdingConnected componentsMatched filtersHistograms
Slide credit: Kristen Grauman
105
![Page 105: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/105.jpg)
Coming up
• Texture– Read Szeliski 10.5
106
![Page 106: Edges and Binary Image Analysis - University of California ...yjlee/teaching/ecs189g...Edge Detection with Structured Random Forests (Dollar Zitnick ICCV 2013) • Goal: quickly predict](https://reader034.vdocuments.net/reader034/viewer/2022042215/5ebd190c2c516371fe060028/html5/thumbnails/106.jpg)
Questions?
See you Tuesday!
107