计算机视觉表征与识别 chapter 5: edges
TRANSCRIPT
![Page 1: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/1.jpg)
计算机视觉表征与识别Chapter 5: Edges
王利民
媒体计算课题组
http://mcg.nju.edu.cn/
1
![Page 2: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/2.jpg)
Today’s class
◼ Introduction to edge detection.
◼ Gradients and edges.
◼ Canny edge detector.
◼ Object contour.
◼ Pb edge detector.
◼ Recent advances in edge detection.
◼ Straight line detection
25
![Page 3: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/3.jpg)
From Pixels to Perception
TigerGrass
Water
Sand
Tiger
tail
eye
legs
head
back
shadow
mouthMid-level operations of
Segmentation and Grouping
27
![Page 4: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/4.jpg)
First problem: edge detection
◼ Goal: compute something like a line drawing of
a scene
33
![Page 5: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/5.jpg)
Issues
◼ No precise problem formulation
◼ Much harder than it seems to be
◼ Edge detectors usually work by detecting “big
changes” in image intensity
◼ Boundary is contour in the image plane that
represents a change in pixel ownership from
object or surface to another.
36
![Page 6: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/6.jpg)
Edge detection
◼ Goal: map image from 2d array of pixels to a set of
curves or line segments or contours.
◼ Main idea: look for strong gradients, post-process
Figure from J. Shotton et al., PAMI 2007
37
![Page 7: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/7.jpg)
What causes an edge?
Depth discontinuity:
object boundary
Change in surface
orientation: shape
Cast shadows
Reflectance change:
appearance
information, texture
Slide credit:
Kristen Grauman
39
![Page 8: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/8.jpg)
Edges/gradients and invariance
Slide credit:
Kristen Grauman
40
![Page 9: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/9.jpg)
Derivative of Gaussian filters
x-direction y-direction
Source: L. Lazebnik52
![Page 10: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/10.jpg)
The Sobel Operator: A common
approximation of derivative of gaussian
53
![Page 11: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/11.jpg)
Smoothing with a Gaussian
Recall: parameter σ is the “scale” / “width” / “spread” of the
Gaussian kernel, and controls the amount of smoothing.
…
55
![Page 12: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/12.jpg)
Effect of σ on derivatives
The apparent structures differ depending on
Gaussian’s scale parameter.
Larger values: larger scale edges detected
Smaller values: finer features detected
σ = 1 pixel σ = 3 pixels
Slide credit:
Kristen Grauman
56
![Page 13: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/13.jpg)
Laplacian of Gaussian
Consider
Laplacian of Gaussian
operator
Where is the edge? Zero-crossings of bottom graphSlide credit: Steve Seitz
58
![Page 14: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/14.jpg)
2D edge detection filters
• is the Laplacian operator:
Laplacian of Gaussian
Gaussian derivative of Gaussian
Slide credit: Steve Seitz
59
![Page 15: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/15.jpg)
Designing an edge detector
• Criteria for an “optimal” edge detector: Good detection: minimize the probability of false positives (detecting spurious
edges caused by noise), as well as that of false negatives (missing real edges)
Good localization: must be as close as possible to the true edges
Single response: the detector must return one point only for each true edge point;
60
![Page 16: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/16.jpg)
62
![Page 17: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/17.jpg)
Gradients -> edges
Primary edge detection steps:
1. Smoothing: suppress noise
2. Edge enhancement: filter for contrast
3. Edge localization
Determine which local maxima from filter output are actually edges vs. noise
Thresholding and thinning
63
![Page 18: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/18.jpg)
69
![Page 19: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/19.jpg)
Canny edge detector
• This is probably the most widely used edge detector in computer vision
• Theoretical model: step-edges corrupted by additive Gaussian noise
• Canny has shown that the first derivative of the Gaussian closely approximates the operator that optimizes the product of signal-to-noise ratio and localization
J. Canny, A Computational Approach To Edge Detection, IEEE Trans.
Pattern Analysis and Machine Intelligence, 8:679-714, 1986.
70
![Page 20: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/20.jpg)
Canny edge detector
◼ Filter image with derivative of Gaussian
◼ Find magnitude and orientation of gradient
◼ Non-maximum suppression:
Thin wide “ridges” down to single pixel width
◼ Linking and thresholding (hysteresis):
Define two thresholds: low and high
Use the high threshold to start edge curves and
the low threshold to continue them
◼ MATLAB: edge(image, ‘canny’);
◼ >>help edge
Source: D. Lowe, L. Fei-Fei
71
![Page 21: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/21.jpg)
The Canny edge detector
original image (Lena)
Slide credit: Steve Seitz
72
![Page 22: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/22.jpg)
The Canny edge detector
norm of the gradient
73
![Page 23: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/23.jpg)
The Canny edge detector
thresholding
74
![Page 24: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/24.jpg)
The Canny edge detector
thresholding
How to turn
these thick
regions of the
gradient into
curves?
75
![Page 25: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/25.jpg)
Non-maximum suppression
Check if pixel is local maximum along gradient direction,
select single max across width of the edge
requires checking interpolated pixels p and r
76
![Page 26: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/26.jpg)
Bilinear Interpolation
http://en.wikipedia.org/wiki/Bilinear_interpolation77
![Page 27: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/27.jpg)
Sidebar: Interpolation options
◼ imx2 = imresize(im, 2, interpolation_type)
◼ ‘nearest’ Copy value from nearest known
Very fast but creates blocky edges
◼ ‘bilinear’ Weighted average from four nearest known
pixels
Fast and reasonable results
◼ ‘bicubic’ (default) Non-linear smoothing over larger area
Slower, visually appealing, may create negative pixel values
Examples from http://en.wikipedia.org/wiki/Bicubic_interpolation
78
![Page 28: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/28.jpg)
Before non-max suppression
79
![Page 29: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/29.jpg)
After non-max suppression
Problem:
pixels along
this edge
didn’t
survive the
thresholding
80
![Page 30: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/30.jpg)
Hysteresis thresholding
◼ Threshold at low/high levels to get weak/strong edge pixels
◼ Do connected components, starting from strong edge
pixels
81
![Page 31: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/31.jpg)
Closing edge gaps
◼ Check that maximum value of gradient value is
sufficiently large
drop-outs? use hysteresis
◼ use a high threshold to start edge curves and a low threshold to
continue them.
Gra
die
nt m
ag
nitu
de
t1
t2
Labeled as edge Pixel number in linked list
along gradient maxima
Not an edge
82
![Page 32: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/32.jpg)
Hysteresis thresholding
Source: S. Seitz
strong edge
pixel
weak but
connected edge
pixels
strong edge
pixel
83
![Page 33: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/33.jpg)
Hysteresis thresholding
original image
high threshold
(strong edges)
low threshold
(weak edges)
hysteresis threshold
84
![Page 34: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/34.jpg)
Final Canny Edges
85
![Page 35: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/35.jpg)
Effect of (Gaussian kernel spread/size)
Canny with Canny with original
The choice of depends on desired behavior• large detects large scale edges
• small detects fine features
86
![Page 36: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/36.jpg)
Example: Canny Edge Detection
87
![Page 37: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/37.jpg)
Example: Canny Edge Detection
88
![Page 38: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/38.jpg)
Background Texture Shadows
Low-level edges vs. perceived contours
90
![Page 39: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/39.jpg)
91
![Page 40: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/40.jpg)
Low-level edges vs. perceived contours
◼ Berkeley segmentation database:http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/
image human segmentation gradient magnitude
92
![Page 41: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/41.jpg)
93
![Page 42: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/42.jpg)
94
![Page 43: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/43.jpg)
Berkeley Segmentation Data Set
David Martin, Charless Fowlkes, Doron Tal, Jitendra Malik
95
![Page 44: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/44.jpg)
Annotation tool
96
![Page 45: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/45.jpg)
97
![Page 46: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/46.jpg)
98
![Page 47: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/47.jpg)
Segmentation Error Measure
◼ A segmentation is a division of the pixels an
image into sets.
◼ A segmentation error takes two segmentations
as input and produces a real-valued output.
◼ We define a measure error at each pixel that is
tolerant to refinement.
99
![Page 48: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/48.jpg)
GCE vs. LCE
100
![Page 49: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/49.jpg)
101
A
B C
![Page 50: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/50.jpg)
Error Measure Validation
102
![Page 51: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/51.jpg)
103
![Page 52: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/52.jpg)
104
Dataset Summary
• 30 subjects, age 19-23
– 17 men, 13 women
– 9 with artistic training
• 8 months
• 1,458 person hours
• 1,020 Corel images
• 11,595 Segmentations
– 5,555 color, 5,554 gray, 486 inverted/negated
![Page 53: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/53.jpg)
105
![Page 54: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/54.jpg)
[D. Martin et al. PAMI 2004] Human-marked segment boundaries
Learn from
humans which
combination of
features is most
indicative of a
“good” contour?
106
![Page 55: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/55.jpg)
[D. Martin et al. PAMI 2004]
What features are responsible for
perceived edges?
Feature profiles
(oriented energy,
brightness, color,
and texture
gradients) along
the patch’s
horizontal diameter
107
![Page 56: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/56.jpg)
[D. Martin et al. PAMI 2004]
What features are responsible for
perceived edges?
Feature profiles
(oriented energy,
brightness, color,
and texture
gradients) along
the patch’s
horizontal diameter
108
![Page 57: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/57.jpg)
109
![Page 58: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/58.jpg)
110
Boundaries of image regions
defined by a number of cues
Brightness
Color
Texture
Motion (in video)
Binocular Diparity (if available)
Familiar objects
![Page 59: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/59.jpg)
111
![Page 60: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/60.jpg)
112
![Page 61: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/61.jpg)
113
Brightness and Color Features
• 1976 CIE L*a*b* colorspace
• Brightness Gradient BG(x,y,r,)
– 2 difference in L* distribution
• Color Gradient CG(x,y,r,)
– 2 difference in a* and b*
distributions
+
−=
i ii
ii
hg
hghg
22 )(
2
1),(
r
(x,y)
![Page 62: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/62.jpg)
114
Texture Feature
• Texture Gradient TG(x,y,r,)
– 2 difference of texton histograms
– Textons are vector-quantized filter outputs
TextonMap
![Page 63: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/63.jpg)
119
Non-Boundaries Boundaries
TG
TG
+
−=
f
fff~ˆ
(1) Fit cylindrical parabolas to raw oriented
signal to get local shape: (Savitsky-Golay)
(2) Localize peaks:
Boundary Localization
![Page 64: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/64.jpg)
120
Non-Boundaries Boundaries
I
T
B
C
![Page 65: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/65.jpg)
121
Cue Combination Models
• Classification Trees
– Top-down splits to maximize entropy, error bounded
• Density Estimation
– Adaptive bins using k-means
• Logistic Regression, 3 variants
– Linear and quadratic terms
– Confidence-rated generalization of AdaBoost (Schapire&Singer)
• Hierarchical Mixtures of Experts (Jordan&Jacobs)
– Up to 8 experts, initialized top-down, fit with EM
• Support Vector Machines (libsvm, Chang&Lin)
– Gaussian kernel, -parameterization
➢ Range over bias, complexity, parametric/non-parametric
![Page 66: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/66.jpg)
122
Computing Precision/Recall
Recall = Pr(signal|truth) = fraction of ground truth found by the signal
Precision = Pr(truth|signal) = fraction of signal that is correct
• Always a trade-off between the two
• Standard measures in information retrieval (van Rijsbergen XX)
• ROC from standard signal detection the wrong approach
Strategy
• Detector output (Pb) is a soft boundary map
• Compute precision/recall curve:
– Threshold Pb at many points t in [0,1]
– Recall = Pr(Pb>t|seg=1)
– Precision = Pr(seg=1|Pb>t)
![Page 67: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/67.jpg)
123
ROC vs.
Precision/Recall
Sig
nal
Truth
P N
P TP FP
N FN TN
ROC Curve
Hit Rate = TP / (TP+FN) =
False Alarm Rate = FP / (FP+TN) =
PR Curve
Precision = TP / (TP+FP) =
Recall = TP / (TP+FN) =
/
/
/
/
+=
rp
pr
tF
2max
![Page 68: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/68.jpg)
124
Classifier
Comparison Goal
More
Noise
More
Signal
![Page 69: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/69.jpg)
125
Cue Calibration
➢All free parameters optimized on training data
➢All algorithmic alternatives evaluated by experiment
• Brightness Gradient
– Scale, bin/kernel sizes for KDE
• Color Gradient
– Scale, bin/kernel sizes for KDE, joint vs. marginals
• Texture Gradient
– Filter bank: scale, multiscale?
– Histogram comparison: L1, L2, L, 2, EMD
– Number of textons, Image-specific vs. universal textons
• Localization parameters for each cue
![Page 70: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/70.jpg)
126
![Page 71: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/71.jpg)
127
Classifier
Comparison
![Page 72: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/72.jpg)
128
![Page 73: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/73.jpg)
129
Alternate Approaches
• Canny Detector
– Canny 1986
– MATLAB implementation
– With and without hysteresis
• Second Moment Matrix
– Nitzberg/Mumford/Shiota 1993
– cf. Förstner and Harris corner detectors
– Used by Konishi et al. 1999 in learning framework
– Logistic model trained on full eigenspectrum
![Page 74: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/74.jpg)
130
Pb ImagesCanny 2MM Us HumanImage
![Page 75: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/75.jpg)
131
Pb Images IICanny 2MM Us HumanImage
![Page 76: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/76.jpg)
132
Pb Images IIICanny 2MM Us HumanImage
![Page 77: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/77.jpg)
133
Two Decades
of Boundary
Detection
![Page 78: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/78.jpg)
134
Findings
1. A simple linear model is sufficient for cue combination
– All cues weighted approximately equally in logistic
2. Proper texture edge model is not optional for complex natural images
– Texture suppression is not sufficient!
3. Significant improvement over state-of-the-art in boundary detection
– Pb(x,y,) useful for higher-level processing
4. Empirical approach critical for both cue calibration and cue combination
![Page 79: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/79.jpg)
[D. Martin et al. PAMI 2004] Kristen Grauman, UT-Austin
135
![Page 80: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/80.jpg)
Brightness
Color
Texture
Combined
Human
136
![Page 81: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/81.jpg)
Results
Human (0.95)
Pb (0.88)
137
![Page 82: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/82.jpg)
Results
Human
Pb
Human (0.96)
Global PbPb (0.88)
138
![Page 83: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/83.jpg)
Human (0.95)
Pb (0.63)
139
![Page 84: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/84.jpg)
Human (0.90)
Pb (0.35)
For more:
http://www.eecs.berkeley.edu/Research/Projects/CS/v
ision/bsds/bench/html/108082-color.html140
![Page 85: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/85.jpg)
141
![Page 86: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/86.jpg)
142
Exploiting global constraints:Image Segmentation as Graph Partitioning
Build a weighted graph G=(V,E) from image
V: image pixels
E: connections between
pairs of nearby pixels
Partition graph so that similarity within group is large and
similarity between groups is small -- Normalized Cuts
[Shi & Malik 97]
![Page 87: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/87.jpg)
143
Wij small when intervening contour strong, small when weak..
Cij = max Pb(x,y) for (x,y) on line segment ij; Wij = exp ( - Cij /
![Page 88: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/88.jpg)
144
Spectral Pb
![Page 89: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/89.jpg)
Global pB boundary detector
145
![Page 90: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/90.jpg)
Contour Detection
◼ Local and Global cues combination
146
![Page 91: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/91.jpg)
Global pB boundary detector
147
![Page 92: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/92.jpg)
148
![Page 93: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/93.jpg)
149
![Page 94: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/94.jpg)
150
![Page 95: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/95.jpg)
151
![Page 96: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/96.jpg)
Edge Detection with Structured
Random Forests
◼ Goal: quickly predict whether each pixel is an edge
◼ Insights Predictions can be learned from training
data
Predictions for nearby pixels should not be independent
◼ Solution Train structured random forests to split
data into patches with similar boundaries based on features
Predict boundaries at patch level, rather than pixel level, and aggregate (average votes)
http://research.microsoft.com/pubs/202540/DollarICCV13edges.pdf
Boundaries
in patch152
![Page 97: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/97.jpg)
Edge Detection with Structured
Random Forests
◼ Algorithm
1. Extract overlapping 32x32
patches at three scales
2. Features are pixel values and
pairwise differences in feature
maps (LUV color, gradient
magnitude, oriented gradient)
3. Predict 𝑇 boundary maps in the
central 16x16 region using 𝑇trained decision trees
4. Average predictions for each pixel
across all patches
153
![Page 98: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/98.jpg)
Standard Decision Tree Learning
154
![Page 99: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/99.jpg)
155
General Information Gain
![Page 100: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/100.jpg)
156
![Page 101: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/101.jpg)
Edge Detection with Structured
Random Forests
Results
BSDS 500 NYU Depth dataset edges
157
![Page 102: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/102.jpg)
158
![Page 103: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/103.jpg)
159
![Page 104: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/104.jpg)
Holistically nested edge detection
https://arxiv.org/pdf/1504.06375.pdf160
![Page 105: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/105.jpg)
161
![Page 106: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/106.jpg)
State of edge detection
◼ Local edge detection is mostly solved Intensity gradient, color, texture
HED on BSDS 500 is near human performance
◼ Some room for improvement by taking advantage of higher-level knowledge (e.g., objects)
◼ Still hard to produce all objects within a small number of regions
162
![Page 107: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/107.jpg)
Finding straight lines
163
![Page 108: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/108.jpg)
Finding line segments using
connected components
1. Compute canny edges
– Compute: gx, gy (DoG in x,y directions)
– Compute: theta = atan(gy / gx)
2. Assign each edge to one of 8 directions
3. For each direction d, get edgelets:
– find connected components for edge pixels with directions in
{d-1, d, d+1}
4. Compute straightness and theta of edgelets using eig
of x,y 2nd moment matrix of their points
5. Threshold on straightness, store segment
( ) ( )( )( )( ) ( )
−−−
−−−=
2
2
yyx
yxx
yyx
yxx
M )eig(],[ Μ=λv
))2,1(),2,2(2(atan vv=
12 / =conf
Larger eigenvector
164
![Page 109: 计算机视觉表征与识别 Chapter 5: Edges](https://reader033.vdocuments.net/reader033/viewer/2022060918/62995d091e2940465d4b8874/html5/thumbnails/109.jpg)
Things to remember
◼ Canny edge detector = smooth →derivative → thin → threshold → link
◼ Pb: learns weighting of gradient, color, texture differences More recent learning approaches give at
least as good accuracy and are faster
◼ Straight line detector = canny + gradient orientations → orientation binning → linking → check for straightness
165