introduction
DESCRIPTION
Introduction. CSc I6716 Fall 2010. Part I Feature Extraction (2). Edge Detection. Zhigang Zhu, City College of New York [email protected]. Edge Detection. What’s an edge? “He was sitting on the Edge of his seat.” “She paints with a hard Edge.” - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/1.jpg)
3D Computer Vision
and Video Computing IntroductionIntroduction
Part IFeature Extraction (2)
Edge Detection
CSc I6716Fall 2010
Zhigang Zhu, City College of New York [email protected]
![Page 2: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/2.jpg)
3D Computer Vision
and Video Computing Edge DetectionEdge Detection
n What’s an edge?l “He was sitting on the Edge of his seat.”l “She paints with a hard Edge.”l “I almost ran off the Edge of the road.”l “She was standing by the Edge of the woods.”l “Film negatives should only be handled by their Edges.”l “We are on the Edge of tomorrow.”l “He likes to live life on the Edge.”l “She is feeling rather Edgy.”
n The definition of Edge is not always clear.n In Computer Vision, Edge is usually related to a
discontinuity within a local set of pixels.
![Page 3: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/3.jpg)
3D Computer Vision
and Video Computing DiscontinuitiesDiscontinuities
n A: Depth discontinuity: abrupt depth change in the worldn B: Surface normal discontinuity: change in surface orientationn C: Illumination discontinuity: shadows, lighting changesn D: Reflectance discontinuity: surface properties, markings
A
C
B
D
![Page 4: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/4.jpg)
3D Computer Vision
and Video Computing Illusory EdgesIllusory Edges
n Illusory edges will not be detectable by the algorithms that we will discuss
n No change in image irradiance - no image processing algorithm can directly address these situations
n Computer vision can deal with these sorts of things by drawing on information external to the image (perceptual grouping techniques)
Kanizsa Triangles
![Page 5: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/5.jpg)
3D Computer Vision
and Video Computing Another OneAnother One
![Page 6: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/6.jpg)
3D Computer Vision
and Video Computing GoalGoal
n Devise computational algorithms for the extraction of significant edges from the image.
n What is meant by significant is unclear.l Partly defined by the context in which the edge detector
is being applied
![Page 7: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/7.jpg)
3D Computer Vision
and Video Computing EdgelsEdgels
n Define a local edge or edgel to be a rapid change in the image function over a small areal implies that edgels should be detectable over a local
neighborhoodn Edgels are NOT contours, boundaries, or lines
l edgels may lend support to the existence of those structuresl these structures are typically constructed from edgels
n Edgels have propertiesl Orientationl Magnitudel Position
![Page 8: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/8.jpg)
3D Computer Vision
and Video Computing OutlineOutline
n First order edge detectors (lecture - required)l Mathematicsl 1x2, Roberts, Sobel, Prewitt
n Canny edge detector (after-class reading)n Second order edge detector (after-class reading)
l Laplacian, LOG / DOGn Hough Transform – detect by voting
l Linesl Circlesl Other shapes
![Page 9: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/9.jpg)
3D Computer Vision
and Video Computing Locating EdgelsLocating Edgels
Rapid change in image => high local gradient => differentiation
f(x) = step edge
1st Derivative f ’(x)
2nd Derivative -f ’’(x)
maximum
zero crossing
![Page 10: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/10.jpg)
3D Computer Vision
and Video Computing RealityReality
![Page 11: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/11.jpg)
3D Computer Vision
and Video Computing Properties of an EdgeProperties of an Edge
Original
Orientation
Magnitude
Orientation
Position
![Page 12: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/12.jpg)
3D Computer Vision
and Video Computing Quantitative Edge DescriptorsQuantitative Edge Descriptors
n Edge Orientationl Edge Normal - unit vector in the direction of
maximum intensity change (maximum intensity gradient)
l Edge Direction - unit vector perpendicular to the edge normal
n Edge Position or Centerl image position at which edge is located
(usually saved as binary image)n Edge Strength / Magnitude
l related to local contrast or gradient - how rapid is the intensity variation across the edge along the edge normal.
![Page 13: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/13.jpg)
3D Computer Vision
and Video Computing Edge Degradation in NoiseEdge Degradation in Noise
Ideal step edge Step edge + noise
Increasing noise
![Page 14: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/14.jpg)
3D Computer Vision
and Video Computing Real ImageReal Image
![Page 15: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/15.jpg)
3D Computer Vision
and Video Computing Edge Detection: TypicalEdge Detection: Typical
n Noise Smoothingl Suppress as much noise as possible while retaining
‘true’ edgesl In the absence of other information, assume ‘white’
noise with a Gaussian distributionn Edge Enhancement
l Design a filter that responds to edges; filter output high are edge pixels and low elsewhere
n Edge Localizationl Determine which edge pixels should be discarded as
noise and which should be retainedu thin wide edges to 1-pixel width (nonmaximum
suppression)u establish minimum value to declare a local maximum from
edge filter to be an edge (thresholding)
![Page 16: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/16.jpg)
3D Computer Vision
and Video Computing Edge Detection MethodsEdge Detection Methods
n 1st Derivative Estimatel Gradient edge detectionl Compass edge detectionl Canny edge detector (*)
n 2nd Derivative Estimatel Laplacianl Difference of Gaussians
n Parametric Edge Models (*)
![Page 17: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/17.jpg)
3D Computer Vision
and Video Computing Gradient MethodsGradient Methods
F(x)
xF’(x)
x
Edge= sharp variation
Large first derivative
![Page 18: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/18.jpg)
3D Computer Vision
and Video Computing Gradient of a FunctionGradient of a Function
n Assume f is a continuous function in (x,y). Then
n are the rates of change of the function f in the x and y directions, respectively.
n The vector (Dx, Dy) is called the gradient of f.n This vector has a magnitude:
and an orientation:
n q is the direction of the maximum change in f.n S is the size of that change.
s = x2+ y
2
Dx
Dyq = tan-1 ( )
y
f
x
fyx
,
![Page 19: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/19.jpg)
3D Computer Vision
and Video Computing Geometric InterpretationGeometric Interpretation
n Butl I(i,j) is not a continuous function.
n Thereforel look for discrete approximations to the gradient.
y
x
f(x,y) S
f
y
x
![Page 20: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/20.jpg)
3D Computer Vision
and Video Computing Discrete ApproximationsDiscrete Approximations
df(x)dx = lim
Dx 0
f(x + Dx) - f(x)Dx
df(x)dx
f(x) - f(x-1)1
@ xx-1
f(x)
Convolve with -1 1
![Page 21: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/21.jpg)
3D Computer Vision
and Video Computing
-1 1
In Two DimensionsIn Two Dimensions
n Discrete image function I
n Derivatives Differences
i
j
Image
row i-1
row i
row i+1
col j-1 col j col j+1
I(i-1,j-1)
I(i,j-1)
I(i+1,j-1)
I(i-1,j)
I(i,j)
I(i+1,j)
I(i-1,j+1)
I(i,j+1)
I(i+1,j+1)
1
-1DjI = DiI =
![Page 22: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/22.jpg)
3D Computer Vision
and Video Computing 1x2 Example1x2 Example
1x2 Vertical 1x2 Horizontal
Combined
![Page 23: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/23.jpg)
3D Computer Vision
and Video Computing Smoothing and Edge DetectionSmoothing and Edge Detection
n Derivatives are 'noisy' operationsl edges are a high spatial frequency phenomenonl edge detectors are sensitive to and accent noise
n Averaging reduces noisel spatial averages can be computed using masks
n Combine smoothing with edge detection.
1 1 1
1 1 1
1 1 1
1 / 9 x
1 1 1
1 1 1
1 1 1
1 1 1
1 0 1
1 1 1
1 / 8 x
![Page 24: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/24.jpg)
3D Computer Vision
and Video Computing Effect of BlurringEffect of Blurring
Original Orig+1 Iter Orig+2 Iter
Image
Edges
ThresholdedEdges
![Page 25: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/25.jpg)
3D Computer Vision
and Video Computing Combining the TwoCombining the Two
n Applying this mask is equivalent to taking the difference of averages on either side of the central pixel.
-1 -1 -1
11 1
0 0
Average
Average
![Page 26: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/26.jpg)
3D Computer Vision
and Video Computing Many Different KernelsMany Different Kernels
n Variablesl Size of kernell Pattern of weights
n 1x2 Operator (we’ve already seen this one
-1 11
-1DjI = DiI =
![Page 27: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/27.jpg)
3D Computer Vision
and Video Computing Roberts Cross OperatorRoberts Cross Operator
n Does not return any information about the orientation of the edge 22 ),1()1,()1,1(),( yxIyxIyxIyxI
or
1 00 -1
0 1-1 0+
| I(x, y) - I(x+1, y+1) | + | I(x, y+1) - I(x+1, y) |
[ I(x, y) - I(x+1, y+1) ]2 + [ I(x, y+1) - I(x+1, y) ]2S =
S =
![Page 28: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/28.jpg)
3D Computer Vision
and Video Computing Sobel OperatorSobel Operator
-1 -2 -1 0 0 0 1 2 1
-1 0 1-2 0 2 -1 0 1
S1= S2 =
Edge Magnitude =
Edge Direction =
S1 + S22 2
tan-1S1
S2
![Page 29: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/29.jpg)
3D Computer Vision
and Video Computing Anatomy of the SobelAnatomy of the Sobel
1 2 1
0 0 0
-1 -2 -1
-1 0 1 -2 0 2
-1 0 1
= 1/4 * [ 1 2 1]
1 0 -1
Sobel kernel is separable!
1
-1
-2
1
1
2 Averaging done parallel to edge
1/4
1/4
= 1/4 * [-1 0 -1]
1 2 1
+
![Page 30: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/30.jpg)
3D Computer Vision
and Video Computing Prewitt OperatorPrewitt Operator
-1 -1 -1 0 0 0 1 1 1
-1 0 1-1 0 1 -1 0 1
P1= P2 =
Edge Magnitude =
Edge Direction =
P1 + P22 2
tan-1P1
P2
![Page 31: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/31.jpg)
3D Computer Vision
and Video Computing Large MasksLarge Masks
1 x 2
1 x 5
1 x 9
1 x 9 uniform weights
-1 0 0 0 0 0 0 0 1
-1 -1 -1 -1 0 1 1 1 1
-1 0 0 0 1
-1 1
What happens as the mask size increases?
![Page 32: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/32.jpg)
3D Computer Vision
and Video Computing Large KernelsLarge Kernels
7x7 Horizontal Edges only
13x13 Horizontal Edges only
![Page 33: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/33.jpg)
3D Computer Vision
and Video Computing Compass MasksCompass Masks
n Use eight masks aligned with the usual compass directions
n Select largest response (magnitude)n Orientation is the direction associated with the largest
response
SE
E
NENNW
W
SE S
(-)
(+)
![Page 34: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/34.jpg)
3D Computer Vision
and Video Computing Many Different KernelsMany Different Kernels
1
111
1
-1 -1
Prewitt 1
-2
-1
-3
555
-3
-3 -3 -3
Kirsch
0
0
111
0
-1 -1 -1
Prewitt 2
0 0
121
0
-1 -2 -1
Sobel
0
-1 -1
11
0 0 0
- 2
2
Frei & Chen
![Page 35: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/35.jpg)
3D Computer Vision
and Video Computing Robinson Compass MasksRobinson Compass Masks
-1 0 1-2 0 2 -1 0 1
0 1 2-1 0 1 -2 -1 0
1 2 1 0 0 0 -1 -2 -1
2 1 0 1 0 -1 0 -1 -2
1 0 -1 2 0 -2 1 0 -1
0 -1 -2-1 0 -1 2 1 0
-1 -2 -1 0 0 0 1 2 1
-2 -1 0-1 0 1 0 1 2
![Page 36: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/36.jpg)
3D Computer Vision
and Video Computing Analysis of Edge KernelsAnalysis of Edge Kernels
n Analysis based on a step edge inclined at an angle q (relative to y-axis) through center of window.
n Robinson/Sobel: true edge contrast less than 1.6% different from that computed by the operator.
n Error in edge directionl Robinson/Sobel: less than 1.5 degrees errorl Prewitt: less than 7.5 degrees error
n Summaryl Typically, 3 x 3 gradient operators perform better than 2 x 2.l Prewitt2 and Sobel perform better than any of the other 3x3 gradient
estimation operators.l In low signal to noise ratio situations, gradient estimation operators of
size larger than 3 x 3 have improved performance.l In large masks, weighting by distance from the central pixel is
beneficial.
![Page 37: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/37.jpg)
3D Computer Vision
and Video Computing Prewitt ExamplePrewitt Example
Santa Fe Mission Prewitt Horizontal and Vertical Edges Combined
![Page 38: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/38.jpg)
3D Computer Vision
and Video Computing Edge ThresholdingEdge Thresholding
n Global approach
0
1000
2000
3000
4000
5000
Num
ber
of P
ixel
s
Edge Gradient Magnitude
64 128
T=64T=128
See Haralick paper for thresholding based on statistical significance tests.
Edge Histogram
![Page 39: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/39.jpg)
3D Computer Vision
and Video Computing Demo in PhotoshopDemo in Photoshop
- Go through slides 40-71 after class
- Reading: Chapters 4 and 5
- Homework 2: Due after two weeks
You may try different operators in Photoshop, but do your homework by programming … …
![Page 40: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/40.jpg)
3D Computer Vision
and Video Computing Canny Edge DetectorCanny Edge Detector
n Probably most widely usedn LF. Canny, "A computational approach to edge detection",
IEEE Trans. Pattern Anal. Machine Intelligence (PAMI), vol. PAMI vii-g, pp. 679-697, 1986.
n Based on a set of criteria that should be satisfied by an edge detector:l Good detection. There should be a minimum number of false
negatives and false positives. l Good localization. The edge location must be reported as
close as possible to the correct position. l Only one response to a single edge.
Cost function which could be optimized using variational methods
![Page 41: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/41.jpg)
3D Computer Vision
and Video Computing Canny ResultsCanny Results
I = imread(‘image file name’);BW1 = edge(I,'sobel');BW2 = edge(I,'canny');imshow(BW1)figure, imshow(BW2)
s=1, T2=255, T1=1
‘Y’ or ‘T’ junction problem with Canny operator
![Page 42: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/42.jpg)
3D Computer Vision
and Video Computing Canny ResultsCanny Results
s=1, T2=255, T1=220 s=1, T2=128, T1=1 s=2, T2=128, T1=1
M. Heath, S. Sarkar, T. Sanocki, and K.W. Bowyer, "A Robust Visual Method for Assessing the Relative Performance of Edge-Detection Algorithms" IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 19, No. 12, December 1997, pp. 1338-1359.
http://marathon.csee.usf.edu/edge/edge_detection.html
![Page 43: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/43.jpg)
3D Computer Vision
and Video Computing
n Second derivatives…
![Page 44: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/44.jpg)
3D Computer Vision
and Video Computing Edges from Second DerivativesEdges from Second Derivatives
n Digital gradient operators estimate the first derivative of the image function in two or more directions.
f(x) = step edge
1st Derivative f’(x)
2nd Derivative f’’(x)
maximum
zero crossing
GR
AD
IEN
TM
ET
HO
DS
![Page 45: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/45.jpg)
3D Computer Vision
and Video Computing Second DerivativesSecond Derivatives
n Second derivative = rate of change of first derivative.n Maxima of first derivative = zero crossings of second
derivative.n For a discrete function, derivatives can be approximated by
differencing.n Consider the one dimensional case:
..... f(i-2) f(i-1) f(i) f(i+1) f(i+2) .....
f(i-1) f(i) f(i+1) f(i+2)
f(i-1)2 2 f(i+1) f(i)2
D f(i) = D f(i+1) - D f(i)2
= f(i+1) - 2 f(i) + f(i-1)
-2 11Mask:
![Page 46: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/46.jpg)
3D Computer Vision
and Video Computing Laplacian OperatorLaplacian Operator
n Now consider a two-dimensional function f(x,y).n The second partials of f(x,y) are not isotropic.n Can be shown that the smallest possible isotropic
second derivative operator is the Laplacian:
n Two-dimensional discrete approximation is:
1
1 1
1
-4
2
2
2
22
y
f
x
ff
![Page 47: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/47.jpg)
3D Computer Vision
and Video Computing
-1 -1 24 -1 -1-1 -1 -1 -1 -1-1 -1 -1 -1 -1
-1 -1 -1 -1 -1-1 -1 -1 -1 -1
-1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 +8 +8 +8 -1 -1 -1 -1 -1 -1 +8 +8 +8 -1 -1 -1 -1 -1 -1 +8 +8 +8 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
Example Laplacian KernelsExample Laplacian Kernels
n Note that these are not the optimal approximations to the Laplacian of the sizes shown.
5X5
9X9
![Page 48: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/48.jpg)
3D Computer Vision
and Video Computing Example ApplicationExample Application
5x5 Laplacian Filter 9x9 Laplacian Filter
![Page 49: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/49.jpg)
3D Computer Vision
and Video Computing Detailed View of ResultsDetailed View of Results
![Page 50: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/50.jpg)
3D Computer Vision
and Video Computing Interpretation of the LaplacianInterpretation of the Laplacian
n Consider the definition of the discrete Laplacian:
n Rewrite as:
n Factor out -5 to get:
n Laplacian can be obtained, up to the constant -5, by subtracting the average value around a point (i,j) from the image value at the point (i,j)!l What window and what averaging function?
I = I(i+1,j)+I(i-1,j)+I(i,j+1)+I(i,j-1) - 4I(i,j)
looks like a window sum
I = I(i+1,j)+I(i-1,j)+I(i,j+1)+I(i,j-1)+I(i,j) - 5I(i,j)
I = -5 {I(i,j) - window average}
![Page 51: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/51.jpg)
3D Computer Vision
and Video ComputingEnhancement using the LaplacianEnhancement using the Laplacian
n The Laplacian can be used to enhance images:
n If (i,j) is in the middle of a flat region or long ramp: I-Ñ2I = In If (i,j) is at low end of ramp or edge: I-Ñ2I < In If (i,j) is at high end of ramp or edge: I-Ñ2I > I
n Effect is one of deblurring the image
I(i,j) - I(i,j) =
5 I(i,j) -[I(i+1,j) + I(i-1,j) + I(i,j+1) + I(i,j-1)]
![Page 52: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/52.jpg)
3D Computer Vision
and Video Computing Laplacian EnhancementLaplacian Enhancement
Blurred Original 3x3 Laplacian Enhanced
![Page 53: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/53.jpg)
3D Computer Vision
and Video Computing NoiseNoise
n Second derivative, like first derivative, enhances noise
n Combine second derivative operator with a smoothing operator.
n Questions:l Nature of optimal smoothing filter.l How to detect intensity changes at a given
scale.l How to combine information across multiple
scales.n Smoothing operator should be
l 'tunable' in what it leaves behindl smooth and localized in image space.
n One operator which satisfies these two constraints is the Gaussian:
![Page 54: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/54.jpg)
3D Computer Vision
and Video Computing 2D Gaussian Distribution2D Gaussian Distribution
n The two-dimensional Gaussian distribution is defined by:
n From this distribution, can generate smoothing masks whose width depends upon s:
e(x + y )2 2
2 2 2
1G(x,y) =
x
y
![Page 55: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/55.jpg)
3D Computer Vision
and Video Computing s Defines Kernel ‘Width’s Defines Kernel ‘Width’
s2 = .25 s2 = 1.0 s2 = 4.0
![Page 56: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/56.jpg)
3D Computer Vision
and Video Computing Creating Gaussian KernelsCreating Gaussian Kernels
n The mask weights are evaluated from the Gaussian distribution:
n This can be rewritten as:
n This can now be evaluated over a window of size nxn to obtain a kernel in which the (0,0) value is 1.
n k is a scaling constant
W(i,j) = k * exp (- )i2 + j2
2 s2
= exp (- )i2 + j2
2 s2
W(i,j)k
![Page 57: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/57.jpg)
3D Computer Vision
and Video Computing ExampleExample
n Choose s = 2. and n = 7, then:2
.011 .039 .082 .105 .082 .039 .011
.011 .039 .082 .105 .082 .039 .011
.039 .135 .287 .368 .287 .135 .039
.105 .039 .779 1.000 .779 .368 .105
.082 .287 .606 .779 .606 .287 .082
.082 .287 .606 .779 .606 .287 .082
.039 .135 .287 .368 .287 .135 .039
-3
-2
-1
0
1
2
3
-3 -2 -1 0 1 2 3
i
j
W(1,2)k
= exp(-12+ 222*2
) To make this value 1, choose k = 91.
![Page 58: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/58.jpg)
3D Computer Vision
and Video Computing ExampleExample
1 4 7 10 7 4 1
7 26 55 71 55 26 7
4 12 26 33 26 12 4
1 4 7 10 7 4 1
7 26 55 71 55 26 7
4 12 26 33 26 12 4
10 33 71 91 71 33 10
W(i,j) = 1,115j = -3
3
i = -3
3
7x7 Gaussian Filter
Plot of Weight Values
![Page 59: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/59.jpg)
3D Computer Vision
and Video Computing Kernel ApplicationKernel Application
7x7 Gaussian Kernel 15x15 Gaussian Kernel
![Page 60: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/60.jpg)
3D Computer Vision
and Video Computing Why Gaussian for SmoothingWhy Gaussian for Smoothing
n Gaussian is not the only choice, but it has a number of important propertiesl If we convolve a Gaussian with another Gaussian, the
result is a Gaussianu This is called linear scale space
l Efficiency: separablel Central limit theorem
![Page 61: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/61.jpg)
3D Computer Vision
and Video Computing Why Gaussian for SmoothingWhy Gaussian for Smoothing
n Gaussian is separable
![Page 62: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/62.jpg)
3D Computer Vision
and Video Computing Why Gaussian for Smoothing – cont.Why Gaussian for Smoothing – cont.
n Gaussian is the solution to the diffusion equation
n We can extend it to non-linear smoothing
![Page 63: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/63.jpg)
3D Computer Vision
and Video Computing Ñ2G FilterÑ2G Filter
n Marr and Hildreth approach:1. Apply Gaussian smoothing using s's of increasing size:
2. Take the Laplacian of the resulting images:
3. Look for zero crossings.
n Second expression can be written as:
n Thus, can take Laplacian of the Gaussian and use that as the operator.
G I*
(G I)*
(2G ) * I
![Page 64: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/64.jpg)
3D Computer Vision
and Video Computing Mexican Hat FilterMexican Hat Filter
n Laplacian of the Gaussian
n Ñ2G is a circularly symmetric operator.n Also called the hat or Mexican-hat operator.
-1 4
G (x,y) =2 2
(x + y )2 21 - e 2 2
(x + y )2 22
![Page 65: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/65.jpg)
3D Computer Vision
and Video Computing s2 Controls Sizes2 Controls Size
s2 = 0.5 s2 = 1.0 s2 = 2.0
![Page 66: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/66.jpg)
3D Computer Vision
and Video Computing KernelsKernels
n Remember the center surround cells in the human system?
0 0 -1 0 0
-1 -2 16 -2 -1 0 -1 -2 -1 0
0 -1 -2 -1 0
0 0 -1 0 0
0 0 0 0 0 0 -1 -1 -1 -1 -1 0 0 0 0 0 0 0 0 0 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 0 0 0 0
0 0 0 0 -1 -1 -1 -1 -1 -1 -1 -1 -1 0 0 0 0
0 0 -1 -1 -1 -2 -3 -3 -3 -3 -3 -2 -1 -1 -1 0 0
0 0 -1 -1 -1 -2 -3 -3 -3 -3 -3 -2 -1 -1 -1 0 0
0 0 -1 -1 -2 -3 -3 -3 -3 -3 -3 -3 -2 -1 -1 0 0
0 0 -1 -1 -2 -3 -3 -3 -3 -3 -3 -3 -2 -1 -1 0 0
0 -1 -1 -2 -3 -3 -3 -2 -3 -2 -3 -3 -3 -2 -1 -1 0 0 -1 -2 -3 -3 -3 0 2 4 2 0 -3 -3 -3 -2 -1 0
0 -1 -1 -2 -3 -3 -3 -3 -3 -3 -3 -3 -3 -2 -1 -1 0 0 -1 -2 -3 -3 -3 0 2 4 2 0 -3 -3 -3 -2 -1 0
-1 -1 -3 -3 -3 0 4 10 12 10 4 0 -3 -3 -3 -1 -1
-1 -1 -3 -3 -3 0 4 10 12 10 4 0 -3 -3 -3 -1 -1
-1 -1 -3 -3 -2 2 10 18 21 18 10 2 -2 -3 -3 -1 -1
-1 -1 -3 -3 -2 2 10 18 21 18 10 2 -2 -3 -3 -1 -1 -1 -1 -3 -3 -3 4 12 21 24 21 112 4 -3 -3 -3 -1 -1
5x5
17 x 17
![Page 67: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/67.jpg)
3D Computer Vision
and Video Computing ExampleExample
13x13 Kernel
![Page 68: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/68.jpg)
3D Computer Vision
and Video Computing ExampleExample
13 x 13 Hat Filter Thesholded Positive
Thesholded Negative Zero Crossings
![Page 69: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/69.jpg)
3D Computer Vision
and Video Computing Scale SpaceScale Space
17x17 LoG Filter Thresholded Positive
Thresholded Negative Zero Crossings
![Page 70: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/70.jpg)
3D Computer Vision
and Video Computing Scale SpaceScale Space
2 = 2 2 = 2
2 =2 2 2 = 4
![Page 71: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/71.jpg)
3D Computer Vision
and Video Computing Multi-Resolution Scale SpaceMulti-Resolution Scale Space
n Observations:l For sufficiently different s 's, the zero crossings will be
unrelated unless there is 'something going on' in the image.l If there are coincident zero crossings in two or more
successive zero crossing images, then there is sufficient evidence for an edge in the image.
l If the coincident zero crossings disappear as s becomes larger, then either:
u two or more local intensity changes are being averaged together, oru two independent phenomena are operating to produce intensity changes
in the same region of the image but at different scales.
n Use these ideas to produce a 'first-pass' approach to edge detection using multi-resolution zero crossing data.
n Never completely worked outn See Tony Lindbergh’s thesis and papers
![Page 72: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/72.jpg)
3D Computer Vision
and Video Computing Color Edge DetectionColor Edge Detection
n Typical Approachesl Fusion of results on R, G, B separately
l Multi-dimensional gradient methods
l Vector methodsl Color signatures: Stanford (Rubner and Thomasi)
![Page 73: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/73.jpg)
3D Computer Vision
and Video Computing Hierarchical Feature ExtractionHierarchical Feature Extraction
n Most features are extracted by combining a small set of primitive features (edges, corners, regions)
l Grouping: which edges/corners/curves form a group?u perceptual organization at the intermediate-level of vision
l Model Fitting: what structure best describes the group?
n Consider a slightly simpler problem…..
![Page 74: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/74.jpg)
3D Computer Vision
and Video Computing From Edgels to LinesFrom Edgels to Lines
n Given local edge elements:
n Can we organize these into more 'complete' structures, such as straight lines?
n Group edge points into lines?
n Consider a fairly simple technique...
![Page 75: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/75.jpg)
3D Computer Vision
and Video Computing Edgels to LinesEdgels to Lines
n Given a set of local edge elementsl With or without orientation information
n How can we extract longer straight lines?n General idea:
l Find an alternative space in which lines map to pointsl Each edge element 'votes' for the straight line which it
may be a part of.l Points receiving a high number of votes might
correspond to actual straight lines in the image.n The idea behind the Hough transform is that a change
in representation converts a point grouping problem into a peak detection problem
![Page 76: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/76.jpg)
3D Computer Vision
and Video Computing Edgels to LinesEdgels to Lines
n Consider two (edge) points, P(x,y) and P’(x’,y’) in image space:
n The set of all lines through P=(x,y) is y=mx + b, for appropriate choices of m and b.l Similarly for P’
n But this is also the equation of a line in (m,b) space, or parameter space.
P
P 'L
x
y
![Page 77: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/77.jpg)
3D Computer Vision
and Video Computing Parameter SpaceParameter Space
b
m
b = -mx+y
b’ = -m’x'+y'
x,y; x',y' are fixed
(m,b)
L 1
L2
n The intersection represents the parameters of the equation of a line y=mx+b going through both (x,y) and (x',y').
n The more colinear edgels there are in the image, the more lines will intersect in parameter space
n Leads directly to an algorithm
![Page 78: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/78.jpg)
3D Computer Vision
and Video Computing General IdeaGeneral Idea
n General Idea: l The Hough space (m,b) is a representation of every
possible line segment in the planel Make the Hough space (m and b) discretel Let every edge point in the image plane ‘vote for’ any
line it might belong to.
![Page 79: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/79.jpg)
3D Computer Vision
and Video Computing Hough TransformHough Transform
n Line Detection Algorithm: Hough Transforml Quantize b and m into appropriate 'buckets'.
u Need to decide what’s ‘appropriate’
l Create accumulator array H(m,b), all of whose elements are initially zero.
l For each point (i,j) in the edge image for which the edge magnitude is above a specific threshold, increment all points in H(m,b) for all discrete values of m and b satisfying b = -mj+i.
u Note that H is a two dimensional histogram
l Local maxima in H corresponds to colinear edge points in the edge image.
![Page 80: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/80.jpg)
3D Computer Vision
and Video Computing Quantized Parameter SpaceQuantized Parameter Space
n Quantization
b
single votes two votes
m
The problem of line detection in image space has been transformed into the problem of clusterdetection in parameter space
![Page 81: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/81.jpg)
3D Computer Vision
and Video Computing ExampleExample
n The problem of line detection in image space has been transformed into the problem of cluster detection in parameter space
Image Edges
AccumulatorArray
Result
![Page 82: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/82.jpg)
3D Computer Vision
and Video Computing ProblemsProblems
n Vertical lines have infinite slopesl difficult to quantize m to take this into account.
n Use alternative parameterization of a linel polar coordinate representation
r1
q1
q2
r2
y
x
r 1 x 1 cos y 1 sin +=
r = x cos q + y sin q
![Page 83: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/83.jpg)
3D Computer Vision
and Video Computing Why?Why?
n ( ,r q) is an efficient representation:l Small: only two parameters (like y=mx+b)l Finite: 0 £ r £ Ö(row2+col2), 0 £ q £ 2pl Unique: only one representation per line
![Page 84: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/84.jpg)
3D Computer Vision
and Video Computing Alternate RepresentationAlternate Representation
n Curve in ( ,r q) space is now a sinusoid l but the algorithm remains valid.
r
2 q
r 1 x 1 cos q y 1 sin q+=
r 2 x 2 cos q y 2 sin q+=
![Page 85: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/85.jpg)
3D Computer Vision
and Video Computing ExampleExample
x
y
P = (4, 4)
P = (-3, 5)
1
2
P2 P1
r
q
r 4 c 4 s+=
r 3 c 5 s+=
s 2 c 2+ 1=
s 750
50=
c 150
50=
1.4289=
r 4.5255=
Solve for r and
r 3 cos 5 sin +=
r 4cos 4 sin +=
Two Constraints
(r, q )(r, ) Space
![Page 86: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/86.jpg)
3D Computer Vision
and Video Computing Real ExampleReal Example
Image Edges
AccumulatorArray
Result
![Page 87: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/87.jpg)
3D Computer Vision
and Video Computing ModificationsModifications
n Note that this technique only uses the fact that an edge exists at point (i,j).
n What about the orientation of the edge?l More constraints!
n Use estimate of edge orientation as q.n Each edgel now maps to a point in Hough space.
Image
Origin is arbitrary
The three edges have same (r, q)
![Page 88: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/88.jpg)
3D Computer Vision
and Video Computing Gradient DataGradient Data
n Colinear edges in Cartesian coordinate space now form point clusters in (m,b) parameter space.
E1
E2
E3
L1L2
L3
b
L1
L2
L3
mm
![Page 89: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/89.jpg)
3D Computer Vision
and Video Computing Gradient DataGradient Data
n ‘Average’ point in Hough Space:
n Leads to an ‘average’ line in image space:
b
L1
L2
L3
m m
ba = -m
ax + y
Average line in coordinate space
![Page 90: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/90.jpg)
3D Computer Vision
and Video Computing Post HoughPost Hough
n Image space localization is lost:
n Consequently, we still need to do some image space manipulations, e.g., something like an edge 'connected components' algorithm.
n Heikki Kälviäinen, Petri Hirvonen, L. Xu and Erkki Oja, “Probabilistic and nonprobabilistic Hough Transforms: Overview and comparisons”, Image and vision computing, Volume 13, Number 4, pp. 239-252, May 1995.
both sets contribute to the same Hough maxima.
![Page 91: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/91.jpg)
3D Computer Vision
and Video Computing Hough FittingHough Fitting
n Sort the edges in one Hough clusterl rotate edge points according to ql sort them by (rotated) x coordinate
n Look for Gapsl have the user provide a “max gap” thresholdl if two edges (in the sorted list) are more than max gap
apart, break the line into segmentsl if there are enough edges in a given segment, fit a
straight line to the points
![Page 92: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/92.jpg)
3D Computer Vision
and Video Computing GeneralizationsGeneralizations
n Hough technique generalizes to any parameterized curve:
n Success of technique depends upon the quantization of the parameters:l too coarse: maxima 'pushed' togetherl too fine: peaks less defined
n Note that exponential growth in the dimensions of the accumulator array with the the number of curve parameters restricts its practical application to curves with few parameters
parameter vector (axes in Hough space)
f(x,a) = 0
![Page 93: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/93.jpg)
3D Computer Vision
and Video Computing Example: Finding a CircleExample: Finding a Circle
n Circles have three parametersl Center (a,b)l Radius r
n Circle f(x,y,r) = (x-a)2+(y-b)2-r2 = 0n Task:
n Given an edge point at (x,y) in the image, where could the center of the circle be?
Find the center of a circle with known radius r given an edge image with no gradient direction information (edge location only)
![Page 94: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/94.jpg)
3D Computer Vision
and Video Computing Finding a CircleFinding a Circle
(i-a)2+(j-b)2-r2 = 0
Imagefixed (i,j) Parameter space (a,b)
Parameter space (a,b)
Parameter space (a,b)
Circle Center(lots of votes!)
![Page 95: Introduction](https://reader038.vdocuments.net/reader038/viewer/2022110405/5681339b550346895d9aac2c/html5/thumbnails/95.jpg)
3D Computer Vision
and Video Computing Finding CirclesFinding Circles
n If we don’t know r, accumulator array is 3-dimensionaln If edge directions are known, computational
complexity if reducedl Suppose there is a known error limit on the edge
direction (say +/- 10o) - how does this affect the search?n Hough can be extended in many ways….see, for
example:l Ballard, D. H. Generalizing the Hough Transform to
Detect Arbitrary Shapes, Pattern Recognition 13:111-122, 1981.
l Illingworth, J. and J. Kittler, Survey of the Hough Transform, Computer Vision, Graphics, and Image Processing, 44(1):87-116, 1988