csc508 what you should be doing code, code, code –programming gaussian convolution sobel edge...
TRANSCRIPT
![Page 1: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/1.jpg)
CSC508
What You Should Be Doing
• Code, code, code– Programming
• Gaussian Convolution
• Sobel Edge Operator
![Page 2: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/2.jpg)
CSC508
Effects of Threshold Selection
![Page 3: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/3.jpg)
CSC508
Effects of Threshold Selection
• The preceding images were edge-detected with a Sobel operator then binarized with various thresholds
• The overall trend is predictable but the individual results are not
• This threshold dependency is the curse of computer vision
• We need smarter algorithms
![Page 4: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/4.jpg)
CSC508
Scale-Space Processing
• One method of dealing with the threshold dependency is through scale-space processing– An image is processed at various scales revealing
different sets of features at each scale– Features common to all [or many] sets are deemed
“important”
![Page 5: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/5.jpg)
CSC508
Gaussian Pyramid
• Selection of the “appropriate” parameters (sigma and width) of a Gaussian filter is analogous to setting a binarization threshold
• The Gaussian Pyramid helps in such a situation
![Page 6: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/6.jpg)
CSC508
Gaussian Pyramid Formation
1. Select a sigma and width
2. Filter the image
3. Subsample the image by removing every other line and sample– Resultant image is ¼ the size of the original
4. Repeat on the subsampled image
![Page 7: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/7.jpg)
CSC508
Gaussian Pyramid Formation
• You end up with a series of images (a pyramid) each exposing different features– Coarse features at the top of the pyramid– Fine features at the bottom
• You can now use the coarse features to select where to concentrate your processing of the fine features
![Page 8: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/8.jpg)
CSC508
Effects of Threshold Selection
• Another method of dealing with the threshold dependency is through more complex algorithms
![Page 9: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/9.jpg)
CSC508
A Better Kernel
• The Sobel mask is a derivative mask (computes the weighted difference of neighborhood pixels)
• So is the Laplacian
• From calculus: the 2nd derivative of a signal is zero when the derivative magnitude is extremal
![Page 10: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/10.jpg)
CSC508
A Better Edge Detector
• Combining these observations we can derive the Laplacian-Gaussian filter due to Marr and Hildreth– Also known as a DoG (Difference of Gaussians)
because it can be approximated by subtracting two Gaussian kernels of same width and differing sigmas
![Page 11: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/11.jpg)
CSC508
Laplacian-Gaussian Filter
• Bypassing the math and jumping to the results of applying the rules of calculus…
eyx yx
2
22
22
22
42
11
• Similar in form to the Gaussian kernel
![Page 12: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/12.jpg)
CSC508
Laplacian-Gaussian Filter
• In 1 dimension it looks like this…
Convolution Filter (1 Line)
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
0 2 4 6 8 10 12 14 16
![Page 13: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/13.jpg)
Laplacian-Gaussian Filter
• In 2 dimensions it looks like this…
CSC508
![Page 14: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/14.jpg)
CSC508
Laplacian-Gaussian Filter
• Convolution with this kernel results in edge magnitudes of both positive and negative values
• We’re interested in where the sign changes occur
• These are where the edges are (recall the calculus result)
![Page 15: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/15.jpg)
Zero-Crossing Detection
• For each pixel, look at it’s 4 previous (in scan line order) neighbors
• If the sign is different in the edge detected image, there is an edge at pixel I(i, j) in the original image
• Why don’t we look at the other 4 neighbors?
CSC508
+-- - -
i
j
-++ + +
i
j
![Page 16: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/16.jpg)
CSC508
Width = 15, Sigma = 1.6
![Page 17: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/17.jpg)
CSC508
Comparable Sobel vs. Laplacian-Gaussian
![Page 18: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/18.jpg)
CSC508
Width = 15, Sigma = 2.6
![Page 19: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/19.jpg)
CSC508
Comparable Sobel vs. Laplacian-Gaussian
![Page 20: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/20.jpg)
CSC508
Improvement
• Results of the Marr-Hildreth method are better than those of the Sobel/Binarization method– Edge placement accuracy– Removal of edges due to noise
• But, there’s still something missing…
![Page 21: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/21.jpg)
CSC508
What Else Can We Do?
• What are we ignoring in the following edge detection processes?
1.Gaussian filter
2.Sobel edge detection
3.Threshold based binarization of Sobel magnitude
And1.Laplacian-Gaussian filter
2.Zero crossing detection
![Page 22: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/22.jpg)
CSC508
Context!
• We merely selected each edge based on its magnitude and zero-crossing
• We completely ignored its immediate surroundings• Why is this bad?
– Detects edges we don’t want• Prone to noise
• Prone to clutter
– Doesn’t detect edges we do want• Weak magnitude edges are thresholded (thresheld?) away
• We need a smarter algorithm!
![Page 23: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/23.jpg)
CSC508
Canny Edge Detector
• John Canny – MIT 1986• Developed a process (series of steps) for
extracting edges from an image– Gaussian smoothing of input image– Basic derivative operator to detect edge candidates– Removal of edges that are clustered around a
single location– Adaptive thresholding algorithm for final edge
selection
![Page 24: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/24.jpg)
CSC508
Canny Edge Detector
• Gaussian smooth original image– Reduce the effects of noise– In the code I posted the implementation is
somewhat strange• Broken up based on filter size
• Some odd scaling goes on
– This is due to constraints of the original system on which the implementation was created
![Page 25: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/25.jpg)
CSC508
Canny Edge Detector
• Select two thresholds– The high threshold is used to identify large
magnitude (strong) edges• This is applied first
– The low threshold is used to identify small (weak) magnitude edges that are spatially connected to large (strong) magnitude edges
– My code accepts these as percentages of the total range of edge magnitudes
![Page 26: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/26.jpg)
CSC508
Canny Edge Detector
• Non-maximal Suppression– This process thins out clusters of edge points– Ideal step edges rarely occur in natural images
![Page 27: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/27.jpg)
CSC508
Non-Maximal Suppression
• Life would be grand if edges looked like this (1 dimension)
• But, in reality they usually look like this (1 dimension)
• This is why our Sobel magnitude images look so awful
![Page 28: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/28.jpg)
CSC508
Non-Maximal Suppression
• We need to employ an algorithm that eliminates all the weak edges (due to noise) and retains the strong edges (due to signal)
• The basic algorithm is given on page 180 of the textbook (this actually combines two steps)– The implementation in the posted code is
somewhat cryptic but follows the basic philosophy which is
![Page 29: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/29.jpg)
CSC508
Non-Maximal Suppression
• Find an intermediate or strong edge• Search a neighborhood along the direction of the
gradient (perpendicular to the edge) for stronger edges
• Eliminate those that are of lesser magnitude than the strongest found
• The result is the removal of [relatively] weak edges in the neighborhood of a [relatively] strong edge
• Notice that we have not applied any thresholds to this point in the process (other than Gaussian kernel parameters)
![Page 30: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/30.jpg)
Non-Maximal Suppression
• Assume the “boldness” of the arrow represents the magnitude of the edge
• Assume the direction of the arrow represents the direction of the edge (perpendicular to the direction of the gradient)
CSC508
![Page 31: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/31.jpg)
CSC508
Non-Maximal Suppression
![Page 32: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/32.jpg)
CSC508
Contour Following
• We now have a set of “reasonable” candidate edges– We used some local context to eliminate some
candidates (non-maximal suppression)
• Next we use global context to eliminate additional candidates
![Page 33: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/33.jpg)
CSC508
Contour Following
• Find a strong edge– Apply the high threshold– This isn’t so bad as we can set it very high with the
expectation that some real edges will meet this criteria
![Page 34: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/34.jpg)
CSC508
Strong Edges
![Page 35: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/35.jpg)
CSC508
Contour Following
• Compute the gradient direction of the edge– Similar to the computation done in Sobel
• Search for a neighboring edge in the direction of the edge (perpendicular to the gradient) that is above the weak threshold– We’re using a strong edge as an anchor point for a contour
of weak and strong edges – known as Hysteresis• Repeat the process until we have no more connected
weak edges• Find the next disconnected strong edge and repeat the
entire contour following process
![Page 36: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/36.jpg)
CSC508
Final Thinning
• An additional thinning step is performed
• This is similar to the non-maximal suppression step
• Just one last clean-up operation
![Page 37: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/37.jpg)
CSC508
Final Results
![Page 38: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/38.jpg)
CSC508
Canny Summary
All edges Weak edges
Strong edges Contours
![Page 39: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/39.jpg)
CSC508
Canny Summary
• Through more complex processing we arrive at a set of edges in which we have confidence– Minimal reliance on thresholds
• The cost is complex (slow, memory intensive) processing
• We must trade resources for accuracy!
![Page 40: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/40.jpg)
CSC508
Things To Do
• Reading for Next (few) Week(s)– Chapter 9 – Texture
• 9.1 – 9.3 describe one approach
• I’ll describe a second approach
– Chapter 14 – Segmentation by Clustering• We’ll consider various clustering methods
– Chapter 15 – Segmentation by Models• We’ll analyze the Hough Transform
![Page 41: CSC508 What You Should Be Doing Code, code, code –Programming Gaussian Convolution Sobel Edge Operator](https://reader035.vdocuments.net/reader035/viewer/2022070400/56649f0e5503460f94c225ec/html5/thumbnails/41.jpg)
CSC508
Things To Do
• Additions to current programming assignment– Use your Gaussian filter code to create a Gaussian Pyramid
of Lenna of 4 levels• Resultant image sizes: (512x512), (256x256), (128x128), (64x64)
– Use your Sobel filter code to create edge images of the Gaussian Pyramid images
– Turn in your resultant image files– Comment on the results