Exploring Methods to Improve Edge Detection withCanny Algorithm
A.Mehta, H.Lehri, P.Thakur & S.TambeJanuary 2015
1 Introduction 1.1 Image as a Signal1.2 Theory of Edge Detection1.3 Edge Detector Types
1.3.1 Gradient Based Edge Detection1.3.2 Second Ordered Derivative Based Edge Detection1.3.3 The Canny Edge Detector
1.4 Problems with Standard Edge Detection Method2 Literature Survey 3 Methods Explored
3.1 Recursive Method for Edge Detection3.1.1 Basics of Recursion3.1.2 Using Recursion with Canny Algorithm
3.2 Edge Filtering by determining Conditional Probability of a point being an Edge3.2.1 Polynomial Regression3.2.2 Edge Linking3.2.3 Algorithm for the Edge Filtering Technique
3.3 Results4 Future Work 5 Conclusion
CONTENT
INTRODUCTION
Edge detection is the process of finding the set of pixels that represent the boundary of disjoint regions in an image.
Edge detection
INTRODUCTIONEdge detection is important for image segmentation, since many image processing algorithms are first required to identify the objects and then process them.
Image Medical Imaging
Finger print detection
Face recognition
Traffic control
processingpre-processing
Edge detection
INTRODUCTION• Edge detection is not without its challenges• The image on which the detection is carried out has noise, due to random
variation of brightness, color or electronic noise likeGaussian noise, Salt pepper noise, Shot noise, Gradient noise etc..
• Noise leads to false edge detection
Gradient noise = False edges
Image as a Signal
• An Image is described as a function of x and y• A captured image must be treated as a discrete signal rather than a
continuous functions since the image can be recorded on a 2D grid• The conversion is done by a process called sampling• Uniform rectangular sampling is the most common
Original Image Spectrum Sampled Image Spectrum
Ld(n, m) = L(n∆x, m∆y)
sinc(x) = sinc(x)/x
introduction
Theory of Edge Detection• Difficulty: Changes in intensity occur over a wide range of scales in
an image and no single filter can optimally reduce noise at allscales
• Reason: Physical phenomena that give rise to intensity changes like illumination and surface reflectance are spatially localized
• Uncertainty Principle: The two requirements lie in different domains, we cannot achieve both at the same time
SpatialDomain
FrequencyDomain
FourierTransform
InverseFourier
Transform
IlluminationSurface Reflectance
introduction
Theory of Edge Detection
• Possible solution• Take local averages of the image at various
resolutions and then detect the changes in intensity that occur at each of the resolutions.
• Determine the optimal smoothing filter to reduce the range of scales over which the intensity changes take place
• There is only one distribution that optimizes this relationship• Gaussian distribution:
=
introduction
Intensity changes can be detected by either finding the maxima or minimaof the first derivative or finding zero crossings in the second derivative of the imageHence we can write it as:
f(x, y) = ( G(x, y) I∗ (x, y) )Gaussian
Kernel Image
Second order directional derivative
Theory of Edge Detection introduction
Edge Detector Types
• Gradient BasedDetects edges by looking for the maxima or minima of the first derivate of the image
• Laplacian basedDetect edges by searching for the zero crossing of the second derivate of the image
introduction
Edge
Edge
Edge Detector Types
Gradient Based Edge Detection
Roberts Operator
Sobel Operator
Prewitt Operator
Filter out noise
Convolve Image inhorizontal & verticalDirection using gradientoperators
Gaussian Operators used
For all pixels If magnitude>threshold then pixel=edgeElse pixel edge
introduction
Edge Detector Types
Second Order Derivative Based Edge Detection
Filter out noise
Convolve Image inusing LaplacianKernel
Laplacian of Gaussian matrix
For all pixels If then pixel=edgeElse pixel edge
Marr Hildrith Operator
• Isotropic in nature • Utilizes only one mask• Cheaper to implement
introduction
Edge Detector TypesThe Canny Edge Detector
Remove noiseusing Gaussianfilter
first order derivativesof the Image using operatorslike Sobel operator
For every pixelcalculateNon Maximalsuppression
For every pixelPerformhysteresis thresholding
The process
Canny is Optimal because:• Less sensitive to noise• It removes streaking by using two thresholds • Offers good localization of edges and utilizes gradient of the
edge to generate thin, one-pixel wide edges
introduction
Illumination when the image is taken
introductionProblems with Standard Edge Detection Method
(x-1,y+1) (x,y+1) (x+1,y+1)
(x-1,y) (x, y) (x+1,y)
(x-1,y-1) (x,y-1) (x+1,y-1)
Intensity difference between two neighboring pixels
Edge Detection Depends on
Smoothing to filter out noise also spreads the edges
introductionProblems with Standard Edge Detection Method
Edge Detection Depends on
Edge linking that may lead to detection of false edges
Difficult to generalize themethod of thresholding
introductionProblems with Standard Edge Detection Method
How Canny deals with these problems
Canny EdgeDetection
• Adaptively sensitive to digital noise• Detects sharper and thinner edges• Used as a benchmark
LITERATURE SURVEYModern Non-Canny Approaches
Wavelet transform
• Similar to Canny’s Kernel• Selectivity is high• Difficult to select Wavelet functions
Mathematical Morphology
• Deng Caixia et al[13]. proposed a technique, wherein they use 2 structural elements, one of size 3x3 and the other of size 5x5 and orient them along different directions
• Gives them two edge setsthat fuse into edge image
LITERATURE SURVEYResearch on Canny Edge Detector
T Rupalatha et al[15] & Qian Xu et al
Distributed Canny Edge detector on Field Programmable Gate Array(FPGA) based architecture.
Ogawa K
Canny Edge Detection algorithm on CUDA(Compute Unified Device Architecture)
LITERATURE SURVEYResearch on Canny Edge Detector
Philip Worthington
Canny Edge detector using Curvature Consistency
Ping Zhou et al[5]
Canny Edge Detection algorithm on Otsu method for Adaptive Thresholding
LITERATURE SURVEYOther Approaches
Aleksandar Jevtic and Bo Li[7
Ant Based Approach for adaptive edge detection
Piotr Dollar and C.Lawrence Zitnick[10]
Edge detection algorithm using Structured Forests
METHODS EXPLOREDRecursive Method for Edge Detection
Original Image Canny output for 300 x 300
Canny output for 400 x 400 Canny output for 600 x 600
Our Observation - Canny seems to work better with smaller size images
Our Approach –• Recursively divide the image into
sub-images• Apply Canny on sub-images• Merge sub-outputs to obtain
result
METHODS EXPLOREDRecursive Method for Edge Detection
Our Observation - Canny seems to work better with smaller size images
Mathematical Reasoning
NxN ImageW
Gaussian FilterKernel
(N −W2 )(N −W2 )Convolutions required
METHODS EXPLOREDRecursive Method for Edge Detection
Our Observation - Canny seems to work better with smaller size images
Mathematical Reasoning
WGaussian Filter
Kernel
(S−W2 )(S−W2 )Convolutions required per block
Block size
S
METHODS EXPLOREDRecursive Method for Edge Detection
Our Observation - Canny seems to work better with smaller size images
Mathematical Reasoning
We have, such blocks
Hence total number of convolutions
S > 1 number of operations have decreased
METHODS EXPLOREDBasics of Recursion
f(x)=
Problem can be divided into multiple independent problems(Image) unlike dynamic programming. Recursion is part of the Divide and Conquer strategy. Recursion is usually of the following format:
T(n) = a ∗ T+ O()Time complexity of recursive algorithms is give as
The solution to this problem is given by the Master’s Theorem as:
T(n) =
METHODS EXPLOREDUsing Recursion with Canny Algorithm
EDGEDETECTION (Image)
1. Divide the image into four regions into , i {0, 1, 2, 3}
2. For Each Region do:
a) IF sizeof is less than Base Case:
i. Filter and Perform Canny Edge Detection on it
b) Else Call EDGEDETECTION()
3. Join the results obtained from each region
4. Return the joint Image
METHODS EXPLOREDEdge Filtering by determining Conditional Probability of a point being an Edge
Noise and
Intensity
changesInput image
CannyNo
unnecessary edges
Edge detected image
Edge FilteringBy determining conditional probability of points being on an edge
Post Processing
Noise and
Intensity
changesInput image
CannyIrrelevant
AndSmalledges
Edge detected image
Our Observation
Our Approach
METHODS EXPLOREDEdge Filtering by determining Conditional Probability of a point being an Edge
noise and intensity changes
Input image
Canny
For every pixel detected by the Canny algorithm:1. Analyze Surrounding 2. Measure Order in the neighborhood of the
pixels• Determine if the pixels can be aligned
along a particular line using regression fitting
• Get conditional Probability of pixel falling on the line
3. Probability whether the pixel is part of an edge or not
4. Filter out pixels based on certain threshold
Post-Processing
No unnecessary
edges
Edge detected image
METHODS EXPLOREDEdge Linking
In Global Processing the voting is done by points on a curve
y = mx + c or c = -xm+yc and m are Variablesx and y are Parameters
To find he straight line that best fits given n-points, we can work in the m-c domain instead of the x-y domain and apply Hough transform.
The point through which maximum number of lines pass through gives the m and c of the best fitting line.
METHODS EXPLOREDEdge Linking
In localized algorithm for each edge point, we find the approximate direction of the edge in one of the eight directions (0o, 45o, 90o, 135o, 180o, 225o, 270o, 315o)
Canny List L = Edges Endpoints While is not empty• Analyze the 8-
neighborhood of L0 and find the direction (from the 8 directions) of the edge
• If reflection(ir,jr) of Lk(i,j) is not in the list L:• Add (ir,jr) to List L• Mark (ir,jr) as white
• Remove Lk from List L
END
METHODS EXPLOREDEdge Linking
Algorithm for Edge Filtering Technique
Canny For each point detected as an edge by Canny algorithm:
1. Analyze the neighborhood around the point. Using regression, determine a line ax+b which best fits all the points in the neighborhood which are white. Once we determine a and b, we find the number of points (x,y) for which y = ax+b. We then compute Probability P(i,j).
2. Determine the probability Pr. If Pr > 0.5 the we retain (i,j) else we set pixel (i,j) as black
Remove small edges and
perform edge linking
RESULTS
OriginalImage
Canny
RecursiveCanny
Edge Filtering
OriginalImage Cann
y
RecursiveCanny
Edge Filtering
OriginalImage Cann
y
RecursiveCanny
Edge Filtering
ANALYSIS OF RESULTSCanny Recursive Canny
Region 1
Region 2
Region 1
Region 2
Good Hysteresis linking preserves edges Reduced scale of Hysteresis linking doesn't preserves some edges
Recursive Canny helps us analyze Canny without hysteresis linking
ANALYSIS OF RESULTSCanny Recursive Canny
Region 1
Region 2
Bad Hysteresis linking preserves noise Reduced scale of Hysteresis linking doesn't preserves noise
Region 1
Region 2
Because linking at lower scales gives fewer false edges than linking on a large scaleTrade off between noise filtering and true edge filtering
ANALYSIS OF RESULTSEffects of variable kernel size
Kernel size=5
Kernel size=7
Kernel size=9
Kernel size=11
ObservationsOutlines (important edges) become more significant as kernel size increasesFalse Edges decrease with increase in kernel sizeLoss of Actual Edges after size = 7
ANALYSIS OF RESULTS
Effects of variable recursion base case (block) size
Block size=8
Block size=16 Block size=32 Block size=64
Jittery and unsmooth output
Approaches to Regular Canny
ANALYSIS OF RESULTS
Optimal Kernel and Block size
block size of 32 and kernel size of 5 improves the quality of output generated.• Larger block size the performance suffers• Smaller block size the recursion leads to a
jittery output with the edges appearing mismatched.
• Larger kernel size smoothen the image, but it increases the time taken for convolving the image with the kernel and hence may not b e useful for practical purposes.
ANALYSIS OF RESULTS
Canny Probabilistic Edge Filtering
Region 1
Region 2
Region 1
Region 2
• No Significant differences since it uses original Canny image used for edge linking• Marked regions are maintained since they satisfactorily fit on a line unlike in recursive canny• Noise density is reduced to some extent
ANALYSIS OF RESULTS
Canny Probabilistic Edge Filtering
Noise removal depends on minimum edge length and hence not a fault of refinement process
Region 1
Region 2
Region 1
Region 2
ANALYSIS OF RESULTSEffects of variable length thresholds
Canny Output
Length threshold=1 Length threshold=5
Length threshold=10
Significant-Edge Retention
• However after linking the edges result is similar to Canny• This is a fault of the linking algorithm as the original Canny is
used for linking broken edges
ANALYSIS OF RESULTSEffects of variable probability thresholds
Canny Output
Probability threshold= 0.5
Probability threshold= 0.6
Probability threshold=0.7
Strong step edges (lines)are retained always, showing the algorithm returns high probability for nearly straight lines
Number of edges decreases
ANALYSIS OF RESULTSEffects of variable truth values
Canny Output
Canny Truth value = 0.4
Canny Truth value = 0.6
Canny Truth value = 0.7
True Edges are lost
False edges increase• Strong step edges (lines)are not retained• Because we assume the output of the Canny algorithm itself has very small chance of being true and hence reduces
the overall probability
ANALYSIS OF RESULTSEffects of variable window size
Canny Output
Window Size = 4 Window Size = 14 Window Size = 18
Ineffective neighborhood judgment
Increased processing time & lines integrate to curves
ANALYSIS OF RESULTS
Optimal probability threshold and truth value and window size
probability threshold = 0.5truth value = 0.7window size = 6
• Configuration must be chosen carefully • Absolute Optimal configuration is not
possible
FUTURE SCOPE
• Applying Tsalli’s model• Color Space Research• Machine Learning Approach• Adaptive Thresholding Using
Image Histograms• 2D Entropy and Probability
application• Texture Analysis
Thank You