syde 575: introduction to image processing image enhancement by smoothing textbook sections 3.4 –...
TRANSCRIPT
SYDE 575: Introduction to Image Processing
Image Enhancement by Smoothing
Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2
Noise Reduction by Image Averaging
We want to be able to reduce the amount of noise in an image
Assume that the noise process has a mean of zero
One approach is to take many images f(x,y) of the same scene and use these to generate a noise-free image g(x,y)
Noise Reduction by Image Averaging
Assume noise is additive and Gaussian distributed with zero mean
~ (0, )n N s Suppose we take the average of q number
of noise samples at a point in the image
1 q
kk i
m nq =
= å
1 2, , ...., qn n n
Noise Reduction by Image Averaging
Now, apply concept to each pixel in the whole image
Given an infinite number of noise samples, the average approaches the mean of the distribution, which in this case is 0
1 1
1 1( , ) ( , ) ( , )
q q
kk k
g x y f x y n x yq q= =
= +å å
1
1A s , ( , ) 0 , ( , ) ( , )
q
kk
q n x y g x y f x yq =
® ¥ ® ®å
Example
Source: Gonzalez and Woods
Noise Reduction by Spatial Filtering
Image averaging takes advantage of information redundancy from the individual images to reduce noise
Not always possible to acquire so many images!
Images may not be perfectly registered so errors due to spatial misalignment
Alternative option: Take advantage of information redundancy from different pixels within the same image to reduce noise
Spatial Filtering - Convolution
2D discrete convolution
output input point spread function (2d) or impulse response (1d)
[ , ] [ , ] [ , ] [ , ] [ , ]j i
g m n f m n h m n f i j h m i n j¥ ¥
=- ¥ =- ¥
= * = × - -å å
Impulse response h[m,n] can be viewed as a spatial filter for an input image f[m,n] to produce output image g[m,n]
Spatial Filtering
Source: Gonzalez and Woods
Let us represent w(x,y) as a 2D convolution mask
( 1) / 2 ( 1) / 2
( 1) / 2 ( 1) / 2
( , ) ( , ) ( , )m n
s m t n
g x y w s t f x s y t- -
=- - =- -
= + +å å
Spatial Filtering
Spatial filtering of an image f with 2D convolution using symmetrical mask w of size m x n can be expressed as
output inputconvolution mask; the “system”
Spatial Filtering
Source: Gonzalez and Woods
Local Average Filter for Smoothing
Instead of averaging between images, we can average neighboring pixels using the following point spread function (PSF)
Recall similar 1-d mask [1/3 1/3 1/3] Why is the 1/9 required?
Averaging Filter
Source: Gonzalez and Woods
Removes noise but also blurs
Properties of Averaging Filter
• Linear?• Shift Invariant?• Memory?• Causal?• Stable?• Invertible?
Weighted Average Filter
Problem: Simple averaging of neighboring pixels lead to over-smoothing
Possible solution: Instead of weighting all neighboring pixels equally, assign higher weights to pixels that are closer to the input pixel
Weighted Averaging Filter: Example
Source: Gonzalez and Woods
Weighted Averaging Filter: Example
Noisy Average WeightedAverage
Gaussian Smoothing
• A weighted average can be produced using a Gaussian as a weight
• In 1d
• In 2d
1D Gaussian
Source: Wikipedia
2d Gaussian
Source: Wikipedia
Characteristics of Local Averaging
a) h(m,n) >= 0 for all (m,n)
b) S h(m,n) = 1 (DC gain is 1)
c) Non-causal
d) Typically odd-dimensions (practical)
e) Even symmetry
Order-Statistic Filters
Nonlinear spatial filters, i.e.,
Best known example: median filter
a1f1 (m,n) + a2f2 (m,n) ≠ a1g1 (m,n) + a2g2 (m,n)
Median Filter
Provides good noise reduction for certain types of noise such as impulse noise
Considerably less blurring than weighted averaging filter
Forces a pixel to be like its neighbors
Steps Order pixels within an area Replace value of center pixel with median
value (half of all pixels have intensities greater than or equal to the median value)
Median Filter: Example
255 10 9
10 255 10
8 10 10
255109 10 10108 10 255
255 10 9
10 10
8 10 10
10
median=10
Median Filter: Example
Source: Gonzalez and Woods
S-statistic Filter
• Another method to avoid impact of outliers on smoothing is the S-statistic filter
• Basically, only use those values within a region that are within a certain range relative to the mean
• Typically, this is done by calculating the m and s of the local region and determining a local average based only on pixels within a certain number of s
Recap: Spatial smoothing filters
All of the filters discussed so far are spatial smoothing filters
Weight of each pixel in the neighborhood covered by the filter depends on the proximity of the pixel to the center pixel being filtered Typically, the closer the pixel is to the
center pixel, the higher the weight
Range Smoothing Filters
Problem: oversmoothing of edges and other fine image detail using ideal and Gaussian filters
Alternative solution: range smoothing filters Weight of each pixel in the neighborhood covered by the
filter depends on the similarity of the pixel's intensity value to that of the center pixel being filtered The closer the pixel's intensity value is to that of the
center pixel, the higher the weight
Bilateral Filtering
Problem: range smoothing filter simply remaps intensity values No notion of space Poor noise reduction performance (in fact, not
really useful by itself) Idea: Combine spatial smoothing filters with range
smoothing filters! Good noise reduction (Spatial smoothing) Good edge and detail preservation (Range
smoothing)
Bilateral Filtering
Resulting filter is non-linear
( ( , ) ( , ) ) ( , )bf c c c ch h f x y f x y h x x y y= - - -
Range smoothing filter Spatial smoothing filter
Example
Suppose we use Gaussian models for the range and spatial smoothing filters
Source: Tomasi et al. 1998
Uses: Noise Suppression
noisy Gaussian spatial filter Bilateral filter
Uses: Special Effects
Bilateral filter reduces small image details while preserving large edge details
What would happen if we apply bilateral filtering multiple times? More and more smaller details get smoothed out Large edges remain well-preserved
Result: cartoon-like image
Uses: Special Effects
original 5 iterations
Implementing Symmetrical 2-D Filters
• Large images in use today (e.g., 1k x 1k, 5k x 5k, and even larger) require increased computational requirements
• Symmetrical filters are commonly used in image processing systems
• There is a faster way to implement a 2-d symmetrical mask using 2 1-d arrays
Example: 2d as 2 x 1d
1 1 1
1 1 1
1 1 1
* 1/9
1 2 3 4
1 1 1 1
1 2 2 3
1 1 1 1
“Image” Smoothing Mask – Note Symmetry
Representing 2d Mask as 2 x 1d
1 1 1
1 1 1
1 1 1
1 1 1
1/9 * 1/3
1
1
1
= 1/3
*
Top row and bottom row are equivalent
1 2 3 4
1 1 1 1
1 2 2 3
1 1 1 1
1 2 3 4
1 1 1 1
1 2 2 3
1 1 1 1
*
1 1 1
1 1 1
1 1 1
1 1 1 *1/3
1
1
1
1/3
- - - -
- 14 19 -
- 11 15 -
- - - -
= same
= 1/91/9
Computational Complexity
• Assume M by N image and m by n filter mask• For 2d mask, number of multiplications is:
M x N x m x n
• For 2 1d masks, number of multiplication is
M x N x (m + n)
• So, for a 15 by 15 mask, 2d implementation is 225MN and 1d implementation is 30MN
• Huge Savings! But note that 1d implementation requires storing an additional image in memory