image processing ib paper 8 – part a ognjen arandjelović ognjen arandjelović oa214
TRANSCRIPT
![Page 1: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/1.jpg)
Image ProcessingImage ProcessingIB Paper 8 – Part AIB Paper 8 – Part A
Ognjen ArandjeloviOgnjen Arandjelovićć
http://mi.eng.cam.ac.uk/~oa214/
![Page 2: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/2.jpg)
Lecture RoadmapLecture Roadmap
Face geometry Lecture 1:
Geometric image transformations
Lecture 2:
Colour and brightness enhancement
Lecture 3:
Denoising and image filtering
![Page 3: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/3.jpg)
– – Image Denoising and Filtering –Image Denoising and Filtering –
![Page 4: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/4.jpg)
Image Noise SourcesImage Noise Sources
Image noise may be produced by several sources:
Quantization
Photonic
Thermal
Electric
![Page 5: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/5.jpg)
DenoisingDenoising
To effectively perform denoising, we need to consider the following issues:
Signal (uncorrupted image) modelTypically piece-wise constant or linear
Noise model (from the physics of image formation)Additive or multiplicative, Gaussian, white, salt and pepper…
![Page 6: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/6.jpg)
Salt and Pepper NoiseSalt and Pepper Noise
![Page 7: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/7.jpg)
Gaussian NoiseGaussian Noise
![Page 8: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/8.jpg)
Modelling NoiseModelling Noise
Most often noise is additive:
Observed pixel luminance
True luminance Noise process
![Page 9: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/9.jpg)
Additive Gaussian Noise – ExampleAdditive Gaussian Noise – Example
A clear original image was corrupted by additive white Gaussian noise:
Original, uncorrupted image Additive Gaussian noise
![Page 10: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/10.jpg)
Additive Gaussian Noise – ExampleAdditive Gaussian Noise – Example
A clear original image was corrupted by additive white Gaussian noise:
Additive Gaussian noise Additive Gaussian noise
![Page 11: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/11.jpg)
Additive Gaussian Noise – ExampleAdditive Gaussian Noise – Example
Taking a slice through the image can help us visualize the behaviour of noise better:
![Page 12: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/12.jpg)
Temporal Average for Video DenoisingTemporal Average for Video Denoising
A video feed of a static scene can be easily denoised by temporal averaging, under the assumption of zero-mean additive noise:
Pixel luminance estimate
Pixel luminance in frame i
Average noise energy is reduced by a factor of N:
![Page 13: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/13.jpg)
Temporal Averaging – ExampleTemporal Averaging – Example
Consider our noisy CCTV image from the previous lecture and the result of brightness enhancement:
Original image Brightness enhanced image
![Page 14: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/14.jpg)
Temporal Averaging – ExampleTemporal Averaging – Example
The effect of temporal averaging over 100 frames is dramatic:
But note that moving objects cause blur.
The clarity of image detail is much improved.
![Page 15: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/15.jpg)
Spatial AveragingSpatial Averaging
Although attractive, a static video feed is usually not available. However, a similar technique can be used by noting:
Images are mostly smoothly varying
Original smoothly varying signal and the signal corrupted with zero mean Gaussian noise
![Page 16: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/16.jpg)
Simple Spatial AveragingSimple Spatial Averaging
Thus, we can attempt to denoise the signal by simple spatial averaging:
The result of averaging each neighbouring 7 (± 3) pixels
![Page 17: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/17.jpg)
Simple Spatial Averaging – ExampleSimple Spatial Averaging – Example
Using out synthetically corrupted image:
Additive Gaussian noise Spatially averaged using 5 х 5 neighbourhood
![Page 18: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/18.jpg)
Simple Spatial Averaging – ExampleSimple Spatial Averaging – Example
Consider the difference between the uncorrupted image and the corrupted and denoised images:
Before averaging After averaging
RMS difference = 29 RMS difference = 12
![Page 19: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/19.jpg)
Simple Spatial Averaging – AnalysisSimple Spatial Averaging – Analysis
The result of averaging looks good, but a closer inspection reveals some loss of detail:
Difference image Magnified patch
![Page 20: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/20.jpg)
Simple Spatial Averaging – AnalysisSimple Spatial Averaging – Analysis
To formally analyze the filtering effects, rewrite the original averaging expression:
Rectangular pulseConvolution integral
![Page 21: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/21.jpg)
1D Convolution1D Convolution
A quick convolution re-cap:
f(x) h(x)
Flip and slide over
![Page 22: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/22.jpg)
Discrete 1D ConvolutionDiscrete 1D Convolution
In dealing with discrete signals:
Flip and slide over
… 234 233 228 240 241 241 … 0 0 1 2 2 1
f(x)
h(x)
… 234 233 228 240 241 241 …
1 2 2 1 0 01 2 2 1 0 0
228+ 480+ 482+ 241 + …
![Page 23: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/23.jpg)
2D Convolution2D Convolution
The concept of linear filtering as convolution with a filter (or kernel) extends to 2D and the integral becomes:
We shall be dealing with separable filters only
in which this is equivalent to two 1D convolutions:
![Page 24: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/24.jpg)
Simple Spatial Averaging – AnalysisSimple Spatial Averaging – Analysis
By considering the effects of convolution in the frequency domain, we can now see why there was loss of detail:
Rectangular pulse function
Fourier transform
The sinc function
High frequencies are damped
![Page 25: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/25.jpg)
White Noise ModelWhite Noise Model
This insight allows to devise the denoising filter in a principled way by considering the SNR over different frequencies:
Signal frequency spectrum
Noise frequency spectrum
Frequency
Ene
rgy
![Page 26: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/26.jpg)
Frequency
Ene
rgy
White Noise ModelWhite Noise Model
This insight allows to devise the denoising filter in a principled way by considering the SNR over different frequencies:
Pass Do not pass
![Page 27: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/27.jpg)
The Ideal LPF AgainThe Ideal LPF Again
As when we dealt with reconstructing a signal from a set of samples, we can low-pass filter by convolving with the sinc function in the spatial domain:
The key limitation is that the sinc function has a wide spatial support
Thus, in practice we often use filters that offer a better trade-off in terms of spatial support and bandwidth
![Page 28: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/28.jpg)
Gaussian Low Pass FilterGaussian Low Pass Filter
The Gaussian LPF is one of the most commonly used LPFs. It possesses the attractive property of minimal space-bandwidth product.
1D Gaussian 2D Gaussian as a surface
2D Gaussian as an image
![Page 29: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/29.jpg)
Gaussian LPF – Toy ExampleGaussian LPF – Toy Example
Using the Gaussian filter on our toy 1D example produces a nearly perfect filtering result:
RMS error reduction from 0.1 to 0.02
![Page 30: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/30.jpg)
Gaussian LPF – ExampleGaussian LPF – Example
Using out synthetically corrupted image:
Additive Gaussian noise LP filtered using a Gaussian with
![Page 31: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/31.jpg)
Low, Band and High-Pass FiltersLow, Band and High-Pass Filters
A quick recap of relevant terminology:
Frequency
Ga
in
Low-pass Band-pass High-pass
![Page 32: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/32.jpg)
Low, Band and High-Pass FiltersLow, Band and High-Pass Filters
A summary of main uses:
Low-pass: denoising
High-pass: removal of non-informative low frequency components
Band-pass: combination of low-pass and high-pass filtering effects
![Page 33: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/33.jpg)
Gaussian High-Pass FilterGaussian High-Pass Filter
A high pass filter can be simply constructed from the Gaussian LPF:
Convolution with the delta function leaves the function unchanged
High-pass filter Low-pass filter
![Page 34: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/34.jpg)
Gaussian HPF – Toy ExampleGaussian HPF – Toy Example
Consider the effects of high pass filtering our 1D toy example:
Original signal High-pass filter output
The result is not dependent on the signal mean
Maximal responses around discontinuities
![Page 35: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/35.jpg)
Gaussian HPF – ExampleGaussian HPF – Example
Consider the effects of high pass filtering an image:
Original image High-pass filtered image
Information rich intensity discontinuities are extracted.
![Page 36: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/36.jpg)
High Frequency Image ContentHigh Frequency Image Content
An example of the importance of high-frequency content:
?
+Low-pass filter High-pass filter
![Page 37: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/37.jpg)
High Frequency Image ContentHigh Frequency Image Content
And the result of the experiment is…
![Page 38: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/38.jpg)
HPFs in Face Recognition HPFs in Face Recognition
High-pass filters are used in face recognition to achieve quasi-illumination invariance:
Original image of a localized face
High-pass filtered
![Page 39: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/39.jpg)
Filter Design – Matched FiltersFilter Design – Matched Filters
Consider the convolution sum of a discrete signal with a particular filter:
When is the filter response maximal?
… 234 233 228 240 241 241 …
1 2 2 1 0 01 2 2 1 0 0
228+ 480+482+ 241 + …
![Page 40: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/40.jpg)
Filter Design – Matched FiltersFilter Design – Matched Filters
The summation is the same as for vector dot product:
The response is thus maximal when the two vectors are parallel i.e. when the filter matches the local patch it overlaps.
… 234 233 228 240 241 241 …
1 2 2 1 0 0
![Page 41: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/41.jpg)
Filter Design – Intensity DiscontinuitiesFilter Design – Intensity Discontinuities
Using the observation that maximal filter response is exhibited when the filter matches the overlapping signal, we can start designing more complex filters:
Kernel with maximal response to intensity edges
0.5 0.0 -0.5
![Page 42: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/42.jpg)
Filter Design – Intensity DiscontinuitiesFilter Design – Intensity Discontinuities
Better yet, perform Gaussian smoothing to suppress noise first:
Noise suppressing kernel with high response to intensity edges
Gaussian kernel
![Page 43: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/43.jpg)
Unsharp Masking EnhancementUnsharp Masking Enhancement
The main principle of unsharp masking is to extract high frequency information and add it onto the original image to enhance edges:
image
HPF
+ output
Original edge Enhanced
![Page 44: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/44.jpg)
Unsharp Masking EnhancementUnsharp Masking Enhancement
Unsharp mask filtering performs noise reduction and edge enhancement in one go, by combining a Gaussian LPF with a Laplacian of Gaussian kernel:
Gaussian smoothing Convolution with –ve Laplacian of Gaussian
+ =
Result
![Page 45: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/45.jpg)
Unsharp Masking – ExampleUnsharp Masking – Example
Consider the following synthetic example:
Gaussian smoothed then corrupted with Gaussian noise
![Page 46: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/46.jpg)
Unsharp Masking – ExampleUnsharp Masking – Example
After unsharp masking:
Gaussian smoothed then corrupted with Gaussian noise
![Page 47: Image Processing IB Paper 8 – Part A Ognjen Arandjelović Ognjen Arandjelović oa214](https://reader035.vdocuments.net/reader035/viewer/2022062404/551ac7635503466b6a8b4ffa/html5/thumbnails/47.jpg)
– – That is All for Today –That is All for Today –