digital image processing · image restoration • image restoration: recover an image that has been...
TRANSCRIPT
Digital Image ProcessingCOSC 6380/4393
Lecture – 21Mar 30th, 2020Pranav Mantini
Objective
• To introduce a model for image restoration• To study noise that arise in images
Image Restoration
• Similar to enhancement, the goal is to improve an image.
Image Restoration
• Similar to enhancement, the goal is to improve an image.
Enhancement Restoration
Subjective process Objective process
Use psychophysical aspects of HVS to manipulate image
Use prior knowledge of degradation torestore image
Manipulate image to please viewer Recover original image by removing degradation
Image Enhancement
• Enhancement: – Point operations
• Full contrast stretch• Histogram equalization• Histogram flattening
• Restoration:– Removing blur
• By applying a deblurring function
3/30/2020 6
Image Restoration
• Image restoration: recover an image that has been degraded by using a prior knowledge of the degradation phenomenon.
• Model the degradation and applying the inverse process in order to recover the original image.
A Model of Image Degradation/Restoration Process
𝑓𝑓 𝑥𝑥, 𝑦𝑦
A Model of Image Degradation/Restoration Process
𝑓𝑓 𝑥𝑥, 𝑦𝑦
MotionTurbulence
A Model of Image Degradation/Restoration Process
ℎ 𝑥𝑥, 𝑦𝑦 𝑓𝑓 𝑥𝑥,𝑦𝑦
MotionTurbulence
⊗
A Model of Image Degradation/Restoration Process
ℎ 𝑥𝑥, 𝑦𝑦 𝑓𝑓 𝑥𝑥,𝑦𝑦
MotionTurbulence
Noise:Image acquisition: light, temperature, etc.
Transmission: atmospheric , electronic disturbance
⊗
A Model of Image Degradation/Restoration Process
ℎ 𝑥𝑥, 𝑦𝑦 𝑓𝑓 𝑥𝑥,𝑦𝑦
MotionTurbulence
Noise:Image acquisition: light, temperature, etc.
Transmission: atmospheric , electronic disturbance
𝜂𝜂 𝑥𝑥, 𝑦𝑦Additive noise
⊗
A Model of Image Degradation/Restoration Process
ℎ 𝑥𝑥, 𝑦𝑦 𝑓𝑓 𝑥𝑥,𝑦𝑦
MotionTurbulence
Noise:Image acquisition: light, temperature, etc.
Transmission: atmospheric , electronic disturbance
𝜂𝜂 𝑥𝑥, 𝑦𝑦Additive noise
⊗
⊗
A Model of Image Degradation/Restoration Process
ℎ 𝑥𝑥, 𝑦𝑦 𝑓𝑓 𝑥𝑥,𝑦𝑦
𝜂𝜂 𝑥𝑥, 𝑦𝑦
⊗
⊗
Ideal image
A Model of Image Degradation/Restoration Process
ℎ 𝑥𝑥, 𝑦𝑦 𝑓𝑓 𝑥𝑥,𝑦𝑦
𝜂𝜂 𝑥𝑥, 𝑦𝑦
⊗
⊗
Ideal image⊗
Degradation function
A Model of Image Degradation/Restoration Process
ℎ 𝑥𝑥, 𝑦𝑦 𝑓𝑓 𝑥𝑥,𝑦𝑦
𝜂𝜂 𝑥𝑥, 𝑦𝑦
⊗
⊗
Ideal image⊗
Degradation function
A Model of Image Degradation/Restoration Process
ℎ 𝑥𝑥, 𝑦𝑦 𝑓𝑓 𝑥𝑥,𝑦𝑦
𝜂𝜂 𝑥𝑥, 𝑦𝑦
⊗
⊗
Ideal image⊗
Degradation function
A Model of Image Degradation/Restoration Process
ℎ 𝑥𝑥, 𝑦𝑦 𝑓𝑓 𝑥𝑥,𝑦𝑦
𝜂𝜂 𝑥𝑥, 𝑦𝑦
⊗
⊗
Ideal image⊗
Degradation function
A Model of Image Degradation/Restoration Process
ℎ 𝑥𝑥, 𝑦𝑦 𝑓𝑓 𝑥𝑥,𝑦𝑦
𝜂𝜂 𝑥𝑥, 𝑦𝑦
⊗
⊗
Ideal image⊗
Degradation function
𝑔𝑔(𝑥𝑥,𝑦𝑦)
A Model of Image Degradation/Restoration Process
ℎ 𝑥𝑥, 𝑦𝑦 𝑓𝑓 𝑥𝑥,𝑦𝑦
𝜂𝜂 𝑥𝑥, 𝑦𝑦
⊗
⊗
Ideal image⊗
Degradation function
𝑔𝑔(𝑥𝑥,𝑦𝑦)
Goal of image restoration is to estimate 𝑓𝑓′ 𝑥𝑥,𝑦𝑦that is as close as possible to 𝑓𝑓(𝑥𝑥,𝑦𝑦)
A Model of Image Degradation/Restoration Process
ℎ 𝑥𝑥, 𝑦𝑦 𝑓𝑓 𝑥𝑥,𝑦𝑦
𝜂𝜂 𝑥𝑥, 𝑦𝑦
⊗
⊗
Ideal image⊗
Degradation function
𝑔𝑔(𝑥𝑥,𝑦𝑦)
Noise
• Source• Types of Noise• Estimation of Noise• Image Restoration
3/30/2020 22
Noise Sources
• The principal sources of noise in digital images arise during image acquisition and/or transmission
Image acquisition (CCD cameras)e.g., light levels, sensor temperature, etc.
Transmission (interference on channel)e.g., lightning or other atmospheric disturbance in wireless network
Types of Noise
• Spatially independent• Spatially dependent
Types of Noise
• Spatially independent
• Spatially dependent
The noise at location 𝑥𝑥, 𝑦𝑦 , 𝜂𝜂 𝑥𝑥,𝑦𝑦 is defined by a function 𝐻𝐻 that is not dependent on the (𝑥𝑥, 𝑦𝑦)
𝜂𝜂 𝑥𝑥, 𝑦𝑦 → 𝐻𝐻
Types of Noise
• Spatially independent
• Spatially dependent
The noise at location 𝑥𝑥, 𝑦𝑦 , 𝜂𝜂 𝑥𝑥,𝑦𝑦 is defined by a function 𝐻𝐻 that is not dependent on the (𝑥𝑥, 𝑦𝑦)
𝜂𝜂 𝑥𝑥, 𝑦𝑦 → 𝐻𝐻
The noise at location 𝑥𝑥, 𝑦𝑦 , 𝜂𝜂 𝑥𝑥,𝑦𝑦 is defined by a function 𝐻𝐻 that is dependent on the location (𝑥𝑥,𝑦𝑦)
𝜂𝜂 𝑥𝑥, 𝑦𝑦 → 𝐻𝐻(𝑥𝑥,𝑦𝑦)
Types of Noise
• Spatially independent
• Spatially dependent
The noise at location 𝑥𝑥, 𝑦𝑦 , 𝜂𝜂 𝑥𝑥,𝑦𝑦 is defined by a function 𝐻𝐻 that is not dependent on the (𝑥𝑥, 𝑦𝑦)
𝜂𝜂 𝑥𝑥, 𝑦𝑦 → 𝐻𝐻
The noise at location 𝑥𝑥, 𝑦𝑦 , 𝜂𝜂 𝑥𝑥,𝑦𝑦 is defined by a function 𝐻𝐻 that is dependent on the location (𝑥𝑥,𝑦𝑦)
𝜂𝜂 𝑥𝑥, 𝑦𝑦 → 𝐻𝐻(𝑥𝑥,𝑦𝑦)
Statistical noise
Periodic noise
Types of Noise
• Spatially independent
• Statistical noise:– Defined by the function H (which usually is a
probability distribution)– Different H arise in various aspects of imaging.– H is parametric in nature
The noise at location 𝑥𝑥, 𝑦𝑦 , 𝜂𝜂 𝑥𝑥,𝑦𝑦 is defined by a function 𝐻𝐻 that is not dependent on the (𝑥𝑥, 𝑦𝑦)
𝜂𝜂 𝑥𝑥, 𝑦𝑦 → 𝐻𝐻
Gaussian Noise
Electronic circuit noiseSensor noise due to poor illuminationSensor noise due to high temperature
3/30/2020 29
3/30/2020 30
Gaussian Noise
2 2( ) /2
The PDF of Gaussian random variable, z, is given by1 ( )
2z zp z e σ
πσ− −=
where, represents intensity
is the mean (average) value of z is the standard deviation
z
zσ
3/30/2020 31
Gaussian Noise
2 2( ) /2
The PDF of Gaussian random variable, z, is given by1 ( )
2z zp z e σ
πσ− −=
70% of its values will be in the range
95% of its values will be in the range
[ ])(),( σµσµ +−
[ ])2(),2( σµσµ +−
Rayleigh Noise
Range imaging: Image showing the distance to points in a scene from a specific point (Radar).
3/30/2020 33
3/30/2020 34
Rayleigh Noise
2( ) /
The PDF of Rayleigh noise is given by2 ( ) for
( )0 for
z a bz a e z ap z b
z a
− − − ≥= <
2
The mean and variance of this density are given by
/ 4(4 )
4
z a bb
ππσ
= +−
=
Erlang (gamma) and Exponential Noise
• Laser imaging: The Laser images are used to get 3D images with the help of laser. The images are captured by sensors which are mounted on laser.
3/30/2020 36
3/30/2020 37
Erlang (Gamma) Noise
1
The PDF of Erlang noise is given by
for 0 ( ) ( 1)!
0 for
b baza z e z
p z bz a
−−
≥= − <
2 2
The mean and variance of this density are given by
/ /
z b ab aσ
=
=
3/30/2020 38
Exponential Noise
The PDF of exponential noise is given by
for 0 ( )
0 for
azae zp z
z a
− ≥=
<
2 2
The mean and variance of this density are given by
1/ 1/
z aaσ
=
=
3/30/2020 39
Faulty switching results in quick transients
Impulse noise
3/30/2020 40
Faulty switching results in quick transients
Impulse noise
3/30/2020 41
3/30/2020 42
Impulse (Salt-and-Pepper) Noise
The PDF of (bipolar) impulse noise is given by for
( ) for 0 otherwise
a
b
P z ap z P z b
== =
If either or is zero, the impulse noise is calleda bP Punipolar
if , gray-level will appear as a light dot, while level will appear like a dark dot.
b a ba
>
Uniform noise
• Not common in practical situations• Basis for numerous random number
generators
3/30/2020 44
3/30/2020 45
Uniform Noise
The PDF of uniform noise is given by1 for a
( )0 otherwise
z bp z b a
≤ ≤= −
2 2
The mean and variance of this density are given by
( ) / 2 ( ) /12
z a bb aσ
= +
= −
3/30/2020 46
Examples of Noise: Original Image
3/30/2020 47
Examples of Noise: Noisy Images(1)
3/30/2020 48
Examples of Noise: Noisy Images(2)
3/30/2020 49
Periodic Noise
► Periodic noise in an image arises typically from electrical or electromechanical interference during image acquisition.
► It is a type of spatially dependent noise
► Periodic noise can be reduced significantly via frequency domain filtering
3/30/2020 50
An Example of Periodic Noise
3/30/2020 51
An Example of Periodic Noise
Estimation of Noise
• Periodic noise:– Produces frequency spikes– Estimated by inspecting the Fourier spectrum.– Can be automated if the spikes are pronounced– Infer periodicity directly from images (only
simplistic cases)
Estimation of Noise
• Statistical noise:– Known partially from the sensor specifications– If imaging system is available, capture a set of flat
images (solid grey board) and study the histogram– If only images are available, select an area of
image with reasonably constant background and study the histogram.
3/30/2020 54
Estimation of Noise Parameters
The shape of the histogram identifies the closest PDF match
3/30/2020 55
Rayleigh Noise
2( ) /
The PDF of Rayleigh noise is given by2 ( ) for
( )0 for
z a bz a e z ap z b
z a
− − − ≥= <
2
The mean and variance of this density are given by
/ 4(4 )
4
z a bb
ππσ
= +−
=
3/30/2020 56
Estimation of Noise Parameters
Consider a subimage denoted by , and let ( ), 0, 1, ..., -1, denote the probability estimates of the intensities of the pixels in . The mean and variance of the pixels in :
s iS p z i LS
S
=
1
01
2 2
0
( )
and ( ) ( )
L
i s ii
L
i s ii
z z p z
z z p zσ
−
=
−
=
=
= −
∑
∑
Next
Restoration: How to remove noise
End of Lecture