chapter 5: image restoration - kasetsart universityjan/204584/05-image_restoration..pdfimage...

Digital Image ProcessingChapter 5:

Digital Image ProcessingChapter 5: Chapter 5:

Image RestorationChapter 5:

Image RestorationImage RestorationImage Restoration

Concept of Image Restoration Concept of Image Restoration

Image restoration is to restore a degraded image back tothe original image while image enhancement is to manipulate the image so that it is suitable for a specificapplication.

Degradation model:

),(),(),(),( yxyxhyxfyxg η+∗=

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

where h(x,y) is a system that causes image distortion andη(x,y) is noise.

)()()()( yxyxhyxfyxg η+∗=

Concept of Image Restoration Concept of Image Restoration ),(),(),(),( yxyxhyxfyxg η+∗

We can perform the same operations in the frequency domain where convolution is replaced by multiplicationdomain, where convolution is replaced by multiplication, and addition remains as addition, because of the linearity of the Fourier transform.

),(),(),(),( vuNvuHvuFvuG +=If we knew the values of H and N we could recover F by writing the above equation as

),()()( vuNvuGvuF −=),(

),(),(vuH

vuGvuF =

However, as we shall see, this approach may not be practical. Even though we may have some statistical information about the noise, we will not know the value of n(x,y) or N(u,v) for all, or

l A ll di idi b H(i j) ill diff ltieven any, values. As well, dividing by H(i,j) will cause diffulties if there are values which are close to, or equal to, zero.

NoiseNoise

W d fi i b d d i i h i i lWe may define noise to be any degradation in the image signal, caused by external disturbance. If an image is being sent electronically from one place to another via satellite or wirelesselectronically from one place to another, via satellite or wireless transmission, or through networked cable, we may expect errors to occur in the image signal. These errors will appear on the image g g pp goutput in different ways depending on the type of disturbance in the signal.

Usually we know what type of errors to expect, and hence the type of noise on the image; hence we can choose the most appropriateof noise on the image; hence we can choose the most appropriate method for reducing the effects. Cleaning an image corrupted bynoise is thus an important area of image restoration.g

We will investigate some of the standard noise forms, and the diff t th d f li i ti d i th i ff t thdifferent methods of eliminating or reducing their effects on the image.

Salt and pepper noiseSalt and pepper noise

Al ll d i l i h t i bi i Thi d d ti bAlso called impulse noise, shot noise, or binary noise. This degradation can be caused by sharp, sudden disturbances in the image signal; its appearance is randomly scattered white or black (or both) pixels over the image. To add noise, we use the Matlab function imnoise, which takes a number of different parameters. To add salt and pepper noise:>> cameraman=imread('cameraman.tif');>> figure;imshow(cameraman);>> cameraman_sp=imnoise(cameraman,'salt & pepper');>> figure;imshow(cameraman_sp);to add more or less noise we include an optional parameter, being a value between 0 and 1 indicating the fraction of pixels to be corrupted. Thus,for example>> cameraman_sp20=imnoise(cameraman,'salt & pepper',0.2);

fi i h ( 20)>> figure;imshow(cameraman_sp20);

Gaussian noiseGaussian noise

G i i i id li d f f hi i hi h iGaussian noise is an idealized form of white noise, which is caused by random fluctuations in the signal. We can observe white

noise by watching a television which is slightly mistuned to anoise by watching a television which is slightly mistuned to a particular channel. Gaussian noise is white noise which is normally distributed. If the image is represented as I, and the y g pGaussian noise by N, then we can model a noisy image by simply adding the two: I + N>> i i ( ‘ i ')>> cameraman_gn=imnoise(cameraman,‘gaussian');>> figure;imshow(cameraman_gn);The gaussian parameter also can take optional values, giving the mean and variance of the noise. The default values are 0 and 0.1

Speckle noiseSpeckle noise

Wh G i i b d ll d b d l dd dWhereas Gaussian noise can be modelled by random values added to an image; speckle noise (or more simply just speckle) can be modelled by random values multiplied by pixel values hence it ismodelled by random values multiplied by pixel values, hence it isalso called multiplicative noise. Speckle noise is a major problem in some radar applications. As above, imnoise can do speckle:pp p>> cameraman_spk=imnoise(cameraman,'speckle');>> figure;imshow(cameraman_spk);In Matlab, speckle noise is implemented as I(1 + N)In Matlab, speckle noise is implemented as I(1 N)Although Gaussian noise and speckle noise appear superficially similar, they are formed by two totally different methods, and, as we shall see, require different approaches for their removaldifferent approaches for their removal.

Periodic noisePeriodic noise

If h i i l i bj i di h h dIf the image signal is subject to a periodic, rather than a random disturbance, we might obtain an image corrupted by periodic noise The effect is of bars over the image The function imnoisenoise. The effect is of bars over the image. The function imnoise does not have a periodic option, but it is quite easy to create, by adding a periodic matrix (using a trigonometric function), to an g p ( g g )image:>> s=size(cameraman);>> [x y]=meshgrid(1:s(1) 1:s(2));>> [x,y]=meshgrid(1:s(1),1:s(2));>> p=sin(x/3+y/5)+1;>> cameraman_pn=(im2double(cameraman)+p/2)/2;>> fi i h ( )>> figure;imshow(cameraman_pn)

Salt and pepper noise, Gaussian noise and speckle noise can all be cleaned by using spatial filtering techniques Periodicall be cleaned by using spatial filtering techniques. Periodic noise, however, requires the use of frequency

domain filtering. This is because whereas the other forms of i b d ll d l l d d ti i di i inoise can be modelled as local degradations, periodic noise is

a global effect.

Noise Models Noise Models

N i t b di t d b t b i t l d ib d iNoise cannot be predicted but can be approximately described instatistical way using the probability density function (PDF)

Gaussian noise:22 2/)(

21)( σμ−−= zezp

2)(

πσp

Rayleigh noise⎧2

⎪⎩

⎪⎨⎧

<

≥−=−−

azfor0

for )(2)(

/)( 2

azeazbzp

baz

⎪⎩ < azfor 0

Erlang (Gamma) noise⎧

⎪⎩

⎪⎨⎧

≥−−=

−−

0f0

0for )()!1()(1

zeazb

zazp

azbb

⎪⎩ < 0zfor 0

Noise Models (cont.) Noise Models (cont.)

Exponential noiseazaezp −=)(

Uniform noise

aezp =)(

⎪⎩

⎪⎨⎧ ≤≤=

h i0

afor a-b

1)( bzzp

I l ( lt & ) i

⎪⎩ otherwise 0

Impulse (salt & pepper) noise

⎧ =for azP

⎪⎩

⎪⎨

⎧==

otherwise0for for

)( bzPazP

zp b

a

⎩ otherwise 0

PDF: Statistical Way to Describe NoisePDF: Statistical Way to Describe Noise

PDF tells how mucheach z value occurs.


Image Degradation with Additive Noise Image Degradation with Additive Noise

),(),(),( yxyxfyxg η+=

Degraded images

Original image

Histogram


Image Degradation with Additive Noise (cont.) Image Degradation with Additive Noise (cont.)

),(),(),( yxyxfyxg η+=

Degraded images

Original image

Histogram


Estimation of Noise Estimation of Noise

We cannot use the image ghistogram to estimate noise PDF.

It i b tt t thIt is better to use the histogram of one areaof an image that hasof an image that has constant intensity to estimate noise PDF.


Periodic Noise Periodic Noise

Periodic noisePeriodic noise looks like dotsIn the frequency


domain

Periodic Noise Periodic Noise

Periodic Noise Reduction by Freq. Domain Filtering Periodic Noise Reduction by Freq. Domain Filtering

Degraded image DFTPeriodic noisecan be red ced bcan be reduced bysetting frequencycomponentscomponentscorresponding to noise to zero.


Band reject filter Restored image

Band Reject Filters Band Reject Filters

U t li i t f t i b dUse to eliminate frequency components in some bands

Periodic noise from theprevious slide that is Fil dFiltered out.


Notch Reject Filters Notch Reject Filters

A t h j t filt i d t li i t f tA notch reject filter is used to eliminate some frequency components.


Notch Reject Filter: Notch Reject Filter: Notch filter

Degraded image DFT (freq. Domain)

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition. Restored imageNoise

Example: Image Degraded by Periodic Noise Example: Image Degraded by Periodic Noise Degraded imageDegraded image

DFT( hift)(no shift)


Restored imageNoiseDFT of noise

Mean Filters Mean Filters Degradation model:g

),(),(),(),( yxyxhyxfyxg η+∗=

To remove this part

Arithmetic mean filter or moving average filter (from Chapter 3)

1 ∑∈

=xySts

tsgmn

yxf),(

),(1),(ˆy

Geometric mean filter mn = size of moving window

mn

tsgyxf

1

)()(ˆ⎟⎟⎞

⎜⎜⎛

= ∏Sts xy

tsgyxf),(

),(),( ⎟⎟⎠

⎜⎜⎝

= ∏∈

Geometric Mean Filter: Example Geometric Mean Filter: Example

Original Image d

O gimage corrupted

by AWGN

Image Image gobtained

using a 3x3t i

Image obtained

using a 3x3geometric mean filter

arithmetic mean filter

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.AWGN: Additive White Gaussian Noise

Harmonic and Contraharmonic FiltersHarmonic and Contraharmonic FiltersHarmonic mean filterHarmonic mean filter

=mnyxf ),(ˆ

W k ll f lt i∑

∈ xySts tsg

yxf

),( ),(1),( Works well for salt noise

but fails for pepper noise∈ xySts g),( ),(

Contraharmonic mean filter mn = size of moving window

∑∈

+

Sts

Qtsg)(

1),(ˆ

Positive Q is suitable for li i ti i

∑∈= xy

St

QSts

tsgyxf

)(

),(

),(),( eliminating pepper noise.

Negative Q is suitable for eliminating salt noise.∈ xySts ),(

Q = the filter order

eliminating salt noise.

For Q = 0, the filter reduces to an arithmetic mean filter.For Q = -1, the filter reduces to a harmonic mean filter.

Contraharmonic Filters: ExampleContraharmonic Filters: Example

Image corrupted b

Image corrupted

by pepper noise with prob = 0 1

by salt noise with

b 0 1prob. 0.1 prob. = 0.1

Image obtained

Image obtained

using a 3x3contra-

harmonic

using a 3x3contra-

harmonicharmonic mean filter

With Q = 1.5

harmonic mean filter

With Q=-1.5


Q Q

Contraharmonic Filters: Incorrect Use ExampleContraharmonic Filters: Incorrect Use Example

Image corrupted b

Image corrupted


by salt noise with

b 0 1prob. 0.1 prob. = 0.1

Image obtained

Image obtained

using a 3x3contra-

harmonic

using a 3x3contra-

harmonicharmonic mean filter

With Q=-1.5

harmonic mean filterWith Q=1.5


Q Q

OrderOrder--Statistic Filters: RevisitStatistic Filters: RevisitOriginal image

subimageOriginal image

Statistic parametersM M di M dMean, Median, Mode, Min, Max, Etc.

Moving windowwindow

Output image

OrderOrder--Statistics FiltersStatistics Filters

M di filtMedian filter

{ }),(median),(ˆ tsgyxf = { }),(ed a),(),(

gyfxySts ∈

Max filter

{ }),(max),(ˆ),(

tsgyxfxySts ∈

= Reduce “dark” noise(pepper noise))( xy

Min filter

{ }ˆ

(pepper noise)

R d “b i h ” i{ }),(min),(),(

tsgyxfxySts ∈

= Reduce “bright” noise(salt noise)

Midpoint filter

{ } { }⎟⎞

⎜⎛ )()(1)(f̂ { } { }⎟

⎠

⎞⎜⎝

⎛ +=∈∈

),(min),(max2),(

),(),(tsgtsgyxf

xyxy StsSts

Median Filter : How it works Median Filter : How it works A median filter is good for removing impulse isolated noiseA median filter is good for removing impulse, isolated noise

Salt noisePepper noisePepper noiseMedian

Degraded image MovingSorted arrayDegraded image

Salt noise

Movingwindow

array

Filter outputPepper noise

Normally, impulse noise has high magnitudeand is isolated When we sort pixels in theand is isolated. When we sort pixels in themoving window, noise pixels are usuallyat the ends of the array.

Therefore, it’s rare that the noise pixel will be a median value.

Median Filter : Example Median Filter : Example 1 2

Image corrupted b lby salt-

and-pepper noise withnoise with pa=pb= 0.1

43


Images obtained using a 3x3 median filter

Max and Min Filters: Example Max and Min Filters: Example

Image corrupted b

Image corrupted


by salt noise with

b 0 1prob. 0.1 prob. = 0.1

Image obtained

Image obtained

using a 3x3max filter

using a 3x3min filter


AlphaAlpha--trimmed Mean Filtertrimmed Mean Filter

1

Formula:

∑∈−

=xySts

r tsgdmn

yxf),(

),(1),(ˆy

where gr(s,t) represent the remaining mn-d pixels after removing the d/2 highest and d/2 lowest values of g(s,t).

Total mn pixels

d/2 d/2Lowest Gray Level

HighestGray Level

Total mn pixels

This filter is useful in situations involving multiple typesof noise such as a combination of salt-and-pepper andof noise such as a combination of salt and pepper and Gaussian noise.

AlphaAlpha--trimmed Mean Filter: Exampletrimmed Mean Filter: ExampleImage1 2

Image d

Image additionallycorrupted

corrupted by additiveuniform

pby additivesalt-and-

uniform noise

pepper noise

Image 2 Image 2gobtained

using a 5x5ith ti

obtained using a 5x5geometricarithmetic

mean filtergeometric mean filter


AlphaAlpha--trimmed Mean Filter: Example (cont.)trimmed Mean Filter: Example (cont.)Image1 2

Image d

Image additionallycorrupted

corrupted by additiveuniform

pby additivesalt-and-

uniform noise

pepper noise

Image 2Image obtained

2g

obtained using a 5x5

di filt

using a 5x5alpha-

trimmedmedian filter trimmed mean filterwith d = 5

AlphaAlpha--trimmed Mean Filter: Example (cont.)trimmed Mean Filter: Example (cont.)

Image obtained i 5 5

Image obtained i 5 5using a 5x5

arithmetic mean filter

using a 5x5geometric mean filtermean filter mean filter

Image Image obtained g

obtained using a 5x5

di filt

using a 5x5alpha-

trimmedmedian filter trimmed mean filterwith d = 5

Adaptive FilterAdaptive Filter

G l t

-Filter behavior depends on statistical characteristics of local areas i id i i d

General concept:

inside mxn moving window

- More complex but superior performance compared with “fixed”filfilters

Statistical characteristics:

∑= tsgm )(1Local mean:

2σ

Noise variance:

∑∈

=xySts

L tsgmn

m),(

),(

Local variance:

ησ

Local variance:

∑ −= LL mtsg 22 )),((1σ ∑∈ xySts

LL gmn ),(

)),((

Adaptive, Local Noise Reduction FilterAdaptive, Local Noise Reduction FilterPurpose: want to preserve edgesPurpose: want to preserve edges

Concept:

1. If ση2 is zero, No noisethe filter should return g(x,y) because g(x,y) = f(x,y)the filter should return g(x,y) because g(x,y) f(x,y)

2. If σL2 is high relative to ση

2, Edges (should be preserved), ηthe filter should return the value close to g(x,y)

3 If σ 2 = σ 2 Areas inside objects3. If σL2 = ση

2, Areas inside objectsthe filter should return the arithmetic mean value mL

Formula:

( )myxgyxgyxf )()()(ˆ2ση

Formula:

( )LL

myxgyxgyxf −−= ),(),(),( 2ση

Adaptive Noise Reduction Filter: Example Adaptive Noise Reduction Filter: Example

I IImage corrupted by additive

Imageobtained

using a 7x7by additiveGaussian noise with

using a 7x7arithmeticmean filter

zero meanand σ2=1000

Imageobtained

Imageobtained sing a 7 7obtained

using a 7x7geometric

using a 7x7adaptive

noisegmean filter

noise reduction

filter


Adaptive Median Filter Adaptive Median Filter Purpose: want to remove impulse noise while preserving edges

Algorithm:Level A: A1= zmedian – zminA2

Purpose: want to remove impulse noise while preserving edges

A2= zmedian – zmaxIf A1 > 0 and A2 < 0, goto level BElse increase window sizeElse increase window size

If window size <= Smax repeat level AElse return zxyy

Level B: B1= zxy – zminB2= zxy – zmaxIf B1 > 0 and B2 < 0 return zIf B1 > 0 and B2 < 0, return zxyElse return zmedian

z = minimum gray level value in Swhere zmin = minimum gray level value in Sxyzmax = maximum gray level value in Sxyzmedian = median of gray levels in Sxy

where

median g y xyzxy = gray level value at pixel (x,y)Smax = maximum allowed size of Sxy

L l A A1

Adaptive Median Filter: How it worksAdaptive Median Filter: How it works

D iLevel A: A1= zmedian – zminA2= zmedian – zmax

If A1 > 0 d A2 < 0 t l l B

Determine whether zmedianis an impulse or not

Else Window is not big enoughincrease window size

If A1 > 0 and A2 < 0, goto level B is an impulse or not

increase window sizeIf window size <= Smax repeat level A

Else return zxyy

zmedian is not an impulseLevel B:Determine

B1= zxy – zminB2= zxy – zmax

Determine whether zxyis an impulse or noty

If B1 > 0 and B2 < 0, zxy is not an impulsereturn zxy to preserve original details

ElElse return zmedian to remove impulse

Adaptive Median Filter: Example Adaptive Median Filter: Example

Image corrupted by salt-and-pepper

Image obtained using a 7x7

Image obtained using an adaptive

noise with pa=pb= 0.25

median filter median filter withSmax = 7

More small details are preserved


Estimation of Degradation Model Estimation of Degradation Model Degradation model:Degradation model:

),(),(),(),( yxyxhyxfyxg η+∗= )()()()( yyyfyg η

),(),(),(),( vuNvuHvuFvuG +=or

Purpose: to estimate h(x,y) or H(u,v)

Wh ? If k l h( ) dl f i dWhy? If we know exactly h(x,y), regardless of noise, we can do deconvolution to get f(x,y) back from g(x,y).

Methods:1. Estimation by Image Observation

2. Estimation by Experiment


3. Estimation by Modeling

Estimation by Image ObservationEstimation by Image ObservationOriginal image (unknown) Degraded image

f( ) f( )*h( ) ( )

Original image (unknown) Degraded image

f(x,y) f(x,y)*h(x,y) g(x,y)

Observation

Subimage),( vuGs )( yxg

DFTEstimated Transfer ),(s ),( yxgs

Restorationprocess by

ˆ),()()( vuGvuHvuH s=≈

function

Reconstructedbi

DFT)(ˆ vuF

p ocess byestimation),(ˆ),(),(

vuFvuHvuH

ss≈

This case is used when weSubimage

),(ˆ yxfs

),( vuFsknow only g(x,y) and cannot repeat the experiment!

Estimation by ExperimentEstimation by ExperimentUsed when we have the same equipment set up and can repeat theUsed when we have the same equipment set up and can repeat the

experiment.Input impulse image

Response image fromthe systemInput impulse image the system

SystemH( )H( )

),( yxg),( yxAδ

DFTDFT

),( vuG{ } AyxADFT =),(δ

AvuGvuH ),(),( =


Estimation by ModelingEstimation by ModelingUsed when we know physical mechanism underlying the imageUsed when we know physical mechanism underlying the image

formation process that can be expressed mathematically.

Example:O i i l i S t b l

AtmosphericT b l d l

Example:Original image Severe turbulence

Turbulence model6/522 )(),( vukevuH +−=)(

k = 0.0025

Low turbulenceMild turbulence Low turbulenceMild turbulence


k = 0.00025k = 0.001

Estimation by Modeling: Motion BlurringEstimation by Modeling: Motion BlurringAssume that camera velocity is ))()(( tytxAssume that camera velocity is ))(),(( 00 tytxThe blurred image is obtained by

T

dttyytxxfyxgT

))(),((),( 000

++= ∫h iwhere T = exposure time.

d dG vyuxj∫ ∫∞ ∞

+− )(2)()( π dxdyeyxgvuG

T

vyuxj∫ ∫⎤⎡

=

∞ ∞

∞− ∞−

+ )(2),(),( π

dxdyedttyytxxf vyuxj∫ ∫ ∫⎤⎡

⎥⎦

⎤⎢⎣

⎡++=

∞− ∞−

+− )(2

000 ))(),(( π

dtdxdyetyytxxfT

vyuxj∫ ∫ ∫ ⎥⎦

⎤⎢⎣

⎡++=

∞

∞−

∞

∞−

+−

0

)(200 ))(),(( π

Estimation by Modeling: Motion Blurring (cont.)Estimation by Modeling: Motion Blurring (cont.)T ∞ ∞ ⎤⎡

dtdxdyetyytxxfvuG vyuxj∫ ∫ ∫∞− ∞−

+−⎥⎦

⎤⎢⎣

⎡++=

0

)(200 ))(),(( ),( π

[ ]dtevuFT

tvytuxj∫ +−=0

))()((2 00),( π

dtevuFT

tvytuxj∫ +−=0

))()((2 00),( π

0

Then we get, the motion blurring transfer function:T

dtevuH tvytuxj∫ +−=0

))()((2 00),( π

For constant motion ),())(),(( 00 btattytx =

)()(2 ))(i ()( vbuajT

vbuaj bTdH ++∫ ππ )(

0

)(2 ))(sin()(

),( vbuajvbuaj evbuavbua

TdtevuH +−+− ++

== ∫ ππ ππ

Motion Blurring ExampleMotion Blurring ExampleFor constant motionFor constant motion

)())(sin()(

),( vbuajevbuab

TvuH +−++

= πππ )( vbua +π

Original image Motion blurred imagea = b = 0 1 T = 1


a = b = 0.1, T = 1

Motion Blur in Motion Blur in MatlabMatlab>> f=checkerboard(8);>> f=checkerboard(8);>> PSF=fspecial('motion',7,45);>> PSF

PSF =

0 0 0 0 0 0.0145 00 0 0 0 0 0376 0 1283 0 01450 0 0 0 0.0376 0.1283 0.01450 0 0 0.0376 0.1283 0.0376 00 0 0.0376 0.1283 0.0376 0 00 0.0376 0.1283 0.0376 0 0 0

0.0145 0.1283 0.0376 0 0 0 00 0 0145 0 0 0 0 00 0.0145 0 0 0 0 0

>> gb= imfilter(f PSF 'circular');


>> gb imfilter(f,PSF, circular );>> imshow(f);figure;imshow(gb);

Inverse Filter Inverse Filter From degradation model:From degradation model:

),(),(),(),( vuNvuHvuFvuG +=

after we obtain H(u,v), we can estimate F(u,v) by the inverse filter:

),(),(),(

),(),(),(ˆ

vuHvuNvuF

vuHvuGvuF +==

Noise is enhancedwhen H(u,v) is small. To avoid the side effect of enhancing

noise, we can apply this formulation to freq. component (u,v) with in a radius D0 from the center of H(u,v).


In practical, the inverse filter is notPopularly used.

Inverse Filter Inverse Filter

i d(' if')>> w=imread('cameraman.tif');;>> wf=fftshift(fft2(w));>> b=lbutter(256 256 15 2);>> b=lbutter(256,256,15,2);>> wb=wf.*b;

% image degraded>> wba=abs(ifft2(fftshift(wb)));

b i 8(255* 2 ( b ))>> wba=uint8(255*mat2gray(wba));>> figure;imshow(wba);

% recover>> w1=fftshift(fft2(wba))./b;>> w1a=abs(ifft2(fftshift(w1)));>> fi i h ( t2 ( 1 ))


>> figure;imshow(mat2gray(w1a));

Inverse Filter: Example Inverse Filter: Example

Original imageResult of applying

the full filterResult of applying

the filter with D0=40

Blurred imageDue to Turbulence

Result of applyingthe filter with D0=70

Result of applyingthe filter with D0=85

6/522 )(0025.0),( vuevuH +−=

Due to Turbulence the filter with D0 70 the filter with D0 85

Wiener Filter: Minimum Mean Square Error Filter Wiener Filter: Minimum Mean Square Error Filter Objective: optimize mean square error: { }22 )ˆ( ffEObjective: optimize mean square error: { }22 )( ffEe −=

⎤⎡

Wiener Filter Formula:

),(),(),(),(

),(),(),(ˆ

2

*

vuGvuSvuHvuS

vuSvuHvuF

f

f

⎥⎥⎦

⎤

⎢⎢⎣

⎡

+=

η

),()(/)()(

),( 2

*

vuGSSH

vuH

f

⎥⎥⎤

⎢⎢⎡

=

⎦⎣ η

)(),(1

),(/),(),(2

2

GvuH

vuSvuSvuH f

⎥⎤

⎢⎡

⎥⎦⎢⎣ + η

),(),(/),(),(

),(),(

1 2 vuGvuSvuSvuHvuH f ⎥

⎥⎦⎢

⎢⎣ +

=η

wherewhereH(u,v) = Degradation functionS (u v) = Power spectrum of noise


Sη(u,v) Power spectrum of noiseSf(u,v) = Power spectrum of the undegraded image

Approximation of Wiener FilterApproximation of Wiener FilterWiener Filter Formula:

),(),(1),(ˆ

2

2

vuGvuH

vuF ⎥⎤

⎢⎡

=

Wiener Filter Formula:

),(),(/),(),(),(

),( 2 vuSvuSvuHvuH f ⎥⎥⎦⎢

⎢⎣ + η

Approximated Formula:Difficult to estimate

),()(

),()(

1),(ˆ2

2

vuGKvuH

vuHvuH

vuF⎥⎥⎦

⎤

⎢⎢⎣

⎡

+=

),(),( KvuHvuH ⎥⎦⎢⎣ +

Practically, K is chosen manually to obtained the best visual result!

Wiener Filter: Example Wiener Filter: Example

Original imageResult of the

full inverse filterResult of the inverse

filter with D0=70 0


Result of the full Wiener filterDue to Turbulence full Wiener filter

Wiener Filter: Example (cont.) Wiener Filter: Example (cont.)

Original imageResult of the inverse

filter with D0=70 g g 0

Result of the Wi fil

Blurred imageD T b l Wiener filterDue to Turbulence

Example: Wiener Filter and Motion Blurring Example: Wiener Filter and Motion Blurring Image Result of the Result of theImage degradedby motion

Result of theinverse filter

Result of theWiener filter

yblur +AWGN ση

2=650η

ση2=325

Note: K is chosenmanually


ση2=130

Degradation model: W itt i t i f

Constrained Least Squares Filter Constrained Least Squares Filter Degradation model:

),(),(),(),( yxyxhyxfyxg η+∗=Written in a matrix form

ηHfg +=

Objective: to find the minimum of a criterion function

[ ]∑∑− −

∇=1 1 22 )(

M N

yxfC [ ]∑∑= =

∇0 0

),(x y

yxfC

Subject to the constraint22ˆ ηfHg =−

We get a constrained least square filter

where www T=2

),()()(

),(),(ˆ22

*

vuGPHvuHvuF

⎥⎥⎤

⎢⎢⎡

=

We get a constrained least square filter

),(),( 22 vuPvuH ⎥⎦⎢⎣ + γ

where⎥⎤

⎢⎡ − 010

P(u,v) = Fourier transform of p(x,y) =⎥⎥⎥

⎦⎢⎢⎢

⎣ −−−010141

Constrained Least Squares Filter: Example Constrained Least Squares Filter: Example Constrained least square filter

),(),(),(ˆ22

*

vuGvuHvuF⎥⎥⎤

⎢⎢⎡

=

Constrained least square filter

),(),(),(

),( 22 vuPvuH ⎥⎥⎦⎢

⎢⎣ + γ

γ is adaptively adjusted to achieve the best result.

Results from the previous slide obtained from the constrained least square filter

Constrained Least Squares Filter: Example (cont.) Constrained Least Squares Filter: Example (cont.) Image Result of the Result of theImage degradedby motion

Result of theConstrainedLeast square

Result of theWiener filter

yblur +AWGN

qfilter

ση2=650η

ση2=325

ση2=130

Constrained Least Squares Filter:Adjusting Constrained Least Squares Filter:Adjusting γγ

ˆ 2)( TφDefine fHgr ˆ−= It can be shown that 2)( rrr == Tγφ

We want to adjust gamma so that a±22 ηr 1We want to adjust gamma so that a±= ηr

where a = accuracy factor1. Specify an initial value of γ

1

2. Compute 2r

3. Stop if is satisfiedOtherwise return step 2 after increasing γ if

1a−< 22 ηrOtherwise return step 2 after increasing γ if

or decreasing γ if

a< ηr

a+> 22 ηrUse the new value of γ to recompute

)(),()(ˆ*

vuGvuHvuF ⎥⎤

⎢⎡

= ),(),(),(

),( 22 vuGvuPvuH

vuF⎥⎥⎦⎢

⎢⎣ +

=γ

Constrained Least Squares Filter:Adjusting Constrained Least Squares Filter:Adjusting γ γ (cont.)(cont.)* ⎤⎡

),(),(),(

),(),(ˆ22

*

vuGvuPvuH

vuHvuF⎥⎥⎦

⎤

⎢⎢⎣

⎡

+=

γ

),(ˆ),(),(),( vuFvuHvuGvuR −=2rFor computing

∑∑−

=

−

=

=1

0

1

0

22 ),(1 M

x

N

y

yxrMN

ry

∑∑− −

=1 1

)(1 M N

yxm η

[ ]∑∑− −1 1

22 1 M N

∑∑= =0 0

),(x y

yxMN

m ηη

2ηFor computing[ ]∑∑= =

−=0 0

22 ),(1x y

myxMN ηη ησ

[ ]

ηFor computing

[ ]ηησ mMN −= 22η

Constrained Least Squares Filter: Example Constrained Least Squares Filter: Example

O i i l i Use correct noiseOriginal image Correct parameters:Initial γ = 10-5

Correction factor = 10-6

Use correct noise parameters

Correction factor = 10 6

a = 0.25ση

2 = 10-5η


Use wrong noise parameters

Wrong noise parameterσ 2 = 10-2ση = 10


Results obtained from constrained least square filters

Geometric Mean filter Geometric Mean filter This filter represents a family of filters combined into a

1 α−⎤⎡

This filter represents a family of filters combined into a single expression

)(),(),()(ˆ

1

**

GvuHvuHF

α

α

−

⎥⎥⎤

⎢⎢⎡

⎥⎤

⎢⎡

),(

),(),(

),(

),(),(),(),(

22 vuG

vuSvuS

vuHvuH

vuF

f

ηβ⎥⎥⎥⎥

⎦⎢⎢⎢⎢

⎣⎥⎦

⎤⎢⎣

⎡+⎥

⎥⎦⎢

⎢⎣

=

),( vuS f ⎥⎦⎢⎣ ⎦⎣

α = 1 the inverse filterα 1 the inverse filterα = 0 the Parametric Wiener filterα = 0, β = 1 the standard Wiener filterβ = 1, α < 0.5 More like the inverse filterβ = 1, α > 0.5 More like the Wiener filter

Another name: the spectrum equalization filter

Geometric TransformationGeometric Transformation

Th t f ti ft ll d bb h t t f tiThese transformations are often called rubber-sheet transformations: Printing an image on a rubber sheet and then stretch this sheet accordingto some predefine set of rules.to some predefine set of rules.

A geometric transformation consists of 2 basic operations:1 A spatial transformation :1. A spatial transformation :

Define how pixels are to be rearranged in the spatiallytransformed image.transformed image.

2. Gray level interpolation :Assign gray level values to pixels in the spatiallytransformed image.

Spatial TransformationSpatial Transformation


One of the most commonly used forms of spatial transformations is affine transform which can be writtenaffine transform, which can be written.

This transfomation can scale, rotate, translate, or shear f i d di h l h f h l fa set of points, depending on the values chosen for the elements of

T, which can be create in Matlab :

Image RegistrationImage Registration

I i i h d k li i f hImage registration method seek to align two images of the same scene . For example, it may be of interest to align two ore more images taken at roughly the same time but using differentimages taken at roughly the same time, but using different instruments, such an MRI scan and a PET scan.

Image RegistrationImage Registration

Geometric Transformation : AlgorithmGeometric Transformation : Algorithm

)( ′′)( ),( yx ′′),( yx

Distorted image gImage f to be t d

1 3

1. Select coordinate (x,y) in f to be restored

restored

( y) f2. Compute

),( yxrx =′ 4. get pixel value atBy gray level interpolation

),( yxg ′′

),( yxsy =′

3. Go to pixel ),( yx ′′

By gray level interpolation

5. store that value in pixel f(x,y)3. Go to pixel in a distorted image g

),( y

5

Spatial TransformationSpatial TransformationTo map between pixel coordinate (x,y) of f and pixel coordinateTo map between pixel coordinate (x,y) of f and pixel coordinate

(x’,y’) of g),( yxrx =′ ),( yxsy =′),( y ),( yy

For a bilinear transformation mapping between a pair of Quadrilateral regionsQuadrilateral regions

4321),( cxycycxcyxrx +++==′

8765),( cxycycxcyxsy +++==′ ),( yx ′′ ),( yx

To obtain r(x,y) and s(x,y), we needt k 4 i f di tto know 4 pairs of coordinates

and its correspondingwhich are called tiepoints

),( yx ′′),( yx


which are called tiepoints.

Gray Level Interpolation: Nearest NeighborGray Level Interpolation: Nearest Neighbor

i b i di d)( ′′Since may not be at an integer coordinate, we need to Interpolate the value of

),( yx ′′),( yxg ′′

Example interpolation methods that can be used:1. Nearest neighbor selection2. Bilinear interpolation3. Bicubic interpolation


Gray Level Interpolation: Bilinear interpolationGray Level Interpolation: Bilinear interpolation

Geometric Distortion and Restoration ExampleGeometric Distortion and Restoration ExampleOriginal image and Tiepoints of distortedOriginal image and

tiepointsTiepoints of distorted

image

Distorted image Restored image

Use nearestneighborneighbor intepolation


Geometric Distortion and Restoration Example Geometric Distortion and Restoration Example (cont.)(cont.)

Original image and Tiepoints of distortedOriginal image andtiepoints

Tiepoints of distortedimage

Distorted image Restored image

Use bilinear i t l tiintepolation

(Images from Rafael C. Gonzalez and Richard EGonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Example: Geometric RestorationExample: Geometric RestorationOriginal image Geometrically distortedOriginal image Geometrically distorted

Image

Use the sameUse the sameSpatial Trans (Bilinear).( )as in the previousexample

Restored image


Difference between 2 above images

chapter 5: image restoration - kasetsart universityjan/204584/05-image_restoration..pdfimage...

Documents