CoE4TN3Image Processing

Chapter 5Image Restoration

2

Image Restoration• Similar to image enhancement, the ultimate goal of restoration

techniques is to improve an image• Restoration: a process that attempts to reconstruct or recover a

degraded image by using some a priori knowledge of the degradation phenomenon.

• Technique: model the degradation -> apply the inverse process to recover the original image

• Enhancement technique are heuristic while restoration techniques are mathematical

3

Degradation Model

• Problem: given g(x,y) and H find an approximate of f(x,y). • Some statistical knowledge of n(x,y) is available.

g(x,y)=h(x,y)*f(x,y)+n(x,y)

4

Noise model• Principal sources of noise in digital images: during image

acquisition, during image transmission• Image acquisition: image sensor might produce noise because

of environmental conditions or quality of sensing elements• Image transmission: interference in the channel • Assumptions: noise is independent of spatial coordinates

(except for periodic noise) and independent of the image • Spatial description of noise: statistical behavior of the values

of the noise (PDF)• Most common PDFs found in image processing: Gaussian

noise, Rayleigh noise, Erlang (Gamma) noise, Exponential noise, Uniform noise, Impulse noise

5

6

Noise model• Different PDFs provide useful tools for modeling a broad

range of noise corruption situations:• Gaussian noise: due to factors such as electronic circuit noise,

sensor noise (due to poor illumination or high temperature)• Rayleigh noise: model noise in range imaging • Exponential and Gamma: laser imaging• Impulse noise: found in quick transients (e.g., faulty

switches)

7

Noise model

8

Noise model

9

Noise model

10

Periodic Noise• Periodic noise: from electrical or electromechanical

interference during image acquisition• Frequency domain filtering can be used to remove this noise• Fourier transform of a pure sinusoid is a pair of conjugate

impulses• In the Fourier transform of an image corrupted with periodic

noise should have a pair of impulses for each sine wave

11

Noise model

12

• Parameters of periodic noise are typically estimated by inspection of the FT of the image

• Estimation of parameters of the PDF of noise:– If the imaging system is available, one simple way is to capture a set

of images of “flat” environments• Flat environments: a solid gray board illuminated uniformly

– If only the images are available, the parameters of the PDF can be estimated from small patches of reasonably constant gray levels

Noise model

13

Noise model

14

Restoration in the presence of noise• When the only degradation is noise:

g(x,y)=f(x,y)+n(x,y)G(u,v)=F(u,v)+N(u,v)

• Spatial filtering is the method of choice in this case: Mean filters, Order-statistics filters, Adaptive filters

15

Mean filters• Sxy set of coordinates in a subimage of size mXn• Arithmetic mean filter:

– Smoothes local variations (noise is reduced because of blurring)

• Geometric mean filter:

– Achieves smoothing comparable to arithmetic mean, but tends to lose less image details

∑∈

=xySts

tsgmn

yxf),(

^),(1),(

mn

Sts xy

tsgyxf

1

),(

^),(),(

= ∏

∈

16

Mean filters• Harmonic mean filter:

– Suitable for salt noise, does well with Gaussian noise

• Contraharmonic mean filter

– Negative Q: Suitable for salt noise– Positive Q: Suitable for pepper noise

∑∈

=

xySts tsg

mnyxf

),(

^

),(1),(

∑

∑

∈

∈

+

=

xy

xy

Sts

Q

Sts

Q

tsg

tsgyxf

),(

),(

1

^

),(

),(),(

17

18

19

20

Order-statistics filters• Order-statistics filters: spatial filters whose response is based

on ordering (ranking) the pixels in the subimage• Median filters:

– Effective for salt and pepper noise

• Max and Min filters

– Max filter: useful for finding brightest points in an image (remove pepper noise)

– Min filter: useful for finding darkest points in an image (remove salt noise)

)},({),(),(

^tsgmedianyxf

xySts ∈=

)},({max),(),(

^tsgyxf

xySts ∈= )},({min),(

),(

^tsgyxf

xySts ∈=

21

Order-statistics filters • Midpoint filter:

– Works best for Gaussian and uniform noise

• Alpha-trimmed mean filter• d/2 lowest and d/2 highest gray-levels are removed

– Useful for combination of salt-pepper and Gaussian noise

)}],({min)},({max[21),(

),(),(

^tsgtsgyxf

xyxy StsSts ∈∈+=

∑∈−

=xyStsr tsg

dmnyxf

),(

^),(1),(

22

23

24

Adaptive filters• Adaptive filters: behavior of the filter changes based on

statistical characteristics of the image inside the subimage(Sxy)

• Adaptive filters have superior performance• Price: increase in filter complexity

25

Adaptive, local noise reduction filter• Response of the filter is based on four quantities:1. g(x,y) 2. : variance of noise 3. mL: mean of pixels in Sxy

4. σ2L: variance of pixels in Sxy

• The behavior of the filter:– If is zero, filter should return g(x,y)– If σ2

L is high (edges) the filter should return g(x,y)– If the two variances are almost equal, the filter should return the mean

of the pixels in Sxy

2ησ

2ησ

]),([),(),( 2

2^

LL

myxgyxgyxf −−=σση

26

27

Adaptive median filtering• Median filters perform well as long as the density of impulse

noise is not large• Adaptive median filter:

– Handle dense impulse noise– Smoothes non-impulse noise– Preserves details

• zmin: minimum gray level in Sxy

• zmax: maximum gray level in Sxy

• zmed: median gray level of Sxy

• zxy: gray level at coordinate (x,y)• Smax: maximum allowed size of Sxy

28

Adaptive median filtering• A

– A1=zmed-zmin

– A2=zmed-zmax

– If A1>0 and A2<0 go to B else increase the window size– If window size<Smax repeat A– Else output zxy

• B– B1=zxy-zmin

– B2=zxy-zmax

– If B1>0 and B2<0 output zxy

– Else output zmed

29

Adaptive median filtering

30

Periodic noise reduction• Bandreject filters remove or attenuate a band of frequencies

=

101

),( vuH

2),(

2),(

2

2),(

0

00

0

WDvuD

WDvuDWD

WDvuD

+>

+≤≤−

−<

n

DvuDWvuD

vuH 2

20

2 ),(),(1

1),(

−

+

=

220

2

),(),(

21

1),(

−−

−= WvuDDvuD

evuH

31

Periodic noise reduction

32

Periodic noise reduction

33

• A bandpass filter performs the opposite of a bandreject filter. – Hbp(u,v)=1-Hbr(u,v)

• A notch filter rejects (or passes) frequencies in predefined neighborhood about a center frequency

• Due to symmetry of the FT, notch filters must appear in symmetric pairs about the origin in order to obtain meaningful results

Periodic noise reduction

34

−

−=20

21 ),(),(21

1),( DvuDvuD

evuH

n

vuDvuDD

vuH

+

=

),(),(1

1),(

21

0

=10

),( vuHotherwise

DvuDDvuD 0201 ),(,),( ≤≤

Periodic noise reduction

20

201 )()(),( vvuuvuD −+−=

20

202 )()(),( vvuuvuD +++=

35

Periodic noise reduction

36

37

Estimation of degradation • Since degradations are modeled as being the result of

convolution, restoration is sometimes called deconvolution. • Estimation of the degradation function:

1. Observation2. Experimentation 3. Mathematical modeling

38

Estimation by image observation• We are given a degraded image without any knowledge of H.• We look at a small section of the image containing simple

structures (e.g., part of an object and the background)• Using sample gray levels of the object and background, we

can construct an unblurred image of the subimage ),(^

yxf s

),(

),(),( ^

vuF

vuGvuH

s

ss =

39

Estimation by Experimentation • If equipment similar to the equipment used to acquire the

degraded image is available it is possible a to obtain an accurate estimate of the degradation.

• The idea is to obtain the impulse response of the degradation by imaging an impulse (small dot of light) using the system

AvuGvuH ),(),( =

40

Estimation by modeling • Approach: derive a mathematical model starting from basic

principles• Example: Images blurred by motion

41

42

Inverse filtering

43

Wiener filtering

44

Wiener filtering

45

A Simple Example• 1-D signal• h is a 2 point signal (h(0) and h(1))• f is also a 2 point signal• g is observed: find f(0) and f(1) from g(0), g(1), g(2)

h(x)f(x)

n(x)

g(x)

46

A Simple Example

)2()1()1()2()1()1()0()0()1()1(

)0()0()0()0(

)2()1()1()0()2()2()1()1()0()0()1()1(

)0()1()1()0()0()0(

)()()()(

)()(*)()(1

0'

''

nhfgnhfhfg

nhfg

nhfhfgnhfhfgnhfhfg

xnxhxxfxg

xnxfxhxg

x

+=++=

+=

++=++=

+−+=

+−=

+=

∑=

47

A Simple Example

• If the noise is not ignored, we have five unknowns and only three equations.

• The original signal (f) cannot be determined uniquely. • This is an example of an ill-conditioned problem. • Some other knowledge about f is necessary.

})]1()0(min{[

)2()1()1()2()1()1()0()0()1()1(

)0()0()0()0(

2ff

nhfgnhfhfg

nhfg

−

+=++=

+=

48

Constrained Least Square Filtering• g=Hf+n• Find a restoration filter such that the output of the filter is

smooth:

• After a lengthy procedure (which has been removed in the new edition of the book) using the fact that the matrix H is circular it can be shown that:

22^

^2

1

0

1

0

2^

2

)1,()1,(),1(),1(),(4),(

)],([

nfHg

yxfyxfyxfyxfyxfyxf

yxfCM

x

N

y

=−

−−+−−−+−≈∇

∇= ∑∑−

=

−

=

49

Constrained Least Square Filtering

• P(u,v) is the Fourier transform of the Laplacian operator:

γ is a parameter.• p(x,y) should be padded to a proper size.• Although mathematical method exists for finding γ, in most

practical cases γ is selected by trial and error.

22

*

),(),(),(),(

vuPvuHvuHvuM

γ+=

−−−

−=

010141

010),( yxp

50

Constrained Least Square Filtering

51

Geometric Mean Filter• Generalization of Wiener filter:

α and β : positive real constants α=1 : inverse filterα=0 : parametric Wiener filter α=0, β=1 : Wiener filterα=1/2, β=1 : spectrum equalization filter

α

α

β

−

+

=

1

2

*

2

*

),(),(

),(

),(),(),(),(

vuSvuS

vuH

vuHvuHvuHvuM

f

n

52

Geometric Transformations• Geometrical transformations: modify the spatial relationships

between pixels in an image• Geometrical transformations: called rubber-sheet

transformations– They may be viewed as printing an image on a sheet of rubber and

then stretching this sheet according to some predefined rules

53

Geometric Transformations• Geometrical transformation consists of two basic operations:

1. Spatial transformation: defines the rearrangement of pixels on the image plane

2. Gray level interpolation: deals with the assignment of gray levels to pixels in the spatially transformed image

54

Spatial Transformations• Image f with pixels coordinates (x,y) has undergone

geometric distortion to produce an image g with coordinates (x’,y’)– x’=r(x,y), y’=s(x,y)

• Example: x’=r(x,y)=x/2, y’=s(x,y)=y/2– Distortion is a shrinking of the size of f(x,y) by one-half in both

directions.

55

Spatial Transformations• If r(x,y) and s(x,y) are known analytically: the inverse of r

and s is applied to g(x’,y’) to recover f(x,y).• In practice finding a single set of r(x,y) and s(x,y) is not

possible• Solution: spatial relocation is formulated by the use of

tiepoints. • Tiepoints: a set of pixels whose locations in distorted and

corrected images are known

56

Spatial Transformations• Suppose the geometrical distortion process within the region

is modeled by a pair of bilinear equations:– x’=r(x,y)=c1x+c2y+c3yx+c4

– y’=s(x,y)=c5x+c6y+c7yx+c8

– 8 known tiepoints, 8 unknown ci

– The model is used for all the points inside the region

57

Spatial Transformations

for x=1 to horizontal size {for y=1 to vertical size {

x’=r(x,y)=c1x+c2y+c3yx+c4

y’=s(x,y)=c5x+c6y+c7yx+c8

f^(x,y)=g(x’,y’)}}

58

Gray-level Interpolation• Depending on the values of ci x’ and/or y’ can be noninteger

for integer values of (x,y)– x’=r(x,y)=c1x+c2y+c3yx+c4

– y’=s(x,y)=c5x+c6y+c7yx+c8

• g is a digital image and its pixel values are defined only at integer values of (x,y).

• We need inferring gray-level values at noninteger locations (gray-level interpolation)

59

Gray-level Interpolation• Simplest scheme: nearest neighbor approach (zero-order

interpolation)1. Mapping (x,y) to (x’,y’)2. Selection of closest integer coordinate neighbor to (x’,y’) 3. Assign the gray-level of this nearest neighbor to the pixel at (x,y)

60

Gray-level Interpolation• Nearest neighbor interpolation: simple to implement, has the

drawback of producing undesirable artifacts• Example: distortion of straight edges in an image• More sophisticated techniques: better results, costly in terms

of computations• A reasonable compromise: bilinear interpolation approach

61

Gray-level Interpolation• (x’,y’): a noninteger coordinate• v(x’,y’): the gray-level at (x’,y’)• v(x’,y’)=ax’+by’+cx’y’+d• The four unknowns (a,b,c,d) are determined using the gray-

levels of four neighbors of (x’,y’)• When the coefficients (a,b,c,d) have been determined,

v(x’,y’) is computed and this values is assigned to the location (x,y).

62

Geometric Transformations