part 3: fourier transform and filtering in the frequency...

2011-04-19

Digital Image Processing

Achim J. Lilienthal

AASS Learning Systems Lab, Dep. Teknik

Room T1209 (Fr, 11-12 o'clock)

[email protected]

Digital Image ProcessingPart 3: Fourier Transform and

Filtering in the Frequency Domain

Course Book Chapter 4

Achim J. Lilienthal

Derivatives and their Fourier Transform

Laplacian in the Fourier Domain

[ ])()()( xfujdx

xfd nn

n

FF =

)(),( 22 vuvuH Laplacian +−=⇒

),()()],([ 222 vuFvuyxf +−=∇F

2 Relation Between Spatial and Frequency Filters

Achim J. Lilienthal

Derivatives and their Fourier Transform


[ ])()()( xfujdx

xfd nn

n

FF =

( ) ( ) ( )vuFNvMuyxf ,22),(222

−+−−⇔∇


Achim J. Lilienthal



Achim J. Lilienthal


Laplacian in the Spatial Domain

( ) ( ) yxvuvu ++− −−+− 1]1)([ 221F

2

Achim J. Lilienthal

Deriving Spatial Filter Masks

General Idea select a filter in the frequency domain

transform this filter to the spatial domain

try to specify a small filter mask that captures the "essence" of the filter function

2

Achim J. Lilienthal

Filtering in the Frequency Domain

Other Filters bandpass

allows frequencies in a band between the two frequencies D0 and D1

bandstop: stops frequencies in a band

between the two frequencies D0 and D1

non-symmetric filters: allow different frequencies in the u and v direction

2

Achim J. Lilienthal

Recovering Intrinsic Images,Homomorphic Filtering

→ Contents

Achim J. Lilienthal

Reminder: Image Formation Model illumination i(x,y) from a source reflectivity r(x,y) = reflection / absorption in the scene

f(x,y) = r(x,y) i(x,y)

Image Formation3

Achim J. Lilienthal

Intrinsic Images "midlevel description" of scenes

proposed by Barrow and Tenebaum[H.G. Barrow and J.M. Tenenbaum. Recovering Intrinsic Scene Characteristics from Images". In Computer Vision Systems. Academic Press, 1978]

not a full 3D description of the scene

viewpoint dependent

physical causes of changes in illumination are not made explicit

Intrinsic Images3

Achim J. Lilienthal


proposed by Barrow and Tenebaum[H.G. Barrow and J.M. Tenenbaum. Recovering Intrinsic Scene Characteristics from Images". In Computer Vision Systems. Academic Press, 1978]

"The observed image is a product of two images: an illumination image and a reflectance image."

Intrinsic Images3

Achim J. Lilienthal


(input) image is decomposed into two images …

Intrinsic Images3

from "Deriving Intrinsic Images From Image Sequences",

Yair Weiss , Proc. ICCV 2001

Achim J. Lilienthal


(input) image is decomposed into two images … a reflectance image

Intrinsic Images3



r(x,y)

Achim J. Lilienthal


(input) image is decomposed into two images … a reflectance image and

an illumination image

Intrinsic Images3



r(x,y) i(x,y)

Achim J. Lilienthal


"The observed image is a product of two images: an illumination image and a reflectance image." segmentation on the intrinsic reflectance should be much simpler

than on the original image

3D information can be obtained from the illumination picture

Intrinsic Images3



r(x,y) i(x,y)

Achim J. Lilienthal


(input) image is decomposed into two images … a reflectance image and

an illumination image

but: decomposition is an ill-posed problem number of unknowns is twice as high as the number of equations

(for example: set i(x,y) = 1 r(x,y) = f(x,y))

Intrinsic Images3



),(),(),( yxryxiyxf =

Achim J. Lilienthal

Homomorphic Filtering

Idea: Separate Illumination and Reflectance

not separable directly …

… but the logarithm is separable


)],([)],([)],([ yxryxiyxf FFF ≠

[ ]),(ln),( yxfyxz ≡

[ ][ ] [ ][ ]),(),(),(),(ln),(ln)],([

vuFvuFvuZyxryxiyxz

ri +==+= FFF

3

Achim J. Lilienthal

Frequency Domain Approximation to Homomorphic Filtering – Assumption illumination component varies slowly

reflectance component tends to vary abruptly

⇒ use filter that affects low- and high-frequency components in a different way (decreases influence of illumination, increases influence of reflectance)



γL<1

γH >1

3

Achim J. Lilienthal


Idea: Separate Illumination and Reflectance

[ ][ ] [ ][ ]),(ln),(ln),(),(),( yxryxivuFvuFvuZ ri FF +=+=

),(),(),(),(),(),(),( vuFvuHvuFvuHvuZvuHvuS ri +==

[ ] [ ] [ ]),('),('

),(),(),(),(),(),( 111

yxryxivuFvuHvuFvuHvuSyxs ri

+=+== −−− FFF

),('),('),(),( zxrzxiyxs eeeyxg ==

3

Achim J. Lilienthal

Example consider non-uniform illumination

Homomorphic Filtering3

disturbance pattern

Achim J. Lilienthal

Example histogram equalization does not perform well


histogram equalization

3

Achim J. Lilienthal

Example homomorphic filtering

Homomorphic Filtering3

homomorphic filtering

Achim J. Lilienthal

Homomorphic Filtering Assumption True?["Deriving Intrinsic Images From Image Sequences", Yair Weiss, Proc. ICCV 2001]

edges due to illumination often have as high a contrast as those due to reflectance changes possible solution: deriving intrinsic images from image sequences

Remarks3

Achim J. Lilienthal

Recovering Intrinsic Images from a Single Image["Recovering Intrinsic Images from a Single Image", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]

use colour information

Remarks3

Achim J. Lilienthal

Recovering Intrinsic Images from a Single Image[" Recovering Intrinsic Images from a Single Image ", Marshall F. Tappen, William T Freeman, Edward H Adelson, MIT AI Memo, 2002, http://hdl.handle.net/1721.1/6703]

use colour information find chromaticity changes by classifying image derivatives by

thresholding scalar product of normalized RGB vector neighbours

Remarks3

Achim J. Lilienthal



learn appearance models of shading patterns classify gray-scale image

Remarks3

Achim J. Lilienthal



learn appearance models of shading patterns classify gray-scale image

propagate evidence (MRF model with learned parameters)

Remarks3

Achim J. Lilienthal


use colour information +

learn appearance models of shading patterns +

propagate evidence (MRF model with learned parameters)

Remarks3

Achim J. Lilienthal


Remarks3

Achim J. Lilienthal

Properties of the Fourier Transform

→ Contents

Achim J. Lilienthal


a shift in f(x,y) does not affect |F(u,v)|

)//(200

00),(),( NyvMxujevuFyyxxf +−⇔−− π

Translation

4

Achim J. Lilienthal


Translation a shift in f(x,y)

does not affect the spectrum |F(u,v)|

4

Achim J. Lilienthal

Distributive Over Addition

Not Distributive Over Multiplication

Scaling

Rotation


)],([)],([)],(),([ 2121 yxfyxfyxfyxf FFF +=+

)],([)],([)],(),([ 2121 yxfyxfyxfyxf FFF ≠

),(),( vuaFyxaf ⇔ )/,/(1),( bvauFab

byaxf ⇔

),(),( 00 θϕωθθ +⇔+ Frf

4

Achim J. Lilienthal


rotating f(x,y) rotates F(u,v) by the same angle

F(u,v)

f(x,y)

4

Rotation

Achim J. Lilienthal


Periodicity

the discrete Fourier transform is periodic

also the inverse of the discrete Fourier transform is periodic

Conjugate Symmetry

the spectrum is symmetric about the origin

),(),(),(),( NvMuFNvuFvMuFvuF ++=+=+=

),(),(),(),( NyMxfNyxfyMxfyxf ++=+=+=

),(),( * vuFvuF −−= ),(),( * vuFvuF −−=⇒

4

Achim J. Lilienthal


Separability

F(x,v) is the Fourier transform along one row

F(u,v) can be obtained by two successive applications of the simple 1D Fourier transform instead of by one application of the more complex 2D Fourier transform

4

[ ][ ]),(),(1

),(11

),(1),(

1

0

/2

1

0

/21

0

/2

1

0

1

0

)//(2

yxfvxFeM

eyxfN

eM

eyxfMN

vuF

vu

M

x

Muxj

N

y

NvyjM

x

Muxj

M

x

N

y

NvyMuxj

FF==

==

==

∑

∑∑

∑∑

−

=

−

−

=

−−

=

−

−

=

−

=

+−

π

ππ

π

Achim J. Lilienthal


Fourier Transform

4

from "Computer Vision – A Modern Approach", Forsyth and Ponce,

Prentice Hall, 2002

( )1 ,box x ysin sinu v

u v( )( ),u vF f a b

ab

Achim J. Lilienthal

Correlation

Definition of Correlation

compare with convolution

also the need for padding

∑∑−

=

−

=

++=1

0

1

0

* ),(),(1),(),(M

m

N

nnymxhnmf

MNyxhyxf

∑∑−

=

−

=

−−=∗1

0

1

0),(),(),(),(

M

m

N

nnymxhnmfyxhyxf

4

Achim J. Lilienthal

Correlation

Template Matching

f(x,y) is the image

h(x,y) is a template

if h(x,y) matches somewhere in f(x,y) the correlation will be maximal there

∑∑−

=

−

=

++=1

0

1

0

* ),(),(1),(),(M

m

N

nnymxhnmf

MNyxhyxf

4

),(),(),(),( * vuHvuFyxhyxf ⇔

),(),(),(),(* vuHvuFyxhyxf ⇔

Achim J. Lilienthal

Correlation

Template Matching – Example 1

4

f1(x,y) – padded h(x,y) – padded

Achim J. Lilienthal

Correlation


4

F -1[F1*(u,v)H(u,v)] – rescaledf1(x,y) – padded

Achim J. Lilienthal

Correlation


4

F -1[F1*(u,v)H(u,v)] 4 – rescaled4f1(x,y) – padded

Achim J. Lilienthal

Correlation

Definition of Correlation (from previous slide)

Correlation Theorem

∑∑−

=

−

=

++=1

0

1

0

* ),(),(1),(),(M

m

N

nnymxhnmf

MNyxhyxf

),(),(),(),( * vuHvuFyxhyxf ⇔

),(),(),(),(* vuHvuFyxhyxf ⇔

4

Achim J. Lilienthal

Correlation


4


Achim J. Lilienthal

Correlation


4

f2(x,y) – padded F -1[F2*(u,v)H(u,v)] 4 – rescaled4

Achim J. Lilienthal

Correlation


4


Achim J. Lilienthal

Correlation


4

f3(x,y) – padded F -1[F3*(u,v)H(u,v)] 8 – rescaled8

Achim J. Lilienthal

Filters as Templates

Filters as Templates filters respond most strongly to patterns

that look like the filter

the kernel looks like the effect it is intended to detect

filtering has an analogy to computing a dot product measures similarity to the filter kernel

stronger response in brighter areas normalized correlation

filtering as changing the basis convolution can be seen as changing the base of an image

• base: vectors δ-functions base: shifted versions of the filter

• this process will typically loose information (coefficients on the new base can be redundant) but it might expose image structurein a useful way

4

Achim J. Lilienthal

Nyquist-Shannon Sampling Theorem

→ Contents

Achim J. Lilienthal


Aliasing / Undersampling, Moiré Pattern

5

decrease resolution

Achim J. Lilienthal


How to avoid Aliasing Problems? sampling

continuous function (irradiance in the camera) discrete grid

number of samples relative tothe function seems important a signal sampled too slowly is

misrepresented by the samples

5

from "Computer Vision – A Modern Approach", Forsyth and Ponce, Prentice Hall, 2002

Achim J. Lilienthal


Sampling a Signal in 1D

Reconstruction of the Original Continuous Signal which sample rate?

how to derive the continuous signal from the samples?

how to model the sampling process?

f(x)

x

5

x… …

Achim J. Lilienthal


Sampling a Signal in 1D

How to Model the Sampling Process? continuous model of a sampled signal needed

sum of delta functions (2D: "bed-of-nails function") sampling process = multiplication with a sampling function fIII(x)

f(x)

x x… …

5

… …

×

Achim J. Lilienthal


Considering Band-Limited Signals

signal band-width: band/range of non-zero frequencies

a band-limited signal is constrained in terms of how fast it can change

f(x)

x

F(u)

w-w

u

F

→

5

Achim J. Lilienthal


Fourier Transform of a Sampled Signal

sampling = multiplication with the sampling function in the spatial domain

f(x)⋅fIII(x)

x… …

5

Achim J. Lilienthal




equals convolution in the frequency domain

f(x)⋅fIII(x)

x… …

F(u)∗FIII(u)

5

F

→Fourier transform of a

"Dirac comb" is again a Dirac comb

( Poisson summation)

Achim J. Lilienthal




equals convolution in the frequency domain

F

→

…

f(x)⋅fIII(x)

x… …

F(u)∗FIII(u)

u2 w

5

Achim J. Lilienthal



non-overlapping support of the "shifted Fourier Transforms" we can reconstruct the signal from the sampled versions

F

→

…

f(x)⋅fIII(x)

x… …

F(u)∗FIII(u)

u2 w

5

Achim J. Lilienthal


Reconstruction of the Signal

5

Cut out by multiplication with box filterInverse

Fourier Transform

from "Computer Vision – A Modern Approach", Forsyth and Ponce, Prentice Hall, 2002

Achim J. Lilienthal


Reconstruction of the Signal but if support regions do overlap?

we can't reconstruct the signal

Fourier transform in the regions that overlap can't be determined

5

Achim J. Lilienthal


Fourier Transform of a "Dirac comb"

reciprocal behaviour of Δx and Δu

F

→

x

fIII(x)

… …

Δxu

FIII(u)

… …

1/Δx

5

2 w

Achim J. Lilienthal


Sampling Theorem there should be no overlap

between the repetitions of the FT of the signal

the sampling interval should be at least the double of the highest frequency (1/w) present in the signal

wxxw 2112 ≤∆⇒∆≤⇒

5

u

FIII(u)

… …

1/Δx2 w

part 3: fourier transform and filtering in the frequency...

Documents