LINEAR SYSTEMS THEORY

Ho Kyung Kim, Ph.D.hokyung@pusan.ac.kr

School of Mechanical EngineeringPusan National University

Introduction to Medical Engineering

Even / odd / periodic functions

2

• Think about cosine & sine functions!

• Even if s(-x) = s(x);

• Odd if s(-x) = -s(x);

• Can write any signal as the sum of an even and an odd part:

• Periodic if s(x + X) = s(x)

∫∫∞∞

∞−=

0)(2)( dxxsdxxs ee

0)( =∫∞

∞−dxxso

)()(

2

)(

2

)(

2

)(

2

)()(

xsxs

xsxsxsxsxs

oe +=

−−+

−+=

• In Cartesian representation

• In polar representation

),(),(),( 22 yxvyxuyxs +=

Complex function

3

φ

|s(x, y)|

s(x, y)

),(),(),( yxivyxuyxs +=

),(),(),( yxieyxsyxs φ=

),(

),(arctan),(

yxu

yxvyx =φ

Real axis

Imaginary axis

u

v

where

modulus or amplitude

argument or phase

Important signal functions

4

• Exponential axeax =)exp(

• Complex exponential or sinusoid

[ ])2sin()2cos()2( φ+π+φ+π=φ+π kxikxAAe kxiA = amplitudek = spatial frequencyφ = phase

5

• Rectangular function (width = 2L)

• Step function (or Heaviside’s function)

Lx

Lx

LxL

x

>=

==

=

==

7

• Sinc function

x

xx

)sin()(sinc =

• Dirac impulse

00 for 0)( xxxx ≠=−δ

1)( 0 =−δ∫∞

∞−dxxx

)()()( 00 xsdxxxxs =−δ∫∞

∞−

AdxxxA =−δ∫∞

∞−)( 0

shifting

scaling xx0

Linear systems

8

Modeling: the process of finding a mathematical relationship btwn input & output signal

Lsi soinput signal(excitation)

output signal(response)

e.g., an amplifier with gain A;

{ }io sLs =

{ } )()()( tAstsLts iio ==

Linear system if the superposition principle holds; { } { } { }22112211 sLcsLcscscL +=+

L

s1

so+

s2

Ls1

so+

s2 L

{ }

{ } { }22112211

22112211

)()(

)(

sLcsLc

sAcsAc

scscAscscL

+=+=

+=+

e.g., amplifiers with gain A;

Nonlinear system; { }2

222

11

222112211

)()(

)(

scsc

scscscscL

+≠

+=+

Shift-invariant system

9

• Shift-invariant system if its properties do not change with spatial position;

{ })()( XxsLXxs io −=−

shift shift

shift invariant(no change)

shift variant(changes with position)

• LSI systems = linear & shift-invariant systems

Lx x

• Impulse response, h(x), to a Dirac impulse

For an arbitrary signal; ∫∞

∞−ξξ−δξ= dxsxs ii )()()(

Then, the response of an LSI system;

{ } { } ∫∫∞

∞−

∞

∞−ξξ−ξ=ξξ−δξ== dxhsdxLsxsLxs iiio )()()()()()(

convolution; hss io ∗=

PSF(Point spread function)

Convolution

10

)()()()()()()()( 12212121 xsxsdxssdsxsxsxs ∗=−=−=∗ ∫∫∞

∞−

∞

∞−ξξξξξξ

• Procedure– mirroring s2 about ξ = 0 by changing ξ to –ξ– translating the mirrored s2 by ξ = x– multiplying s1 to the shifted & mirrored s2– integrating the resulting signal (represented by area)– repeating the previous steps for each value of x

11

• Convolution for digital signals

0 1

1

0-2 3

2

0.5

0-2 4

3

1.51

0.5

4

∗ =

12

• Convolution for multidimensional signals;

– the convolution values are represented by volumes.

• Properties– commutativity:

– associativity:

– distributivity:

∫∞

∞−ζξζξζ−ξ−=∗ ddsyxsyxsyxs ),(),(),(),( 2121

1221 ssss ∗=∗

)()( 321321 ssssss ∗∗=∗∗

3121321 )( sssssss ∗+∗=+∗

Response of an LSI system

13

ikxi Aexs

π= 2)(

{ }

)()()(

)(

)(

)()()()(

2

22

)(2

kHxskHAe

dheAe

dhAe

dhxsxsLxs

iikx

ikikx

xik

iio

==

ξξ=

ξξ=

ξξξ−==

π

∞

∞−

ξπ−π

∞

∞−

ξ−π

∞

∞−

∫

∫

∫

∫∞

∞−

ξπ− ξξ= dhekH ik )()( 2

∫∞

∞−

π= dkekSxs ikxii2)()(

{ })()()()()( 2 kHkSdkekHkSxs iikxio 1FT−∞

∞−== ∫

π

For an input signal (sinusoid);

The response of an LSI system;

where = Fourier transform of the PSF h(x)= transfer function (or filter)

Inverse Fourier transform;

Then, we have;

Discuss; )()()( xhxsxs io ∗= )()()( kHkSkS io =in x domain vs. in k domain

Any input signal can be written as an integral of weighted sinusoids with different spatial frequencies

{ } { }[ ])()()( kHkSxs io 1FTFTFT −=

Frequency

14

• Recall

0.0 0.2 0.4 0.6 0.8 1.0

-1

0

1

k = 4

k = 2

No

rma

lize

d A

mp

litu

de

x

k = 1

)2sin()2cos(2 φ+π+φ+π=π kxikxe ikx

– as k increases, so does the frequency of the oscillation– the higher k, the higher the signal resolution, that is, one can represent smaller signal

details (signal that varies more quickly)

Signal synthesis

15

• Any periodic signal can be created by a combination of weighted and shifted sinusoids at different frequencies.

( )

∫

∫

∫

∫

∞

∞−

π

∞

∞−

πφ

∞

∞−

φ+π

∞

∞−

=

=

=

φ+π+φ+π=

dkekS

dkeeA

dkeA

dkkxikxAxs

ikxi

ikxik

kxik

kkko

k

k

2

2

)2(

)(

)2sin()2cos()(

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

box sin(x) 1/3 sin(3x) sum1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

box sin(x) 1/3 sin(3x) 1/5 sin(5x) sum2

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

box sin(x) 1/3 sin(3x) 1/5 sin(5x) 1/7 sin(7x)

1/9 sin(9x) 1/11 sin(11x) 1/13 sin(13x) 1/15 sin(15x) sumf

Inverse FT!!!

Fourier transform

16

• Forward transform

{ } ∫∞

∞−

π−=ℑ= drersrskS irk2)()()(

{ } ∫∞

∞−

π− =ℑ= dkekSkSrs irk21 )()()(

{ } ∫∞

∞−

⋅π−=ℑ= rdersrskS krirrrr

rr2)()()(

{ } ∫∞∞− ⋅π− =ℑ= kdekSkSrs krirrr

rr21 )()()(

• Inverse transform

• Conjugate variables− if r is time dimension "seconds", k is temporal frequency with dimension "hertz"− if r is spatial position with dimension "mm", k is spatial frequency with dimension "mm-1"

FT{rect}

17

• A finite signal in the x-domain creates an infinite signal in the k-domain.– the same is true vice versa

)2(sinc2

)2sin(22

)(2

22

22

2

2

kLAL

kLk

Aee

ik

A

dxAe

dxeL

xA

L

xA

ikLikL

L

L

ikx

ikx

π=

ππ

=−π

−=

=

∏=

∏ℑ

ππ−

−

π−

∞

∞−

π−

∫

∫

2AL

A 2AL

k

k = 1/2L

k = 1/L

FT{step × exp}

18

– real part:

– imaginary part:

– modulus:

– phase:

{ }

222222

0

)2(

2

4

2

4

2

1

)()(

ka

ki

ka

aika

dxe

dxeexuexu

xika

ikxaxax

π+π−

π+=

π+=

=

=ℑ

∫

∫∞ π+−

∞

∞−

π−−−

222 4 ka

a

π+

222 4

2

ka

k

π+π−

222 4

1

ka π+

π−a

k2arctan

FT{Dirac impulse}

19

{ }02

200 )()(

ikx

ikx

e

dxexxxx

π−

∞

∞−

π−

=

−δ=−δℑ ∫

xx0 k

1

FT{cosine}

20

• The spectrum consists of two impulses at spatial frequencies k0 and –k0.• A periodic function has a discrete spectrum (i.e., not all spatial frequencies are present).• An aperiodic function has a continuous spectrum.

{ }

)(2

1)(

2

12

1

2

12

)2cos()2cos(

00

)(2)(2

222

200

00

00

kkkk

dxedxe

dxeee

dxexkxk

xkkixkki

ikxxikxik

ikx

+δ+−δ=

+=

−=

π=πℑ

∫∫

∫

∫

∞

∞−

+π−∞

∞−

−π−

∞

∞−

π−π−π

∞

∞−

π−

Fourier transform pairs

21

• Image space • Fourier space

)(kδ

)(xδ

)2cos( 0xkπ ( ))()(2

100 kkkk −δ++δ

)2sin( 0xkπ ( ))()(2

100 kkkk −δ−+δ

∏L

x

2)2(sinc2 LkL π

ΛL

x

2)(sinc2 LkL π

)(xGn2222 σπ− ke

1

1

Properties

22

• Linearity:

• Scaling:

• Translation:

• Convolution:

• Parseval's theorem:

• Separability:

22112211 ScScscsc +↔+

↔a

kS

aaxs

1)(

)()( 020 kSexxskixπ−↔−

2121 SSss ⋅↔∗ 2121 SSss ∗↔⋅

∫∫∞

∞−

∞

∞−= dkkSdxxs 22 )()(

{ } { } { })(sinc)(sinc)(sinc)(sinc yxyx ℑℑ=ℑ

modifying only its “phase” spectrum

23

• In imaging, the FT of the PSF is known as the optical transfer function (OTF).

– the modulus of the OTF is the modulation transfer function (MTF)

• The PSF (mm) and OTF (mm-1 or lp/mm) characterize the resolution of the system.

)()( kHxh ↔

• Transfer function and impulse response (or PSF) are an FT pair

Note

24

• If the signal is discrete but infinite, then the frequency spectrum is continuous but is periodic (has aliases).

• If the signal is discrete and finite (N samples), then the frequency spectrum is discreteand periodic in N.

FT in polar coordinates

25

• Forward transform

∫ ∫

∫ ∫

∫ ∫

π ∞

∞−

θ+θπ−

π ∞ θ+θπ−

∞

∞−

∞

∞−

+π−

θθ=

θθ=

=

0

)sincos(2

2

0 0

)sincos(2

)(2

),(

),(

),(),(

drdrers

rdrders

dxdyeyxskkS

rkrki

rkrki

ykxkiyx

yx

yx

yx

rrr

ry

r

y

x

r

x

J =θ+θ=θθθ−θ

=

θ∂∂

∂∂

θ∂∂

∂∂

≡ )sin(coscossin

sincos 22

∫ ∫π ∞

∞−

φ+φπ φφ=0

)sincos(2),(),( dkdkeksyxs ykxki

• Inverse transform

Note:

Sampling

26

• Sampled signal:– comb function or impulse train– ∆x = sampling distance– information may be lost by sampling– can we recover a continuous signal completely from its samples?

)()()()()( xxsxnsxsxs s III⋅=∆=→

• Sampling theorem (Nyquist criterion)− if the FT of a given signal is band-limited and if the sampling frequency is larger than twice the

max. spatial frequency present in the signal, then the samples uniquely define the signal

)( definesuniquely )()(then

21

0)( If

max

max

xsxnsxs

kx

kkkS

s ∆=

>∆

>∀=

∑∞

−∞=∆−δ=

n

xnxx )()(III

{ })()()( xkSkSs IIIℑ∗= { } ∑∞

−∞=−δ=ℑ

l

lKkKx )()(IIIx

K∆

= 1& with

Hence, ( )L+++−+++−+= )2()2()()()()( KkSKkSKkSKkSkSKkS s

2 0)(

KkkS ≥∀=

∏=K

kkSkKS s )()(Note that because

Infinite spatial extent – it’s a band-limited FT

27

Finite spatial extent – it’s a not-band-limited FT

28

• If the signal S(k) is not band limited, or if it is band limited but 1/∆x ≤ 2kmax, the shifted replicas of S(k) will overlap.

• Therefore, the spectrum of S(k) cannot be recovered by multiplication with a rectangular pulse.

• Known as aliasing and unavoidable if the original signal s(x) is not band limited.

– “Patients” always have a limited spatial extent!

Aliasing: A commonly observed phenomenon

29

Taken from Dr. K. Mueller’s Slides

Anti-aliasing

30

Taken from Dr. K. Mueller’s Slides

Discrete FT

31

• Forward transform

• Inverse transform

• Fast Fourier transform (FFT)– for the number of samples in a power of two– nlogn flops in 1D– n2logn flops in 2D

∑∑−

=

−

=

+π−∆∆=∆∆

1

0

1

0

2

),(),(N

q

M

p

N

nq

M

mpi

yx eyqxpsknkmS

∑∑−

=

−

=

+π∆∆=∆∆

1

0

1

0

2

),(),(N

n

M

m

N

nq

M

mpi

yx eknkmSyqxps