lect6 notes v1

8/13/2019 Lect6 Notes v1

1/59

General Image Transforms andApplications

Lecture 6, March 3rd, 2008

Lexing Xie

EE4830 Digital Image Processinghttp://www.ee.columbia.edu/~xlx/ee4830/

thanks to G&W website, Min Wu, Jelena Kovacevic and Martin Vetterli for slide materials
http://www.ee.columbia.edu/~xlx/ee4830/http://www.ee.columbia.edu/~xlx/ee4830/


2/59


3/59

outline

Recap of DFT and DCT

Unitary transforms

KLT

Other unitary transforms

Multi-resolution and wavelets Applications

Readings for today and last week: G&W Chap 4,7, Jain 5.1-5.11


4/59

recap: transform as basis expansion


5/59

recap: DFT and DCT basis

1D-DFT

real(A) imag(A)

1D-DCT

AN=32


6/59

recap: 2-D transforms

2D-DFT and 2D-DCT are separable transforms.


7/59

separable 2-D transforms

Symmetric 2D separable transforms canbe expressed with the notations of itscorresponding 1D transform.

We only need to discuss 1Dtransforms


8/59


9/59

Exercise

How do we decompose this picture?

DCT2

11

11

2

12

A

11

11

??

??

DCT2 basis image

DCT2

=-1

=1

What if black=0, does the transform coefficients look similar?

10

00=0

=1

?


10/59

Unitary Transforms

This transform is called unitary when A is a unitary matrix,orthogonal when A is unitary and real.

The Hermitian of matrix A is:

Two properties implied by construction Orthonormality

Completeness

A linear transform:


11/59

Exercise

Are these transform matrixes unitary/orthogonal?

1

1

2

1

j

j

2

2

j

j

cossin

sincos

2

1

2

1

2

1

2

1

21

32


12/59

properties of 1-D unitary transform

energy conservation

rotation invariance

the angles between vectors are preserved

unitary transform: rotate a vector in Rn,i.e., rotate the basis coordinates


13/59

observations about unitary transform

Energy Compaction

Many common unitary transforms tend to pack a large fraction ofsignal energy into just a few transform coefficients De-correlation

Highly correlated input elements quite uncorrelated outputcoefficients

Covariance matrix

display scale: log(1+abs(g))

linear display scale: g

f: columns of image pixels


14/59

one question and two more observations

transforms so far are data-independent

transform basis/filters do not depend on the signal beingprocessed

optimal should be defined in a statistical sense sothat the transform would work well with many images

signal statistics should play an important role

Is there a transform with best energy compaction maximum de-correlation is also unitary ?


15/59

review: correlation after a linear transform

x is a zero-mean random vector in

the covariance (autocorrelation) matrix of x

Rx(i,j) encodes the correlation between xiand xj

Rxis a diagonal matrix iff. all N random variables in x areuncorrelated

apply a linear transform: What is the correlation matrix for y ?


16/59

transform with maximum energy compaction


17/59

proof. maximum energy compaction

a*uare the eigen vectors of Rx


18/59

Karhunen-Love Transform (KLT)

a unitary transform with the basis vectors in A being

the orthonormalized eigenvectors of Rx

assume real input, write ATinstead of AH denote the inverse transform matrix as A, AAT=I Rx is symmetric for real input, Hermitian for complex input

i.e. RxT=Rx, Rx

H= Rx Rxnonnegative definite, i.e. has real non-negative eigen values

Attributions Kari Karhunen 1947, Michel Love 1948 a.k.a Hotelling transform (Harold Hotelling, discrete formulation 1933) a.k.a. Principle Component Analysis (PCA, estimate Rxfrom samples)


19/59

Decorrelation by construction

note: other matrices (unitary or nonunitary) may also de-correlate

the transformed sequence [Jains example 5.5 and 5.7]

Properties of K-L Transform

Minimizing MSE under basis restriction Basis restriction: Keep only a subset of m transform coefficients

and then perform inverse transform (1m N)

Keep the coefficients w.r.t. the eigenvectors of the first m largesteigenvalues


20/59


21/59

energy compaction properties of DCT

DCT is close to KLT when ... x is first-order stationary Markov

DCT basis vectors are eigenvectors of a symmetric tri-diagonal matrixQc

[trigonometric identity cos(a+b)+cos(a-b)=2cos(a)cos(b)]

Rxand 2Rx

-1have thesame eigen vectors

2Rx-1~ Qcwhen is

close to 1


22/59

DCT energy compaction

DCT is close to KLT for

DCT is a good replacement for KLT Close to optimal for highly correlated data

Not depend on specific data

Fast algorithm available

highly-correlated first-order stationary Markov source


23/59

DCT/KLT example for vectors

fraction of

coefficient values inthe diagonal

0.0136

*= 0.8786x: columns of image pixels

1.0000

display scale: log(1+abs(g)), zero-mean

0.1055

0.1185

transform basis


24/59

KL transform for images

autocorrelation function 1D2D

KL basis images are the orthonormalized eigen-functions of R

rewrite images into vector forms (N2

x1) solve the eigen problem for N2xN2matrix ~ O(N6)

if Rxis separable

perform separate KLT on the rows and columns

transform complexity O(N3)


25/59

KLT on hand-written digits

1100 digits 616x16 pixels

1100 vectors of size 256x1


26/59

The Desirables for Image Transforms

Theory Inverse transform available

Energy conservation (Parsevell)

Good for compacting energy

Orthonormal, complete basis

(sort of) shift- and rotation invariant Transform basis signal-independent

Implementation Real-valued

Separable Fast to compute w. butterfly-like structure

Same implementation for forward andinverse transform

DFT KLT

X

X

?XX

x

XX

X

DCT

X

X

?XX

X

XX

X

X

X

XX?

X

x

xx


27/59

Walsh-Hadamard Transform


28/59

slant transform

Nassiri et. al, Texture Feature Extraction using Slant-Hadamard Transform


29/59

energy compaction comparison


30/59

implementation note: block transform

similar to STFT (short-time Fourier transform) partition a NxN image into mxn sub-images

save computation: O(N) instead of O(NlogN)

loose long-range correlation

8x8 DCT coefficients


31/59

applications of transforms

enhancement

(non-universal) compression

feature extraction and representation

pattern recognition, e.g., eigen faces dimensionality reduction

analyze the principal (dominating) components


32/59

Image Compression

where P is average power and A is RMS amplitude.


33/59

Gabor filters

Gaussian windowed Fourier Transform Make convolution kernels from product of Fourier

basis images and Gaussians

=

Odd

(sin)

Even(cos)

Frequency


34/59

Example: Filter Responses

from Forsyth & Ponce

Filterbank

Inputimage


35/59

outline

Recap of DFT and DCT Unitary transforms

KLT

Other unitary transforms

Multi-resolution and wavelets

Applications


36/59

sampling (dirac) FT

STFT


37/59

FT does not capture discontinuities well


38/59


39/59

one step forward from dirac

Split the frequency in half means we can downsample by 2 toreconstruct upsample by 2.

Filter to remove unwanted parts of the images and add

Basic building block: Two-channel filter bank

t

f

x x

analysis synthesisprocessing

h h

g g


40/59

orthogonal filter banks

1. Start from the reconstructed signal

Read off the basis functions


41/59


2.

We want the expansion to be orthonormal The output of the analysis bank is

3. Then The rows of Tare the basis functions The rows of Tare the reversed versions of the filters

The analysis filters are


42/59


4. Since is unitary, basis functions are orthonormal

5.

Final filter bank


43/59

orthogonal filter banks: Haar basis


44/59


45/59


46/59

DWT

Iterate only on the lowpass channel

t

f


47/59

wavelet packet

t

f


48/59

wavelet packet

First stage: full decomposition


49/59

wavelet packet

Cost(parent)


50/59

wavelet packet: why it works

t

f

Holy Grail of SignalAnalysis/Processing Understand the blob-like

structure of the energy

distribution in the time-frequency space

Design a representationreflecting that

Dirac basis


51/59

t

fFT

t

fSTFT

t

fWP

t

fWT


52/59

are we solving x=x? sort of: find matrices such that

after finding those Decomposition

Reconstruction

in a nutshell if is square and nonsingular, is a basis and is its dual basis

if is unitary, that is, * = I, is an orthonormal basis and=

if is rectangular and full rank, is a frame and is its dualframe

if is rectangular and * = I , is a tight frame and =


53/59

overview of multi-resolution techniques


54/59

applications of wavelets

enhancement and denoising

compression and MR approximation

fingerprint representation with wavelet packets

bio-medical image classification

subdivision surfaces Geris Game, A BugsLife, Toy Story 2


55/59

fingerprint feature extraction

MR system

Introduces adaptivity Template matching

performed on differentspace-frequency regions

Builds a different

decomposition for eachclass


56/59

fingerprint identification results

0

10

20

30

40

50

60

70

80

90

100

SCF Average IER = 18.41% WDCF Average IER = 1.68%

IdentificationErrorRate(%)

SCF 0 0 9 .7 8 3 .2 6 4 .3 5 35 .9 0 3 3.7 0 8 9.9 6 6 .5 2 9 .7 8 0 3 .2 6 66 .3 0 1 5.2 2 8 .7 0 21 .7 4 3 3.7 0 1 4.1 3 0 1 1.9 6

WDCF 0 0 0 0 0 5.43 0 0 0 0 0 0 15.22 0 0 13.04 0 0 0 0

1 2 3 4 5 6 7 8 9 10 11 1 2 13 14 15 16 17 1 8 19 20

Standard Correlation Filters

Wavelet Correlation Filters

NIST 24 fingerprint database10 people (5 male & 5 female), 2 fingers20 classes, 100 images/class


57/59

references for multiresolution Light reading

Wavelets: Seeing the Forest -- and the Trees, D. Mackenzie, Beyond Discovery,

December 2001. Overviews

Books

Wavelets and Subband Coding, M. Vetterli and J. Kovacevic, Prentice Hall, 1995. A Wavelet Tour of Signal Processing, S. Mallat, Academic Press, 1999.

Ten Lectures on Wavelets, I. Daubechies, SIAM, 1992.

Wavelets and Filter Banks, G. Strang and T. Nguyen, Wells. Cambr. Press, 1996.

ELEN E6860 Advanced Digital Signal Processing
http://localhost/var/www/apps/conversion/tmp/scratch_1/Papers/SeeingTheForrest.pdfhttp://andrew.cmu.edu/user/jelenak/Book/index.htmlhttp://andrew.cmu.edu/user/jelenak/Book/index.htmlhttp://localhost/var/www/apps/conversion/tmp/scratch_1/Papers/SeeingTheForrest.pdfhttp://localhost/var/www/apps/conversion/tmp/scratch_1/Papers/SeeingTheForrest.pdfhttp://localhost/var/www/apps/conversion/tmp/scratch_1/Papers/SeeingTheForrest.pdfhttp://localhost/var/www/apps/conversion/tmp/scratch_1/Papers/SeeingTheForrest.pdf


58/59

summary

unitary transforms theory revisited

the quest for optimal transform

example transforms

DFT, DCT, KLT, Hadamard, Slant, Haar, multire-solution analysis and wavelets

applications compression

feature extraction and representation

image matching (digits, faces, fingerprints)


59/59

10 yrs

lect6 notes v1

Documents