Wavelets (Chapter 7)CS474/674 Prof. Bebis

*

STFT (revisited)Time/Frequency localization depends on window size.Once you choose a particular window size, it will be the same for all frequencies.Many signals require a more flexible approach - vary the window size to determine more accurately either time or frequency.

The Wavelet TransformOvercomes the preset resolution problem of the STFT by using a variable length window:

Use narrower windows at high frequencies for better time resolution.

Use wider windows at low frequencies for better frequency resolution.

The Wavelet Transform (contd)

Wide windows do not provide good localization at high frequencies.

The Wavelet Transform (contd)

Use narrower windows at high frequencies.

The Wavelet Transform (contd)

Narrow windows do not provide good localization at low frequencies.

The Wavelet Transform (contd)

Use wider windows at low frequencies.

What are Wavelets?Wavelets are functions that wave above and below the x-axis, have (1) varying frequency, (2) limited duration, and (3) an average value of zero.

This is in contrast to sinusoids, used by FT, which have infinite energy.

Sinusoid Wavelet

What are Wavelets? (contd)Like sines and cosines in FT, wavelets are used as basis functions k(t) in representing other functions f(t):

Span of k(t): vector space S containing all functions f(t) that can be represented by k(t).

What are Wavelets? (contd)

There are many different wavelets:

MorletHaar Daubechies

(dyadic/octave grid)What are Wavelets? (contd)

What are Wavelets? (contd)time localizationscale/frequency localizationj

Continuous Wavelet Transform (CWT)Continuous Wavelet Transform of signal f(t)translation parameter, measure of timescale parameter (measure of frequency)Mother wavelet (window)normalization constantForwardCWT:Scale = 1/j = 1/Frequency

CWT: Main StepsTake a wavelet and compare it to a section at the start of the original signal.

Calculate a number, C, that represents how closely correlated the wavelet is with this section of the signal. The higher C is, the more the similarity.

CWT: Main Steps (contd)3. Shift the wavelet to the right and repeat steps 1 and 2 until you've covered the whole signal.

CWT: Main Steps (contd)4. Scale the wavelet and repeat steps 1 through 3.

5. Repeat steps 1 through 4 for all scales.

Coefficients of CTW TransformWavelet analysis produces a time-scale view of the input signal or image.

Continuous Wavelet Transform (contd)Inverse CWT:

double integral!

FT vs WTweighted by F(u)weighted by C(,s)

Properties of WaveletsSimultaneous localization in time and scale

- The location of the wavelet allows to explicitly represent the location of events in time.- The shape of the wavelet allows to represent different detail or resolution.

Properties of Wavelets (contd)Sparsity: for functions typically found in practice, many of the coefficients in a wavelet representation are either zero or very small.

Linear-time complexity: many wavelet transformations can be accomplished in O(N) time.

Properties of Wavelets (contd)Adaptability: wavelets can be adapted to represent a wide variety of functions (e.g., functions with discontinuities, functions defined on bounded domains etc.). Well suited to problems involving images, open or closed curves, and surfaces of just about any variety.Can represent functions with discontinuities or corners more efficiently (i.e., some have sharp corners themselves).

Discrete Wavelet Transform (DWT)(inverse DWT)(forward DWT)where

DFT vs DWTFT expansion:

WT expansion

orone parameter basistwo parameter basis

Multiresolution Representation using

jfinedetailscoarsedetailswider, large translations

Multiresolution Representation using

jfinedetailscoarsedetails

Multiresolution Representation using

jfinedetailscoarsedetailsnarrower, small translations

Multiresolution Representation using high resolution (more details)low resolution (less details)

j

Prediction Residual Pyramid (revisited)

In the absence of quantization errors, the approximation

pyramid can be reconstructed from the prediction residual pyramid.

Prediction residual pyramid can be represented more

efficiently.(with sub-sampling)

Efficient Representation Using Detailsdetails D2L0 details D3

details D1(no sub-sampling)

Efficient Representation Using Details (contd)representation: L0 D1 D2 D3A wavelet representation of a function consists of a coarse overall approximation detail coefficients that influence the function at various scales.

(decompositionor analysis)in general: L0 D1 D2 D3DJ

Reconstruction (synthesis)

H3=L2+D3details D2L0 details D3H2=L1+D2H1=L0+D1details D1(no sub-sampling)

Example - Haar WaveletsSuppose we are given a 1D "image" with a resolution of 4 pixels:

[9 7 3 5]

The Haar wavelet transform is the following:

L0 D1 D2 D3(with sub-sampling)

Example - Haar Wavelets (contd)Start by averaging the pixels together (pairwise) to get a new lower resolution image:

To recover the original four pixels from the two averaged pixels, store some detail coefficients.

1

Example - Haar Wavelets (contd)Repeating this process on the averages gives the full decomposition:

1

Example - Haar Wavelets (contd)The Harr decomposition of the original four-pixel image is:

We can reconstruct the original image to a resolution by adding or subtracting the detail coefficients from the lower-resolution versions.

21 -1

Example - Haar Wavelets (contd)Note small magnitudedetail coefficients!DjDj-1D1L0How tocompute Di ?

Multiresolution ConditionsIf a set of functions can be represented by a weighted sum of (2jt - k), then a larger set, including the original, can be represented by a weighted sum of (2j+1t - k):

time localizationscale/frequency localizationlow resolutionhigh resolutionj

Multiresolution Conditions (contd)If a set of functions can be represented by a weighted sum of (2jt - k), then a larger set, including the original, can be represented by a weighted sum of (2j+1t - k):

Vj: span of (2jt - k): Vj+1: span of (2j+1t - k):

Nested Spaces Vj(t - k) (2t - k) (2jt - k) V0V1VjVj : space spanned by (2jt - k) Multiresolution conditions nested spanned spaces:f(t) Vj

Basis functions:i.e., if f(t) V j then f(t) V j+1

How to compute Di ? (contd)f(t) Vj

IDEA: define a set of basisfunctions that span thedifferences between Vj

Orthogonal Complement Wj Let Wj be the orthogonal complement of Vj in Vj+1

Vj+1 = Vj + Wj

How to compute Di ? (contd) If f(t) Vj+1, then f(t) can be represented using basis functions (t) fromVj+1:

Vj+1 = Vj + WjAlternatively, f(t) can be represented using two basis functions, (t) from Vj and (t) from Wj:Vj+1

Think of Wj as a means to represent the parts of a function in Vj+1 that cannot be represented in VjVj, WjHow to compute Di ? (contd)

differencesbetweenVj and Vj+1

How to compute Di ? (contd) using recursion on Vj:

V0 W0, W1, W2, basis functions basis functionsVj+1 = Vj-1+Wj-1+Wj = = V0 + W0 + W1 + W2 + + Wjif f(t) Vj+1 , then:Vj+1 = Vj + Wj

Summary: wavelet expansion (Section 7.2)Wavelet decompositions involve a pair of waveforms (mother wavelets):

The two shapes are translated and scaled to produce wavelets (wavelet basis) at different locations and on different scales.

(t) (t)(t-k) (2jt-k)encode lowresolution infoencode details orhigh resolution info

Summary: wavelet expansion (contd) f(t) is written as a linear combination of (t-k) and (2jt-k) :

scaling function wavelet function

1D Haar Wavelets Haar scaling and wavelet functions:

computes averagecomputes details(t) (t)

1D Haar Wavelets (contd) Think of a one-pixel image as a function that is constant over [0,1)

We will denote by V0 the space of all such functions.

Example:0 1

1D Haar Wavelets (contd) Think of a two-pixel image as a function having two constant pieces over the intervals [0, 1/2) and [1/2,1)

We will denote by V1 the space of all such functions.

Note that

Examples:0 1=+

1D Haar Wavelets (contd)V j represents all the 2j-pixel imagesFunctions having constant pieces over 2j equal-sized intervals on [0,1).

Note that

Examples:width: 1/2j Vj Vj

1D Haar Wavelets (contd)V0, V1, ..., V j are nestedi.e., VJ

V2V1coarse detailsfine details

1D Haar Wavelets (contd)Mother scaling function:

Lets define a basis for V j :

0 1note alternative notation:1

1D Haar Wavelets (contd)

1D Haar Wavelets (contd)Suppose Wj is the orthogonal complement of Vj in Vj+1

1D Haar Wavelets (contd)Mother wavelet function:

Note that (x) . (x) = 0 (i.e., orthogonal)

1-10 1/2 10 1.= 0 1-10 1/2 11

1D Haar Wavelets (contd)Mother wavelet function:

Lets define a basis ji for Wj :

1-10 1note alternative notation:

1D Haar Wavelets (contd)j=1basis W 1 :basis for V 1 :

Notethat innerproductis zero!

1D Haar Wavelets (contd)Basis functions ji of W j Basis functions ji of V j

form a basis in V j+1

1D Haar Wavelets (contd)V3 = V2 + W2

1D Haar Wavelets (contd)V2 = V1 + W1

1D Haar Wavelets (contd)V1 = V0 + W0

1D Haar Wavelets- Summary(t)(t)

Example - Haar basis (revisited)

1

Decomposition of f(x)V2f(x)=

Decomposition of f(x) (contd)V1and W1V2=V1+W1

Example - Haar basis (revisited)

1

Decomposition of f(x) (contd)

Decomposition of f(x) (contd)V2=V1+W1=V0+W0+W1V0 ,W0 and W1

Example - Haar basis (revisited)

1

Example

Example (contd)

Filter banks (analysis)The lower resolution coefficients can be calculated from the higher resolution coefficients by a tree-structured algorithm (filter bank).

a0k (j=0)a1k (j=1)(t - k) (2t - k) Example:

Filter banks (analysis) (contd)The lower resolution coefficients can be calculated from the higher resolution coefficients by a tree-structured algorithm (filter bank).

h0(-n) is a lowpass filter and h1(-n) is a highpass filterSubbandencoding!

Example - Haar basis (revisited)[9 7 3 5]low-pass,down-samplinghigh-pass, down-sampling(9+7)/2 (3+5)/2 (9-7)/2 (3-5)/2V1 basis functions

Filter banks (analysis) (contd)

Example - Haar basis (revisited)[9 7 3 5]high-pass, down-samplinglow-pass,down-sampling(8+4)/2 (8-4)/2 V1 basis functions

Convention for illustrating 1D Haar wavelet decompositionx x x x x x x x

detailaverage

re-arrange:re-arrange:

V1 basis functions

Convention for illustrating1D Haar wavelet decomposition (contd)x x x x x x x x

detailaverage

Orthogonality and normalizationThe Haar basis forms an orthogonal basisIt can become orthonormal through the following normalization:

since

Examples of lowpass/highpass analysis filtersDaubechiesHaarh0h1h0h1

Filter banks (synthesis) The higher resolution coefficients can be calculated from

the lower resolution coefficients using a similar structure.

Filter banks (synthesis) (contd)

Examples of lowpass/highpass synthesis filtersDaubechiesHaar (same as for analysis):+g0g1g0g1

2D Haar Wavelet TransformThe 2D Haar wavelet decomposition can be computed using 1D Haar wavelet decompositions (i.e., 2D Haar wavelet basis is separable).Two decompositionsStandard decompositionNon-standard decompositionThe two decompositions correspond to a different set of 2D basis functions.

Standard Haar wavelet decompositionSteps

(1) Compute 1D Haar wavelet decomposition of each row of the original pixel values.

(2) Compute 1D Haar wavelet decomposition of each column of the row-transformed pixels.

Standard Haar wavelet decomposition (contd)x x x xx x x x

.x x x ... x

(1) row-wise Haar decomposition:

detailaverage .

.

re-arrange terms

Standard Haar wavelet decomposition (contd)

(1) row-wise Haar decomposition:

detailaverage

.

.

row-transformed result

Standard Haar wavelet decomposition (contd)

(2) column-wise Haar decomposition:

detailaverage

.

.

row-transformed resultcolumn-transformed result

Example

Example (contd)

.

column-transformed result

2D Haar basis for standard decompositionTo construct the standard 2D Haar wavelet basis, consider all possible outer products of 1D basis functions.V2=V0+W0+W1Example:

2D Haar basis for standard decompositionTo construct the standard 2D Haar wavelet basis, consider all possible outer products of 1D basis functions.00(x), 00(x) 00(x), 00(x)01(x), 00(x)

2D Haar basis of standard decompositionV2

Non-standard Haar wavelet decompositionAlternates between operations on rows and columns.

(1) Perform one level decomposition in each row (i.e., one step of horizontal pairwise averaging and differencing).

(2) Perform one level decomposition in each column from step 1 (i.e., one step of vertical pairwise averaging and differencing).

(3) Repeat the process on the quadrant containing averages only (i.e., in both directions).

Non-standard Haar wavelet decomposition (contd)x x x xx x x x

.x x x . . . xone level, horizontalHaar decomposition:

.

.

one level, vertical Haar decomposition:

Note: averaging/differencingof detail coefficients shown as

Non-standard Haar wavelet decomposition (contd)one level, horizontalHaar decompositionon green quadrant

one level, vertical Haar decompositionon green quadrant

.

re-arrange terms

.

Example

.

re-arrange terms

Example (contd)

.

2D Haar basis for non-standard decomposition

Defined through 2D scaling and wavelet functions:

2D Haar basis for non-standard decomposition (contd)

LLLH: intensity variations along columns (horizontal edges)HL: intensity variations along rows (vertical edges)HH: intensity variations along diagonalsLL: averageDetail coefficientsThree sets of detail coefficients (i.e., subband encoding)

2D Haar basis for non-standard decomposition (contd)V2

Forward/Inverse DWT(using textbooks notation)

H LHV HLD HHLLi=H,V,D

2D DWT using filter banks (analysis)H LHV HLD HHLHHHLLHL

Illustrating 2D wavelet decompositionThe wavelet transform can be applied again on the lowpass-lowpass version of the image, yielding seven subimages.LLLHHLHHLLHHHLLH

2D IDWT using filter banks (synthesis)H LHV HLD HH

Wavelets ApplicationsNoise filteringImage compressionFingerprint compressionImage fusionRecognition

G. Bebis, A. Gyaourova, S. Singh, and I. Pavlidis, "Face Recognition by Fusing Thermal Infrared and Visible Imagery", Image and Vision Computing, vol. 24, no. 7, pp. 727-742, 2006. Image matching and retrieval

Charles E. Jacobs Adam Finkelstein David H. Salesin, "Fast Multiresolution Image Querying", SIGRAPH, 1995.

Image Querying Using Wavelets query by content or query by example

Typically, the K best matches are reported.

Fast Multiresolution Image Queryingpaintedlow resolutiontarget

queries

*