a tutorial onhelper.ipam.ucla.edu/publications/es2002/es2002_1441.pdfconnections with fourier...
TRANSCRIPT
A Tutorial on
Wavelets and their Applications
Martin J. Mohlenkamp
University of Colorado at Boulder
Department of Applied Mathematics
This tutorial is designed for people with little or noexperience with wavelets. We will cover the basicconcepts and language of wavelets and computationalharmonic analysis, with emphasis on the applicationsto numerical analysis. The goal is to enable those whoare unfamiliar with the area to interact moreproductively with the specialists.
Thanks to Gregory Beylkin, Willy Hereman, and
Lucas Monzon for help with this tutorial.
Goals
• Enable the wavelet novice to interact
more productively with the specialists, by
– Introducing the basic concepts and
language
– Doing some physics related examples
– Explaining why people like them
Not Goals
• Give the history and assign credit
• Convince you that wavelets are better
than any particular technique for any
particular problem
Outline
• Multiresolution Analysis
• wavelets (traditional)
• properties
• fast algorithms
• Connections with Fourier Analysis
• Local Cosine and the phase plane
• 1D Schrodinger example
• wavelet packets
• Operators in Wavelet Coordinates
• density matrix example
• Review and Questions
Multiresolution Analysis
A multiresolution analysis is a decomposition
of L2(R), into a chain of closed subspaces
· · · ⊂ V2 ⊂ V1 ⊂ V0 ⊂ V−1 ⊂ V−2 ⊂ · · · ⊂ L2(R)
such that
1.⋂
j∈Z Vj = {0} and⋃
j∈Z Vj is dense in L2(R)
2. f(x) ∈ Vj if and only if f(2x) ∈ Vj−1
3. f(x) ∈ V0 if and only if f(x− k) ∈ V0
for any k ∈ Z.
4. There exists a scaling function ϕ ∈ V0
such that {ϕ(x− k)}k∈Z is an orthonormal
basis of V0.
Multiresolution Analysis
Let Wj be the orthogonal complement
of Vj in Vj−1:
Vj−1 = Vj ⊕ Wj,
so that
L2(R) =
⊕
j∈Z
Wj.
Selecting a coarsest scale Vn and finest scale
V0, we truncate the chain to
Vn ⊂ · · · ⊂ V2 ⊂ V1 ⊂ V0
and obtain
V0 = Vn
n⊕
j=1
Wj.
From the scaling function ϕ we can define
the wavelet ψ, such that {ψ(x− k)}k∈Z is an
orthonormal basis of W0.
Multiresolution Analysis
Example: Haar wavelets
ϕ(x) =
1 for 0 < x < 1
0 elsewhere.
V0 = span({ϕ(x− k)}k∈Z) are piecewise
constant functions with jumps only at
integers.
ψ(x) =
1 for 0 < x < 1/2
−1 for 1/2 ≤ x < 1
0 elsewhere.
,
W0 = span({ψ(x− k)}k∈Z) are piecewise
constant functions with jumps only at
half-integers, and average 0 between integers.
Multiresolution Analysis
?
?
?
BBBBBBBBBN
BBBBBBBBBN
BBBBBBBBBN
V3 W3
V2 W2
V1 W1
V0
V0 = V3 ⊕ W3 ⊕ W2 ⊕ W1
Where’s the Wavelet?
Since Wj is a dilation of W0, we can define
ψj,k = 2−j/2ψ(2−jx− k)
and have
Wj = span({ψj,k(x)}k∈Z).
In this example,
W1 = span
W2 = span
W3 = span
The Wavelet Zoo
φ ψ
Haar
0.2 0.4 0.6 0.8 1
0.2
0.4
0.6
0.8
1
0.2 0.4 0.6 0.8 1
-1
-0.5
0.5
1
daub40.5 1 1.5 2 2.5 3
-0.25
0.25
0.5
0.75
1
1.25
-1 -0.5 0.5 1 1.5 2
-1
-0.5
0.5
1
1.5
daub12 2 4 6 8 10
-0.4
-0.2
0.2
0.4
0.6
0.8
1
-4 -2 2 4 6
-1
-0.5
0.5
1
coif4-2 -1 1 2 3
0.5
1
1.5
-2 -1 1 2 3
-2
-1.5
-1
-0.5
0.5
1
coif12-5 -2.5 2.5 5 7.5 10
-0.2
0.2
0.4
0.6
0.8
1
-7.5 -5 -2.5 2.5 5 7.5
-1
-0.5
0.5
Vanishing Moments
Wavelets are usually designed with vanishing
moments:∫ +∞
−∞ψ(x)xmdx = 0, m = 0, . . . ,M − 1,
which makes them orthogonal to the low
degree polynomials, and so tend to compress
non-oscillatory functions.
For example, we can expand in a Taylor series
f(x) = f(0) + f ′(0)x+ · · · + f(M−1)(0)xM−1
(M − 1)!
+ f(M)(ξ(x))xM
M !
and conclude
|〈f, ψ〉| ≤ maxx
∣
∣
∣
∣
∣
f(M)(ξ(x))xM
M !
∣
∣
∣
∣
∣
.
Haar has M = 1.
Quadrature Mirror Filters
Wavelets are designed through properties of
their “quadrature mirror filter” {H,G}.
Haar ↔H =1√2[1,1]
G =1√2[1,−1]
daub4 ↔H =1
4√
2[1 +
√3,3 +
√3,3 −
√3,1 −
√3]
G = [H(3),−H(2), H(1),−H(0)]}
(The values are usually not in closed form.)
For instance, vanishing moments
∫ +∞
−∞ψ(x)xmdx = 0, m = 0, . . . ,M − 1,
are a consequence of
∑
i
G(i)im = 0, m = 0, . . . ,M − 1.
Trade-offs
You can get
• higher M
• more derivatives
• closer to symmetric
• closer to interpolating (coiflets)
if you pay by increasing the filter length,
which causes
• longer (overlapping) supports, and so
worse localization.
• slower transforms
The cost is linear (in M etc.).
Fast Wavelet Transform
Sample onto the finest resolution and then
apply the “quadrature mirror filter” {H,G}.
V3 • ◦ W3
V2 • • ◦ ◦ W2
V1
• • • • ◦ ◦ ◦ ◦ W1
V0 • • • • • • • •
H�
����
AAAAUG
H�
����
AAAAUG
HAAAAU
��
���G
The total cost of this cascade is 2 ·N · L,
where L is the length of the filter.
There are many, many ’lets
By loosening the definitions, you can get
• symmetric
• interpolating
• 2D properties (brushlets)
• . . .
Rules of thumb:
• Use a special purpose wavelet if and only
if you have a special need.
• Use one vanishing moment per digit
desired (truncation level).
• Do not use more derivatives than your
function typically has.
Multiwavelets
(Polynomial version on [0,1])
Fix k ∈ N, and let Vn be the space of
functions that are polynomials of degree less
than k on the intervals (2nj,2n(j + 1)) for
j = 1, . . . ,2−n − 1, and 0 elsewhere.
V0 is spanned by k scaling functions.
W0 is spanned by k multiwavelets.
By construction, the wavelets have k
vanishing moments, and so the same sparsity
properties as ordinary wavelets.
The wavelets are not even continuous, but
this allows weak formulations of the
derivative, which allows better treatment of
boundaries.
Connections with Fourier Analysis
Fourier analysis gives an understanding of
frequency, but “non-stationary” signals beg
for space (time) localization.
This need motivates Computational Harmonic
Analysis and its tools, such as wavelets and
local cosine.
The theory and intuition are still based on
Fourier analysis.
Local Cosine Basis
Partition the line (interval, circle) with
· · · ai < ai+1 · · · , Ii = [ai, ai+1] .
Construct a set of bells: {bi(x)}with
∑
i b2i (x) = 1, bi(x)bi−1(x) even about ai,
and bi(x)bj(x) = 0 if j 6= i± 1.
Construct the cosines which are even on the
left and odd on the right
cji(x) =
√
√
√
√
2
ai+1 − aicos
(
(j + 1/2)π(x− ai)
ai+1 − ai
)
.
{bi(x)cji(x)} forms an orthonormal basis with
fast transform based on the FFT.
Phase Plane Intuition
If a function has ‘instantaneous frequency’
ν(x), it should be represented by those Local
Cosine basis elements whose rectangles
intersect ν(x). There are max{∆l,1} of these.
6
-
ξ
x|I| = l
1/l
∆
ν(x)
Local Cosine Phase Plane
Consider the eigenfunctions of −∆ − C/x,
which have instantaneous frequency
νn =√
C/x+ λn.
6
-x
ξ
��
��
��
νn
P
We can get an efficient representation (with
proof) by adapting, but this is not very
flexible, especially in higher dimensions.
Wavelet Phase Plane
The multiresolution analysis divides the phase
plane differently:
6
-
x
ξ
W1
W2
W3
W4
V4
For wavelets this is intuitive but not rigorous.
Wavelet Phase Plane
Consider again the instantaneous frequency
νn =√
C/x+ λn.
6
-
x
ξ
P
We get an efficient representation without
adapting, so the location of the discontinuity
is not important.
Tones
Sustained high frequencies, such as those in
spherical harmonics
are a problem for Wavelets, but not for local
cosine.
To enable wavelets to handle such functions,
“wavelet packets” were developed.
Wavelet Packets and Best Basis Searches
Idea: by filtering the wavelet spaces, we can
partition phase space in different ways:
V3 • ◦
V2 • • ◦ ◦
V1
• • • • ◦ ◦ ◦ ◦ W1
V0 • • • • • • • •
H�
����
AAAAUG H
��
���
AAAAUG H
��
���
AAAAUG H
��
���
AAAAUG
H�
����
AAAAUG H
��
���
AAAAUG
HAAAAU
��
���G
Any choice of decompositions gives a wavelet
packet representation.
A fast tree search can find the “best basis”.
Wavelet Packets Phase Plane
If we choose:
����
AAAU
����
AAAU
����
AAAU
����
AAAU
����
AAAU
����
AAAU
AAAU
����
then our phase
plane looks like:
-
6
x
ξ
Operators
There are many competing, adaptive ways to
represent functions.
It is more interesting to consider operators
and develop operator calculus.
=
d1
d2
d 3
s 3
d1
d^2
d^ 3
s 3
1Α
2Α
3Α
3Τ
21Γ3Γ 1
1Γ 42Γ 3
2Γ 43Γ 4
4Β 1Β 12 3Β 1
3Β 2 2Β 4
3Β 4
Many operators are sparse in Wavelet bases.
Operators in the Nonstandard Form
The nonstandard form gives a more isotropic,
and often more sparse, representation.
=
d1
s 1
d^2
s 2
d^ 3
s 3
1Α
1Γ
Β 1
2Β2Α
2Γ
3Β3Α
3Γ 3Τ
d1
s1
d2
s2
d 3
s 3
Operators in Wavelets
Hamiltonian
−∆ − 300
|x|
Density Matrix
with 15
eigenfunctions
Operators in Wavelets
Some operators can be computed rapidly.
0 5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Number of iterations
Sig
nific
ant c
oeffi
cien
t rat
io
straightforward disretizationfull wavelet representationadapted wavelet projection at 1e−8adapted wavelet projection at 1e−5
Here the density matrix is computed via the
sign iteration
T0 = T/||T ||2Tk+1 = (3Tk − T3
k )/2, k = 0,1, . . .
Philosophical Review
Multiscale assumption: Efficient when high
frequencies (sharp features) happen for a
short amount of time/space.
Cleanly adaptive: Refine or coarsen the
“grid” by adding or deleting basis
functions.
Automatically adaptive: Simply truncate
small coefficients.
Tunable: Decide on the needed precision and
properties first, then choose which
wavelet.