Computational Geophysics and Data Analysis 1Spectral analysis: foundations
Spectral analysis: Foundations
Orthogonal functions Fourier Series Discrete Fourier Series Fourier Transform: properties Chebyshev polynomials Convolution DFT and FFT
Scope: Understanding where the Fourier Transform comes from. Moving from the continuous to the discrete world. (Almost) everything we need to understand for filtering.
Computational Geophysics and Data Analysis 2Spectral analysis: foundations
Fourier Series: one way to derive them
The Problem
we are trying to approximate a function f(x) by another function gn(x) which consists of a sum over N orthogonal functions (x) weighted by some coefficients an.
)()()(
0
xaxgxfN
iiiN
Computational Geophysics and Data Analysis 3Spectral analysis: foundations
... and we are looking for optimal functions in a least squares (l2) sense ...
... a good choice for the basis functions (x) are orthogonal functions. What are orthogonal functions? Two functions f and g are said to be
orthogonal in the interval [a,b] if
b
a
dxxgxf 0)()(
How is this related to the more conceivable concept of orthogonal vectors? Let us look at the original definition of integrals:
The Problem
!Min)()()()(
2/12
2
b
a
NN dxxgxfxgxf
Computational Geophysics and Data Analysis 4Spectral analysis: foundations
Orthogonal Functions
... where x0=a and xN=b, and xi-xi-1=x ...If we interpret f(xi) and g(xi) as the ith components of an N component
vector, then this sum corresponds directly to a scalar product of vectors. The vanishing of the scalar product is the condition for orthogonality of
vectors (or functions).
N
iii
b
aN
xxgxfdxxgxf1
)()(lim)()(
figi
0 ii
iii gfgf
Computational Geophysics and Data Analysis 5Spectral analysis: foundations
Periodic functions
-15 -10 -5 0 5 10 15 200
10
20
30
40
Let us assume we have a piecewise continuous function of the form
)()2( xfxf
2)()2( xxfxf
... we want to approximate this function with a linear combination of 2 periodic functions:
)sin(),cos(),...,2sin(),2cos(),sin(),cos(,1 nxnxxxxx
N
kkkN kxbkxaaxgxf
10 )sin()cos(
2
1)()(
Computational Geophysics and Data Analysis 6Spectral analysis: foundations
Orthogonality
... are these functions orthogonal ?
0,00)sin()cos(
0
0,,0
)sin()sin(
0
02
0
)cos()cos(
kjdxkxjx
kj
kjkj
dxkxjx
kj
kj
kj
dxkxjx
... YES, and these relations are valid for any interval of length 2.Now we know that this is an orthogonal basis, but how can we obtain the
coefficients for the basis functions?
from minimising f(x)-g(x)
Computational Geophysics and Data Analysis 7Spectral analysis: foundations
Fourier coefficients
optimal functions g(x) are given if
0)()(!Min)()(22
xfxgorxfxg nan k
leading to
... with the definition of g(x) we get ...
dxxfkxbkxaaa
xfxga
N
kkk
kn
k
2
10
2)()sin()cos(
2
1)()(
2
Nkdxkxxfb
Nkdxkxxfa
kxbkxaaxg
k
k
N
kkkN
,...,2,1,)sin()(1
,...,1,0,)cos()(1
with)sin()cos(2
1)(
10
Computational Geophysics and Data Analysis 8Spectral analysis: foundations
Fourier approximation of |x|
... Example ...
.. and for n<4 g(x) looks like
leads to the Fourier Serie
...
5
)5cos(
3
)3cos(
1
)cos(4
2
1)(
222
xxxxg
xxxf ,)(
-20 -15 -10 -5 0 5 10 15 200
1
2
3
4
Computational Geophysics and Data Analysis 9Spectral analysis: foundations
Fourier approximation of x2
... another Example ...
20,)( 2 xxxf
.. and for N<11, g(x) looks like
leads to the Fourier Serie
N
kN kx
kkx
kxg
12
2
)sin(4
)cos(4
3
4)(
-10 -5 0 5 10 15-10
0
10
20
30
40
Computational Geophysics and Data Analysis 10Spectral analysis: foundations
Fourier - discrete functions
iN
xi2
.. the so-defined Fourier polynomial is the unique interpolating function to the function f(xj) with N=2m
it turns out that in this particular case the coefficients are given by
,...3,2,1,)sin()(2
,...2,1,0,)cos()(2
1
*
1
*
kkxxfN
b
kkxxfN
a
N
jjj
N
jjj
k
k
)cos(2
1)sin()cos(
2
1)( *
1
1
****0 kxakxbkxaaxg m
m
km kk
... what happens if we know our function f(x) only at the points
Computational Geophysics and Data Analysis 11Spectral analysis: foundations
)()(*iim xfxg
Fourier - collocation points
... with the important property that ...
... in our previous examples ...
-10 -5 0 5 100
0.5
1
1.5
2
2.5
3
3.5
f(x)=|x| => f(x) - blue ; g(x) - red; xi - ‘+’
Computational Geophysics and Data Analysis 12Spectral analysis: foundations
Fourier series - convergence
f(x)=x2 => f(x) - blue ; g(x) - red; xi - ‘+’
Computational Geophysics and Data Analysis 13Spectral analysis: foundations
Fourier series - convergence
f(x)=x2 => f(x) - blue ; g(x) - red; xi - ‘+’
Computational Geophysics and Data Analysis 14Spectral analysis: foundations
Gibb’s phenomenon
f(x)=x2 => f(x) - blue ; g(x) - red; xi - ‘+’
0 0.5 1 1.5-6
-4
-2
0
2
4
6
N = 32
0 0.5 1 1.5-6
-4
-2
0
2
4
6
N = 16
0 0.5 1 1.5-6
-4
-2
0
2
4
6
N = 64
0 0.5 1 1.5-6
-4
-2
0
2
4
6
N = 128
0 0.5 1 1.5-6
-4
-2
0
2
4
6
N = 256
The overshoot for equi-spaced Fourier
interpolations is 14% of the step height.
Computational Geophysics and Data Analysis 15Spectral analysis: foundations
Chebyshev polynomials
We have seen that Fourier series are excellent for interpolating (and differentiating) periodic functions defined on a regularly spaced grid. In many circumstances physical phenomena which are not periodic (in space) and occur in a limited area. This quest leads to the use of Chebyshev polynomials.
We depart by observing that cos(n) can be expressed by a polynomial in cos():
1cos8cos8)4cos(
cos3cos4)3cos(
1cos2)2cos(
24
3
2
... which leads us to the definition:
Computational Geophysics and Data Analysis 16Spectral analysis: foundations
Chebyshev polynomials - definition
NnxxxTTn nn ],1,1[),cos(),())(cos()cos(
... for the Chebyshev polynomials Tn(x). Note that because of x=cos() they are defined in the interval [-1,1] (which - however - can be extended to ). The first polynomials are
0
244
33
22
1
0
and]1,1[for1)(
where188)(
34)(
12)(
)(
1)(
NnxxT
xxxT
xxxT
xxT
xxT
xT
n
Computational Geophysics and Data Analysis 17Spectral analysis: foundations
Chebyshev polynomials - Graphical
The first ten polynomials look like [0, -1]
The n-th polynomial has extrema with values 1 or -1 at
0 0.2 0.4 0.6 0.8 1-1
-0.5
0
0.5
1
x
T_
n(x)
nkn
kx extk ,...,3,2,1,0),cos()(
Computational Geophysics and Data Analysis 18Spectral analysis: foundations
Chebyshev collocation points
These extrema are not equidistant (like the Fourier extrema)
nkn
kx extk ,...,3,2,1,0),cos()(
k
x(k)
Computational Geophysics and Data Analysis 19Spectral analysis: foundations
Chebyshev polynomials - orthogonality
... are the Chebyshev polynomials orthogonal?
02
1
1
,,
0
02/
0
1)()( Njk
jkfor
jkfor
jkfor
x
dxxTxT jk
Chebyshev polynomials are an orthogonal set of functions in the interval [-1,1] with respect to the weight functionsuch that
21/1 x
... this can be easily verified noting that
)cos()(),cos()(
sin,cos
jxTkxT
ddxx
jk
Computational Geophysics and Data Analysis 20Spectral analysis: foundations
Chebyshev polynomials - interpolation
... we are now faced with the same problem as with the Fourier series. We want to approximate a function f(x), this time not a
periodical function but a function which is defined between [-1,1]. We are looking for gn(x)
)()(2
1)()(
100 xTcxTcxgxf
n
kkkn
... and we are faced with the problem, how we can determine the coefficients ck. Again we obtain this by finding the extremum
(minimum)
01
)()(2
1
1
2
x
dxxfxg
c nk
Computational Geophysics and Data Analysis 21Spectral analysis: foundations
Chebyshev polynomials - interpolation
... to obtain ...
nkx
dxxTxfc kk ,...,2,1,0,
1)()(
2 1
12
... surprisingly these coefficients can be calculated with FFT techniques, noting that
nkdkfck ,...,2,1,0,cos)(cos2
0
... and the fact that f(cos) is a 2-periodic function ...
nkdkfck ,...,2,1,0,cos)(cos1
... which means that the coefficients ck are the Fourier coefficients ak of the periodic function F()=f(cos )!
Computational Geophysics and Data Analysis 22Spectral analysis: foundations
Chebyshev - discrete functions
iN
xi
cos
... leading to the polynomial ...
in this particular case the coefficients are given by
2/,...2,1,0,)cos()(cos2
1
* NkkfN
cN
jjjk
m
kkkm xTcTcxg
1
**0
* )(2
1)( 0
... what happens if we know our function f(x) only at the points
... with the property
N0,1,2,...,jj/N)cos(xat)()( j* xfxgm
Computational Geophysics and Data Analysis 23Spectral analysis: foundations
Chebyshev - collocation points - |x|
f(x)=|x| => f(x) - blue ; gn(x) - red; xi - ‘+’
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1 N = 8
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1 N = 16
8 points
16 points
Computational Geophysics and Data Analysis 24Spectral analysis: foundations
Chebyshev - collocation points - |x|
f(x)=|x| => f(x) - blue ; gn(x) - red; xi - ‘+’
32 points
128 points
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1 N = 32
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1 N = 128
Computational Geophysics and Data Analysis 25Spectral analysis: foundations
Chebyshev - collocation points - x2
f(x)=x2 => f(x) - blue ; gn(x) - red; xi - ‘+’
8 points
64 points
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
1.2 N = 8
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1
1.2 N = 64
The interpolating function gn(x) was shifted by a small amount to be visible at all!
Computational Geophysics and Data Analysis 26Spectral analysis: foundations
Chebyshev vs. Fourier - numerical
f(x)=x2 => f(x) - blue ; gN(x) - red; xi - ‘+’
This graph speaks for itself ! Gibb’s phenomenon with Chebyshev?
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1 N = 16
0 2 4 6-5
0
5
10
15
20
25
30
35
N = 16
Chebyshev Fourier
Computational Geophysics and Data Analysis 27Spectral analysis: foundations
Chebyshev vs. Fourier - Gibb’s
f(x)=sign(x-) => f(x) - blue ; gN(x) - red; xi - ‘+’
Gibb’s phenomenon with Chebyshev? YES!
Chebyshev Fourier
-1 -0.5 0 0.5 1-1.5
-1
-0.5
0
0.5
1
1.5 N = 16
0 2 4 6-1.5
-1
-0.5
0
0.5
1
1.5 N = 16
Computational Geophysics and Data Analysis 28Spectral analysis: foundations
Chebyshev vs. Fourier - Gibb’s
f(x)=sign(x-) => f(x) - blue ; gN(x) - red; xi - ‘+’
Chebyshev Fourier
-1 -0.5 0 0.5 1-1.5
-1
-0.5
0
0.5
1
1.5 N = 64
0 2 4 6 8-1.5
-1
-0.5
0
0.5
1
1.5 N = 64
Computational Geophysics and Data Analysis 29Spectral analysis: foundations
Fourier vs. Chebyshev
Fourier Chebyshev
iN
xi2
iN
xi
cos
periodic functions limited area [-1,1]
)sin(),cos( nxnx
cos
),cos()(
x
nxTn
)cos(2
1
)sin()cos(
2
1)(
*
1
1
**
**0
kxa
kxbkxa
axg
m
m
k
m
kk
m
kkkm xTcTcxg
1
**0
* )(2
1)( 0
collocation points
domain
basis functions
interpolating function
Computational Geophysics and Data Analysis 30Spectral analysis: foundations
Fourier vs. Chebyshev (cont’d)
Fourier Chebyshev
coefficients
some properties
N
jjj
N
jjj
kxxfN
b
kxxfN
a
k
k
1
*
1
*
)sin()(2
)cos()(2
N
jjj kf
Nck
1
* )cos()(cos2
• Gibb’s phenomenon for discontinuous functions
• Efficient calculation via FFT
• infinite domain through periodicity
• limited area calculations
• grid densification at boundaries
• coefficients via FFT
• excellent convergence at boundaries
• Gibb’s phenomenon
Computational Geophysics and Data Analysis 31Spectral analysis: foundations
The Fourier Transform Pair
deFtf
dtetfF
ti
ti
)()(
)(2
1)(
Forward transform
Inverse transform
Note the conventions concerning the sign of the exponents and the factor.
Computational Geophysics and Data Analysis 32Spectral analysis: foundations
The Fourier Transform Pair
)(
)(arctan)(arg)(
)()()()(
)()()()(
22
)(
R
IF
IRFA
eAiIRF i
)(
)(
A Amplitude spectrum
Phase spectrum
In most application it is the amplitude (or the power) spectrum that is of interest.
Computational Geophysics and Data Analysis 33Spectral analysis: foundations
The Fourier Transform: when does it work?
Gdttf )(
Conditions that the integral transforms work:
f(t) has a finite number of jumps and the limits exist from both sides
f(t) is integrable, i.e.
Properties of the Fourier transform for special functions:
Function f(t) Fouriertransform F()
even even
odd odd
real hermitian
imaginary antihermitian
hermitian real
Computational Geophysics and Data Analysis 34Spectral analysis: foundations
… graphically …
Computational Geophysics and Data Analysis 35Spectral analysis: foundations
Some properties of the Fourier Transform
Defining as the FT: )()( Ftf
Linearity
Symmetry
Time shifting
Time differentiation
)()()()( 2121 bFaFtbftaf
)(2)( Ftf
)()( Fettf ti
)()()( Fi
t
tf nn
n
Computational Geophysics and Data Analysis 36Spectral analysis: foundations
Differentiation theorem
Time differentiation )()()( Fi
t
tf nn
n
Computational Geophysics and Data Analysis 37Spectral analysis: foundations
Convolution
')'()'(')'()'()()( dttgttfdtttgtftgtf
The convolution operation is at the heart of linear systems.
Definition:
Properties:)()()()( tftgtgtf
)()()( tfttf
dttftHtf )()()(
H(t) is the Heaviside function:
Computational Geophysics and Data Analysis 38Spectral analysis: foundations
The convolution theorem
A convolution in the time domain corresponds to a multiplication in the frequency domain.
… and vice versa …
a convolution in the frequency domain corresponds to a multiplication in the time domain
)()()()( GFtgtf
)()()()( GFtgtf
The first relation is of tremendous practical implication!
Computational Geophysics and Data Analysis 39Spectral analysis: foundations
The convolution theorem
From Bracewell (Fourier transforms)
Computational Geophysics and Data Analysis 40Spectral analysis: foundations
Discrete Convolution
Convolution is the mathematical description of the change of waveform shape after passage through a filter (system).
There is a special mathematical symbol for convolution (*):
Here the impulse response function g is convolved with the input signal f. g is also named the „Green‘s function“
)()()( tftgty
nmk
fgym
iikik
,,2,1,00
migi ,....,2,1,0
njf j ,....,2,1,0
Computational Geophysics and Data Analysis 41Spectral analysis: foundations
Convolution Example(Matlab)
>> x
x =
0 0 1 0
>> y
y =
1 2 1
>> conv(x,y)
ans =
0 0 1 2 1 0
>> x
x =
0 0 1 0
>> y
y =
1 2 1
>> conv(x,y)
ans =
0 0 1 2 1 0
Impulse response
System input
System output
Computational Geophysics and Data Analysis 42Spectral analysis: foundations
Convolution Example (pictorial)
x y„Faltung“
0 1 0 0
1 2 1
0 1 0 0
1 2 1
0 1 0 0
1 2 1
0 1 0 0
1 2 1
0 1 0 0
1 2 1
0 1 0 0
1 2 1
0
0
1
2
1
0
y x*y
Computational Geophysics and Data Analysis 43Spectral analysis: foundations
The digital world
Computational Geophysics and Data Analysis 44Spectral analysis: foundations
The digital world
j
s jdtttgtg )()()(
gs is the digitized version of g and the sum is called the comb function. Defining the Nyquist frequency fNy as
dtfNy 2
1
after a few operations the spectrum can be written as
1
)2()2()(1
)(n
NyNys nffGnffGfGdt
fG
… with very important consequences …
Computational Geophysics and Data Analysis 45Spectral analysis: foundations
The sampling theorem
dtfNy 2
1
The implications are that for the calculation of the spectrum at frequency f there are also contributions of frequencies f±2nfNy, n=1,2,3,…
That means dt has to be chosen such that fN is the largest frequency contained in the signal.
Computational Geophysics and Data Analysis 46Spectral analysis: foundations
The Fast Fourier Transform FFT
... spectral analysis became interesting for computing with the introduction of the Fast Fourier Transform (FFT). What’s so fast about it ?
The FFT originates from a paper by Cooley and Tukey (1965, Math. Comp. vol 19 297-301) which revolutionised all fields where Fourier transforms where essential to progress.
The discrete Fourier Transform can be written as
1,...,1,0,1
1,...,1,0,
/21
0
/21
0
NkeFN
f
NkefF
NikjN
jjk
NikjN
jjk
Computational Geophysics and Data Analysis 47Spectral analysis: foundations
The Fast Fourier Transform FFT
... this can be written as matrix-vector products ...for example the inverse transform yields ...
1
2
1
0
1
2
1
0
)1(1
22642
132
2
1
1
1
11111
NNNN
N
N
f
f
f
f
F
F
F
F
.. where ...
Nie /2
Computational Geophysics and Data Analysis 48Spectral analysis: foundations
FFT
... the FAST bit is recognising that the full matrix - vector multiplicationcan be written as a few sparse matrix - vector multiplications
(for details see for example Bracewell, the Fourier Transform and its applications, MacGraw-Hill) with the effect that:
Number of multiplicationsNumber of multiplications
full matrix FFT
N2 2Nlog2N
this has enormous implications for large scale problems.Note: the factorisation becomes particularly simple and effective
when N is a highly composite number (power of 2).
Computational Geophysics and Data Analysis 49Spectral analysis: foundations
FFT
.. the right column can be regarded as the speedup of an algorithm when the FFT is used instead of the full system.
Number of multiplicationsNumber of multiplications
Problem full matrix FFT Ratio full/FFT
1D (nx=512) 2.6x105 9.2x103 28.4 1D (nx=2096) 94.98 1D (nx=8384) 312.6
Computational Geophysics and Data Analysis 50Spectral analysis: foundations
Summary
The Fourier Transform can be derived from the problem of approximating
an arbitrary function.
A regular set of points allows exact interpolation (or derivation) of arbitrary
functions
There are other basis functions (e.g., Chebyshev polynomials) with similar
properties
The discretization of signals has tremendous impact on the estimation of
spectra: aliasing effect
The FFT is at the heart of spectral analysis