Fourier Methods of Spectral Estimation
C.S.Ramalingam
Department of Electrical EngineeringIIT Madras
C.S.Ramalingam Fourier Methods of Spectral Estimation
Outline
Definition of Power Spectrum
Deterministic signal example
Power Spectrum of a Random Process
The Periodogram Estimator
The Averaged Periodogram
Blackman-Tukey Method
Use of Data Windowing in Spectral Analysis
Spectrogram: Speech Signal Example
C.S.Ramalingam Fourier Methods of Spectral Estimation
What is Spectral Analysis?
Spectral analysis is the estimation of the frequency content ofa random process
By “frequency content” we mean the distribution of powerover frequency
Also called Power Spectral Density, or simply spectrum
What frequency components are present?What is the intensity of each component?
C.S.Ramalingam Fourier Methods of Spectral Estimation
The Earliest Spectral Analyzer
All colours are present with equal intensity
C.S.Ramalingam Fourier Methods of Spectral Estimation
What is Frequency?
Our notion of frequency comes from sin(2πf0t) and cos(2πf0t)
both are called “sinusoids”
Frequency ≡ Sinusoidal Frequency: f0 cycles/sec (Hz)
exp(j2πf0t) is the basis function needed for representing acomponent with frequency f0
for an arbitrary frequency component, it becomes exp(j2πft)
As f varies from −∞ to ∞, we get the Fourier basis set!
f is sometimes called “Fourier frequency”
C.S.Ramalingam Fourier Methods of Spectral Estimation
Spectral Analysis ≡ Expansion Using Fourier Basis
Spectral analysis is nothing but expanding a signal x(t) usingthe Fourier basis
X (f ) =
∫ ∞−∞
x(t) exp(−j2πft) dt ← inner product!
“X (f ) is the continuous-time Fourier transform of x(t)”
|X (f1)| large ⇒ dominant frequency component at f = f1
|X (f1)| = 0⇒ no frequency component at f = f1
Plot of |X (f )|2 as a function of f is called the “powerspectrum”
C.S.Ramalingam Fourier Methods of Spectral Estimation
Deterministic Signal Example
−2 0 2 4 6 8 10 12 14 16
−1
0
1
Time (ms)
Gated sinusoid with f0 = 1 kHz
−4000 −3000 −2000 −1000 0 1000 2000 3000 40000
20
40
60
Frequency (Hz)
Mag
nitu
de
−4000 −3000 −2000 −1000 0 1000 2000 3000 4000−20
0
20
40
Frequency (Hz)
Log
Mag
nitu
de
C.S.Ramalingam Fourier Methods of Spectral Estimation
What About PSD of a Random Process?
“Spectral analysis is the estimation of the frequency contentof a random process”
An ensemble of sample waveforms constitute a randomprocess
X (f )?=
∫ ∞−∞
x(t) exp(−j2πft) dt
Does it exist?Even if it does, is it meaningful?
We’ll focus on discrete-time random processes, i.e., ensembleof x [n], where n ∈ Z
C.S.Ramalingam Fourier Methods of Spectral Estimation
PSD of a WSS Random Process
Let x [n] be a complex wide-sense stationary process
Its autocorrelation sequence (ACS) is defined as
rxx [k] = E{x∗[n] x [n + k]}
Wiener-Khinchine Theorem:
Pxx(f ) =∞∑−∞
rxx [k] exp(−j2πfk) − 1
2≤ f ≤ 1
2
That is, ACSDTFT←→ PSD
C.S.Ramalingam Fourier Methods of Spectral Estimation
An Alternative Definition for PSD
If the ACS decays sufficiently rapidly,
Pxx(f ) = limM→∞
E
1
2M + 1
∣∣∣∣∣M∑−M
x [n] exp(−j2πfn)
∣∣∣∣∣2
The so-called “Direct Method” is based on the above formula
C.S.Ramalingam Fourier Methods of Spectral Estimation
Why is the Problem Difficult?
ACS is not available
Finite number of samples from one realization
We are only given x [0], x [1], . . . , x [N − 1]
No “best” spectral estimator exists
Many practical signals, such as speech, are non-stationary
Pxx(f ) obtained from given data is a random variable
Bias versus Variance trade-off
C.S.Ramalingam Fourier Methods of Spectral Estimation
The Periodogram Estimator
Recall
Pxx(f ) = limM→∞
E
1
2M + 1
∣∣∣∣∣M∑−M
x [n] exp(−j2πfn)
∣∣∣∣∣2
In practice we drop
limM→∞
because data are finite
the expectation operator E since we have only one realization
The Periodogram estimator is defined as
P̂PER(f )def=
1
N
∣∣∣∣∣N−1∑n=0
x [n] exp(−j2πfn)
∣∣∣∣∣2
“Direct Method”, since it deals with the data directly
C.S.Ramalingam Fourier Methods of Spectral Estimation
Example: Two Sine Waves + Noise
x [n] =√
10 exp(j 2π 0.15n) +√
20 exp(j 2π 0.2n) + z [n]
z [n] ∼ complex N (0, 1), N = 20
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5−30
−20
−10
0
10
20
30
40
50
Frequency
Mag
nitu
de (
dB)
C.S.Ramalingam Fourier Methods of Spectral Estimation
Periodogram is a Biased Estimator For Finite Data
For finite N, periodogram is a biased estimator
Bias is the difference between the true and expected values
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5−30
−20
−10
0
10
20
30
40
50
Frequency
Mag
nitu
de (
dB)
averagednoiseless
N = 20
C.S.Ramalingam Fourier Methods of Spectral Estimation
Periodogram: Bias Decreases With Increasing N
If data length is increased, bias decreases:
limN→∞
E{
P̂xx(f )}
= Pxx(f )
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5−30
−20
−10
0
10
20
30
40
50
60
Frequency
Mag
nitu
de (
dB)
N = 100
C.S.Ramalingam Fourier Methods of Spectral Estimation
The More (Samples) the Merrier?
For most estimators, bias and variance decrease withincreasing N
An estimator is said to be consistent if
limN→∞
Pr(∣∣∣θ̂ − θ∣∣∣ > ε
)= 0
where θ̂ is the estimate of θ
This implies that, as N →∞,
bias → 0variance → 0
C.S.Ramalingam Fourier Methods of Spectral Estimation
Is the Periodogram Consistent?
Consider white noise sequence for various N
True Pxx(f ) = constant
If the Periodogram estimator were consistent,P̂xx(f )→ constant as N increases
Consider noise sequences of length 32, 64, 128, and 256
N = 32; % white noise sequencex = randn(N,1); % of length 32
Does P̂xx(f ) tend to a constant as N increases?
C.S.Ramalingam Fourier Methods of Spectral Estimation
White Noise Example
−0.5 0 0.5
0
40PSD of White Noise
−0.5 0 0.5
0
40
−0.5 0 0.5
0
40
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5
0
40
Frequency
N=256
N=32
As N increases, variance of the estimate does not decrease
Periodogram is an inconsistent estimator
C.S.Ramalingam Fourier Methods of Spectral Estimation
What Went Wrong?
“In practice we drop
limM→∞
because data are finite
the expectation operator E since we have only one realization”
For white noise, increasing the data length did not help
What can be done to capture the benefits of E{·} ?
C.S.Ramalingam Fourier Methods of Spectral Estimation
Averaging: The Poor Man’s Expectation Operator
Expectation operator can be approximated by averaging
Averaged Periodogram:
P̂AVPER(f ) =1
M
M∑m=1
P̂(m)PER (f )
where P̂(m)PER (f ) is periodogram of m-th segment of length N
For independent data records
var{
P̂AVPER(f )}
=1
Mvar{
P̂PER(f )}
C.S.Ramalingam Fourier Methods of Spectral Estimation
Averaged Periodogram of White Noise
Result of averaging 8 periodograms
−0.5 −0.25 0 0.25 0.5−10
−5
0
5
10
15
20
25
30
35
40
Frequency
Mag
nitu
de (
dB)
Averaged Periodogram
C.S.Ramalingam Fourier Methods of Spectral Estimation
Variance Decreases, But Bias Increases!
Two Sines + Noise example
−0.5 −0.25 0 0.25 0.5−50
−40
−30
−20
−10
0
10
20Averaged Periodogram for Two Sines + Noise
Frequency
Mag
nitu
de (
dB)
N=256, M=1N=64, M=4N=16, M=16
C.S.Ramalingam Fourier Methods of Spectral Estimation
Welch’s Method
Overlapping blocks by 50%
Reduces variance without worsening bias
−0.5 −0.25 0 0.25 0.5−25
−20
−15
−10
−5
0
5
10
15Welch’s Method
Frequency
Mag
nitu
de (
dB)
Block Length = 64No. of blocks = 7
Overlap = 50%
C.S.Ramalingam Fourier Methods of Spectral Estimation
Why Did The Periodogram Fail?
Periodogram was defined as
P̂PER(f ) = 1N
∣∣∣∑N−1n=0 x [n] exp(−j2πfn)
∣∣∣2Equivalent to
P̂PER(f ) =N−1∑−(N−1)
r̂xx [k] exp(−j2πfk)
where
r̂xx [k] =
1
N
N−1−k∑n=0
x∗[n] x [n + k] k = 0, 1, . . . ,N − 1
r̂∗xx [−k] k = −(N − 1), . . . ,−1
Note that r̂xx [N − 1] = x∗[0]x [N − 1]/N
No averaging ⇒ estimate with high variance!
C.S.Ramalingam Fourier Methods of Spectral Estimation
Blackman-Tukey Method
Recall
Pxx(f ) =∞∑−∞
rxx [k] exp(−j2πfk) − 1
2≤ f ≤ 1
2
In practice: (a) replace rxx [k] by estimate r̂xx [k], (b) truncatethe summation, and (c) apply “lag window”
P̂BTf ) =M∑−M
w [k] r̂xx [k] exp(−j2πfk)
where
0 ≤ w [k] ≤ w [0] = 1 w [k] = 0 for |k| > Mw [−k] = w [k] W (f ) ≥ 0
“Indirect Method”, since it does not deal with the datadirectly
C.S.Ramalingam Fourier Methods of Spectral Estimation
Example: Two Sine Waves + Noise
Data length N = 100, Correlation Lag M = 10
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5−30
−20
−10
0
10
20
30
40
50
60
Frequency
Mag
nitu
de (
dB)
C.S.Ramalingam Fourier Methods of Spectral Estimation
Periodogram Vs. Blackman-Tukey
−0.5 −0.25 0 0.25 0.5
−20
0
20
40
60
Mag
nitu
de (
dB)
Periodogram
−0.5 −0.25 0 0.25 0.5
−20
0
20
40
60
Frequency
Mag
nitu
de (
dB)
Blackman−Tukey
Blackman-Tukey method: reduction in variance comes at theexpense of increased bias
Speech Analysis
C.S.Ramalingam Fourier Methods of Spectral Estimation
Data Windowing in Spectral Analysis
Useful for data containing sinusoids + noise
Sidelobes of a stronger sinusoid may mask the main lobe of anearby weak sinusoid
We multiply x [n] by data window w [n] before computingperiodogram
Weaker sinusoid becomes more visible
Main lobe of each sinusoid broadens: two close peaks maymerge into one
C.S.Ramalingam Fourier Methods of Spectral Estimation
Example: How Many Sine Waves Are There?
0.1 0.12 0.14 0.16 0.18 0.2 0.22−60
−40
−20
0
20
40
60
Frequency
Mag
nitu
de (
dB)
How Many Sinusoids Are There?
C.S.Ramalingam Fourier Methods of Spectral Estimation
Example: Three Sine Waves
0.1 0.12 0.14 0.16 0.18 0.2 0.22−60
−40
−20
0
20
40
60
Frequency
Mag
nitu
de (
dB)
Three Sinusoids: Rectangular Window
0.150.15 0.157
C.S.Ramalingam Fourier Methods of Spectral Estimation
Example: Three Sine Waves
0.1 0.12 0.14 0.16 0.18 0.2 0.22−60
−40
−20
0
20
40
60
Frequency
Mag
nitu
de (
dB)
Three Sinusoids: Hanning Window
0.150.15 0.157
C.S.Ramalingam Fourier Methods of Spectral Estimation
Commonly Used Windows
Name w [k] Fourier transform
Rectangular 1 WR(f ) =sin πf (2M + 1)
sin πf
Bartlett 1− |k |M
1
M
(sin πfM
sin πf
)2
Hanning 0.5 + 0.5 cosπk
M0.25 WR
(f − 1
2M
)+ 0.5 WR(f ) +
0.25 WR
(f + 1
2M
)Hamming 0.54 + 0.46 cos
πk
M0.23 WR
(f − 1
2M
)+ 0.54 WR(f ) +
0.23 WR
(f + 1
2M
)w [k] = 0 for |k | > M
C.S.Ramalingam Fourier Methods of Spectral Estimation
Hamming Vs. Hanning
−0.05 −0.025 0 0.025 0.05−60
−40
−20
0
Frequency
Mag
nitu
de (
dB)
−0.5 −0.25 0 0.25 0.5
−100
−50
0Fourier Transforms of Hamming and Hanning Windows
Frequency
Mag
nitu
de (
dB)
HammingHanning
C.S.Ramalingam Fourier Methods of Spectral Estimation
Three Sine Waves
0.1 0.12 0.14 0.16 0.18 0.2 0.22−60
−40
−20
0
20
40
60
Frequency
Mag
nitu
de (
dB)
Rectangular Vs. Hamming Vs. Hanning
0.150.15 0.157
C.S.Ramalingam Fourier Methods of Spectral Estimation
How Can We Analyze Non-Stationary Signals?
Consider a “linear chirp”, i.e., a signal whose frequencyincreases linearly from f1 Hz to f2 Hz over a time interval T
What is its magnitude spectrum?
0 0.01 0.02 0.03 0.04 0.05−1
0
1
Time
Am
plitu
de
0 200 400 600 800 10000
20
40
60
Frequency
Mag
nitd
ue (
dB)
C.S.Ramalingam Fourier Methods of Spectral Estimation
Need a More Useful Representation
In Fourier analysis, even if a signal is non-stationary, it is stillrepresented using stationary sinusoids
An unsatisfactory approach
Power spectrum is identical to x(−t), whose frequencydecreases from f2 to f1
x(t) and x(−t) differ only in the phase of the Fouriertransform
What we really want to know is how frequency varies withtime
Can it still be called “frequency” ?
C.S.Ramalingam Fourier Methods of Spectral Estimation
Spectrogram
Plot of power spectrum of short blocks of a signal as afunction of time
Over each short block, signal is considered to be stationary
Speech is a classic example of a commonly occurringnon-stationary signal
Voiced sounds: /a/, /e/, /i/, /o/, /u/ (quasi-periodic)
Unvoiced sounds: /s/, /sh/, /f/ (noise-like)
Plosives: /p/, /t/, /k/ (transient sounds)
C.S.Ramalingam Fourier Methods of Spectral Estimation
Spectrogram of Linear Chirp
Time
Fre
quen
cy
0 0.1 0.2 0.3 0.40
100
200
300
400
500
600
C.S.Ramalingam Fourier Methods of Spectral Estimation
Non-stationarity in Speech Signal
1.16 1.17 1.18 1.19 1.2 1.21 1.22 1.23 1.24−0.2
0
0.2
/k/
1.24 1.26 1.28 1.3 1.32 1.34 1.36 1.38−1
0
1
/ow/
1.75 1.8 1.85 1.9−0.05
0
0.05
Time
/s//s/
C.S.Ramalingam Fourier Methods of Spectral Estimation
Application to Speech Analysis
0 0.25 0.5 0.75 1 1.25 1.5 1.75 2−1
−0.5
0
0.5
1Should We Chase Those Cowboys?
Am
plitu
de
Time
Fre
quen
cy
0 0.25 0.5 0.75 1 1.25 1.5 1.75 20
1000
2000
3000
4000
5000
C.S.Ramalingam Fourier Methods of Spectral Estimation
Application to Speech Analysis
0 0.25 0.5 0.75 1 1.25 1.5 1.75 2−1
−0.5
0
0.5
1Should We Chase Those Cowboys?
Am
plitu
de
Time
Fre
quen
cy
0 0.25 0.5 0.75 1 1.25 1.5 1.75 20
1000
2000
3000
4000
5000
C.S.Ramalingam Fourier Methods of Spectral Estimation
Application to Speech Analysis
0 0.25 0.5 0.75 1 1.25 1.5 1.75 2−1
−0.5
0
0.5
1Should We Chase Those Cowboys?
Am
plitu
de
Time
Fre
quen
cy
0 0.25 0.5 0.75 1 1.25 1.5 1.75 20
1000
2000
3000
4000
5000
C.S.Ramalingam Fourier Methods of Spectral Estimation
Application to Speech Analysis
0 0.25 0.5 0.75 1 1.25 1.5 1.75 2−1
−0.5
0
0.5
1A
mpl
itude
Should We Chase Those Cowboys?
Time
Fre
quen
cy
0 0.25 0.5 0.75 1 1.25 1.5 1.750
1000
2000
3000
4000
5000
C.S.Ramalingam Fourier Methods of Spectral Estimation
Summary
Definition of Power Spectrum
Deterministic signal example
Power Spectrum of a Random Process
The Periodogram Estimator
The Averaged Periodogram
Bias versus Variance
Blackman-Tukey Method
Use of Data Windowing in Spectral Analysis
Spectrogram: Speech Signal Example
C.S.Ramalingam Fourier Methods of Spectral Estimation