fourier methods of spectral estimationcsr/teaching/spectest/lecnotes/classical_psd.pdf · fourier...

Fourier Methods of Spectral Estimation

C.S.Ramalingam

Department of Electrical EngineeringIIT Madras

C.S.Ramalingam Fourier Methods of Spectral Estimation

Outline

Definition of Power Spectrum

Deterministic signal example

Power Spectrum of a Random Process

The Periodogram Estimator

The Averaged Periodogram

Blackman-Tukey Method

Use of Data Windowing in Spectral Analysis

Spectrogram: Speech Signal Example


What is Spectral Analysis?

Spectral analysis is the estimation of the frequency content ofa random process

By “frequency content” we mean the distribution of powerover frequency

Also called Power Spectral Density, or simply spectrum

What frequency components are present?What is the intensity of each component?


The Earliest Spectral Analyzer

All colours are present with equal intensity


What is Frequency?

Our notion of frequency comes from sin(2πf0t) and cos(2πf0t)

both are called “sinusoids”

Frequency ≡ Sinusoidal Frequency: f0 cycles/sec (Hz)

exp(j2πf0t) is the basis function needed for representing acomponent with frequency f0

for an arbitrary frequency component, it becomes exp(j2πft)

As f varies from −∞ to ∞, we get the Fourier basis set!

f is sometimes called “Fourier frequency”


Spectral Analysis ≡ Expansion Using Fourier Basis

Spectral analysis is nothing but expanding a signal x(t) usingthe Fourier basis

X (f ) =

∫ ∞−∞

x(t) exp(−j2πft) dt ← inner product!

“X (f ) is the continuous-time Fourier transform of x(t)”

|X (f1)| large ⇒ dominant frequency component at f = f1

|X (f1)| = 0⇒ no frequency component at f = f1

Plot of |X (f )|2 as a function of f is called the “powerspectrum”


Deterministic Signal Example

−2 0 2 4 6 8 10 12 14 16

−1

0

1

Time (ms)

Gated sinusoid with f0 = 1 kHz

−4000 −3000 −2000 −1000 0 1000 2000 3000 40000

20

40

60

Frequency (Hz)

Mag

nitu

de

−4000 −3000 −2000 −1000 0 1000 2000 3000 4000−20

0

20

40

Frequency (Hz)

Log

Mag

nitu

de


What About PSD of a Random Process?

“Spectral analysis is the estimation of the frequency contentof a random process”

An ensemble of sample waveforms constitute a randomprocess

X (f )?=

∫ ∞−∞

x(t) exp(−j2πft) dt

Does it exist?Even if it does, is it meaningful?

We’ll focus on discrete-time random processes, i.e., ensembleof x [n], where n ∈ Z


PSD of a WSS Random Process

Let x [n] be a complex wide-sense stationary process

Its autocorrelation sequence (ACS) is defined as

rxx [k] = E{x∗[n] x [n + k]}

Wiener-Khinchine Theorem:

Pxx(f ) =∞∑−∞

rxx [k] exp(−j2πfk) − 1

2≤ f ≤ 1

2

That is, ACSDTFT←→ PSD


An Alternative Definition for PSD

If the ACS decays sufficiently rapidly,

Pxx(f ) = limM→∞

E

1

2M + 1

∣∣∣∣∣M∑−M

x [n] exp(−j2πfn)

∣∣∣∣∣2

The so-called “Direct Method” is based on the above formula


Why is the Problem Difficult?

ACS is not available

Finite number of samples from one realization

We are only given x [0], x [1], . . . , x [N − 1]

No “best” spectral estimator exists

Many practical signals, such as speech, are non-stationary

Pxx(f ) obtained from given data is a random variable

Bias versus Variance trade-off



Recall

Pxx(f ) = limM→∞

E

1

2M + 1

∣∣∣∣∣M∑−M


∣∣∣∣∣2

In practice we drop

limM→∞

because data are finite

the expectation operator E since we have only one realization

The Periodogram estimator is defined as

P̂PER(f )def=

1

N

∣∣∣∣∣N−1∑n=0


∣∣∣∣∣2

“Direct Method”, since it deals with the data directly


Example: Two Sine Waves + Noise

x [n] =√

10 exp(j 2π 0.15n) +√

20 exp(j 2π 0.2n) + z [n]

z [n] ∼ complex N (0, 1), N = 20

−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5−30

−20

−10

0

10

20

30

40

50

Frequency

Mag

nitu

de (

dB)


Periodogram is a Biased Estimator For Finite Data

For finite N, periodogram is a biased estimator

Bias is the difference between the true and expected values

−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5−30

−20

−10

0

10

20

30

40

50

Frequency

Mag

nitu

de (

dB)

averagednoiseless

N = 20


Periodogram: Bias Decreases With Increasing N

If data length is increased, bias decreases:

limN→∞

E{

P̂xx(f )}

= Pxx(f )

−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5−30

−20

−10

0

10

20

30

40

50

60

Frequency

Mag

nitu

de (

dB)

N = 100


The More (Samples) the Merrier?

For most estimators, bias and variance decrease withincreasing N

An estimator is said to be consistent if

limN→∞

Pr(∣∣∣θ̂ − θ∣∣∣ > ε

)= 0

where θ̂ is the estimate of θ

This implies that, as N →∞,

bias → 0variance → 0


Is the Periodogram Consistent?

Consider white noise sequence for various N

True Pxx(f ) = constant

If the Periodogram estimator were consistent,P̂xx(f )→ constant as N increases

Consider noise sequences of length 32, 64, 128, and 256

N = 32; % white noise sequencex = randn(N,1); % of length 32

Does P̂xx(f ) tend to a constant as N increases?


White Noise Example

−0.5 0 0.5

0

40PSD of White Noise

−0.5 0 0.5

0

40

−0.5 0 0.5

0

40

−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5

0

40

Frequency

N=256

N=32

As N increases, variance of the estimate does not decrease

Periodogram is an inconsistent estimator


What Went Wrong?

“In practice we drop

limM→∞

because data are finite

the expectation operator E since we have only one realization”

For white noise, increasing the data length did not help

What can be done to capture the benefits of E{·} ?


Averaging: The Poor Man’s Expectation Operator

Expectation operator can be approximated by averaging

Averaged Periodogram:

P̂AVPER(f ) =1

M

M∑m=1

P̂(m)PER (f )

where P̂(m)PER (f ) is periodogram of m-th segment of length N

For independent data records

var{

P̂AVPER(f )}

=1

Mvar{

P̂PER(f )}


Averaged Periodogram of White Noise

Result of averaging 8 periodograms

−0.5 −0.25 0 0.25 0.5−10

−5

0

5

10

15

20

25

30

35

40

Frequency

Mag

nitu

de (

dB)

Averaged Periodogram


Variance Decreases, But Bias Increases!

Two Sines + Noise example

−0.5 −0.25 0 0.25 0.5−50

−40

−30

−20

−10

0

10

20Averaged Periodogram for Two Sines + Noise

Frequency

Mag

nitu

de (

dB)

N=256, M=1N=64, M=4N=16, M=16


Welch’s Method

Overlapping blocks by 50%

Reduces variance without worsening bias

−0.5 −0.25 0 0.25 0.5−25

−20

−15

−10

−5

0

5

10

15Welch’s Method

Frequency

Mag

nitu

de (

dB)

Block Length = 64No. of blocks = 7

Overlap = 50%


Why Did The Periodogram Fail?

Periodogram was defined as

P̂PER(f ) = 1N

∣∣∣∑N−1n=0 x [n] exp(−j2πfn)

∣∣∣2Equivalent to

P̂PER(f ) =N−1∑−(N−1)

r̂xx [k] exp(−j2πfk)

where

r̂xx [k] =

1

N

N−1−k∑n=0

x∗[n] x [n + k] k = 0, 1, . . . ,N − 1

r̂∗xx [−k] k = −(N − 1), . . . ,−1

Note that r̂xx [N − 1] = x∗[0]x [N − 1]/N

No averaging ⇒ estimate with high variance!



Recall

Pxx(f ) =∞∑−∞

rxx [k] exp(−j2πfk) − 1

2≤ f ≤ 1

2

In practice: (a) replace rxx [k] by estimate r̂xx [k], (b) truncatethe summation, and (c) apply “lag window”

P̂BTf ) =M∑−M

w [k] r̂xx [k] exp(−j2πfk)

where

0 ≤ w [k] ≤ w [0] = 1 w [k] = 0 for |k| > Mw [−k] = w [k] W (f ) ≥ 0

“Indirect Method”, since it does not deal with the datadirectly


Example: Two Sine Waves + Noise

Data length N = 100, Correlation Lag M = 10

−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5−30

−20

−10

0

10

20

30

40

50

60

Frequency

Mag

nitu

de (

dB)


Periodogram Vs. Blackman-Tukey

−0.5 −0.25 0 0.25 0.5

−20

0

20

40

60

Mag

nitu

de (

dB)

Periodogram

−0.5 −0.25 0 0.25 0.5

−20

0

20

40

60

Frequency

Mag

nitu

de (

dB)

Blackman−Tukey

Blackman-Tukey method: reduction in variance comes at theexpense of increased bias

Speech Analysis


Data Windowing in Spectral Analysis

Useful for data containing sinusoids + noise

Sidelobes of a stronger sinusoid may mask the main lobe of anearby weak sinusoid

We multiply x [n] by data window w [n] before computingperiodogram

Weaker sinusoid becomes more visible

Main lobe of each sinusoid broadens: two close peaks maymerge into one


Example: How Many Sine Waves Are There?

0.1 0.12 0.14 0.16 0.18 0.2 0.22−60

−40

−20

0

20

40

60

Frequency

Mag

nitu

de (

dB)

How Many Sinusoids Are There?


Example: Three Sine Waves

0.1 0.12 0.14 0.16 0.18 0.2 0.22−60

−40

−20

0

20

40

60

Frequency

Mag

nitu

de (

dB)

Three Sinusoids: Rectangular Window

0.150.15 0.157


Example: Three Sine Waves

0.1 0.12 0.14 0.16 0.18 0.2 0.22−60

−40

−20

0

20

40

60

Frequency

Mag

nitu

de (

dB)

Three Sinusoids: Hanning Window

0.150.15 0.157


Commonly Used Windows

Name w [k] Fourier transform

Rectangular 1 WR(f ) =sin πf (2M + 1)

sin πf

Bartlett 1− |k |M

1

M

(sin πfM

sin πf

)2

Hanning 0.5 + 0.5 cosπk

M0.25 WR

(f − 1

2M

)+ 0.5 WR(f ) +

0.25 WR

(f + 1

2M

)Hamming 0.54 + 0.46 cos

πk

M0.23 WR

(f − 1

2M

)+ 0.54 WR(f ) +

0.23 WR

(f + 1

2M

)w [k] = 0 for |k | > M


Hamming Vs. Hanning

−0.05 −0.025 0 0.025 0.05−60

−40

−20

0

Frequency

Mag

nitu

de (

dB)

−0.5 −0.25 0 0.25 0.5

−100

−50

0Fourier Transforms of Hamming and Hanning Windows

Frequency

Mag

nitu

de (

dB)

HammingHanning


Three Sine Waves

0.1 0.12 0.14 0.16 0.18 0.2 0.22−60

−40

−20

0

20

40

60

Frequency

Mag

nitu

de (

dB)

Rectangular Vs. Hamming Vs. Hanning

0.150.15 0.157


How Can We Analyze Non-Stationary Signals?

Consider a “linear chirp”, i.e., a signal whose frequencyincreases linearly from f1 Hz to f2 Hz over a time interval T

What is its magnitude spectrum?

0 0.01 0.02 0.03 0.04 0.05−1

0

1

Time

Am

plitu

de

0 200 400 600 800 10000

20

40

60

Frequency

Mag

nitd

ue (

dB)


Need a More Useful Representation

In Fourier analysis, even if a signal is non-stationary, it is stillrepresented using stationary sinusoids

An unsatisfactory approach

Power spectrum is identical to x(−t), whose frequencydecreases from f2 to f1

x(t) and x(−t) differ only in the phase of the Fouriertransform

What we really want to know is how frequency varies withtime

Can it still be called “frequency” ?


Spectrogram

Plot of power spectrum of short blocks of a signal as afunction of time

Over each short block, signal is considered to be stationary

Speech is a classic example of a commonly occurringnon-stationary signal

Voiced sounds: /a/, /e/, /i/, /o/, /u/ (quasi-periodic)

Unvoiced sounds: /s/, /sh/, /f/ (noise-like)

Plosives: /p/, /t/, /k/ (transient sounds)


Spectrogram of Linear Chirp

Time

Fre

quen

cy

0 0.1 0.2 0.3 0.40

100

200

300

400

500

600


Non-stationarity in Speech Signal

1.16 1.17 1.18 1.19 1.2 1.21 1.22 1.23 1.24−0.2

0

0.2

/k/

1.24 1.26 1.28 1.3 1.32 1.34 1.36 1.38−1

0

1

/ow/

1.75 1.8 1.85 1.9−0.05

0

0.05

Time

/s//s/


Application to Speech Analysis

0 0.25 0.5 0.75 1 1.25 1.5 1.75 2−1

−0.5

0

0.5

1Should We Chase Those Cowboys?

Am

plitu

de

Time

Fre

quen

cy

0 0.25 0.5 0.75 1 1.25 1.5 1.75 20

1000

2000

3000

4000

5000


Application to Speech Analysis

0 0.25 0.5 0.75 1 1.25 1.5 1.75 2−1

−0.5

0

0.5

1A

mpl

itude

Should We Chase Those Cowboys?

Time

Fre

quen

cy

0 0.25 0.5 0.75 1 1.25 1.5 1.750

1000

2000

3000

4000

5000


Summary

Definition of Power Spectrum

Deterministic signal example

Power Spectrum of a Random Process


The Averaged Periodogram

Bias versus Variance


Use of Data Windowing in Spectral Analysis

Spectrogram: Speech Signal Example


fourier methods of spectral estimationcsr/teaching/spectest/lecnotes/classical_psd.pdf · fourier...

Documents