1

7.10 MLE ExamplesWe’ll now apply the MLE theory to several examples of practical signal processing problems.

These are the same examples for which we derived the CRLB in Ch. 3

1. Range Estimation – sonar, radar, robotics, emitter location

2. Sinusoidal Parameter Estimation (Amp., Frequency, Phase)– sonar, radar, communication receivers (recall DSB Example), etc.

3. Bearing Estimation – sonar, radar, emitter location

4. Autoregressive Parameter Estimation– speech processing, econometrics

See Book

We Will

Cover

2

Ex. 1 Range Estimation ProblemTransmit Pulse: s(t) nonzero over t∈[0,Ts]

Receive Reflection: s(t – τo)

Measure Time Delay: τo

max,);(

0)()()( osts

o TTttwtstxo

τττ

+=≤≤+−= !"!#$

C-T Signal Model

tTs

s(t)

tT

s(t – τo)

BandlimitedWhite Gaussian

BPF& Amp

x(t)PSD of w(t)

f B–B

No/2

3

Range Estimation D-T Signal Model

Sample Every ∆ = 1/2B secw[n] = w(n∆)

DT White Gaussian Noise

Var σ2 = BNo

f

ACF of w(t)

τ1/2B

B–B1/B 3/2B

PSD of w(t)No/2 σ2 = BNo

1,,1,0][][][ −=+−= Nnnwnnsnx o …

s[n;no]… has M non-zero samples starting at no no ≈ τo /∆

−≤≤+

−+≤≤+−

−≤≤

=

1][

1][][

10][

][

NnMnnw

Mnnnnwnns

nnnw

nx

o

ooo

o

4

Range Estimation Likelihood FunctionWhite and Gaussian ⇒ Independent ⇒ Product of PDFs

3 different PDFs – one for each subinterval

2

3#

1

2

2

2#

1

2

2

1#

1

02

2

2

1

2][exp

2])[][(exp

2][exp);(

πσ

σσσ

=

−•

−−−•

−= ∏∏∏

−+

+=

−+

=

−

=

C

nxCnnsnxCnxCnpMn

Mnn

Mn

nn

on

no

o

o

o

o

o

!!!! &!!!! '(!!!!!!! &!!!!!!! '(!!! &!!! '(

x

Expand to get an x2[n] term… group it with the other x2[n] term

−+−−−•

−= ∑∑ −+

=

−

=

!!!!!!! "!!!!!!! #$!!! "!!! #$

12

022

1

0

2

])[][][2(2

1exp2

][exp);(

Mn

nno

N

nNo

o

o

nnsnnsnxnx

Cnpσσ

x

must minimize this or maximize its negative over values of noDoes not depend on no

5

Range Estimation ML Condition

!!! "!!! #$!!! "!!! #$∑∑

−+

=

−+

=−+−

12

1

0 ][][][Mn

nno

Mn

nn

o

o

o

o

nnsnnsnx

Doesn’t depend on no! …Summand moves with the limits as no changes.

Because s[n – no] = 0 outside summation

range… so can extend it!

2So maximize this:

∑−

=−

1

00 ][][

N

nnnsnxSo maximize this:

So…. MLE Implementation is based on Cross-correlation: “Correlate” Received signal x[n] with transmitted signal s[n]

,][][][][maxargˆ1

00∑−

=−≤≤−==

N

nxsxs

MNmo mnsnxmCmCn

6

Range Estimation MLE Viewpoint

mno

Cxs[m]

,][][][1

0∑−

=−=

N

nxs mnsnxmC

Doesn’t depend on no! …Summand moves with the limits as no changes.

Warning: When signals are complex (e.g., ELPS) take find peak of |Cxs[m] |

• Think of this as an inner product for each m• Compare data x[n] to all possible delays of signal s[n]

! pick no to make them most alike

7

Ex. 2 Sinusoid Parameter Estimation ProblemGiven DT signal samples of a sinusoid in noise….

Estimate its amplitude, frequency, and phase

1,,1,0][)cos(][ −=++Ω= NnnwnAnx o …φ

Ωo is DT frequency in cycles/sample: 0 < Ωo < π

DT White Gaussian NoiseZero Mean & Variance of σ2

Multiple parameters… so parameter vector: ToA ][ φΩ=θ

The likelihood function is:

),,(

))cos(][(2

1exp);(1

0

22

φ

φσ

o

N

no

N

AJ

nAnxCp

Ω=

+Ω−−=

∆

−

=∑θx

For MLE: Minimize This

8

Sinusoid Parameter Estimation ML ConditionTo make things easier…

Define an equivalent parameter set:

[α1 α2 Ωo ]T α1 = Acos(φ) α2 = –Asin(φ)

Then… J'(α1 ,α2,Ωo) = J(A,Ωo,φ) α = [α1 α2]T

Define:

c(Ωo) = [1 cos(Ωo) cos(Ωo2) … cos(Ωo(N-1))]T

s(Ωo) = [0 sin(Ωo) sin(Ωo2) … sin(Ωo(N-1))]T

and…

H(Ωo) = [c(Ωo) s(Ωo)] an Nx2 matrix

9

Then: J'(α1 ,α2,Ωo) = [x – H (Ωo) α]T [x – H (Ωo) α]

Looks like the linear model case… except for Ωo dependence of H (Ωo)

Thus, for any fixed Ωo value, the optimal α estimate is

[ ] xHHHα )()()(ˆ 1o

Too

T ΩΩΩ=−

Then plug that into J'(α1 ,α2,Ωo):

[ ] [ ]

[ ][ ]

[ ][ ]

[ ] !!!!!!! "!!!!!!! #$

!!!!!!!! "!!!!!!!! #$

o

oT

ooT

o

oT

ooT

oTT

oT

ooT

oT

ooTTT

oT

ooJ

Ω

−

ΩΩΩΩ−=

−

ΩΩΩΩ−=

ΩΩΩΩ−=

Ω−Ω−=

Ω−Ω−=Ω′

−

w.r.t.minimize

1

)()()()(

21

21

)()()()(

)()()()(

ˆ)()(ˆ

ˆ)(ˆ)(),ˆ,ˆ(

1

xHHHHxxx

xHHHHIx

αHxHαx

αHxαHx

HHHHI

αα

10

Sinusoid Parms. Exact MLE Procedure

[ ]

ΩΩΩΩ=Ω

−

≤Ω≤xHHHHx )()()()(minargˆ 1

0o

Too

To

To

o π

oΩStep 1: Minimize “this term” over Ωo to find

Step 2: Use result of Step 1 to get

[ ] xHHHα )ˆ()ˆ()ˆ(ˆ 1o

Too

T ΩΩΩ=−

Done Numerically

Step 3: Convert Step 2 result by solving

)ˆsin(ˆˆ

)ˆcos(ˆˆ

2

1

φα

φα

A

A

−=

=φ&ˆfor A

11

Sinusoid Parms. Approx. MLE ProcedureFirst we look at a specific structure:

[ ]

Ω

Ω

ΩΩΩΩ

ΩΩΩΩ

Ω

Ω=ΩΩΩΩ

−

−

xs

xc

sscs

sccc

xs

xcxHHHHx

)(

)(

)()()()(

)()()()(

)(

)()()()()(

1

1

oT

oT

ooT

ooT

ooT

ooTT

oT

oT

oT

ooT

oT

!!!!!! "!!!!!! #$

Then… if Ωo is not near 0 or π, then approximately1

20

02

−

≈N

N

and Step 1 becomes

2

0

21

00)(minarg)exp(][2minargˆ Ω=

Ω−=Ω≤Ω≤

−

=≤Ω≤∑ Xnjnx

N

N

noo

o ππ

and Steps 2 & 3 become DTFT of Data x[n]

)ˆ(ˆ

)ˆ(2ˆ

o

o

X

XN

A

Ω∠=

Ω=

φ

12

The processing is implemented as follows:

Given the data: x[n], n = 0, 1, 2, … , N-1

1. Compute the DFT X[m], m = 0, 1, 2, … , M-1 of the data• Zero-pad to length M = 4N to ensure dense grid of frequency points

• Use the FFT algorithm for computational efficiency

2. Find location of peak• Use quadratic interpolation of |X[m]|

3. Find height at peak• Use quadratic interpolation of |X[m]|

4. Find angle at peak• Use linear interpolation of ∠X[m]

oΩ

|X(Ω)|

Ω

∠X(Ω)

ΩoΩ

13

Figure 3.8 from textbook:

)2cos()( φπ += tfAts ot

Ex. 3 Bearing Estimation MLEEmits or reflects

signal s(t)

Simple model

Grab one “snapshot” of all M sensors at a single instant ts:

( ) ][~

cos][)(][ nwnAnwtsnx ssn ++Ω=+= φ

Same as Sinusoidal Estimation!! So Compute DFT and Find Location of Peak!!

If emitted signal is not a sinusoid then you get a different MLE!!