optimal and adaptive signal processinghcso/ee5410_13.pdf · optimal signal processing means finding...
TRANSCRIPT
![Page 1: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/1.jpg)
Optimal and Adaptive Signal Processing Optimum signal processing is concerned with the design, analysis, and implementation of systems that extract information from sampled data in a manner that is ‘best’ or optimal in some sense. These systems are called optimal filters. Adaptive signal processing is concerned with the design, analysis, and implementation of systems whose structure changes in response to the incoming data.
H. C. So Page 1 Summer, 2009-2010
![Page 2: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/2.jpg)
Basic Classes of Optmal Filtering Applications
1.Prediction: use previous samples to predict current samples
Speech Modeling using Linear Predictive Coding (LPC)
Since speech signals are highly correlated, a speech signal ][ns can be accurately modeled by a linear combination of
its past samples:
H. C. So Page 2 Summer, 2009-2010
![Page 3: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/3.jpg)
][][̂][1
inswnsns iP
i−∑=≈
=
where }{ iw are known as the LPC coefficients. This corresponds a speech compression application of storing
instead of the raw data . Optimal signal processing can be used to determine }{ iw in an optimal way.
2. Identification
H. C. So Page 3 Summer, 2009-2010
![Page 4: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/4.jpg)
System Identification
Σ
ΣΣ
H(z)
Unknown system
Optimal filter
W(z)
+
++
+
+
-
s(k)
n (k)i
n (k)o
d(k)
r(k)
e(k)
x(k)
Given noisy input ][kx and/or noisy output ][kr , our aim is to determine the unknown system )(zH using )(zW
H. C. So Page 4 Summer, 2009-2010
![Page 5: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/5.jpg)
3. Inverse Filtering: find the inverse of the system
Signal Recovery Given a noisy discrete-time signal:
][][][][ kwkhkskx +⊗=
where ][ks , ][kh and ][kw represent signal of interest, unknown impulse response and noise. Optimal signal processing means finding ][ks in an optimal way.
H. C. So Page 5 Summer, 2009-2010
![Page 6: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/6.jpg)
4. Interference Canceling: Remove noise using an external reference
Interference Cancellation in Electrocardiogram (ECG) In biomedical engineering, the measured ECG signal is corrupted by the 50Hz power line interference:
][nr
][][][ ninsnr += where is noise-free ECG and represents 50Hz signal. External reference for is another 50Hz signal.
][ns ][ni][ni
H. C. So Page 6 Summer, 2009-2010
![Page 7: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/7.jpg)
Problem Statement for Optimal Filters
W(z)x(n) Σ
y(n)d(n)
EstimationError
+-
InputSignal
Output
e(n)
DesiredResponse
Given the input ][nx and the desired response , we want to find the transfer function
][nd)(zW or its impulse response
such that a statistical criterion or a cost function is optimized.
H. C. So Page 7 Summer, 2009-2010
![Page 8: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/8.jpg)
Some common optimization criteria in the literature are:
1.Least Squares : find )(zW that minimizes ∑−
=
1
0
2 ][N
nne where N
is the number of samples available. This corresponds to least-squares filter design.
2.Minimum Mean Square Error : find )(zW that minimizes
]}[{ 2 neE . This corresponds to Wiener filtering problem. Least Squares Filtering Consider that )(zW is a causal FIR filter of length L so that
∑=−
=
−1
0)(
L
i
ii zwzW (13.1)
H. C. So Page 8 Summer, 2009-2010
![Page 9: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/9.jpg)
The error function is thus given by ][ne
][][][ nyndne −= (13.2) where
][][][1
0nXWinxwny TL
ii =∑ −=
−
=,
T
TLL
LnxLnxnxnxnX
wwwwW
]]1[]2[]1[][[][
][ 1210
+−+−−=
= −−
L
L
The cost function is
21
0
1
0
1
0
2 ][][][)( ∑ ⎥⎦⎤
⎢⎣⎡
∑ −−=∑=−
=
−
=
−
=
N
n
L
ii
N
nLS inxwndneWJ (13.3)
which is a function of the filter coefficients }{ iw and N is the number of ][nx (and ). ][nd
H. C. So Page 9 Summer, 2009-2010
![Page 10: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/10.jpg)
The minimum of the least squares function can be found by differentiating )(WJ LS with respect to 110 ,,, −Lwww L and then setting the resultant expressions to zero as follows,
( )
∑ ∑ −−∑ =∑ −−=∑ −⇒
=−−∑ ⎟⎠⎞
⎜⎝⎛
∑ −−⇒
−==⎪⎭
⎪⎬⎫
⎪⎩
⎪⎨⎧
∑ ⎟⎠⎞
⎜⎝⎛
∑ −−∂
∂=
∂∂
−
=
−
=
−
=
−
=
−
=
−
=
−
=
−
=
−
=
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
21
0
][][][][][][
0][][][2
1,,1,0,0][][)(
L
i
N
ni
N
n
L
ii
N
n
N
n
L
ii
N
n
L
ii
jj
LS
jnxinxwjnxinxwjnxnd
jnxinxwnd
Ljinxwndww
WJL
(13.4)
H. C. So Page 10 Summer, 2009-2010
![Page 11: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/11.jpg)
Denote
Tdxdxdxdxdx LRLRRRR )]1(ˆ)2(ˆ)1(ˆ)0(ˆ[ˆ +−+−−= L (13.5)
where
1,,1,0,][][)(ˆ 1
0−=∑ −=−
−
=LjjnxndjR
N
ndx L
and
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
+−+−+−−+−+−
−+−+−−
=
)1,1(ˆ)1,1(ˆ)1,0(ˆ)2,0(ˆ
)1,0(ˆ)0,1(ˆ)0,2(ˆ)0,1(ˆ)0,0(ˆ
ˆ
LLRLRLRLR
RLRLRRR
R
xxxxxx
xx
xx
xxxxxxxx
xx
O
MOL
O
L
with
∑ −−=−−−
=
1
0][][),(ˆ N
nxx jnxinxjiR
H. C. So Page 11 Summer, 2009-2010
![Page 12: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/12.jpg)
When LN >> , it is convenient to use:
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
−−−
+−+−−
=
)0(ˆ)2(ˆ)1(ˆ)2(ˆ
)1(ˆ)1(ˆ)2(ˆ)1(ˆ)0(ˆ
ˆ
xxxxxx
xx
xx
xxxxxxxx
xx
RLRLRLR
RLRLRRR
RO
MOL
O
L
(13.6)
where
)(ˆ][][)(ˆ 1
0iRinxnxiR xx
iN
nxx −=+∑=
−−
=
As a result, we have
( ) dxxxLSLSxxdx RRWWRR ˆˆˆˆ 1⋅=⇒⋅=
− (13.7)
provided that xxR̂ is nonsingular. H. C. So Page 12 Summer, 2009-2010
![Page 13: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/13.jpg)
Example 13.1 A standard usage for least squares filtering is found in polynomial fitting. For example, line fitting corresponds to a first order polynomial and it is also called linear regression. Given ten measurement points which contain noises:
1 2 3 4 5 6 7 8 9 100
2
4
6
8
10
12
H. C. So Page 13 Summer, 2009-2010
![Page 14: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/14.jpg)
Suppose the desired form corresponds to a straight line, i.e., cmxy += . Hence we can write the “filter output” as:
cnmxny += ][][ where corresponds to the measurement values ][nd According to (13.3), the least squares cost function is:
( ) ( )210
1
210
1][][][),( cmnndcnmxndcmJ
nn−−∑=−−∑=
==
( )
∑∑ +∑=⇒
∑ =−−−=∂
∂
== =
=N
n
N
n
N
n
N
n
ncnmnnd
ncmnndm
cmJ
11 1
2
1
][
0)(][2),(
H. C. So Page 14 Summer, 2009-2010
![Page 15: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/15.jpg)
( )
∑∑ +∑=⇒
∑ =−−−=∂
∂
== =
=N
n
N
n
N
n
N
n
cnmnd
cmnndc
cmJ
11 1
1
1][
0)1(][2),(
⎥⎦
⎤⎢⎣
⎡=⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
∑
∑
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
∑
∑∑=⎥⎦
⎤⎢⎣
⎡⇒⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
∑
∑=⎥⎦
⎤⎢⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
∑∑
∑∑
=
=
−
=
==
=
=
==
==3723.10091.1
][
][
][
][
11
1
1
1
11
2
1
1
11
11
2
N
n
N
nN
n
N
n
N
nN
n
N
nN
n
N
n
N
n
N
n
nd
nnd
Nn
nn
cm
nd
nnd
cm
n
nn
http://www.ee.cityu.edu.hk/~hcso/ee5410/ex13_1.m
H. C. So Page 15 Summer, 2009-2010
![Page 16: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/16.jpg)
1 2 3 4 5 6 7 8 9 100
2
4
6
8
10
12
H. C. So Page 16 Summer, 2009-2010
![Page 17: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/17.jpg)
Example 13.2
Σ h zi=0
4
i-i
Σ w zi=0
4
i-i
Σ
x(n)
d(n)
y(n) e(n)
+
-
Unknown System
Σq(n)+
+
In this example least squares filtering is applied in determining the impulse response of an unknown system. Assume that the unknown impulse response is causal and { }={1,2,3,2,1}. Given ih N samples of ][nx and ][nd where
is a measurement noise. ]n[q
H. C. So Page 17 Summer, 2009-2010
![Page 18: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/18.jpg)
We can use MATLAB to simulate the least squares filter for impulse response estimation. The MATLAB source code is http://www.ee.cityu.edu.hk/~hcso/ee5410/ex13_2.m %define the number of samples N=50; %define the noise and signal powers noise_power = 0.0; signal_power = 5.0; %define the unknown system impulse response h=[1 2 3 2 1]; %generate the input signal which is a Gaussian white noise with power 5 x=sqrt(signal_power).*randn(1,N);
H. C. So Page 18 Summer, 2009-2010
![Page 19: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/19.jpg)
%generate R_xx corr_xx=xcorr(x); for i=0:4 for j=0:4 R_xx(i+1,j+1)= corr_xx(N+i-j); end end
%generate the desired output plus noise d=conv(x,h); d=d(1:N)+sqrt(noise_power).*randn(1,N); %generate R_dx corr_xd = xcorr(x,d); for i=0:4 R_dx(i+1) = corr_xd(N-i); end %compute the estimate impulse response
H. C. So Page 19 Summer, 2009-2010
![Page 20: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/20.jpg)
W_ls = inv(R_xx)*(R_dx)'
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
−−
−−
=
6413.2514044.244998.368182.115115.44044.246413.2514044.244998.368182.114998.364044.246413.2514044.244998.368182.114998.364044.246413.2514044.24
5115.48182.114998.364044.246413.251
ˆxxR
]9508.3340732.6163125.8738827.6588245.390[ˆ =dxR
TLSW ]8176.08501.18291.20151.21095.1[=
When N is increased to 500, we have
TLSW ]9828.09942.19927.29943.19975.0[=
When N is increased to 5000, we have
TLSW ]9984.09972.19976.29981.10000.1[=
For 500=N and the noise power is 0.5 (SNR=10 dB): H. C. So Page 20 Summer, 2009-2010
![Page 21: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/21.jpg)
TLSW ]9925.09773.19728.29826.10158.1[=
For 500=N and the noise power is 5.0 (SNR=0 dB):
TLSW ]0591.10249.29484.20138.20900.1[=
It is observed that
1. The estimation accuracy improves as N increases. It is reasonable because as N increases, the accuracy of xxR̂ and dxR̂ increases due to more samples are involved in their computation.
2. The estimation accuracy improves as the noise power decreases.
H. C. So Page 21 Summer, 2009-2010
![Page 22: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/22.jpg)
Example 13.3 Consider conversation using hand-free telephone between Users A and B. At the side of User A, the microphone will receive User A’s voice as well as the speaker output – User B’s voice. This corrupted signal can be modeled as:
][][][],[][][ ncndnininsny ⊗=+=
where is the desired speech (User A) and is due to speaker output (User B), which is equal to the convolution of speaker output ][nd and speaker-to-microphone channel impulse response . The reference signal (speaker output)
is highly correlated with such that:
][ns ][ni
][nc][nd ][ni
][][1
0lndcni l
L
l−∑=
−
=
Given and , the problem is to find . ][ny ][nd ][ns
H. C. So Page 22 Summer, 2009-2010
![Page 23: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/23.jpg)
load corrupted.dat; %this is y[n] plot(corrupted);
H. C. So Page 23 Summer, 2009-2010
![Page 24: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/24.jpg)
load ref.dat; %this is d[n] plot(ref);
H. C. So Page 24 Summer, 2009-2010
![Page 25: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/25.jpg)
Assume we know that 5=L so that
]4[]3[]2[]1[][
][][
43210
4
0
−+−+−+−+=
−∑==
ndcndcndcndcndc
lndcni ll
The error function is constructed as
][~][][4
0lndcnyke l
l−∑−=
=
where we hope ll cc →~ so that ][][ kske → The least squares cost function is then
21
0
4
0
1
0
2 ][~][][ ∑ ⎥⎦⎤
⎢⎣⎡
∑ −−=∑−
= =
−
=
N
n ll
N
nlndcnyne
H. C. So Page 25 Summer, 2009-2010
![Page 26: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/26.jpg)
Using (13.4)-(13.7), least squares solution is obtained from
( ) ydddLS RRW ˆˆ 1⋅=
−
where
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
=
)0(ˆ)1(ˆ)2(ˆ)3(ˆ)4(ˆ)1(ˆ)0(ˆ)1(ˆ)2(ˆ)3(ˆ)2(ˆ)1(ˆ)0(ˆ)1(ˆ)2(ˆ)3(ˆ)2(ˆ)1(ˆ)0(ˆ)1(ˆ)4(ˆ)3(ˆ)2(ˆ)1(ˆ)0(ˆ
ˆ
dddddddddd
dddddddddd
dddddddddd
dddddddddd
dddddddddd
dd
RRRRRRRRRRRRRRRRRRRRRRRRR
R
with
][][)(ˆ 1
0indndiR
iN
ndd +∑=
−−
=
and
H. C. So Page 26 Summer, 2009-2010
![Page 27: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/27.jpg)
Tydydydydydyd RRRRRR ])4(ˆ)3(ˆ)2(ˆ)1(ˆ)0(ˆ[ˆ −−−−=
with
4,3,2,1,0,][][)(ˆ 1
0=∑ −=−
−
=jjndnyjR
N
nyd
The MATLAB source code is http://www.ee.cityu.edu.hk/~hcso/ee5410/ex13_3.m It is found that
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
=
257.9152 207.4885 197.3658 166.4233144.6380 207.4885 257.9152 207.4885 197.3658 166.4233 197.3658 207.4885 257.9152 207.4885 197.3658 166.4233 197.3658 207.4885 257.9152 207.4885
144.6380 166.4233 197.3658 207.4885 257.9152
ˆ ddR
H. C. So Page 27 Summer, 2009-2010
![Page 28: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/28.jpg)
TydR ]867.0252 402.7274 714.0973 355.28504435.488[ˆ =
which gives
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
=
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
=
5.01393.9788-
3.01792.0122-
0.9588
ˆˆˆˆˆ
4
3
2
1
0
ccccc
W LS
The interference signal is estimated as
][ˆ][4
0lndcni l
l−∑=
=
while the source signal is estimated as
][][][ ninyns −=
H. C. So Page 28 Summer, 2009-2010
![Page 29: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/29.jpg)
The interference signal is plot(interference);
H. C. So Page 29 Summer, 2009-2010
![Page 30: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/30.jpg)
The source signal is plot(source);
H. C. So Page 30 Summer, 2009-2010
![Page 31: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/31.jpg)
Example 13.4 Suppose we want to find the position ),( yx of a target and we have range measurements between the target and M receivers with known positions ),( nn yx , Ni ,,2,1 L= . Without measurement noises, the range is:
( ) ( )22nnn yyxxr −+−=
H. C. So Page 31 Summer, 2009-2010
![Page 32: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/32.jpg)
Each range defines a circle where the target will lie and intersection of at 3 least circles provides the source location. In practice, The obtained range measurements are noisy:
Nnpyyxxr nnnn ,,2,1,)()( 22 L=+−+−=
Squaring both sides, we have:
222222 )()(2)()( nnnnnnn yyxxppyyxxr −+−++−+−= Now the noise component is
222 )()(2 nnnnn yyxxppq −+−+=
H. C. So Page 32 Summer, 2009-2010
![Page 33: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/33.jpg)
Expanding the equation and letting yields 22 yxR +=
222
22222
22222
222
22
,22
22
)()(
nnnnn
nnnnnn
nnnnnn
nnnn
ryxRyyxx
yxRqryxRyyxx
qyyyyxxxxr
qyyxxr
−+≈−+⇒
+=+−+=−+⇒
++−++−=⇒
+−+−=
Similar to Example 13.1, the inputs are now nx2 , and -1,
the desired output is , and the filter weights are
ny2222
nnnn ryxd −+=x, and y R. According to (13.3), the least squares cost
function is:
( )2
122),,( nnn
N
ndRyyxxRyxJ −−+∑=
=
H. C. So Page 33 Summer, 2009-2010
![Page 34: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/34.jpg)
( )
nnN
nn
N
nnn
N
nn
N
n
nnnnN
n
xdxRyxyxx
xdRyyxxx
RyxJ
∑=∑−∑+∑⇒
=−−+∑=∂
∂
====
=
111
2
1
1
22
0)2(22),,(
Similarly, we get:
nnN
nn
N
nn
N
nnn
N
nydyRyyyxx
yRyxJ
∑=∑−∑+∑⇒=∂
∂
==== 11
2
11220),,(
and
nN
nn
N
nn
N
ndRNyyxx
RRyxJ
∑=−∑+∑⇒=∂
∂
=== 111220),,(
H. C. So Page 34 Summer, 2009-2010
![Page 35: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/35.jpg)
In matrix form:
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
∑
∑
∑
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
−∑∑
∑−∑∑
∑−∑∑
=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡⇒
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
∑
∑
∑
=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
−∑∑
∑−∑∑
∑−∑∑
=
=
=
−
==
===
===
=
=
=
==
===
===
nN
n
nnN
n
nnN
n
nN
nn
N
n
nN
nn
N
nnn
N
n
nN
nnn
N
nn
N
n
nN
n
nnN
n
nnN
n
nN
nn
N
n
nN
nn
N
nnn
N
n
nN
nnn
N
nn
N
n
d
yd
xd
Nyx
yyyx
xyxx
Ryx
d
yd
xd
Ryx
Nyx
yyyx
xyxx
1
1
1
1
11
1
2
11
11
2
1
1
1
1
11
1
2
11
11
2
1
22
22
22
22
22
22
H. C. So Page 35 Summer, 2009-2010
![Page 36: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/36.jpg)
Suppose we have 4 receivers with positions (0,0), (0,10), (10,0) and (10,10) and the target is at (2,3). The noise-free ranges are:
( ) ( ) ( ) ( ) 3.6056==−+−=−+−= 130302 2221
211 yyxxr
( ) ( ) ( ) ( ) 7.2801==−+−=−+−= 5310302 2222
222 yyxxr
( ) ( ) ( ) ( ) 8.5440==−+−=−+−= 7303102 2223
233 yyxxr
( ) ( ) ( ) ( ) 10.6301==−+−=−+−= 113103102 2224
244 yyxxr
Under small range errors, , we get }6.10,6.8,3.7,6.3{}{ =nr
)(),,( 5318,13.4421.9983,3.0=Ryx .
Under large range errors, }11,9,7,4{}{ =nr , we get )(),,( 0750,14.2501.5750,3.1=Ryx .
http://www.ee.cityu.edu.hk/~hcso/ee5410/ex13_4.m H. C. So Page 36 Summer, 2009-2010
![Page 37: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/37.jpg)
Adaptive Signal Processing
Application areas are similar to those of optimal signal processing but now the environment is changing, the signals are nonstationary and/or the parameters to be estimated are time-varying. For example, Echo cancellation for Hand-Free Telephones (The speech echo is a nonstationary signal)
Equalization of Data Communication Channels (The channel impulse response is changing, particularly in mobile communications)
Time-Varying System Identification (the system transfer function to be estimated is non-stationary in some control applications)
H. C. So Page 37 Summer, 2009-2010
![Page 38: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/38.jpg)
Adaptive Filter Development
Year Application Developers
1959 Adaptive pattern recognition system Widrow et al
1960 Adaptive waveform recognition Jacowatz
1965 Adaptive equalizer for telephone channel
Lucky
1967 Adaptive antenna system Widrow et al
1970 Linear prediction for speech analysis Atal
Now numerous applications, structures, algorithms
H. C. So Page 38 Summer, 2009-2010
![Page 39: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/39.jpg)
Adaptive Filter Definition An adaptive filter is a time-variant filter whose coefficients are adjusted in a way to optimize a cost function or to satisfy some predetermined optimization criterion.
Characteristics:
Can automatically adapt (self-optimize) in changing environments and changing system requirements
Can be trained to perform specific filtering and decision-making tasks according to some updating equations
It can automatically operate in changing environments (e.g. signal detection in wireless channel)
nonstationary signal conditions (e.g. LPC of speech signal) time-varying parameter estimation (e.g. position tracking of moving source)
H. C. So Page 39 Summer, 2009-2010
![Page 40: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/40.jpg)
Block diagram of a typical adaptive filter is shown below:
AdaptiveFilter
AdaptiveAlgorithm
Σx(k)+-
{h(k)}
d(k)
e(k)
x(k) : input signal y(k) : filtered outputd(k) : desired responseh(k) : impulse response of adaptive filterThe cost function may be E{e (k)} or Σ e (k)2 2
k=0
N-1
y(k)
FIR or IIR adaptive filter filter can be realized in various structures adaptive algorithm depends on the optimization criterion
H. C. So Page 40 Summer, 2009-2010
![Page 41: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/41.jpg)
Basic Classes of Adaptive Filtering Applications 1. Prediction: signal encoding, linear prediction coding,
spectral analysis
H. C. So Page 41 Summer, 2009-2010
![Page 42: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/42.jpg)
2.Identification: adaptive control, layered earth modeling, vibration studies of mechanical system
H. C. So Page 42 Summer, 2009-2010
![Page 43: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/43.jpg)
3.Inverse Filtering: adaptive equalization for communication channel, deconvolution
H. C. So Page 43 Summer, 2009-2010
![Page 44: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/44.jpg)
4.Interference Canceling: adaptive noise canceling, echo cancellation
H. C. So Page 44 Summer, 2009-2010
![Page 45: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/45.jpg)
Widrow’s Least Mean Squares (LMS) Algorithm For simplicity, it is assumed that the adaptive filter is of causal FIR type and is implemented in direct form. Therefore, its system block diagram is
w (n)0
z-1
w (n)1
z-1
+
w (n)L-2
z-1
+
w (n)
+
L-1
...
...
x(n)
+d(n)
y(n)e(n) +
-
H. C. So Page 45 Summer, 2009-2010
![Page 46: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/46.jpg)
1. Optimization Criterion
To minimize the mean square error ]}[{ 2 neE where E denotes the expectation operator. The error signal
][ne at time n is:
][][][ nyndne −= where
)()(][][][1
0nXnWinxnwny TL
ii =∑ −=
−
=,
T
TLL
LnxLnx
W nwnwnwnwn
nxnxnX ]]1[]2[]1
]][][][][[)( 1210
+−+−
= −−
L
L
[][[)( −= H. C. So Page 46 Summer, 2009-2010
![Page 47: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/47.jpg)
2. Adaptation Procedure
ethod:
It is based on steepest descent m
)(nW ()()1 nWnW Δ+=+ where
⎟⎟⎠
⎞⎜⎜⎝
⎛
∂∂
μ−=Δ)(]}[{)(
2
nWneEnW (13.8)
and a step-size parameter which controls convergence ate and ensures stability
pectation operator is ignored:
μ is r In LMS algorithm, the ex
H. C. So Page 47 Summer, 2009-2010
![Page 48: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/48.jpg)
[ ])(][)(
)(nXnenW
AnW,)()(][][)(
)(][
][][
2)(
)(][
2)()1(
2
2
BBAnXnWndnenW
nWne
nenenW
nWnenWnW
TT=
⋅∂⋅−∂⋅μ−=
∂∂
⋅∂
∂μ−=
∂∂μ
−=+
r μ+=
∂∂
o 1,,1,0],[][][]1[ −=−μ+=+ Liinxnenwnw ii L (13.9) . Advantages
Low computational complexity Simple to implement Allow real-time operation
3
H. C. So Page 48 Summer, 2009-2010
![Page 49: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/49.jpg)
Example 13.5
This example considers an application of linear prediction. Suppose a signal ][nx is a second-order autoregressive (AR) process that satisfies the following difference equation:
][]2[81.0]1[558.1][ nvnxnxnx +−−−=
where white noise process such that
LMS filter to predict
][nv is a
⎩⎨⎧ =σ=+=
otherwise,00,]}[][{)(
2 mmnvnvEmR vvv
We want to use a two-coefficient
[][][][ˆ2
=∑ −= nwinxnwnx
)(nx by
]2[][]1[] 21 −+− nxnwnx 1=i
i
H. C. So Page 49 Summer, 2009-2010
![Page 50: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/50.jpg)
Upon convergence, we desire
558.1]}[{ 1and
→nwE
81.0]}[{ 2 −→nwE
z -1
w (n)1
z -1x(n)
w (n)
+d(n)
x(n)
e(n)+
-
2
++
+
H. C. So Page 50 Summer, 2009-2010
![Page 51: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/51.jpg)
The error function or prediction error
Thus the LMS algorithm for this problem is
][ne is given by
]2[][]1[][][
][][][][
21
2
1
−−−−=
∑ −−==
nxnwnxnwnx
inxnwndnei
i
]1[][][][
][][][
2][
][][
2][]1[
1
1
2
11
2
11
−μ+=∂∂
⋅∂
∂⋅
μ−=
∂∂
⋅μ
−=+
nxnenwnwne
nenenw
nwnenwnw
and
]2[][][][][
2][]1[ 2
2
2
22 −μ+=∂∂
⋅μ
−=+ nxnenwnwnenwnw
The computational requirement for each sampling interval is
Multiplications : 5 Addition/subtraction : 4
H. C. So Page 51 Summer, 2009-2010
![Page 52: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/52.jpg)
Two values of μ, 0.02 and 0.004, are investigated:
02.0=μ Convergence characteristics for LMS predictor with
H. C. So Page 52 Summer, 2009-2010
![Page 53: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/53.jpg)
004.0=μ Convergence characteristics for LMS predictor with
H. C. So Page 53 Summer, 2009-2010
![Page 54: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/54.jpg)
Observations: 1.When 02.0=μ , we had a fast convergence rate (the
parameters converged to the desired values in approximately 200 iterations) but large fluctuation existed in
2.When
][1 nw and ][n . 2w
004.0=μ , small fluctuation in ][1 nw and ][2 nw but the filter coefficients did not converge to the desired values of 1.558 and -0.81 respectively after the 300th iteration.
3.The learning behaviours of and agreed with
those of ][1 nw ][2 nw
]}[{ 1 nwE and ]}[{ 2 nwE . Notice that ]}[{ 1 nwE and ]}[{ 2 nwE can be derived using some advanced mathematics
H. C. So Page 54 Summer, 2009-2010
![Page 55: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/55.jpg)
We can use MATLAB to simulate this example
ww.ee.c
http://w ityu.edu.hk/~hcso/ee5410/ex13_5.m lear all Generate x[n]
] (2 ndn;
c%x(1)=randn; %v[n is white noise with power 1 x
xe
e % LMS alu = 0.001; %step-size=0.001
)=ra=10000;
for n=3:5000 (n)=1.558*x(n-1)-0.81*x(n-2)+randn; nd
N
for n=5001:N x(n)=1*x(n-1)-0.5*x(n-2)+randn; nd
gorithm mH. C. So Page 55 Summer, 2009-2010
![Page 56: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/56.jpg)
w1(1)=0; w1(2)=0; w1(3)=0; w2(1)=0; w2(2)=0;
for n=3:N n)*x(n-1)-w2(n)*x(n-2);
xlabel('number of iterations')
w2(3)=0;
e=x(n)-w1(w1(n+1)=w1(n)+mu*e*x(n-1); w2(n+1)=w2(n)+mu*e*x(n-2); end n=1:N; plot(n,w1(1:N),n,w2(1:N),'r');
nd('w1[n]','w2[n]') lege
H. C. So Page 56 Summer, 2009-2010
![Page 57: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/57.jpg)
1000 2000 3000 4000 5000 6000 7000 8000 9000 10000-1
-0.5
0
0.5
1
1.5 w1[n]
w2[n]
number of iterations
H. C. So Page 57 Summer, 2009-2010
![Page 58: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/58.jpg)
Example 13.6
This example considers an application of system identification. Given the input signal ][nx and output signal estimate the impulse response of the system or plant using the LMS algorithm.
Suppose the transfer function of the plant is
][nd , we can
∑=
−2
0i
ii zh which is
a causal FIR unknown system, then can be represented as
][nd
∑ −==
2
0][][
ii inxhnd
H. C. So Page 58 Summer, 2009-2010
![Page 59: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/59.jpg)
Assuming that the order the transfer function is unknown and we use a 2- coefficient LMS filter to model the system function as follows,
Σ h zi=0 i
-i2
Σ w z1
i=0 i-i
Σ
x(n) d(n)
e(n)
Plant
+
y(n) -
H. C. So Page 59 Summer, 2009-2010
![Page 60: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/60.jpg)
The error function is computed as
][][][][][][1∑ −−=−= inxnwndnyndne i
]1[][][][][ 10
0
−−−==
nxnwnxnwndi
Thus the LMS algorithm for this problem is
)()()(][
][][][
2][
][][
2][]1[
0
0
2
00
2
00
nxnenwnw
nenenenw
nwnenwnw
μ+=∂∂
⋅∂
∂⋅
μ−=
∂∂
⋅μ
−=+
and
]1[][][][][
2][]1[
1
1
2
11
−μ+=∂∂
⋅μ
−=+
nxnenwnwnenwnw
H. C. So Page 60 Summer, 2009-2010
![Page 61: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/61.jpg)
We can use MATLAB to simulate th
is example
http://www.ee.cityu.edu.hk/~hcso/ee5410/ex13_6.m % Generate signals N=5000; x=randn(1,N); %x[n] is white Gaussian h=[3,2,1]; %set system impulse response
d=d(1:length(x)); % LMS algorithm mu = 0.0025; %step-size=0.001 w0(1)=0; w0(2)=0; w0(3)=0; w1(1)=0; w1(2)=0; w1(3)=0;
n=3:N e=d(n)-w0(n)*x(n)-w1(n)*x(n-1); w0(n+1)=w0(n)+mu*e*x(n); w1(n+1)=w1(n)+mu*e*x(n-1); end
d=conv(h,x);
for
H. C. So Page 61 Summer, 2009-2010
![Page 62: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/62.jpg)
n=1:N;
terations')
plot(n,w0(1:N),n,w1(1:N),'r'); legend('w0[n]','w1[n]') xlabel('number of i
3.5
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000-0.5
0
0.5
1
1.5
2
2.5
3
w0[n]w1[n]
number of iterations
H. C. So Page 62 Summer, 2009-2010
![Page 63: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/63.jpg)
If we add:
e5410/ex13_6_2.mw2(n+1)=w2(n)+mu*e*x(n-2);
~hcso/ehttp://www.ee.cityu.edu.hk/
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000-0.5
0
0.5
1
1.5
2
2.5
3
w0[n]w1[n]w2[n]
number of iterations H. C. So Page 63 Summer, 2009-2010
![Page 64: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/64.jpg)
If we further add: W3(n+1)=w3(n)+mu*e*x(n-3); http://www.ee.cityu.edu.hk/~hcso/ee5410/ex13_6_3.m
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000-0.5
0
0.5
1
1.5
2
2.5
3
number of iterations
w0[n]w1[n]w2[n]w3[n]
H. C. So Page 64 Summer, 2009-2010
![Page 65: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/65.jpg)
Example 13.7
This example considers an application of interference cancellation. Given a received signal ][kr which consists of a source signal
][ks and a sinusoidal interference with task is to extract
known frequency. The ][ks from ][kr .
Notice that the amplitude and phase of the sinusoid are unknown. A well-known application is to remove 50/60 Hz power line interference in ECG recording.
H. C. So Page 65 Summer, 2009-2010
![Page 66: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/66.jpg)
Source Signal + Sinusoidal Interference
r(k)=s(k) + Acos(ω k + φ)0
Reference Signal
sin(ω k)0
90 Phase-Shift0cos(ω k)0
b0
b1
Σe(k)
+
-
-
The interference cancellation system consists of a phase-shifter and a two-weight adaptive filter. By properly adjusting the weights, the reference waveform can be changed in magnitude and phase in any way to model the
soid.
090
interfering sinuH. C. So Page 66 Summer, 2009-2010
![Page 67: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/67.jpg)
The filtered output is of the form
)cos(][)sin(][][][ 0100 kkbkkbkrke ω−ω−= The LMS algorithm is
)sin(][][][][2][2 00 kbkekb ∂∂∂
][][)(][][]1[
00
2
0
2
00
kkekw
kekekbkkbkb
ωμ+=
∂⋅
∂⋅
μ−=⋅−=+
nd
e∂μ
a
)cos(][][][][][]1[
2
11kekbkb ∂μ
−=+2
01
1
kkekbkb
ωμ+=∂
⋅
H. C. So Page 67 Summer, 2009-2010
![Page 68: Optimal and Adaptive Signal Processinghcso/ee5410_13.pdf · Optimal signal processing means finding s k[ ] ... is a causal FIR filter of length L so that = ... signal_power = 5.0;](https://reader031.vdocuments.net/reader031/viewer/2022030407/5a83b1997f8b9aee018efbda/html5/thumbnails/68.jpg)
H. C. So Page 68 Summer, 2009-2010
For sufficiently large number of iterations, we expect
)sin(][0 φ−→ Akb and
)cos(][1 φ→ Akb The filtered output is then approximated as
)cos(][)cos()cos()sin()sin(][][
0
00
kAkrkAkAkrke
φ+ω−=ωφ−ωφ+≈
][ks=