EE 6422 – Adaptive Signal Processing

NANYANG TECHNOLOGICAL UNIVERSITY SINGAPORE

School of Electrical & Electronic Engineering

JANUARY 2009

Dr Saman S. Abeysekera School of Electrical Engineering

Room: S1-B1c-87 ; Ph: 6790 4515 Email: esabeysekera@ntu.edu.sg

ii

Lecturing Team: • Assoc. Prof Saman S. Abeysekera (SSA)

• Assoc. Prof Lin Zhi Ping (LZP) ----- Course Coordinator Acad Unit : 3.0 Pre-Requisite : Nil Text: 1. Adaptive Filter Theory, by Simon Haykin, Prentice

Hall, 2002. References: 1. Widrow B. and Stearns S., Adaptive Signal

Processing, Prentice Hall 1990. 2. Kalouptsidis, N. and Theodoridis S., Adaptive

System Identification and Signal Processing Algorithms, Prentice Hall 1996.

3. Triechler, Johnson and Larimore, Theory and Design

of Adaptive Filters, John-Wiley, 1995. 4. Johnson D. H. and Dudgeon D. E., Array Signal

Processing, Concepts and Techniques, Prentice-Hall, 1993.

iii

SCOPE 1. Fundamentals – 7 hrs SSA 2. Gradient Based Techniques - 7 hrs SSA 3. Least Squares Estimation - 4 hrs SSA 4. Recursive Least Squares - 6 hrs LZP 5. Lattice Filters - 4 hrs LZP 6. Nonlinear Adaptive Filtering - 6 hrs LZP 7. Applications - 5hrs LZP/SSA TOTAL - 39 hours

iv

TIME ALLOCATION - (SSA) Fundamentals – 6 hrs

1. Introduction to Adaptive Filtering

2. DSP Preliminaries

3. Stochastic Processes

4. Power Spectral Density

5. Eigen-analysis Linear Optimum Filtering - 3 hrs

6. Weiner Filter 7. Gradient Based Adaptation Linear Adaptive Filtering - 8 hrs

8. Least-Mean-Square (LMS) Algorithm 9. Other Adaptive Algorithms

11. Applications – Adaptive Line Enhancer Recursive Least Squares - 3 hrs

10. Method of Least Squares ASSIGNMENT --- 1

1

1. INTRODUCTION TO ADAPTIVE FILTERING

1.1 The Filter

- A filter is a device (hardware or software) that is applied to a set of noisy data in order to extract information about a known quantity of interest. (e.g. recovering a useful signal that has been corrupted by the transmission through a communication channel.)

- A filter can perform three basic tasks: -----

(i) Filtering – the information extracted at time t uses data measured up to and including time t.

- (ii) Smoothing – the data measured at later

time is also used in obtaining the information. There is an inherent delay in this case.

- (iii) Prediction – the data measured up to

and including time t is used to forecast the information at some later time.

- If the filtered, smoothed or the predicted

output is a linear function of the input then the filter is linear. Otherwise the filter is nonlinear. A Linear filter is completely described by its Impulse Response.

2

1.2 How do we Design the Filter ?

- We use a statistical approach. That is we assume that some statistical parameters of the wanted signal and noise are available. (i.e. mean and correlation function.)

- We then design the filter to minimize the

effect of the noise at the output, under some statistical criterion.

- Criterion: We evaluate the error between the

output of the filter and the desired response. (Desired response, for example can be the noise free signal.) For simplicity, (mathematical tractability) our filter design criterion can be selected to minimize the mean square error (error as defined above).

- When the inputs are stationary (i.e. the

statistics do not change with time) the resulting optimum filter is known as the Weiner Filter. That is the Weiner Filter is optimum in the mean square sense.

- When the input to the filter is nonstationary

(i.e. the signal and/or noise statistics are time varying) the resulting filter solution is known as the Kalman Filter. (This has variety of engineering applications, however, we do not discuss it here.)

3

1.3 Adaptive Filters

- The Weiner Filter needs a priori information about the statistics of the filter input. If this information is unavailable the filter design is no longer possible or the resulting filter is not optimum.

- In the absence of a priori information we

could use the data first to estimate the statistics and then use this statistical information to design the filter. This is a non-recursive technique that is not suitable for real time applications.

- A more elaborate approach is to let the filter self-design itself. This results in an Adaptive Filter that operates with the help of a recursive algorithm. The algorithm starts form a set of initial conditions. The set of conditions are recursively updated as more number of data samples are available to the input of the filter. When the input is stationary, the successive iterations of the algorithm converges to the optimum Weiner solution.

4

1.4 Types of Recursive Algorithms

- Because of the recursive nature of the filter, in reality an Adaptive filter is a nonlinear filter. However, in the nomenclature of Adaptive Filters, an Adaptive Filter is said to be linear if the recursive algorithm is computed adaptively as a linear combination of the filter inputs. Otherwise, it is classified as a nonlinear adaptive filter.

- A wide variety of algorithms are available

for the operation of linear adaptive filters. These algorithms are usually compared on the following properties.

• Rate of Convergence: The number of iterations

required to come “close enough” to the Weiner solution.

• Mis-adjustment: The deviation of the final mean square error from the mean square error produced by a Weiner filter.

• Tracking: If the input is non-stationary (but slowly varying) how well the algorithms can track the slow variations.

• Robustness: Sensitivity to small disturbances (should be small).

• Computations: Should be efficient. • Numerical Stability: Should not be prone to

quanitization errors. • Structure: Filter structure should be attractive for

hardware implementation (e.g. for VLSI).

5

1.5 Linear Filter Structures (i) FIR (finite-duration impulse response) Filter

• Also known as Transversal Filter or Tapped-

Delay Line Filter. • Inherently stable. Hence the popular choice in

adaptive filter structures.

• However, the filter length could be very long.

6

(ii) IIR(infinite-duration impulse response) Filter

• Excellent from hardware implementation point of view.

• However, the presence of feedback paths may

cause the filter unstable. Therefore, special precautions are necessary to avoid instability.

7

(iii) Lattice Filter

• Acts as an FIR filter. • Consists of a number of stages, each of which

appear as a lattice.

• Will be discussed later !!!

8

1.6 Adaptive Filter Applications

- Adaptive Filter is a powerful tool or device

that is used in many signal processing and control application, e.g. in communications, radar, sonar, biomedical engineering etc.

- Although these applications are quite

different, the adaptive filter has one common feature. The filter output and a desired response are used to compute an estimation error, which in turn used to control a set of adjustable filter coefficients.

- The essential difference between the various

adaptive filter applications arises in the manner in which the desired response is extracted. In this context, there are 4 basic classes of adaptive filtering applications.

9

1.7 Extraction of the Desired Response !

- (a) Identification Problem: Adaptive filter is used to identify a system (a plant) transfer function. The desired response is the plant output.

- (b) Inverse Modeling: The adaptive filter is

used to model the inverse of the plant transfer function. A delayed version of the plant input is the desired response. (The delay could also be zero, in certain applications.)

- (c) Prediction: The adaptive filter is used to

determine the characteristics of a random signal. For example the adptive filter is used to predict the present value of the random signal using the past values. (Either the predicted value or the prediction error could be useful for subsequent processing.)

- (d) Interference Canceling: The adaptive

filter is used to cancel unknown interference contained in the information bearing primary signal. A reference (auxiliary) signal is used as the input of the filter. The primary signal serves as the desired response. The reference signal is obtained by keeping sensors in such a way that the information bearing signal component is essentially weak in the reference signal.

10

11

1.8 Example 1: Adaptive Noise Canceller

- To improve voice communication in a noisy site such as the cockpit of a military aircraft.

12

13

- To suppress 50Hz interference in Electrocardiography (ECG).

14

15

- Cancelling Maternal ECG from Fetal ECG.

16

17

1.9 Example 2: Modelling of a Multipath Channel

- To decrease the communication bit errors due to multipath effects, the communication channel need to be modeled and equalized. This is usually achieved by transmitting a known PN sequence to the receiver. (The cross correlator measures the performance of the adaptive filter.)

18

19

1.10 Example 3: Adaptive Self Tuning Filter

- In many instances the signal of interest (narrow band) is corrupted by an unwanted wideband interference. In this situation, the adaptive self tuning filter can be used to suppress the interference without the use of a reference signal.

- A variation of the circuit can also be used to

detect a low level sine wave buried in noise. This circuit then serves as an “adaptive line enhancer (ALE)” which also can be considered as a spectrum analyzer (instead of a conventional DFT techniques.)

- ALE will be discussed later.

20