nonlinear mmse estimation and soft decisions stefano galli dr. stefano galli telcordia technologies,...

Nonlinear MMSE Estimation and Soft Decisions Nonlinear MMSE Estimation and Soft Decisions

Stefano GalliStefano Galli

Dr. Stefano GalliTelcordia Technologies, Inc.Room: MCC-1J124B445 South StreetMorristown, NJ 07960-6438Tel. : (973) 829-4980Fax : (973) 829-5886Email: [email protected] Copyright © 2003 Telcordia Technologies. All Rights Reserved

John Hopkins University, April 17, 2003.John Hopkins University, April 17, 2003.

Telcordia Technologies Proprietary - Copyright 2003.

Telcordia Technologies Applied ResearchCore Competencies

Network and Services Management– Services Planning and Provisioning– Broadband Network Management– Wireless Network Management– Global Services Management

Software Technology and Engineering– High Availability Software– Distributed Systems– Scalable Systems– Software Architecture and Testing


Internet Evolution– Internet Architecture/Performance/Economics– Quality of Service (QoS) in Converged Networks

Next-Generation Networks– Voice over Packet (VoP)/Voice over IP (VoIP)– Multi-Protocol Label Switching (MPLS)– Integrated Access Networking– Internet Appliances and Premises Interworking– Next-Generation Signaling and Control

Telcordia Technologies Applied ResearchCore Competencies, Cont’d


Wireless Networking– 3rd Generation (3G)– RF Technology– Digital Signal Processing for Wireless Comms– Wireless Architecture and Middleware– Wireless Applications/Support (WAP)

Optical Networking– Dense Wavelength Division Multiplexing (DWDM)– Routing in All-Optical Networks– Optical Network Engineering and Management– Quantum Computing and Quantum Cryptography



E-Business and Information Assurance– E-Commerce for Mobile Users (M-Commerce)– Data Mining and Information Extraction/Integration– Virtual Private Networks– Network Security

Plus Specialized Expertise In:– Speech Technology and Applications– Mathematical Sciences/Statistics (Algorithms, Network

Traffic Modeling, Cryptography, etc.)– Cable Interconnection Technology



Telcordia Technologies Applied ResearchBroadband Networking Group

David Waring, Kenneth Kerpez, Thomas Banwell, Stefano Galli

Various aspects of DSL– Standardization efforts;– Loop and crosstalk identification;– Crosstalk modeling;– Dynamic Spectrum Management.

Power Line Communications

Home Networking (wired/wireless)


Summary of presentation

1) Hard and Soft Decisions in Adaptive Equalization

2) Linear and Nonlinear MMSE Channel Estimation

3) The Proposed Approach to Soft Detection

4) Practical Applications

5) Conclusions


• Hard decisions (Hard-Statistics) are extensively used communications, e.g. in adaptive receivers for channel estimation and tracking, in the feedback section of a DFE, etc.

• Hard-Decisions don’t give us an index of the reliability of the decisions but are the best we can do, if the decisions are correct.

• If the decisions are wrong, the use of Hard-Decisions may cause severe performance degradation (e.g. channel tracking loss, catastrophic error propagation, etc.).

Hard-Statistics and Soft-Statistics


• Soft-Decisions (Soft-Statistics) are decisions on a transmitted symbol which also contain an index of the reliability of the decision.

• In general, Soft-Decisions contain all the information contained in Hard-Decisions, whereas the viceversa is not true.

Hard-Statistics and Soft-Statistics (cont.)

Recently, Soft-Decisions have been proven to be a useful tool in the following areas:

• Channel estimation and tracking;

• Combined adaptive equalization and decoding;

• Blind equalization based on second order statistics;

• Enhanced DFE with soft feedback section;

• Iterative decoding of parallel/serial concatenated coded streams.


Essentially three kinds of soft-information:Essentially three kinds of soft-information:

• Nonlinear function applied to an estimate of the symbol.

• A Posteriori Probabilities (APPs);

• Estimate (usually, MMSE) of transmitted symbols;

Is there any formal justification for their use?Is there any formal justification for their use?

Are they related in any way?Are they related in any way?

Hard-Statistics and Soft-Statistics (cont.)


The Considered Communications System

Time Division Multiple Access

Transmission of independent time slots with preamble or midamble

a(i)

a(i-D)

H (f)T

exp(j2 f t)

xx

n(t) exp(j2 f t) +

QAM Modulator

H (f) x

q(t)

y(i)R

AdaptiveReceiver

(t-iT )Si

0

0

exp(-j2 f t)0

Radio

Channel

r(t) exp(j2 f t)0

QAM Demodulator

The System Model


The System Model (cont.)

Model of the observations (channel as linear filter, convolution):

where:

(vector of the TS-sampled input delay-spread function g(t;));

(ISI channel state vector);

(complex zero-mean Gaussian noise sequence);

)()()()()(v)();()(1

0ivizivixiGikiskigiy T

L

k

C

)(

CC

LTLigigiG C )1;( ... )0;()(

C

C

L

STLisisix A )1( ... )()(

)()()( C isjvicviv


Adaptive Receivers Based on Hard-Statistics

Goal of an adaptive receiver

• Recover the transmitted data stream after the distortion caused by the channel, by continuously adapting to the time-varying channel.

Basic components of an adaptive receiver

• Adaptive channel estimator: estimates on the basis of a certain criterion (LMS, MMSE, etc.) the distortion introduced by the channel and provides this estimate to the symbol/sequence detector.

• Symbol/sequence detector (equalizer): using the estimation of the channel, decides on the basis of a certain criterion (MAP, ML, ZF, etc.) what symbols were transmitted and then feeds them to the channel estimator.


Adaptive Receivers Based on Hard-Statistics: major problems

• Decisions on symbols are best made (minimization of probability of error) if the detector “waits” and gathers more information from the received signal before deciding on the transmitted symbols. The amountof “waiting” is the decision delay.Example: Viterbi algorithm optimal if decision delay is infinite, but in practice a delay of five times the memory of the channel L is sufficient.

• Channel estimators process both the received signal and the decisions of the symbol/sequence detector to estimate the channel and output the estimate of the channel at the time the decisions were made.

If the decision delay is too large, channel estimate is old.

If decision delay is too small, decisions are not reliable.


MLS detection and decision driven channel estimator

Low-delay tentative hard-decisions (d(L-1)) for channel estimation.Final hard-decisions at large delay (D5(L-1)).Trade-off between decision delay and prediction order of the channel estimate.

r(i) a(i-D)^

Observation MLSE-VA

Channel estimator

Channelestimates

with delay d

Final decisionswith high delay D

Tentative decisionswith low delay d

a(i-d)

B

Trainingsymbols

A

S

Adaptive Receivers Based on Hard-Statistics: examples


Effects of predicted channel estimates on system performance(BPSK over Rayleigh channel with Land Mobile fading spectrum)



Receivers based on the Per Survivor Processing (PSP) principle

Kubo, Murakami, Fujino (1994) and Polydoros, Raheli (1995)

• There are S(L-1) distinct estimators for the S(L-1) states, respectively;• Higher tracking capability at the expense of a much larger implementation complexity.

MLSE detector

S distinct tentative decisions, zero delay

large-delay final decisions

channel estimator #1

channel estimator #2

channel estimator #S

...

...

.........

...S distinct

channel estimates

L-1

L-1

L-1

received �data y(i)



Channel ModelChannel Model:

ObservationObservation :

GoalGoal : , i.e. MMSEMMSE estimation of the channel impulse response

The Linear MMSE recursive estimate of the channel impulse response:

)()1()( idiGiG

)()()()( inixiGiy T

iyiGEiiG 1/)()/(ˆ

)i(vix)i(G)i()ki(s)k;i(g)i(y TL

k

C

)(

1

0v

)i/i(y)i(yc)i/i(Gy|)i(GE)i/i(G ii 1111

MMSE Channel Estimation: Linear or Nonlinear?


The one-step MMSE prediction of the observation

Common semplification: assume correct correct hard decisions.

,y|)i(x)i/i(GE

y|)i(yE)i/i(y

iT

i

1

1

11

11


1111 iT y),i(x|)i/i(GE)i/i(y

pdf of pdf of GG((ii) conditioned to ) conditioned to yy((ii) ) andand xx((ii) is Gaussian) is Gaussian

Linear MMSE estimation (Kalman) of Linear MMSE estimation (Kalman) of GG((ii) is optimal) is optimal


However, the problem is intrinsecally non-Gaussian:

pdf of pdf of xx((ii) or ) or GG((ii) conditioned to ) conditioned to yy((ii) is ) is notnot Gaussian Gaussian

Optimal MMSE channel estimator is nonlinearOptimal MMSE channel estimator is nonlinear


Goal: , nonlinear nonlinear MMSE estimation of the channel impulse response

iyiGEiiG 1/)()/(ˆ


In estimation theory, it is useful to define the following sequences:

1

1/)()1/(ˆ iyiGEiiG

1

1/)()1/(ˆ iyiyEiiy

In fact, the following sequences:

(estimate innovation sequence)

(observation innovation sequence)

have special properties depending on the nature of the estimator:

Linear estimation : uncorrelated sequences.

Nonlinear estimation : Martingale Difference (MD) sequences.

)1/(ˆ)/(ˆ)( iiGiiGi

)1/(ˆ)()( iiyiyi

Nonlinear MMSE Channel Estimation


M D R e p r e s e n t a t i o n T h e o r e m

T h e M D s e q u e n c e ( i ) c a n b e e x p r e s s e d a s a t r a n s f o r m a t i o n o f t h e

M D s e q u e n c e ( i ) a s i n t h e f o l l o w i n g :

)()/()( iiiCi

w h e r e t h e o b s e r v a t i o n d e p e n d e n t f i l t e r i n g g a i n C ( i / i ) i s a n a d a p t e d

s e q u e n c e a n d i s g i v e n b y :

ii yiiEyiiEiiC 11 /)(*)( //)(*)()/(

Nonlinear MMSE Channel Estimation (cont.)


Given a probability space (,F,P), the sequence {C} is said to be adaptedadapted to {B}, if C(i/i) is measurablemeasurable with respect to Bi for any i, where {B}={Bi , i=1, 2, ...} is an increasing sequence of -algebras of subsets in F.

Since Bi can be viewed as containing the past of all sequences of interest up to instant i,

the property of being an adapted sequence implies that the filtering gain C(i/i) is not recursively computablenot recursively computable.

ii yiiEyiiEiiC 11 /)(*)( //)(*)()/(



In order to have a recursive estimator, it would be necessary that the gain sequence C(i/i) be predictablepredictable not adapted or, equivalently, that C(i/i) is measurablemeasurable with respect to Bi-1 for any i. Therefore:

Unfortunately, the MD-Representation theorem has been proven to be falsefalse in the predictable form for the case of

1

11

1 /)(*)( //)(*)()1/()/( ii yiiEyiiEiiCiiC

discrete-time observations in white Gaussian noise!!!discrete-time observations in white Gaussian noise!!!



Non Linear MMSE estimation:Non Linear MMSE estimation:E. Baccarelli, R. Cusani, S. Galli, “A Novel Adaptive Receiver with Enhanced Tracking Capability for TDMA-Based Mobile Radio Communications”, IEEE JSAC-Special Issue on Wireless Communications (Part II), Vol. 16, No. 8, Dec. 1998.

• Formal approach via martingales is not trivial;

• The exact solution does not exist in the recursive and finite dimensional form;

• Structure of the estimator is fixed: Kalman-like;

• Computational complexity issues;

Is there another way?Is there another way?



Let us consider again the linear MMSE recursive estimate of the channel impulse response:

)i/i(y)i(yc)i/i(Gy|)i(GE)i/i(G ii 1111

11 11 iT y|)i(x)i/i(GE)i/i(y

1 11 iT y|)i(xE)i/i(G

Conjecture: this approximation is less harsh than assuming correct hard decisions.

Soft-Decision Based Channel Estimation


1111 iT y|)i(xE)i/i(G)i/i(y

NL-MMSENL-MMSE prediction of the states of the ISI channel prediction of the states of the ISI channeloror

NL-MMSE filtered and fixed-lag smoothed estimates NL-MMSE filtered and fixed-lag smoothed estimates of the transmitted symbolsof the transmitted symbols

11

11

11

11

|)1(

|)1(

|)(

|)(

i

i

i

i

yLisE

yisE

yisE

yixE

Soft-Decision Based Channel Estimation (cont.)


Goal: Non-linear MMSE estimation of transmitted symbol Goal: Non-linear MMSE estimation of transmitted symbol ss((ii))

iMMSENL yisEiis 1|)()/(ˆ

NL-MMSENL-MMSE estimation of the channel impulse response estimation of the channel impulse response

NL-MMSENL-MMSE estimate of the ISI channel state vector estimate of the ISI channel state vector

11

11

11

11

|)1(

|)1(

|)(

|)(

i

i

i

i

yLisE

yisE

yisE

yixE



NL- MMSE estimation of transmitted symbolsNL- MMSE estimation of transmitted symbols

Tarköy (ISIT ’95), Wang & Poor (IEEE Trans. Comm. ’99)

Only filtered estimates may be obtained.Therefore, it is optimal only for channels with no memory.

kkk

MMSENLiysisPs

APPsiis

1|)(

symbols message theof )/(ˆ f



NL- MMSE estimation of transmitted symbolsNL- MMSE estimation of transmitted symbols

(ISIT ’00; IEEE Trans. on Comm., Dec. 2002)

New approachNew approach:

where 0 D L-1

Both filtered and fixed-lag smoothed estimates may be obtained.

Therefore, it is optimal also for channels with memory.

vectorstate channel ISI theof )/(ˆ APPsDiis MMSENL f



)(

|)(

|)(

|)(

)/1(ˆ

)/1(ˆ

)/(ˆ

)/(ˆ

1

12

11

)()2()1(

)(2

)2(2

)1(2

)(1

)2(1

)1(1

i/i

yixP

yixP

yixP

iLis

iis

iis

iis

iN

i

i

NLLL

N

N

MMSENL

MMSENL

MMSENL

MMSENL

Ξ

Optimal NL-MMSE symbol estimation as a linear transformation of the APP vector of the channel state.



The channel estimator is fed by the more informative NL-MMSE estimates of the transmitted symbols and not by the usual hard-decided data.

The APPs are recursively computed and delivered to the channel estimatorwith no delay.

Practical applications: adaptive equalization


A MLS equalizer can operate efficiently at a high decision delay, e.g. at a decision delay equal to the length of the TDMA-slot.

The VA can build the trellis with more reliable zero-delayed channel estimates and in parallel with channel tracking.

Practical application: adaptive equalization (cont.)


Six equal-powered taps with land mobile fading spectrum

(BPSK modulation - Lp=12, Lf=60 - BDTS = 10-4)



Six equal-powered taps with land mobile fading spectrum

(BPSK modulation - Lp= 20, Lf= 100 - BDTS = 10-3)



HF link: CCIR Moderate conditions (= 1 ms;BD=0.5 Hz)

(QPSK modulation - Lp=15, Lf=50 - BDTS = 4.17·10-4)



Practical application: enhanced DFE

Digital Feedback Equalization (DFE)Digital Feedback Equalization (DFE)

Pros: optimum (MMSE sense), cheap, adaptive channel estimation.

Cons: risk of catastrophic error propagation.

MAP or MLSE are better than the MAP or MLSE are better than the idealideal DFE (at least 3 dB) DFE (at least 3 dB)

If channel impulse response is short:

MAP or MLSE are the preferred choice.

If channel impulse response is long:

DFE is the only practical approach unless reduced state techniques are employed for the MAP or MLSE solutions


Practical application: enhanced DFE (cont.)

The soft approachThe soft approach(Globecom 1999)(Globecom 1999)

• Reduced state MAP or MLSE receivers with feedback filter to shorten the long impulse response;

• Relax the overly optimistic assumption of error free decisions;

• Do not use hard decisions in the feedback section but the non-linearnon-linear MMSE MMSE estimates of the transmitted symbols;

• The non-linear MMSE estimates are computed on the basis of the A Posteriori ProbabilitiesA Posteriori Probabilities (APP, soft statistics) delivered at small delay by a MAP receiver;


Fig. 5

1.00E-06

1.00E-05

1.00E-04

1.00E-03

1.00E-02

1.00E-01

1.00E+00

7 8 9 10 11 12 13 14 15 16 17 18 19

SNR (dB)

BE

R

ZF-DFE

MAP/SDF[3/121]

MAP/HDF[3/121]

MAP/SDF[2/121]

MAP/HDF[2/121]

Practical application: enhanced DFE (cont.)


Practical application: multiuser detection

Successive Cancellation MUDSuccessive Cancellation MUD • Orders the users from the strongest to the weakest: P1>P2> … >PK;

• Detect the data sequence of the first (strongest) user;

• Subtract the decoded stream (hard decisions) from the observations;

• Detect the data sequence of the second user and so on;

KkdttriTtsib kkk ..., ,2 ,1 , )()(sgn)(ˆ

1

1

)(ˆ)()(k

llllk iTtsbPtrtr


Practical application: multiuser detection (cont.)

The soft approachThe soft approach(DARPA MIMO project, 2002)

• Do not use hard decisions in the subtraction of the decoded users but the non-linear MMSEnon-linear MMSE estimates of the transmitted symbols;

• Better performances are obtained if the fixed-lag smoothed estimates are used in place of the filtered ones;

w0 Detector

0ˆks

hk0

Hard-decisions

Conventional Hard SIC

w0APP

Computer

hk0

LinearTransformation

0

~ks

NL-MMSE Fixed-lag Smoothed Estimates of the Transmitted Symbol

Proposed Soft SIC


s(i)Unknown Channel

g(i) +

n(i)

y(i)Blind Equalizer s(i)^

Sufficient condition for channel identification: Probability distribution of must be equal to the probability distribution of {s(i)}.

This condition implies that the following cost function must be minimized:

, with p>2

Sato (‘75), Godard (‘80), Benveniste-Goursat (‘80), Picchi-Prati (‘80), Shalvi-Weistein (‘90)

Main disadvantages:•Slow convergence: several thousands of observation samples are needed to achieve channel identification;•The channel is estimated with a high residual MSE due to the use of nonconvex cost functions.•The channel estimate is not always available

)(ˆ is

pp ksksE )(ˆ)(

)(ˆ ig

Practical Applications: Blind Equalization


y(i)APPs

Computer

ChannelEstimator

G(i)^

(i/i)

Proposed blind equalizer

+

v(i)

z(i)Channel{g(k)}

MAPDetector

s(i-L+1)^

QAMModulator

s(i)

QAMDemodulator

S1S2

a(i-L+1)^

DifferentialEncoder

a(i)

DifferentialDecoder

The channel-estimator is fed with the soft information given by the APPs of the channel

Practical Applications: Blind Equalization (cont.)

IEEE Trans. on Signal Processing, July 2001


Channel: g=[1, 0, -1]

1.00E-05

1.00E-04

1.00E-03

1.00E-02

1.00E-01

1.00E+00

4 5 6 7 8 9 10 11 12 13 14 15 16

Eb/No (dB)

Bit

Err

or R

ate

No ISI (AWGN only)Known Channel with MAP detection (D=2)Blind MAP detection (D=2)Soft procedure

Practical Applications: Blind Equalization (cont.)


• As the transmitted bit-rate increases, ISI increases and powerfulAs the transmitted bit-rate increases, ISI increases and powerful

channel estimators become more necessary. channel estimators become more necessary.

• Hard-decision driven channel estimators are sub-optimal.Hard-decision driven channel estimators are sub-optimal.

• Optimal channel estimation implies the computation of the non-Optimal channel estimation implies the computation of the non-

linear filtered and fixed-lag smoothed estimates of the transmitted linear filtered and fixed-lag smoothed estimates of the transmitted

symbols. symbols.

• A new and simpler method for generating NL-MMSE filtered andA new and simpler method for generating NL-MMSE filtered and

fixed-lag smoothed estimates of the transmitted symbols via APPs fixed-lag smoothed estimates of the transmitted symbols via APPs

has been proposed. has been proposed.

Conclusions


Conclusions

• NL-MMSE filtered and fixed-lag smoothed estimates of theNL-MMSE filtered and fixed-lag smoothed estimates of the

transmitted symbols can be seen as transmitted symbols can be seen as optimaloptimal soft information, and soft information, and

their use is a consequence of the correct statement of the problem their use is a consequence of the correct statement of the problem

of MMSE estimation. of MMSE estimation.

•The proposed method makes SbS-MAP receivers very appealing.The proposed method makes SbS-MAP receivers very appealing.

• The proposed approach can be applied to all those problems thatThe proposed approach can be applied to all those problems that

admit a space-state representation. admit a space-state representation.

• Several fields of application, especially all those situations whereSeveral fields of application, especially all those situations where

hard decisions are employed despite their low reliability. hard decisions are employed despite their low reliability.

nonlinear mmse estimation and soft decisions stefano galli dr. stefano galli telcordia technologies,...

Documents

control telcordia technologies

contd slide

wireless networking

testing slide

hard decisions hardstatistics

harddecisions dont

soft decisions stefano

management quantum computing