une introduction au codage de canal - enseeiht

Rappels de theorie de l’information introduction au codage de canal

Une introduction au codage de canal

C. Poulliat

2 novembre 2011


Plan

1

2

Rappels de theorie de l’information Principales notions et definitions Capacit ´ emoiree d’un canal discret sans m´Capacit ´ ` ees et/ou sorties continues e d’un canal a entr ´

introduction au codage de canal Quelques definitions Crit ` ecodageeres de D´


Entropie, entropie conjointe

X une variable al ´ ete `eatoire discr ` a valeurs dans l’alphabet X de d.d.p. p(x) = Prob(X = x), x ∈ X

Entropie

H(X ) = −

x∈Xp(X = x) log2 (p(X = x))

= −E(log2 p(X )) (1)

Entropie conjointe

H(X ,Y ) = −

x∈X

y∈Yp(X = x ,Y = y) log2 (p(X = x ,Y = y))

= −E(log2 p(X ,Y )) (2)


Entropie binaire

X ∈ {0,1} avec p(X = 1) = p

H(X ) = −p log2 (p)− (1 − p) log2 (1 − p) � H2(p)


Information mutuelle

Information mutuelle

I(X ;Y ) = −x∈X y∈Y

p(X = x ,Y = y) log2 (p(X = x)p(Y = y)p(X = x ,Y = y)

)

= −E(log2 (p(X )p(Y )

p(X ,Y ))) ≥ 0 (4)

Proprietes

I(X ;Y ) = H(X )− H(X |Y )

= H(Y )− H(Y |X )

= H(X ) + H(Y )− H(X ,Y )

= I(Y ;X ) (5)

I(X ;X ) = H(X )


Interpretations


Plan

1

2




Capacite d’un canal discret sans memoire Definition

Codeur Canal Décodeur

S X Y S estimé

• X ∈ X , Y ∈ Y• Canal sans memoire caracterise par p(Y |X )

C = max p(X)

I(X ;Y ) (6)

= max p(X)

H(X )− H(X |Y ) = max p(X)

H(Y )− H(Y |X )

Max. atteint pour distribution uniforme pour les canaux symetriques.

Definition


Capacite d’un canal discret sans memoire Canal a effacement (BEC)

0

1

0

e

1

1−pe

pe

pe

1−pe

C = 1 − pe (7)

atteint pour une distribution d’entree uniforme


Canal binaire symetrique (BSC)

0

1

0

1

1−p

p

p

1−p

C = 1 − H2(p) (8)

atteint pour une distribution d’entree uniforme


Theoreme du codage de canal Canal discret sans memoire

Th´ eme du codage de canal (1/2) eor`

Soit un canal discret sans memoire de capacite C, on peut communiquer a tout debit de transmission inferieur a C. En particulier, ∀R < C, il existe un code C(N,R) tel que

C(N,R) : {0,1}NR −→ {0,1}N

telle que la probabilite d’erreur bloc en sortie de decodage optimal soit arbitrairement petite pour N suffisamment grand.


Plan

1

2



Rappels de th´ introduction au codage de canal eorie de l’information

Theoreme du codage de canal (2/2) Canal a temps discret et entrees/sorties continues

Th´ eme du codage de canal eor`

• Extension au cas d’entrees ou de sorties continues. • Les expressions precedentes mettent en jeu des densites de

probabilites. • Application principale : le cas du canal Gaussien.


Canal additif gaussien(AWGN)

B(ω)

↓X (ω) −→ E −→ Y (ω)

(9)

• X (ω) tel que σ2 x ≤ P

• B(ω) ∼ N (0, σ2 b)

C =1 2

log2 (1 + σ2 x/σ

2 b) bits/symbol (10)

=1 2

log2 (1 + 2RbEb/N0) (11)

max. atteint pour X (ω) ∼ N (0, σ2 x = P)


Canal additif gaussien a entrees binaires (BI-AWGN)

B(ω)

↓ X (ω) −→ E −→ Y (ω)

(12)

• X (ω) ∈ X = {0, 1}, avec p(X = 1) = 1/2 • B(ω) ∼ N (0, σ2

b = N0/2)• canal symetrique : p(y |x = +1) = p(−y |x = −1)


Canal additif gaussien a entrees M-aire (CI-AWGN)

20 Coding and Capacity

in AWGN in the problems.) Thus, we begin with CM -ary = h(Y )− h(Y |X) =h(Y )− h(Z), from which we may write

CM -ary = −∫ +∞

−∞p(y)log2(p(y))dy − log2(2πeσ

2), (1.20)

where h(Z) = log2(2πeσ2) because the noise is now two-dimensional (or complex)

with a variance of 2σ2 = N0, or a variance of σ2 = N0/2 in each dimension. In this

expression, p(y) is determined via

p(y) =1

M

M∑

m=1

p(y|sm),

where p(y|sm) is the complex Gaussian pdf with mean sm and variance 2σ2 = N0.The first term in (1.20) may be computed as in the binary case using (1.14). Notethat an M -ary symbol is transmitted during each symbol epoch and so CM -ary

has the units information bits per channel symbol. Because each M -ary symbolconveys log2(M) code bits, CM -ary may be converted to information bits per codebit by dividing it by log2(M).For QPSK, 8-PSK, 16-PSK, and 16-QAM,CM -ary is plotted in Figure 1.9 against

Eb/N0, where Eb is related to the average signal energy Es = E[|sm|2] via thechannel rate (channel capacity) as Eb = Es/CM -ary. Also included in Figure 1.9is the capacity of the unconstrained-input (two-dimensional) AWGN channel,CShannon = log2(1 + Es/N0), described in the previous subsection. Recall that theShannon capacity gives the upper limit over all signaling schemes and, hence, itscurve lies above all of the other curves. Observe, however, that, at the lower SNR

–2 0 2 4 6 8 10 12 14 160

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Cap

acity

(bi

ts/c

hann

el s

ymbo

l)

16–QAM

8–PSK

QPSK

16–PSK

Eb/N0 (dB)

log2(1 + Es/N0)

Figure 1.9 Capacity versus Eb/N0 curves for selected modulation schemes.

B(ω)

↓ X (ω) −→ E −→ Y (ω)

X (ω) ∈ X = {0, . . . ,M}, avec p(X = x) = 1/MB(ω) ∼ CN (0, σ2

b = N0)


Canaux selectif en frequence a entrees binaires(BI-ISI) 24 Coding and Capacity

–2 0 2 4 6 8 10 120

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Cap

acity

(in

form

atio

n bi

ts/c

ode

bit)

PR1, PR4Proakis' Channel C

BI-AWGN

Eb/N0 (dB)

Figure 1.11 Capacity plots for the BI-AWGN channel, the PR1 and PR4 channels, and Proakis’Channel C.

As examples of applications of this approach, Figure 1.11 compares the capac-ity of the BI-AWGN channel with Ciud for (1) partial-response channels of theform 1±Dν , ν = 1, 2, . . ., which includes the Class I partial response (PR1) poly-nomial 1 +D and the Class IV partial response (PR4) polynomial 1−D2; and(2) Proakis’ Channel C (see Figure 10.2-5(c) in [8]), which is represented by thepolynomial 0.227 + 0.46D + 0.688D2 + 0.46D3 + 0.227D4. Proakis’ Channel C ischaracterized by a spectral null near the middle of the band, so it is not surprisingthat its (Ciud) capacity is significantly less than that of the other channels. ThePR1 channel has a spectral null at DC and the PR4 channel has spectral nullsboth at DC and at the Nyquist frequency (half of the channel bit rate). We observethat these two partial-response channels have the same Ciud curve (which is alsothe curve for all 1±Dν PR channels), and that there is a rate loss relative to theBI-AWGN channel.

Problems

1.1 A single error has been added (modulo 2) to a transmitted (7,4) Hammingcodeword, resulting in the received word r = (1011 100). Using the decoding algo-rithm described in the chapter, find the error.

y [n] =L−1

k=0

h[k ]x [n − k ] + b[n] (13)

X ∈ X = {−1,+1}, avec p(X = x) = 1/2 B ∼ N (0, σ2

b = N0/2)


Plan

1

2




Codes en bloc lineaires Quelques definitions

On consid` efinis sur le corps binaire F2ere des codes d´ = GF (2).

Codes lineaires en blocs

Un code en blocs binaire C(N,K ) de longueur N est une application g(.) de l’ensemble FK

2 = {0, 1}K vers l’ensemble FN

2 = {0,1}N qui associe a tout bloc de donnees u un mot de code c.

g : FK2 → FN

2

u → c = g(u) (14)

# mots de code : 2K . Rendement : R = K/N (K symb. d’inf., N symb. codes). C(N,K ) est dit lineaire si g(.) est une application lineaire (les mots de codes sont un sous-espace vectoriel de FN

2 ).


Codes en bloc lineaires Matrice generatrice

Matrice g´ eratrice en´

On note c = [c0, . . . , cN−1] et u = [u0, . . . ,uK−1]

la matrice generatrice G de dimensions K × N est definie comme etant l’application linaire definie comme

c = uG

Espace du code : Im(C) = {c ∈ FN2 |c = uG, ∀u ∈ FK

2 } rang(G) = K (les lignes de G sont K mots de codes independants) et G non unique. G est dite systematique si ∀k ∈ [0,K − 1],∃n ∈ [0,N − 1] tel que c[n] = u[k ]. G peut alors se mettre sous la forme

G = [P|IK ]


Codes en bloc lineaires Matrice de parite

Matrice de parite

Le code C⊥(N − K ,K ), dit code dual, verifie que tout mot du code dual est orthogonal a tout mot du code C(N,K ). On note sa matrice generatrice H. On a alors {c ∈ C(N,K )|cHT = 0}Relation avec G : GHT = 0Pour un code systematique, H = [IN−K |PT]. Detection d’erreur a l’aide du syndrome : r = c + e

s = rHT = eHT

Si e est un mot de code, alors on parle d’erreurs non detectable.


Matrice de parite

Codes en bloc lineaires Distance minimum et spectre de distance du code

Distance de Hamming : dH(ci , cj) = w(ci ⊕ cj)

Distance minimale :

dmin = min {dH(ci , cj)|ci , cj ∈ C(N,K ); ci = cj} = min {w(c)|c ∈ C(N,K ), c = 0} (15)

Spectre de distance d’un code :

∀i = 1 . . .N,Ai = #c ∈ C(N,K ), w(c) = i

{A0,A1 . . .AN} est appele spectre de distance du code dmin est egale au plus petit nombre de colonnes dont la somme est le vecteur nul. dmin − 1 erreurs detectables, l(dmin − 1)/2J erreurs corrigibles sur BSC.


Codes en bloc lineaires Exemples

code de v´ eerification de parit ´code de rep´´ etition

Un code de repetition consiste en la repetition de N fois un bit d’information. On obtient un code C(N,1)de matrice generatrice

G1 = [1 . . .1 . . .1 � �� N

]

cN−1 = u0 ⊕ u1 ⊕ . . .⊕ uN−2 definissant un code C(N,N − 1)

G2 =

⎛ ⎜⎜⎜⎜⎜⎜⎝

1 . . .

IN−1 1 . . . 1

⎞ ⎟⎟⎟⎟⎟⎟⎠

Relation les deux codes sont duaux :

G1GT2 = 0


Plan

1 Rappels de theorie de l’information Principales notions et definitions Capacit ´ emoiree d’un canal discret sans m´Capacit ´ ` ees et/ou sorties continues e d’un canal a entr ´

2 introduction au codage de canal Quelques definitions Crit ` ecodageeres de D´


Criteres de decodage

Decodage par Maximum a Posteriori (MAP)

c = arg max cl

p(c'|y) (16)

= arg max cl

p(y|c')p(c')

p(y)(17)

Decodage par Maximum de Vraisemblance (ML)

c = arg max cl

p(y|c') (18)

Exemple de canaux

canal BSC : c = arg mincl dH(y,c')

canal BI-AWGN : c = arg mincl dE(y,c') = arg minclL

n (yn − c'n)

2

une introduction au codage de canal - enseeiht

Documents