Channelcoding

WatcharapanSuwansan5suk

#11EIE467DigitalCommunica5ons

KingMongkut’sUniversityofTechnologyThonburi

Chapter8:Detec5on,coding,anddecoding

Today’stopicsconcernwiththechannelencoderandchanneldecoder

Sampler

message(inputwaveform)

Modulator

Interpolator Demodulator

message(outputwaveform)

Channel

transmiNer

receiver

binaryinterface

analogsequence

symbolsequence

Sourceencoder

Sourcedecoder

Quan5zer

Tablelookup

2

Channeldecoder

Channelencoder

Topics•  Blockcoding(§8.5)•  Thenoisy-channelcodingtheorem(§8.6)

3

AZerthislecture,youwillbeableto•  State the difference between a soZ decision and a harddecision

•  Listallcodewordsofablockcodeforasmallgeneratormatrix•  Decodeanyblockcodeusinganop5malbutnaiveapproach,theminimum-distancerule

•  Understandanopera5onalmeaningofthechannelcapacity•  ComputethecapacityoftheAWGNchannel•  Giveanexampleofchannelcodesthatachievethecapacity

4

BACKGROUNDONCHANNELCODING

5

Channelencoderaddsredundancytocombatnoise

•  Thisexampleshowsarepe55oncode,whereeachinputbitisrepeated35mes

Channelencoder

010... 000111000...

6

Channeldecodercorrectsanyerrorandremoveredundancy

• Weassumethroughoutthislecturethatthechanneldecoderusestheminimum-distancerule

•  For the repe55on code, theminimum-distance rule outputsthemajoritybit(amongagroupof3bits)

Channeldecoder

010... 000111000...

Channeldecoder

010... 010111000...Anotherexample:

Anerroroccursattheinput

7

Inputtothechanneldecodercanbebitsorrealnumbers

•  Forsimplicity,assumePAMthroughoutthislecture

8

Channeldecoder

010... 011..

bits

Iftheinputisasequenceofbits(0’s,1’s),wesaythatthedemodulatorhasperformedaharddecision

Iftheinputisasequenceofrealnumbers,thedemodulatorhasperformaso,decision

Channeldecoder

100... −0.31.83.1...

realnumbers

Toimplementaharddecision,usethesamestructureofthedemodulatorthatwecovered

•  Thesignaldecoderwhichturnsthesequenceofrealnumbersintoasequenceofbits

9

bits

signalencoder{0, 1} ! ±a

U0, U1, U2, . . .

Gaussiannoise

Z0, Z1, Z2, . . .V0, V1, V2, . . .

channelencoder

Cm =

(1, if Vm � 0

0, if Vm < 0

signaldecoder

C0, C1, C2, . . .

bits

B0, B1, B2, . . .

binaryinterface

channeldecoder

±a

realnumber

Toimplementaso,decision,removethesignalencoder

10

bitssignal

encoder{0, 1} ! ±a

U0, U1, U2, . . .

Gaussiannoise

Z0, Z1, Z2, . . .V0, V1, V2, . . .

channelencoder

B0, B1, B2, . . .

binaryinterface

realnumbers

channeldecoder

±a

realnumber

CASESTUDY:REPETITIONCODETHATREPEATSEACHBIT3TIMES

11

Considerachannelencoderthatrepeatseachinputbit35mes

•  Thischannelcodeiscalledarepe55oncode•  Eachoutput“000”or“111”iscalledacodeword

12

Channelencoder(repe55oncode)

0 000

Channelencoder(repe55oncode)

1 111

acodeword

acodeword

Example1:Channeldecoder(harddecision)

•  Thedemodulatorperformsaharddecision•  Thechanneldecoderusestheminimum-distancerule

•  Ques5on:whatistheoutputfromthechanneldecoder?

•  Answer:0

13

Channeldecoder(repe55oncode)

? 001

(Solu5on)twohypotheses:000and111

•  The channel decoder decideswhether the input is closer indistanceto[000]orto[111]

14

channelencoder

channeldecoder

000 signalencoder

�a,�a,�a0

signaldecoder

000

noiselesschannel

channeldecoder

channelencoder

1111

signaldecoder

111

noiselesschannel

signalencoder

a, a, a

Solu5on:thedistancebetweeneachhypothesisand[001]

15

inputtochannelencoder

hypothesis:outputfromchannelencoder(codeword)

distance

0

1

c

= 1MinimumdistanceSodecode[001]tobit0

|c� v|

���⇥0 0 0

⇤�⇥0 0 1

⇤���

=p

(0� 0)2 + (0� 0)2 + (0� 1)2

���⇥1 1 1

⇤�⇥0 0 1

⇤���

=p

(1� 0)2 + (1� 0)2 + (1� 1)2

=p2

⇥1 1 1

⇤

111orasavector

⇥0 0 0

⇤

000orasavector

v =⇥0 0 1

⇤

Example2:Channeldecoder(soZdecision)

•  Themodulatorusestheamplifica5onfactor•  ThedemodulatorperformasoNdecision•  Thechanneldecoderusestheminimum-distancerule.

•  Ques5on:whatistheoutputfromthechanneldecoder?

•  Answer:1

16

Channeldecoder? 5.17.7−4.5

a = 4

(Sol.)2hypotheses:[−4−4−4],[444]

•  The channel decoder decideswhether the input is closer indistanceto[−4−4−4]orto[444]

17

channelencoder

channeldecoder

0000

channeldecoder

channelencoder

1111

signalencoder

�4,�4,�4

{0, 1} ! ±4

signalencoder

{0, 1} ! ±4

4, 4, 4

noiselesschannel

444

noiselesschannel

−4−4−4

Solu5on:computethedistancebetweeneachhypothesisand[5.17.7−4.5]

18

inputtochannelencoder

codeword Hypothesis Distancebetweenhypothesisand[5.17.7−4.5]

0

1

c

⇥�4 �4 �4

⇤

⇥4 4 4

⇤ ���⇥4 4 4

⇤�

⇥5.1 7.7 �4.5

⇤���

⇡ 9.3

���⇥�4 �4 �4

⇤�

⇥5.1 7.7 �4.5

⇤���

⇡ 14.8

Minimumdistance

Sodecode[5.17.7−4.5]tobit1

⇥1 1 1

⇤

111orasavector

⇥0 0 0

⇤

000orasavector

Wecancomputeprobabili5es.oferrorforsoZandharddecisions

19

X 2 {0, 1}channelencoder

channeldecoder

Y 2 {0, 1}PAMdemod.

(repe55oncode)

P {X 6= Y }

AWGNChannel

noisePSD=�2

PAMmod.

amplifica5onfactor=a

Eachbitisrepeatedn5mes,(nisassumedtobeodd:n=1,or3,or5,...)

Incaseyouarecurious,theprobabili5esoferrorequals...

•  Detailsofcalcula5onarenotcoveredinthiscourse

20

perr(hard decision) =

nX

m=dn/2e

✓n

m

◆hQ⇣ a

�

⌘im h1�Q

⇣ a

�

⌘in�m

ceiling(roundup),e.g.,d1.3e = 2d4e = 4

perr(soft decision) = Q⇣p

na

�

⌘

nisthenumberofrepeatedbits(assumetobeodd,)n = 1, 3, 5, . . .

−10 −5 0 5 10

10−4

10−3

10−2

10−1

100

Comparingtheirprobabilityoferrors,soZdecisionisbeNerthanharddecision

21

harddecision

soZdecision

ProbabilityoferrorP {B 6= C}

SNRperbit(dB)a2

�2

Thelargerthenumberofrepeatedbits,thelowertheprobabilityoferror

•  Adrawbackofalarge#ofrepe55onsisaslowdatarate

22

−10 −5 0 5 10

10−4

10−3

10−2

10−1

100

SNRperbit(dB)a2

�2

ProbabilityoferrorP {B 6= C}

hardsoZ

hardsoZ

repeatn=35mes

n=195mes

INTRODUCTIONTOTHECAPACITYOFTHEAWGNCHANNEL

23

Asharptransi5onexistsbetweenthedatarateandtheprobabilityoferror

•  The plot is for the best channel code, whichminimizes theprobabilityoferroratagivendatarate

24

Thistransi5onpointiscalledthechannelcapacity

probabilityoferror

0datarate(transmissionspeed)bit/sec

aposi5venumber

C

CapacityoftheAWGNchannelequals...

•  ClaudeE.Shannondiscoveredthisexpressionin1949

25

bit/sec

Transmissionbandwidth(Hz)

0.5×thenoisePSD(waN/Hz)

Receivedpower(waN) Thisra5oisthereceivedsignal-to-noisera5o(SNR)

C = W log2

✓1 +

P

WN0

◆

Thechannelcapacityisthemaximumreliablyspeed

Theorem(Channelcodingtheorem):•  IfthetransmiNertransmitsatthedatarate<C(bit/sec),then–  theprobabilityoferrorcanbemadearbitrarilysmall– byusingasuitablechannelcoding

•  IfthetransmiNertransmitsatthedatarate>C(bit/sec),then–  theprobabilityoferrorbecomes1–  regardlessofwhatchannelcodingbeingused

26

Thestatementisimprecisebutsufficientforthisintroductorycourse

Channelcodescanbeclassifiedinto2majorclasses:blockandconvolu5onal

27

CharacterisYc Linearblockcode ConvoluYonalcode

Atthechannelencoder,acurrentblockofoutputbitsdependson...

Thiscourseonlycoversthelinearblockcodes

onlythecurrentblockofinputbits

boththecurrentandthepastblocksofinputbits

Methodstoencodeincludes...

•  finitestatemachine•  shiZregister

•  matrixmul5plica5on•  polynomial

mul5plica5on

Methodstodecodeincludes...

•  minimum-distancerule•  sum-productalgorithm

•  Viterbialgorithm

Examplesofcodesinthisclass

•  Repe55oncodes•  Reed-Solomoncodes•  Low-density-parity

checkcodes(LDPCs)

•  Turbocodes

TherearetwocodesthatachievethecapacityoftheAWGNchannel

•  Low-densityparitycheckcode(LDPC)–  isanexampleofalinearblockcode– wasproposedbyGallager in1960butwerenotwidelystudiedforthenext20years

– appears in many standards (hard drive, digital television, etc.)duetoitsexcellentperformance

•  Turbocode–  isanexampleofaconvolu4onalcode– wasproposedbyBerrou,Glavieux,andThi5majshimain1993

28

LINEARBLOCKCODES

29

Checkyourbackgroundonmatrixmul5plica5on

•  QuesYon1:mul5plythesematrices

•  Modulo-2means:oddnumbersare1,evennumbersare0•  QuesYon2:mul5plythesematrices(modulo-2)

30

⇥1 1 0

⇤2

41 0 1 10 1 0 10 0 1 0

3

5 =

⇥1 1 0

⇤2

41 0 1 10 1 0 10 0 1 0

3

5 =

Fortherepe55oncode,theencodermul5pliesaninputbitwithamatrix

•  Fortheinput,theoutputis

•  Fortheinput,theoutputis

31

Channelencoder(repe55oncode)

u 2 {0, 1}

G =⇥1 1 1

⇤

u = 0

uG

uG = 1⇥1 1 1

⇤=

⇥1 1 1

⇤

uG = 0⇥1 1 1

⇤=

⇥0 0 0

⇤

u = 1

G =

2

66664

3

77775

Matrixiscalledthegeneratormatrixandcontroltheoutput

•  #ofbitsin≤#bitsout,i.e.•  Ifwechange,theoutputchanges

32

G

G

anymatrixof0’sand1’s

#ofrows=k

=#ofbitsthatarereadintotheencoder#ofcolumns= n

=#ofbitsthatareoutpuNed

k n

Linearblockcodeencodesbymatrixmul5plica5on(modulo-2)

•  Theencoderreadsbitsandcombinesthemintoavector:

•  Computesthemodulo-2mul5plica5on

•  Outputsthecodeword,avectorofbits:

33

n

k

c =⇥c1 c2 . . . cn

⇤

u =⇥u1 u2 . . . uk

⇤

=⇥c1 c2 . . . cn

⇤

c = uG thegeneratormatrix

Mnemonic:codewordconsistsof0’sand1’ssoweneedmodulo-2

Inthiscourse,weonlyconsider.inthesystema4cform

34

G

G =

2

66664

1 . . . 0

1...

.... . .

0 . . . 1

����������

3

77775anymatrixof0’sand1’s

iden5tymatrixofsize

rowsk

columnsn

k ⇥ k

=⇥I | P

⇤

Forasystema5c,thecodeword.consistsoftwoparts

•  Example:for,andforinput•  Theoutputis

35

G uG

uG = u⇥I | P

⇤=

⇥u | uP

⇤

Thecopyofinputbits

paritycheck

G =

1 0 10 1 1

�u =

⇥u1 u2

⇤

=⇥u1 u2 u1 + u2

⇤

paritycheck

modulo-2addi5on

c =⇥u1 u2

⇤ 1 0 10 1 1

�

Thecopyofinputbits

Example2:Linearblockencoding•  Supposethegeneratormatrixis•  Then,thecodewordforeachinputis

36

input output(codeword)u c = uG

⇥0 0

⇤ ⇥0 0 0

⇤

⇥0 1 1

⇤

⇥1 0 1

⇤

⇥1 1 0

⇤

⇥0 1

⇤

⇥1 0

⇤

⇥1 1

⇤

G =

1 0 10 1 1

�

0 + 0

0 + 1

1 + 0

1 + 1

(con5nued)Example2

•  Supposetheinputtochannelencoderis001011•  Then,theoutputis

37

Channelencoder

G =

1 0 10 1 1

�

00 1011

Example3•  Supposethegeneratormatrixis

•  Iftheinputtochannelencoderis

•  Then,thecorrespondingcodewordis

38

G =

2

41 0 0 1 0 1 10 1 0 1 1 1 00 0 1 0 1 1 1

3

5

u =⇥0 1 1

⇤u1 u2 u3

uG = 0 1 1

= u1 + u2 = 0 + 1

1

= u2 + u3 = 1 + 1

0

= u1 + u2 + u3 = 0 + 1 + 1

0 1

= u1 + u3 = 0 + 1

[ ]

Ques5on:RefertothegeneratormatrixinExample3

•  Supposethecodewordis

• Whatistheinput?

•  Answer:

•  Thefirstbitsofcodewordareinputbits.

39

uG

uG =⇥1 1 0 0 1 0 1

⇤

u

u =⇥1 1 0

⇤

G =

2

41 0 0 1 0 1 10 1 0 1 1 1 00 0 1 0 1 1 1

3

5

k = 3

Todecodetheblockcode,usetheminimum-distancerule

•  Theminimum-distanceruleissimplebutslow•  In prac5ce, a decoder uses a sum-product algorithm todecode– whichiscomplicated– butdecodesfastandisnear-op5mal

40

Herearestepstodecodethehard-decisionedinput

•  Supposethedemodulatorperformsaharddecision•  Sotheinputtothechanneldecoderconsistsof0’sand1’s

•  Supposetheinputis•  Todecode,

1.  Generateallcodewords2.  Findacodewordthatisnearestto:

3.  Decodetothefirstbitsof

41

v =⇥v1 v2 . . . vn

⇤

k

(Selectanycodewordincaseof5e)

vc

c =⇥c1 c2 . . . cn

⇤

c

Example4:decodingahard-decisionedinput

•  Supposethegeneratormatrixis•  Channelencoderusestheminimum-distancerule•  Theinputtochanneldecoderis•  Ques5on:whatistheoutputfromthechanneldecoder?•  Answer:Either00,or01,or11

42

Channeldecoder? 010

G =

1 0 10 1 1

�

v =⇥0 1 0

⇤

(con5nued)Solu5on

43

input codeword squareofdistance

u c = uG

⇥0 0

⇤ ⇥0 0 0

⇤

⇥0 1 1

⇤

⇥1 0 1

⇤

⇥1 1 0

⇤

⇥0 1

⇤

⇥1 0

⇤

⇥1 1

⇤

1

1

3

1

Thereare5es.Decodetoanyofthethreepossibili5es:⇥0 0

⇤

⇥0 1

⇤

⇥1 1

⇤

|c� v|2

v =⇥0 1 0

⇤

|c� v|2 = (0� 0)2 + (0� 1)2 + (0� 0)2 = 1

(con5nued)Matlabclear all;!G = [1 0 1; ... % generator matrix! 0 1 1 ];!v = [0 1 0]; % hard-decisioned input to channel decoder! !%----------------------------%!% channel encoder %!%----------------------------%![k,n] = size(G); % dimension of G!p2k = 2^k; % number of codewords! !% a list of k-bits: !% ulist(1) = [ 0 0 0 ... 0 0 ]!% ulist(2) = [ 0 0 0 ... 0 1 ]!% ...!% ulist(p2k) = [ 1 1 1 ... 1 1 ]!ulist = (dec2bin( 0:(p2k-1) ) == '1' );!

44

(con5nued)% a list of codewords!% codebok(m,:) = a codeword for the input ulist(m)!codebook = rem( ulist * G, 2 ) % modulo-2 multiplication! !%----------------------------%!% channel decoder %!%----------------------------%!% compute the squared distance between 'y' !% and each row of ’codebook'!dist_sq = sum( ( kron( v, ones(p2k,1) ) - codebook ).^2, 2)! !% find the row with the least distance![~, I] = min( dist_sq );! !% decode v to these bits!codebook(I,1:k)

45

HerearestepstodecodethesoZ-decisionedinput

•  SupposethedemodulatorperformsasoNdecision•  Sotheinputtothechanneldecoderconsistsofrealnumbers•  Supposetheinputis•  Todecode,

1.  Generateallcodewords2.  Mapeachcodewordtoavectorof’s:and3.  Findavector(hypothesis)thatisnearestto:

4.  Maptothecodeword:and5.  Decodetothefirstbitsof

46

v =⇥v1 v2 . . . vn

⇤

±a

w v

w =⇥w1 w2 w3 . . . wn

⇤

k

w

c

�a ! 0 a ! 1

0 ! �a 1 ! a

Amplifica5onfactorofPAM(aposi5veconstant)

(Selectanycodewordincaseof5e)

Example5:decodingasoZ-decisionedinput

•  Supposethegeneratormatrixis•  Theamplifica5onfactorofPAMis•  Channelencoderusestheminimum-distancerule•  Theinputtochanneldecoderis•  Ques5on:whatistheoutputfromthechanneldecoder?•  Answer:01

47

a = 2

Channeldecoder? −2.21.8−1.5

G =

1 0 10 1 1

�

v =⇥�2.2 1.8 �1.5

⇤

(con5nued)Solu5on

48

input codeword hypothesis squareofdistance

u c = uG

⇥0 0

⇤ ⇥0 0 0

⇤

⇥0 1 1

⇤

⇥1 0 1

⇤

⇥1 1 0

⇤

⇥0 1

⇤

⇥1 0

⇤

⇥1 1

⇤

⇥�2 �2 �2

⇤

w |w � v|2

⇥�2 2 2

⇤

⇥2 �2 2

⇤

⇥2 2 �2

⇤

minimumdistance

14.73

12.33

44.33

17.93

|w � v|2 = (�2 + 2.2)2 + (�2� 1.5)2 + (�2 + 1.9)2 = 14.73

v =⇥�2.2 1.8 �1.5

⇤

(con5nued)Matlabclear all;!G = [1 0 1; ... % generator matrix! 0 1 1 ];!a = 2; % amplification factor !% soft-decisioned input to channel decoder!v = [-2.2 1.8 -1.5];!%----------------------------%!% channel encoder %!%----------------------------%![k,n] = size(G); % dimension of G!p2k = 2^k; % number of codewords !% a list of k-bits: !% ulist(1) = [ 0 0 0 ... 0 0 ]!% ulist(2) = [ 0 0 0 ... 0 1 ]!% ...!% ulist(p2k) = [ 1 1 1 ... 1 1 ]!ulist = (dec2bin( 0:(p2k-1) ) == '1' );!!

49

% a list of codewords!% codebok(m,:) = a codeword for the input ulist(m)!codebook = rem( ulist * G, 2 ) % modulo-2 multiplication!% convert 0 to -a and 1 to a!hypbook = a*(2*codebook-1)!%----------------------------%!% channel decoder %!%----------------------------%!% compute the squared distance between 'y' !% and each row of 'hypbook'!dist_sq = sum( ( kron( v, ones(p2k,1) ) - hypbook ).^2, 2)! !% find the row with the least distance![~, I] = min( dist_sq );!!% decode v to these bits!codebook(I,1:k)!

50

HereareacronymsandsymbolsSymbol Meaning

AWGN addi5vewhiteGaussiannoise

PAM pulseamplitudemodula5on

PSD powerspectrumdensity

generatormatrix

thenumberofrowsof

thenumberofcolumnof

amplifica5onfactorusedbyPAM

51

G

k

n

a (a > 0)

G

G

Summary•  Overviewofchannelencoderanddecoder–  func5ons– harddecisionvssoZdecision

•  Casestudy:repe55oncode– encoder– decoder– probabilityoferror:hardvssoZdecisions

•  CapacityoftheAWGNchannel–  formula–  channelcodingtheorem–  capacityasthemaximumreliabledatarate– blockcodesvsconvolu5onalcodes– examples of capacity-achieving codes: low-density parity checkcodes,Turbocode

52

Summary•  Linearblockcodes– encoder(matrixmul5plica5on)– decoder(minimum-distancerule)– hard-decision–  soZ-decision– examples

53