-,

... 1

r -,

\ 1

"

"

o

"

"rACK t\LtivtU 'l"rlMS FOR DECODING CONVOLUTJ ONAL CODES

-- ..

, \

1

(

.. '

/

'.

, , ~ .:.,...- .";.':.-~.~_~'.L_'.~:jJrtrnttr~~;

...................... ------------~--~--

\

.'

r

MJjLTIPlE - PATH' STAK ALGORITHMS FOR , ,

~ DECODING CONVOlUT10NAl.. CODES

by

Da,:id Haccoun, B. Sc. A. & Ing. Phys. (U of M) S. M. (M. 1: T. )

Deportment of Electrical Engineerrn('

A thesis ~ubmitted to the Faculty of Gradua te Studies and Research

in' portial fulfjllm~nt of tt'e requiremenh for the degree of

Doctor of Philo50P~Y. ! : (

- \

\ . Depaftment of Electreol Engineering

.... /"

( " - 1

\

,.

,

j

, ..

"

MUlTIPlE-PATH STAK AlGORITHMS FOR

DECODING CONVOLUTlONAl CODES

by,

David Hoccoun, B.Sc'.A, & Ing, Phys. (Universitde Montral)

S . M. (M. 1. T . )

Def?Ortment of Electricel En9in~erin9

.. , (

"

-'

ABSTRACT

~

. A new class of generalized stack a/gorithms for decodi'1g convol-

utional codes is presented. It is based on th: Zigangirov-JeHnek olgo.rit~m but

insteqd of extending just th .top node of the st&ck at ail times, a nu~ber of th

"--mest likely paths are simultaneously extended. This number of paths moy be 1 ) ,

'

Il

J ,. , .. SOMMAIRE ..

, Une classe nouvelle d'algorithmes gnraliss utilisant une pile

pour le dcodage des c-?des de convolution est propose. 'Ces algorithmes drivent

de l'algorithme de Zigangirov et Jelinek, mais au lieu de prolonger seulement le , -

" .sommet de la pile, on prolonge simultanment un sous-ensemble des chemins les

plus vraisemblables. Ce nombre de chemins peut demeurer fixe ou peut tre vari , 'J

pour adopter l'effort de.dco'dage courant a Ptat actuel du bruit aans 'lb voie de

transmission. De plus la structure en treillis des codes de convolution est mise en

va'ieur par la reconnaissance et l'exploitatim de tOute convergence entre les chemins

. c:;xplors. La variabilit du nombre de calculs peut ainsi tre rduite jusqu ' une \

limite the'orique correspondant l'algorithme idal. De plus la probabilit d'erreur

est borne (borne suprieure) par cetle u dcodeur squentiel. Ces algorithmes , r

font le 1 ien entre le dcodeur squentieJ et le dcodeur Ide Viterbi.

Le chemin correct tant prolong avant d'atteindre le sommet de la o

pife, la chut apparente de sa mtrique cumulee est rduite'. Bien que la queue de la , 1

distribution de l'effort de, calcul suive la loi Pareto, 'on dmontre et on observe pai' . . simulatiol;l avec des codes de taux 1/2 et de IOl1guurs de contrainte courtes

,t

(K s; 9)/que ,compares ou dcodeur squentieJ,les'distributions du nombre de calculs

par bit de

"

\ i ii , <

, ACKNOWlEDGEMENT

, 1 am greatly ~.in,debted to Professor Mic~ael J. Ferguson {or h is - ,

\

guidance th~ughout *is resebrch. 1 wish Id express my sincere thanks for his

stimulating discussions anq criticisms, and for his many helpful suggestions during

the preparation of this dissertation.

The financial support of the E.cole Polytechnique de Montral ~ - ,

and Hydro-Qubec is gratefully acknowledged. Sp,cial tha~ks are also due to

my wife, Lyson for her encouragement during the course df this research. '.

/

, ,

\ '

(1,

"

.. : " . -, " .

:

..

l' 1 1 1

. -

:

o

...

., 4 . ~ -" , iv

e' ~ ~ABLE OF CONTENTS .?

/ - Page ...

~StRACt

SOM~AIRE ii

ACKNOWLEDGEMENT iii

TABLE OF CONTENTS IV

CHAPTER- . INTRODUC'HON . " 1.1 Outline of the Thesis' , t 6

, "1 .. 2 Claim of Cont;ibutions of this. Research 8

* C:HAPTER " STRUCTURE OF 'CONVOLUTIONAL ,CODES ln 2. 1 Conv.plutionaJ Encoder 10 2.2 The Structure of Convo'uti~nal Codes 17 2.2. 1 T ree StructUTe

\ 2'.2.2 State DiogrQ'hl 21 ,-1 2.2.3 T re" is Structure 24

~ t CHAPTER Iii DECODI NG OF CONVOlUTlONAL CODES 27

. \. 3.1 Introduction . ~ 27 3.2 Maximum likelihood Decoding of Convolutiont;J1

.. .6l "Codes. 31 :' ' .

~.(.,I,' . 3.3 35 Suboptimum DeC~ing . "l 3.4 Probabilistic Dec ding Algorithms 37 ',' 1 -" .. ~ . ...... ~< "}Ii: 3.4. 1 Tree Search: Se ential Oecoding 38 ')

3.4.2 Treflis Sarch: Viterbi\ Oecoding 55 3.4.3 Compa~iSO!:l5 and limitatio~s of Sequentil and

Viterbi Decoding 63

CHAPTER ' IV MULTIPLE l'ATH' SJACK OECODING\AlGORITHM 67 .,

4. 1 Introduction: Generalized Stock Algorith~ 67 4.2 Exploitation of the Remergers 70 4.2.1 Z-J,Algorithm With a Purging Cycle 75 ~.2.2 "" 79 c Purgi.ng Procedure ~ , 4.2.3 Speeding Up the Procedur'e 81 e, 4.3 Z-) Algorithm With a Multiple Path Extension Cycle 84 ~

" "

~.

,r . -v 0 .

/

" Pag~ ..

4.~. 1 Search Prope/fies ofS'equ~n~ol Deco"ding 85 4.3.2 . Mu Itiple Poth E~ten5ion f" 91 4.3.-3 An' M-pth Generalized Stock AlgOr'ithm 93 ,. 4.3.4 f;~mputotionol B~haviour of ~he M-.Pa"th Algbrithm 95-4.3.5 rror'Probobi 1 ity 104 4:4' '- CCI1}pUter Simulation Results 110 4.4. l

b ,. , 110 Sysfem Description .

...., 4.41:'2 Results of th Simulation ,; 115 , . . ..

/ CHAPTER V "ADAPTIVE SEQUENTIAL DECODING 135

\ 5. l Variants to th.e M-path AJgorithm 135 .5.2 Ideol Adoptive Stock Algorithm 143 5.3 An Adoptive Stock Algorithm 149 5.4 Model of the Adoptive Stock Algo,ithm 152 5.4.1 Average Distance Between No-Search Node.s 155

> . 5.4.2 SE"orch Mode Probobility .. 164 5.5 Computer Simulation Results 170

CIiAPTER VI CONCLUSIONS AND SUGGESTIONS FOR , FURTHER RESEARCH 185

APPENDIX THE GENERALIZED STACK DECODER' 189

APPENDIX /1 MARKOV CHAIN MODEL OF THE C'oRRECT .-PATH METRIC 193 ..

J

. APPENOIX "' BRANCHING PROCESS.MODEL OF THE INCORRECT PATHS 204 APPENOIX IV. SEARCH MODE PROBABILITY Z14

.-1

REFERENCES 218

.",. , .'" ~ l' ,t' .. '

" ~ '\ ~ . .

#

J

'e ,. .. ~

\

" } J

,.

'- ,

'tI CHAPTER

INTRODUCTION

.. Communication i,s essentially the process of transmitting information , from one point t~ another through a noisy channel: ln a generol communi~atton , . s;stem,shown i~ Fig. (1.1), a message sou.rce generates m~ssages that must b~ transmifted to a user over the channel. A specifie signol is assigned to each of

,1

the possib)e M messotles wh ich the systm can transmit, nd the selection rU,le . , ._ ~ thot Jssigns a transmitted signol tcf'each- possible messa9~ is cailed the code. , --

Beca~se of the channel noise th~ tra~smitted signal does ~ot ~rrive at the receiver "'. " l

_ ~xactly as transmitted, and hence rt'brs ore made in cOflveying the source message

, to the user. The coding theorem of Shannon (1948) demonstrates the existence of

~ .. (,

codes that achieve reliable communication if and oilly if the infornption transmission

rate R is less than some maximum rate C - cal/ed th~ channel capacity.' If ~

Since Shannon 's r.sult ~ppe~red, ofnsiderob'e omoun! of work hos been concerned with the se~rch of coding-deoding tchniques that permit commun-

"1\ ication with low error probobility, and which are simple to implemnt. The first code "

investigated were cal/ed bloGk codes. In these codes, a s~uence Jf K information

symbols is encoded in a bl9ck of N ;ymbols to b'e transmitted over the channel. ,

Block codes h~ve been studied extensively (8erlekomp ~ 968, Gal/oger 1968).

'Atthough Shannon prpved that block codes exist for which the error probability 1

',J " d - ,:fecrease5 exponentially with the black length N, th long codes which are simply ,~ -

~ ,,', ,..." ~ .. '" ": d''coded yi~ld a significantly greater ~rror probability than predicted by the theory.

, ,

l '

..

/-~ 2 '~ ,

" 1 1 e. '.

"

... II) III

" ::>

... ( , , , p

,. -;, . .. . f1

... ., f " / 0

" , 0 U II)

q If . , / " ~

", . ~ ;JI i' "

1 .

, (., ~

E -' II) ~ .... .., ~. ~I III ~

,$

~~

c: -1 ~Q -, ....

II) 0,

. 0 ..s: , U

~ c: ,~

"

E .. ~

Q U , .

, '0 ... QI . c: QI

C> . , ... II)

"8 -u ... c:

w (

... -II) ... :> , .2> . u.

... .

. e

...

. 1

..

..

, .

, . p

. ,

.,.. .

.. . ~.. :\. ~ Convolutional cofes ore codes which are par'ticu1orly suitoble ,. ,-

when the inf~rmation symbols to be tronsmitted arr.~v~ seria'lIy rother t~an .in r-

.. ' 'arge I:>I9cks. Randomly chosen 'convolutional ~odes are known which achieve

" . . . er r probobiliHS decreasing xponentially with the constr.nt iength-of the

-

..

." <

"

, encod. Jn g~neral convolutional codFs outperform block c~des df the, sortie' ) ... .'

ontr of d c.oder complexi ty. _ . -

.. . . -In 1957 Wozencraf( (1957) proposed a decoding proced~re colled

, " , (J "

. sequential decoding wherel;>y the tree-lilte str.ucture of the co'nv~utional code ) .

/ ,

is used in a step >by- step ,search of ~he most Jikely ~ransmitt~d ~$~age. As long

. ~ . as the fote does "not eX'Ced a qlJontity caHed R , which ;9 less thon the

, . .: comp ft . - . chonn~1 copocity, th average number of comgutotions necessary tp ~code one ,

infor~tion digit is small and independent of the constrciint length of the code . . " .

There are two principal sequential decoding algprithms:~e Fano . ~

algorithm (Fno 1903) and th~ Zigongirov-J~linek (Z-d) stQcj< olgorithm introdu,ced .,. "

,independently by Zigongirov (1966) and Jelinek tl969a~ Regordls~ of'the olgorithm', d

a maior problem with sequ~n'tial 'decoding is the variability in 'the n~mbe( of computotio~s . . - .. ~ ,

required per decoded'digit. The cumulative distribution P (.~ N ) of the number of . . o .,

, co.mputations performed per decoded digit is upper and lower bounded for the

/' .r 1,.

l'

r

.'

. .

J

..

J

J'

. , ',4

, L , ,

r" 0' J ' 1 inearly with the cOl]sfraint le~gth. ~onsquently, convolutional ~ncoding and

~:

seq\,Jential decoding are' among th;-mOst attractive means of a~hieving digita,l l' 1 ~ f -y r) l'

communicati~n over memoryless chann~ls at the low er~r pro bah 1 il ity predicted' ,0

, , .' by the theory.

,In the tree represnta~ of a c~"'rivalut,ional cod~ a ~t,~ is . .spedfied by,the input message th,Qt entered the encoder. But' in ,a convdlutional

, ~lof. 1

1

encoder of finite constfaint fength' K, the encoder outputs depend onIY"upon the

..

span of K', K' ,

e.

\

1

. ' $

.. .. 5

Given th'e received sequence hom the channel, sequentiol and

Viterbi decoding apper te be at the extremes for determining the most likely

transmitte,d message. Thefprime motivation for this researoo h~: been the desire '~1 fi, ~ ~ , ,.,

to alleviate the variability of the computational effort of sequentiol decoding .~

., , , " fY,closing the 9C;1P between the 'single-path seqvential decodlng and the ,,11-

paths Vit~rbi decodil19.

or Keeping the sealich properties oT sequentiol decoding while using

" . some of th~ concepts of Viterbi decoding, we develop a c10ss of generolized

stock decoding a 19orithms which unify these two seemingly different decoding' f '

techniques. The variabifity of the computati~nal effort of sequentiel decodir'lg is

shown to be reduced without degrading the error performance.

The fundamentol idea of the generolized stock olgorithm is to ovoid

b ' back-searching for the most likely tronsmitted sequenc.e th!:ough the si multaneous

extension of a subset of the most likely paths., Moreover some redundant and useless li, '

'decoding effort is elimi~'y usin'g the trall.is structure of the code. In the process, ~ . -,...

_" 1. 1 (~ .vents which could lead to erroTs-for sequential decoding are shown to be corrected

. .

/ , ,

by this technique. However, unless 011 distinct paths are constantly extended, a

:"1earch mode of operation si mi lor to that of seq~ntjal decoding appears to be , ~

~

asymptotica.lly inescapoble .

f The generalized stock algorithm is further refined by exploiting the -------~ ___ ,

information about post c{ebding behavioul"l and current stote of the, chann'el noise

. in deciding o!, the actuol number of paths that must be extende~. Hence the d~coding

J ( s

~ .. ----------------~, ~~-----I

',.

r

, \ 6

effort tends to ,be adapted to the particular requireme~ts imposed by the chQnnel

noise, As a co~:equepce both the computational' behaviour and error pe.rJormOnce l' r:' j ..

can be furt~,~;:mprov~d. The limit to this adoptive ~roced~~e is shown jo b~~given by an optirp6)m unrealizable ideel stock algorithm..wt.ich lower bounds fhe compu-

Q

----- . , ~

1 / .,~

tational 'Wformance d~ any stac~ decoding algorithm (including Viterbi and ,y'

sequential decoding).

"'" 1. l Outline of the Thesi!'

Chapter Il is a review of the structure of convolutionol codes. The

essentiel concepts,error performance and computotional behoviour of the sequential

(Z-J) and Viterbi decoding algorithms are presented in Chopter III. The two .

decoding algorithms br~ compared, and inethods to alleviote the computationol

variability of sequential decoding ore introduced.

- ln Chapter IV we first show that the Z-J and Viferpi decoding

algorithms are in fac~ only particular cases of a more general dass of stock decoders.

We th~n proceed ta bridge the gap between these two algorithms by generalizing~the,

Z-J decoding algorithm. A new M-path stock deGoding lgorithm is described,

and its computational behaviour and errOr' performance ore anolyzed. In this

algorifhm the useless computations ohequentiol decoding are eliminated by

exploiting the reconvergence of the explored poth1, andJhe variability of th~

computational effort is reduced by the simultaneous extension of the set of M,

M}I-l- most Iikely pat~s. Results of a simulation of the M'-path algorithm with fI' ~ .

1

7

1

codes of constraint length K ~ 7, over a discrete memoryless gaussian

'channel are presented md compored to the Z-J algorithm.

The e'xploittion of some of the information imbedded in the stock,

ta help reduce further the voriobility of the computationol effort without unduly

incresing the average amountJof computation is exomined in Chapter V

. Simulation results for variants of the M-poth algorithm are presented in Section

( 5.1. An ideal adaptive stock decoding algorithm is proposed in Section 5.2 and . shbwn to be the best algorith~' in the class of generalizea stock algorithms. A

practical adoptive procedure is then described, modelled and its modes of operation

o onalyzed by Markov chain techn iques. The chapter ends with a discussion of

,simulation results for sorne implementotions of the Qdaptive algorithm with short.

constrant length cpdes (K:s: 9 ).

Chapter VI conta ins" fina 1 co~clusions and 'Suggestions for future

research.

ri' ! Appendix 1 contains a bri~f description of the generalized stock

decoding agorith'm as we have programmed it: Massey's Markov Chain model of ~I '

the correct path metric differences is r~iewed in Appendix Il. This model is used

ta determine the average separation between breakout nodes of the correcf poth.

ln Appendix III a branching process mode 1 of the incorrect paths is used for a

newapproach in upper bounding th, average decoding effort of sequentiel qecocHng ....,.

Appendix IV is an altemative method of determining the probability of entering

a search mode of opefation presented in Section 5.4. '

..

;. ,

1. 2 Claim of Contributions of this Research

Some spec ific contributions of this research are summarized below.

A new clos! of generaljzed stack decoding algorithrns is proposed.

ln this 'class, depending on whether the poth extension decision rul~ fixed o.r

variable we distinguish two important subclasses of algotithms.

(o) The M-poth algorithm which unifies the l-J and Viterbi decodi'ng

algorithms.

(b) The adoptive algorithm where~ variable number of paths are' extended

" depending on how much the maximum stock metric has dipped. In

{a}

this subclass the ideol algorithm is shown to be the best possib..le stock , decoding algorithm.

ln thes~ algorithms we developped:

A practicat procdur!! to recognize ~nd exp,loit the convergences of , \

the poths.

(b) li A simple rMthod of extracting the exa~l top node of the stac~

We showed and verifieEl through simylation with short constroint length

codes (K ~ 9) thot:

(o)

\

Compared to squential decoding, the vQriability of the computotionQI ".., . 1 ~

~ffort is redued .. .IIowever 1 the tait of the computational distribution

is sti /1 Pareto

\ " , ..

,

--------.. ----------~--~----------

/

9

"(b) The errOr probability i, upper bounded by that of the ordinary

sequentfol decoder and lower bounded b)C. that of the Viterbf :

decoder. t

il

, J

., .' 2.1

CHAPTER Il

i' .

STRUCTURE OF CONVOLUTIONAL CODES

Convolutional Encoder

A binary convolutional code of rote l/V bits/symbol moy be

10

generoted by a lineir ~inite-state machine consisting of a K-stage $hift register,

V modulo-2 adders connected to some of the shift rE'gistet stoges, and a commu-

tator thot scons the output of the modulo-2 adders. The mchine is ca"ed a

convolutionol enc~deT and is sketched in Fig. (2. la).'

'Let us assume that the information to be encoded is a sequence

(2. 1)

of binary letters, O.or 1. Assuming the shift register to be initially filled with

an ali-zerQ sequence (or any other known sequence), the first binary digit u1 . . ,

is fed into the first stage of the shift register. The V rnodulo-2 odders are then

sampled in sequence by the commutator, produc ing V output caded symbo Is

(1) (2)' (V) Xl ' Xl , .. Xl (2.2) .

, " where the output of ~n adder is "1" j~.9pd'-~nly if the totqJ. number of l's in

.. ,,~- ..

the"shift register stages conne:ted to the odder is odd. Otherwise the adder

output is "0". After the Vth output 'symbol X~) 150btained, the second

ir,:lput 42 is}h i fted ir1to the first stage of the sh ift register cQUsing the contents

of 011 other stages to move one step to the right. The right-most di~it leaving

,'~

.. ~

,.

1

, .

, 11

1

the encoder is lost. The V modulo-2 adders are again sampled, yielding V new<

output symlls x~l), ~~2) , ... x~V). ~his procedure continues unt~ the lost . --.,

input di~it u L enters the shift register. In order to retum the shift registrno

il, in;t~al zero 'Ial~, wilh'eacf! ,hm of uL a zero 'Y~~I i. fed-in unlil '~l " " . leav~s the shjft' registr. Th is terminating sequence.of (K; 1) zeros is calJed '1 ~,' ...

"

the tail of the message.

The L -bit message sequence U of Eq. (2.1) produces the output

sequence or codeword

, . r (1) (2) xc' { )Q _ \ 1 ' Xl '

of length (K +L-l ) V binary digits. The rate of the code is

and for l K

l ~ = (L +K-l ) V bits/symbol

('"

1 l' R::.:f V bits symbol.

let the row vector

G j '= (g1j' g2j' ... gVi)' i = 1,2" . K ..

specify the connection between the stage of the shift register and the V

, (2.4)

(2.5)

(2.6)

modulo-2 adders. The component g.. is a

" 1 if the i ~h modu 10-2 aerder is

connected to stage j, otherwise it is a O. For example, the rate 1/3

,1

.'

u

\, /

, '04

K. stages

V modulo-2 add~rs -

/ ~~ ___ ~_. ___________ . x

Figure (2. la) Convolutional encoder.

."

u

~----___________ X

~

G1

= 111

G 2 = 1"10 ~

G3 = 10l

"Figure (2. lb) Convolutional encoder with K = 3, V = 3,.d = 1.

\.

! .

'. 1

J2

\ /

1

J

,,J

. , , ( 13

.( , , , convolutionol encoder encoderrshown in Fig. (2. lb) hos the following connection

"

vectors '-

(2.7)

G 3 = ( l,? 'n l ) "

From the operation of the encoder, the V output digits (l) (2) (V)

x. ,x" 1 X. 1 1 1

produced whe'n the input binary digjt u. is first shifted in the shift register are 1

(2.8)

where the sign Gl represents the modulo-2 addition. Relation (2.8) is the ,

convolution of,the vectors GiS and the K-bit long input sequence u., u. l' .. -- 1 1-

ui-

K+

1, The term "convolutional" is taken From the frm of the relation (2.8)

and c learly the vectors GiS specify the cpde. We observe that the V outp~t

symbols corresponding to a particular 'input digit depend 'Ipon that digit and the

( K-l ) preceding digits that entered th,e encoder., The constraiRt length of the

code is defined as the number of shifts ov~r which Q' single- information symbol can - . \

influence the encoder output. For the simple binary c~nvolutional code, the

constr~int lef1gth 4S equalnto, K, the length of the shift register. ~

, Considering the input binary sequence OS' on l-dimensional row ..

vector U, the (l+K-l) V - component output X con be written, usin~ modulo~ -

arithm~fjc, as

,

\

\ \

14.

X==U[Gl (2.9)'

where

( L +K-l ) V columns ... ,

G 1 G2 G3 GK , .. G1 cf. G3 ~K -2' ..

" G1 G2 G3 G K ,

[G)= l rows

.. l

l r , G ;. G3 GK (2.10) 1 -2 The matrix [G} is cal/ed the. generator matrix of the-code, and the KV"

component vector r -

:G* = ( G1, G2, G 3,. (2.'11)

is cal/ed the generotor of th code. The first row of the matrix, [G J is the , ,

generator G* followed by (l_-l ) V zeros, 'and each succeeding row is the . .. . , , previ~us row shifted V places to the right, with ail elements to the left of G

1

equol to zero. The number of rows is the length L of the input sequence, and ,

the number of columns is (K+L-l ) V. Every codeword X con be expres$ed os \. -' . .

a linear combination of the rws of [] and the code may thus be seen os the' -

..set of aU 2l

I-inear combinotions of the rows of [G,. for the exomple of .

Fig. (2. lb), the generator of the code is , . -G* :;: (111 110 101) 'f

1 1

, i

'.r' '" .

1

15

and ~r the 4 bit long inp'ut sequence

U t:: (1011 )

x = ( 11 L 11 0, 01 ()., 00 1, 011, 1 01 )

" , One can generalize this binary convo'utional encocrer by

aLiowing the K-stage shif~ register to rec\ive groups of d q-a'ry information . . \. -

symbols of an input sequence whose components ore element; of a finite Galois . -fi~ld ' GF ( q )., Then the input to the encoder is the sequence

(2. 12)

"

whre

J _ (1) (2) (d) u, - (u. 1 U. 1'" U. ), -1 1 1 1

u~j) E GF (q ) 1 i = 1, 21" ,' d, i = ], 2, " l 1 C

4

The V moCiulo-2 odders,' V> d and their connections are reploced by V

inner produ.ct computers, eoch of which compute the inner p,roduct in GF (q )

of the shift register con~ents anq so~fie~ vec~r. In o.ther'words, the

lements of the generator matrix [G 1 bel\"9 to GF ( q)/ and the operatIons

of Eq. (2.9) are performed by arithmetic operations in' GF ( q). The length

K ~$ ch~sen to be 0 multiple of the integer d,

,,'

, ,

.. . . , . . , . ..

-,

. \

.'

1 ,.

16

K = kd (2.14)

. , and-~e constroint length of the code' bec~mes the numb~! J = K/d of groups of

d shifts over which ~ 5ingte ih~rmation S~rF' dqn inf!~ence the encoder output. "i'>

:''---~~p~~.~.--------------~----~ .... ~( ............... ........ (

. , ~ .. 17 I-er .. ,

G * :::' ( 1 01 00 1 111 011) -2 " ...

;

.. ' ~ is sho~n in Fig.

encod~d as 010

(2.2). The input seque~ce sta~ing W(ith 11001001

010 O 1, ~1 .. , .,' . . . -

"'/ wiltbe , ,

The Structure of Convol~io~al Cod!!s. ,

From the above description of. the operation of'a convolu'tionaJ ,"

encoder, we observe that

(1) ;;, ~ l'; l ,

The code symbols depend on post values of the lnputs. " . , '.

~ (2) The post dependency does not extend to the infin ite post but s

limited to the length of the shift register 1

. ~ (3) The code is line~r sin,ce the out,put symbols are linear combinations

of the post inputs, , ... ... ~

We now sho~ that the first tv.y,bservations leJd ta two d~ffere~t

tations of the convoJutional codes.

2.2,1 ee Structure ..

" o . .) ," d

:'The .fact that ,at any time, an input may ta'ke q values hen,ce

leading te, qd ~utput sequences, ~Ubge~S ~e~~esenting the out~ut Qf a ~onvolutio~al 1 t ., '10 ,',

'1

od b th dn 'bl 'enc er y a tree WI q,' poSSI e sequences, corresporlding ta inplJts. The. .

n ~

r

~ ., .' ,

~ . CI .q

1

\

t> ~

;\ e ' " ."" " ..

.> a.. ." + """-- ~ 18 ~ '\: .. "

/ 'c e" / '-. ~ , t '" .. "r 0 ' ,,, 0 ~

'C ., ~ . 1 ,1 "tJ> . '() 11. d'_ .. l4> , ~

, 0 " " .. (l'Y> (1) (1) u3 u2 u , ~

,~ o .~ ,

\

x3 ~ . , J) c;o;

x2

M

X .. , y--- " ~ -,. . 0 , "' ~,

. ' ~1

~ , , . .. (2) (2~ (2) ", ~ , "

'~3 ~2 u 1 . ~ , .

~

'.

\/ ". .. " . . " .... ~

, \~ ..

, ,

- ,

19

\ ." -root node of the tree hos no predecessor whi le every other node hos one predecessor. " \

There ore qd branches stf:!~ming out of eachl:node, with the exception of the nodes

corresponding to the taU oflhe message which hav only a single branch extension.

"' Ail bronches corry V coded symbols. Fig. (2.3) depicts the tree representatio"\

for the outputs of the encoder of I\ig. (2. lb).

. .... A poth . in the tr~ is troced from Jeft to ri~bt occordin~/o the .

input sequence lhot speci fies i t. For the b inary tree of F i~. '(2,.3), 0 0 input , means toking the upper bronch leoving a node, and a 1 meo~~ tcrking the low~.r

, "

branch. Hence a message sequence ~1' ~2' ~3' ". traces 0 poth throJ.!g~ the _1 ~_ , ,

i'>

tree, and the correspoAding coded ~Is of fh( bronches of the poth ore the

channel inputs that ore\transmitteQ, ' ln' Fig. (2.f>, the input message storting . t. 1 ('

'with the sequence 01100 .determines the path in~iate~ by the thic!f'U'}e having

tht'! encoded sequence 000 III 001 0) l 101. The tree of Fig. (2.3) is said to hav , r

fi.ve levels, 'one for eoch bronc'j99 olong a possibl-e patb. The number of fevels in

a tree con be extended ind~f,Lt~I;, and noturally there iSf{] one-to~one corres-'dl 1 \ ~pondence between the set of al! q informti?n sequences of ength dl an

l'

'Set of poths through the L levels of the tree. A code thot con be represented by .., , , a tree IS calred a't.ree code,

. ! .. ln general, tree ~odes ore codes where coded symbals

1

depenCl on post input sequences and where poths moy be re~resented as branch

cho ices on a tree.

..

f

\ . . ,

20

...

..

,j

000 000

111 . 000 - S -0' 111 1

110 (Jo'l " 000 S

-0 110 lUl

111 010

~1 001 1 011 100

S .yOOO 101 1 -0 " . 110 {;I_ I 111

,~. ~2 010 110 001 - 111 01

~1 101 - . 011 .-010 0', ' 001 ~3 011 - , 100 1 -100 .

Root node 000 . 000 1 1 ! ,l .. - 15 111 , 101 ~ 110 111 001 H.,9 -.

~2 101 1101 010

,J - 010 " . --. ~1 001 011 100 ~1 000 101 " . 111, -Olt . ,

~2' " 010 11 () 001 001 , 23 OlT

101 4 . .

100 1 010 S 011 , -3 100

, 100 ,

\ "

Figure (2'.3) Trec representation of encoder of Fig. (2.1b)

. . \

..

21,

2.2.2 :Stote Piogram,

'In the causal transformatian of the input seque'nce U into the

coded sequence X, at any given time the V output digits are ,completely

determined by the most recent k groups (inc/uding the present one) of d

information symbols that entered the encoder, and are residing in the sh ift r;gister. , .

For every new shift, of input symbols, the output will d'epend upen the previous

( k-1 ) shifts of input data, and c1early, if the first nd components of 2 or

more input sequences are equal, the correspending output sequences wi Il be identicol

in their first nV components.

, A cOJlvolutional enc:;.od~r w.as defined as a fi.nite stote machine,

and since a fin ite state macit,ine changes From one stote to another occording to .the

input it receives, the state of a convolutional encoder is then simply the (1

22

The toto 1 nomber of state's is then

( k-l ) d 'K-d q , =, '

d and From one state th~ machine moy move to q other, states. Naturally 1 sorne.

input symbols moy cause the mach ine to stay in the same state .

... This descripti0'f 5IJggests representing the encoder by astate

diagram with q~-d states or nodes and qd branche's leaving and entering each

state. These branhes represent the transitions of the encoder from one state fo

another, and therefore they cerry the V coded symbols delivered by the encoder

in the transi tion. For example, the state diagram representotion of the encoder

of'Fig. (2.10) is shown in Fig. (2.4). \t hos 4 states and~ach branch is labelled

by the corresponding input (above) and output V-tuple (below).

The stote diagrom is a very compact representotion of the encoder.

Storting from the initial stote S := ( 0, 0, ... 0), the coded output corresponding -0

to the, input sequence U =: (u1, u2

' u3

, ... ) is reodily obtoined by writing the

branch symbols corresponding ta the transitions due to u1, u2, u

3, ... For exomple,

from Fig. (2.4), the output ta the message 110100 i~ 111, 001, 011, 010, 110, 101.

A potentially infinlte tree hos been reduced ta a mere 4-stote diagraln~

Regarding the sfote diogrol11 a a signol flow graph, particular codes

can be analyzed via the transfer function of eir dia9fom (Viterbi 1971). However, , th~ greot diffiulty in finding the transfer fu ction for long constraint length codes

limits t~e'use of this technique to very short constmint length codes.

2

u

(1)

1

.. ,.

24

2.2.3 Trellis Structure

Using the definition of astate, to each node of the tree tnere

corresponds an encoder state. For~ input sequence of dl symbols, there are

qdl terminal nodes in the L-Iever tree, but only qK-d different states. There-

fore 1 for L ~ (KI d ) 1 the number of nodes in the tree exceeds the numbe r of p

states of the encoder. This means tha,t sevial nodes must correspond to the some ..

encoder state~ and hence the coded tree paths are not distinct over their entire j

length. From our discussion on the state diagram we know that after the input

symbols have been identical for (K-d) consecutive symbols, the encoded

symbols are the same: the paths remerge. However, th is fact is obscured in the

tree'representation of the code since the tree keeps an branching off with q~

branches From each node, as if the states were ail different. This behaviour is

equivalent to considering an infinite number of states or a dependenc~ over the

infinite post. Realizing that the number of sta,tes is finite, we see that the tree

contqins much redundant information which con be eliminated by merging together 1

at any some level, ail nod~s corr'esponding ta the some encoder state. The

redrawing of the tree with remerging paths is called a trellis.

Taking the example of Fig. (2.3), let the four states by ~ :=: (0,0)

~1 = (0,1 ); ~2 =: ( 1,0); ~3 = ( 1,1 ) Xfter the third level, the data sequence

100 abc ... , and 000 abc. .. generate the same code symbols, and hence both

-nodes in the figure labelled ~ can be ioinecl together: The Same reason,ing applies

for the nodes labelled ~1' ~2 and ~3' and the tree collapses in the trellis of

25

Fig. (2.5). More insight about the remergng of the tree paths in general is l"

,

1 given by the following properties (Viterbi 1967).

r Property 1 :

If tw~ sequences of i{lfbrmotion symbols are identical except for .'

n consecutive groups of d information symbols, the corresponding codewords

w'nl be distinct over (n+k-1 ) consecutive branches.

Proof:

The paths in the tree code, corresponding to the 2 codewords , ..

will be identical as long as the input sequences are the sorne (prior to the nd

different symbols), and will diverge, i.e. will be different, for the n

consecutive branches where the input symbols are different. Thereafter, although

the input symbols are the same for the two data sequences, the code symbols will

be different as long as the encoder states are different. Hence, (k-1) additional

shifts ore necessary before obtaining identical outputs, causing the paths to diverge

over a total of (n+k-1) branches.

Property 2:

If the information symbols of two paths agree at sorne point ,in r

(k-l ) branches, the $ubtrees extending From these two paths thereafter must be \ .

identical.

.'

.. )

26

The trelli-s representotion of a convolutionol code is more

instructive thon the tree representation for it explicitly uses the foct that the ... encoder is a finite state machine. Each stote of the trellis hos q~ possible

d J. . successon, q possible predecessors, and From level to levl the struct.ure is

repetitive. "".

This redution of the tree str~ture of the code into a trellis

is c;redited to Fomey, (1967) and the trellis structure is at the heart of an

optimum decoding tec~nique for convolutionol codes due to Viterbi (19'67).

This decoding technique will be present~d in section 3.4.

" 1

"

,.., "

"

..

~

,

27

CHAPTER HI

DECODING OF CONVOLUTIONAL CODES

. 3. 1 Introduction

" ln a terminated convolutional code, the message input sequence !

U is enc6ded as a sequence X and is represented by a porticular poth through , - ~ . the tree or trellis of the code. This codeword 'X (0150 called the transmtted

s~quence) is sent through a noisy memoryless channe,l ta a decoder and a use'r. 0

A discrete memoryless cflannel (OMC) is an idealized model of

a noisy channel with digital input and quantized or digital outputs. The input

ta a DMC is a sequence of letfers From a Q-symbol alphabet, and the output is

a sequenc~ of letters from a J-symbol alphabet. During each channel use, a

letter of the input sequence is fransmitted, and the corresponding output letter > 1

is received. A r~,ceived letter j, \ l, 2, ... J is assumed to be statistically

dependent only on the corresponding input letter i, i = l, 2, ... Q, and is

determined by a fixed conditional probobility a~ignment P ( i/i ). S!Jccessive 1

inpuf-output transitions are random and statistically independe~t.

The decoder observes the output sequence of the DMC corres-

pondi'ng to th'e transmitted sequence, and determines which of the possible qL

q-ary data sequences of length l entered the encoder. Fig. (3. 1) shows a G

comrrlunicatiol'l system employing a convolutional code and (t . DMC.

l The encoder simply maps the integers 1 to M = q onta the

codewords X(l) ta X(M}. As message m enters the encoder, ~(m) is

, ,e, ('

-Data

y

.-./

.. .. Trnsmitted

Convolutiono 1

.

" ;0

Encoder

binary input

(Q = 2)

.sequence X - --

,

~

.

.. D.M.C.

" ~

Received sequence

Y - .. , . .... ,

. .,

' .. ..

,....

Decoder .'

8 outputs

DMC with

Q = 2; J = 8

Figure (3. 1) Communication system and Discrete Memoryless channel. ~ ,

tf'

1':1

,

--~

...

r

'"

'"

User

" U

;;

~

\ tr~l')smitted, and on the basis of the corres~nding receivd sequence '1., the

, decoder pro(juces an integer m', Decoding errors occur if m t ~', "

"

30

Likewise let

v. ::;: (y.'" Y'2' ." y. ) LI 'l, 1 IV (3.4) c

"

represent ,l'he V received symbols when x~U) is transmitted. Then~ E~. (3.1) \} -Y' \ .. .

the like/ihood function for the jth branch of that poth s.

, , "

(3.5)

;'" 't' since the channel is ossumed memoryl~ss, .

(.

. Gen;r~lIy it is more convenient to use the logarithm of the

likelihood function since the logorithm i5 a monotonely, increasing function, and i: ~ \.

, . , tHerefore does, not alter the final result. Defining the log-liklihood Junction

'-

;y ~U) for th'e jth branch on the path speciHed by the input se~~nce b -as cl,

, . (U) A

y.- = 1

log

, 0 V ,

1

1

1 ~

1

'1' f

,.

1

, ,

, 1>

....

, J

, . ~ 0 a

31

. . maximum likelihood decoding pf convolutional .codes may be ~imply regarded as

(J 1 ~ n

~~ finding of the poth with tJi'e I~rg~st accumulated me!ric, giw;n the received 'l-' , ~. , p ,

sequence. However; dePending on whether the tr~e or trellis'structure of the 1-

code is used, the d.terrrlinatior'l of t~ MOst likel};' poth will lead to either an

. imp~ical or practic~1 de~oding technique. ., " '-,'< , J .. '--

, Maxim(Jm tikelihood Deeoding of C?nvolutionol aCod~ ..

By d~finition, n;lOximum likelihood decoding implies comporing

/ , ' "the reeeived sequen-ce with ail possible transmittd seqences before making 0

-) , J _

decision. ije,rc;~" in g~ne'raJ for memoryless chonnels, the optImal decoding 'of (1' ., - fi

an l-bit long bina'ry seque,nce requl'tes c~parin,g the ',2L ~'Cc'umulated metrics

of the 2L

di,fferent c'odew?rdsJh~t .;cQuld have ~een transmitted~ and picking the . . 1

best one .. 6ecouse of this exponential inc'rease of the decoding effo~t with the . ' . Jength t of the sequen~~1 maximum likeli~ood ~ecoding is u;~o!t difficult to

, , \

, . 'its use as a standard with wh1ch other practica'i suboptimum deodi~~ techniq~es mey

, 'S- -'>- r _"'"

be coh1pored. Mor.eover"..it is used to q~rer!"ine thebperformance of codes, since a

meas~~ of ~er!~rmance fOl\apy c>de is the probability ~f error witp the optimum ..

. . ~ :~ ~/ , 1

,For the decodi~9 of 'ton vo lvtiono 1 codes, either the tr~ or trellis c. '" ';

decoder.'

, structur~ of -the c0cte may be used. Ir the tfee representati~n of. th~ ~ode, \the fact

Q \

, \ that the paths remerge is entirely ignored, and hence the decodin-g of an L-bV

, C" \ '\ , .., 1 . -'.

b

Q

1

. .

L

,>

...

32

:-.. seq,:,ence reqyires the exhaustive c'tmpariSon of 2

L ~~cumt.itd ~etrics. Conse"" l'

. " qumtly over the tree structure of th~ ~ode: optimal decoding is not proctical, , ,

and g'i~en th~TVed sequence, the estimation of th~ most Ukely' data. seque'nce would coll for a suboptimum but more practicol decoding technique.

~onsl..der now the trellis st?ucture of the ~ode where the rfd~ncioncy "

of-the trie was elimmoted through merging. -In princ iple this redundoncy o;ouid be 1 ,

\

" exploited by a clever decoder which would consider only those paths thot could 1 '"

'ever maximize the Ifkelihood function ov~r the whole set of paths. In th~ ~~o(jing ~"j

#'

. " process, iF at some point it is realized thaf a poth connot possib~y yield-the lorgest

, ' , '/

.. C'> 1. ....

metric, thn that.pa!h would be ignored' by the decoder. The decoded pot~ would , "".1 ',' 0

then b -ho;e~: From amon9 the set of rerhG:lining Or "surV'fving ll poths that reoched ~" ~ l'

"the lost level l. Such o'decoder wou Id st~e optimum in the sense thot the 1 ~

decoded poth would be the some os the decoded ROth of 0 IIbrute force" maximum

.'kelihood de~er~""1 the eorly rejeclion ~f'unlikely polh. WOu Id r.duce Ih~ decoding effort. 'Th, objective is naturo lIy to fj~d a procedure that breaks the ~

, p

exponerttiaJ message sequence-Iength dependency of the decoding effor~ ond yet . '

~ields maximum likelihoo9 decisions. We now show that the trellrs structure of

\ the code allo~sosuch ~ ~ec

e

,/

(

..

/

/ l ',J;

l,

l 33

1

this set, the path having the '?rgest accumulated metric is c,al'ed the survivor at

s"( t ) d Il h \ h . '"\... h . S (-t) 1 . state . , on a ,ot er pat s constlture t e non-survlvor set at . . t IS -1 -1

readi Iy ~that an optimum decoder con d-t$card thJ non-survivor set~f poths ',' L . withbufaltering the optimum decsion. /.' l,' . '

~ --' , Since the sub rees issud From a set of converging nnfhs are / r":,,

'

, '{

Poth metrics.

1

"

Remerging level

, /

, , ,

34

Identicb 1 subtrees.

. ........ .. -----.... ...--e

.:..__e

;-------+-------+-----~~-----+------~------- -Tree levels

~I . ...... l t -1 t+1

1

Figure (3.2) Poth metrics of remerging poths. .

, -.

35

A very proctical optimu.m decoding technique is the Viterbi

decoding aigorithm1tlVented by Viterbi (1967). This.deco~n9 procedure uses (

~tre/lis structure of the code, and is very efficient in the way the periodic , ~- ),

remerging of the poths is sysklmaticlly exploited to determine the survivors.

The optimolity of the Viterbi decoder wos demorstrated by Forney (1967) and

Omura (1969). Forney showed that for cny convoJutionaLcode, the o~tput of

the decoder is a sequence of estimated information digits whic~ is maximum lik~-

lihood conditioned upon the received sequence, and Omura showed that the '\

Viterbi olgorithm wos in foct a forword dynamic progromming technique.

(,

,~

3.3 Suboptimum Decodiog

Since the number of states increoses exponentiafly with the , .

constraint length, !JIOximum likelihood decoding becomes v~ry ropidly impractital.

t

However, -given ,the received sequence, the seorch for the moSt likely poth con be ,

attempted sequentiallYi one branch at 0 time, making tentative decisions (or \ \ .,

hYpQtheses) pn successive branches of explored poths, in such a woy that the poth

being followed is the most li ke t'y omong the subset of paths currently examined.

Whene.ver a subsequent choice indicates an earlier'incorrect decision, tflis, j

decision is properly modified, and hence at each decoding step, the cur,rent mast

likely poth is chosen among dfferent paths Iyin~ ~t different levels in the tree

(or trellis). Clearly the exploNtdpath is o'nly locolly most IikelY- .

t, p

1

> \

... '"

. '

J

36

Th is step by step procedu re is faci 1 itated by the tree (or he Il is)

structure of the ~ode, and by following only the current most likely path, unlikely

s~quences ore eliminated until 0 single sequence of length l remains; this

sequence is occepted os the decoded poth.

The elimination of on unlikely poth ba~sed on the observation of " ,

its first m bronches, m < l, is of course equivalent ,to' the elimination of every

poth beginning with,these m bronches, i.e., 0 whole subtree of length (L-m).

The earlier the rejection, the larger (exponentiol/y) the ,number of paths discarded,

re

~ .......... ------~--------~--------1

1

1

1

1

37

decoded sequence is obtained by probabilistic considerations 'rather than bya

Fixed sequence of algebraic operations. ( These two techniques are presented in the next section, whereos..

the new decoding technique that extends the M-most likely poths will be

pre.sented in the next chapter.

There exists also a number of non-probabilistic (or algebraic)

suboptimum decoding techniques for convolutional codes, for example Mossey's

threshold decoding (Massey 1963), suitable for codes having certain algebraic

(orthogonal) properties. In general these decoding techniques lead to inferior

. ,

performance compared to Viterbi or sequential decoding, and wi Il not be discussed-, \

in this thesis.

3.4 Probabilistic Decoding Algorithms

This section is a brief introduction to the two main probobilistic 1

decoding techn iques introduced in the preceding sections: ,the tree search of

sequ.entiol decoding and the trellis search of Viterbi decoding. ,

For both decoding techniques, the decoder is assumed to have a . replica of the encoder and hence is able to duplicate ail possible transmit~ed

sequences that constitute the code. Successive sequences of V channel output

symbols. caUed "received branches" are received From the channel, a,nd from

. th~se the corresponding bronch metries con be computed." Assu~ing 0 DMC" the,

metric along the poths i$ cumulative and th~refore, for both techniques, the

f

objective oF the deoder is to find the poth that yields the largest accumulated \

t

metric, given the received sequence. Viterbi (or optimum) deco~g represents

C?n exhaustive search that takes advontage of the code topolC;;gy 1 whereas

sequentlQI decoding uses an intuitive trial-and-en-or search method to reduce

the average decoding effort.

1 3.4. 1 Tree Search: Sequential Decoding

The central idea of sequentia 1 decoding is the decoding of the

received message one symbOl at a time rathe~han decoding 011 information \

symbols simultaneously as in maximum likelihood decoding. Intuitively, one'

38

could suspect that in many instances, the correct poth has a larger metric thon the

incorrect Paths diverging From it, and hence could be accurately estimoted by

considering only the succeeding Few branches. Starting From the origin of th~

tree, the poth se /ected to be- searched one step Fu rther (i. e. , extended one level '~

deeper in its qd branches) is the pcth that has the largest accumulated metric

amang those already examined. Consequently, by extending only the poth that

appears the most promising, most of the computations necessary for an optimum

decoding can be avoided. This idea is common ta various olgorithms known as

sequential decoding algorithms, the specifie method of,searching and selecting

the poth to be' extended depending on the particulor algorithm . Among the subset

of exp/ored paths, the poth that reaches the last 1evel of the tree (including the

tail) with the highest metric is accepted as the decoded pa!h.

, Jacobs ~nd Berlekamp (1967) have given two conditions which

.

39

a decoding algor7thm must satisfy in order to be a sequential decoding "0 1 gO ri thm'. '

(1) The branches of the explored part of the tree are examined ) ..

sequentia Ily, and the deci,siQn on wh ich poth to extend is based

on Iy On those paths already examined. Each new poth is thus

an extension of a previously examined one.

(2) The decode40.performs at least dne computation for each of every

examined path.

Algorithms which do not have these two properties are not considered to be,

s,equential decoding algorithms.

The interesting feature of su ch a decoding procedure is ,that the

rejection of a branch ;5 in fact the rejection of a whole 'subtree whose root node

is the end node of the rejected branch.

J As the decoding. proceeds, the number of poths in the subset of

explored paths grows, and,occasionally the decoder goes boek in the tree and , 1

extend early ~nd posSib!y incorrect paths. A fundamental idea of sequential is

to bias the metries (on which the seareh deeision5 are ba.sed) in ,5uch a way that .:

the backing-up and, extension of unlikely paths is reduced to a minimum. for a

, 1",

memoryless chonne l, the bronch metric (U)

y.-, of the branch (U) (U)

( xiT ' xi2 ' x~~) ) of a poth ~,.i'v

,

U is generally taken os

(U)

40 . ,

(U) ::: X.--1

y~U)::: L [log p ( y .. 'x.-:- ) Il ,I f( y .. ) JI

- B] (3.8)

i=l

\

where r. = (y. l' y'2' ... y,V) is the sequence f V received symbols , 1 1 1

corresponding to the transmi"tted branch X~U), and wh~re f ( i ) is the nominal -1 .

probobility of output.j for a DMC with an input probabili,ty ossignment w ( i ),

i::: 1,2, ... Q and transitiOj1 probabilities P (i/i), i::: 1,2, ... J. That is

Q f ( i )::: L w ( i ), P ( i/i ) (3.9)

i=l

the bias term B is chosen in such a way that ,_ on the 6verage 1 the branch me trics

ore positive along the correct poth and negotive olong the incorrect paths. As

the-"\'netric ossigned to eoch node on a poth is the sm of the branch metrics olong

the path leading to thot node, the accumuloted metric for node UN is ..

N

= l i=l

(U) y.-

1

, N = 1, 2, 3, (3. 10)

The metric of Eq. (3.8) is then a "tilting" of a modified form of the log-likelihood ,. .

function of Eq. (3.6). This biosirfg 'or "tilting" of the likelihood function is ...

" common to 011 sequentiol decoding algorithms, and the value of the bias which

)

rninimizes the average computat~onol effort per dec4ed dfgit is equal to the rate

'.

"

!. ~

".

41

. of the code, (Fano 1963, Bucher 1970), Clearly then, the consequence of biasing

the likelihood function is that aJ.ong the correc.t poth the accumulated metric tends to

incras, while alon9 any incorrect path it tends to decrease. Therefore, although

the objective of any sequential decoder is to firid the poth that yields th~ largest

accumulated metric, the strategy is to searc!l and follow the path of increasing

metric value.

There ore two main sequential decoding oJgorithms: the Fano algorith~(Fano 1963) and a ~arch algorithm introduced independently by Jelinek

(1969a)andZigangirov (1966). Although seemingly different, Qoth algorithms

search the tree From its root node out, trying to "match" sequentially the beginning

~egment ~i 1 i = l, 2, . '.' L of the received sequence ~l ta the corresponding

initial segments of the various paths of the tree. The Fano al

by, Fano in 1963 is a modification of the original dlgorithm present d by Wozencraft in . 1957. A detai/ed descriptTon of the algorithm may be found in ei ner Wozencraft

and Jacobs (1965) or Gallager (1968), and will not be given here. The Fano

oJgorithm is quite complex, the simplicity of the decoding principles being somewhat - "

lost in the details of the rules thot govern the motion of the decoder. It is however 1 , .

quite populor, and has been used ta derive mast of the theoretical results on

s~quential decoding. ."

, ,

...................... ~----------------------~.~

'

42

(a) The Zigangirov-Jelinek (Z-J) Algorithm

_ Th~ l-J algoritf is a very simple algorithm y.'here the es'intial

concepts of sequential decoding are ra(lily apparent. The decoder consists of a Iist , 1.

~

or stock of the olready searched paths ordered in decreasing order of their metric -- .' valves. The "top" of the stock is th poth of largest occumuloted metric among ,

-the poths,in the stock, and wi Il be seorched fu~ther (extended one lev~ further J

. in its qd branches) since 7t is the one which is the most likely to be the correct

" t

,path. As,the stac:k is reordered after eoch extension, 0 poth w,hose metric is ever-

! increasing wi Il continue te) be seorched further. Should the metric decreose al

drop from the top position, thot path will be properly stored in the stock and the

new top node wi Il be extended.

Denoting eoch explored path by the .npde of its extremity, the

stock con equivalentfy be considered as a list of nOdes ordered according to their ;'

~

metric values. The objective of the decoder is the finding of the top node, the

extension of its successors, and the proper rerdering of the stack. After initially ~

. ~ loading the stock with the origin node whose metric is token as zero, the decoding \

olgorithm consists of the foJ/owing rules:

:'(1) Compute the metrics of 011 successors of the top node and enter

_ them in th'ttr proper place in the stock.

(2) . R.emove From the stock the node whose successors were just . . . " ,inserted .

") ,

< ..

(3) If th new tt ~ode is the final node, stop. Otherwise go to (1).

, 1

Wh~n thE\aJgorithm gets out of the loop, the top ':'Iode is m' 1

end node of the decoded path. Tf,e who le- path' is then e~si1y',recovered From the ", <

, ,

information stored in the stock .

43

A decoding example for a binaJ}ree\;s illustrated in Fjg. (3.3). ,',

,

The paths are associated with their termjn~1 nodes wh'ich are ~umbered in the ,

figure. 'The, numbers Ylritten on top of the branches represent the 'corresponding

" ' \ branch metrics for som~ received sequence ~4 ~f length 4 branches. Ordering.

the nod'es From Jeft to right 1 the contents of the sto~ during decoding is then:

Stee No.

Initiafization 1

2

3

4

5

. " . ' \ ... .

. .

'Stock contMts,: node (metric) . ' 0(0)., t' .

2(-1),1(-7) , 5 ( -5 ), 6 ( -5 ), 1. ( -7 ) ~(-5),11 ,(-6),1 (-7), 12(-12)

, 11 ( -6 ), 1 ( -7 ), 13 ( -9 ), 14 ( -9 ), 12 ( - 12 ) . - o

)

6 1 ( -7 ), 13 ( -9/), 14 ( -9 ), 23 ( -10 ), 24 ( -10 ), 12 ( -12 ) - . ~

" A

7 4 ( -8 ), 13 ( -9 ), 14 ( -9 ), 23 ( - 10), 24 (- 10),: 12 ( - 12), 3 ( - 14 ) , 8

. 9

1 0 ( - 8 ), 13 ( - 9 ), 14 ( - 9 ), 23 ( - 1 0 ), 24 ( - 10 ), 9 ( - 10 ), 12(-12),3(-14)

21 ( -6 ), 13 ( -9 ), 14 ( -9 ), 23 ( -10 ), 24 ( -10 ), 9 ( -10 ), 12 (f'" 12 ), 3 ( -14 ), 22 ( -18 ) l . .

1

The decoded path iSa thu&s specified by t~e doto sequence,. terminat,ing ot nqde 21, thot,

is (0, 1, l, 0). (

a

J J 44

"

1 It ;~, 15 /~ , '- ... ., " . -7

16 Ci

" "1 .y

~

-7 ..

1 ~ 4

"11\ .- , , 19 -2

~

'" -1 0 ,- 1

0i-

J

1 . '2 21

l

-10 22 .,.

, )} -4

./

11 -1 '~ \

\ -4 24 .. -4'

~

25 ''''''7-. . . t 26 1,1-:-

~ t 1 ~ -1 ~ ,,," 27

"'4 "

28 -4

1 ->-4

'fi 1

~ "

45 1

" WHO. the keJp .of this exal'nple ,1) the Z-J 'afgorithm n'loy be . \.,

rgorded as th most natural way ta extend thct explored path having the largest

biased metric, By olways 'extending the most fikely'poth omongthose ,a 1 reody ,

" explored (~hot is, stor~d ~n the. st?) 1 t~e ~Igorithm ~x~mizes' the -probabii~y

that the next step will be token alorlg the correct poth in the encoded 'tr~, ~ " .. ~~~-~

The m~~han ics of the seorh 'ore quite s~mple in principle. After . . 1

" d .. the'top node\is eliminoted, th- q new successors are inserted in the stock ot 1 _

th place ossigned by their totQI metr}os .. The sto~k grows by (q d -1 ) entries ~o~h d~codin~ step,' so thot of~er, i steps, it contoins (1 +j ( q d -1 )) paths

of vorious le~gths terminoting in different nodes in the tree. Defin~ng a compu-

tation as the exe'cution of step l of the algorthm, the toto 1 number qf computotiorRl ,()

. ~ , .to decode a binary tree is equol to (W-1) where W is the size of the stock when

" , dec~din9 stops.

, , . .

J\ groph of~e metric values of the poths called the "received \

) '- p

value tree U may be constructed as in Fig, {3.4) for the example of Fig. (3.3). At , '

the 4th levet' obvilsly' the decOd~ poth ~os ~e highest value. However, t~i~ is . not th~ ca~e for any"'other interrnediate no,de lev~1 and by_ the rul~~ of the olgorlthm,

011 poths whose metric is higher thon the correct path metric must be ev-entuolly . ~ 1

extended. A typical example of th~ plot of the ~orrect poth wet,t.lc is shown in

, 1

Fig. (3.5), A span of bod noise causes the metric to.dro~, and therefore some

incorrect path~ may occupy the top of the stack and be extended. OI~r1y, before o

the~ correct.path reoches the top'of the stock and is e~ended anew (henc-e hos

.-

"

-6

-8

-JO , / / ! 1

/-12 1

1

1

j f 14 ! -

1

/

'r .

:\' Metric valu.es ai the explored paths. Figu~(3.~)

.~/;'; 'L, Cor~:; ~~, i~

T', min

Incorrect poth, ,

.J, __________ -_

" / tSOd ~i~ \ \

, " ,

\ \ ,

~

"

\ \

... , t , , , ,

~'~~ , ,

Figure (3,5) \T.)1)cal correct and jncor;e~{~ths metria.

(

. . / . .

( ,1'\

~

'"

----------------~~~--~-~-

)

.)

1

, . ./

/

) 47

,possed ~e region of ~d noise), ail incorrect paths with a metric hj~eI thon the smal/est value of the correct path metric must have been extended. Th is .obser-

vation i5 fundamental ta understc:;"ding the computational behaviour of sequential

decoding.' ft is the occasional drop in the correot path metric and the conco~itant

extension of inco;rect paths that is responstble for the ron~om nature of the decoding

, effort. ! '

(b) Quantized Z-J Algorithm

The simplicity of the Z-J a/gorithm is paid for by requiring a

large memory for storing ait the information about the exp/ored paths. However, L 1,_ '

the rea/ ptob/em is keeping the stock exoctly ordered. An exact ordering of the

nodes is 50 time consuming thot the algorithm becomes practieally worth/esse To 'p

overcome ~is difficulty; Jelinek (19690) pro'posed a quantized version of the

algorithm in which the nodes ore p/aced at random into substacks (aise ca/led

bins) accorQing to their metric va~ues. In each substack are stored ail l'hose nodes Q

. ~hose ~tric value ~ie5 with;n a ce!1.tfift range. ThOt is, anOde m of metric r m

is inserted at random into substack ci if

c

Q H $ rm

.. ,..

\

48

. . #

The node which is to be extended is thn taken ot rondo';; From this top substack,

usuolly in the lost-in first-out mode. Jhis quantized version of the l-J algorithm

is sometimes called the Jelinek algorithrn and the rules becorne:

(1 )

(2)

(3)

Compute the metric$ of the successors of any node from the r~ ,

highesfn-emPt~ substack and enter them in. their proper

s~ck.. 1 ~emove f1m+he stock the node whose successors were just

Inseoetr:'

If any node in the highest non-emplty substack is the final node,

stop. Otherwisegoto (1).

Under this quantized form, the Z-J algorithm becomes practicat ..

and competitive wah the ~Ider Fano ~Igorithm. In a compprative study of the two . , algorithms,' Geist (1970) has show ... that although quite different, both algorithms

essentiafly folfow the some rules of poth extension. Considering the decoding time

as a measure ~f the decoding effort, Geist observed by simulation that foncoses of

low noise, the)no decoder perfo{"" better thon the S~J,.it.coder, but the advantage goes rapidly to the stock decoderas the l')oise increases. Hence the

stack decq-r would be superior than the Fano decoder for periods of high chann~f . "- . noise and vic-verso for periods of calm channel noise. The choice of the

../

"r 1 partiular decoder will df!IJpend on a trade-off between storage; speed o:nd sofOtwore

soph istication. -/

l , --~----

49

(c) The Computational Problem of Sequential Decoding

Regardless of the algorithm used, sequential decoding involves a

random motion into the tree, where the exact moves of the decoder are determined . -by the re~j"ed seque~ce and the particular algorithm. Since any decoder move

implies a computation, the num~rr of computgtions to decode a given block of

~-' , information symbols is a random variable. Conseqtlently, the analysis of sequential

~o decoding is not only concemed wJth the error probability but al50 with the distribution

of the computationaf effort. This variability of the computation is one of the draw-

bocks of sequential decoding .

A combination of asymptotic resulls by Jacobs and Berlekamp (1967) 1

Savage (1965), Falconer (1967), and Jelinek (1969b) ~hows that the number of

computations required to .decode one information symbol for any sequential

decoding algorithm has a Pareto distribution (under the assumption of an infin ite

constraint length code). Thus ignoring the converge'nces of paths,

P (C ~ N) 1=:1 k1

N -0

(3. 12)

yr large N. k1

i$ a constant. The exponent a is cal/ed the Pareto exponent

and is giveQ by.the porametric equation

tE (a) o

R=---a

(3.13)

whttre R is the information rote of the code and E~ ( a) is the Gal/oger. fU'hctic:>n

(Gallager 1968) deFined os

... "

.'

50

(3. 14)

for a discrete memoryless channel with Q inputs having a distribution L) - -'

fw (i ) , i = 1, 2, ... Q}, J outputs, and transition probabilities ! ( Vi ).

E (a) is a concave monoton ic non-decreasing function of a "and is determined \ .0 .1 by the channel statistics. This fU!1ction has the properties that

E, (0)=0 ()

E (1) A R o ,. comp 1

d (E (a o

lim da

a .... 0 =c

(3. 15)

Where C is the channel capacity 1 and the rate R = E (1) is called the comp 0

,

computationai ,~ut-off ~te of sequential decoding. FOr a discrete memoryless , ,

1 channel R

, ...

..

51

The theoretical values of the exponent 0 and R/R moy be comp

obtained by a graphical construction On the graph of Eo (0) as shown i,:" Fig. (3.6).

Observing that E (a) is monotone non-decreasing, then o

E (0) ~ R comp'

o ~ 1 " 0 , (3. 16)

and from Eq. (3. 13) we obtain

R o ~ com~ for R s: R

R comp (3. 17)

Simi larly for o s:

R o ~ comE for R ~ R

R comp (3. 18)

"

Observe that a larger Pareto-exponent wi" moke the tai! of

the ,distribution of the computation'" ~ C ~ N) dec!ease more rapidly. This (s

a desirable situation which for a given MC can be obtained only by decreasing

the in'tormotion rate of the code.

Many experimentaf 'fhvestigat'ons '(Jordan & 8lu~stein 1'963,

Niessen 1965) confirm the Paretean nature of the distribution of the computation

f

for the coherent and incoherent di1rete memotyle channel (Ha.ccoun 1966, , 1

Helier 1967), and the fading :Iispersive ch'-nel

1

. , ..

E (p) o

R(a) comp

R(a)

E (p) o

R (a)

R(a) comp

"/ /

/ /,

/

,/ /

. 'O.

- - --jf 1 /1 1 / 1 1"'/

---A / 1

/ 1 /

R

;'

/

comp a R ..

------------------ -If /1

/ 1 / J ----------------. ---

1 1 1

1 1 ~.

Z

- ~ ,

'1 /

R cony R

, , . 1

a > 1

a

'-

53

with the penetration into the subset, the number of required computations will also \

~ ,

grow exponentially with the length of the correct poth metric dip. How~ver, on

the discrete mernoryless channel, the interval of high noise thot caused the cOrrect

poth metric to dip, occurs with a probabi lit y that is exponentially decreasing with

- its length. The combined effect of these two exponential behaviours results in a

distribution which decreases at rnost algebraicolly.

. Another unpleasant consequence of the variabi 1 itr of, the comi

tation is that incoming dota may l')ot be always immed'iately processed as it arr.ives,

and buffer storage must be provided. Regardless of the size of the buffer, there is

a non-zero probabi 1 ity that it may fi II-up, leading to on overflow and complete

communication breakdown. This overflow problem is the most serious problem with \.

\ sequentiaJ/decoding on the discrete memoryless channel. The probability of over-

flow has been bounded by Savage (1965) os

(3. 19)

where S i~ the decoder speed factor in computations per informatipn digit inter-

arrivai time, B is the size of the buffer and a is the Paret~ exponent. -The

probobility"of overfJow is rather difficult to combat because of its relative

insensitivity to buffer size and defoder speed.

To overcome this major diffieulty, dota s sent in blocks, usuolly

in thel order of 500 to 1000 branches, and known sequences (the toil of thl

messag.) cleor the shift register of the convoJutional encoder of ter each block.

"

\ \

1 ! i

54 \

~

ln case of an overflow, the entire block is lost, but the buffer is cleared and

decoding can be re~umed in the following block.

The sequential decoder is soid to have made an error if it makes

an incorrect decision On an information bit and never returns to correct it. The

error probability in sequential decoding decreases exponentially with the constraint

length of the code and hence can be made arbitrarily sma". Sequential decoding

being a subopti,mal procedure, a true lower bound on its error probability is

obviously the lower bound on the errOr probabi lit y for'the optimal decoding o~

Viterbi '(1967). Asymptotically the bound is

E (a) o

-K E (a )/R o

e 1 R""" P ( E ) > k2 a (3.20)

where k2

i5 a constant, ,K the constraint length of the code, R the rate and

E (a) the Gallager function of Eq. (3.14). o

Long constraint lengths present no problem in actual sequential

decoders bec ouse the coder complexity varies only tinearly with the constraint

len9th and the prObabilit~verflow is inoensitive ta it. Th~refore, t'I.e real problems with sequential decoding are the variability of the computational effort

and the buffer overflow.

These difficulties have somewhat been alleviated by a hybrid

scherne due to Falconer (1967), where instead of using one sequential decoder,

a mixture of several sequential decoders working in parallel and an algebraic

block decoder are used. The system 'works in such a way that periodically the / /

1

\ 1

55

algebraic decoder helps those sequential decoders involved in long searches by

putting them in the right poth. Compored fo straight sequential decoding" the

computational variability of Falconer's scheme is reduced, but the distribution

of the computation remains osymptoticolly of a Pareto-type. The increosed'

complexify of the hybdd scheme has so far prevented ifs implementation in

proctice.

A new method to reduce the computotional voriobility is to allow

the decoder to ex tend sever~1 poths simultoneously, and by using the trellis of

the code exploit the convergence of the pcths and eliminate unnecessary

compufptions and storage. This method which will be presented i~ the next

chapter, involves only a slight modification of the Z-J algorithm, and in

'addition mak~s sequential decoding att+ctive for short constrainf' length codes.

3.4.2 Trellis Search: Viterbi Decoding

The Viterbi decoding algorithm (~i~erbi 1967) is a slmple decoling

procedure, which delivers a sequence of estimated ilformation digits, maximum-

~~ 1 < likelihood conditioned upon the received sequence. In contrast with sequential

decodini' the Viterbi de~oded sequence corresponds to the poth hoving the lorgest

occumulated metrie of:e.!..! possible distinct paths.

We recal! From section 3.2 thpt of 011 the paths- remerging ot a

given node, only the survivor at thaf no de needs to be retoined by the decoder,

~nd consequently the trellis is the structure to use for decoding. At ony trellis

\

depth (or level) t furtherthanoneconstraint length, thesurvi,vorforeach of the

qK-d distinct states must be determi~ed, which implies that ~ distinct paths of

length t branches must have been examined, Since none of the possibly volid

paths are discarded, there is no need to bias the liklihood function, and hence

the metrie will be the log-li.kelihood function of Eq; (3.6).

56

ln the trellis structure, anode is entirely specified by its stote

and its level. Thot is, s~t) 'specifies- anode at level t, t= 1, 2, ... , having -1

~, . 'state S., i = l, 2, .. , ttK- d . Given the survivor for each node cff some level t,

-1

the survivor at any ,node ~~t+l) is eosily obtained by considering only those- qd

l-branch ~xtensions ~,merging from th~ qd survivors ot level t) that merge ('{'+1) . (t+l) . .

onto node S. . Consequently, the survlvor at node S. IS the extension of -1 - -1 .

that survivor which yields the largest accumulated metric at node s~t+l). For -\

exomple in Fig. (3.7), the sl,Irvivor ot ~~t+)) 'is the extension of the sur~r;~r --,

at node S(t) and not ot the survivors ot either Sl(,{,) Or S (.t) l" -3 - -0

~--

ln case none of the extensions of a survivor yi~td a su~Jvivor ot the

next le~el, then cleorly thot svrvivor connot become port of the decoded path and

hence may be discorded. This situation is illustroted in Fig. (3.-7), where none of

. (t) (t+l) , the successors of the survlvors ot nodes ~1 and ~3 were chosen, and hence

.. these survivors are etiminoted. Summorizing we see thot a surv'iving path ot ~ny

level is alwoys a surviving poth ot:all of its precedin~ levels, but it, 1 not necessorily o part of 0 surviving path at any following level. rThis observation is at the heart of

the Viterbi decoding olgorithm which 'can be regorded os a set of rules for obtoining ,?

the survivor ot eoch stote of every trellis level. -1

1

S -0 ('\ ,.

~1

~5

.. ,

r=15 T"=19

r=19

t +1 t+2'

The heavy lines represent the surviving paths.

Figure (3.7) Surviving and non-surviving paths in the tre"is. ~

-,

57

_ 0

, -

"

o

1

(a) The Viterbi De'~oding lgorithm p" ~ 0 " \

The Viterbi decoding algorithm operates os follows where' we may\ , ,

assume d = 1 without loss of generality. Given the first (K-1) branches of the ~ ~ J 1

received sequence, ~K-1 (= 11' 2::2' ... 1K-l ), starf by e~amining, fro~ the \_

origin out, 011 qK-1 paths of length (K-'l) ~ro.nGhes and compute their total me~ics

r (i) K71 ' K -1 ' i = 1, 2, ... q Since no merging takes place for the first (K: 1 ) con-

secutive branchesi9f the, tree, those q K-1 poths are distinct and identical f?r bath

\ ,

-59

J

factor"of q, resulting in a constant nu-mber of survivors. From our ea~lier

. ~

1

.',

O( . ~\

.) 0 ( ..

, ~ be performed at every state, the cOl1'lplexity of the decoder is proportional to the

number of ~tates, hence groQws exponentiolly .with the constllOint length. To

decode a block of L information symbo'ls the total number of Vitethi op~rations

/ 60

... . ... , 1

K-l ~ 1\....5L L 15 L q ,and for L >;>- K l.i' is far smoller ,thcr the tot~umber ~ 1 (q -1 )

. ',. ..l.. tf of operations that would b~ required to perir;;'~ ma~"),um l''kelihood decoding

on a tree. However, as ci practical decoc{,ing.teohnique, clearly the eXpOnential ~ ~ ... 1) )

, , . growth of the number of ~tates with K will lirn,it Vherbi deco~ing to convolutional

... codes of short constraint lengths (K ~ 7 ). . r

, "_1 . .,.

>

.. t, 1 ) ~. J

The probability of error fur the optimal decodjng of a convolutional

code of rate R over a memoryless channe! has been boutrd~ Viterbi (1967) as 1) \

'\

\ 'where L

\

..,

/.7 P(E)~ l(q q e-NE(R) / ) -E/tR

,: l-q 7 1

1 " j

rrllle number of inforrhation symbofs thot ore encode', ~ "

N = KV is' , 0 \'

the symbol constraif,'lt lengfh of the COde Glnd where \

O

Using computer searcn techniques, good short constraint length

codes of rates 1/2 and 1/3 have been ob(ained by Odenwalder (1970) for .

K s: 9, and excelle~t decoding performance 'w'jth these codes was reported~Heller & Jacobs, 1971). More recent/y Larsen (1973) extended the list of good codes

1\ ~

~ith constraint lengths up to K = 14 and ;ates 1/2, 1/3 and 1/4 .

. Iifications of the Algorithm

. , The great odvontage.of the Viterbi decoder is that the number of

decoding op~rations performed per received branch is constant.. These operations

-ore always of the sorne nature and do not require a sophisticoted, logie. The main

.. 'disadva~s the huge memory necssary to ~tore ail the paths. Moreover,

forcing the encoder into a known final state by using Cl toi! of o,known sequence

of symbo fs !s equivalent to waiting until al/ L information symbols have been ,:

,. D

decoded before starting the delivery of the, fil'St syml;>ol ta the user. In a real , -

... system th t~ delay 'may be operationa!ly undeshab/e.

, ft is observed ~t ail the survl~in~ ~ths do not stay distinct over

61

{ll ~. , . ' -

their entfre length but ha\l"e a tend~cy te stemJrom a single node s~veral constraint i "

fengths ebrlier (Hell~~CobS, 19711. For/example in Fig. ,(3.7) te survivors at , . (t +2). ( t + ( .( +2) ( t) . . nodes ~ , $1 c' ~4' merge together at node ~3 . Supposmg that , . . ail the suro,rs ot the present ~eeoding depth n merge togethe.r ot ~me node

~

S~n-M) JYin9,~Vel~ back~, M"> K, the'; ;!gordless of whch poth ;5 'fi~Qlly -,1. _

'h's~n as the ~odec:f path,' 'the survi~in9 poth ~eadin9 to node S~n-M) mu~t be ~ -1

(

..

................... ------------~~-----,----

)

62

. ,J--a part of it. Hence, it is not necessary te wait until the unique decoded path has

been obtained before starting to deliver the decoded symbo/s. A final decision may be made on these branches prior to the point of common convergence, ahd the

decoder con safely deliver ,the information (with a small delay corresponding to

M) as i t progresses in the tre"is. With th is procedure, clearly the decoder con , ~

eliminote the toi 1 altogether.

Ge~tin~ rid of the tail in this way leads to a v,ery attractive situation

with respect to the memory requirement. C/early instead of storing the entire,pa)s

hiS\ry for the total length l, the decoder needs to s!ore only the history of the

poth~. to and inc lding the point of comman convergence. If the point of .... common conv~e lies M leve Is bock (M is ca lied the ~mory of the decoder),

then the total amount W of path storoge required is

(3.23)

A practical decoder hos a given amount of storage ovailable, 50

that in fact M becomes fixed. A \practical refinement of the above method of

symbol decision consists in deciding on the oldest symbol of the most likely of the

q K -1 reta ined poths. It hos been shown theoreticolly by Forney (1967) th'il t with J , ,

this procedure, asymptotically the expected value of the~vror probability on that ~

oldest symbol is not increosed if the memory M is large enogh. It has been found , , '

through simulation (Odenwold~r 1970, Hel./er & Jac6bs 1971) thot a memory of . . ~

4, Or 5 constraint lengths is sufficient to insure a negligible degradotion from the

error probab il ity of the optimum dec6der. Should an error o~cur by this method it

...

....

\.

, \

63

does not I~od to catostrophy, for although- the Viterbi decoder was assumed to start

operating from a known starting,state, it has been found through simulation (Helle

& Jacobs 197"1) that if mey in fact start deeoding From any state. The fi rst 3 or 4

constraint lengths of data symbols may be unreliable but norrnatopefations will

. prevarJ th~reofter. Therefore, for a practical decoder the tail con be eliminated

and the memory reduced without altering the performmce of 0 (theoretical) optimum

dQcoder.

Fina/ly a quontized integer metrie may be used instead of the

, exact log-like/ihood function of Eq. (3.6) without much difference in error

performanq,e: the Viterbi decoder is relatively insensitive to metric quantization.

3.4.3 Comparisons and Limitatiol'ls of Sequential and Viterbi Decoding ,

"

'.

64

distribution of the number of computations to decode one branch prevai 15 regardless

of the rate (Falconer 1967). Stoorage requirements and decoding speed must be

traded in suc'h a way thet a decrease in buffer storage, must be compenseted by

an increase of the decoding speed over the bit rate, thu5 limiting the maximum 0

bit rate capability. Below R , if the number of computations that the decoder comp .

con perform per unit time is greater thon the mean value of the corresponding

. Pareto distribution of computations, then on the average the decoder con keep-up

with the data, although buffering is necessary. Regardless of the buffer storage "

provided and even for rates below R , occasional long searches do occur . camp ,

~

resulting in a pos~j'ble buffer overftow and the consequent erasing of long sequ~nces.

For ;equential decoders using long constraint lengths the probabi 1 ity of undetected , ,

errer is indeed very small, and usually the main contribution of errors cornes From

the buffer overflow.

For the Viterbi decoder such a situation obviously does not occur

due to the fixed nature of the computationol effort. Storage history and decoding

speed requirements may be definitely set for a pcirticular application.

, Since both the number of op~rations per decoded branch and the

total storage are prolfrtional to the l'}umber of s~ates, the complexity of the Viterbi

decoder is' Pfoportional to the number of states, and therefore it increases expmentio lIy ~

with the constraint length of the code. A,.lthough the errer probability decreoses

exponentially with the constraint Jength, clearlyaoy improvement on' the error

performance by increasing K, is expnsive in terms of computationol effort and

- -- - - - -- ~--------------

65

storage. Therefore., Viterhi decoders are limited to small (K 's 7) constrajnt ,

length codes, and consequently where very moderate error probability is sufficient.

J However, the very simple nature of a Viterbi operation and its identical repetition

, 4

from level to level permits a porallel implementation of a decoder and hence

decoding at very h igh data rotes in the tens of Megabits/sec ronge.

')

Decoding delays are inescepoble for both sequentia', and Vit~rbi ,

decoders. For sequential decoders the deloy is a direct consequence of the 1

variability of the computational effort, and for Viterbi decodrs it corresponds

to the poth memory storage of each state. Sequentiel decoders tend to have ,<

buffers longer thon the poth memory storage of Viter!;>i decoders. However, in a

real-time situation where a fixed time deloy is required for putting ouf data, the

total number of computations required in one delay g~s only lineady with ~he

constraint length K for sequentio 1 decodrs wherees it grows exppnentially with

K for Vi terbi decoders.

ln generel, perforrTlQnce end complexity combine in such a way

that sequential decoding is chosen over Viterbi decoding ;"hen high performance

;s required (P.( E ) s: 10-7 ), provided the information rate is sufficiently low that

operation below R is assured. M,?reover 1 since sequential decode'rs will be comp

extrerhely sensitive to bursty noise patterns, they wi" usually be restricted to

weil beheved rflemoryJess channels suc~ as the $pace cha~~el.

" , _ Viterbi decoders mey be preferred when.the performance demanded is more modest but Jhe dota rate very high. The crossover point occurs for bit error . .

. " -\

J

- ---~ - ~-------------------

~....... #

\ \

p~bability in the' 10-3 to 10.-5 range and constraint length between 6 1 .....

aod 8. "'

Most of the drawbacks of sequential decoding are th~ conse- currently

\ ' \'

most likely paths.

.,

\ \ "

1(

~

c l~ /

l \

\"\ \ )

1 .. ..-:

/'

\

J

"

'\

67

CHAPTER IV

MUlTIPl{ PATH STACK DECODING AlGORITHM

4.1 1 nt-roduct ion: Generalized Stock A/gorithm 1

The two a/gorithms presented in the preceding chapter appear to b~e

the extremes for determining the most likely transmitted sequence of a convolutionol

code, given the received sequence. The Viterbi algoritMm examines ~ distinct

paths ot every level while a sequentiel decoding olgorithm follows only that poth

'" thot appears to be the most likely. The step by step procedure of sequential decoding ,tends to reduce to a minimum the average value of the number of computations to

decode one information digit. Unfortunate/)'l( the procedure mokes this computational

effort variable with 0 Pareto distribution. By comporison, a Viterbi decoder carries

~Iong the one correct rth and ail incorrect poths of the trellis. As a consequence

'the computationol eff1rt becomes rather large but remains constant for the enti re decoding.

<

We now tjJm our attention to improving the computotionol behd'viour. j .~~ J ' - ,

of sequential decodin~ by reducing the yoriability of the" cdmputational effort without

degreding the efr?r performance. Th is variabi lit Y wi" be reduced when the distribution

function P ( C ~ t) is ltself reduced, mostly when t becomes relatively large. '-

The fundomentol idea is to keep the search proper~ies of sequentiel decoding and

the notion of a stock while using sorne cf the features of Viterbi decoding.

1

Denoting each explored poth by the node of its extremity, we recall ,." 1

tht a stock can be considered as a 1 tst 9f nodes ordere~ (in decreasing order)

'.

1

\ ,

;

68

according to their metric values. Allowing for the extension of sJt.eral paths From

the stock we now generalize the concepts M the stock algorithm of Zigangirov , /

, 1

and Jelinek.

Let a poth extension cycle consist of omputing the mefrics of the

successors of the extended poths, entering them in their proper pliee in the stock,

, and removing from the stac~ the nodes just extended. In a poth extension cycle,

the extension of the Jth node of the stock, J ~ 1, implies th~simultaneous

extension of a Il nodes stored h igher in the stac k. In other words if J paths are --; ,

extended, they ore the J most likely explored poths. The actual number of

extended -paths in 0 poth extension cycle is assumed determined by 0 decision

rule which is specifid in the algorithm.

Following the poth exten~'ion cycle a purging cycle eliminates from 1

/ -the stock some unwonted nodes and r rders the stock. The purging cycle is

directed by sorne pUfging rule 0150 s ecified by the user. The decision ond purging

rule; need not be kept fixed over e entire decoding, but" may be vori~ according

to any information pertinent ta the decoding as it progresses (post stpck contents,

metric behoviours, channel measurements, etc.).

The set of operations comprising a path extension cycle and a purging

cycle is defined as a decoding cycle. An ~Iso'rithm that uses this decoding cycle . at every step is called a generolized stock algorithm.

If is now easy to show that bWh the 'Z-J _cOO

&

e.

Il

\

l'

69

ln the Z-J algorithm ~e decoding eye le is degenerate since it eontains no purging

cycle. The decision rule is th~simplest possible as it directs the extension of only

the top node of the stock for eoch path extension cycle. (The quontized form of

the l-J algorithm introduces only ~~proctical, detail in the poth extension cyc,le).

Now: suppose the Viterbi decod~r uses a stack to store and order ail

explored paths. For a binary code the decision rule is.simply to extend the"

M = 2K- 1 ,v

(4.1)

, highest metric paths (the survivors at any depth) in the stack at each poth extension

cycle, whe'reas the purging rule is recognized as the elimination of ail non-surviving

paths. Ali redundancy is eliminated from the stock whose size remoins constant. , l

The M -path extension together with the purging fuIe c1early guarantees the v ..