struggle ! two practical classes of channel (error- correcting) codes cyclic codes...

struggle !

two practical classes of channel (error- correcting) codescyclic codes （巡回符号）

a subclass of linear codeslinear-time encoding and error detection (not correction)

convolutional codes （畳み込み符号）different principle from linear codessoft-decision optimal decoding is available

both codes widely used todayexpected to be replaced by “next-generation” codes

1

what’s wrong with linear codes?

Textbooks say thatlinear codes are a practical class of channel codes,but they are not fully practical ...

not scalable complexity for encoding/decoding,

due to matrix computations

not easy to constructgenius idea or mathematical talent is needed

2

channel codes evolution

3

general channel (error-correcting) codes

linear codes(linear block codes) convolutional codes

cyclic codes

LDPC codes turbo codes

BCHReed-Solomon

HammingGolay

more structure ...

more efficiency...

more power...

more power... more power...

next class next class

this class(& next class)

cyclic codes

Linear codes are characterized by matrix operations.Encoders/decoders are realized by combinatorial circuits.

simple but not scalable ... for the code length

cyclic codesa subclass of linear codesEncoders/decoders are realized by shift registers.

slightly complicated but scalable ... complexity

4

codes linearcodes

cyclic codes

preliminary (1)

Binary vectors can be written as binary polynomials （多項式） .

5

11101 x4 + x3 + x2 + x + 1

addition (=subtraction) ⇒ XOR

x4 + x3 + x2 + 1 x3 + x + 1+)

x4 + x2 + x

111010101110110

+)

multiplication ⇒ (unnamed operation)

x4 + x3 + x2 + 1 x3 + x + 1×)x4 + x3 + x2 + 1

x5 + x4 + x3 + xx7 + x6 + x5 + x3

x7 + x6 + x3 + x2+ x + 1 11001111

1110101011×)11101

1110111101

multiplication with xm

= left-shift of m bits

preliminary (2)

6

division (unnamed operation)⇒

x6 + x4x4 + x3 + x2 + 1 )x2 + x + 1

x6 + x5 + x4 + x2

x5 + x2

x5 + x4 + x3 + xx4 + x3 + x2 + xx4 + x3 + x2 + 1

x + 1

11101 ) 101000011101

1001011101

1111011101

11

111

the division circuit is easily implemented by a shift register

1. store p(x) to the registers2. if MSB = 1, then AND gates are activated,

and registers are XOR’ed with q(x)3. left-shift, and go to step 1

division circuit (1)

divide p(x) = x6 + x4 by q(x) = x4 + x3 + x2 + 1:

7

1 1 1 0 1

1 0 1 0 0 0 0

q(x)

p(x)

quotient( 商 ) remainder ( 剰余 )

𝑝 (𝑥 ) ÷𝑞 (𝑥)M

SBdivisor

dividend

division circuit (2)

8

0 1 0 0 1

1 1 1 0 1

0 0

11101 ) 101000011101010010

111011111011101

11

111

1 0 1 0 0

1 1 1 0 1

0 0

one-cycle of operation= one-step of the calculuscontents of the register= remainder of the division complexity

definition of cyclic codes

How to construct an cyclic code: bit codeword, information bits, parity bits

Step 1: choose an degree-m polynomial G(x) that divides xn + 1.Step 2: code C = {multiples of G(x) with degree < n}

Example: n = 7, m = 4:choose G(x) = x4 + x3 + x2 + 1,because x7+1 = (x3 + x2 + 1) (x4 + x3 + x2 + 1).

G(x)...generator polynomial

9

1000000111101 )

111011101001

11101

1110111101

0

1101

G(x)x7+1

construction of a cyclic code

Step 1: choose an degree-m polynomial G(x) that divides xn + 1.Step 2: C = {multiples of G(x) with degree < n}

n = 7, m = 4: G(x) = x4 + x3 + x2 + 1

10

G(x)(x2+x+0)×

C=

00000000011101011101001001111110100110100110011101010011

G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×G(x)(x2+x+1)×

=

properties of cyclic codes (1)

Anyway, we have defined a set of vectors, but is it a linear code?

Lemma: a cyclic code is a linear code

proof: Show that c1+c2C for any c1, c2C;

c1 C c1 = f1(x)G(x)

c2 C c2 = f2(x)G(x)

c1+c2 = (f1(x)+f2(x)) G(x) C

11

properties of cyclic codes (2)

Lemma: if (an-1, an-2, ..., a0)C, then (an-2, ..., a0, an-1)C

proof:Let W(x) = an-1xn-1 + ... + a0 and W’(x) = an-2xn-1 + ... + a0x + an-1.

W(x) is a multiple of G(x), because (an-1, an-2, ..., a0)C

W’(x) = an-2xn-1 + ... + a0x + an-1

= an-1xn + an-2xn-1 + ... + a0x + an-1+ an-1xn

= xW(x) + an-1(xn + 1)

12

multiple of G(x) multiple of G(x) ... construction step 1

W’(x) is a multiple of G(x), and (an-2, ..., a0, an-1)C.

A cyclic code C is closed for a cyclic shift.

three approaches for encoding

three approaches for an encoding procedure:matrix approach

use a generator matrix ... no advantage of cyclic codesmultiplication approach

codeword = (information bits) × G(x)the code not systematic (cf. p.10)

division approachslightly complicated (for me)make the code systematiceasily implemented by shift registers

13

3-step encoding by division

1. A(x) = polynomial of information bits2. B(x) = remainder of A(x)xm divided by G(x)3. codeword = A(x)xm + B(x)

encode 011, with n = 7, k = 3, m = 4, G(x) = x4 + x3 + x2 + 1 1. A(x) = x + 12. A(x)x4 = x5 + x4 = x(x4 + x3 + x2 + 1) + (x3 + x), and B(x) = x3 + x3. A(x)x4 + B(x)= x5 + x4 + x3 + x, the codeword = 0111010

14

A(x)

(not necessary)G(x)

B(x)

x4

÷divisor

dividend

quotient

remainder

W(x) = A(x)x4 + B(x)

multiply,divide,

and add

did we really make “encoding”?

simple question: Is A(x)xm + B(x) really a codeword? Is A(x)xm + B(x) divided by

G(x)?

Yes, note that...B(x) is a remainder of A(x)xm

in the binary world, A(x)xm + B(x) = A(x)xm – B(x)“A(x)xm – B(x)” = “removing the remainder”

if A(x)xm – B(x) is divided by G(x), then there is no remainder...

15

example

n = 7, k = 3, m = 4, G(x) = x4 + x3 + x2 + 1

16

data000001010011100101110111

A(x)x2+x+0x2+x+1x2+x+1x2+x+1x2+x+1x2+x+1x2+x+1x2+x+1

A(x)x4

00000000010000010000001100001000000101000011000001110000

B(x)x3+x2+x+0x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1x3+x2+x+1

A(x)xm + B(x) 00000000011101010011101110101001110101001111010011110100

systematic codeencoder ≈ division circuit O(n) < O(n2) of matrix operation

error “detection” of cyclic codes

error “detection” is easy for cyclic codesu C u (in a polynomial representation) is divided by G(x)

17

G(x)

= 0 ... no error 0 ... error÷divisor

dividend

quotient

remainderreceived u

no need of parity check matrix, division circuit sufficesencoding and error detection can share one division circuit

reduces the cost of realizationCyclic Redundancy Check (CRC)

... used in many communication systems

error “correction” of cyclic codes

general algorithm for all cyclic codeserror-trapping decoder [Kasami 61]

18

Tadao Kasami1930-2007

E. Berlekamp,1940-

J. L. Massey,1934-2013

I. Reed, 1923-2012 (left)andG. Solomon,1930-1996

special algorithms for special cyclic codesBerlekamp-Massey algorithm for...

BCH (Bose-Chaudhuri-Hocquenghem) codesReed-Solomon Codes

intermission

and

convolutional code

19

the channel and modulation

We have considered “digital channels”.At the physical-layer,

almost all channels are continuous.digital channel=modulator + continuous channel + demodulator

20

modulator( 変調器 )

demodulator( 復調器 )

a naive demodulator translates the waveform to 0 or 1 we are losing something...

0010 0010

0010 0010

continuous(analogue)

“more informative” demodulator

From the viewpoint of error correction, the waveform contains more information than the binary output of the demodulator.

demodulators with multi-level output can help error correction.

21

0

1

definitely 0maybe 0

definitely 1maybe 1

to make use of this multi-level demodulator,the decoding algorithm must be able to handle multi-level inputs

hard-decision vs. soft-decision

hard-decision decodingthe input to the decoder is binary (0 or 1)decoding algorithms discussed so far are hard-decision type

soft-decision decodingthe input to the decoder can have three or more levelsthe “check matrix and syndrome” approach does not workthe use of polynomials is not obvious

complicated, but should have more power

22

soft-decision decoding as optimization problem

outputs of the demodulator0+ (definitely 0), 0- (maybe 0), 1- (maybe 1), 1+ (definitely 1)

code C = {00000, 01011, 10101, 11110}for a received vector 0- 0+ 1+ 0- 1-,

find a codeword which minimizes the penalty .

23

received0+

0-

1-

1+

penalty of “0”

0123

penalty of “1”

3210

(hard-decision ... penalty = Hamming distance)

0- 0+ 1+ 0- 1-

0 0 0 0 01 0 3 1 2 7

r

c0+ + + + =

1 0 1 0 12 0 0 1 1 4+ + + + =

c2

=

=

smaller penalty,more likely

algorithms for the soft-decision decoding

We just formalized the problem... how can we solve it?by exhaustive search?

... not practical for codes with many codewordsby matrix operation?

... need to solve an integer programming, NP-hardby approximation?

... yes, this is one practical approach anyway...

design a special codefor which soft-decision decoding is not “too difficult”

convolutional code ( 畳み込み符号 )

24

convolutional codes

the codes we studied so far... block codesa block of k-bit data is encoded to a codeword of length nthe encoding is done independently for block to block

convolutional codesencoding is done in a bit-by-bit mannerprevious inputs are stored in shift-registers in the encoder, and affects future encoding

25

input data

combinatorial logicencoder outputs

encoding of a convolutional code

at the beginning, the contents of registers are all 0when a data bit is given, the encoder outputs several bits,

and the contents of registers are shifted by one-bitafter encoding, give 0’s until all registers are filled by 0

26

r3 r2 r1

encoder exampleconstraint length = 3 ( = # of registers)the output is constrained by three previous input bits

encoding example (1)

to encode 1101...

27

0 0 0 1

11 0 0 1 1

10

0 1 1 0

01 1 1 0 1

10

give additional 0’s to push-out 1’s in the register...

encoding example (2)

28

1 0 1 0

00 0 1 0 0

00

1 1 0 1

10

1 0 0 0

01

the output is 11 10 01 10 00 00 01

encoder as a finite-state machine

An encoder with k registers has 2k internal states.An input causes a state transition, accompanied with outputs

29

r2 r1

inpu

t

outp

ut

internal state = (r2, r1)

0123

(0,0)(0,1)(1,0)(1,1)

(r2, r1)0

1 2

3

0 / 00

1 / 11

1 / 11

0 / 00

1 / 100 / 01

0 / 01

1 / 10

input / output

constraint: initial state = final state = 0

encoding = state transition

30

0 0 111

0 1 111

1 1 001

1 0 001

input 0input 1

2

3

00

11

11

00

10 01

01

10

0

1

00

11

11

00

10 01

01

10

1

3

00

11

11

00

10 01

01

10

0

2

00

11

11

00

10 01

01

10

0 0

at the receiver’s end

the receiver knows...the state diagramthe initial state and the final statethe transmitted sequence but corrupted by errors

31

encoder receiver

01001... 01100...

errors

to correct errors = to estimate the most-likely transition

trellis diagram

expand the state diagram to the time axis...

32

0

1

2

3

0

1

2

3

0011

10

01

11

00

1001

0

1

2

3

0

1

2

3

0

1

2

3

trellis diagram time 0 1 2 3 4

initial state final state

input 0input 1

0

1

00

11

11

00

10 01

01

10

2

3

trellis diagram and code

33

possible encoding sequences= paths connecting the initial state and the final state

the transmitted sequence = the path with the minimum penalty

10

0

1

2

3

0

1

2

3

0011

10

01

11

00

01

0

1

2

3

0

1

2

3

0

1

2

3

error correction = shortest path problem

Viterbi algorithm

given a received sequence...the demodulator defines penalties for symbols at each positionthe penalties are assigned to edges of the trellis diagramfind the path with the minimum penalty using a good algorithm

Viterbi algorithmDijkstra algorithm over a trellis diagramrecursive width-first search

34

Andrew Viterbi1935-

s0,0

pA

pB

qA

qB

the minimum penaltyof this state ismin (pA+qA, pB+qB)

soft-decision decoding for convolutional codes

the complexity of Viterbi algorithm ≈ the size of the trellis diagram

for convolutional codes (with constraint length k):the size of trellis ≈ 2k×data length ... manageablewe can extract 100% performance of the code

for block codes:the size of trellis ≈ 2data length ... too largeit is difficult to extract the full performance

35

block code high potential,but difficult to use

performance

convolutionalcode

moderated potential,full power available

summary

cyclic codesscalable, and have good mathematical structure

some good codes have been discoverederror-correction is not straight-forward

convolutional codessoft-decision decoding is practically realizableno good algorithm for constructing good codes

design in the “trial-and-error” manner with computer

still used widely, but “better” codes are studied recently...

36