daniel j. costello, jr. · daniel j. costello, jr. dept. of electrical engineering, spatially...

Daniel J. Costello, Jr.

Dept. of Electrical Engineering,

Spatially Coupled LDPC Codes:From Theory to Practice

University of Notre Dame

BIRS Workshop on Mathematical Coding Theory in Multimedia Streaming

Research Collaborators: David Mitchell,Michael Lentmaier, and Ali Pusane

D. J. Costello, Jr., “Spatial Coupling vs. Block Coding: A Comparison” / 39

Outline

LDPC Block Codes

Spatially Coupled LDPC Codes

Parity-check matrix and Tanner graph representations, minimumdistance bounds, iterative decoding thresholds, protograph-based constructions

Protograph representation, edge-spreading construction, termination

Iterative decoding thresholds, threshold saturation, minimum distance

Practical Considerations

Window decoding; performance, latency, and complexity comparisonsto LDPC block codes; rate-compatibility; implementation aspects

LDPC Block Codes

Definition by parity-check matrix: [Gallager, '62]

Code:(J,K)-regular LDPC

block code:

Bipartite graph representation: [Tanner, '81]

n = 20 variable nodes of degree J = 3

l = 15 check nodes of degree K = 4

LDPC Block Codes

Definition by parity-check matrix: [Gallager, '62]

Code:(J,K)-regular LDPC

block code:

Bipartite graph representation: [Tanner, '81]

n = 20 variable nodes of degree J = 3

l = 15 check nodes of degree K = 4

Graph-based codes can be decoded iteratively with low complexity by

exchanging messages in the graph using Belief Propagation (BP).

For an asymptotically good code ensemble, the minimum distance grows linearly with the block length n

Minimum Distance Growth Rates of

(J,K)-Regular LDPC Block Code Ensembles

where is called

the typical minimum

distance ratio, or

minimum distance

growth rate.

(J,K)-regular block codeensembles areasymptotically good, i.e.,

As the density of (J,K)-regular ensembles increases,approaches theGilbert-Varshamovbound.

where is called

the typical minimum

distance ratio, or

minimum distance

growth rate.

(J,K)-regular block codeensembles areasymptotically good, i.e.,

[RU01] T. J. Richardson, and R. Urbanke, “The capacity of low-density parity-check codes under message

passing decoding”, IEEE Transactions on Information Theory, vol. 47 no. 2, Feb. 2001.

AWGNC thresholdsBEC thresholds

Iterative decoding thresholds can be calculated for (J,K)-regular

LDPC block code ensembles using density evolution (DE).

Thresholds of (J,K)-regular LDPC

Block Code Ensembles

There exists a relatively large gap to capacity.

Iterative decoding thresholds get further from capacity as the graph

density increases.

Large LDPC codes can be obtained from a small base parity-checkmatrix B by replacing each nonzero entry in B with an M x M permutation matrix, where M is the lifting factor.

Protographs (Matrix Description)

length 6M = 24rate R = 1/2

Example: Irregular code with M = 4

length 6M = 24rate R = 1/2

Example: Irregular code with M = 4

Irregularcodes havevariable rowand columnweights(check nodeand variablenode degrees)

Protographs are often represented using a bipartite Tanner graph

Protographs (Graphical Description)

3 check nodes

6 variable nodes

[Tho05] J. Thorpe, “Low-Density Parity-Check (LDPC) codes constructed from

protographs”, Jet Propulsion Laboratory INP Progress Report, Vol. 42-154 Aug. 2003.

Protographs are often represented using a bipartite Tanner graph

Protographs (Graphical Description)

3 check nodes

6 variable nodes

The collection of all possible parity-check matrices with lifting factor M forms a code ensemble, where all the codes share a common structure

[Tho05] J. Thorpe, “Low-Density Parity-Check (LDPC) codes constructed from

protographs”, Jet Propulsion Laboratory INP Progress Report, Vol. 42-154 Aug. 2003.

.Quasi-cyclic (QC) LDPC codes are of great interest in practice, since

they have efficient encoder and decoder implementations

Quasi-Cyclic LDPC Codes

Example: protograph construction of a (2,3)-regular QC-LDPC block code

For QC codes, the permutationmatrices are shifted identities

[DDJA09] D. Divsalar, S. Dolinar, C. Jones, and K. Andrews, “Capacity-approaching protograph

codes”, IEEE Journal on Select Areas in Communications, vol. 27, no. 6 Aug. 2009.

Multi-Edge Protographs

Protographs can have repeated edges (corresponding to integer valuesgreater than one in B)

Note that this makes nosense without lifting

Repeated edges in aprotograph correspond tousing sums of permutationmatrices to form LDPC codeensembles

denser graphs!

can also be QC (using circulant matrices)!

2e variablenodes

Rate Threshold Capacity Distancegrowth

1/2 0.628 0.187 0.01450

2/3 1.450 1.059 0.00582

3/4 2.005 1.628 0.00323

4/5 2.413 2.040 0.00207

5/6 2.733 2.362 0.00145

6/7 2.993 2.625 0.00108

'Good' Protographs

Ensemble average properties can be easily calculated from a protograph,thus simplifying the construction of 'good' code ensembles.

Iterative decoding thresholds close to capacity for irregular protographs

Minimum distance growing linearly with block length (asymptotically good) for regular and some irregular protographs

Outline

LDPC Block Codes

Spatially Coupled Protographs

Consider transmission of consecutive blocks (protograph representation):

... (3,6)-regularLDPC-BC

base matrix

Blocks are spatially coupled (introducing memory) by spreading edges

over time:

base matrix

Blocks are spatially coupled (introducing memory) by spreading edges

over time:

Spreading constraint: ( has size )

Transmission of consecutive spatially coupled (SC) blocks results in aconvolutional protograph:

... ...

The bi-infinite convolutional protograph corresponds to a bi-infiniteconvolutional base matrix:

Constraint length:

SC-LDPC Code Ensembles

An ensemble of (3,6)-regular SC-LDPC codes can be created from theconvolutional protograph by the graph lifting operation

Graph lifting: is an permutation matrix

If each permutation matrix is circulant, the codes are quasi-cyclic

Code rate:

Consider terminating to a (block code) base matrix of length Lbv:

Terminated Spatially Coupled Codes

Code rate:

For large L, RL approaches the unterminated code rate .

Example: (3,6)-regular base matrix , ms = 2, L = 4, R4 = 1/4

Code rate:

(check node degrees lowerat the ends)

Example: (3,6)-regular base matrix , ms = 2, L = 4, R4 = 1/4

Code rate:

(check node degrees lowerat the ends)

Codes can be lifted to different lengths and rates by varying M and L .

Wave-like Decoding of Terminated

Spatially Coupled Codes

Variable nodes all have the same degree as the underlying block code.

Check nodes with lower degrees (at the ends) improve the BP decoder.

10 20 30 40 50 60 70 80 90 10010-6

10 iterations

Evolution of message probabilities: (3,6)-regular SC-LDPC code (L = 100)

10 20 30 40 50 60 70 80 90 10010-6

20 iterations

10 20 30 40 50 60 70 80 90 10010-6

50 iterations

10 20 30 40 50 60 70 80 90 10010-6

100 iterations

10 20 30 40 50 60 70 80 90 10010-6

200 iterations

10 20 30 40 50 60 70 80 90 10010-6

300 iterations

10 20 30 40 50 60 70 80 90 10010-6

340 iterations

10 20 30 40 50 60 70 80 90 10010-6

340 iterations

Note: the fraction of lower degree nodes tends to zero as i.e., the codes are asymptotically regular.

Density evolution can be applied to the protograph-based ensembles

with [Sridharan et al. '04]:

Thresholds of Terminated

Example: BEC

L=4, gap=0.115

Example: BEC

L=4, gap=0.115

Example: BEC

L=10, gap=0.095

L=4, gap=0.115

Example: BEC

L=10, gap=0.095

(3,6)-regular block code:

BEC AWGN

Iterative decoding thresholds (protograph-based ensembles)

We observe a significant improvement in the thresholds of SC-LDPC codes compared to the associated LDPC block codes (LDPC-BCs) due to the lowerdegree check nodes at the ends of the graph and the wave-like decoding.

[LSCZ10] M. Lentmaier, A. Sridharan, D. J. Costello, Jr., and K.Sh. Zigangirov, “Iterative decoding threshold

analysis for LDPC convolutional codes,” IEEE Trans. Inf. Theory, 56:10, Oct. 2010.

BEC AWGN

Iterative decoding thresholds (protograph-based ensembles)

We observe a significant improvement in the thresholds of SC-LDPC codes compared to the associated LDPC block codes (LDPC-BCs) due to the lowerdegree check nodes at the ends of the graph and the wave-like decoding.

In contrast to LDPC-BCs, the iterative decoding thresholds of SC-LDPC codes improve as the graph density increases.

When symbols are perfectly known (BEC), all adjacent edges can be removedfrom the Tanner graph.

Why are Terminated Spatially Coupled

Thresholds Better?

The threshold saturates (converges) to a fixed value numericallyindistinguishable from the maximum a posteriori (MAP) threshold of the (J, K)-regular LDPC-BC ensemble as [LSCZ10].

Thresholds Better?

The threshold saturates (converges) to a fixed value numericallyindistinguishable from the maximum a posteriori (MAP) threshold of the (J, K)-regular LDPC-BC ensemble as [LSCZ10].

For a more random-like ensemble, this has been proven analytically, first forthe BEC [KRU11], then for all BMS channels [KRU13].

[KRU11] S. Kudekar, T. J. Richardson and R. Urbanke, “Threshold saturation via spatial coupling: why

convolutional LDPC ensembles perform so well over the BEC”, IEEE Trans. on Inf. Theory, 57:2, 2011

[KRU13] S. Kudekar, T. J. Richardson and R. Urbanke, “Spatially coupled ensembles universally achievecapacity under belief propagation”, IEEE Trans. on Inf. Theory, 59:12, 2013.

Threshold Saturation (BEC)

0.32 0.34 0.36 0.38 0.4 0.42 0.44 0.46 0.48 0.5

epsilon

rasure

threshold

(3,6)(3,6)

BP = iterative (suboptimal) decoding thresholdMAP = (optimal) maximum a posteriori threshold

0.32 0.34 0.36 0.38 0.4 0.42 0.44 0.46 0.48 0.5

epsilon

rasure

MAP MAPBP BP

(5,10)

0.32 0.34 0.36 0.38 0.4 0.42 0.44 0.46 0.48 0.5

epsilon

rasure

MAP MAPBP BP

SC-LDPC

(5,10)

0.32 0.34 0.36 0.38 0.4 0.42 0.44 0.46 0.48 0.5

epsilon

rasure

MAP MAPBP BP

SC-LDPC

(5,10)

optimal decoding performance with a suboptimal iterative algorithm!

Threshold Saturation (AWGNC)

(3,6)-regularblock code

capacity

~0.5dB

capacity

~1.25dB

capacity

spatially coupledcodes

(3,6)(4,8)

optimal decoding performance with a suboptimal iterative algorithm!

capacity

spatially coupledcodes

(3,6)(4,8)

BEC Thresholds vs Distance Growth

By increasing J and K, we obtain capacity achieving (J,K)-regular SC-LDPC code ensembles with linear minimum distance growth.

(J,K)-regular SC-LDPC codes combine the best features of irregularand regular LDPC-BCs, i.e., capacity approaching thresholds and lineardistance growth.

AWGNC Thresholds vs. Distance Growth

[MLC10] D. G. M. Mitchell, M. Lentmaier and D. J. Costello, Jr., “AWGN Channel Analysis of Terminated

LDPC Convolutional Codes”, Proc. Information Theory and Applications Workshop, San Diego, Feb. 2011.

Similar results are obtained for the AWGNC

Distance Measures for SC-LDPC Codes

As the minimum distance growth rates of terminated SC-LDPCcode ensembles tend to zero. However, the free distance growth rates ofthe unterminated ensembles remain constant.

(3,6)-regular unterminated SC-LDPC free distance growth rates

(3,6)-regular terminated SC-LDPCminimum distance growth rates

For large L, thestrength ofunterminatedensembles scaleswith theconstraint length and isindependent of L.

independent of L

normalized by L

For large L, thestrength ofunterminatedensembles scaleswith theconstraint length and isindependent of L.

An appropriatedistance measurefor 'convolutional-like' terminatedensembles shouldbe independent of L.

independent of L

normalized by L

Outline

LDPC Block Codes

D. J. Costello, Jr., “Spatial Coupling vs. Block Coding: A Comparison” / 39 25

Decoding SC-LDPC Codes

SC-LDPC codes can be decoded with standard iterative decoding schedules.

Reliable messagesfrom the endspropagate throughthe graph toward thecenter as iterationsproceed.

The highly localized (convolutional) structure is well-suited for efficient

decoding schedules that reduce memory and latency requirements.

Sliding window decoding (WD) updates nodes onlywithin a localized windowand then the window shiftsacross the graph [Lentmaieret al '10, Iyengar et al '12].

width W

Window Decoding Performance

[LPF11] M. Lentmaier, M. M. Prenda, and G. Fettweis, “Efficient Message Passing Scheduling for

Terminated LDPC Convolutional Codes”, Proc. IEEE ISIT, St. Petersburg, Russia, July 2011.

Latencies:LDPC:SC-LDPC:

For equal liftingfactors, SC-LDPCcodes display alarge convolutionalgain at the cost ofincreased latency.

convolutional gain

For equal latency,SC-LDPC codesstill display asignificantperformance gain.

equallatency

Trade-off in M vs W

Required to achieve a BER of as a function of latency:

decreases as W (and thus thelatency) increases.

does not decreasesignificantly beyond acertain W

Equal Latency Comparison for

(3,6)-Regular LDPC Codes

large improvescode performance.

large W improvesdecoder performance.

When choosingparameters:

Complexity Tradeoffs

For equal latency, SC-LDPC codes display a performance gain comparedto the underlying LDPC-BCs

(including non-binary codes)

With standardstopping rules, thecomputationalcomplexity ishigher for SC-LDPC codes

LDPC-BCs cannot achieve equalperformance byincreasing thenumber ofiterations

For equal performance (BER of 10-5 at 1.5dB), SC-LDPC codes display alarge reduction in latency compared to LDPC-BCs of similar complexity

Complexity/Latency Tradeoffs

With increasing(small) field sizes q,latency decreasesfor increasingcomplexity

For larger q, bothlatency and complexityincrease (for both SC-LDPC codes andLDPC-BCs)!

SC-LDPC codesover GF(4) offer agood balancebetween complexityand latency

For larger q, bothlatency and complexityincrease (for both SC-LDPC codes andLDPC-BCs)!

Rate-compatible Punctured Codes

A linear code with rate R is punctured by removing a set of p columnsfrom its generator matrix, reducing the codeword length from n to

A variety of code rates can be achieved using the same decoder bypuncturing different numbers of symbols (rate compatibility)

puncturing fraction:

punctured rate:

The code rate is increased by puncturing:

Randomly Punctured

SC-LDPC Codes (BEC)

M = 500, W=8Latency = 2MW = 8000

Randomly punctured SC-LDPC codes drawn from

Max its. = 10

Randomly Punctured

SC-LDPC Codes (BEC)

M = 500, W=8Latency = 2MW = 8000

Max its. = 10

Robust decodingperformance

Small gap tothreshold for allrates

No error floorsobserved

Gaps decreasewith increasing M

Robust decodingperformance

Small but slightlyincreasing gap tothreshold (~1-1.25dB at 10-5) forincreasing

Gaps decreasewith increasing M

M = 500, W=8Latency = 2MW = 8000Max its. = 10

No error floorsobserved

Randomly Punctured

SC-LDPC Codes (AWGNC)

Regular SC-LDPC Codes vs.

Irregular LDPC-BCs

Consider a comparison of a (3,6)-regular SC-LDPC code vs. anirregular-repeat-accumulate (IRA) LDPC-BC with optimizedprotograph taken from the WiMAX standard

Irregular LDPC-BCs

The IRA LDPC-BC ensemble has rate R=0.5, BEC threshold , and AWGNC threshold dB.

Irregular LDPC-BCs

We compare this to a (3,6)-regular SC-LDPC code ensemble withL=50, R=0.49, and thresholds and dB.

Irregular LDPC-BCs

We compare this to a (3,6)-regular SC-LDPC code ensemble withL=50, R=0.49, and thresholds and dB.

For the SC-LDPC code, we choose W=6 and M=500 so that thelatency of both codes is 6000 bits. (Since a code symbol is present inW=6 'windows', we allow fewer iterations per position for the SC-LDPC window decoder.)

Gaps tothreshold willreduce withincreasinglatency

Irregular LDPC-BCs

Theasymptoticallygood regularSC-LDPC codeshows no sign ofan error floor

Irregular LDPC-BCs

Theasymptoticallygood regularSC-LDPC codeshows no sign ofan error floor

The regular SC-LDPC codestructure hasimplementationadvantages

Irregular LDPC-BCs

Randomly Punctured LDPC Codes

Randompuncturing can beapplied to LDPCcodes to increasethe rate

Randomly Punctured LDPC Codes

Randompuncturing can beapplied to LDPCcodes to increasethe rate

Equal latencyperformancecomparisons areconsistent for higherrate codes

Regular SC-LDPCcodes displayrobust decodingperformancecompared toirregular LDPC-BCs

Spatial Coupling of Irregular Codes

We can also couple irregular codes to construct an irregular SC-LDPCcode ensemble. Consider the ARJA LDPC-BC protograph:

A spatially coupled version can be created by edge spreading:

... ...

AWGNC Thresholds vs Distance Growth

Irregular SC-LDPC code ensembles also display excellent asymptotic

properties

[MLC10] D. G. M. Mitchell, M. Lentmaier and D. J. Costello, Jr., “AWGN Channel Analysis of Terminated

LDPC Convolutional Codes”, Proc. Information Theory and Applications Workshop, San Diego, Feb. 2011.

Implementation Aspects

As a result of their capacity approaching performance and simplestructure, regular SC-LDPC codes may be attractive for future codingstandards. Several key features will require further investigation:

Hardware advantages of QC designs obtained by circulant liftings

Hardware advantages of the 'asymptotically-regular' structure

Design advantages of flexible frame length and flexible rateobtained by varying M, L, and/or puncturing

Implementation Aspects

As a result of their capacity approaching performance and simplestructure, regular SC-LDPC codes may be attractive for future codingstandards. Several key features will require further investigation:

Hardware advantages of QC designs obtained by circulant liftings

Hardware advantages of the 'asymptotically-regular' structure

Design advantages of flexible frame length and flexible rateobtained by varying M, L, and/or puncturing

Of particular importance for applications requiring extremely lowdecoded bit error rates (e.g., optical communication, data storage) isan investigation of error floor issues related to stopping sets,trapping sets, and absorbing sets.

Conclusions

Spatially coupled LDPC code ensembles achieve threshold saturation,i.e., their iterative decoding thresholds (for large L and M) approach theMAP decoding thresholds of the underlying LDPC block code ensembles.

The threshold saturation and linear minimum distance growth propertiesof (J,K)-regular SC-LDPC codes combine the best asymptotic featuresof both regular and irregular LDPC-BCs.

With window decoding, SC-LDPC codes also compare favorably toLDPC-BCs in the finite-length regime, providing flexible tradeoffsbetween BER performance, decoding latency, and decoder complexity.

SC-LDPC codes can be punctured to achieve robustly good performanceover a wide variety of code rates.

daniel j. costello, jr. · daniel j. costello, jr. dept. of electrical engineering, spatially...

Documents

por que deus usa pessoas como sansão? daniel santos jr

costello daniel september 2010 - university of minnesota

frank costello

presented by daniel garcia, jr. - frigolaw.com - dan...

advanced dsp - visvesvaraya technological...

date printed: 06-01-21 daniel wright jr high school

multi-level trellis coded modulation and · pdf file/va _a...

costello fish

theodore j. boutrous jr., sbn 132099 daniel g. swanson

monte carlo simulation for next generatio n source and...

onde estudar teologia? | daniel santos jr

modern coding theory...modern coding theory daniel j....

daniel j. o'connor jr. - recorder.franklincountyohio.gov

supplementary information global fishery prospects under...

daniel j. costello, jr. - itsoc.org · daniel j. costello,...

muscles of the back daniel young, jr isenberg, lexi morton

costello characters

marriage records croneen, daniel, jr.. m rachel adams 1779...

veterans’ benefits overview 2007 public benefits training...

2020 daniel d. richter, jr. nicholas ... - duke university