Hierarchies of Local-Optimality Characterizations

in Decoding Tanner Codes

Nissim Halabi Guy Even

School of Electrical Engineering, Tel-Aviv University

July 6, 2012

1/17

Tanner Graphs and Tanner Codes

v1x1

v2x2

v4x4

v3x3

v5x5

v6x6

v7x7

v8x8

v10x10

x9 v9

G = (V ∪ J , E)

Variable Nodes Local-Code NodesV J

C4 C4

C1 C1

C2 C2

C3 C3

C5 C5

Tanner code C(G, CJ ) representedby bipartite graph

x ∈ C(G, CJ ) iff x ∈ Cj for everyj ∈ {1, . . . , J}

degrees: can be regular, irregular,bounded, or arbitrary

can allow arbitrary linear localcodes

minimum local distanced∗ , minj dj

2/17

Decoding of Tanner Codes over MBIOS Channels

ChannelEncoder

ChannelDecodercodeword

Noisy Channelnoisy codeword

λ(y) ∈ RN

u ∈ {0, 1}k

c ∈ {0, 1}Nu ∈ {0, 1}k c ∈ C ⊂ {0, 1}N

Memoryless binary-input output-symmetric channel (MBIOS)channels characterized by a log-likelihood ratio (LLR)observations λ:

λi(yi) , ln

(

Pr(yi | ci = 0)

Pr(yi | ci = 1)

)

ML-decoding:ml(λ) , argmin

x∈conv(C)〈λ, x〉

LP-decoding [following Feldman-Wainwright-Karger’05]:

lp(λ) , argminx∈P(G,CJ )

〈λ, x〉

where P(G, CJ ) , generalized fundamental polytope of aTanner code C(G, CJ )

3/17

Local-Optimality: Sufficient Condition for Successful

Decoding of Finite-Length Codes

Set of deviations B(w)d ⊂ R

N : finite set of vectorscorresponding to projections of w-weighted d-trees incomputation trees with height 2h of the Tanner graph

Definition ([Even-H’11])

A codeword x ∈ C is (h,w, d)-locally optimal w.r.t. λ ∈ RN if for

all vectors β ∈ B(w)d ,

〈λ, x⊕ β〉 > 〈λ, x〉

Theorem ([Even-H’11])

Let 2 6 d 6 d∗. If x ∈ C is (h,w, d)-locally optimal w.r.t. λ, then:

1 x is the unique ML codeword w.r.t. λ

2 x is the unique optimal solution of the LP-decoder w.r.t. λ

4/17

Parameters (h, w, d) for Deviations of Local-Optimality

B(w)d ⊂ R

N : finite set of projections of w-weighted d-treeswith height 2h in computation trees of the Tanner graph

h ∈ N - tree height /2w ∈ R

h+ - level weights

d ∈ N - degree of local-codes nodes in the sub-tree, 2 6 d 6 d∗

h = 2

wT (p)w1

w2 3-tree (d = 3)

An hierarchy statement: lo with parameter h ⇒ lo withparameter h′ > h

5/17

Height Hierarchy of Local-Optimality: Motivation

An iterative decoding algorithm (nwms) is guaranteed todecode lo-certified codeword in h iterations [Even-H’11]

Questions: what is the effect of increasing the number ofiterations? even when number of iterations exceeds the girth?

Theorem (Hierarchy of local-optimality based on height)

An (h,w, d)-strongly locally optimal codeword x w.r.t. λ is also

(k · h, w, d)-strongly locally optimal w.r.t. λ for any k ∈ N

x is slo with height parameter h ⇒ Iterative message-passingdecoding by nwms is guaranteed to decode the ML-certified xafter k · h iterations ∀k ∈ N+

Insight on convergence: If a codeword x is slo-certified afterh iterations, then x is the outcome of nwms infinitely manytimes

6/17

Strong Local-Optimality

Definition ((h,w, d)-Strong Local Optimality)

A codeword x ∈ C is (h,w, d)-strongly locally optimal w.r.t.

λ ∈ RN if for all vectors β ∈ B

(w)d ,

〈λ, x⊕ β〉 > 〈λ, x〉

B(w)d ⊂ R

N : finite set of projections of w-weighted reducedd-trees in computation trees with height h of the Tanner graphReduced: degT (root) = degG(root)− 1 (as if the root itselfhangs from an edge)

3−treedefines deviations for

local−optimalitydefines deviations for

Reduced 3−tree

strong local−optimality7/17

Strong Local-Optimality vs. Local-Optimality

Set of pairs (x, λ) s.t. x is locally-optimal w.r.t. λ:

loC(h,w, d) ,{

(x, λ) ∈ C ×R | x is (h,w, d)−lo w.r.t. λ}

Set of pairs (x, λ) s.t. x is strongly locally-optimal w.r.t. λ:

sloC(h,w, d) ,{

(x, λ) ∈ C×R | x is (h,w, d)−slo w.r.t. λ}

Lemma

sloC(h,w, d) ⊆ loC(h,w, d)

Corollary

slo ⇒ unique ml

slo ⇒ unique lp opt.

Empirically: sloapproaches lo as hincreases (h >> girth(G))

1 10 100 3200

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

h

|LO0N ,Λp(h, 1h, 2)|, p = 0.04

|SLO0N ,Λp(h, 1h, 2)|, p = 0.04

|LO0N ,Λp(h, 1h, 2)|, p = 0.05

|SLO0N ,Λp(h, 1h, 2)|, p = 0.05

|LO0N ,Λp(h, 1h, 2)|, p = 0.06

|SLO0N ,Λp(h, 1h, 2)|, p = 0.06

girth=6h=3

8/17

Height Hierarchy of Local-Optimality

Theorem (h-hierarchy of local-optimality)

For every k ∈ N and geometric level-weights w,

sloC(h,w, d) ⊆ sloC(k · h,w, d)

SLO0N (h, w, d)

RN

ML(λ) = 0N

LP(λ) = 0N

SLO0N (k · h, w, d)

SLO0N (2 · h, w, d)

Remark: Assume x is transmittedover an MBIOS channel with somebounded noise level, then x is slow.r.t. the received LLR with highprobability

Interesting because an iterativemessage-passing decodingalgorithm is guaranteed to find slo

codewords

9/17

Proof Method for Height Hierarchy of Local-Optimality

Proof via contrapositive statement:

Lemma (Symmetry of slo)

x is slo w.r.t. λ iff 0N is slo w.r.t. λ0 , (−1)x ∗ λ

x not (k · h,w, d)-slo w.r.t. λ

⇐⇒ 0N is not (k · h,w, d)-slo w.r.t. λ0

[symmetry]

⇐⇒ ∃β based on reduced d-tree of height2 · k · h s.t. 〈λ0, β〉 < 0 [slo]

⇒ ∃β′ based on reduced d-tree of height2 · h s.t. 〈λ0, β′〉 < 0 [“averaging”]

⇐⇒ 0N is not (h,w, d)-slo w.r.t. λ0 [slo]

⇐⇒ x not (h,w, d)-slo w.r.t. λ[symmetry]

pi

T

T i

2h

2h

2h

2 · k · h

10/17

Implications of Height Hierarchy to Message-Passing

Decoding

Setting:

Irregular Tanner codes

Local-codes = single parity-check code

Local-optimality: d = 2, arbitrary height h (not limited bygirth)

Normalized Weighted Min-Sum (NWMS) Algorithm [Even-H’11]

Normalize: take care of irregular degrees

Weighted: allow level weights

Min-Sum: based on Max-Product/Min-Sum algorithm

nwms(λ, h,w):

Input: λ ∈ RN - LLRs from the channel

h ∈ N - number of iterationsw ∈ R

h+ - level weights

Output: x ∈ {0, 1}N

11/17

BP-Based Decoding - nwms(λ, h, w): Init

Initialize:

Init check-to-variable messages with 0:

∀C ∈ J , ∀v ∈ N (C) : µ(−1)C→v ← 0

12/17

BP-Based Decoding - nwms(λ, h, w): Iterations

Iterate: for ℓ = 0 to h− 1:

message: variable node → check node

C

µ v→C

v µ(ℓ)v→C ←

wh−ℓ

degG(v)λv+

1

degG(v)− 1

∑

C′∈N (v)\{C}

µ(ℓ−1)C′→v

[degree normalization, level weights]

message: check node → variable node

C

v µC→

v

µ(ℓ)C→v ←

(

∏

u∈N (C)\{v}

sign(

µ(ℓ)u→C

)

)

· minu∈N (C)\{v}

{

|µ(ℓ)u→C |

}

13/17

BP-Based Decoding - nwms(λ, h, w): Decision

Decision:

for all v ∈ V do

µv ←∑

C∈N (v) µ(h−1)C→v

xv ←

{

0 if µv > 0,

1 otherwise.end for

14/17

NWMS: Decoding Guarantee via Local-Optimality

Theorem (Even-H’11)

If x is (h,w, 2)-locally optimal w.r.t. λ then nwms(λ, h,w)returns x

Height Hierarchy of Local-Optimality Implies:

If nwms finds an slo certified codeword x after h iterations, then

1 nwms outputs x every k · h iterations (infinitely many times)

2 nwms never outputs a codeword y 6= x for any number ofiterations

Holds for all h, decoding guarantee not limited by the girth!

No issue of convergence/divergence (as in density evolution)

ML-certificate by local-optimality

15/17

Degree Hierarchy of Local-Optimality Characterization

What is the effect of increasing the minimum distance of thelocal codes in Tanner codes?

Theorem (Hierarchy of local-optimality based on degrees)

An (h,w, d)-locally optimal codeword x w.r.t. λ is also

(h,w, d′)-locally optimal w.r.t. λ for any degree parameter d′ > d

Insight on the improvement of suboptimal decodings ofexpander codes as the minimum distance of local codesincreases.

16/17

Summary

Conclusions

Hierarchies of Local Optimality:

1 Degree hierarchy ⇒ take d as large as possible

2 Strong local-optimality ⇒ local-optimality (⇒ lp-opt. ⇒ML-opt.)

3 Height hierarchy of local-optimality

A strongly locally optimal codeword is infinitely often stronglylocally optimal (w.r.t. height parameter)A BP-Based algorithm nwms decodes x with slo certificateafter h iterations ⇒ nwms decodes x with slo certificateevery k · h iterations, for every k ∈ N

Open Questions

Height hierarchies for belief-propagation (sum-product)algorithm and other BP-based algorithms [no probabilisticassumptions as in monotonicity of DE]

17/17