constructing error-correction codes from scale-free networks
TRANSCRIPT
Constructing ErrorConstructing Error--Correction Codes Correction Codes from Scalefrom Scale--Free NetworksFree Networks
Francis C.M. LauFrancis C.M. Lau
Department of Electronic and Information EngineeringHong Kong Polytechnic University
International Workshop on Complex Systems and Networks 2007 Guilin, China
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
IWCSN' 2007, Guilin, China 2
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 1: Communications and CodingPart 1: Communications and Coding
IWCSN' 2007, Guilin, China 3
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Communications without CodingHow are you today ?
Hxw au& u%$ wqo .
Welf affi zv iol bxg.
How aruyox tuday ?
Information can be easily corrupted when sent through a channel !
IWCSN' 2007, Guilin, China 4
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
More Reliable Communications
How are you today ?How are you today ?How are you today ?
How aru yox tuday ?How aee yeu todey ?Hoe are you toxak ?
How are you today ?
Error-correction capability
channel
IWCSN' 2007, Guilin, China 5
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Bit-level Communications
111100
011010
101001
110000
10
Without coding schemes:
0,1 0,0
InfoSource
channel InfoSink
noise
0,1 0,1000, 111 000, 110
1 information bit2 check bits
Code Rate=number of information bits/ block length=1/3
Block Length =1+2=3
With coding schemes:Info
SourceEncoder channel Decoder Info
Sink
noise
IWCSN' 2007, Guilin, China 6
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Reliable Communications
Add redundant information at transmitterDecode information intelligently at receiver
Error-Correction Capability
Any better ways than to repeat the information several times?Any performance bounds?
IWCSN' 2007, Guilin, China 7
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Shannon’s Capacity Theorem
Channel
Additive White Gaussian NoiseBandwidth W
Average Received Signal Power SAverage Noise Power N
⎟⎠⎞
⎜⎝⎛ +=
NSWC 1log2
System Capacity of the channel
C. E. Shannon “A mathematical theory of communications,” Bell Syst. Tech. J., vol. 27, pp. 379–423, 623–656, 1948.
IWCSN' 2007, Guilin, China 8
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Shannon’s Capacity Theorem
possible theoretically to transmit information at any rate R ≤ C with an arbitrarily small error probability (with coding)if R > C, not possible to transmit information with an arbitrarily small error probability (even with coding)
⎟⎠⎞
⎜⎝⎛ +=
NSWC 1log2
ChannelInformation with rate R
IWCSN' 2007, Guilin, China 9
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Normalized channel capacity versus SNR
⎟⎠⎞
⎜⎝⎛ +=
NS
WC 1log2
IWCSN' 2007, Guilin, China 10
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Normalized channel bandwidth versus SNR
⎟⎠⎞
⎜⎝⎛ +
=
NSC
W
1log
1
2
The graph is not telling the whole story !
IWCSN' 2007, Guilin, China 11
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Because …
noise power is proportional to bandwidth
WNN 0=
⎟⎠⎞
⎜⎝⎛ +
=
NSC
W
1log
1
2
IWCSN' 2007, Guilin, China 12
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Further …
when bit rate R equals channel capacity C
CS
RSEb ==energy per bit
IWCSN' 2007, Guilin, China 13
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Shannon Limit
⎟⎠⎞
⎜⎝⎛ +=
NS
WC 1log2
( )12 /
0
−= WCb
CW
NE
WNN 0=
CS
RSEb ==
rearrangement
IWCSN' 2007, Guilin, China 14
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
( )12 /
0
−= WCb
CW
NE
Shannon Limit
0//
dB 59.1/ 0
→⇔∞→⇒−→
WCCW
NEb
Channel capacity approaches zero, regardless of the channel bandwidth
No error-free communications below dB 59.1/ 0 −=NEb
IWCSN' 2007, Guilin, China 15
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Typical error performance of coded and uncodedmodulations
IWCSN' 2007, Guilin, China 16
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Improved error performance;more bandwidth required to
add redundancy bits
IWCSN' 2007, Guilin, China 17
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Reduction in requirement;more bandwidth required to add
redundancy bits
0/ NEb
coding gain
IWCSN' 2007, Guilin, China 18
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Significance/Conclusions of Shannon’s work
proved theoretically that there exists codes that could improve the error probability performance from uncoded modulation schemesthere is a minimum requirement0/ NEb
IWCSN' 2007, Guilin, China 19
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BUT ….
IWCSN' 2007, Guilin, China 20
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
How should we design coding schemes, with reasonable
complexity, that work as close to the Shannon limit as possible?
IWCSN' 2007, Guilin, China 21
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Solutions
Not provided by Shannon !
So do research on Coding !
IWCSN' 2007, Guilin, China 22
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 2: ParityPart 2: Parity--Check CodesCheck Codes
IWCSN' 2007, Guilin, China 23
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Parity-Check Codes
single-parity-check code
110011010100
parity bit
message bits
IWCSN' 2007, Guilin, China 24
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Parity-Check Codes
single-parity-check code
even-parity code
can detect all single-and triple-error patterns (e.g. 0100 or 0010) but cannot correct errors
IWCSN' 2007, Guilin, China 25
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
001111110011110000000110100001101011
Parity-Check Codes
rectangular code (or product code)
horizontal parity check
vertical parity check
message
IWCSN' 2007, Guilin, China 26
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
001111110011110000000110100001101011
Parity-Check Codes
rectangular code (or product code)
can correct a single error pattern
001111110011110000001110100001101011
horizontal parity check fails
vertical parity check fails
bit in error
channel
IWCSN' 2007, Guilin, China 27
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Linear Block Codes
a class of parity-check codesdenoted by (n, k)
codeword length message length
maps k-bit messages (k-tuples) linearlyand uniquely to n-bit codewords (n-tuples)
IWCSN' 2007, Guilin, China 28
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Subset S of a vector space is a subspace if
it contains the all-zeros vectorsum of any two vectors in S is also in S
IWCSN' 2007, Guilin, China 29
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Packing as many Packing as many codewordscodewords in the entire in the entire space as possible improves space as possible improves coding efficiency coding efficiency
Putting the Putting the codewordscodewords as as far apart from one far apart from one another as possible another as possible increases the chance of increases the chance of decoding the decoding the codewordscodewordscorrectlycorrectly
IWCSN' 2007, Guilin, China 30
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
(6, 3) Code Example
form a subspace
011101+
Modulo-2 Addition
all-zeros vector
IWCSN' 2007, Guilin, China 31
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Modulo-2 Multiplication• 0 10 0 01 0 1
Modulo-2 Addition and Multiplication
addition can be accomplished electronically using an Exclusive-OR gatemultiplication can be accomplished using an AND gate
Modulo-2 Addition+ 0 10 0 11 1 0
IWCSN' 2007, Guilin, China 32
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Encoding the Messages
Table look-up is possible for small kFor large k, table look-up may become extremely difficult
e.g., 301026.12100 ×≈⇒= kk
Use of Generator Matrix
IWCSN' 2007, Guilin, China 33
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Generator Matrix G (size k x n)
a basis set of k linearly independent n-tuples that spans the subspace
n-tuple
n-tuple
n-tuple
][ 21 kmmm L=mmessage
mGU =codeword (size 1 x n)
IWCSN' 2007, Guilin, China 34
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
(6, 3) Code Example
][ 654321 uuuuuu=
IWCSN' 2007, Guilin, China 35
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Parity-Check Matrix H (size (n-k) x n)
For each generator matrix G, there exists an (n-k) x nmatrix H such that rows of G are orthogonal to rows of H.
0GH =Tk x (n-k) all-zeros matrix
0mGHUH == TT
H can be used to test whether a received vector is a valid codeword.
HG ↔
IWCSN' 2007, Guilin, China 36
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Parity-Check Matrix H (size (n-k) x n)
0GH =T
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
110100011010101001
H
rows areorthogonal
IWCSN' 2007, Guilin, China 37
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Parity-Check Matrix H
000
101110011100010001
][
653
542
641
654321
=++=++=++
⇒
=
⎥⎥⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢⎢⎢
⎣
⎡
⇒=
uuuuuuuuu
uuuuuuT 00UH
IWCSN' 2007, Guilin, China 38
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Bipartite Graph
)0(5422 =++= uuuc
)0(6533 =++= uuuc
1u
2u
3u
4u
5u
6u
variable nodes
check nodes
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
110100011010101001
H
)0(6411 =++= uuuc
IWCSN' 2007, Guilin, China 39
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Decoding
received vector r’
Codeword UAdditive White Gaussian Noise Channel0 +1 volt
1 −1 volt
IWCSN' 2007, Guilin, China 40
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Eight codewords in a 6-tuple space
Hard Decoding
received vector r’(AWGN channel)
decoded codeword after error correction
after making hard decision on each bit
ri > 0 volt 0ri < 0 volt 1
IWCSN' 2007, Guilin, China 41
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
r’
)'|011101( rU =P
)'|101110( rU =P
)'|110100( rU =P
maximuma posteriori (MAP) decision rule: Select codeword U that has the largest )'|( rUP
)'|101001( rU =Pa posteriori probability (APP)
Soft Decoding
IWCSN' 2007, Guilin, China 42
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Performance of Some Well-known Block Codes (Coherent BPSK over an AWGN channel)
t = maximum number of guaranteed correctable errors per codeword
IWCSN' 2007, Guilin, China 43
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Performance of BCH Codes (Coherent BPSK over an AWGN channel)
IWCSN' 2007, Guilin, China 44
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 3: LowPart 3: Low--DensityDensity--ParityParity--Check (LDPC) CodesCheck (LDPC) Codes
IWCSN' 2007, Guilin, China 45
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Parity-Check Matrix H (size (n-k) x n)
0GH =T
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
110100011010101001
H
IWCSN' 2007, Guilin, China 46
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Low-Density-Parity-Check Codes
proposed by Gallager(1960)parity-check matrix H
sparse (most elements are zeros)fraction of 1’s ~ O(n)
elements of H determine the connections between variable nodes and check nodes degree of variable
node u6 = 2
degree of checknode c3 = 3
IWCSN' 2007, Guilin, China 47
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Low-Density-Parity-Check Codes
sparse (low-density) parity-check matrix Himplies that all variable nodes and check nodes have very few connections
HG ↔
IWCSN' 2007, Guilin, China 48
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Error rates achieved by different coding schemes under the binary AWGN channel.Codeword length = 106. Rate =0.5.
IWCSN' 2007, Guilin, China 49
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Types of LDPC Codes
Regular LDPC all nodes of the same type (variable node or check node) have the same degree
A (3, 6)-regular LDPC code of length 10 and rate one-half.
check node degree
variable node degree
IWCSN' 2007, Guilin, China 50
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Types of LDPC Codes
Irregular LDPC: the degrees of each set of nodes are chosen according to some distribution
IWCSN' 2007, Guilin, China 51
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Degree Distribution of Nodes
Degree distribution of variable nodes
Degree distribution of check nodes
fraction of edges connected to the variable nodes with degree k
∑=
−=vd
k
kk xx
2
1)( λλ
∑=
−=cd
k
kk xx
2
1)( ρρfraction of edges connected to the
check nodes with degree k
maximum variable node degree
maximum check node degree
IWCSN' 2007, Guilin, China 52
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Code Rate
∫∫−= 1
0
1
0
d)(
d)(1
xx
xxR
λ
ρ
IWCSN' 2007, Guilin, China 53
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Question
Given specific
∑=
−=vd
k
kk xx
2
1)( λλ ∑=
−=cd
k
kk xx
2
1)( ρρand .
How would the LDPC code perform?
IWCSN' 2007, Guilin, China 54
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
AnswerDependent on the actual design
(connections),
the channel type, e.g. AWGN, binary symmetric channel (BSC),
binary erasure channel (BEC)
and the decoding algorithm.
IWCSN' 2007, Guilin, China 55
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
More Question
Any idea on the optimal performance of
practical LDPC decoders?
IWCSN' 2007, Guilin, China 56
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 4: Part 4: Belief Propagation (BP) Decoding Belief Propagation (BP) Decoding AlgorithmAlgorithm
IWCSN' 2007, Guilin, China 57
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Belief Propagation (BP) Decoding Algorithm
A kind of message-passing decoding algorithmApplicable to both regular and irregular LDPC codesProduces very good error performance
IWCSN' 2007, Guilin, China 58
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Belief Propagation (BP) Decoding Algorithm
Define Log Likelihood Ratio (LLR):
⎥⎦
⎤⎢⎣
⎡==
info)|1bit(info)|0bit(log
PP
variable nodes
check nodes
IWCSN' 2007, Guilin, China 59
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Compute initial Log Likelihood Ratio (LLR) for each variable node based on the received signal vector r (real number elements)
⎥⎦
⎤⎢⎣
⎡==
)|1bit()|0bit(log
i
i
rPrP
1r)(LLR 10 r
iterationnumber
ir
IWCSN' 2007, Guilin, China 60
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Set iteration number k = 1Pass the LLR messages from variable nodes to the connected check nodes
)(LLR 10 r
IWCSN' 2007, Guilin, China 61
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Check nodes received the LLR messages from the connected variable nodes
IWCSN' 2007, Guilin, China 62
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each check node computes a message for each of its connected variable nodes, based on the messages from all other variables nodes
IWCSN' 2007, Guilin, China 63
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each check node computes a message for each of its connected variable nodes, based on the messages from all other variables nodes
IWCSN' 2007, Guilin, China 64
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each check node computes a message for each of its connected variable nodes, based on the messages from all other variables nodes
IWCSN' 2007, Guilin, China 65
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each check node computes a message for each of its connected variable nodes, based on the messages from all other variables nodes
IWCSN' 2007, Guilin, China 66
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each check node computes a message for each of its connected variable nodes, based on the messages from all other variables nodes
IWCSN' 2007, Guilin, China 67
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each check node computes a message for each of its connected variable nodes, based on the messages from all other variables nodes
IWCSN' 2007, Guilin, China 68
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each variable node update its LLR based on the messages passed from the check nodes and the initial LLRBased on the updated LLR, estimate the codeword
)(LLR)(LLR 1110 rr →
IWCSN' 2007, Guilin, China 69
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Estimate the codeword as
)(LLR)(LLR 1110 rr →
[ ]nccc ˆˆˆˆ 21 L=c
where ⎭⎬⎫
⎩⎨⎧
<>
=0)(LLR if1 0)(LLR if0
ˆ1
1
rr
ck
ki
If , is the decoded codeword.
0Hc =Tˆ c
IWCSN' 2007, Guilin, China 70
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
)(LLR 10 r
If , increment the iteration number k.Each variable node computes a message for each of its connected check nodes, based on its initial LLR and the messages from all other connected check nodes
0Hc ≠Tˆ
IWCSN' 2007, Guilin, China 71
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
)(LLR 10 r
Each variable node computes a message for each of its connected check nodes, based on its initial LLR and the messages from all other connected check nodes
IWCSN' 2007, Guilin, China 72
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
)(LLR 10 r
Each variable node computes a message for each of its connected check nodes, based on its initial LLR and the messages from all other connected check nodes
IWCSN' 2007, Guilin, China 73
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Check nodes received the LLR messages from the connected variable nodes
IWCSN' 2007, Guilin, China 74
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
BP Decoding Algorithm
Each check node computes a message for each of its connected variable nodes, based on the messages from all other variables nodesSame iterative process repeated …. until convergence to a valid codeword or maximum number of iterations exceeded
IWCSN' 2007, Guilin, China 75
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Capacity (Threshold) of LDPC codesGiven the degree distributions
and the channel type (AWGN, BSC or BEC)and the use of BP decoding algorithm.
∑=
−=vd
k
kk xx
2
1)( λλ ∑=
−=cd
k
kk xx
2
1)( ρρ
Richard and Urbanke (2001) proposed an effective algorithm – density evolution – to determine the capacity (threshold) of LDPC codes.
T. J. Richardson and R. Urbanke, “The capacity of low-density parity-check codes under message-passing decoding,” IEEE Trans. Inform. Theory, vol. 47, pp. 599–618, Feb. 2001.
IWCSN' 2007, Guilin, China 76
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Density Evolution Algorithm
channel type
∑=
−=vd
k
kk xx
2
1)( λλ
∑=
−=cd
k
kk xx
2
1)( ρρDensity
Evolution Algorithm
iterations
threshold value *σ
a higher threshold value indicates a higher achievableachievable performance of the code
IWCSN' 2007, Guilin, China 77
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Good Degree Distribution Pairs (Rate = 0.5)
T. J. Richardson et al., “Design of capacity-approaching irregular low-density parity-check codes,”IEEE Trans. Inform. Theory, vol. 47, pp. 619–637, Feb.2001.
IWCSN' 2007, Guilin, China 78
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Good Degree Distribution Pairs (Rate = 0.5)
IWCSN' 2007, Guilin, China 79
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
the threshold value , for example indicates the maximum noise power that can be tolerated for error-free communication in AWGN channelsthe threshold value can be achievedachieved if
the message-passing process does not contain any cyclesnumber of iterations tends to infinitycodeword length is infinitecodeword length is infinite
Density Evolution Algorithm*σ
*σ
IWCSN' 2007, Guilin, China 80
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Density Evolution AlgorithmProblems:
optimizing the codes based on DE algorithm is not a simple task
codeword length cannot be infiniteoptimizing the threshold value may give a more complex code
number of connections
∑=
−=vd
k
kk xx
2
1)( λλ ∑=
−=cd
k
kk xx
2
1)( ρρ
vary to maximize the threshold value
IWCSN' 2007, Guilin, China 81
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 5: Review of Complex NetworksPart 5: Review of Complex Networks
IWCSN' 2007, Guilin, China 82
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Basic properties
Path length: the distance between two nodes, which is defined as the number of edges along the shortest path connecting them
IWCSN' 2007, Guilin, China 83
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Basic properties
Betweenness centrality: the fraction of shortest paths going through a given nodeAssortative mixing: preference of high-degree nodes attach to other high-degree nodesDisassortative mixing: preference of high-degree nodes attach to low-degree nodes
IWCSN' 2007, Guilin, China 84
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Examples of Complex Networks
Random NetworksGiven a network with N nodes. Each pair of nodes are connected with a probability of p. Poisson distribution
( )!
kep kk
μμ −
=
μ =
0.1ERp =
IWCSN' 2007, Guilin, China 85
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Examples of Complex Networks
Regular Coupled Networks
high clusteringlarge average path length
Fully-connected Networks
2 2.5 3 3.5 4 4.50
0.2
0.4
0.6
0.8
1
<k>
P(k
)
IWCSN' 2007, Guilin, China 86
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Examples of Complex Networks
Small-World NetworksEach edge of a regular coupled network is re-wired with a probability of phigh clusteringsmall average path length
0WSp =
0.2WSp =
IWCSN' 2007, Guilin, China 87
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Examples of Complex Networks
Scale-Free Networks
IWCSN' 2007, Guilin, China 88
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Examples of Complex NetworksScale-Free (SF) Networks
γii nn −~)Pr( γxxf −~)(
IWCSN' 2007, Guilin, China 89
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Characteristics of Typical Complex Networks
HighUniformLongRegular Coupled Network
-
High
Low
Clustering Coefficient
Power-LawVery ShortScale Free Networks
-Short Small World Networks
PoissonShortRandom Networks
Degree Distribution
Average Distance
(log( ))NΟ
(log(log( )))*NΟ
*The exponent parameter should be valued between 2 and 3.See reference “Scale-Free Networks Are Ultrasmall”, PRL, vol. 90, no. 5
(log( ))NΟ
( )NΟ
Fast to disseminate information
IWCSN' 2007, Guilin, China 90
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 6: ScalePart 6: Scale--free Networks to free Networks to LDPC CodesLDPC Codes
IWCSN' 2007, Guilin, China 91
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Scale-free Networks meet LDPC Codes
Can the “very short distance” property of scale-free network helps passing/spreading messages quickly in the decoding of LDPC codes?
If so, how?
IWCSN' 2007, Guilin, China 92
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Scale-free Networks meet LDPC Codes
??
IWCSN' 2007, Guilin, China 93
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
From Bipartite Graph to Unipartite Graph
IWCSN' 2007, Guilin, China 94
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
From Bipartite Graph to Unipartite Graphpower-law degree distribution
Power-law degree distribution !
IWCSN' 2007, Guilin, China 95
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Typical node degree distribution of the corresponding unipartite graph. Codeword length = 10000 and maximum variable node degree = 20.X. Zheng, F.C.M. Lau and C.K. Tse, " Study of LDPC Codes Built on Scale-Free Networks," Proceedings, NOLTA'06, Bologna, Italy, September 2006, pp. 563-566.
IWCSN' 2007, Guilin, China 96
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Building LDPC Codes From SF Networks
Assume that the variable nodes have the power-law degree distribution and the check nodes obey the Poisson-law degree distribution
Use the (Density Evolution) DE to select the optimized parameters and .
( ) ~P k k γλ
−
γ μ
!)(
lelP
l μ
ρμ −
=
IWCSN' 2007, Guilin, China 97
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Threshold value and average variable node degrees <k> of LDPC codes built from scale-free networks and the optimized ones reported in [1] for an AWGNchannel. Rate equals 0.5.
*σ
[1]
[1] T. J. Richardson et al., “Design of capacity-approaching irregular low-density parity-check codes,”IEEE Trans. Inform. Theory, vol. 47, pp. 619–637, Feb.2001.
IWCSN' 2007, Guilin, China 98
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Typical node degree distribution of the corresponding unipartite graph. Codeword length = 1000 and maximum variable node degree = 15.
X. Zheng, F.C.M. Lau and C.K. Tse, “Error Performance of Short-Block-Length LDPC Code Built on Scale-Free Networks,” Proceedings, The Third Shanghai International Symposium on Nonlinear Sciences and Applications, Shanghai, China, June 2007, pp. 55-57.
IWCSN' 2007, Guilin, China 99
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Building LDPC Codes From SF Networks
Threshold values lower compared with those reported in the literatureAverage variable node degrees <k> lower compared with those reported in the literature
*σ
Which one is better in practice ?
IWCSN' 2007, Guilin, China 100
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
The threshold value can be achievedachieved ifthe message-passing process does not contain any cycles andnumber of iterations tends to infinity andcodeword length is infinitecodeword length is infinite
Threshold Value
*σ
IWCSN' 2007, Guilin, China 101
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
PEG and Enhanced PEG
Progressive Edge-Growth algorithm (PEG)an effective method to construct codes with girth average as large as possible based on the given degree distributions
Enhanced PEG (E-PEG) proposed by usstopping set and the near codeword are also checked after each variable node is added.
X. Y. Hu, E. Eleftheriou and D. M. Arnold, “Regular and irregular progressive edge-growth tanner graphs,” IEEE Trans. Inform. Theory, vol. 51, no. 1, pp. 386–398, 2005.
IWCSN' 2007, Guilin, China 102
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 7: Simulation ResultsPart 7: Simulation Results
IWCSN' 2007, Guilin, China 103
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Block Error RatesBlock length=1008Code rate=0.5Max. no. of iterations = 50
IWCSN' 2007, Guilin, China 104
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Bit Error Rates
Block length=1008Code rate=0.5Max. no. of iterations = 50
IWCSN' 2007, Guilin, China 105
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
0 0.5 1 1.5 2 2.5 310-6
10-5
10-4
10-3
10-2
10-1
100
SNR(dB)
Blo
ck E
rror R
ate
DE10 (PEG), <k>=3.66 DE10 (E-PEG), <k>=3.66
PEG and E-PEG Algorithms
Block length=1008Code rate=0.5Max. no. of iterations = 50
IWCSN' 2007, Guilin, China 106
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
0 0.5 1 1.5 2 2.5 310-8
10-6
10-4
10-2
100
SNR(dB)
Bit
Erro
r Rat
e
DE10 (PEG), <k>=3.66 DE10 (E-PEG), <k>=3.66
PEG and E-PEG Algorithms
Block length=1008Code rate=0.5Max. no. of iterations = 50
IWCSN' 2007, Guilin, China 107
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
0 0.5 1 1.5 2 2.5 310-7
10-6
10-5
10-4
10-3
10-2
10-1
100
SNR(dB)
Blo
ck E
rror R
ate
SF20 (PEG), <k>=3.72 SF20 (E-PEG), <k>=3.72
PEG and E-PEG Algorithms
Block length=1008Code rate=0.5Max. no. of iterations = 50
IWCSN' 2007, Guilin, China 108
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
0 0.5 1 1.5 2 2.5 310-8
10-6
10-4
10-2
100
SNR(dB)
Bit
Erro
r Rat
e
SF20 (PEG), <k>=3.72 SF20 (E-PEG), <k>=3.72
PEG and E-PEG Algorithms
Block length=1008Code rate=0.5Max. no. of iterations = 50
IWCSN' 2007, Guilin, China 109
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Part 8: SummaryPart 8: Summary
IWCSN' 2007, Guilin, China 110
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Summary
Coding for a reliable communicationOperation principles of parity-check codesLow-density-parity-check (LPDC) codes Belief propagation decoding algorithm Density evolution
IWCSN' 2007, Guilin, China 111
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
SummaryLPDC codes with scale-free variable-node-degree distribution achieve very good theoretical threshold (error correction performance)Short LPDC codes built with scale-free variable-node-degree distribution outperform other well-known LPDC codes with similar complexity
IWCSN' 2007, Guilin, China 112
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Some of Our Related Work1. X. Zheng, F.C.M. Lau and C.K. Tse, “Error Performance of Short-Block-Length
LDPC Code Built on Scale-Free Networks,” Proceedings, The Third Shanghai International Symposium on Nonlinear Sciences and Applications, Shanghai, China, June 2007, pp. 55-57.
2. X. Zheng, F.C.M. Lau, Chi K. Tse and S.C. Wong, “Study of Bifurcation Behavior of LDPC Decoders", International Journal of Bifurcation and Chaos, vol. 16, no. 11, pp. 3435-3449, Nov. 2006.
3. X. Zheng, F.C.M. Lau and Chi K. Tse, “Study of LDPC Codes Built on Scale-Free Networks,” Proceedings, International Symposium on Nonlinear Theory and Its Applications (NOLTA'06), Bologna, Italy, September 2006, pp. 563-566.
4. X. Zheng, F.C.M. Lau, C.K. Tse and S.C. Wong, “Techniques for Improving Block Error Rate of LDPC Decoders,” Proceedings, IEEE International Symposium on Circuits and Systems (ISCAS'06), Kos, Greece, May 2006, pp. 2261-2264.
5. X. Zheng, F.C.M. Lau, Chi K. Tse and S.C. Wong, “Study of Nonlinear Dynamics of LDPC Decoders", Proceedings, European Conference on Circuit Theory and Design (ECCTD ‘2005), Dublin, Ireland, August 2005, paper 207. (CD version)
IWCSN' 2007, Guilin, China 113
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Collaborators
Dr Wai-man TAMProf. Chi K. TSEDr Siu C. WONGMiss Xia ZHENG
Constructing ErrorConstructing Error--Correction Codes Correction Codes from Scalefrom Scale--Free NetworksFree Networks
Francis C.M. LauFrancis C.M. Lau
Department of Electronic and Information EngineeringHong Kong Polytechnic University
International Workshop on Complex Systems and Networks 2007 Guilin, China
TTHEHE HHONGONG KKONGONG
PPOLYTECHNICOLYTECHNIC UUNIVERSITYNIVERSITY
Thank You !