making error correcting codes work for flash memory · flash memory summit 2013, santa clara, ca 16...
TRANSCRIPT
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Making Error Correcting Codes Work for FlashMemory
Part II: Algebraic and Graph-based Codes with Applications toFlash Memory
Lara Dolecek
Laboratory for Robust Information Systems (LORIS)Center on Development of Emerging Storage Systems (CoDESS)
Department of Electrical Engineering, UCLA
Flash Memory Summit 2013
Flash Memory Summit 2013, Santa Clara, CA 1 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Outline
1 Preliminaries2 Algebraic codes
Algebra reviewBCH codesAlgebraic codes for Flash
3 Graph-based codesLDPC code constructionIterative DecodingLDPC codes for Flash
4 Advanced Coding ApproachesGraded algebraic codesNon-binary LDPC codes
5 Summary and Outlook
Flash Memory Summit 2013, Santa Clara, CA 2 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Concepts of interest
!"#$%&'( )&#$%&'(
*"+,-(.&//01&( #$%&2$'%( '&-'*&3&%(2$'%( %&#$%&%(.&//01&(
4(5*-/( 4(5*-/("(5*-/( "(5*-/(
6$*/7(890""&:(
A channel code C maps a message m of length k into acodeword c of length n, with n > k (encoder)
Total number of codewords: 2k
Code rate: R = k/n.
Structure of C is used to determine the stored message(decoder).
Flash Memory Summit 2013, Santa Clara, CA 3 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Concepts of interest
Linear block code C of dimension k and codeword length n can berepresented by
a k × n generator matrix G
mG = c
a (n − k)× n parity check matrix H
cHT = 0
G specifies the range space of C and H specifies the nullspace of C .
The two representations are mathematically equivalent.
Flash Memory Summit 2013, Santa Clara, CA 4 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Concepts of interest
Linear block code C of dimension k and codeword length n can berepresented by
a k × n generator matrix G mG = c
a (n − k)× n parity check matrix H
cHT = 0
G specifies the range space of C and H specifies the nullspace of C .
The two representations are mathematically equivalent.
Flash Memory Summit 2013, Santa Clara, CA 4 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Concepts of interest
Linear block code C of dimension k and codeword length n can berepresented by
a k × n generator matrix G mG = c
a (n − k)× n parity check matrix H cHT = 0
G specifies the range space of C and H specifies the nullspace of C .
The two representations are mathematically equivalent.
Flash Memory Summit 2013, Santa Clara, CA 4 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Concepts of interest
Linear block codes can be divided in two categories:
algebraic codes (BCH codes, Hamming codes, Reed-Solomoncodes)
graph-based codes (LDPC codes, Turbo codes)
A good practical channel code should
be able to correct as many transmission errors as possible
be equipped with a simple decoding algorithm
Flash Memory Summit 2013, Santa Clara, CA 5 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
Algebraic Codes
Flash Memory Summit 2013, Santa Clara, CA 6 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
Brief review of finite fields
Suppose q is prime.
GF (q) can be viewed as the set {0, 1, . . . , q − 1}.Operations are performed modulo q.
Example:
GF (5) has elements {0, 1, 2, 3, 4} such that
product 0 1 2 3 40 0 0 0 0 01 0 1 2 3 42 0 2 4 1 33 0 3 1 4 24 0 4 3 2 1
sum 0 1 2 3 40 0 1 2 3 41 1 2 3 4 02 2 3 4 0 13 3 4 0 1 24 4 0 1 2 3
Flash Memory Summit 2013, Santa Clara, CA 7 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
Brief review of finite fields
GF (q) can also be expressed as{α−∞ = 0,α0 = 1,α,α2, . . . ,αq−1}, for suitably chosen α.
Example:
In GF (5): 0 → α−∞, 1 → α0, 2 → α, 3 → α3 and 4 → α2
Consider an element α of GF (q) such that α �= 0 and α �= 1.
Let s be the smallest positive integer such that αs = 1. Then,s is the order of α.
If s = q − 1, then α is called a primitive element of GF (q).
GF (q) is thus generated by powers of a primitive element α.
Flash Memory Summit 2013, Santa Clara, CA 8 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
Brief review of finite fields
We are often interested in the extension field GF (qm) ofGF (q), where q is prime and m is a positive integer.GF (qm) is then {α−∞ = 0,α0 = 1,α,α2, . . . ,αqm−1}, whereα denotes a primitive element of GF (qm) and is a root ofso-called primitive polynomial.
Example:
GF (8) = GF (23).Here, α is a root of the polynomial x3 + x + 1.We then have
α0 = 1α1 = αα2 = α2
α3 = α+ 1
α4 = α2 + αα5 = α2 + α+ 1α6 = α2 + 1α−∞ = 0
Flash Memory Summit 2013, Santa Clara, CA 9 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
BCH code construction
BCH code C is a linear, cyclic code described by a (d − 1)× nparity check matrix H with elements from GF (qm) with α havingorder n:
H =
1 αb α2b · · · α(n−1)b
1 αb+1 α2(b+1) · · · α(n−1)(b+1)
......
... · · ·...
1 αb+d−2 α2(b+d−2) · · · α(n−1)(b+d−2)
b is any (positive) integer and d is integer 2 ≤ d ≤ n.
Minimum distance of C is at least d . The code corrects atleast t = �d−1
2 � errors.
Flash Memory Summit 2013, Santa Clara, CA 10 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
BCH code construction
If α is a primitive element, then the blocklength is n = qm − 1(largest possible).
If b = 1, BCH code is called narrow-sense (simplifies someencoding and decoding operations).
For m = 1, BCH codes are also known as Reed-Solomoncodes.
Flash Memory Summit 2013, Santa Clara, CA 11 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
BCH code properties
A code C is called a cyclic code if all cyclic shifts of acodeword in C are also codewords.
Example:Suppose (0, 1, 0, 1, 1) ↔ x3 + x + 1 is a codeword in C . Thenso are (1, 0, 1, 1, 0), (0, 1, 1, 0, 1), (1, 1, 0, 1, 0) and(1, 0, 1, 0, 1).
Cyclic code is generated by a generator polynomial g(x), suchthat each codeword c corresponds to a polynomialpc(x) = m(x)g(x). All rows of the generator matrix G arecyclic shifts of g(x).BCH code: Each codeword c corresponds to a polynomialpc(x) = m(x)g(x) where g(x) is LCM of(x − αb)(x − αb+1) · · · (x − αb+d−2).
Flash Memory Summit 2013, Santa Clara, CA 12 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
BCH code properties
A code C is called a cyclic code if all cyclic shifts of acodeword in C are also codewords.
Example:Suppose (0, 1, 0, 1, 1) ↔ x3 + x + 1 is a codeword in C . Thenso are (1, 0, 1, 1, 0), (0, 1, 1, 0, 1), (1, 1, 0, 1, 0) and(1, 0, 1, 0, 1).
Cyclic code is generated by a generator polynomial g(x), suchthat each codeword c corresponds to a polynomialpc(x) = m(x)g(x). All rows of the generator matrix G arecyclic shifts of g(x).BCH code: Each codeword c corresponds to a polynomialpc(x) = m(x)g(x) where g(x) is LCM of(x − αb)(x − αb+1) · · · (x − αb+d−2).
Flash Memory Summit 2013, Santa Clara, CA 12 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
BCH code example
Let’s construct a narrow-sense BCH code over GF (8)correcting t = 1 error and of length n = 7.
We consider a primitive element α that satisfiesα3 + α+ 1 = 0. Notice that α7 = 1.
Then,
H =
�1 α α2 α3 α4 α5 α6
1 α2 α4 α6 α8 α10 α12
�
Flash Memory Summit 2013, Santa Clara, CA 13 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
BCH code example
Let’s construct a narrow-sense BCH code over GF (8)correcting t = 1 error and of length n = 7.
We consider a primitive element α that satisfiesα3 + α+ 1 = 0. Notice that α7 = 1.
Then,
H =
�1 α α2 α3 α4 α5 α6
1 α2 α4 α6 α α3 α5
�
Flash Memory Summit 2013, Santa Clara, CA 13 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
BCH code example
We can interpret this code in the binary domain by substituting
1 →
100
α →
010
α2 →
001
α3 →
110
α4 →
011
α5 →
111
α6 →
101
0 →
000
Flash Memory Summit 2013, Santa Clara, CA 14 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
BCH code example
We can then interpret this parity check matrix in the binarydomain as
H =
1 0 0 1 0 1 10 1 0 1 1 1 00 0 1 0 1 1 11 0 0 1 0 1 10 0 1 0 1 1 10 1 1 1 0 0 1
Here H is 6× 7 and has rank 3. This code can correct 1 error.
Flash Memory Summit 2013, Santa Clara, CA 15 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
Decoding BCH codes
Decoding algorithm heavily relies on the algebraic structure of thecode: recall that each codeword polynomial c(x) must have asroots αb, αb+1,..,αb+d−2.
1 Compute the syndromes of the received polynomial r(x)– tellsus which of α’s are not the roots.
2 Based on the syndromes, compute the locations of the errors(system of linear equations).
3 Compute the error values at these location (system ofnon-linear equations that are in the Vandermode form)
4 Based on steps 2 and 3, build error polynomial e(x).
5 Add e(x) to r(x) to produce the estimate of c(x).
Flash Memory Summit 2013, Santa Clara, CA 16 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
Decoding BCH codes
If the system of equations cannot be solved, declare adecoding failure. This is a hard limit on the number ofcorrectable errors.
Implementation can be greatly reduced using theshift-registers viewpoint in the Berlekamp-Massey algorithm.
Flash Memory Summit 2013, Santa Clara, CA 17 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
Performance evaluation
Figure: Theoretical bound for length n = 1023 binary BCH code fordifferent error correction capability t (and different code rate).
4 4.2 4.4 4.6 4.8 5 5.2
10−3
10−2
10−1
100
SNR(dB)
FER
t=10, rate= 0.9022t=11, rate= 0.8925t=12, rate= 0.8827t=13, rate= 0.8729t=14, rate= 0.8631t=15, rate= 0.8534t=16, rate= 0.8436
Flash Memory Summit 2013, Santa Clara, CA 18 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebra reviewBCH codesAlgebraic codes for Flash
Graph-Based Codes
Flash Memory Summit 2013, Santa Clara, CA 19 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Low Density Parity Check (LDPC) Codes
Definition 1: LDPC code
An LDPC block code C is a linear block code whose parity-checkmatrix H has a small number of ones in each row and column.
Invented by Gallager in 1963 but were all but forgotten untillate 1990’s.
In the limit of very large block-lengths LDPC codes are knownto approach the Shannon limit (i.e., the highest rate at whichthe code can be designed that guarantees reliablecommunication)
LDPC codes are amenable to low-complexity iterativedecoding.
Flash Memory Summit 2013, Santa Clara, CA 20 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
An Example
LDPC code described by the sparse parity check matrix H:
H =
1 0 0 1 0 0 1 0 00 1 0 0 1 0 0 1 00 0 1 0 0 1 0 0 11 0 0 0 0 1 0 1 00 1 0 1 0 0 0 0 10 0 1 0 1 0 1 0 0
Matrix H has 9 columns and 6 rows.
There are 9 coded bits and 6 parity-check equations.Each coded bit participates �=2 parity-check equations and eachparity-check equation contains r = 3 coded bits.
Flash Memory Summit 2013, Santa Clara, CA 21 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
An Example
LDPC code described by the sparse parity check matrix H:
H =
1 0 0 1 0 0 1 0 00 1 0 0 1 0 0 1 00 0 1 0 0 1 0 0 11 0 0 0 0 1 0 1 00 1 0 1 0 0 0 0 10 0 1 0 1 0 1 0 0
Matrix H has 9 columns and 6 rows.There are 9 coded bits and 6 parity-check equations.Each coded bit participates �=2 parity-check equations and eachparity-check equation contains r = 3 coded bits.
Flash Memory Summit 2013, Santa Clara, CA 21 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
LDPC Preliminaries
Definition 3: Tanner graph
A Tanner graph of a code C with a parity check matrix H is thebipartite graph such that:
each coded symbol i is represented by a variable node vi ,
each parity-check equation j is represented by a check node cj ,
there exists an edge between a variable node and a checknode if and only if H(j , i) = 1.
Flash Memory Summit 2013, Santa Clara, CA 22 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
An Example
LDPC code: parity check matrix H and its Tanner graph
H =
100100
010010
001001
100010
010001
001100
100001
010100
001010
c1c2c3c4c5c6
v1 v2 v3 v4 v5 v6 v7 v8 v9
Flash Memory Summit 2013, Santa Clara, CA 23 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
An Example
LDPC code: parity check matrix H and its Tanner graph
H =
100100
010010
001001
100010
010001
001100
100001
010100
001010
c1c2c3c4c5c6
v1 v2 v3 v4 v5 v6 v7 v8 v9
Flash Memory Summit 2013, Santa Clara, CA 23 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
An Example
LDPC code: parity check matrix H and its Tanner graph
H =
100100
010010
001001
100010
010001
001100
100001
010100
001010
c1c2c3c4c5c6
v1 v2 v3 v4 v5 v6 v7 v8 v9
Flash Memory Summit 2013, Santa Clara, CA 23 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Message Passing Decoding
Message-passing (belief propagation) is an iterative decodingalgorithm that operates on the Tanner graph of the code.In each iteration of the algorithm:
1 (bit-to-check) Each variable node sends a message to eachcheck node it is connected to,
2 (check processing) Each check node then computes theconsistency of incoming messages,
3 (check-to-bit) Each check node then sends a message to eachvariable node it is connected to,
4 (bit processing) Each variable node (coded symbol) updatesits value.
Flash Memory Summit 2013, Santa Clara, CA 24 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Message Passing Decoding
Message-passing (belief propagation) is an iterative decodingalgorithm that operates on the Tanner graph of the code.In each iteration of the algorithm:
1 (bit-to-check) Each variable node sends a message to eachcheck node it is connected to,
2 (check processing) Each check node then computes theconsistency of incoming messages,
3 (check-to-bit) Each check node then sends a message to eachvariable node it is connected to,
4 (bit processing) Each variable node (coded symbol) updatesits value.
Flash Memory Summit 2013, Santa Clara, CA 24 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Message Passing Decoding
Message-passing (belief propagation) is an iterative decodingalgorithm that operates on the Tanner graph of the code.In each iteration of the algorithm:
1 (bit-to-check) Each variable node sends a message to eachcheck node it is connected to,
2 (check processing) Each check node then computes theconsistency of incoming messages,
3 (check-to-bit) Each check node then sends a message to eachvariable node it is connected to,
4 (bit processing) Each variable node (coded symbol) updatesits value.
Flash Memory Summit 2013, Santa Clara, CA 24 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Message Passing Decoding
Message-passing (belief propagation) is an iterative decodingalgorithm that operates on the Tanner graph of the code.In each iteration of the algorithm:
1 (bit-to-check) Each variable node sends a message to eachcheck node it is connected to,
2 (check processing) Each check node then computes theconsistency of incoming messages,
3 (check-to-bit) Each check node then sends a message to eachvariable node it is connected to,
4 (bit processing) Each variable node (coded symbol) updatesits value.
Flash Memory Summit 2013, Santa Clara, CA 24 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Message Passing Decoding
Message-passing (belief propagation) is an iterative decodingalgorithm that operates on the Tanner graph of the code.In each iteration of the algorithm:
1 (bit-to-check) Each variable node sends a message to eachcheck node it is connected to,
2 (check processing) Each check node then computes theconsistency of incoming messages,
3 (check-to-bit) Each check node then sends a message to eachvariable node it is connected to,
4 (bit processing) Each variable node (coded symbol) updatesits value.
Flash Memory Summit 2013, Santa Clara, CA 24 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Message Passing Decoding
Passed messages can be either
Hard decisions: 0 or 1
Soft decisions/likelihoods: real numbers
Flash Memory Summit 2013, Santa Clara, CA 25 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
An Example
!"#$%&'( )&#$%&'(
*"+,-(.&//01&( #$%&2$'%( '&-'*&3&%(2$'%( %&#$%&%(.&//01&(
4( 5(4464( 4664(
7$*/8(9:0""&;(
Message m Codeword y m1 y1y2y3y4
0 ! 0 0 0 0 1 ! 1 1 0 1
y1 y2 y3 y4
y1+y2+y3 = 0 y2+y3 + y4= 0 y1+y3+y4 = 0
Flash Memory Summit 2013, Santa Clara, CA 26 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Message Passing Decoding
Bit-flipping algorithm
Flash Memory Summit 2013, Santa Clara, CA 27 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Received Codeword
1 0 0 1
y1+y2+y3 y1+y3+y4 y2+y3+y4
Flash Memory Summit 2013, Santa Clara, CA 28 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Bit-to-Check Messages
1 0 0 1
1+y2+y3 1+y3+y4 y2+y3+y4
Flash Memory Summit 2013, Santa Clara, CA 29 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Bit-to-Check Messages
1 0 0 1
1+0+y3 1+y3+y4 0+y3+y4
Flash Memory Summit 2013, Santa Clara, CA 29 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Check Processing
1 0 0 1
1+0+0=1 ?? 1+0+y4 0+0+y4
Flash Memory Summit 2013, Santa Clara, CA 30 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Check Processing
1 0 0 1
1+0+0=1 ?? 1+0+1= 0 0+0+1= 1 ??
Flash Memory Summit 2013, Santa Clara, CA 30 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Check-to-Bit Messages
1 0 0 1
1+0+0=1 ?? 1+0+1= 0 0+0+1= 1 ??
Flip, Flip, Flip,
Flash Memory Summit 2013, Santa Clara, CA 31 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Check-to-Bit Messages
1 0 0 1
1+0+0=1 ?? 1+0+1= 0 0+0+1= 1 ??
Flip, Stay Flip, Stay, Flip, Stay,
Flash Memory Summit 2013, Santa Clara, CA 31 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Check-to-Bit Messages
1 0 0 1
1+0+0=1 ?? 1+0+1= 0 0+0+1= 1 ??
Flip, Stay Flip, Flip Flip, Stay, Flip Stay, Flip
Flash Memory Summit 2013, Santa Clara, CA 31 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Bit Processing
1 0 0 1
1+0+0=1 ?? 1+0+1= 0 0+0+1= 1 ??
Flip, Stay Flip, Flip Flip, Stay, Flip Stay, Flip
Flash Memory Summit 2013, Santa Clara, CA 32 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Bit Processing
1 1 0 1
y1+y2+y3 y1+y3+y4 y2+y3+y4
Flash Memory Summit 2013, Santa Clara, CA 32 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Bit-to-Check Messages
1 1 0 1
1+y2+y3 1+y3+y4 y2+y3+y4
Flash Memory Summit 2013, Santa Clara, CA 33 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Bit-to-Check Messages
1 1 0 1
1+1+y3 1+y3+y4 1+y3+y4
Flash Memory Summit 2013, Santa Clara, CA 33 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Bit-to-Check Messages
1 1 0 1
1+1+0 = 0 1+0+y4 1+0+y4
Flash Memory Summit 2013, Santa Clara, CA 33 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Check Processing
1 1 0 1
1+1+0 = 0 1+0+1 = 0 1+0+1 = 0
Flash Memory Summit 2013, Santa Clara, CA 34 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Check-to-Bit Messages
1 1 0 1Stay, Stay, Stay,
1+1+0 = 0 1+0+1 = 0 1+0+1 = 0
Flash Memory Summit 2013, Santa Clara, CA 35 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Check-to-Bit Messages
1 1 0 1Stay, Stay, Stay,
1+1+0 = 0 1+0+1 = 0 1+0+1 = 0
Flash Memory Summit 2013, Santa Clara, CA 35 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Check-to-Bit Messages
1 1 0 1Stay, Stay Stay, Stay Stay, Stay, Stay Stay, Stay
1+1+0= 0 1+0+1= 0 1+0+1= 0
Flash Memory Summit 2013, Santa Clara, CA 36 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Bit Processing
1 1 0 1
1+1+0 = 0 1+0+1 = 0 1+0+1 = 0
Decoded Codeword
Flash Memory Summit 2013, Santa Clara, CA 37 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Soft Iterative Decoding
Improved variants of message passing algorithm use softinformation as messages, i.e., log-likelihood ratio L = log P(xi=0|yi )
P(xi=1|yi ) .
Sum-product algorithm (SPA) [1,2]
bit-to-check L(vi → cj) =�j �∈N(i)\j L(c
�j → vi ) + Lint(vi )
check-to-bit L(cj → vi ) =
Φ−1��
i �∈N(j)\i Φ(|L(v �i → cj)|)�
i �∈N(j)\i sgn(L(v�i → cj))
�
where Φ(x) = − log(tanh(x/2))
Min-sum algorithm (MSA) [3]
check-to-bit L(cj → vi ) =mini �∈N(j)\i |L(v �i → cj)|
�i �∈N(j)\i sgn(L(v
�i → cj))
[1] R. Gallager, MIT Press, 1963.[2] T. Richardson and R. Urbanke, IEEE Trans. on Info. Theory, 2001.[3] M. P. C. Fossorier, M. Mihaljevic, and H. Imai, IEEE Trans. on Comm., 1999.
Flash Memory Summit 2013, Santa Clara, CA 38 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Soft Iterative Decoding
Improved variants of message passing algorithm use softinformation as messages, i.e., log-likelihood ratio L = log P(xi=0|yi )
P(xi=1|yi ) .
Sum-product algorithm (SPA) [1,2]bit-to-check L(vi → cj) =�
j �∈N(i)\j L(c�j → vi ) + Lint(vi )
check-to-bit L(cj → vi ) =
Φ−1��
i �∈N(j)\i Φ(|L(v �i → cj)|)�
i �∈N(j)\i sgn(L(v�i → cj))
�
where Φ(x) = − log(tanh(x/2))Min-sum algorithm (MSA) [3]
check-to-bit L(cj → vi ) =mini �∈N(j)\i |L(v �i → cj)|
�i �∈N(j)\i sgn(L(v
�i → cj))
[1] R. Gallager, MIT Press, 1963.[2] T. Richardson and R. Urbanke, IEEE Trans. on Info. Theory, 2001.[3] M. P. C. Fossorier, M. Mihaljevic, and H. Imai, IEEE Trans. on Comm., 1999.
Flash Memory Summit 2013, Santa Clara, CA 38 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Soft Decoding
Bit values 1 1 0 1
Flash Memory Summit 2013, Santa Clara, CA 39 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Soft Decoding
Bit values 1 1 0 1
Values using BPSK -‐1 -‐1 +1 -‐1
Flash Memory Summit 2013, Santa Clara, CA 39 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Soft Decoding
Bit values 1 1 0 1
Values from channel -‐1.1 0.1 1.2 -‐0.9
Values using BPSK -‐1 -‐1 +1 -‐1
Flash Memory Summit 2013, Santa Clara, CA 39 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Soft Decoding
Bit values 1 1 0 1
Beliefs -‐2.2 0.2 2.4 -‐1.8
Values from channel -‐1.1 0.1 1.2 -‐0.9
Values using BPSK -‐1 -‐1 +1 -‐1
( ) =1 2/2 2
+1 2/2 2 = 22
We assume = 1.
( )
Flash Memory Summit 2013, Santa Clara, CA 39 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Soft Decoding
Bit values 1 1 0 1
Beliefs -‐2.2 0.2 2.4 -‐1.8
Values from channel -‐1.1 0.1 1.2 -‐0.9
0.1666
Values using BPSK -‐1 -‐1 +1 -‐1
0.2
2.4
= 2 tanh 1 tanh 12
Flash Memory Summit 2013, Santa Clara, CA 39 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Soft Decoding
Bit values 1 1 0 1
Beliefs -‐2.2 0.2 2.4 -‐1.8
Values from channel -‐1.1 0.1 1.2 -‐0.9
0.1666
-‐1.3774
-‐1.6119
-‐1.3774
-‐0.1599
1.3051
-‐0.1430
-‐1.6119
0.1666
Values using BPSK -‐1 -‐1 +1 -‐1
= 2 tanh 1 tanh 12
Flash Memory Summit 2013, Santa Clara, CA 39 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Soft Decoding
Beliefs -‐2.2 0.2 2.4 -‐1.8
Values from channel -‐1.1 0.1 1.2 -‐0.9
-‐3.4108 -‐2.7893 3.5621 -‐3.2453
Values using BPSK -‐1 -‐1 +1 -‐1
Bit values 1 1 0 1
0.1666
-‐1.3774
-‐1.6119
-‐1.3774
-‐0.1599
1.3051
-‐0.1430
-‐1.6119
0.1666
= ( ) +
Flash Memory Summit 2013, Santa Clara, CA 39 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Soft Decoding
Beliefs -‐2.2 0.2 2.4 -‐1.8
Values from channel -‐1.1 0.1 1.2 -‐0.9
-‐3.4108 -‐2.7893 3.5621 -‐3.2453
Values using BPSK -‐1 -‐1 +1 -‐1
Bit values 1 1 0 1
0.1666
-‐1.3774
-‐1.6119
-‐1.3774
-‐0.1599
1.3051
-‐0.1430
-‐1.6119
0.1666
All variable nodes are decoded to correct bit value.
1 1 0 1
Flash Memory Summit 2013, Santa Clara, CA 39 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Performance evaluation
Figure: Bnary LDPC codes vs. BCH codes performance comparison forAWGN channel. Code rate is 0.9, block length is 1000 bits. BCH codecorrects 13 errors.
3.5 4 4.5 5
1.0E−4
1.0E−3
1.0E−2
1.0E−1
1
SNR (dB)
FER
Binary LDPCBCH
Flash Memory Summit 2013, Santa Clara, CA 40 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Extracting soft information in SLC Flash
In Flash, levels are represented by distributions
1 read compares against a single threshold
!!! !! !
0 1 !!! ! !
0 1
! !
0 1
Flash Memory Summit 2013, Santa Clara, CA 41 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Extracting soft information in SLC Flash
Idea: multiple word line reads
2 reads compare against two thresholds
Maximize mutual information of the induced channel todetermine the best thresholds (here q and −q)
Flash Memory Summit 2013, Santa Clara, CA 42 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Extracting soft information in SLC Flash
Idea: multiple word line reads
2 reads compare against two thresholds
!!! !! !
0 1 !!! ! !
0 1
! !
0 1
Maximize mutual information of the induced channel todetermine the best thresholds (here q and −q)
Flash Memory Summit 2013, Santa Clara, CA 42 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Extracting soft information in SLC Flash
Idea: multiple word line reads
2 reads compare against two thresholds
!!! !! !
0 1 !!! ! !
0 1
! !
0 1
0 0
1 1
e
p1
p1
p2p3
p2
p3
0
1
(a) Two reads
Fig. 3. Equivalent discrete memoryless channel models for SLC with (a) two reads and (b) three reads with distinct word-line
voltages.
A. PAM-Gaussian Model
This subsection describes how to select word-line voltages to achieve MMI in the context of
a simple model of the flash cell read channel as PAM transmission with Gaussian noise. MMI
word-line voltage selection is presented using three examples: SLC with two reads, SLC with
three reads, and 4-level MLC with six reads.
For SLC flash memory, each cell can store one bit of information. Fig. 2 shows the model of
the threshold voltage distribution as a mixture of two identically distributed Gaussian random
variables. When either a “0” or “1” is written to the cell, the threshold voltage is modeled as a
Gaussian random variable with variance N0/2 and either mean −√Es (for “1” ) or mean +
√Es
(for “0” ).
For SLC with two reads using two different word line voltages, which should be symmetric
(shown as q and −q in Fig. 2), the threshold voltage is quantized according to three regions
shown in Fig. 2: the white region, the black region, and the union of the light and dark gray
regions (which essentially corresponds to an erasure). This quantization produces the effective
discrete memoryless channel (DMC) model as shown in Fig. 3 (a) with input X ∈ {0, 1} and
output Y ∈ {0, e, 1}.
Assuming X is equally likely to be 0 or 1, the MI I(X;Y ) between the input X and output
Maximize mutual information of the induced channel todetermine the best thresholds (here q and −q)
Flash Memory Summit 2013, Santa Clara, CA 42 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Extracting soft information in SLC Flash
Idea: multiple word line reads
2 reads compare against two thresholds
!!! !! !
0 1 !!! ! !
0 1
! !
0 1
0 0
1 1
e
p1
p1
p2p3
p2
p3
0
1
(a) Two reads
Fig. 3. Equivalent discrete memoryless channel models for SLC with (a) two reads and (b) three reads with distinct word-line
voltages.
A. PAM-Gaussian Model
This subsection describes how to select word-line voltages to achieve MMI in the context of
a simple model of the flash cell read channel as PAM transmission with Gaussian noise. MMI
word-line voltage selection is presented using three examples: SLC with two reads, SLC with
three reads, and 4-level MLC with six reads.
For SLC flash memory, each cell can store one bit of information. Fig. 2 shows the model of
the threshold voltage distribution as a mixture of two identically distributed Gaussian random
variables. When either a “0” or “1” is written to the cell, the threshold voltage is modeled as a
Gaussian random variable with variance N0/2 and either mean −√Es (for “1” ) or mean +
√Es
(for “0” ).
For SLC with two reads using two different word line voltages, which should be symmetric
(shown as q and −q in Fig. 2), the threshold voltage is quantized according to three regions
shown in Fig. 2: the white region, the black region, and the union of the light and dark gray
regions (which essentially corresponds to an erasure). This quantization produces the effective
discrete memoryless channel (DMC) model as shown in Fig. 3 (a) with input X ∈ {0, 1} and
output Y ∈ {0, e, 1}.
Assuming X is equally likely to be 0 or 1, the MI I(X;Y ) between the input X and output
Maximize mutual information of the induced channel todetermine the best thresholds (here q and −q)
Flash Memory Summit 2013, Santa Clara, CA 42 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Extracting soft information in SLC Flash
Idea: multiple word line reads
3 reads compare against three thresholds
Maximize mutual information of the induced channel todetermine the best thresholds (here q1, −q1 and 0)
Flash Memory Summit 2013, Santa Clara, CA 43 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
Extracting soft information in SLC Flash
Idea: multiple word line reads
3 reads compare against three thresholds
!!! !! !
0 1 !!! ! !
0 1
! !
0 1
0
1
e
00
10
p1
p1
p2p3
p2
p3
01
11
p4
p4
0
1
(b) Three reads
Fig. 3. Equivalent discrete memoryless channel models for SLC with (a) two reads and (b) three reads with distinct word-line
voltages.
A. PAM-Gaussian Model
This subsection describes how to select word-line voltages to achieve MMI in the context of
a simple model of the flash cell read channel as PAM transmission with Gaussian noise. MMI
word-line voltage selection is presented using three examples: SLC with two reads, SLC with
three reads, and 4-level MLC with six reads.
For SLC flash memory, each cell can store one bit of information. Fig. 2 shows the model of
the threshold voltage distribution as a mixture of two identically distributed Gaussian random
variables. When either a “0” or “1” is written to the cell, the threshold voltage is modeled as a
Gaussian random variable with variance N0/2 and either mean −√Es (for “1” ) or mean +
√Es
(for “0” ).
For SLC with two reads using two different word line voltages, which should be symmetric
(shown as q and −q in Fig. 2), the threshold voltage is quantized according to three regions
shown in Fig. 2: the white region, the black region, and the union of the light and dark gray
regions (which essentially corresponds to an erasure). This quantization produces the effective
discrete memoryless channel (DMC) model as shown in Fig. 3 (a) with input X ∈ {0, 1} and
output Y ∈ {0, e, 1}.
Assuming X is equally likely to be 0 or 1, the MI I(X;Y ) between the input X and output
Maximize mutual information of the induced channel todetermine the best thresholds (here q1, −q1 and 0)
Flash Memory Summit 2013, Santa Clara, CA 43 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
LDPC vs. BCH code performance
Figure: Performance comparison for 0.9-rate LDPC and BCH codes oflength n = 9100.
10−2 10−110−6
10−5
10−4
10−3
10−2
10−1
100Frame Error Rate vs. Raw Bit Error Rate (BPSK)
Channel Bit Error Probability
Fram
e Er
ror R
ate
BCH theoryLDPC softLDPC 2−bit (3 reads)LDPC 1.6−bit (2 reads)LDPC 1−bit (1 read)Capacity softCapacity 2−bitCapacity 1.6−bitCapacity 1−bit
Fig. 10: Simulation results for SLC.
10−2 10−110−6
10−5
10−4
10−3
10−2
10−1
100Frame Error Rate vs. Raw Bit Error Rate (4PAM)
Channel Bit Error Probability
Fram
e Er
ror R
ate
BCH theoryLDPC softLDPC 2.8−bit (6 reads)LDPC 2−bit (3 reads)Capacity softCapacity 2.8−bitCapacity 2−bit
Fig. 11: Simulation results for 4-level MLC.
signal-to-noise ratio Es/N0 in Figure 12 for SLC, whereeach Es/N0 corresponds to an equivalent raw bit error ratein Figure 10.Of course the BCH code will also benefit from the use
of soft information. However, we were unable to performsimulations of a BCH decoder utilizing soft information (suchas erasures) for inclusion in this paper.
2 3 4 5 6 7 810−6
10−5
10−4
10−3
10−2
10−1
100Frame Error Rate vs. SNR (BPSK)
Es/N0 (dB)
Fram
e Er
ror R
ate
BCH theoryLDPC softLDPC 2−bit (3 reads)LDPC 1.6−bit (2 reads)LDPC 1−bit (1 read)Capacity softCapacity 2−bitCapacity 1.6−bitCapacity 1−bit
Fig. 12: Simulation results for SLC.
V. CONCLUSION
This paper explores the benefit of using soft information inan LDPC decoder for flash memory. Using a small amountof soft information improves the performance of LDPC codessignificantly and demonstrates a clear performance advantageover conventional BCH codes. In order to maximize theperformance benefit of the soft information, we develop aword-line-voltages-selection method that maximizes the mu-tual information between the input and output of the equivalentread channel. Possible directions for future research includeextending these results to more precise channel models, thedesign of better high-rate LDPC codes for flash memory, andthe analysis of the corresponding error-floor properties.
REFERENCES
[1] Y. Li, S. Lee, and et al. A 16 Gb 3b/cell NAND Flash Memory in 56nmWith 8MB/s Write Rate. In Proc. of ISSCC, pages 506–632, Feb. 2008.
[2] C. Trinh, N. Shibata, and et al. A 5.6MB/s 64 Gb 4b/Cell NAND FlashMemory in 43nm CMOS. In Proc. of ISSCC, page 246, Feb. 2009.
[3] J.-D. Lee, S.-H. Hur, and J.-D. Choi. Effects of floating-gate interferenceon NAND flash memory cell operation. IEEE Electron Device Letters,23(5):264–266, May 2002.
[4] T. Richardson, M. Shokrollahi, and R. Urbanke. Design of capacity-approaching irregular low-density parity-check codes. IEEE Trans.Inform. Theory, 47(2):616–637, Feb. 2001.
[5] R. Bez, E. Camerlenghi, A. Modelli, and A. Visconti. Introduction toFlash Memory. Proc. IEEE, 91(4), April 2003.
[6] Y. Maeda and K. Haruhiko. Error Control Coding for Multilevel CellFlash Memories Using Nonbinary Low-Density Parity-Check Codes. In24th IEEE Int. Symp. on Defect and Fault Tolerance in VLSI Systems,Chicago, IL, Oct. 2009.
[7] S. Li and T. Zhang. Improving Multi-Level NAND Flash MemoryStorage Reliability Using Concatenated BCH-TCM Coding. IEEE Trans.VLSI Systems, 18(10):1412–1420, Oct. 2010.
[8] G. Dong, N. Xie, and T. Zhang. On the Use of Soft-Decision Error-Correcting Codes in NAND Flash Memory. IEEE Trans. Circ. and Sys.,58(2):429–439, Feb. 2011.
[9] T. Tian, C. Jones, J. D. Villasenor, and R. D. Wesel. Selective Avoidanceof Cycles in Irregular LDPC Code Construction. IEEE Trans. Comm.,52(8):1242–1247, Aug. 2004.
[10] X.-Y. Hu, E. Eleftheriou, and D.-M. Arnold. Progressive edge-growthTanner graphs. In Proc. IEEE GLOBECOM, San Antonio, TX, Feb.2001.
[11] T. Richardson. Error-floors of LDPC codes. In Proc. 41st AnnualAllerton Conf., Monticello, IL, Oct. 2003.
[12] J. Wang, L. Dolecek, and R.D. Wesel. Controlling LDPC Absorbing Setsvia the Null Space of the Cycle Consistency Matrix. In Proc. IEEE Int.Conf. on Comm. (ICC), Kyoto, Japan, June. 2011.
[13] J. Wang, L. Dolecek, and R. D Wesel. LDPC Absorbing Sets, the NullSpace of the Cycle Consistency Matrix, and Tanner’s Constructions. InProc. Info. Theory and Appl. Workshop, San Diego, CA, Feb. 2011.
[14] M. Ivkovic, S. K. Chilappagari, and B. Vasic. Eliminating trapping setsin low-density parity-check codes by using Tanner graph covers. IEEETrans. Inform. Theory, 54(8):3763–3768, Aug. 2008.
[15] D. V. Nguyen, B. Vasic, and M. Marcellin. Structured LDPC Codesfrom Permutation Matrices Free of Small Trapping Sets. In Proc. IEEEInfo. Theory Workshop (ITW), Dublin, Ireland, Sept. 2010.
[16] Q. Huang, Q. Diao, S. Lin, and K. Abdel-Ghaffar. Cyclic and quasi-cyclic LDPC codes: new developments. In Proc. Info. Theory and Appl.Workshop, San Diego, CA, Feb. 2011.
[17] A. Ramamoorthy and R. D. Wesel. Construction of Short Block LengthIrregular LDPC Codes. In Proc. IEEE Int. Conf. on Comm. (ICC), Paris,France, June. 2004.
Caution: AWGN-optimized LDPC codes may not be the bestfor the quantized Flash channel !
Flash Memory Summit 2013, Santa Clara, CA 44 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
LDPC code constructionIterative DecodingLDPC codes for Flash
LDPC vs. BCH code performance
Figure: Performance comparison for 0.9-rate LDPC and BCH codes oflength n = 9100.
10−2 10−110−6
10−5
10−4
10−3
10−2
10−1
100Frame Error Rate vs. Raw Bit Error Rate (BPSK)
Channel Bit Error Probability
Fram
e Er
ror R
ate
BCH theoryLDPC softLDPC 2−bit (3 reads)LDPC 1.6−bit (2 reads)LDPC 1−bit (1 read)Capacity softCapacity 2−bitCapacity 1.6−bitCapacity 1−bit
Fig. 10: Simulation results for SLC.
10−2 10−110−6
10−5
10−4
10−3
10−2
10−1
100Frame Error Rate vs. Raw Bit Error Rate (4PAM)
Channel Bit Error Probability
Fram
e Er
ror R
ate
BCH theoryLDPC softLDPC 2.8−bit (6 reads)LDPC 2−bit (3 reads)Capacity softCapacity 2.8−bitCapacity 2−bit
Fig. 11: Simulation results for 4-level MLC.
signal-to-noise ratio Es/N0 in Figure 12 for SLC, whereeach Es/N0 corresponds to an equivalent raw bit error ratein Figure 10.Of course the BCH code will also benefit from the use
of soft information. However, we were unable to performsimulations of a BCH decoder utilizing soft information (suchas erasures) for inclusion in this paper.
2 3 4 5 6 7 810−6
10−5
10−4
10−3
10−2
10−1
100Frame Error Rate vs. SNR (BPSK)
Es/N0 (dB)
Fram
e Er
ror R
ate
BCH theoryLDPC softLDPC 2−bit (3 reads)LDPC 1.6−bit (2 reads)LDPC 1−bit (1 read)Capacity softCapacity 2−bitCapacity 1.6−bitCapacity 1−bit
Fig. 12: Simulation results for SLC.
V. CONCLUSION
This paper explores the benefit of using soft information inan LDPC decoder for flash memory. Using a small amountof soft information improves the performance of LDPC codessignificantly and demonstrates a clear performance advantageover conventional BCH codes. In order to maximize theperformance benefit of the soft information, we develop aword-line-voltages-selection method that maximizes the mu-tual information between the input and output of the equivalentread channel. Possible directions for future research includeextending these results to more precise channel models, thedesign of better high-rate LDPC codes for flash memory, andthe analysis of the corresponding error-floor properties.
REFERENCES
[1] Y. Li, S. Lee, and et al. A 16 Gb 3b/cell NAND Flash Memory in 56nmWith 8MB/s Write Rate. In Proc. of ISSCC, pages 506–632, Feb. 2008.
[2] C. Trinh, N. Shibata, and et al. A 5.6MB/s 64 Gb 4b/Cell NAND FlashMemory in 43nm CMOS. In Proc. of ISSCC, page 246, Feb. 2009.
[3] J.-D. Lee, S.-H. Hur, and J.-D. Choi. Effects of floating-gate interferenceon NAND flash memory cell operation. IEEE Electron Device Letters,23(5):264–266, May 2002.
[4] T. Richardson, M. Shokrollahi, and R. Urbanke. Design of capacity-approaching irregular low-density parity-check codes. IEEE Trans.Inform. Theory, 47(2):616–637, Feb. 2001.
[5] R. Bez, E. Camerlenghi, A. Modelli, and A. Visconti. Introduction toFlash Memory. Proc. IEEE, 91(4), April 2003.
[6] Y. Maeda and K. Haruhiko. Error Control Coding for Multilevel CellFlash Memories Using Nonbinary Low-Density Parity-Check Codes. In24th IEEE Int. Symp. on Defect and Fault Tolerance in VLSI Systems,Chicago, IL, Oct. 2009.
[7] S. Li and T. Zhang. Improving Multi-Level NAND Flash MemoryStorage Reliability Using Concatenated BCH-TCM Coding. IEEE Trans.VLSI Systems, 18(10):1412–1420, Oct. 2010.
[8] G. Dong, N. Xie, and T. Zhang. On the Use of Soft-Decision Error-Correcting Codes in NAND Flash Memory. IEEE Trans. Circ. and Sys.,58(2):429–439, Feb. 2011.
[9] T. Tian, C. Jones, J. D. Villasenor, and R. D. Wesel. Selective Avoidanceof Cycles in Irregular LDPC Code Construction. IEEE Trans. Comm.,52(8):1242–1247, Aug. 2004.
[10] X.-Y. Hu, E. Eleftheriou, and D.-M. Arnold. Progressive edge-growthTanner graphs. In Proc. IEEE GLOBECOM, San Antonio, TX, Feb.2001.
[11] T. Richardson. Error-floors of LDPC codes. In Proc. 41st AnnualAllerton Conf., Monticello, IL, Oct. 2003.
[12] J. Wang, L. Dolecek, and R.D. Wesel. Controlling LDPC Absorbing Setsvia the Null Space of the Cycle Consistency Matrix. In Proc. IEEE Int.Conf. on Comm. (ICC), Kyoto, Japan, June. 2011.
[13] J. Wang, L. Dolecek, and R. D Wesel. LDPC Absorbing Sets, the NullSpace of the Cycle Consistency Matrix, and Tanner’s Constructions. InProc. Info. Theory and Appl. Workshop, San Diego, CA, Feb. 2011.
[14] M. Ivkovic, S. K. Chilappagari, and B. Vasic. Eliminating trapping setsin low-density parity-check codes by using Tanner graph covers. IEEETrans. Inform. Theory, 54(8):3763–3768, Aug. 2008.
[15] D. V. Nguyen, B. Vasic, and M. Marcellin. Structured LDPC Codesfrom Permutation Matrices Free of Small Trapping Sets. In Proc. IEEEInfo. Theory Workshop (ITW), Dublin, Ireland, Sept. 2010.
[16] Q. Huang, Q. Diao, S. Lin, and K. Abdel-Ghaffar. Cyclic and quasi-cyclic LDPC codes: new developments. In Proc. Info. Theory and Appl.Workshop, San Diego, CA, Feb. 2011.
[17] A. Ramamoorthy and R. D. Wesel. Construction of Short Block LengthIrregular LDPC Codes. In Proc. IEEE Int. Conf. on Comm. (ICC), Paris,France, June. 2004.Caution: AWGN-optimized LDPC codes may not be the best
for the quantized Flash channel !
Flash Memory Summit 2013, Santa Clara, CA 44 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Graded algebraic codesNon-binary LDPC codes
Advanced Coding Approaches
Flash Memory Summit 2013, Santa Clara, CA 45 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Graded algebraic codesNon-binary LDPC codes
Graded algebraic codes
Motivation: Raw error rate for TLC flash
0 500 1000 1500 2000 2500 3000 3500 4000 4500 500010−5
10−4
10−3
10−2
P/E Cycles
Erro
r Rat
e
Error Rates for TLC Flash
LSBCSBMSBSymbol Error Rate
Student Version of MATLAB
LSB: least significant bitCSB: center significant bitMSB: most significant bit
Table: Mapping between VoltageLevels and Triple-bit Words
Voltage Level Triple-bit Word0 1111 1102 1003 1014 0015 0006 0107 011
Flash Memory Summit 2013, Santa Clara, CA 46 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Graded algebraic codesNon-binary LDPC codes
Error patterns within a TLC cell
Number of bits in symbol that err Percentage of errors1 0.96172 0.03143 0.0069
Standard error-correction codes are designed to correct allsymbol–to–symbol errors and do not differentiate among theseerrors.
Usage of standard codes: overkill in terms of redundancy, ascertain symbol–to–symbol errors are not as important.
Flash Memory Summit 2013, Santa Clara, CA 47 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Graded algebraic codesNon-binary LDPC codes
Error patterns within a TLC cell
Number of bits in symbol that err Percentage of errors1 0.96172 0.03143 0.0069
Standard error-correction codes are designed to correct allsymbol–to–symbol errors and do not differentiate among theseerrors.
Usage of standard codes: overkill in terms of redundancy, ascertain symbol–to–symbol errors are not as important.
Flash Memory Summit 2013, Santa Clara, CA 47 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Graded algebraic codesNon-binary LDPC codes
Graded algebraic codes
Idea: Design codes for the observed intracell error patterns
Approach: Algebraic codes that simultaneously control thenumber of symbols in error and the number of bits in errorper erroneous symbol
Construction: Tensor-product operations
Flash Memory Summit 2013, Santa Clara, CA 48 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Graded algebraic codesNon-binary LDPC codes
Performance evaluation
2800 3000 3200 3400 3600 3800 4000 4200 4400
104
103
102
101
P/E Cycles
Page E
rror
Rate
Error Rates of Codes Applied to TLC Flash
Non binary BCH [4095,3534,80]8
Binary BCH [4095,3531,47]2
TPC [81,7;1,3]8
Binary BCH [4096 , 3351, 62]2 (LSB), [4096 , 3339, 63]
2 (MSB), [4096, 3915, 15]
2 (CSB)
All codes are of rate 0.86 and length 4000 bits.
Flash Memory Summit 2013, Santa Clara, CA 49 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Graded algebraic codesNon-binary LDPC codes
Performance evaluation
2800 3000 3200 3400 3600 3800 4000 4200 4400
104
103
102
101
P/E Cycles
Page E
rror
Rate
Error Rates of Codes Applied to TLC Flash
Non binary BCH [4095,3534,80]8
Binary BCH [4095,3531,47]2
TPC [81,7;1,3]8
Binary BCH [4096 , 3351, 62]2 (LSB), [4096 , 3339, 63]
2 (MSB), [4096, 3915, 15]
2 (CSB)
40 % lifetime improvement
20 % lifetime improvement
All codes are of rate 0.86 and length 4000 bits.
Flash Memory Summit 2013, Santa Clara, CA 49 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Graded algebraic codesNon-binary LDPC codes
Non-binary LDPC codes
Entries in the parity check matrix H are taken from GF (q).Example: GF (8) = 0, 1, 2, ..., 7. (with αk → k + 1 for 0 ≤ k ≤ 6)
H =
1 0 0 3 0 0 5 0 00 2 0 0 6 0 0 2 00 0 3 0 0 1 0 0 11 0 0 0 0 5 0 7 00 3 0 2 0 0 0 0 40 0 6 0 7 0 1 0 0
c1c2c3c4c5c6
v1 v2 v3 v4 v5 v6 v7 v8 v9
Parity check c3: 3v3 + v6 + v9 ≡ 0 mod 8.
Flash Memory Summit 2013, Santa Clara, CA 50 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Graded algebraic codesNon-binary LDPC codes
Non-binary LDPC codes
Decoding is more complex than in the binary case. Keep trackof q − 1 likelihoods on each edge.
Popular approaches:
Direct implementation has complexity on the order of O(q2)FFT-based SPA has complexity on the order of O(q log q)Min-sum and its variants can further reduce the complexity
Flash Memory Summit 2013, Santa Clara, CA 51 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Graded algebraic codesNon-binary LDPC codes
Performance evaluation
Figure: Non-binary LDPC codes vs. BCH codes performance comparisonfor AWGN channel. Code rate is 0.9, block length is 1000 bits. BCHcode corrects 13 errors.
3.5 4 4.5 5
1.0E−8
1.0E−6
1.0E−4
1.0E−2
1
SNR (dB)
FER
GF(2)GF(4)GF(8)GF(16)BCH
Flash Memory Summit 2013, Santa Clara, CA 52 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Algebraic codes (BCH)
– Performance is acceptable
+ Guaranteed error correctioncapability
+ Structure allows for efficientdecoder implementation
– Not amenable for soft decoding
Graph-based codes (LDPC)
+ Performance is excellent
– No guaranteed error correctioncapability (but we have ideas)
– Decoder complexity is acceptablebut now low
+ Amenable for soft decoding
With the move to MLC/TLC technologies, advanced codingschemes will need to be considered!
Flash Memory Summit 2013, Santa Clara, CA 53 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Further information, papers, references etc. available athttp://loris.ee.ucla.edu
Selected list:
R. Gabrys, E. Yaakobi and L. Dolecek, ”Graded bit error correcting codes with applications to Flash
memory,” IEEE Transactions on Information Theory, vol. 59(4), pp. 2315 – 2327, Apr. 2013.
J. Wang, L. Dolecek and R. Wesel, ”The Cycle Consistency Matrix Approach to Absorbing Sets in
Separable Circulant-Based LDPC Codes,” IEEE Transactions on Information Theory, vol. 59(4), pp. 2293 –2314, Apr. 2013.
B. Amiri, J. Kliewer, and L. Dolecek, ”Analysis and Enumeration of Absorbing Sets for Non-Binary
Graph-Based Codes,” submitted to IEEE Transactions on Communications, 2013. (Conference version inISIT 2013.)
Flash Memory Summit 2013, Santa Clara, CA 54 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
UCLA Coding talks and posters at 2013 Flash Summit
R. Gabrys, “Coding for Unreliable Flash Memory Cells,”Session 301-A: Flash Controller Design Options - from 8:30 to9:40 am on Thursday, August 15.
B. Amiri, “Low Error Floor LDPC Codes and Their PracticalDecoders for Flash Memory Applications,” Hall B, booths916-920 – Exhibit Hours
K. Vakilinia, “Non-Binary LDPC Code Design fromInter-Connected Cycles,” Hall B, booths 916-920 – ExhibitHours
Flash Memory Summit 2013, Santa Clara, CA 55 / 56
OutlinePreliminaries
Algebraic codesGraph-based codes
Advanced Coding ApproachesSummary and Outlook
Announcement
New center on Coding for Storage at UCLA:http://www.loris.ee.ucla.edu/codess
Kick-off day on Thursday 9/19/2013!
Registration is free. Register early, space is limited.
Flash Memory Summit 2013, Santa Clara, CA 56 / 56