information theory and channel coding
TRANSCRIPT
Information Theory and Channel Coding
Prof. Rodrigo C. de Lamare
CETUC, PUC-Rio, Brazil
XI. Polar codes
A. Introduction
B. Encoding and code structure
C. Decoding
β’ Polar codes were invented by Erdal Arikan in 2009.
β’ Polar codes are the first provably capacity-achieving efficient coding scheme.
β’ The encoding of polar codes has low complexity (π log2 π ).
β’ The decoding of polar codes can be carried out via successive cancellationdecoding with cost (π log2 π ), list decoding or belief propagation.
β’ They have been adopted for control channels in extended Mobile Broadband (eMBB) of 5G systems.
A. Introduction
β’ Let us consider the block diagram of a polar coding system.
β’ A unique feature of polar codes is the use of polarization of bit channels, which results in noisy useless channels and error-free channels.
β’ Bits that are transmitted over useless channels are frozen.
Encoding Channel Decoding
π π π ΰ·π, ΰ·π
1 Γ π 1 Γ π 1 Γ π
Polarization,Frozen bits
β’ A building block of polar codes is the basic channel transformationillustrated by
where π is a discrete memoryless channel (DMC).
β’ For an error-free/ perfect channel:o π determines πo No need to encode data.
β’ For a useless channel:o π is independent of πo No need to encode.
ππ1
π2
π
π1
π2
π1
π2
β’ Concatenation of DMCs:
o Channels that are perfect or useless but nothing in between-> polarization
o In a polar coding system a given DMC π is used π times to transmit a codeword.
Definition: Basic Channel Transformation (BCT)
π β (πβ,π+)
π: ΰΈ0,1π³
β π΄
πβ: 0,1 β π΄2: π°1β π1, π2 (output)
π+: 0,1 β π΄2 Γ 0,1 : π°2β π1, π2, π1 (output)
Inputs: π1 = π1 βπ2π2 = π2
ππ1
π2
π
π1
π2
π1
π2
Example 1
Consider that π is a binary erasure channel (BEC) with erasure probabilityπ
and the BCT
What are πβ,π+ ? Consider π1, π2 β {0,1}
ππ1
π2
π
π1
π2
π1
π2
π₯0 = 0
π₯1 = 1
π¦0 = 0
π¦1 =1
π
1 β π
1 β π
π
π
Solution:
πβ: input π°1 , output (π1, π2)
i) No erasure: π1, π2 = (π°1 βπ°2, π°2)
ii) Erasure on the 1st BEC: π1, π2 = (? , π°2 )
iii) Erasure on the 2nd BEC: π1, π2 = (π°1 βπ°2, ? )
iv) Two erasures: π1, π2 = (? , ? )
ππ1
π2
π
π1
π2
π1
π2
π₯0 = 0
π₯1 = 1
π¦0 = 0
π¦1 =1
π
1 β π
1 β π
π
π
π+: input π°2 , output (π1, π2, π°1)
i) No erasure: π1, π2, π°1 = (π°1 βπ°2, π°2, π°1)
ii) Erasure on the 1st BEC: π1, π2, π°1 = (? , π°2 , π°1)
iii) Erasure on the 2nd BEC: π1, π2, π°1 = (π°1 βπ°2, ? , π°1)
iv) Two erasures: π1, π2, π°1 = (? , ? , π°1)
ππ1
π2
π
π1
π2
π1
π2
π₯0 = 0
π₯1 = 1
π¦0 = 0
π¦1 =1
π
1 β π
1 β π
π
π
Polarization
β’ The polarization effect can be obtained by recursively applying theBCT.
π
ΰ·ͺΰ·ͺπ1
π
π1
π2
π1
π2
π
π
π3
π4
π3
π4
ΰ·ͺΰ·ͺπ2
π1 =ΰ·ͺΰ·ͺπ1
π3 = ΰ·ͺπ2
ΰ·ͺΰ·ͺπ3
ΰ·ͺΰ·ͺπ4
π2 = ΰ·ͺπ3
π4 = ΰ·ͺπ4
β’ 1st copy of π: input π1, output π1β’ 2nd copy of π: input π2, output π2
πβ: ΰ·ͺΰ·ͺπ1 β π1, π2 , where π1 =ΰ·ͺΰ·ͺπ1 β
ΰ·ͺΰ·ͺπ2
π+:ΰ·ͺΰ·ͺπ2 β π1, π2,ΰ·ͺΰ·ͺπ1 , where π2 =
ΰ·ͺΰ·ͺπ2
β’ 3rd copy of π: input π3, output π3β’ 4th copy of π: input π4, output π4
πβ:ΰ·ͺΰ·ͺπ1 β π1, π2 , where π1 =ΰ·ͺΰ·ͺπ1 β
ΰ·ͺΰ·ͺπ2
π+: ΰ·ͺΰ·ͺπ2 β π1, π2,ΰ·ͺΰ·ͺπ1 , where π2 =
ΰ·ͺΰ·ͺπ2
π
ΰ·ͺΰ·ͺπ1
π
π1
π2
π1
π2
π
π
π3
π4
π3
π4
ΰ·ͺΰ·ͺπ2
π1 =ΰ·ͺΰ·ͺπ1
π3 = ΰ·ͺπ2
ΰ·ͺΰ·ͺπ3
ΰ·ͺΰ·ͺπ4
π2 = ΰ·ͺπ3
π4 = ΰ·ͺπ4
β’ Application of BCT to πβ:
1st copy of π: input ΰ·ͺΰ·ͺπ1, output (Y1, π2)
2nd copy of π: input ΰ·ͺΰ·ͺπ3, output (π3, π4)
πββ: ΰ·ͺΰ·ͺπ1 β π1, π2, π3, π4 , where ΰ·ͺΰ·ͺπ1 = ΰ·ͺπ1 β ΰ·ͺπ3
πβ+: ΰ·ͺΰ·ͺπ2 β π1, π2, π3, π4, ΰ·ͺπ1 , whereΰ·ͺΰ·ͺπ3 = ΰ·ͺπ3
β’ Application of BCT to π+:
1st copy of π: input ΰ·ͺΰ·ͺπ2, output (Y1, π2, π1)
2nd copy of π: input ΰ·ͺΰ·ͺπ4, output (π3, π4,ΰ·ͺΰ·ͺπ3)
π+β: ΰ·ͺΰ·ͺπ2 β π1, π2, π3, π4,ΰ·ͺΰ·ͺπ1,
ΰ·ͺΰ·ͺπ3, , where ΰ·ͺΰ·ͺπ2 = ΰ·ͺπ2 β ΰ·ͺπ4
π++: ΰ·ͺΰ·ͺπ2 β π1, π2, π3, π4,ΰ·ͺΰ·ͺπ1,
ΰ·ͺΰ·ͺπ3, π2 Η , whereΰ·ͺΰ·ͺπ4 = ΰ·ͺπ4
π
ΰ·ͺΰ·ͺπ1
π
π1
π2
π1
π2
π
π
π3
π4
π3
π4
ΰ·ͺΰ·ͺπ2
π1 = ΰ·ͺπ1
π3 = ΰ·ͺπ2
ΰ·ͺΰ·ͺπ3
ΰ·ͺΰ·ͺπ4
π2 = ΰ·ͺπ3
π4 = ΰ·ͺπ4
β’ By renaming π1 = ΰ·ͺπ1, π2 = ΰ·ͺπ2, π3 = ΰ·ͺπ3, π4 = ΰ·ͺπ4, we obtain
πββ: π1 β π1, π2, π3, π4
πβ +: π2 β π1, π2, π3, π4, π1
π+β : π3 β π1, π2, π3, π4, π1, π2
π+ +: π4 β π1, π2, π3, π4, π1, π2, π3
General procedure
β’ πππ 1β¦π π βΆ π° π 1β¦π π +1 β π1, β¦ , π2π , π°1, β¦ , π°(π 1β¦π π) ,
where π 1β¦π π β 0,β, 1, +
π = 2π
Inputs are π’(π 1β¦π π)
Binary string is (π 1β¦π π)
β’ The mutual information associated with this procedure is given by
πΌ πππ 1β¦π π = πΌ(π° π 1β¦π π , π1, β¦ , π2π , π°1, β¦ , π° π 1β¦π π )
β’ By measuring the capacity of these concatenated channels, we obtain
πΆ ππ = πΌ πππ 1β¦π π ,
which is illustrated by
πΆ ππ
π
πΆ ππ
1 1
channels
Example 2
Consider the BCT as the following Kronecker matrix
π» =1 11 0
Illustrate how this matrix can be used for polarization with π = 1,2
Solution:
We can write for π = 1 the inputs π1 and π2 as follows:
π = π1 π2 = π°1 π°21 11 0
= π π»
For π = 2 the inputs π1, π2, π3 and π4 as follows:
π = π1 π2 π3 π4 = π°1 π°2π°3 π°4
1 11 0
1 11 0
1 11 0
0 00 0
= π π»β π° = ππ» π»π» π
B. Encoding
β’ The encoding stage of polar codes relies on the following steps:
o To repeatedly apply the BCT to π DMCs.
o To select among the polarized channels those that are best.
o The message bits π are transmited over good channels
o The useless channels transmit the frozen bits ππ.
β’ The encoding can be illustrated by
Encoding
π π
1 Γ π 1 Γ π
Polarization,Frozen bits
Frozenbits
π
1 Γ π 1 Γ π
πBCTπ»π
π
1 Γ π
β‘
β’ For a codeword with length π = 2π, we define the (π, π, β±, ππ) polar coding scheme πππ as follows.
β’ The length π codeword π of the polar code is given by
π = ππ»π,
where π»π = π·ππ»βπ is the π-fold BCT transformation, π·π is a permutation
matrix and π β βπ is the input vector structured as
π = π | ππ ,
where ππ β βπβπ contains the π β π frozen bits.
β’ The code rate is given by
π =π
π
β’ The encoding complexity is given by O(π πππ2π), which involves savingarithmetic operations with the π-fold BCT.
β’ It is common to employ other channel codes such as cyclic redundancycheck (CRC) codes to further enhance the performance of polar codes.
Example 3
Consider the BCT as the following Kronecker matrix
π» =1 11 0
Show how this matrix can be used to produce a codeword with length π = 4using bit reversal ordering via the matrix
π·π =
1 00 0
0 01 0
0 10 0
0 00 1
Solution:
We can write for π = 1 the inputs π1 and π2 as follows:
π = π1 π2 = π°1 π°21 11 0
= π π»
For π = 2 the inputs π1, π2, π3 and π4 constitute the codeword as follows:
π = π1 π2 π3 π4 = π°1 π°2π°3 π°4
1 11 0
1 11 0
1 11 0
0 00 0
= π π»β π° = ππ» π»π» π
Using the bit-reversal matrix, we can write
π = ππ·4and
π = π π»β π° = ππ·4π» π»π» π
= ππ·4π»βπ = ππ»4
= π°1 π°2π°3 π°4
1 11 1
1 10 0
1 01 0
1 00 0
Therefore, in general we have
π»π = π·ππ»βπ
Puncturing
β’ Puncturing is important to adjust the codeword and/or the rate to therequirements of standards and applications.
β’ Due to the nature of polar codes they have codewords that are powersof 2, i.e., π = 2π, which often requires puncturing of the frozen bits.
Encoding Puncturing
π π
1 Γ π 1 Γ π 1 Γ πβ
Polarization,Frozen bits
πβ²
π =π
ππ β² =
π
πβ²
Example 4
Encode the message π = 1 0 0 1 0 = π°3 π°5π°6 π°7 π°8 using the frozen bits
ππ = 0 1 0 designated by the set β± = 1,2,4 and the matrix
π»8 =
1 11 1
1 11 1
1 11 1
0 00 0
1 10 0
1 10 0
1 10 0
0 00 0
1 01 0
1 01 0
1 01 0
0 00 0
1 00 0
1 00 0
1 00 0
0 00 0
Consider also puncturing to produce a rate R =5
7polar code
Solution:
The codeword is given by
π = π π»β π° = ππ·8π»βπ = ππ»8
= 0 1 1 0 0 0 1 0
1 11 1
1 11 1
1 11 1
0 00 0
1 10 0
1 10 0
1 10 0
0 00 0
1 01 0
1 01 0
1 01 0
0 00 0
1 00 0
1 00 0
1 00 0
0 00 0
= 1 0 1 1 0 1 1 1
In order to puncture this code to obtain a rate R =5
7polar code, we need
to discard 1 frozen bit, which would result in
π = 0 1 1 0 1 1 1
C. Decoding
β’ The most used decoding strategy is based on successive cancellation and isgiven by the following decisions:
ΰ·π’π = απ’π, ππ π β β±
ππ(π , ΰ·π’πβ1 ) ππ π β β±,
where the decoding functions is given by
ππ π , ΰ·π’πβ1 = ΰ΅1, ππ log
π π , ΰ·π’πβ1 |π = 1
π π , ΰ·π’πβ1 |π = β1β₯ 0
0 otherwise
Channel Decoding
π π ΰ·π, ΰ·π
1 Γ π
Example 5
Compare the performance of an LDPC code and a polar code with
puncturing, π = 1920 and rate π =1
4.
