enumerative source coding and shaping · c outline piet schalkwijk feedback source coding two-way...

Enumerative Source Coding and Shaping

Frans M.J. Willems

ICT-Lab,Department of Electrical EngineeringEindhoven University of Technology

Munich, DoktorandenSeminar, July 19-20, 2018

LABT

IC

Where innovation starts1/55 July 21, 2018

LABT

IC

OutlinePIET SCHALKWIJK

FeedbackSource CodingTwo-Way Channel

ENUMERATIVE SOURCE CODINGBinary IID SourceLexicographical OrderingPascal-∆ Algorithm

VARIATIONSSch-Antonio-PetrySch-Petry CodeShell Mapping (D & C)Rounding

SHAPING CODESGap to CapacityProbablistic Shaping, Distribution Matching

ENUMERATIVE SHAPINGShaping for 802.11

REMARK/REFERENCES

2/55 July 21, 2018

LABT

IC

PIET SCHALKWIJK: Algorithms, Algorithms, Algorithms, ...

1963 - 1965: PhD Stanford (supervisor Thomas Kailath)1968 - 1972: Assistant Professor University California San Diego.1972 - 1996: Professor “Telecommunicatie” Technische Hogeschool Eindhoven

3/55 July 21, 2018

LABT

IC

Feedback, AWGN Channel, Wideband Case [1966]

1967 Information Theory Society Paper Award

4/55 July 21, 2018

LABT

IC

Feedback, AWGN Channel, Band-Limited Case [1966]

5/55 July 21, 2018

LABT

IC

Feedback, Binary Symmetric Channel [1971]

6/55 July 21, 2018

LABT

IC

Source Coding [1972]

7/55 July 21, 2018

LABT

IC

AND - Channel [1982]

8/55 July 21, 2018

LABT

IC

Binary Sources, Sequences, IID

Binary Sourcex1x2 · · ·xN

The binary source produces a sequence xN1 = x1x2 · · ·xN with components ∈ {0, 1} withprobability P (xN1 ).

Definition (Binary IID Source)For an independent identically distributed (i.i.d.) source with parameter θ, for 0 ≤ θ ≤ 1,

P (xN1 ) =N∏n=1

P (xn), where P (1) = θ, and P (0) = 1− θ.

A sequence xN1 containing N − w zeros and w ones has probability

P (xN1 ) = (1− θ)N−wθw.

Entropy IID Source

The ENTROPY of this source is h(θ) ∆= (1− θ) log21

1−θ + θ log21θ

(bits).9/55 July 21, 2018

LABT

IC

Lexicographical OrderingIDEA:Sequences having the same composition (and probability) only need to be INDEXED. Thebinary representation of the index can be taken as codeword.

Definition (Lexicographical Ordering)In a lexicographical ordering (0 < 1) we say that xN1 < yN1 if xn < yn for the smallestindex n such that xn 6= yn.Consider a subset S of the set {0, 1}N . Let iS(xN1 ) be the lexicographical index ofxN1 ∈ S, i.e. the number of sequences yN1 < xN1 for yN1 ∈ S.

ExampleLet N = 5 and S = {xN1 : w(xN1 ) = 2} where w(xN1 ) is the weight of xN1 . Then|S| =

(52

)= 10 and:

iS(11000) = 9 iS(01100) = 4iS(10100) = 8 iS(01010) = 3iS(10010) = 7 iS(01001) = 2iS(10001) = 6 iS(00110) = 1iS(01100) = 5 iS(00011) = 0

10/55 July 21, 2018

LABT

IC

Pascal-∆ Algorithm

IDEA:Index sequences of fixed weight. Use a Huffman code (or a fixed-length code) to describethe weights.

Example (Schalkwijk (1972))Let N = 5 and S = {xN1 :

∑xn = 2}. Then |S| =

(52

)= 10.

10

4

6

1

3

3

1

2

1

1

1

1

1

1

1

1

1

1

1

10

0

0

0

0

0

0

0

0

Index from Sequence:

i(10100) = 6 + 2 = 8.

Sequence from Index:Index i = 8, nowa) 8 ≥ 6 hence x1 = 1,b) i < 6 + 3 hence x2 = 0,c) i ≥ 6 + 2 hence x3 = 1,d) x4 = x5 = 0.

11/55 July 21, 2018

LABT

IC

Sequential Enumeration

Theorem (Cover, 1973)From the sequence xN1 ∈ S we can compute index

iS(xN1 ) =∑

n=1,N :xn=1

#S(x1, x2, · · · , xn−1, 0),

where #S(x1, x2, · · · , xk) denotes the number of sequences in S having prefixx1, · · · , xk.Moreover from the index iS(xN1 ) the sequence xN1 can be computed if numbers#S(x1, x2, · · · , xn−1, 0) for n = 1, N are available.

The index of a sequence can be represented by a codeword of fixed length dlog2 |S|e.

ExampleIndex iS(10100) = 8 hence, since |S| = 10 the corresponding codeword is 1000b.

12/55 July 21, 2018

LABT

IC

Analysis Pascal-∆ Method

First note that

Nh(θ) = H(XN1 ) = H(XN

1 , w(XN1 )) = H(W ) +H(XN

1 |W ).

If we use enumerative coding for XN1 given weight w, since all sequences with a fixed

weight have equal probability

E[L(XN1 |W )] =

∑w=0,1,N

P (w) log2d(Nw

)e

<∑

w=0,1,N

P (w) log2(Nw

)+ 1 = H(XN

1 |W ) + 1.

If W is encoded using a Huffman code we obtain

E[L(XN1 )] = E[L(W )] + E[L(XN

1 |W )]≤ H(W ) + 1 +H(XN

1 |W ) + 1= H(XN

1 ) + 2.

Worse than Huffman, but no large code-table needed.

13/55 July 21, 2018

LABT

IC

Pascal-∆ Method (again): Weight = 3, Length = 8

,

56

21

35

6

15

20

1

5

10

10

.

1

4

6

4

.

.

1

3

3

1

.

.

.

1

2

1

.

.

.

.

.

1

1

.

.

.

.

.

.

.

1

.

.

.

00

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

001

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

11

14/55 July 21, 2018

LABT

IC

Schalkwijk Antonio Petry: Weight ≤ 3, Length = 8

93

29

64

7

22

42

1

6

16

26

.

1

5

11

15

.

.

1

4

7

8

.

.

.

1

3

4

4

.

.

.

.

1

2

2

2

.

.

.

.

.

1

1

1

1

00

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

011

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

J.P.M. Schalkwijk, F. Antonio, and R. Petry,Hawaii Int. Conf. Syst. Sciences [1972]

15/55 July 21, 2018

LABT

IC

Schalkwijk-Petry Code: Variable-to-Fixed Length Code

55

21

34

8

13

21

3

5

8

13

1

2

3

5

8

.

1

1

2

3

5

.

.

.

1

1

2

3

.

.

.

.

.

1

1

2

.

.

.

.

.

.

.

1

1

00

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

11

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

21 3

J.P.M. Schalkwijk, Benelux Symp. Inform. Theory [1981]

16/55 July 21, 2018

LABT

IC

Schalkwijk-Petry Code: AnalysisIDEA:Modify the Tunstall segment sets such that the segments can be indexed.

Again let 0 < θ ≤ 1/2. It can be shown that a proper-and-complete segment set is aTunstall set (maximal E[N(X∗)] given the number of segments) if and only if for all nodesn and all leaves l

P (n) ≥ P (l).

ConsequenceIf the segments x∗ in a proper-and-complete segment set satisfy

P (x∗−1) > γ ≥ P (x∗),

this segment set is a Tunstall set. Constant γ determines the size of the set.

SinceP (x∗) = (1− θ)n0(x∗)θn1(x∗),

where n0(x∗) is the number or zeros in x∗ and n1(x∗) the number of ones in x∗, this isequivalent to

An0(x∗−1) +Bn1(x∗−1) < C ≤ An0(x∗) +Bn1(x∗)

for A = − logb(1− θ), B = − logb θ, C = − logb γ, and some log-base b.17/55 July 21, 2018

LABT

IC

Schalkwijk-Petry Code: Analysis

Note that log-base b has to satisfy

1 = (1− θ) + θ = b−A + b−B .

For special values of θ, A and B are integers. E.g. for θ = (1− θ)2 we obtain that A = 1and B = 2 for b = (1 +

√5)/2. Now C can also assumed to be integer. The corresponding

codes are called Petry codes.

Definition (Petry (Schalkwijk), 1982)Fix integers A and B. The segments x∗ in a proper-and-complete Petry segment set satisfy

An0(x∗−1) +Bn1(x∗−1) < C ≤ An0(x∗) +Bn1(x∗).

Integer C can be chosen to control the size of the set.

Linear ArrayPetry codes can be implemented using a linear array.

18/55 July 21, 2018

LABT

IC

Schalkwijk-Petry Code: Analysis

ExampleConsider A = 1, B = 2.For given C, let S(C) denote the resulting segment set and σ(C) its cardinality. LetS(−1) = S(0) = ∅, then S(1) = {0, 1}, S(2) = {00, 01, 1}, etc. Moreover nowσ(−1) = σ(0) = 1, σ(1) = 2 and σ(2) = 3, etc. It is easy to see that

σ(C) = σ(C − 1) + σ(C − 2),

and therefore σ(3) = 5, σ(4) = 8, σ(5) = 13, σ(6) = 21, σ(7) = 34, and σ(8) = 55.Now take C = 8. Note that 010010 ∈ S(8).We can now determine the index i(010010) using Cover’s formula:

i(010010) = #(00) + #(01000) = σ(6) + σ(2) = 21 + 3 = 24.

55 34 21 13 8 5 3 2 1 10

1

0

1

0

1

0

1

0

1

0

1

0

1

0

1

19/55 July 21, 2018

LABT

IC

Schalwijk-Petry Code: Analysis

Theorem (Tjalkens & W. (1987))A Petry code with parameters A < B, and C is a Tunstall code for parameter q whereq = b−B when b is the solution of b−A + b−B = 1.For arbitrary θ the rate

log2 σ(C)E[N(X∗)]

≤C + (B − 1)

C(h(θ) + d(θ||q)).

ExampleIn the table q for several values of A and B:

A B 2 3 4 51 0.382 0.318 0.276 0.2452 0.4302 0.382 0.3463 0.450 0.412

20/55 July 21, 2018

LABT

IC

Shell Mapping (Divide & Conquer)

56

21

35

6

15

20

1

5

10

10

.

1

4

6

4

.

.

1

3

3

1

.

.

.

1

2

1

.

.

.

.

.

1

1

.

.

.

.

.

.

.

1

.

.

.

00

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

001

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

11

20

101

i(0100010) = 20 + 10 + 1 = 31

21/55 July 21, 2018

LABT

IC


1

1

1

1

2

1

1

3

3

1

.

4×1

6×4

4×6

1×4

.

.

1

3

3

1

.

.

.

1

2

1

.

.

.

.

.

1

1

.

.

.

.

.

.

.

1

.

.

.

00

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

001

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

11

i(0100010) = 4 + 24 + · · ·

22/55 July 21, 2018

LABT

IC


6

3

3

1

2

1

.

1

1

.

.

.

1

.

.

.

.

4

.

.

.

.

1

3

.

.

.

.

.

1

2

.

.

.

.

.

.

1

1

.

.

.

.

.

.

.

1

.

.

.

00

00

00

00

0

0

001

1

1

1

11

11

11

Lang and Longstaff [1989]...

Laroia, Farvardin, and Tretter [1994]

i(01100010) = 4 + 24 + (1 + 1) ∗ 4 + 1 = 37 = 100101b23/55 July 21, 2018

LABT

IC

Pascal-∆ Method (again): Weight = 3, Length = 8

56

21

35

6

15

20

1

5

10

10

.

1

4

6

4

.

.

1

3

3

1

.

.

.

1

2

1

.

.

.

.

.

1

1

.

.

.

.

.

.

.

1

.

.

.

00

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

001

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

11

20

101

i(0100010) = 20 + 10 + 1 = 31 = 011111b24/55 July 21, 2018

LABT

IC

Rounding (2-bit mantissa), Encoding

96

32

48

8

24

24

1

6

12

12

.

1

4

6

4

.

.

1

3

3

1

.

.

.

1

2

1

.

.

.

.

.

1

1

.

.

.

.

.

.

.

1

.

.

.

00

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

001

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

11

24

121

Folklore in arithmetic coding (Pasco [1976], Rissanen [1976]),and in enumerative coding (Tjalkens [1987]).

i(0100010) = 24 + 12 + 1 = 37 = 0100101b25/55 July 21, 2018

LABT

IC

Rounding (2-bit mantissa), Decoding

96

32

48

8

24

24

1

6

12

12

.

1

4

6

4

.

.

1

3

3

1

.

.

.

1

2

1

.

.

.

.

1

1

.

.

.

.

.

.

1

.

.

00

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

001

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

11

24

121

37 < 48→ 037 ≥ 24→ 1

37 ≥ 24 + 12→ 137 < 24 + 12 + 4→ 0

37 < 24 + 12 + 3→ 037 < 24 + 12 + 2→ 0

37 ≥ 24 + 12 + 1→ 1forced → 0

26/55 July 21, 2018

LABT

IC

SHAPING: Gap to Channel CapacityFor signal-to-noise ratio SNR horizontally in dB the AGN capacity CAGN in bits/channeluse is depicted in black in the figure below.

-5 0 5 10 15 20 25 30 35

SNR (dB)

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5C

agn

, C

2, C

4, C

8, C

16, C

32 (

bit)

The curves for uniform 2-PAM, 4-PAM, 8-PAM, 16-PAM and 32-PAM are depicted in blue.A gap to AGN-capacity appears since the PAM inputs are not Gaussian, but Uniform.

27/55 July 21, 2018

LABT

IC

SHAPING: Gap in bit and in SNR lossAssumptions: (a) M -PAM where M →∞ and(b) SNR→∞ or equivalently that σ2

n → 0.

-5 -4 -3 -2 -1 0 1 2 3 4 5

x

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

pg(x

), p

u(x

)

Consider difference of the capacity I(Xg ;Yg), where Xg is the Gaussian channelinput and Yg the corresponding output, and the mutual information I(Xu;Yu), whereXu is a uniform channel input and Yu the corresponding output:

I(Xg ;Yg)− I(Xu;Yu)= h(Yg)− h(Yg |Xg)− h(Yu) + h(Yu|Xu)= h(Yg)− h(Yu)

=12

log2(2πeσ2x)− log2 12σ2

x =12

log2πe

6= 0.2546 bit,

or equivalently 1.53 dB in SNR.28/55 July 21, 2018

LABT

IC

Probabilistic Shaping: Equidistant Signal Points

Eight equidistant signals x ∈ {−7γ,−5γ, · · · ,+7γ} for γ = 1.089.Non-uniform probability distribution P (x): {0.0521, 0.0989, 0.1562, 0.1927, 0.1927,0.1562, 0.0989, 0.0521}. In the plot:

p(x, y) = P (x)p(y|x) for x ∈ {−7γ,−5γ, · · · ,+7γ}

p(y) =∑x

P (x)p(y|x) where p(y|x) =1√

2πexp(−

(y − x)2

2).

Moreover I(X;Y ) = 2.001 bit at SNR = 11.96 dB.29/55 July 21, 2018

LABT

IC

Probabilistic Shaping, Distribution Matching

Q: How can we generate sequences with a given composition?

We can use Distribution Matching (Boecherer [2014], Schulte and Boecherer [2016]).

Distribution matching is inspired by arithmetic data compression techniques (e.g.Langdon and Rissanen [1979], Witten, Neal, and Cleary [1987]). In their methodssequences are represented by intervals.

30/55 July 21, 2018

LABT

IC

Sequences and Intervals, ExampleWe want to generate binary sequences of length 5 containing 2 ones. There are(5

2

)= 10 such sequences.

Each of these sequences corresponds to a subinterval of length 1/10 of the [0, 1) interval.This interval can be computed sequentially from the sequence. The first digit of thesequence splits the interval in fractions 3/5 and 2/5. After a first 0 the interval [0, 3/5) issplit according to 2/4 and 2/4, etc.Note that the sequences and their intervals are now in a lexicographical order.

0.0

1.0

0

1

0.6

0

1

0

1

0.3

0.9

0

1

0

1

0

1

0

0.1

0.5

0.8

1

0

1

0

1

0

0

1

0

0

0.2

0.4

0.7

1

1

0

1

0

0

1

0

0

0

31/55 July 21, 2018

LABT

IC

Constant Composition Sequence Intervals, IndicesConsider 3-digit indices 000, 001, · · · , 111. Index b1b2b3 connects to a constantcomposition sequence if b12−1 + b22−2 + b32−3 is in the interval corresponding to theconstant composition sequence.

0.0

1.0

0

1

0.6

0

1

0

1

0.3

0.9

0

1

0

1

0

1

0

0.1

0.5

0.8

1

0

1

0

1

0

0

1

0

0

0.2

0.4

0.7

1

1

0

1

0

0

1

0

0

0

0.000(000)

0.500(100)

0.250(010)

0.750(110)

0.125(001)

0.375(011)

0.625(101)

0.875(111)

(a) Only one index can connect to a sequence since 2−3 ≥ 1/10.(b) Observe that for not all constant composition sequences there is an index.(c) For all indices there is a sequence however.

32/55 July 21, 2018

LABT

IC

Combining Shaping with Coding

Schematic:

m distrib.matcher

a Graydemapper

b3

b2systematic

rate 23

coder b3

b2

b1

Graymapper

x

The distribution matcher converts message m into amplitude sequence a of thedesired composition.The Gray demapper (sign bit missing!) converts the amplitude sequence a into thetwo amplitude bitstreams b2 and b3 both of length N .

a b2 b31 1 03 1 15 0 17 0 0

Now parity is generated from b2 and b3, using a systematic code of rate 2/3. Thisparity is used as bitstream b1. This bitstream represents the sign bits.

33/55 July 21, 2018

LABT

IC

Combining Shaping with Coding

The three bitstreams are combined into an 8-PAM symbol x stream, using Graymapping.

b1 b2 b3 x0 0 0 −70 0 1 −50 1 1 −30 1 0 −11 1 0 +11 1 1 +31 0 1 +51 0 0 +7

We have described a Bit-Interleaved Coded Modulation construction, where onlysequences with constant amplitude composition are generated.Log-Likelihood Ratio calculation now includes a priori symbol information.

LLRi =

∑b1b2b3:bi=0 p(b1b2b3)p(y|x(b1, b2, b3))∑b1b2b3:bi=1 p(b1b2b3)p(y|x(b1, b2, b3))

34/55 July 21, 2018

LABT

IC

Boecherer Simulations

FER (frame error rate) = 10−3, LDPC codes from DVB-S2. Boecherer, Schulte, andSteiner [2016].

35/55 July 21, 2018

LABT

IC

Alternative to Arithmetic-Coding Approach

Instead of arithmetic-coding approach we can use Schalkwijk’s Pascal-∆ method.

Non-binary case: Use binary splits. Example:

n1357

n13

n57

n1

n3

n5

n7 n1357!n1!n3!n5!n7! =

(n1357n13

)(n13n1

)(n57n5

)Use rounding to reduce complexity.

36/55 July 21, 2018

LABT

IC

Enumerative Shap., from a Partially-Filled Surface to a SphereExample by Y. Gultekin [TU/e, 2017]: In 802.11, in each OFDM symbol 96 realdimensions are used. Now (37, 30, 19, 10) is the Maxwell- Boltzmann composition over 96dimensions with rate 1.75. Composition (37, 30, 19, 10) leads to 2168.72 sequences (rate1.75) and sequence energy 1272.

QUESTION: Can we obtain more sequences such that the average sequence energy doesnot exceed 1272?

Note first that there are more com-positions with energy equal to 1272.Add the corresponding sequences.This leads to 2172.75 sequences.Add all the sequences with an en-ergy smaller than 1272. Now weobtain 2175.04 sequences. More-over the average energy drops to1242.4.

√1272

If we are interested in rate 1.75 we can decrease the radius to√

1120 Now we find 2168.03

sequences with average sequence energy 1096.9. Gain = 12721096.9 = 0.6431 dB.

37/55 July 21, 2018

LABT

IC

Enumerative Sphere Shaping: Bounded-Energy TrellisWuijts, internship TU/e [1991], MSc report TU/e, [1993],W. and Wuijts, IEEE Symp. Comm. Vehic. Techn. Benelux [1993].N = 4, amplitude alphabet is {1, 3, 5, · · · }, Emax = 28 i.e. sphere radius

√28.

0/19 1/11

9/7

17

25/1

2/6

10/4

18/3

26/1

3/3

11/2

19/2

27/1

4/1

12/1

20/1

28/1

1 1 1 1

3 3 3 3

5 5 5 5

1 1 13 3 3

1 13 3

1 1 1

38/55 July 21, 2018

LABT

IC

Lexicographical Ordering. Index of sequence 3131 is 13.

19 11

7

1

6

4

3

1

3

2

2

1

1

1

1

1

1 1 1 1

3 3 3 3

5 5 5 5

1 1 13 3 3

1 13 3

1 1 1

39/55 July 21, 2018

LABT

IC

Lexicographical ordering. Sequence with index 8 is 1331.

19 11

7

1

6

4

3

1

3

2

2

1

1

1

1

1

1 1 1 1

3 3 3 3

5 5 5 5

1 1 13 3 3

1 13 3

1 1 1

40/55 July 21, 2018

LABT

IC

N = 6, Emax = 70, Trellis

517 237

177

87

16

100

82

58

44

32

11

5

38

35

26

20

17

11

7

4

1

13

13

11

8

8

6

4

3

1

4

4

4

3

3

3

2

2

1

1

1

1

1

1

1

1

1

1

41/55 July 21, 2018

LABT

IC

N = 6, Emax = 70, Analysis

Sequences 731111, 731113, 731131, 731311, 733111 are not used if 0 ≤ index < 512.Adding signs leads to 64 · 512 = 215 sequences.Total rate

Rtot =log2 215

6= 2.500 bits/symbol.

Average energy per symbol Eav/N = 9.2422.Gain

G =22Rtot−1

3Eav/N

= 0.4847 dB.

42/55 July 21, 2018

LABT

IC

N = 6, Emax = 70, Rounded Trellis, 6-bit Mantissa

512 236

176

86

16

100

82

58

44

32

11

5

38

35

26

20

17

11

7

4

1

13

13

11

8

8

6

4

3

1

4

4

4

3

3

3

2

2

1

1

1

1

1

1

1

1

1

1

43/55 July 21, 2018

LABT

IC

N = 6, Emax = 70, Analysis

Sequences 173311, 373111, 553311, 731311, 733111 are not used in rounded trellis.Adding signs leads to 64 · 512 = 215 sequences.Total rate

Rtot =log2 215

6= 2.500 bits/symbol.

Average energy per symbol Eav/N = 9.2422.Gain

G =22Rtot−1

3Eav/N

= 0.4847 dB.

44/55 July 21, 2018

LABT

IC

Combining Enumerative Sphere Shaping with Coding

Schematic:

m enumer.shaper

a Graydemapper

b3

b2systematic

rate 23

coder b3

b2

b1

Graymapper

x

The enumerative shaper converts a message m into a bounded energy amplitudesequence a.No difference with Boecherer’s approach.

45/55 July 21, 2018

LABT

IC

Probabilistic Shaping versus Enumerative Shaping

Consider a hypersphere in N (even) dimensions with radius ρ, then

h(X) =h(X1) + h(X2) + · · ·+ h(XN )

N

≥h(X1, X2, · · · , XN )

N=

1N

logπN/2

(N/2)!ρN

≥1N

logπN/2

(N2e )N/2√

e2N2

ρN =12

log 2πeρ2

N−

12N

loge2N

2.

h(X) =h(X1) + h(X2) + · · ·+ h(XN )

N

≤log 2πeE[X2

1 ] + log 2πeE[X22 ] + · · ·+ log 2πeE[X2

N ]2N

≤12

log 2πeE[X2

1 ] + E[X22 ] + · · ·+ E[X2

N ]N

=12

log 2πeρ2

N.

HENCE enumerative shaping for large N results in Gaussians components!Wuijts [1993], continuous case.

46/55 July 21, 2018

LABT

IC

Probabilistic Shaping versus Enumerative Sphere Shaping

Both probabilistic shaping and enumerative sphere shaping lead to Gaussian inputdistributions, which is required to achieve capacity.In a coding environment the bounded energy constraint could be more elementarythan the Gaussian input constraint.For short blocklengths there is something to gain with enumerative shaping.For large blocklengths the gain is negligible (sphere hardening argument).Complexity of enumerative sphere shaping is larger than that of probabilistic shaping(same blocklength).

47/55 July 21, 2018

LABT

IC

Enumerative Sphere Shaping for 802.11

Gultekin, van Houtum, Serbetli, and W., IEEE PIMRC, Montreal [2017]:

Note that shaping does no effect the Peak-Average-Power-Ratio in combination withOFDM. In pulse transmission the PAPR increases.Shaping in combination with the mother convolutional code in 802.11.Amplitudes A7, A13, A25, · · · are prescribed. Input bits can be toggled to hit theamplitude bits. Toggling a bit results in a complemented output pair. Sign bits andPunctured bits follow.

48/55 July 21, 2018

LABT

IC

Enumerative Sphere Shaping for 802.11Puncture-then-Shape Performance:

49/55 July 21, 2018

LABT

IC

Enumerative Sphere Shaping for 802.11Gultekin, van Houtum, and W., Symp. Inform. Th. and Sign. Proc. in the Benelux,Enschede [2018]:

(Enumerative) Sphere Shaping results in Maxwell-Boltzmann Distribution forincreasing blocklengths.Rate-loss (ESS vs. CCDM):

50/55 July 21, 2018

LABT

IC

Enumerative Sphere Shaping for 802.11Enumerative Sphere Shaping in combination with the LDPC codes for 802.11. Nr. ofreal dimensions 162 and 486. Results:

51/55 July 21, 2018

LABT

IC

Approximate Enumerative Sphere Shaping

Gultekin, W., van Houtum, and Serbetli, IEEE ISIT, Vail, Colorado [2018]:

Rounding for Enumerative Sphere Shaping:

Rounding for Shell-Mapping (D & C) Shaping:

Proof that Rounded Trellis Still Gives Unique Decodability.

52/55 July 21, 2018

LABT

IC

Approximate Enumerative Sphere Shaping

Rate-Loss due to rounding:

53/55 July 21, 2018

LABT

IC

REMARK: Shaping Combined with Coded Modulation (Set Partitioning)

W. and Wuijts, IEEE Symp. Comm. Vehic. Techn. Benelux [1993]:

Enumerative Sphere Shaping is combined with the coded modulation techniques asdescribed in “Pragmatic Coded Modulation”, by Viterbi, Wolf, Zehavi, and Padovani[1989].

54/55 July 21, 2018

LABT

IC

REFERENCES (BASIC)

Forney, Gallager, Lang, Longstaff, and Qureshi [1984]: constellation shaping, codingand shaping can be separated.Kschischang and Pasupathy [1990]: variable-rate shaping, geometric shaping.Calderbank and Ozarow [1991]: shaping on regions.Forney [1992]: trellis shaping (more codewords for same data, choose lowest energycodeword), sign-bit shaping, constellation expansion, peak-to-average power ratioexpansion.Sun and van Tilborg [1993]: geometrical shaping.Laroia, Farvardin, and Tretter [1994]: enumerative shaping (two schemes).V34 Modem standard [1994]: shell mapping, enumerative shaping.Fischer [2002]: overview.

55/55 July 21, 2018

enumerative source coding and shaping · c outline piet schalkwijk feedback source coding two-way...

Documents