rateless coding with feedback
DESCRIPTION
Rateless Coding with Feedback. Andrew Hagedorn , Sachin Agarwal , David Starobinski, and Ari Trachtenberg Department of ECE, Boston University, MA, USA IEEE INFOCOM 2009. Outline. Introduction Problem Definition Shifted LT (SLT) Codes Experimental Results Conclusion. Transmitter. - PowerPoint PPT PresentationTRANSCRIPT
1
Rateless Coding with Feedback
Andrew Hagedorn, Sachin Agarwal , David Starobinski, and Ari Trachtenberg
Department of ECE, Boston University, MA, USA
IEEE INFOCOM 2009
2
Outline
1. Introduction2. Problem Definition3. Shifted LT (SLT) Codes4. Experimental Results5. Conclusion
3
Partial Information
• Transmission Channel with Erasures
Transmitter Receiver
Input symbols Received Symbols
4
Partial Information
• Transmission Channel with Erasures
Transmitter Receiver
Input symbols Received Symbols
5
Partial Information
• Transmission Channel with Erasures
Transmitter Receiver
Input symbols Received Symbols
6
Partial Information
• Transmission Channel with Erasures
Transmitter Receiver
Input symbols Received Symbols
7
Partial Information
• Transmission Channel with Erasures
Transmitter Receiver
Input symbols Received Symbols
8
Partial Information
• Transmission Channel with Erasures
Transmitter Receiver
Input symbols Received Symbols
9
Partial Information
• Multiple Receivers may have different erasures
Transmitter
Receiver 1
Receiver 2
Receiver 3
Given the situation of multiple receivers having partial information, how can all of them be updated to full information efficiently, and over a broadcast channel?
10
Partial Information• Multiple mobile devices may have out-
dated informationa. Mobile databasesb. Sensor network information aggregationc. RSS updates for devices
Broadcaster
Mobile device 1
Mobile device 2
Mobile device 3
Latest version of information
11
Problem Definition•Given an encoding host with k input symbols and a decoding host with n out of the k input symbols, the goal is to efficiently determine the remaining k-n input symbols at the decoding host.
– The encoding host has no information of which k-n input symbols are missing at the decoding host
– Different decoding hosts may be missing different input symbols
•Efficiency1.Communication complexity – Information transmitted from the encoding host to the decoding host should be close in size to the transmission size of the missing k-n input symbols
2.Computational complexity – The algorithm must be computationally tractable
12
• Contribution of this paper – Show that a small amount of feedback, whereby
receivers periodically inform the broadcasting sources about the number of successfully decoded input packets, can lead to major communication, memory, and energy usage gains through a judicious modification of the encoding procedure.
13
Rateless Codes - EncodingUsed for content distribution over error-prone channels
Random choice of edges based on a probability density function
At least k Encoded Symbolsk input symbols
1 =A+B
2 =B
3 =A+B+C
4 =A+C
A
B
C
14
Rateless Codes - DecodingUsed for content distribution over error-prone channels
At least k Encoded Symbols
1 =A+B
2 =B
3 =A+B+C
4 =A+C
k input symbols
SolveGaussian Elimination, Belief Propagation
System of Linear Equations
Irrespective of which encoded symbols are lost in the communication channel, as long as sufficient encoded symbols are received, the decoding can retrieve all the k input symbols
A
B
C
15
Decoding Using Belief Propagation
Decoded k Input Symbols
k+ Encoded Symbols
Decoding host
Redundant!
Decode
Input Symbols
16
Digital Fountain CodesLT Codes
1. Class of rateless erasure codes invented by Michael Luby1
2. Computationally practical (as compared to Random Linear Codes)
3. Fast decoding algorithm based on Belief propagation instead of Gaussian Elimination
4. Form the outer code for Raptor Codes3, which have linear decoding computational complexity
5. Designed for the case when no input symbols are available at the Decoding host initially
•Asymptotic Properties2
•Expected number of encoded symbols required for successful decoding
•Expected decoding computational complexity
•k: number of input symbols
2Assuming a constant probability of failure
)ln( 2 kkOk
1Michael Luby, “LT codes,” in The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002, pp. 271–282.3Amin Shokrollahi, “Raptor codes,” IEEE Transactions on Information Theory, vol. 52, no. 6, 2006, pp. 2551–2567.
)ln( kkO
17
Digital Fountain CodesLT Codes’ Robust Soliton Probability Distribution
• Robust Soliton Probability Distribution k, • Probability of an encoded symbol with degree d is k(d) • Property of releasing degree 1 symbols at a controlled, near-constant rate throughout the decoding process
0 200 400 600 800 1000-6
-5
-4
-3
-2
-1
0
Degree
log 10
(Pro
babi
lity)
LT Code (Robust Soliton)
LT code distribution, k = 1000, c = 0.01, = 0.5.
18
Real-Time Oblivious Erasure Correcting
Amos Beimel, Shlomi Dolev, and Noam Singer
IEEE-Information Theory Workshop 2004, San Antonio, Texas[3] Amos Beimel, Shlomi Dolev, and Noam Singer, “Rt oblivious erasure
correcting”, IEEE/ACM Trans. Netw., vol. 15, no. 6, pp. 1321–1332, 2007.
19
Traditional Erasure Codes
k message
Decodingk message
Encoding
n>k symbols
Transmission Channel
Sender
Receiver
k received X X X X X X
Rate-Less Codesn can be ∞
20
Motivation• Problem
– Channels with high loss rate– Expensive feed-back channels– Weak receiving devices
• Current solutions– ARQ – Requires large feed-back– Erasure Codes – Higher Encoding/Decoding complexity,
a single feedback message• Our goal
– Combine their benefits.
21
Real-Time Codes• Complexity
– Fast symbols generation– Efficient message decoding– Balanced decoding over the entire transmission
• Decoding rate– Rate in which
symbols are decoded
22
Protocol Description
Encoded Symbols
Feed-back
d=3
Calculate degree dRandomly pick d symbolsXOR these symbolsTransmit encoded symbols
Check if exactly 1 symbol missingIf so, decode the missing symbolDump the encoded symbolTransmit the number of decoded symbols r
r=4
rk
kd 1
d=4
23
Conclusions of RT Codes
• A combined approach between ARQ and Erasure Codes
• Low memory overhead• Low feedback - O(√k) messages
24
Inefficiency of LT Codes for our Problem
k+ Encoded Symbols
Decoding host
Decode
Input Symbols
n out of k input symbols are known a priori at the decoding host
Many redundant encoded symbols
Inefficiency of LT Codes for our Problem
•The number of these redundant encoded symbols grows with the ratio of input symbols known at the decoder (n) to the total input symbols (k)
•If n input symbols are known a priori, then an additional LT-encoded symbol will provide no new information to the decoding host with probability
…which quickly approaches 1 as n → k
d
i
k
dk ik
ind01
)(
25
26
Intuitive Fix • n known input symbols serve the function of degree 1
encoded symbols, disproportionately skewing the degree distribution for LT encoding
• We thus propose to shift the Robust Soliton distribution to the right in order to compensate for the additional functionally degree 1 symbols
• Questions – 1) How?– 2) By how much?
0 200 400 600 800 1000-6
-5
-4
-3
-2
-1
0
Degree
log 10
(Pro
babi
lity)
LT Code (Robust Soliton)
27
Shifted Code Construction• Definition
The shifted robust soliton distribution is given by
– k : the number of input symbols in the system– n : the number of input symbols already known at the decoder– round(・ ) rounds to the nearest integer
• Intuitionn known input symbols at the decoding host reduce the degree of each encoding symbols by an expected fraction
j
kn
iij nknk
1roundfor )()(,
kn1
1
28
Shifted Code Distribution
0 200 400 600 800 1000-6
-5
-4
-3
-2
-1
0
Degree
log 10
(Pro
babi
lity)
LT Code (Robust Soliton)Shifted Code
LT code distribution and proposed Shifted code distribution, with parameters k = 1000, c = 0.01, = 0.5. The number of known input symbols at the decoding host is set to n = 900 for the Shifted code distribution. The probabilities of the occurrence of encoded symbols of some degrees is 0 with the shifted code distribution.
29
Shifted Code – Communication Complexity
nknkOnkm 2ln)(
Lemma IV.2 A decoder that knows n of k input symbols needs
encoding symbols under the shifted distribution to decode all k input symbols with probability at least 1−.
ProofWe have k-n input symbols comprising the encoded symbols after the n
known input symbols are removed from the decoding graph. The expresson follows from Luby‘s analysis.
30
Shifted Code – Average Degree of Encoded Symbol
• Lemma IV.3 – The average degree of an encoding node under the k,n distribution is
given by
• Proof– The proof follows from the definitions, since a node with degree d in
the μk distribution will correspond to a node with degree roughly
in the shifted code distribution.From Luby‘s analysis,the expresson for the average degree of an LT
encoded symbol is
)ln( nk
nkkO
kn
d
1
)(ln kO
31
Shifted Codes – Computational Complexity•Lemma IV.4
–For a fixed , the expected number of edges E removed from the decoding graph upon knowledge of n input symbols at the decoding host is given by
E = O (n ln(k − n))•Theorem IV.5
–For a fixed probability of decoding failure , the number of operations needed to decode using a shifted LT code (SLT) is
O (k ln(k − n)) *Proof described in: S. Agarwal, A. Hagedorn and A. Trachtenberg, “Rateless Codes Under Partial Information”, Information Theory and Applications Workshop, UCSD, San Diego, 2008
32
Heuristics for practical implementation
• 1) Non-uniform restriction on feedback– In fact, most input symbols are decoded after n
surpasses a certain value n = αk, 0 ≤ α ≤ 1. – A feedback message containing the most recent
value of n is sent only when the average degree changes by a constant (since the previous feedback).
– When n < nNU, the average degree of an encoding symbol increases by
33
Heuristics for practical implementation
– We limit the feedback to every time the average degree changes by √ log k (from its value at the previous feedback), leading to approximately 1/2√ k feedbacks (obtained by dividing (4) by √logk).
– When n ≥ nNU , the heuristic sends at most √ k feedbacks, one each time the degree changes by (at least) √ log k.
– This heuristic sends O( √ k) feedbacks as n increases from 0 to k, which is equal to the RT code’s feedback.
34
Heuristics for practical implementation
• 2) Uniform restriction on feedback– The current value of n is communicated back to
the encoder every time n increases by √k, resulting in √k feedbacks as n increases from 0 to k
– This heuristic has the advantage of not congesting the feedback channel toward the end of decoding, unlike RT codes and the non-uniform restriction on feedback.
35
Fig. 1. Feedback strategies for uniform and non-uniform restrictions on Shifted LT and RT codes. Each circle qualitatively corresponds to a situation in which the current value of n is fed back to the encoder.
36
Simulation Results
• c = 0.9 and δ = 0.1• In each round of the simulation an encoded
packet is generated and transmitted, and decoding is attempted on the received packet (as well as any stored in memory) at the decoder.
• If an input symbol is recovered then feedback is sent as dictated by each code.
37
Simulation Results
• For k=500, on average Shifted LT codes requires 59% less redundancy than RT codes and 21% less redundancy than LT codes (on average, over 100 trials).
• The feedback channel communication complexity for Shifted LT codes is greater than either RT codes or LT codes.
• While RT codes is limited by the changes in its degree and LT codes transmits no feedback, the Shifted LT code transmits feedback every time it recovers one or more input symbols.
38
Memory usage
Fig. 2. Memory usage at the decoder as a function of the number of transmitted symbols.
39
Number of encoded symbols required
Fig. 3. Number of encoded symbols required to disseminate all k input symbols.
40
Number of feedback messages sent
Fig. 4. The number of feedback messages sent for the different codes for increasing number of input symbols k. The “Shifted LT - no restriction” transmits too many (O(k)) feedbacks and has been left out of this figure.
41
Number of encoded symbols needed
Fig. 5. The number of encoded symbols needed to decode 100 input symbols, as a function of the feedback channel rate.
42
Number of encoded symbols needed
Fig. 6. The number of encoded symbols needed to decode 100 input symbols, as a function of the feedback channel loss rate. The forward channel loss rate is fixed at 5%.
43Fig. 7. The number of encoded symbols needed to decode 100 input symbols at 50 receiving nodes, for various forwarded packet loss probabilities.
44
Computational load on the motes
Fig. 8. The amount of time required to decode a randomly chosen encoded packet, as a function of the number of encoded symbols already transmitted.
• 2 TelosB motes, one mote serves a single page (consisting of multiple packets) to the other mote.
45
Total number of packets transmitted
Fig. 9. The total number of packets transmitted on forward and feedback channels in order to disseminate a 5 page program to 10 motes using variants of the Deluge over-the-air programming protocol.
• 11 motes, one of which broadcast 5 pages in memory (totally 11.5K) to the 10 other motes. • All feedback channels from the 10 motes to the broadcaster were set to have a 5% packet loss rate, and the forward channel loss rates were varied from 0% to 9%.
46
Total energy used
Fig. 10. Total energy used by all the motes for communication and decoding during the dissemination of a 5 page program using a variant of the Deluge over-the-air programming protocol.
47
Conclusion
• Shifted LT codes provide an easily implemented improvement over existing rateless codes, LT codes and RT codes.
• The corresponding improvements in communication complexity, energy usage, and, in certain cases, memory requirements are even starker within a broadcast.
48
References• [3] Amos Beimel, Shlomi Dolev, and Noam Singer, “Rt oblivious erasure correcting”, IEEE/ACM
Trans. Netw., vol. 15, no. 6, pp. 1321–1332, 2007.• [4] J.W. Hui and D. Culler, “The dynamic behavior of a data dissemination protocol for network
programming at scale.”, in SenSys’04, Baltimore, Maryland, USA, Nov. 2004.• [10] A. Hagedorn, D. Starobinski, and A. Trachtenberg, “Rateless deluge: Over-the-air programming
of wireless sensor networks using random linear codes”, in IPSN ’08: Proceedings of the 7th International Conference on Information Processing in Sensor Networks, 2008.
• [11] M. Rossi, G. Zanca, L. Stabellini, R. Crepaldi, A. F. Harris, and M. Zorzi, “Synapse: A network reprogramming protocol for wireless sensor networks using fountain codes”, in SECON ’08: Proceedings of the IEEE Conference on Sensor, Mesh and Ad Hoc Communications and Networks, 2008.
• [13] S. Kokalj-Filipovic, P. Spasojevic, E. Soljanin, and R. Yates, “Arq with doped fountain decoding”, in ISSSTA 08’: International Symposium on Spread Spectrum Techniques and Applications, 2008.
• [14] S. Agarwal, A. Hagedorn, and A. Trachtenberg, “Rateless codes under partial information”, in ITA ’08: Information Theory and Applications Workshop, 2008.
• [17] Phil Levis, “Tossim: Accurate and scalable simulation of entire tinyos applications”, in In Proceedings of the First ACM Conference on Embedded Networked Sensor Systems (SenSys 2003), 2003.
• Weiyao Xiao, Sachin Agarwal, David Starobinski, Ari Trachtenberg: Reliable Wireless Broadcasting with Near-Zero Feedback. IEEE INFOCOM 2010: 2543-2551