problems of network coding in p2p - and how to overcome it … · 2011-03-31 · problems of...
TRANSCRIPT
![Page 1: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/1.jpg)
Problems of Network Coding in P2P - and how to overcome it
Christian Schindelhauer
joint work with Christian Ortolf &
Arne Vater
Albert-Ludwig University FreiburgDepartment of Computer Science
Computer Networks and Telematics
presented in SPAA 09 & 10
![Page 2: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/2.jpg)
Global Internet Traffic Shares1993-2004
Source: CacheLogic 2005
FTP
Peer-to-Peer
Web
1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004
CacheLogic Research Trends of Internet Protocols 1993-2004
Share
of In
tern
et tr
affi
c
0
10
20
30
40
50
60
70
2
![Page 3: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/3.jpg)
P2P Share Germany 2007
Quelle: Ipoque 2007
3
![Page 4: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/4.jpg)
What Germans Download 2007by Volume
4Quelle: Ipoque 2007
![Page 5: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/5.jpg)
HTTP 44.4 %
BitTorrent 24.1 %
NNTP 14.2 %SHOUTcast 6.4 %
RTMP 5 %
eDonkey 4 %RTSP 1.2 %Skype 0.8 %
HTTP 14.6 %BitTorrent 64.3 %
NNTP 0.7 %SHOUTcast 0.7 %RTMP 0.4 %
eDonkey 16.3 %
RTSP 0.1 %Skype 3 %
Internet Traffic of a Geman ISPAugust 2009
5
Download
Upload
![Page 6: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/6.jpg)
P2P Systems Germany 2007by Volume
6Quelle: Ipoque 2007
![Page 7: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/7.jpg)
7
Motivation
Peer-to-peer networks - distributed system- equal participants (peers)- no client/server structure- used for
• communication (i.e. Skype)• data storage (i.e. OceanStore)• file sharing
![Page 8: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/8.jpg)
8
Block-Based
Bram Cohen: BitTorrent Block-based file sharing
system- divide file into blocks- efficient download - efficient usage of upload- usage of several sources- fairness amongst peers- uses implicit multicast trees
for the distribution of blocks
file X
x1 x2 x3 x6x4 x5 x8x7
x1
x2
x3
x6
x4x5x8
x7
![Page 9: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/9.jpg)
9
BitTorrent Block Selection
Coupon-Collector-Problem- reason for unevenly distributed blocks- if blocks chosen randomly
Measures: Policies- heuristics to select blocks for distribution- different policies depending on progress
• random first• rarest first• endgame
- very successful for popular files
x1
x2
x3
x6
x4
x5
x8
x7
01541302
![Page 10: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/10.jpg)
10
Solution: Network Coding
Optimal solution for Coupon-Collector-Problem / Policy- optimal network flow
• Ahlswede, Cai, Li, and Yeung, "Network Information Flow"
- practical network coding• Gkantsidis, and Rodriguez, "Network coding for large scale
content distribution“
Method- sender transmits code blocks as linear combinations of the
file‘s blocks- receiver collects code blocks and reconstructs the original file
![Page 11: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/11.jpg)
11
Coding and Decoding (I)
b2 b3 b6 b9 b12 b17 b18 b20
x1
x2
x3
x6
x4
x5
x8
x7x2 x3 x6x4 x5 x8x7x1
decoding
bi
enco
ding
![Page 12: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/12.jpg)
12
Coding and Decoding (II)
file X = (x1, x2, ..., xn)
linear coefficients cij
code blocks b1, b2, ..., bn
if the matrix is invertable then
![Page 13: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/13.jpg)
13
Problems of Network Coding
Overhead of storing linear coefficients- one per block- e.g. 4 GB file with 100 KB blocks
• 4 GB / 100 KB = 40 KB per block• overhead 40%
- better: 4 GB file and 1 MB blocks• 4 KB overhead = 0.4%
Overhead of decoding- Inversion of an (n×n)-matrix needs time O(n3)
Read/write accesses- writing n blocks requires reading each part n times: O(n2)- disk access is much slower than memory access
![Page 14: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/14.jpg)
14
Scenario
BitTorrent is optimal regarding disk access and computation overhead,- but it suffers from the
coupon collector problem (availability).
![Page 15: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/15.jpg)
15
Scenario
Network Coding is optimal regarding availability- but it has a high
computational overhead as well as high disk access overhead
![Page 16: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/16.jpg)
16
Our Goal
We want to find a coding scheme that- performs better than BitTorrent regarding availability,
and- requires less read/write accesses than Network
Coding.
![Page 17: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/17.jpg)
17
Paircoding
Paircoding- is a reduced form of Network Coding- combines only two original blocks into one code block
• pi,j = ci xi + cj yj
x2 x3 x6x4 x5 x8x7x1
p1,2 p4,5 p5,7 p3,6 p4,6 p1,8p4,7p2,3 pi,j. . .
![Page 18: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/18.jpg)
18
Advantages
Easy code block creation- only two original blocks must be read
Recoding- new code blocks can be created from different code blocks - no prior decoding necessary
Decoding - requires little computation (sparse matrix inversion)- can be done lazy
Coding alleviates the coupon collector problem
![Page 19: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/19.jpg)
19
Scenario
Paircoding outperforms BitTorrent- with less overhead than
Network Coding
![Page 20: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/20.jpg)
20
Simulation (I)
one seed one downloading peer seeder sends one random block in each round
![Page 21: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/21.jpg)
21
Policy
Policy- algorithmic choice of creating and uploading a code block
depending on• receiving peer
• current network configuration
• availability
• progress
Policy of BitTorrent- optimize throughput and fairness
Policy of Practical Network Coding- always send linearly independent code blocks- optimal
![Page 22: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/22.jpg)
22
Outperforming
A file sharing system A is at least as good as B,
A ≥ B
if for every scenario and every policy of B there is a policy in A such that A performs at least as well as B.
If A ≥ B and there exists a scenario in which A has larger progress than B, A outperforms B.
A > B
![Page 23: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/23.jpg)
23
Paircoding - Results
Theorems- Paircoding outperforms BitTorrent
• For all scenarios and any BitTorrent policy, Paircoing is at least as efficient as BitTorrent.
• For some scenarios Paircoding is more efficient than Bittorrent, i.e. Paircoding outperforms BitTorrent.
- Encoding and decoding can be performed with an almost linear number of read/write operations: O(n • α(n)).
• α(n) is the inverse Ackerman function
![Page 24: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/24.jpg)
24
Treecoding(κ)
tree structure- fixed linear coefficients for all
blocks xi - Xor of two nodes creates
parent node
κ different trees- with linearly independent
linear coefficients root node is equivalent to a
network coding block leaves are equivalent to
uncoded blocks
![Page 25: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/25.jpg)
25
Treecoding - Results
Performance hierarchy- Treecoding(κ + 1) > Treecoding(κ)
Treecoding performs as well as forward error correction- Treecoding(κ) ≥ FEC(κ)
Treecoding outperforms Fixed Paircoding- ∪κ Treecoding(κ) > FixedPaircoding
- if the number of trees is arbitrary Treecoding and Paircoding are incomparable Treecoding has read/write cost of
- O(n), if κ = 1- O(κ n log2 n), for any κ
![Page 26: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/26.jpg)
26
Class Hierarchy
![Page 27: Problems of Network Coding in P2P - and how to overcome it … · 2011-03-31 · Problems of Network Coding in P2P - and how to overcome it Christian Schindelhauer joint work with](https://reader033.vdocuments.net/reader033/viewer/2022042008/5e7119362503f55362022c30/html5/thumbnails/27.jpg)
Problems of Network Coding in P2P - and how to overcome it
Christian Schindelhauer
joint work with Christian Ortolf &
Arne Vater
Albert-Ludwig University FreiburgDepartment of Computer Science
Computer Networks and Telematics
presented in SPAA 09 & 10