bullet: high bandwidth data dissemination using an overlay mesh
DESCRIPTION
Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh. Introduction. Given a sender and a large set of receivers spread across the Internet, how can we maximize the bandwidth? Problem domain: Software or video distribution Real-time multimedia streaming. Existing Solutions. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/1.jpg)
Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh
![Page 2: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/2.jpg)
Introduction
Given a sender and a large set of receivers spread across the Internet, how can we maximize the bandwidth?
Problem domain: Software or video distribution Real-time multimedia streaming
![Page 3: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/3.jpg)
Existing Solutions
IP multicast does not consider bandwidth when constructing its distribution tree
A promising alternative: Overlay Attempts to mimic the multicast routing
trees Instead of high-speed routers, use
programmable end hosts as interior nodes in the overlay tree
![Page 4: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/4.jpg)
Existing Solutions
A tree structure is problematic Decreasing bandwidth as moving
down a tree Any loss high up the tree will reduce
the bandwidth lower down the tree Bandwidth of a node limited by its
single parent
![Page 5: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/5.jpg)
A New Approach
Transmit disjoint data set to various points in the network
A node download from multiple sources rather than a single parent Higher reliability
![Page 6: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/6.jpg)
Conceptual Model
A B
Root1 Mbps 1 Mbps
1 Mbps
![Page 7: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/7.jpg)
Conventional Model
A B
Root1 Mbps 1 Mbps
1 Mbps
![Page 8: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/8.jpg)
Conventional Model
A B
Root1 Mbps 1 Mbps
1 Mbps
![Page 9: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/9.jpg)
Bullet
A B
Root1 Mbps 1 Mbps
1 Mbps
![Page 10: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/10.jpg)
Bullet
A B
Root1 Mbps 1 Mbps
1 Mbps
![Page 11: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/11.jpg)
Bullet
A B
Root1 Mbps 1 Mbps
1 Mbps
![Page 12: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/12.jpg)
Bullet
A B
Root1 Mbps 1 Mbps
1 Mbps
![Page 13: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/13.jpg)
Bullet
A B
Root1 Mbps 1 Mbps
1 Mbps
![Page 14: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/14.jpg)
Bullet
A B
Root1 Mbps 1 Mbps
1 Mbps
![Page 15: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/15.jpg)
Bullet
A B
Root1 Mbps 1 Mbps
1 Mbps
![Page 16: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/16.jpg)
Bullet
A B
Root1 Mbps 1 Mbps
1 Mbps
![Page 17: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/17.jpg)
Bullet
A B
Root1 Mbps 1 Mbps
1 Mbps
![Page 18: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/18.jpg)
Bullet
A B
Root1 Mbps 1 Mbps
1 Mbps
![Page 19: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/19.jpg)
Bullet Properties TCP friendly
Must respond to congestion signals Low control overhead
Probing resource Locating multiple downloading sources
Decentralized and scalable No global knowledge
Robust to failures, even high up in a tree
![Page 20: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/20.jpg)
Bullet Overview
Use meshes as opposed to trees Bandwidth independent of the
underlying overlay tree Used a 1,000-node overlay and
20,000 network topologies Up to 2x bandwidth improvement
over a bandwidth-optimized tree Overhead of 30 Kbps
![Page 21: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/21.jpg)
System Components Split the data into packet-sized objects Disseminate disjoint objects to clients
at a rate determined by bandwidth to each client
Nodes need to locate and retrieve disjoint data from their peers
Periodically exchange summary tickets Minimize overlapping objects from
each peer
![Page 22: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/22.jpg)
Illustration
S
A
ED
CB
1 2 3 4 5 76
1 2 3 5 1 3 4 6 2 4 5 6
1 2 5 1 3 4
![Page 23: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/23.jpg)
Data Encoding
Multimedia: MDC encoding Large files: erasure encoding
Tornado code Only need to locate 5% extra packets
to reconstruct the original message Faster encoding and decoding time
![Page 24: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/24.jpg)
RanSub Distributes random subsets of
participating nodes During the collect phase, each
node sends a random subset of its descendant nodes up the tree
During the distribute phase, each node sends a random subset of collected nodes down the tree
![Page 25: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/25.jpg)
Informed Content Delivery Techniques
Use summary tickets A summary ticket is an array
array[i] = hashi(working set)
Check ticket elements against a Bloom filter It is possible to have false positives It is possible that B will not send a
packet to A even though A is missing it
![Page 26: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/26.jpg)
TCP Friendly Rate Control (TFRC)
TCP halves the sending rate as soon as one packet loss is detected Too severe
TFRC is based on loss events, or multiple dropped packets within one round-trip time
![Page 27: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/27.jpg)
TCP Friendly Rate Control (TFRC)
Bullet eliminated retransmission from TFRC Easier to recover from other sources
than from the initial sender TFRC does not aggressively seek
newly available bandwidth like TCP
![Page 28: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/28.jpg)
Bullet
Layers a mesh on top of an overlay tree to increase overall bandwidth
![Page 29: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/29.jpg)
Finding Overlay Peers
RanSub periodically delivers subsets of uniformly random selected nodes
Via summary tickets (120 bytes per node)
The working set from each node is associated with a Bloom filter
Peer with nodes with the lowest similarity ratio
![Page 30: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/30.jpg)
Recovering Data from Peers A receiver assigns a portion of the
sequence space to each of its senders, to avoid duplication among senders
A receiver periodically updates each sender with its current Bloom filter and the range of sequences covered in its Bloom filter
Less than 10% of all received packets are duplicates
![Page 31: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/31.jpg)
Making Data Disjoint
Given a randomly chosen subset of peer nodes, it is about the same probability that each node has a particular data packet
A parent decides the portion of its data being sent to each child A function of limiting and sending
factors
![Page 32: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/32.jpg)
Making Data Disjoint
The portion of data a child should own is proportional to The number of its descendants Bandwidth
If not enough bandwidth Each child receives a completely
disjoint data stream
![Page 33: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/33.jpg)
Making Data Disjoint
If ample bandwidth Each child will received the entire
parent stream
![Page 34: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/34.jpg)
Improving the Bullet Mesh
What can go wrong Not enough peers Constant changing network
Use trial senders and receivers Bullet periodically evaluates the
performance of its peers Places the worst performing
sender/receiver
![Page 35: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/35.jpg)
Evaluation
Used Internet environments and ModelNet IP emulation
Deployed on the PlanetLab Built on MACEDON
Specifies the overlay algorithms Core logic under 1,000 lines of
code
![Page 36: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/36.jpg)
Evaluation
ModelNet experiments 50 2 GHz Pentium 4’s running Linux
2.4.20 100 Mbps and 1 Gbps Ethernet
switches 1,000 emulated instances 20,000 INET-generated topologies
![Page 37: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/37.jpg)
Offline Bottleneck Bandwidth Tree
Given global knowledge, what is the overlay tree that will deliver the highest bandwidth to a set of overlay nodes? Finding a tree with a maximum
bottleneck NP hard in general
![Page 38: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/38.jpg)
Offline Bottleneck Bandwidth Tree
Assumptions The path between two overlay nodes is fixed The overlay tree uses TCP-friendly unicast
connections to transfer data point-to-point In the absence of other flows, we can
estimate the throughput of a TCP-friendly flow using a steady-state formula
In the case of sharing, each flow can achieve at most of 1/nth of the total throughput
![Page 39: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/39.jpg)
Bullet vs. Streaming
The maximum bottleneck tree achieves 5x bandwidth compared to random trees
Bullet outperforms the bottleneck tree by a factor up to 100%
![Page 40: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/40.jpg)
Creating Disjoint Data
Without disjoint transmission of data Bullet degrades by 25%
![Page 41: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/41.jpg)
Epidemic Approaches
Bullet is 60% better than anti-entropy (gossiping) approaches Epidemic algorithms have an
excessive number of duplicate packets
![Page 42: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/42.jpg)
Bullet on a Lossy Network
Bullet achieves 2x bandwidth compared to maximum bottleneck trees
![Page 43: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/43.jpg)
Performance Under Failure
With failure detection/recovery disabled 30% performance degradation with a
missing child under the root With failure detection/recovery
enabled Negligible disruption of performance
![Page 44: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/44.jpg)
PlanetLab
47 nodes for deployment Similar results
![Page 45: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/45.jpg)
Related Work Kazza
Perpendicular downloads Does not use erasure code Bandwidth consuming
BitTorrent Centralized tracker
FastReplica Not bandwidth aware
![Page 46: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/46.jpg)
Related Work
Scalable Reliable Multicast Difficult to configure
Epidemic approaches Do not avoid duplicates
Narada Use overlay meshes Bandwidth still limited by parent
![Page 47: Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56814bf3550346895db8e2b6/html5/thumbnails/47.jpg)
Related Work
Overcast More heavyweight when nodes leave
a tree Splitstream
Not bandwidth aware CoopNet
Centralized