a measurement study of piece population in bittorrent
DESCRIPTION
A Measurement Study of Piece Population in BitTorrent. Cameron Dale and Jiangchuan Liu Simon Fraser University Burnaby, BC, Canada [email protected] Globecom, November 29 th , 2007, Washington, D.C. Overview. background on some BitTorrent features describe the experiment - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/1.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
A Measurement Study ofPiece Population in BitTorrent
Cameron Dale and Jiangchuan LiuSimon Fraser University
Burnaby, BC, Canada
Globecom, November 29th, 2007, Washington, D.C.
![Page 2: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/2.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Overview
● background on some BitTorrent features
● describe the experiment
● results for piece population snapshots and the evolution of the piece population for some real BitTorrent swarms*
● PlanetLab simulations of BitTorrent swarms
● discussion and future work*swarm: a group of peers connected to one another via the
BitTorrent file distribution protocol
![Page 3: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/3.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Piece Population
● BitTorrent splits a large file into many pieces
● each piece will have a certain number of copies in all the peers in the swarm
● ideally, all pieces should have approximately the same number of copies
● local knowledge limitations prevent the ideal and introduce an imbalance
● too much variation can lead to inefficiencies and even starvation
![Page 4: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/4.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Rarest-First Policy
● responsible for choosing the next piece that a peer will download
● each client chooses the piece that it believes is the rarest one
● choices are made based only on local knowledge of neighboring peers
● goal is to maintain an even distribution of pieces throughout the swarm
![Page 5: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/5.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Piece Population Distribution
● expect the population to form a distribution around a mean value
● the mean will be determined solely by the macro-characteristics: peer arrival rate, download time, departure rate, etc.
● the width of the distribution will indicate the effectiveness of the rarest-first policy– the narrower, the better
![Page 6: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/6.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Real Swarm Snapshots
● used a modified BitTorrent client to gather data from real swarms
● client constantly requests peers from the tracker* and connects to all returned peers
● client connects to most (90%) peers in a swarm and collects their piece information
● collection occurs very quickly and is terminated, usually after less than an hour
*tracker: a server that coordinates communication between peers attempting to download a file
![Page 7: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/7.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Real Swarm Evolution
● collection was done as in the snapshots
● collection continued for many hours, or even days
● observe changes in the Piece Population as the swarm evolves
● focussed on the early stages of a swarm when there are a large number of leechers*
*leecher: a user who does not yet have a complete copy of the file, and so is currently downloading (and uploading)
![Page 8: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/8.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Swarm Characteristics
*clients: the number of clients administered by us
Torrent Name Pieces Size (MB) Leechers Clients*
KNOPPIX 4125 4325 169 10
FreeBSD 5699 1494 34 10
mandriva 2803 735 89 9
openSUSE 14805 3881 398 9
feisty 1387 727 65-120 20
openSUSE-2 14977 3926 100-150 18
PlanetLab 1497 784 0-340 340
PlanetLab-2 1497 784 0-390 390
![Page 9: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/9.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Piece Population Plots
● x-axis shows the number of copies within the downloaders in the swarm
● y-axis shows the number of pieces that have that number of copies
● x-axis is normalized to range from 0 to 1, by the total number of downloaders
● y-axis can be normalized by the number of pieces, or so that the area under the graph is 1 (to facilitate comparisons)
![Page 10: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/10.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Real Internet Swarm Snapshots
![Page 11: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/11.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Evolution: feisty
![Page 12: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/12.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Evolution: feisty
![Page 13: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/13.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Evolution: feisty
![Page 14: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/14.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Evolution: feisty
![Page 15: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/15.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Evolution: openSUSE-2
![Page 16: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/16.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Evolution: openSUSE-2
![Page 17: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/17.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Evolution: openSUSE-2
![Page 18: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/18.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Evolution: openSUSE-2
![Page 19: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/19.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Simulated Swarms
● run on PlanetLab research network testbed
● all peers are controlled by us
● experiment starts with a single seed* and peers join randomly over first 4 hours
● PlanetLab: identical peers, no peers join after 4 hours
● PlanetLab-2: distribution of peers, new peers arriving continuously
*seed: a peer that has the complete file, and is only uploading
![Page 20: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/20.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Simulation: PlanetLab
● all peers are identical
● peers stay in the system for about 9 hours, then leave forever
● each peer's maximum number of connections is limited to 40 to enhance the local effect
● it takes 9 hours for the initial seed to upload a single copy of the file
![Page 21: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/21.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Simulation: PlanetLab
![Page 22: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/22.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Simulation: PlanetLab
![Page 23: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/23.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Simulation: PlanetLab
![Page 24: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/24.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Simulation: PlanetLab
![Page 25: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/25.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Simulation: PlanetLab-2
● peers have a distribution of download and upload speeds
● peers download the file, seed for a random time, then leave and rejoin as a new peer
● arrivals and departures are grouped together to increase the amount of churn*
● each peer's maximum number of connections is limited to 80 (default)
*churn: the arrival and departure of peers in a P2P system
![Page 26: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/26.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Simulation: PlanetLab-2
![Page 27: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/27.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Simulation: PlanetLab-2
![Page 28: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/28.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Simulation: PlanetLab-2
![Page 29: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/29.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Rarest-first Policy
● is effective– results in normal distributions that are narrow
● improves on random piece selection by 3-4x
– recovers quickly (exponentially) from events that cause a widened distribution
● could be improved– larger swarms show tails on the high end due to
the limited local knowledge of peers– churn causes increases in the width of the
population
![Page 30: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/30.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Suggested Improvement
● peers knowledge of pieces could be extended through gossiping– share all piece information with neighbors
● increased communication cost
– share information on which pieces are the rarest●piece with the most votes gets downloaded next
● should reduce the size of the tails for the larger swarms
![Page 31: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/31.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
Future Work
● use these measurements to evaluate the effect of new piece selection strategies
● further analysis of swarms in the presence of churn to determine the cause of the increased population width
● work is already under way to create an analytical piece-level model describing the piece population of a swarm– complicated due to the complex interactions
between peers
![Page 32: A Measurement Study of Piece Population in BitTorrent](https://reader035.vdocuments.net/reader035/viewer/2022070405/56813efb550346895da97be2/html5/thumbnails/32.jpg)
Cameron Dale and Jiangchuan Liu A Measurement Study of Piece Population in BitTorrent
Introduction BitTorrent Experiment Results Simulation Discussion
A Measurement Study ofPiece Population in BitTorrent
Cameron Dale and Jiangchuan LiuSimon Fraser University
Burnaby, BC, Canada
Globecom, November 29th, 2007, Washington, D.C.