a case study on how economic frameworks can bail out systems research emin gün sirer ryan peterson,...
Post on 21-Dec-2015
220 views
TRANSCRIPT
![Page 1: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/1.jpg)
A Case Study on How Economic Frameworks
Can Bail Out Systems Research
Emin Gün SirerRyan Peterson, Bernard Wong
Department of Computer Science, Cornell UniversityUnited Networks, LLC
September 3, 2009
![Page 2: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/2.jpg)
Problem Domain
What is the most efficient way to
disseminate a large set of files to
a large set of clients?
![Page 3: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/3.jpg)
Client-Serverserver
clients
InefficientInefficientHigh cost of High cost of ownershipownership
![Page 4: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/4.jpg)
Peer-to-Peerpeer
Limited informationLimited informationNo control or performance No control or performance
guaranteesguarantees
![Page 5: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/5.jpg)
Traditional Systems Research
•Complex problems lead to complex protocols with many parameters
• Parameters picked based on a combination of gut feeling and experimentation
• Systems tailored to specific workloads
• Only intellectual honesty keeps researchers from perfect performance on benchmarks!
•Novelty by aggregation• Pick k out of n: CDNs, network coordinates,
geolocation, network tomography, …
![Page 6: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/6.jpg)
BitTorrent
•Robust, resilient protocol• A combination of randomization and heuristics
• Two years until characterized as auctions based on received bandwidth
•Lots of incremental research!• Every 10-line tweak => 1 paper
• Hacks: Share ratios, manual pruning, …
• We lack the tools to tackle difficult problems
![Page 7: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/7.jpg)
Antfarm Goals•High performance•Low cost of deployment•Performance guarantees• Administrator control over swarm
performance
•Accounting• Enables different resource
contribution policies
![Page 8: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/8.jpg)
Antfarm Approach
•Key insight: view content distribution as an optimization problem
•Hybrid architecture
• P2P swarming with a logically centralized coordinator
•Clean slate protocol
![Page 9: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/9.jpg)
Antfarm System Model
coordinatorseeder
altruist
![Page 10: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/10.jpg)
Strawman Coordinator
•One could schedule every data transfer in the system
• All packets for all time
• Unscalable, impractical!
•Antfarm coordinator makes critical decisions based on observed dynamics
![Page 11: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/11.jpg)
Antfarm Coordinator
•Models swarm dynamics• Measures and extracts key
parameters
•Formulates optimization problem• Calculates optimal bandwidth
allocation
•Enacts allocation decisions• Maximizes aggregate bandwidth
• Minimizes average download time
![Page 12: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/12.jpg)
Antfarm Formalization
Maximize system-wide aggregate bandwidth subject to a bandwidth constraint
![Page 13: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/13.jpg)
Response Curves
slope = 1 slope = 0
![Page 14: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/14.jpg)
Response CurvesSwarm aggregate bandwidth (KB/s)
1500
1000
500
0
Seeder bandwidth (KB/s)0 25 75 10050
![Page 15: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/15.jpg)
Swarm Dynamics
Swarms exhibit different dynamics based on size,
peer resources,network conditions. . .
![Page 16: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/16.jpg)
Antfarm Optimization
σA σCσB
σA + σB + σC = B
A
B
C
![Page 17: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/17.jpg)
Performance Control
•Can provide swarm performance guarantees
• Guarantee minimum level of service
• Prioritize swarms
![Page 18: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/18.jpg)
Antfarm Allocation
σA′ σC′σB′
σA′+ σB′+ σC′= B
A
B
C
![Page 19: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/19.jpg)
Problems•The optimization problem has limitations
•Requires knowing total bandwidth constraint
•Applies only to full caches under the control of the coordinator
•At the wire protocol level, there are many opportunities for malfeasance
![Page 20: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/20.jpg)
Antfarm++•Use a currency scheme, called karma, to keep track of peers’ resource consumption and contribution
•Every exchange costs some karma to the peer receiving a block, brings karma to the peer providing the block and bandwidth
![Page 21: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/21.jpg)
peer A
purseledger
Wire Protocol
•Coordinator mints small, unforgeable tokens
•Peers trade each other tokens for blocks
•Peers return spent tokens to the coordinator as proof of contribution
coordinator
peer B
purseledger
![Page 22: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/22.jpg)
Tokens & Exchange Rates
•Every swarm uses a per-swarm currency, every token has source-destinations marked
•Token flows reveal the internal network dynamics to the coordinator
•The coordinator can provide incentives in the form of exchange rates
•Exchange rates encode the differential in the desirability of participating in different swarms => no need to estimate total BW
![Page 23: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/23.jpg)
Benefits of a Virtual Currency•The cost and benefit of a transaction need not be symmetrical
•A block always costs 1 token to the buyer
•A block can bring γii karma to the seller
•Provide an incentive for peers to Provide an incentive for peers to join and seed swarms in needjoin and seed swarms in need
![Page 24: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/24.jpg)
Antfarm Performance
0
500
1000
1500
2000
2500
3000
3500
4000
4500
Zipf, 60 KB/s seeder Zipf, 200 KB/s seeder
Aggr
egat
e ba
ndwid
th (K
B/s)
Client-server BitTorrent Antfarm
![Page 25: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/25.jpg)
Swarm Starvation
BitTorrent starves the singleton swarm
![Page 26: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/26.jpg)
BitTorrent: Starves New Swarm
Swarms, ordered largest to smallest
newself-
sufficientsingleto
n
![Page 27: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/27.jpg)
Antfarm: Seeds New Swarm
Swarms, ordered largest to smallest
newself-
sufficientsingleto
n
![Page 28: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/28.jpg)
Conclusions
•Hard problems in systems research can benefit immensely from the application of economics
• http://flixq.com showcases our current work
![Page 29: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/29.jpg)
Related Work•Content Distribution Networks
• Akamai, CoBlitz, CoDeeN, ECHOS, Coral, Slurpie, YouTube, Hulu, GridCast, Tribler, Joost, Huang et al. 2008, ...
•P2P Swarming
• BitTorrent, BitTyrant, PropShare, BitTornado, BASS, Annapureddy et al. 2007, Guo et al. 2005, ...
•Incentives and microcurrencies
• Dandelion, BAR Gossip, Samsara, Karma, SHARP, PPay, Kash et al. 2007, ...
![Page 30: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/30.jpg)
Questions?
![Page 31: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/31.jpg)
PlanetLab Deployment
![Page 32: A Case Study on How Economic Frameworks Can Bail Out Systems Research Emin Gün Sirer Ryan Peterson, Bernard Wong Department of Computer Science, Cornell](https://reader030.vdocuments.net/reader030/viewer/2022032611/56649d6c5503460f94a4c0f5/html5/thumbnails/32.jpg)
PlanetLab Deployment
0
2375
4750
7125
9500
11875
50 100 200
Seeder bandwidth (KB/s)
Aggre
gate
bandwid
th (KB/s
)
Client-server BitTorrent Antfarm