overlays and p2p networks
TRANSCRIPT
![Page 1: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/1.jpg)
Overlays and P2P Networks
@ CRES, Monreale, Italy, 16 March-15 April 2009
Giovanni NegliaINRIA Sophia Antipolis, France
www-sop.inria.fr/members/Giovanni.Neglia/p2p09.htm
Thanks to: S. Alouf, E. Biersack, D. Carra, R. Cascella,M. Jelasity, D. Kafar, T. Karagiannis, R. Lo Cigno,V. Misra, M. Mitzenmacher, A. Montresor,D. Qiu, K. Ross, D. Rubenstein, M. Steiner, D. Towsley, M. Varvello
![Page 2: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/2.jpg)
Overlays and P2P Networks:Introduction
@ CRES, Monreale, Italy, 16 March 2009
Giovanni NegliaINRIA Sophia Antipolis, France
[email protected]/members/Giovanni.Neglia/
Thanks to Vishal Misra, Don Towsleyto have provided part of the slides
![Page 3: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/3.jpg)
Overlay networks: what they are
Internet
![Page 4: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/4.jpg)
Overlay networks: what they are
Internet
applicationtransportnetworkdata linkphysical
applicationtransportnetworkdata linkphysical
applicationtransportnetworkdata linkphysical
![Page 5: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/5.jpg)
Overlay networks: what they are
Internet
Overlay
Underlay
Logical connections (using TPC or UDP)
![Page 6: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/6.jpg)
6
Overlay graph
Virtual edge❒ TCP connection❒ or simply a pointer to an IP addressOverlay maintenance❒ Periodically ping to make sure neighbor is still alive
❒ Or verify liveness while messaging❒ If neighbor goes down, may want to establish new edge
❒ New node needs to bootstrap
![Page 7: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/7.jpg)
7
Overlays: all in the application layer
Tremendous design flexibility
❍ Topology, maintenance
❍ Message types
❍ Protocol
❍ Messaging over TCP or UDP
Underlying physical net is transparent to developer
❍ But some overlays exploit proximity
applicationtransportnetworkdata linkphysical
applicationtransportnetworkdata linkphysical
applicationtransportnetworkdata linkphysical
![Page 8: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/8.jpg)
8
Examples of overlays
❒ DNS
❒ BGP routers and their peering relationships
❒ Content distribution networks (CDNs), ❍ e.g. Akamai
❒ Application-level multicast❍ economical way around barriers to IP multicast
❒ And P2P apps !
![Page 9: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/9.jpg)
What is Peer-to-Peer (P2P)?
![Page 10: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/10.jpg)
10
P2P Networks
1) They are overlays
❍ at the application level
2) No client-server
❍ interaction among peers
3) Application running at user side
![Page 11: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/11.jpg)
The Client-Server Model
❒ Contact a server and get the service.
❒ Server has all the resources and capabilities.
❒ No interaction among clients
❒ Common model in the Internet (e.g. www).
![Page 12: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/12.jpg)
The P2P Model
❒ A peer’s resources are similar to the resources of the other participants
❒ P2P – peers communicating directly with other peers and sharing resources
❒ Peer = Servent = Server+Client
![Page 13: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/13.jpg)
Client-Server vs P2P
❒ RPC/RMI❒ synchronous❒ Asymmetric❒ Emphasis on language integration and binding models (stub IDL/XDR compilers, etc.)
❒ Kerberos style security – access control crypto
❒ Messages
❒ Asynchronous
❒ Symmetric
❒ Emphasis on service location, content addressing, application layer routing
❒ Anonymity, high availability, integrity
13
![Page 14: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/14.jpg)
14
P2P Networks
1) They are overlays
❍ at the application level
2) No client-server
❍ interaction among peers
3) Application running at user side
![Page 15: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/15.jpg)
15
First example: Napster
❒ the most (in)famous
❒ not the first (c.f. probably Eternity, from Ross Anderson in Cambridge)
❒ but the first popular one
❒ instructive for what it gets right, and
❒ also wrong…
❒ also has a political message…and economic and legal…
![Page 16: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/16.jpg)
16
P2P file sharing software
❒ Allows Alice to open up a directory in her file system
❍ Anyone can retrieve a file from directory
❍ Like a Web server
❒ Allows Alice to copy files from other users’open directories:
❍ Like a Web client
❒ Allows users to search the peers for content based on keyword matches:
❍ Like Google
Seems harmless to me !
![Page 17: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/17.jpg)
17
Napster: how does it work
❒ Application-level, client-server protocol over point-to-point TCP
❒ Centralized directory server
Steps:❒ connect to Napster server❒ upload your list of files to server.❒ give server keywords to search the full list with.❒ select “best” of correct answers. (pings)
![Page 18: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/18.jpg)
18
Napster
File list and IP address is uploaded
1.napster.com
centralized directory
![Page 19: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/19.jpg)
19
Napsternapster.com
centralized directory
Queryand
results
User requests search at server.
2.
![Page 20: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/20.jpg)
20
Napster
pingspings
User pings hosts that apparently have data.
Looks for best transfer rate.
3.napster.com
centralized directory
![Page 21: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/21.jpg)
21
Napsternapster.com
centralized directory
Retrievesfile
User choosesserver
4.
Napster’s
centralized
server farm had
difficult time
keeping
up with traffic
![Page 22: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/22.jpg)
1) They are overlays YES
2) No client-server, Mostly, but one server for look-up
3) Application running at user side, Mostly
Napster as P2P
Internet
napster.com
centralized directory
![Page 23: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/23.jpg)
23
Routers as peers
❒ Routers in the Internet:
❍ discover topology, and maintain it
❍ neither client nor server
❍ continually talk to each other
❒ Internet was born as an overlay on top of the traditional phone network
![Page 24: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/24.jpg)
P2P & Ad-Hoc Wireless Nets
❒ wireless ad hoc networks have many similarities to peer to peer systems ❍ no a priori knowledge
❍ no given infrastructure
❍ have to construct it from “thin air”!
24
![Page 25: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/25.jpg)
P2P Applications
❒ File sharing (Napster, Gnutella, Kazaa, BitTorrent, EMule)
❒ Audio/video conference (Skype)
❒ Streaming (Sopcast, PPLive)
❒ Data storage
❒ Anonymity (Tor, I2P)
❒ Censorship resistance (Infranet, Tangler)
❒ Collaboration (Jabber, Groove)
❒ General Purpose platforms (JXTA)
❒ Distributed Computing (SETI@home)
![Page 26: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/26.jpg)
Why P2P?
❒ Distributed systems pros...❍ Scalability, Reliability, Saving,...
❒ and cons❍ complexity, management, security
❒ The Internet has three valuable fundamental assets...
❍ Bandwidth❍ Computing/Storage resources ❍ Information
❒ ...all of which are vastly under utilized, partly due to the traditional client-server model
![Page 27: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/27.jpg)
Bandwidth
❒ Despite miles of new fiber installed, the new bandwidth gets little use if everyone goes to Yahoo for content and to eBay
❒ Instead, hot spots just get hotter while cold pipes remain cold
Take advantage of users bandwidth
![Page 28: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/28.jpg)
Computing/Storage
❒ Super-computers (Network Attached Storage) are expensive
Take advantage of home PC CPUs❍ Project like SETI@home
28
1
10
100
1000
10000
1 10 100 1000
Rank (log scale)
LinPack perf.GFLOPS (log scale) .
2001 2000
1999 1998
1997 1996
1995
Top500 supercomputer list over time:Zipf distr. Perf(rank) ≈ rank –τ
τ decreases over time =>Increasing interest to aggregate capabilities of machines in the tail
A virtual machine that aggregates the last 10 would
rank 32nd in ’95 but 14th in ‘03
![Page 29: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/29.jpg)
Information
❒ Most of the information is still at users❍ No single search engine can locate and catalog it (size, accessibility, transient)
![Page 30: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/30.jpg)
P2P impact: users#
❒ File sharing❍ Napster (≈2000) 1.5 million simultaneous users❍ 2003 3.8Millions [Big Champagne]
❍ 2006 • 9M (not including BitTorrent), [Big Champagne]
• 5M in Mainline BT DHT [BitTorrent]
• 1M in Azureus BT DHT [Azureus]
❒ Skype: 9 millions [skypestats.com 2008]
❒ PPLive 0.4M daily users [PPLive, 2006]
![Page 31: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/31.jpg)
P2P impact: traffic (file sharing)
❒ 60% [Cachelogic, 2004 et 2006]❍ In 2006 50-65% download, 75/90% upload
❒ 2007❍ 74% of German traffic [ipoque 2007]
❍ 37% North America [Ellacoya networks, 2007]
❒ Total cross-border Traffic [TeleGeography, 2005]❍ 2005: 1 Terabps
❍ 2008: 2-3 Terabps
![Page 32: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/32.jpg)
P2P impact: market
❒ P2P applications market for collaborative environment [Solomon Smith Barney,2003] ❍ 5.8 Billions $ in 2003❍ 36.5B $ in 2004
❒ P2P traffic revenues for carriers❍ 100B $ by 2012 [Insight Research Corp., 2007]❒ Seti@Home savings: $1.5M / year
❒ Microsoft: user-assisted software update spreading
❒ Thomson: set-top box assisted movie spreading
❒ From 2008: P2P market conference, byDistributed Computing Industry Association
![Page 33: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/33.jpg)
Outline (1/3)
❒ File Sharing: the first generation❍ Napster, Gnutella, Kazaa
❒ Interlude: Structured vs Unstructured and P2P topologies
❒ Search in Unstructured Networks
❒ Structured networks (DHT)
❒ File Sharing: state of the art❍ BitTorrent, Emule
❍ Models for P2P file sharing
❍ Free riders, Incentives and Strategic clients (BitThief, BitTyrant)
33
![Page 34: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/34.jpg)
Outline (2/3)
❒ Coding for file sharing❍ Fountain codes, network coding, bloom filters
❒ P2P traffic❍ Tussle between ISPs and Peers (part 1)
❍ P4P
❒ Advantage and Risks of Overlays❍ Tussle between ISPs and Peers (part 2)
❍ Case study: overlays for routing (Resilient Overlay Networks)
34
![Page 35: Overlays and P2P Networks](https://reader035.vdocuments.net/reader035/viewer/2022071601/613d483e736caf36b75b7b35/html5/thumbnails/35.jpg)
Outline (3/3)
❒ Skype
❒ P2P for anonymity
❒ P2P streaming
❒ P2P backup
❒ Virtual Coordinate systems
❒ Distributed Virtual Environments
35