PPSP Protocol Considerations and Tracker Protocol

draft-gu-ppsp-tracker-protocol-01

Y. J. Gu, David A. Bryan, Y. Zhang, H. Liao IETF-78 Maastricht, PPSP Session

2

My Thoughts…• Trying to share a picture of what a PPSP

deployment might look like– When possible, want to reuse protocols. Where

can we and what?• Given an architecture, what does the tracker

protocol for that look like?– Short overview of the protocol details of

BitTorrent– These are are own (very early!) thoughts, so may

be wrong, but hope to stimulate discussion

3

What are we trying to do?• There are two basic protocols (or protocol

operations)– Tracker protocol

• How to find other peers that share the information• Focus of this talk, but need to discuss peer protocol a bit to make

sense…

– Peer protocol• How to get information from those peers once you have found

them

• But we seem to be looking at two different tasks– Offline/timeshifted media (essentially file sharing)– Streaming/realtime media

4

Tracker Protocol• Need some way to locate the peers that are sharing same

content• Don’t have a direct protocol to reuse from IETF• Basic idea is that tracker functionality is on a single server,

but could be distributed (DHT)– Note that this is essentially a client-server protocol. Could distribute

as a DHT underneath, but the tracker-peer operation is basically C/S– I’ve heard proposal to use RELOAD, this works if the tracker is made

up of a distributed set of peers– Anaheim meeting indicated interest in tracker as a server, so seems

RELOADs only application here is possibly as an implementation detail underneath or as an alternate distributed implementation

5

BitTorrent as a Model?• Approach seems well suited to offline use case• May not have all the information we want/need for

streaming– Need to find peers “nearby” in the stream– Should tracker attempt to do this?– If not, many peers may need to be contacted to find one

in “right place” (depending on window size, pause, etc)• Possible issues

– Security (or lack of)– HTTP approach is somewhat heavy (but easy)– Do we want to incorporate metadata into tracker (not

offline in torrent file)?• Need to specify syntax for these metadata files

6

Some BT Basics• BT’s primary purpose/design is for file

sharing (not originally designed for live streaming)

• Peers that share a particular file cluster together to share portions of a file form a swarm

7

BitTorrent Entities• Peers: Hosts that hold some portion of the swarm are

peers. Peers exchange blocks, and a set of blocks makes up a piece of the file– A Seed is a peer with the entire file

• Tracker: A central server that stores a mapping between a swarm and the peers participating in that swarm– Tracker doesn’t store which peers have which pieces, just list of the

peers– Tracker is located offline…

8

File and Metadata• Original person sharing the file splits it up

into pieces, and performs an MD5 hash on each piece

• The list of pieces and their hashes, and the location of a tracker that will serve peers sharing this file are placed into a metadata file called a torrent

• Original user places torrent on a web server, and subscribes to tracker with all chunks, as the initial seed

9

Example Startup1) File is chopped up, MD5 sum generated

TrackerChunk1Chunk2Chunk3

…

2) Torrent file lists chunks, sums, and tracker to use for swarm

TrackerChunk1Chunk2Chunk3

…

3) Torrent file is stored on web server

TP 4) Peer connects to Tracker with entire file as seed

10

Example Join SwarmTrackerChunk1Chunk2Chunk3

…

1) Peer connects to web site and obtains torrent file to locate tracker

P

2) Peer connects to Tracker to find other peers

P

TP

P

P

P

3) Peer connects to other peers in swarm and exchanges chunks

11

Peer Exchange• Peers exchange blocks or chunks

– Swap smaller bits than described in metafile, MD5 verify assembled chunks

• Simple gossip protocol – Generally unstructured, not a DHT– Connect to peers that may have desired chunks– Exchange bitfield msgs indicating which chunks it has– Request to ask for a chunk, returned in a piece msg– Once a piece is downloaded, advertised with have msg– Interested/not interested and choked/not choked are

used in flow control – keep-alive message

12

BitTorrent Protocol Details• Regular HTTP is used to obtain the torrent

from a web server• The tracker protocol is also HTTP,

essentially GET to ask for a list of peers/join swarm

• The peer protocol is a TCP wire protocol (binary)

13

Our (Strawman) Proposal• New -01 version coming soon

– Yingjie Gu, David Bryan, Yunfei Zhang, Hongluan Liao• Currently propose binary protocol (but open)

– Light weight, aesthetic considerations– Could also use HTTP with XML or something similar

• Messages to connect to a tracker/disconnect– Credential verification, verify peer-ID later used by peer protocol– Credential issuance/peer-ID assignment not (necessarily) by tracker

• Messages to join/leave a swarm (and get list of peers)– Currently can store location in stream/get peers at this location…

may be hard to implement• Diagnostics between peers and tracker, keep-alive

messages, query list of swarms from tracker• Will describe in detail in later talk

14

Peer Protocol Considerations• New transport is out of scope.

– Offline and Streaming scenarios• Need to reuse existing protocols• SRTP/RTP for streaming. ??? For offline• Should work of LEDBAT be leveraged here?

• Lightweight gossip protocol between peers– Typical for BT is 20-50 peers, in an unstructured

way– Is RELOAD suited for this, or will we need

something lighter?• Try a RELOAD usage and find out?

15

What might this look like?

P

T

P

P

P

Offline/Time Shifted Scenario

New peer protocol (BT based?) to find peers, get metadata (not specific chunks)

Lightweight gossip protocol between peers to find chunks (RELOAD usage or new?)Existing transport

protocol to obtain chunks from peer (leverage LEDBAT?)

16

What might this look like?

P

T

P

P

P

Streaming Scenario

New peer protocol (BT based?) to find peers, get metadata, where in stream peer is?

Lightweight gossip protocol between peers to start/stop stream to other peersExisting transport

protocol to stream the data (RTP/SRTP?)

17

Protocol Reuse?• May be many places where we can reuse

existing protocols, but in some cases, using for things we haven’t done before– LEDBAT?– RELOAD for lightweight/gossip protocol (not

DHT)?– New protocol for Tracker or HTTP with XML

bodies or something similar?

18

Tracker Protocol Proposal• New -01 version released (quite a few changes)

– Yingjie Gu, David A. Bryan, Yunfei Zhang, Hongluan Liao• Key changes:

– Changed names of several messages• Name and semantic meaning not aligning

– Added XML/HTTP Encoding• Authors don’t view encoding as that important right not -- set of messages and

semantic meaning is what is critical• Encodings are proof of concept

• This is a basic overview, not a detailed description (can read draft shortly when we iterate for all details)

• Still very early work -- primary focus is exploring problems through design/early implementation

• A number of hard questions are being left open for WG input, and I’ll talk about those today

• This is by no means complete right now -- lots of work left to do!!!• Authors are very interested in suggestions

19

Messages• CONNECT/DISCONNECT

– Associate with server, verify credentials– DISCONNECT removes from all swarms, leaves system

• JOIN/LEAVE– Participate in a particular swarm (streaming or file for

VoD)– Possibly a JOIN_CHUNK to allow for specifying where in

a live streaming (but big can of worms…)• FIND

– Given a swarm, locate a number of peers– Leaving room to specify a criteria (where in stream (if we

allow JOIN_CHUNK), capacity, possibly certain layers– Quality (ALTO in the tracker, or do peers do ALTO?)

20

Messages, Continued• STATs messages

– STAT_QUERY/STAT_REPORT– Send info to tracker / query about other peers– Tracker can also poll peers

• KEEPALIVE– Limit on live time for peers to tracker, so if no requests in

a certain time, refresh connections– Another option is to expire either CONNECT or JOIN and

require a subsequent call…• QUERY (do we need this?)

– Search for swarms/list swarms (Tracker protocol or should this be something else?)

– Not currently in the draft

21

Open Issues/Considerations• Binary vs. text encoding

– Transport/security mechanism if not HTTP• Need to define format for metadata

describing the file• Peer-IDs used in many messages, but

assignment is offline. Do we want a version of connect that issues IDs?

• Response of list of peers depends on peer protocol used -- IP address vs. Peer-ID only

• NAT traversal needs to be considered

22

Metafile• Differs slightly for streaming/VoD• VoD:

– Needs to describe chunk format, number of chunks, break down (sizing), support for layered encodings, codec

– MD5 sums of blocks (or collections of blocks)• Streaming

– Codec, chunk size, but likely some (number of chunks, MD5 sums don’t make sense)

• It is very important to get this right, but hard!

23

Impact of Peer Prot. on Tracker Prot.• If we use a DHT style peer protocol, with

lookup, then at CONNECT time, the peers need to insert into the overlay– Tracker then only needs return peer-ID (after

bootstrap to locate peer when connecting)• If be use an unstrutured/gossip protocol, not

clear this is ideal– Random connection to 20 peers in a system of

millions likely means you need to provide an address

– Or, just use a DHT for routing, nothing else?

24

NAT Traversal• Easy for managed systems -- server is

placed in a reachable location.• Issue: unmanaged systems:

– Can’t we guarantee tracker in public space• Bigger issue: distributed tracker

– Peers may very well be behind NATs– Fully distributed RELOAD may solve some of

this– If just a few “super-peers”, how do we decide

promotion (NAT detection is provably difficult/impossible…)

25

Conclusion• This is early work -- much still to do• Decoupling the design of the peer and

tracker protocols may help with design, but some aspects are intertwined (for example peer list structure)

• Very much need advice drawn from existing implementations– Biggest question: Do we have the right

commands?

PPSP Peer Protocol

draft-gu-ppsp-peer-protocol-00

Y.J. Gu, David A. BryanIETF-78 Maastricht, PPSP Session

27

Overview• Draft covers:

– Requirements for Peer Protocol– Simple example of a possible flow– Discussion of open issues– Skeleton (currently empty) for new protocol

• Define binary and text strawman proposals such as in tracker protocol?

• Very early work (less developed than tracker protocol, but we will be extending/expanding)– Conversations with some vendors with deployment

experience – including some since -00• Main emphasis currently around requirements and

open issues

28

Peer Protocol Requirements• Location/Connection Requirements

– Req 1: Once a client has a peerlist it SHOULD be able to locate peers and connect to them with no or minimal Tracker's assistance.

– Req 2: The Peer Protocol MUST provide a mechanism for peers behind different NATs/Firewalls to connect with each other.

29

Peer Protocol Requirements• Information Exchange Requirements

– Req 3: The Peer Protocol SHOULD enable peers to request/return/exchange peerlists.

– Req 4: The Peer Protocol SHOULD enable peers to request/return/exchange data availability, e.g. bitmap of chunks.

– Req 5: The Peer Protocol MUST be able to carry different data structures for different applications.

30

Peer Protocol Requirements• Transportation/Negotiation Requirements

– Req 6: The Peer Protocol MUST be able to negotiate a transportation protocol that both peers can support.

• Security Requirements– Req 7: The Peer Protocol MUST guarantee

peers' privacy.– Req 8: Peers SHOULD be able to verify the

identity of remote peers.

31

Protocol Overview• Currently looking at peer protocol in two logical parts

– Location Portion• Locate and connect with Remote Peers

– Signaling Portion• Get additional peerlist (optional?)• Exchange data availability (e.g. bitmap)• Negotiate transport mechanism (e.g. protocol, port…)

– Could be two protocols…– Actual transfer is yet another aspect, but may be out of scope here

(beyond negotiation using signaling portion)

32

Location Protocol• Ties in intimately with tracker protocol• The decision of identity type included in the

peerlists in the tracker protocol will influence the choice of peer location mechanism– IP address, Peer-ID w/DHT, other?

• Effectiveness/decision on these options will influence the design of the tracker protocol.

33

Candidate Protocols• Candidate protocols include RELOAD, SIP, a new protocol,

combination… – RELOAD

• In theory, works for unstructured, but untested.• Could use structured simply to connect peers/create connections?

– SIP• Potentially heavier than RELOAD. • No direct mechanism to support location by Peer-ID?

– RELOAD w/SIP• Use RELOAD to locate/connect, SIP to negotiate?

– New protocol• Single new protocol for location and negotiation? New for only some

functions with reuse for others?

34

NATs and Choice of Identifier• Big issue in identifiers: NAT• In current network, most peers are behind NAT.• If IP addresses are used, ICE/STUN/TURN server functions

may need to be deployed in PPSP system:– Tracker: Natural server/relay location, but will increase the burden

on tracker and need long-term connection between peers and tracker.

– 3rd party server?

• If Peer-IDs are used (particularly reusing RELOAD) can potentially leverage ICE support, add tracker as a relay/STUN server when needed?

35

Conclusions• One of the biggest questions – and one that

has strong bearing on the tracker protocol is the use of IP addresses or Peer-IDs (and a mechanism to locate them)

• NAT traversal is critical• Need to identify set of operations, determine

if reuse is possible• Perhaps decide on identifier issue, then work

on tracker protocol first?

36

Acknowledgements• We would like to acknowledge the following

for providing advice/suggestions/questions:– Roni Even, Yunfei Zhang, Hongluan Liao, Ning

Zong, Daniel De Vera, Matias Barrios

Backup Slides

38

HTTP/XML Approach• HTTP POST method used

– May not be best approach• Bodies are encoding in XML• Tags defined in draft

39

Encoding, Transport for Binary• Currently, we have proposed a binary protocol

– Light weight, low bandwidth for mobile devices– Basic ideas would apply to other encodings

• Right now, the transport is left unspecified -- looking for feedback from group– Since this is essentially a client-server like operation, preference is

on a persistent secure connection approach (TLS/DTLS)• May want a different approach if we do a distributed tracker

– Is there a good reason to use DTLS (UDP)?

• Fragmentation mechanism borrowed from RELOAD (first/last bits, offset)

40

Basic Shared Header

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PPSP Tracker Protocol Token | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version | Reserved | Method | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Transaction ID +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Fragmentation | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Message Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

41

Example Specific Request (JOIN)

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | PeerID (160 bits...) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SwarmID (128 bits...) ... +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Expiration Time | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

42

Messages/Responses• Transaction ID used for

correlation/retransmission• Responses are numeric codes, optionally

with bodies (for example when requesting a list of peers)

• Currently a pure request/response protocol– No need for anything else so far

43

BitTorrent References• Official Protocol Specification (very limited!)

is at http://bittorrent.org/beps/bep_0003.html• Unofficial Specifications (much more

detailed) at theory.org: http://wiki.theory.org/BitTorrentSpecification