xstream final book.doc

XstreamA Large scale peer-to-peer video on demand

streaming media protocol

Submitted by:

Barak Schlosser and Amiad Stern

Advisors:

Mr.Danny Bickson and Mr.Elan Pavlov

Faculty Member:

Prof. Danny Dolev

1

Contents:

1. ABSTRACT……………………………………………….3

2. STREAMING OVERVIEW …………………………….4

2.1. P2P Streaming Protocol………………………...…….4

2.2. Problem Overview……………………………………5

3. SOLUTION……………………………………………….7

4. PROTOCOL………………………………………............9

4.1. Connection Server…………………………………….9

4.2. Connecting a User……………………………………12

4.3. Updating Relevant Users……………………………..13

4.4. Locality Awareness…………………………………..14

5. ARCHITECTURE……………………………………….14

5.1. UsersServer…………………………………………..14

5.2. BufferManagment……………………………………15

5.3. Communication………………………………………16

5.4. Client…………………………………………………18

5.5. Messages……………………………………………..18

5.6. Network Utilities……………………………………..19

6. DEVELOMENT STATUS……………………………….20

6.1. Development Overview………………………………20

6.2. Goals Achieved……………………………………….21

7. XSTREAM INNOVATIONS……………………………..22

7.1. Reduce Servers’ Workload……………………………22

7.2. Load Balancing of the System………………………...22

7.3. Locality Awareness…………………………………...23

8. SYSTEM PREFORMANCE…………………………….25

8.1. LAN Tests Overview…………………………….…...25

8.2. LAN Results………………….………………………25

8.3. LAN Deferent Scenarios……………………………..26

8.4 Planetlab Tests Overview……………………………..28

9. FUTURE WORK………………………………………... 29

10. REFERENCES .…………………………………………..31

11. APPENDIX A………………………………………….….32

12. APPENDIX B………………………………………….…. 33

2

1. Abstract

Streaming video on demand content to a large number of users (thousands or more)

simultaneously, is one of the challenges of current and future networks.

We have designed Xstream - a large scale P2P video on demand streaming protocol.

Our protocol is a distributed P2P video on demand streaming protocol.

Xstream protocol provides digital video capabilities such as start, stop, pause, fast-

forward, and rewind thus forming a dynamically changing environment of users. The

solution is based on an unstructured topology of users. On entry to the system each user

receives a list of peers who hold content that is of interest to the user. This list is

dynamically updated during running using protocols based on gossiping. Since

distances between users seldom change, the overhead of this gossip is low. Each user

caches a sliding window of content and makes it available to other users.

Our protocol ensures division of workload between users to avoid a situation where a

small number of users must supply content to a large number of users, thus avoiding

bottlenecks of users which slow the system. This fairness condition is notified by taking

into account bandwidth, buffer size and distance between peers. Peer selection and

choice of content between various peers is based on locality awareness content

distribution algorithm.

3

2. Streaming Overview

2.1 P2P Streaming Protocol

A P2P protocol is a protocol in which all users in the system, named peers, acts as

both user and a server.

When connecting to a system running a P2P protocol, you know that in order for the

system to work properly, you can’t only be acting as a ‘consumer’ but as a ‘producer’

as well. The advantages of P2P topologies are that they are unstructured protocols

which mean that the system has great flexibility, it is durable and it is designed to

achieve maximal fairness for all users (peers).

Briefly we’ll mention other protocols that have lack taking in consideration those

advantages for example, a tree topology lack the flexibility, a disconnection of an

upper recipient will cause a great deal of managing new one into the blank space he

left, another example is the naïve solution in which everybody downloads directly

from the server there is no fairness in the system and the server might malfunction if

there are too many users connected to him.

After we explained a little bit what is a P2P topology lets see what exactly a

streaming protocol is. As you probably know ‘Kazaa’, ‘E-mule’ and similar, which

are File Sharing P2P Protocols you’ll easy understand what is Streaming P2P

Protocol.

Let us summarize the differences between File Sharing and Streaming protocols.

In File Sharing you would like to enable the following:

Save the file on locally (for reuse).

Get the file completely (without any lose).

Don’t mind waiting a little bit to download it completely.

You can download packets in an unlimited arbitrary order (i.e. you can

download the last packet and then the first).

4

In Streaming you would like to enable the following:

Don’t want to keep the content locally.

Don’t mind to have a little loss of packets (as long as it doesn’t harm

the video quality).

Can tolerate couple of seconds while buffering, but after starting

viewing you want to see the video continually.

Packets should arrive on a strict time schedule, or should be skipped.

You can download packets in an arbitrary order but there is a limit for

that arbitrary order for you don’t need the last packet for the next

couple of minutes so you want ask to download it.

For conclusion of this small overview Xstream protocol is a P2P protocol like any

File Sharing Protocol but it has been designed and implemented in order to cope with

those differences mentioned above….

2.2 Problem Overview

Given a large video file we would like to enable a large number of users to concurrently

watch the movie over the internet.

Users may leave and enter the system arbitrarily and more over they can join into any

part of the movie (e.g. 5th minute).

As we are dealing with a P-2-P streaming protocol the time constraint is one of the

major tasks to deal with while solving the above problem.

Packets should arrive on time before they should be screend to the viewer, therefore

packets must be "collected" at the host in a serial mode, there can be a slight disorder

while buffering packets but there is a certain limit, meaning if the user currently views

the 1st minute he won't "collect" to his buffer packets from the end of the movie but he

can collect packet from the 5th minute before packet from the 4th minute as long as he

have the next couple of minutes in his buffer.

5

The user might tolerate few seconds delay upon connecting to the system, but after

watching the first frame he would like to view the movie without any stops (small

glitches caused by packets lost are tolerable).

As for the system has a P2P topology it must be designed to achive maximal guarntee

fairness for all users and above all servers’ workload should be kept as low as possible

in order not to cause it to malfunction and enable it to be used for other contents.

The system should be flexibile and durable, meaning that the system should be durable

for reasonable amount of users to connect or disconnect at the same time without any

trouble.

6

3. Solution

In our solution users (hosts) are conceptually assigned into flexible groups where a

group represents a time frame in the video file.

Users (hosts) download packets from users (hosts) located in groups ahead of them on

the video stream. [Figure 1]

Figure 1. Users conceptually assigned into flexible groups and download packets from users (hosts)

located in groups ahead of them on the video stream.

The only exceptional group is the first one in which hosts are downloading their content

directly from the server and share their content between themselves.

Each host "helps" the system by buffering already seen packets by the user/viewer (e.g.

buffers the last 10 minutes), buffering the packets is been carried out in a sliding

window manner [Figure 2], each host download packets he need in order to serve its user

(viewer) but instead of "throwing" them after been seen the host buffers them in order

to serve the hosts (users) in groups located behind him on the video stream. By that

hosts located behind him on the video stream can download the content from him and

don’t need to address the server.

As for the reason that hosts download their desired content from the hosts located in a

group ahead of them on the video stream, a host doesn't need to "search" it's desired

content among the hosts (peers) in the system but he knows that the hosts (peers)

located ahead of him have the content he desires.

In an ideal world, where the ratio of connections of new users (hosts) in each time

frame is constant and each user (host) has enough users (hosts) to download from, the

7

server will work only once for forwarding the stream to the first group of users and then

the stream will continue to be forwarded among consecutive groups.

Figure 2. In this figure we see 3 instances of the users’ buffer from Figures 2a - 2c shows packets

coming in, been buffered and the first packet (which is the oldest one in the buffer) been

thrown out. The different between the instances is seconds.

8

4. Protocol

Conceptually, there is a single video file which we wish too stream to a large number of

users. In practice each video file will be dealt with separately.

In our protocol there are conceptually two types of servers:

A connection server (called UsersServer).

A content server.

The UsersServer is responsible for maintaining a database of users and their place along

the stream (i.e., which offset they are currently viewing).

The content server is responsible for actually serving the content. In practice, the

content server may be implemented by a cluster of servers.

4.1 Connection server (UsersServer)

At the heart of the system is the UsersServer. The UsersServer is responsible for

ensuring that each user has enough peers in the vicinity of its offset to ensure that it

can receive content in a timely fashion. Since all connections to the stream are done

via the UsersServer, the UsersServer knows the offset of all peers. Ideally, when a

peer leaves the system it will gracefully inform the UsersServer of the disconnection.

Unfortunately, this can not be guaranteed in practice. We therefore need to sample

peers to ensure that they have not disconnected. This is done in two fashions.

The first is by new users connecting to the stream. A new user is given a set of peers

to connect to. It notifies the UsersServer of any peers that are not active. The second

way of updating the UsersServer of disconnects are via peers that are active. If at any

time a peer has too few peers to receive content it requests additional peers from the

UsersServer and notifies the UsersServer of failed peers. From the fact that new users

test all of their connections

9

Remark 4.1.1 if each new user in the system checks kc peers then the connection

database is updated to a fraction kc / r , where r is the ratio between connecting and

disconnecting peers.

Remark 4.1.2 It is also possible to bound the accuracy of the database if we

require a user to report whenever a fraction of the peers it received from the

UsersServer fail. We wish to avoid the requirement that the user do so

and provide new peers to receive content from via gossiping.

Another problem that might arise is that of bottlenecks. Since users can connect at

any offset and any number of users can appear at any time a situation can arise in

which a large number of peers needs content from a small number of peers. In order

to deal with this, we define a stretch parameter l s.t. l is the maximum of the ratio

between peers giving and receiving content. We divide the streams into sections

S1, . . . ,Sn of length t which we call time-frame the time-frame represent groups. Note

that a peer can receive information only from members assigned to a group currently

located ahead of him on the stream and can give information only to members of

groups located behind him on the stream. The UsersServer ensures that the l > max

Si/Si+1 by requiring users to simulate additional users. If at any time l < max Si/Si+1

the next user to enter the system simulates a node in Si+1. The user u receives the

content as if it is viewing the desired offset. Due to gossiping mechanisms in our

protocol explained in section 4.3 the users who desire the content held by u are

informed that u holds the desired data.

Due to this fact we get that at a small cost of simulating users we can deal with

arbitrary faults of users leaving the system. Formally,

Corollary 4.3 Under the assumption that there is a ratio r between incoming

and outgoing users, the ratio between the size of adjacent groups is

bounded.

10

The main property of our system is that if there are enough incoming users, the load

on the content server is bounded. Formally,

Lemma 4.1 Under the assumption that there is a ratio r between incoming

and outgoing users and that failures are random, the workload is bounded by

the size of the first group.

Proof: let x denote the size of the first group, let r be the ratio between incoming and

outgoing users and let l be the ratio between any 2 consecutive groups.

The content server must distribute the content to the first group causing a worst case

minimum work load of x. By keeping a ratio l between any 2 consecutive groups we

have ensured that each group demand for content can be resolved by it preceding

group thus minimizing the workload on the server to 0. The only other case in which

the server is used for distributing data is when a user reaches a time schedule

problem and asks the server for a specific packet.

4.2 Connecting a user

For any two users u, v with currently stream offsets Ou, Ov if Ou < Ov < Ou+t then u

can receive data from v. We denote by c the maximal number of users u that u should

know that it can receive content from (the minimal number of such users will be c/2 ).

We denote k as a parameter that determines how many users u is informed of when

asking the UsersServer for peers to connect to. The higher k is the more time it will

take to connect but the less probability there is for u to need to return to the

UsersServer for more peers.

When a new user u wants to connect to the stream it connects to the UsersServer. The

UsersServer determines what offset the user wishes to view (according users’ desired

offset and connection time) and gives the user several peers to receive content from. In

addition the UsersServer assigns the user additional offsets to simulate (i.e., send and

receive content) in order to provide load balancing (if needed). The details of this are

provided in section 4.1.

11

Upon connecting:

1. Connect to UsersServer and get a set of offsets to simulate (one of these offsets is the

offset the user wishes to view).

2. Connect to UsersServer and get a group of users of size kc whose buffers overlap the

buffer for each of these offsets. If there are not enough appropriate users get a link to

the content server. k is a resilience parameter, the higher k is the less probability

there is to need to return to the UsersServer to get more peers. Furthermore, the

higher k is the better updated the UsersServer will be.

3. Connect to all of the users received.

4. Notify UsersServer of all users who did not respond (and which the user has not yet

notified the UsersServer). The UsersServer removes these users from its database.

5. As long as there are c/2 users to provide content or the set of users contain the

content server, retrieve content.

6. If there are more than c peers remove the content server from the set of users (if it is

the list).

7. If there less than c/2 users and the content server is not one of them go to line 2.

4.3 Updating relevant users

As we can see by the protocol a user will connect to the UsersServer whenever the

number of users who can serve content drops too low. In order to minimize the number

of times a user needs to connect to the content servers we update the set of users who

can serve content. During streaming whenever a connection is open between peers for

data we piggyback to give updated information about peers that are watching offsets

close to the desired offset. This allows a user to keep a large set of peers to receive

content from even if peers disconnect. In order to do this each user u maintains a list of

users who are at an offset that is within 2t of the users offset Ou. In order to avoid

keeping redundant and disconnected peers in this list the size of the list is bound to c

users in any time frame of length t. This list is propagated whenever content is passed

between peers. Formally, we have the following protocols:

12

When requesting data:

1. Request data from node v and add v and the relative offset of v to the list of nodes

with a time stamp of the current time.

2. u gives to v a list of all users known to u that are in the interval [Ou, Ou + 2t].

3. Receive list of users from v that are in interval [Ou, Ou + 2t].

4. Both u, v merge lists and purge extra entries (by keeping more updated time stamps).

Note that after this protocol, u has received an additional list of users that it can receive

data from. These are the users in the interval [Ou, Ou +t]. u also receives a list of nodes

in the interval [Ou + t, Ou + 2t] while these nodes are useless to u they are useful for

nodes in the interval [Ou, Ou + t] which are precisely the nodes u contacts to receive

content. Intuitively, if v, v0 are contemporary nodes (i.e., nodes that need the same

content), they never communicate directly but can update each other on new peers via

the services of u. Of course, in practice there v, v0 will have a slight difference in offset

so they might communicate.

4.4 Locality

Another problem streaming protocols often face is the question of locality. The tradeoff

between attempting to receive data from peers which are close locally (i.e., high

bandwidth and low latency) is balanced by the need to get data from peers that might be

far away. In our protocol we balance these conflicting needs by retrieving data from an

arbitrary peer when the buffer is relatively full and from close by nodes when the buffer

is low.

13

5. Architecture

The main modules in our solution include:

UsersServer

BufferManagment

Communication

Client

Messages

Network utilities

5.1. UsersServer

The UsersServer module is the module that keeps track on all of the users currently

connected to the system (it knows the exact position of each user in the system).

A user is been represented as a struct that amongst other things holds users’ name,

port, bandwidth and connection time.

All the users connected to the system are been held in a map data base.

In the map each entrance consist of key and value, the key is the number of the

group (e.g. 1st, 2nd etc.) and the value is a set of users assigned to this group.

The map is cyclic meaning that after a time equal to the movie length passes, the

first group will be deleted and the new group will be inserted in the first group place

and the list of users (hosts/peers), that users from this new group get, is a subset of

users from the last group. [Figure 3]

Upon user (host) connection the UsersServer assign him to one of the groups

according to his connection time and his desired offset, For example:

Lets say a time frame for each group is 5 minutes, two users (hosts) connected at

the same time to the system but one has desired offset of 1st minute and the other

with desired offset of 5th minute, they will be assigned in a two consecutive groups

where the user with desired offset of the 5th minute will be one group ahead of the

user that wanted to view the movie from the 1st minute.

14

Figure 3. The map is cyclic meaning if the time passed since the system had started is smaller than

the movie length the first group takes their content directly from the server (small red

arrow), if the time passed is bigger than movie length the “first” group is actually the first

group for the second time the movie is been “screened” so they have their content at users

currently in group number 6 (big red arrow).

The blue arrows represent the dependences of the groups (e.g. group number 2 gets their

content from users currently in group number 1).

The UsersServer has the abilities of insert and delete one user or list of users.

The ability to deal with list of users is necessary in times where user addressing to

the server and update him about list of users that are connected or disconnected

(right now while there is only one server the user cannot update the server about

connected users because obviously the server know about them but if the system

will be expanded into more than one servers this feature is needed).

5.2. BufferManagment

The BufferManagment module keep track on the packets currently buffered on each

peer (host).

Each user (host/peer) holds buffer (e.g. 20MB buffer size for a 700MB movie) in

which he buffers downloaded packets, those packets are used by the user (viewer)

and by other users (hosts/peers) that are currently downloaded packets from him.

15

We “divided” the buffer into two parts 1/3 of the buffer is used to serve the

user (viewer) and 2/3 of it is used to serve users (hosts/peers) in the system. [Figure 4a]

The buffer has three pointers marking its ‘first frame’, ‘last frame’ and ‘currently

seen by viewer frame’, using these frames the buffer “moves” along the video

stream.

The ‘currently frame seen by viewer’ is the pointer that helps keeping the 1/3 and

2/3 ratio, meaning that if the viewer had seen more than 2/3 of the buffer size that

‘first frame’ and ‘last frame’ should ‘go forward’. [Figure 4b]

Figure 4a. Users’ buffer divided in a 1/3 ratio.

Figure 4b. View over the whole video stream packets and the sliding window of users’ buffer in it.

The buffer size is calculated taking in consideration of the time frame by it (time

16

frame) users are gathered into groups, by that we guarantee that for each two

consecutive groups users from the advanced group have the needed content for

users at the latter group.

The BufferManagment is responsible for all buffer needs manageability, meaning

he knows from which frame the user (viewer) wanted to start see the movie (start or

some offset), he knows which frame the viewer currently sees (in order to keep the

exact ratio as discussed above), he knows which frame is the next frame to ask for,

he knows if a specific frame, been asked by other user (host/peer), currently exist in

the buffer, he know if a specific packet arrived is needed or can be discard (there

are two reasons to discard arrived packets one is if the packet already exist in buffer

and theother is if it came late and there is no place for it in the buffer)

BufferManagment way of action:

The buffer holds structs represent packets in the video stream.

Each struct holds its number (in the video stream), its size and a pointer to the data

it contain. On initialization the packets structs are been numbered in a sequential

manner and they hold NULL values note that they are empty (have no data), the

BufferManagment starts asking for the packets in a serial manner each packet that

have been asked is marked by multiply the packet number by (-1) upon arrival of

the wanted packet its number returns back to be positive and it is no longer hold

NULL only a pointer to the data of this packet.

Because there is a time constraint for missing packets if the number of packet,

currently been asked as next by the buffer management, is greater than a constant

number (currently implemented to be 10) from the first negative packet next packet

will be this packet (first negative), first negative means that it’s the smallest indexed

packet that have been asked but never arrived.

BufferManagment “makes the decision” about when the viewer can start watching

the movie.

Upon connection the viewer needs to wait couple of seconds before he can start

17

watch the stream in order to let the host (peer) buffer packets that would be enough

in order to watch the whole video stream without any problems. In our

implementation BufferManagment gives the o.k sign for the viewer to start

watching the video stream after buffering 2/3 of the buffer size.

BufferManagment is also responsible of writing the packets to the file for the

“player” to read and play.

5.3. Communication

The communication module is in charge of managing both incoming and outgoing

communication done by the user. This module runs in an infinite loop manner during

the session and handles the following tasks:

Receiving incoming connections

Creating outgoing connections

Sending and receiving control messages (TCP)

Dispatch incoming messages to the Client module.

Sending and receiving data messages (UDP)

5.4. Client

This module holds the logic of a user/server in the system. It defines what should be

done on every stage of the session, what messages should be sent and when and many

more.

This module connects between the Communication module to all other modules.

5.5. Messages

This module defines the format of the control messages. All messages inherit from

the basic message which holds the fundamental communication parameters. The type

of messages in the system consists of:

18

RequestFileInfo – The message sent for connecting/reconnecting to the system.

In this message we specify the name of the desired content, the offset and our

bandwidth.

ReplyFileInfo – The reply from the UsersServer to the connecting peers. This

message holds details regarding the desired content and the list of peers to

download from.

RequestChunck – This message is exchanged between the peers and holds the

desired chunks.

ReplyChunk – This message is sent by the peers to indicate the requested data

chunk is being sent.

5.6. Network utilities

This module implements all the low level communication commands. It implements the

TCP and UDP sending and receiving functions, socket handling and other

communication functionality.

19

6. Development Status

6.1 Development Overview

We designed, implemented and tested a large scale video on demand streaming

protocol that enables a large number of users to log in to the system with any desired

offset and on any time. The development process took place in the following realms:

Theoretical development – Before sitting down and designing the protocol we read

several articles dealing with streaming and slowly figured out the main idea of the

protocol. From this point and during the rest of the development stages we have

added and modified minor issues in the protocol.

Coding – This part was the longest and hardest, but in its end we had a working

system, enabling users to connect to a distant server and view a specific content.

Due to the fact that the protocol is influenced by a large number of variables

(bandwidth, packet sizes, communication protocols, buffer size and many more) we

have tried to implement it as general as possible in order to make the fine

configuration based on the test results.

LAN tests – Our first test sessions were done on the Local LAN and during these

tests we have configured the system based on the results.

On the LAN we have made hundreds of tests in different scenarios.

The results we came up with were more than satisfying as can be seen in the next

chapter.

Planetlab testing – Logging into Planetlab was very exciting and was the first time

we have worked on a distributed network, consisting of computers from all over the

world. Before starting the testing itself, we have learned how to distribute and run our

system on the network and then how to collect the logs from the distant computers.

The poor results on Planetlab made us make several improvements to the

implementation of the protocol.

20

6.2 Goals Achieved

Developing an unstructured distributed streaming protocol.

Coping real time constraints.

Testing the protocol on a heterogeneous LAN with no delay.

Testing the protocol on large-scale distributed network (PlanetLab) consist of

different machine types, bandwidth and delays.

Learning and using the newest video protocols – MPEG2, MPEG4 (divx).

Assembly and disassembly of video into frames.

Building a working prototype in C++ and not just simulation.

Debugging a distributed system.

Presenting the protocol in a conference outside of the university.

21

7. Xstream innovations

Our protocol has three main innovations.

7.1 Reduce Servers’ Workload

As Xstream protocol, conceptually grouping the users according to their connection

time and desired offset, the servers’ workload does not depend on the number of users

in the system, but only proportional to the size of the first group of users. [Figure 5]

Figure 5. Servers’ workload is proportional to the size of the first group of users.

In an ideal scenario in which there are continually incoming connections with the same

amount of users upon each connection time, the server would need to “spread” the

video stream content only to the first group and then it will be passed between each two

consecutive groups without any need of servers’ “interference”.

This innovation clearly decreases the servers’ workload as can be seen in the results

graph exhibit later on in this book.

7.2 Load balancing of the system

One of the major problems of conceptually grouping the users in the system into groups

according to their connection time and desired offset is dealing with bottlenecks created

on scenarios in which a small amount of users connected to the system and the ‘group’

after them is much bigger in a way that the small group of users can’t serve them

properly.

In our protocol, whenever such situation is discovered we try to load balance the system

by enforcing some of the users (hosts/peers) from the incoming group to act like two

hosts in two different groups. These users are the ones with the largest bandwidth.

22

In order to make it clear the hosts that been forced to act as two hosts act only as one

viewer and it is the viewer on the host that originally connected to the system, the other

host that was enforced to help the system to cope with the bottleneck does not have a

viewer[Figure 6]

Figure 6. Bottelneck encountering and its solution. Some users from the new group assigned to another

group in order to help the system (Colored hosts). The hosts that are marked with yellow dots

note that they are acting only as a host and don’t have a viewer, they are only helping the

system to cope with the bottleneck situation, the viewer is located on the ‘original’ host which

is the host in the new group.

7.3 Locality awareness

Each peer in the system has a list of peers from which he downloads the video stream

content. The decision from which peers to download is taking into consideration the

peers buffer state.

There are two states of the buffer:

FULL – The peers’ buffer is full and the viewer has enough packets in order to show

the movie without any faults.

NON-FULL – The peers’ buffer is not full and there are some packets that must be

“collected” as soon as possible in order to allow the viewer to show the

movie without any faults.

In times where peers buffer is FULL, the selection of the peers from which to download

will be taken randomly without any consideration to their distance from the peer that

asks for content.

23

In times where peers buffer is NON-FULL, the selection of the peers from which to

download will be taking under consideration of their distance from the peer that ask for

content, the closest peers will be chosen to be the peers to download the content from

them. After the peer coming out from the NON-FULL state he returns to a FULL state

and the peers selection is made according to the FULL state scenario.

Currently distance is based on peers’ bandwidth but it can be expand for latency and

other useful criteria.

24

8. System Performance

8.1 LAN Tests Overview

Xstream protocol has been tested over 200 tests of different scenarios.

The best scenarios were scenarios in which there were continually incoming

Connections with constant intervals and a constant group size, or continually

incoming connections with constant intervals and growing groups’ size.

Some “bad” scenarios were random scenarios in which there were times no users

connected to the system and groups sizes vary between 1 and 15, but in

Any case the results weren’t worse than the naïve solution in which each user that

Connects to the system downloads the content directly from the server.

The test took place over all the available computers in HUJI.CS meaning ‘gx-*’,

‘xil-*’, ‘os-*’, ‘cmos-*’, ‘bmos-*’ and ‘ocean’ used as the server.

Scripts that ran on more than 200 computers were used to test the different scenarios.

8.2 LAN Results

In the following graph [Figure 7] it can be seen clearly that we had improved the servers’

workload in compare to the naïve solution in which every user connected to the

System downloads the content directly from the server. The line representing Xstream

solution is not constant because in some realistic scenarios that were tested we saw the

server still has to help in distributing the content in the system. It can be seen that it is

reaching a constant line.

Figure 7. Xstream results graph.

25

8.3 LAN Deferent Scenarios

In this paragraph we will show some of the scenarios tested and we’ll explain the

results we got. In Appendix B the full results of these scenarios can be found.

The first scenario deals with 38 users connecting to the system on different times.

The numbers on the left (i.e. 0, 1, 2 etc.) shows the group number in the UsersServer.

Each group indicates a time frame in the movie (in this case 10 seconds).

Users in this scenario join the system in a growing rate. As for the results seen at the

end of Figure 8, we can see that the server have distributed the content only once and

the rest of the users exchanged the data between themselves without being helped by

the server.

******************* Date: 2005/05/30, Time: 13:38 *************************0 : User name is - os-01.cs.huji.ac.ilis - 33183 , offset is - 0 time: 11174489531 : User name is - os-02.cs.huji.ac.ilis - 32852 , offset is - 0 time: 1117448963 User name is - os-03.cs.huji.ac.ilis - 32832 , offset is - 0 time: 1117448963 User name is - os-04.cs.huji.ac.ilis - 32838 , offset is - 0 time: 1117448966 User name is - os-06.cs.huji.ac.ilis - 54065 , offset is - 0 time: 1117448965 User name is - os-07.cs.huji.ac.ilis - 32877 , offset is - 0 time: 1117448965 User name is - os-13.cs.huji.ac.ilis - 32955 , offset is - 0 time: 1117448972 User name is - os-16.cs.huji.ac.ilis - 33435 , offset is - 0 time: 11174489722 : User name is - os-08.cs.huji.ac.ilis - 33197 , offset is - 0 time: 1117448975 User name is - os-09.cs.huji.ac.ilis - 32930 , offset is - 0 time: 1117448976 User name is - os-10.cs.huji.ac.ilis - 52487 , offset is - 0 time: 1117448973 User name is - os-11.cs.huji.ac.ilis - 32858 , offset is - 0 time: 1117448973 User name is - os-12.cs.huji.ac.ilis - 33069 , offset is - 0 time: 1117448975 User name is - os-14.cs.huji.ac.ilis - 32978 , offset is - 0 time: 1117448976 User name is - os-15.cs.huji.ac.ilis - 32923 , offset is - 0 time: 11174489753 : User name is - os-17.cs.huji.ac.ilis - 32864 , offset is - 0 time: 1117448986 User name is - os-18.cs.huji.ac.ilis - 32892 , offset is - 0 time: 1117448986 User name is - os-19.cs.huji.ac.ilis - 33051 , offset is - 0 time: 1117448987 User name is - os-20.cs.huji.ac.ilis - 32889 , offset is - 0 time: 1117448984 User name is - os-21.cs.huji.ac.ilis - 32972 , offset is - 0 time: 1117448985 User name is - os-22.cs.huji.ac.ilis - 33012 , offset is - 0 time: 1117448985 User name is - os-23.cs.huji.ac.ilis - 32842 , offset is - 0 time: 1117448984 User name is - os-24.cs.huji.ac.ilis - 54718 , offset is - 0 time: 11174489864 : User name is - os-25.cs.huji.ac.ilis - 32819 , offset is - 0 time: 1117448993 User name is - os-26.cs.huji.ac.ilis - 32839 , offset is - 0 time: 1117448994 User name is - os-27.cs.huji.ac.ilis - 33108 , offset is - 0 time: 1117448998 User name is - os-28.cs.huji.ac.ilis - 32950 , offset is - 0 time: 1117448997 User name is - os-29.cs.huji.ac.ilis - 32842 , offset is - 0 time: 11174489995 : User name is - os-31.cs.huji.ac.ilis - 33286 , offset is - 0 time: 1117449006 User name is - os-32.cs.huji.ac.ilis - 36539 , offset is - 0 time: 1117449009 User name is - os-33.cs.huji.ac.ilis - 36384 , offset is - 0 time: 1117449007 User name is - os-34.cs.huji.ac.ilis - 35883 , offset is - 0 time: 1117449006 User name is - os-35.cs.huji.ac.ilis - 32837 , offset is - 0 time: 1117449006 User name is - os-36.cs.huji.ac.ilis - 36129 , offset is - 0 time: 1117449007 User name is - os-37.cs.huji.ac.ilis - 32841 , offset is - 0 time: 1117449006 User name is - os-38.cs.huji.ac.ilis - 36332 , offset is - 0 time: 1117449008 User name is - os-39.cs.huji.ac.ilis - 32839 , offset is - 0 time: 1117449005 User name is - os-40.cs.huji.ac.ilis - 33090 , offset is - 0 time: 1117449005

26

The number of packets in content is: 508The number of users connected is: 38The number of packets been out is: 508The ratio of packetsOut/packetsContent is: 1.000The number of users in first group is: 1

Figure-8. Xstream scenario I.

The second scenario we show deals with 223 users connecting to the system at random

order and as can be seen in Figure-9 , the results the total workload on the server was

11.791 a bit more than the size of the first group connecting to the system.

***************** Date: 2005/05/31, Time: 19:59 ******************The number of packets in content is: 508The number of users connected is: 223The number of packets been out is: 5990The ratio of packetsOut/packetsContent is: 11.791The number of users in first group is: 8

Figure-9. Xstream scenario II

The third scenario deals with 138 users connecting to the system in a continually and as can be seen in Figure-10 the workload on the server in even less then the size of the first group, meaning the first group have succeeded in exchanging the data between peers in the same group.

***************** Date: 2005/05/29, Time: 23:37 ******************* The number of packets in content is: 508The number of users connected is: 138The number of packets been out is: 2175The ratio of packetsOut/packetsContent is: 4.281The number of users in first group is: 9

Figure-10. Xstream scenario III

The fourth and last scenario deals with 60 users connecting to the system, where the size of the groups increases.In Figure-11 we can see the results indicating the small workload on the server.

**************** Date: 2005/05/30, Time: 09:57 *********************The number of packets in content is: 508The number of users connected is: 60The number of packets been out is: 1695The ratio of packetsOut/packetsContent is: 3.337The number of users in first group is: 2

Figure-11. Xstream scenario IV

27

8.4 Planetlab Tests Overview

Xstream protocol was tested on over than 200 computers of the Planetlab network

located through out the globe. For compiling and preparing the files we used a local

development node of Planetlab (Planet8.cs.huji.ac.il). As server we used another local

node of Planetlab (Planet5.cs.huji.ac.il). Then we were ready to deployed our code on

our designated slice on the Planetlab machines. Then using scripts we wrote we were

able to command the nodes all over the world to connect to the system.

During these tests we have encountered different issues than those we found on the

LAN tests. The main issues we’ve found were:

Dealing with a network limitation on UDP packets size

Network latency

The UDP packet size we used on the LAN for the data packets was 50,000 bytes which

is a bit smaller than the maximum UDP packet size. When trying to use the same

configuration on Planetlab we have seen that packets did not reach their destination.

After we investigate this issue a bit more we have found out that the maximum UDP

packet size transferred in the Planetlab network was 1024 bytes. This issue made our

system fail in distributing the content to the users on time due to the fact that for one

second of movie instead of asking for 1 packet on the LAN we needed to ask for 50

packets on Planetlab. In order to cope with this issue, we have modified our protocol to

enable sending of successive UDP data packets. These packets can be thought of as

“virtual” data packets of 50,000 bytes.

The second issue deals with the fact that the nodes of Planetlab are distributed all over

the world and the time it takes for a packet of data to reach from source to destination is

much larger than the time it takes on the LAN. This issue can be solved either by

requiring a minimal density of users in one geographical area (meaning we will require

at least X amount of users connecting from a country a continent or a network).

Due to this latency problem mentioned above the results we came up with were not

satisfying in terms of the time it takes the user to receive the data packets.

9. Future work

28

Xstream protocol can be expanded in the following directions:

Adjusting the protocol to the network latency – As we changed our

tests environment from HUJIs’ LAN found out that latency is a great

factor for packets to come on time, one of the improvements can be to

design the protocol for been able to adjust itself to the system latency

using “virtual packets” according to system latency.

Adjusting the system to cope with selfish peers – As our system

handles only ‘good’ users (peers) meaning, every one will to help the

system by buffering some packets in order to give them to users (peers)

behind upon request, a more complicate but realistic situations are peers

that only want to get packets but would not buffer for the users behind,

this situation make things worse because it need special treatment if you

‘bump’ into this kind of user while trying to get needed packet.

Enlarge number of servers/contents in the system – In our work we

focused on designing and implementing the protocol so in order to

make things less complicated we dealt with one server and one video

stream, the problem to deal with more than one server and more than

one video stream seems to be a nice expansion to this protocol in order

it to be more attractive.

One problem to solve would be making sure that the workload on all the

different servers will be equal.

Another problem to think about is gossiping, are servers going to

communicate and by that update each other or leaving the update only

between users and servers, the tradeoff between the methods would be

update against network load traffic.

Making the system commercial – As mentioned above our focus was to design

and implement streaming protocol, but it is a great challenge to make it a commercial

29

product, meaning making some optimizations in order to make it attractive some of

the optimizations are the two pointes mentioned above and other might be improving

GUI and player and other stuff related to marketing.

10. REFERNCES

[1] . Xu, Y. Wang, S. P. Panwar and K. W. Ross, A Peer-to-Peer Video-on-Demand System using Multiple Description Coding

30

and Server Diversity, IEEE International Conference on Image

Processing (ICIP), Oct. 2004[2] Y.Guo, K.Suh, J.Kurose, and D.Towsley, "A peer-to-peer on-demand streaming service and its performance evaluation", Department of Computer Science, University of Massachusetts Amherst.

[3] L.Xu, "Efficient and scalable On-Demand Data Streaming Using UEP Codes. ", Department of Computer Science Washington University in St. Louis.

[4] D.Bickson, D.Malkhi and D.Rabinowitz, Locality Aware Content Distribution -. - submitted for publication.

31

11. APPENDIX A

Main files and their description:

File Name Description

Client-main .cpp Holds the main of the program

UsersServer .h/cpp Manages the users in the system

BufferManagment .h/cpp Manages the data of the user

Communication .h/cpp Handles incoming and outgoing

communication issues.

Client .h/cpp Holds the logic of the user

Config .h/cpp Reads and holds parameters read from the

config file.

globals .h/cpp Holds the global objects in the system

messages .h/cpp Defines the format of messages in the

system and how to read and write them.

node .h/cpp Holds the implementation of a single node

in the system.

routing .h/cpp Holds implementation of a routing table of

a user in the system

Stats .h/cpp Gathers information used for statistics

libcommon .a Library holding common used

communication functionalities.

libtopologyd .a Holds functionalities regarding network

topologies.

32

12. APPENDIX B

************************************************ Date: 2005/05/30 , Time: 13:38

0 :

User name is - os-01.cs.huji.ac.il, port is - 33183 , offset is - 0 , connection_time is - 1117448953

1 :








2 :








3 :









4 :






5 :








33




The number of packets in content is: 508

The number of users connected is: 38

The number of packets been out is: 508

The ratio of packetsOut/packetsContent is: 1.000

The number of users in first group is: 1

*************************************************************************

************************************************ Date: 2005/05/31 , Time: 19:59

0 :

User name is - xil-143.cs.huji.ac.il, port is - 52453 , offset is - 0 , connection_time is - 1117558528





User name is - gx-37.cs.huji.ac.il, port is - 57992 , offset is - 0 , connection_time is - 1117558530



1 :












User name is - gx-110.cs.huji.ac.il , port is - 36841 , offset is - 0 , connection_time is - 1117558546



2 :

------ EMPTY ------------

3 :

------ EMPTY ------------

4 :


5 :




34
















6 :












7 :

User name is - cmos-08.cs.huji.ac.il, port is - 56534 , offset is - 0 , connection_time is - 1117558378













8 :






35



User name is - bmos-01.cs.huji.ac.il, port is - 41950 , offset is - 0 , connection_time is - 1117558392







9 :


10 :

------ EMPTY ------------

11 :

User name is - xil-10.cs.huji.ac.il , port is - 59957 , offset is - 0 , connection_time is - 1117558434

































36




12 :

















13 :


















14 :


15 :







37

























16 :




















38


*************************************************************************************

************************************************ Date: 2005/05/29 , Time: 23:37

0 :










1 :











2 :









3 :











4 :

39








5 :








6 :






7 :








8 :

















9 :

40












10 :










11 :





12 :


13 :














14 :





41









15 :




16 :












***********************************************************

************************************************ Date: 2005/05/30 , Time: 09:57

0 :

------ EMPTY ------------

1 :



2 :



3 :

------ EMPTY ------------

4 :








42

5 :








6 :














7 :

























43











44

xstream final book.doc

Documents

demand streaming protocol

users peers

astreaming protocol

streaming protocols

bottlenecks of users

relevant users

abstract streaming video

small number of users