an analysis of the p2p traffic characteristics on file transfers between prefectures and between...

An Analysis of the P2P Traffic Characteristics

on File Transfers Between Prefectures and Between Autonomous Systems

in the Winny Network

Nov. 1, 2010 1

Takashi Koizumi, Masahiro Yoshida, Satoshi Ohzahata, Konosuke Kawashima

Tokyo University of Agriculture and Technology,The University of Tokyo,

The University of Electro-Communications,

APCC2010

1)

1)

3)

2)

1)

3)

2)

APCC2010 2

Outline Introduction File distribution protocol in Winny Analysis method

Definition of the file uploading peers Definition of the file transfer between peers

Measurement environment Measurement results

Analysis between prefectures Analysis between ASs

Conclusion

Nov. 1, 2010

APCC2010 3




Analysis between prefectures Analysis between ASs

Conclusion

Nov. 1, 2010

APCC2010 4

Introduction P2P file sharing network generates huge traffic.

it has an enormous impact to backbone network. however, the technology of P2P is useful.

VOD, Video streaming, etc...

⇒ Analyzing P2P traffic is important.

Problems: many ISPs do not make public report

on their traffic data. difficulty of identifying file transfers

from where did peer download files ?

Nov. 1, 2010

P2P network

APCC2010 5

Introduction Analysis methodology

identify the file transfers between peers in Winny network.

Analyze the characteristics of the traffic matrix. focus on between ASs and between Japanese

Prefectures.

Winny network one of the most popular P2P file sharing applications in

Japan. it generates 20% of traffic in one ISP in 2008.

Reveal the factor of increasing traffic from P2P file-sharing.

Nov. 1, 2010

APCC2010 6




analysis between prefectures analysis between ASs

Conclusion

Nov. 1, 2010

APCC2010 7

File distribution protocol in Winny

Nov. 1, 2010

Feature of Winny network- unstructured P2P

network.- layered structure.- diffuse metadata of files

(KEY)- 2 types of query.

KEY information

file name

file location(IP address, service port)

file hash value

lifetime of KEY (TTL)

etc...

(1) File uploading peer diffuses KEYs to other peers.

(2) File searching peer generates search queries to find KEYs.

(3) Download the file from the location of file uploading peer included in the KEY.

File uploading peer

File searching peer

APCC2010 8





Conclusion

Nov. 1, 2010

APCC2010 9

Analyze time series of file KEY. focus on TTL and the file location in the KEY. 1500 < TTL < 2000. file location is same with the KEY having peer.

Peer A

Definition of the file uploading peers

Nov. 1, 2010

Crawlersearch query

query hit

Peer A is file uploading peer

: File

: File KEY

1500 < TTL < 2000

file location is Peer A.※TTL should be smaller than 2000 seconds to exclude spam KEYs.

TTL of the KEY in file uploading peer is set to be greater than 1500 seconds.

APCC2010 10





Conclusion

Nov. 1, 2010

APCC2010 11

Definition of the file transfer between peers process of the file transfer.

changes of a KEY having peer to a file uploading peer.

Nov. 1, 2010

Peer APeer B

① Peer A is the file uploading peer ② Peer B is the KEY having peer

① Diffusion of KEY

③ Peer B becomes a file uploading peer

Peer B

④ file transfer to Peer B from Peer A

: File

: File KEY

② Download demand

Peer A

with continuous crawling

APCC2010 12





Conclusion

Nov. 1, 2010

APCC2010 13

Measurement environment

Winny Crawling System collects KEYs. crawl more than 100,000 peers within 10 minutes.

Measurement period Aug. 4, 2009 01:00 ~(12 hours)

Nov. 1, 2010

6 crawlers

DB server

Management

server

APCC2010 14

Measurement environment

Types of measured file. 15 popular files (selected in 1 hour crawling). books, movies and compressed files. average file size is 245MB

Nov. 1, 2010

6 crawlers

DB server

Management

server

APCC2010 15





Conclusion

Nov. 1, 2010

APCC2010 16

Measurement results Analysis of the file transfer characteristics

focus on between prefectures and between ASs.

⇒ Origin-Destination file transfer matrix.

measurement summary about 220,000 peers, 280 ASs. about 30,000 instances of the file transfers between 33,000

peers

Nov. 1, 2010

※ results are for 47 prefectures and top 50 ASs with number of file transfers　　 about 15 popular files.

APCC2010 17





Conclusion

Nov. 1, 2010

APCC2010 18

File transfer traffic matrix between prefectures src prefectures on Y-axis, dst prefectures on X-axis large portion of file transfers are generated in few

prefectures.

Nov. 1, 2010

Destination Prefectures

Source Prefectures

Tokyo(dst) Osaka(dst)

Tokyo(src)

Osaka(src)

%

highlow

APCC2010 19

File transfer traffic matrix between prefectures the sum of elements is 100% for each row. similar traffic distribution in every prefectures.

Nov. 1, 2010

Destination Prefectures

Source Prefectures

within-prefectures

highlow

src and dstis same prefecture

APCC2010 20

Comparison of Winny and telephone traffic

upper: Winny below: telephone traffic (in

2007)

traffic within-prefectures Winny: 2% telephone traffic: 70%

similar prefectures are selected as destinations.

Tokyo, Osaka, etc.

Nov. 1, 2010

Tokyo Osaka

APCC2010 21

Comparison of Winny and Internet upper: Winny below: Internet (in 2008)*

traffic within-prefectures. Winny: 2% Internet: 3% few locality of traffic.

distribution patterns are similar.

Nov. 1, 2010

* Kenjiro Cho, Kensuke Fukuda, Hiroshi Esaki, Akira Kato, Jun Murai, “Where does all the traffic go? – observing trends in Japanese residential traffic –,” International Workshop on Protocols for Future, Large-Scale and Diverse Network Transports, 2009.

APCC2010 22





Conclusion

Nov. 1, 2010

APCC2010 23

File transfer traffic matrix between ASs src ASs on Y-axis, dst ASs on X-axis large portion of file transfers is generated by few

ASs. 20 ASs generated 85% of total number of file transfers.

Nov. 1, 2010

Source ASs

Destination ASs

%

highlow

APCC2010 24

File transfer traffic matrix between ASs the sum of elements is 100% for each row. traffic distribution is similar in each AS. very poor locality of AS network in Winny.

Nov. 1, 2010

Source ASs

Destination ASs

within-AS

highlow

APCC2010 25





Conclusion

Nov. 1, 2010

APCC2010 26

Conclusion Analyzed traffic characteristics in Winny

network. many file transfers are generated by few

prefectures or ASs. since there is no mechanism to consider IP network,

many file transfers are not within-prefecture or within-ASs.

Future work long term measurement. analysis and measurement of other P2P networks

(BitTorrent, Share, etc.).

Nov. 1, 2010

APCC2010 27

THANK YOU VERY MUCH FOR YOUR KIND ATTENTION.

Nov. 1, 2010

an analysis of the p2p traffic characteristics on file transfers between prefectures and between...

Documents