global data flow report - cloudtalk.it...in this first edition of the global data flow report you...

© 2012 DiViNetworks and/or its affiliates. All rights reserved.

Global Data Flow Report

September 2012

Global Data Flow Report September 2012

© 2012 DiViNetworks and/or its affiliates. All rights reserved. of 15

Introduction

The present document provides measurements and statistics of data traffic flow throughout the world.

DiViNetworks operates a global network – DiViCloud – delivering virtual capacity to ISPs. Most of the

ISPs served by the network are located in countries where international capacity prices are high, and

thus virtual capacity is the only affordable way to expand international bandwidth.

We hold a unique point of view – global PoPs positioned at strategic Internet junctions, combined with

intelligent software overseeing masses of international traffic. This position enables us to study global

data flow characteristics and trends. We would like to share these observations with you.

In this first edition of the Global Data Flow Report you can view:

¨ Mixture of sources comprising international traffic flowing to different global regions

¨ Changes in traffic mixture during time

¨ Effects of local CDN presence on international traffic mixture

¨ Effects of local caching mechanisms on international traffic mixture

¨ Traffic redundancy left after applying local CDN or local caching

We will be glad to receive your feedbacks, questions and ideas for future global data flow reports. To

provide your feedback and/or register to future reports, please follow us at

www.linkedin.com/company/divinetworks.

Yair Shapira, Ph.D.

VP Marketing & Business Development



Methodology

DiViNetworks operates the DiViCloud network, serving ISPs worldwide with virtual capacity (see

http://www.divinetworks.com/divicloud). DiViCloud comprises of PoPs (points of presence) located at

major Internet junctions (see below – map of PoPs at the time of data acquisition).

Figure 1: The DiViCloud PoPs network at the time of data acquisition for this report

Close to 100 ISPs are served by the DiViCloud network. 100% of the international downstream traffic to

the served ISPs passes through these PoPs. As traffic passes through the PoPs, the software in the PoPs

collects traffic statistics. These statistics serve as the raw data for the current report.

The information gathered includes:

¨ Originating AS (autonomous system)

¨ Destination AS

¨ Traffic volume

¨ Bit-stream repetition

Bit-stream repetition is a measure of the superfluous data passing on the link. It actually measures the

portion of the traffic payload which had already passed on the same link within the measurement

period. Traffic payload is defined as the net cargo of the link, excluding headers (ETH, IP, TCP), and

control packets.

Repetition is measured at the bit-stream level, as opposed to the file or content levels. Payload chunks

of 500 bytes are matched vs. a history map. Those chunks which are matched are considered a

repetition. Note that the chunk-matching is agnostic to protocol. Thus, the same chunk might be carried

multiple times using different protocols, but is still considered repetition.



Only international traffic passes through the DiViCloud PoPs. Local traffic is not accounted for in the

analysis. The figure below depicts a single ISP, for simplicity, highlighting which parts of the traffic are

included in the analysis, and which do not.

Figure 2: Traffic included and excluded from the analysis

AS’s have been categorized into several categories:

¨ Named CDNs: e.g. Limelight, Edgecast, Akamai

¨ Named content: e.g. Google, Facebook

¨ Hosting: Aggregate of hosting companies (e.g. MediaFire, LeaseWeb)

¨ Carrier: Aggregate of telecom companies (e.g. Cogent, Telia)

¨ Usenet: Aggregate of usenet AS’s (e.g. Miniclip, Eweka)

ISPs have been categorized to 5 different regions, served by DiViCloud:

¨ Europe: Specifically remote islands of European countries

¨ CIS: Specifically the *stan countries

¨ S.E. Asia

¨ Africa: Specifically sub-Saharan Africa

¨ LATAM: Including Central and South America, as well as the Caribbean



Source AS mixture in different regions

This analysis takes a look at the main sources of traffic in different regions, split by originating AS.

Hosting, Carrier, and Usenet are an aggregation of AS’s. The analysis does not reach 100%, due to the

long tail of source AS’s not categorized in this analysis.

Figure 3: Source AS mixture in different regions



The following analysis highlights the dominance of the CDNs in the international traffic to different

regions. Note that since the analysis takes only international traffic into account, the traffic from local

CDN PoPs in excluded.

Figure 4: Contribution of CDNs to traffic in different regions



Source AS at different times

The following charts analyze the changes of the source AS mixture flowing over the international link

during times of day. Data was collected from a single ISP in LATAM.

Note that the top chart shows AS mixture in percentage, regardless of the total traffic. The total traffic

(relative to the peak hour) is presented in the bottom graph.

Figure 5: Source AS mixture over 24 hours in a LATAM ISP (top);

Total traffic as % of traffic in peak hour (bottom)



Data from the same LATAM ISP was analyzed to obtain the difference in source AS mixture at 16:00

every day, over a week time.

Figure 6: Source AS mixture over 7 days in a LATAM ISP (top);

Total traffic as % of traffic in peak day (bottom)



Figure 7: Contribution of source AS’s to traffic in different days of the week (measure with a LATAM ISP)



Effect of local CDN on traffic mixture and repetition

The following analysis takes a look at two ISPs in LATAM. The first ISP in a country without PoPs of any

major CDN, so that all non-local traffic actually flows through the DiViCloud PoP in Miami. The second

ISP is in Brazil, where major CDNs are present, hence large portion of the traffic does not pass in the

DiViCloud PoP in Miami. Both ISPs do not use local caching.

Figure 8: International traffic mixture in ISPs with and without local CDN presence

As expected, the CDNs present in Brazil indeed attenuate their respective traffic portions in the

international traffic. Other parts of the traffic increase their relative portion in presence of local CDN.



The next diagram presents the repetition rate (see methodology section) of the international traffic

flowing to the two abovementioned ISPs.

Figure 9: International traffic repetition-rate by AS in ISPs with and without local CDN presence

Surprisingly, repetition rate in the international traffic increases in the presence of a local CDN. This may

be explained by the tendency of local CDNs to inflate local traffic served from the CDN cache. This static

cached data pulls dynamic international data, which may be highly repetitive.

Thus, even in presence of local CDN there is still significant redundancy in the traffic flowing over

international links, which can be utilized to generate virtual capacity.

In these specific two cases CDN is shown to inhibit international traffic volume from its global network,

and to increase repetition rate in this traffic.



Effect of local caching on traffic mixture and repetition

The following analysis examines two ISPs in Africa. The first ISP is not using any caching mechanism, so

that all non-local traffic actually flows through the DiViCloud PoP. The second ISP is using local caching;

hence cached traffic does not pass in the DiViCloud PoP. Both ISPs are in countries without CDN

presence.

Figure 10: International traffic mixture in ISPs with and without local caching

The caching shows effectiveness on several traffic segments, mainly traffic originating at: Google,

Level3, Microsoft, Akamai, Amazon and Facebook.

Other traffic segments, mainly the long tail of hosted content and carrier-originated content are poorly

handled by caching.



The next diagram presents the repetition rate (see methodology section) of the international traffic

flowing to the two abovementioned ISPs.

Figure 11: International traffic repetition-rate by AS in ISPs with and without local caching

Interesting to note that in these segments there is still a high repetition rate in these traffic portions,

which are positively handled by the caching mechanism. Traffic originating at Google, Level3, Microsoft,

Akamai, Limelight – still demonstrates high redundancy over the international link even in presence of

local caching mechanisms.



Long tail breakdown

The table below lists the long tail of hosting, content and enterprise AS sources. The figures refer to the

average percentage each AS contributes to the overall international traffic of the different ISPs.

OVH Systems 3.76% OJSC RTComm.RU 0.10% Donbass Electronic 0.03%

LeaseWeb B.V. 2.72% Pandora Media, Inc 0.10% GoDaddy.com, LLC 0.03%

MediaFire, LLC 2.68% Radore Hosting 0.10% Reseau National tele. 0.03%

Link11 GmbH 2.05% CRYPTIC STUDIOS, INC. 0.10% Savvis 0.03%

SoftLayer Technologies 1.94% Rackspace Hosting 0.10% www.jeuxvideo.com 0.03%

M247 Ltd 1.50% Virgin Media Limited 0.09% HIVELOCITY Ventures 0.03%

LL company Mail.Ru 1.39% Justin.tv, Inc. 0.09% YACAST Media AS 0.03%

WebaZilla European 1.32% EOS - MOJOHOST 0.09% Colo4Dallas LP 0.03%

Portlane Network 1.08% Interactive3D 0.09% Rede Ciencia Tecn. 0.03%

FDCservers.net 0.99% Hosting Solutions Int’l 0.08% Zynga Inc. 0.03%

ADVANCEDHOSTERS 0.93% Hosting Solutions Int’l 0.08% GloboTech Comm. 0.03%

Dailymotion S.A. 0.90% M100 LLC M100 0.08% CWIE - CWIE LLC 0.03%

Free SAS 0.63% Hosting Hizmetleri Ltd 0.08% Hosting Services Inc 0.03%

WZ Communications Inc. 0.62% M6 WEB S.A.S. 0.08% Internet Center Ltd. 0.03%

JSC ""TRC FIORD 0.52% SWIFTWAY AS 0.07% MetaPeer 0.03%

NFOrce Entertainment BV 0.49% LALIB-GESTAO 0.07% Nobis Technology 0.03%

Hewlett-Packard 0.37% Base IP B.V. 0.07% LLC ""Multiscan 0.03%

ThePlanet.com Internet 0.36% Gigabase ltd 0.07% Bluecoat Systems, Inc. 0.03%

Opera Software ASA 0.35% root SA 0.07% Universidade Parana 0.02%

Private Layer INC 0.33% Linux Security Group 0.07% DNSSLAVE.COM 0.02%

Panther Express 0.30% ISPpro Internet KG 0.06% Instituto Costarr. 0.02%

Leaseweb USA, Inc. 0.28% UPLOADING.COM USA 0.06% GigeNET 0.02%

DCP Networks 0.24% PlusServer AG 0.06% Livestream LLC 0.02%

CJSC RuTube 0.23% Versaweb, LLC 0.06% Fasthosts Internet 0.02%

OAO Webalta 0.22% Namesco Limited 0.06% Voxel Dot Net, Inc. 0.02%

New Dream Network, LLC 0.21% WestHost, Inc. 0.05% Media Temple, Inc. 0.02%

Hetzner Online AG RZ 0.20% Redstation Limited 0.05% Internet Archive 0.02%

Choopa, LLC 0.18% MnogoByte LLC 0.05% GARR Italian academic 0.02%

Voxility SRL 0.18% The JANET IP Service 0.05% Server Boost B.V. 0.02%

GoDaddy.com, Inc. 0.16% Giga-Hosting GmbH 0.05% .masterhost AS 0.02%

1&1 Internet AG 0.16% Giga-Hosting GmbH 0.05% conova comm. 0.02%

Bluehost Inc. 0.16% HostDime.com, Inc. 0.04% Gameforge Product. 0.02%

iWeb Technologies Inc. 0.16% CCANet Limited 0.04% Hewlett-Packard 0.02%

Yandex LLC 0.16% UK-2 Ltd AS 0.04% ARKA UNIVERSAL 0.02%

Lemuria Comm. Inc. 0.14% Infomaniak Network 0.04% Netmagic Datacenter 0.02%

Axill Europe Ltd. 0.13% CloudFlare, Inc. 0.04% IPTelligent LLC 0.02%

SingleHop 0.13% eServer.ru - hosting 0.04% USTREAM.TV INC 0.02%

i3d B.V. 0.13% Bluemile, Inc 0.04% Hotmail Corporation 0.02%

Internap Network 0.13% United Network LLC 0.04% MediaCom Ltd. 0.02%

NetByNet Holding 0.12% NationalNet, Inc. 0.04% ABCONNECT AB 0.02%

Tvigo Entertainment 0.12% Accretive Networks 0.04% intergenia AG 0.02%

RCS & RDS S.A. 0.12% ServerClub Inc. 0.04% Liquid Web, Inc. 0.02%

Canonical Ltd 0.11% OXEVA SAS 0.04% JSC Centrohost 0.02%

Ezri Inc 0.10% ISPrime, Inc. 0.03% Leaseweb Germany 0.02%



To learn more about virtual capacity please visit us at www.divinetworks.com. To register to future

global data flow report, please follow us at www.linkedin.com/company/divinetworks.

DiViNetworks, DiViCloud, DiVination, DiViLink are trademarks or registered trademarks of

DiViNetworks and/or its affiliates. Third party trademarks mentioned are the property of their

respective owners.

global data flow report - cloudtalk.it...in this first edition of the global data flow report you...

Documents