CHAPTER 20

BUILDING CONTENT DELIVERY NETWORKS

Numerous “storage cloud” providers (or “Storage as a Service”) that can provide Internet-enabled content storage and delivery capabilities in several continents(across the globe), offering service-level agreement (SLA)-backed performance and uptime promises for their services.

Why cloud only for CDN / storage?

Customers are charged only for their utilization of storage and transfer of content only.

Economies of scale, in terms of cost effectiveness and performance for both providers and end users, could beachieved by leveraging existing (Amazon simple storage and cloud front, Nirvanix delivery network, Rackspace cloud files etc) “storage cloud” infrastructure, instead of investing large amounts of money in their own content delivery platform or utilizing one of the incumbent operators like Akamai.

With the existing cloud providers,

Even though the existing emerging services have reduced the cost of content storage and delivery by several orders of magnitude, but they can be difficult to use for non-developers, because each service is best utilized via unique Web services or programmer APIs and have their own unique quirks.

Howeven,

There is no general-purpose, Reusable framework to interact with multiple storage cloud providers and Leverage their services as a content delivery network. Most “storage cloud” providers are merely provides basic file storage and delivery services and

do not offer the capabilities of a fully featured CDN such as o Automatic replication,

o Fail-over,

o Geographical load re-direction and load balancing.

o Furthermore, a customer may need coverage in more locations than offered by a single

provider.

To overcome all this, Introduced MetaCDN, a system that utilizes numerous storage providers in order to create an overlay network that can be used as a high-performance, reliable, and redundant geographically distributed CDN.

Comparison in existing cloud providers:

Amazon simple storage and cloud front, Nirvanix delivery network, Rackspace cloud files Azure storage service

Feature comparison of cloud storage providers:

Pricing comparison

META CDN: creating an integrated overlay network that aims to provide a low-cost, high-performance,easy-to-use content delivery network for content creators and consumers.

ARCHITECTUR

The MetaCDN service is presented to end users in two ways.

First, it can be presented as aWeb portal, which was developed using (a) Java Enterprise and Java Server Faces (JSF) technologies, with a MySQL back-end to store user accounts and deployments, and (b) the capabilities,pricing, and historical performance of service providers. The Web portal acts as the entry point to the system and also functions as an application-level load balancer for end users that wish to download content that has been deployed by MetaCDN.

The second method of accessing the MetaCDN service is via RESTful Web Services. These Web Services expose all of the functionality of the MetaCDN system.

GENERAL framework of Meta CDN- client can choose preferred cloud provider.

Integrating “Cloud Storage” Providers:

DefaultConnector, prescribes the basic functionality that each provider could be expected to support, and it must be implemented for all existing and future connectors. These include basic operations like creation, deletion, and renaming of replicated files and folders.If an operation is not supported on a particular service, then the connector for that service throws a FeatureNotSupportedException.

Overall Design and Architecture of the System

The MetaCDN service has a number of core components that contain the logicand management layers

These components include

MetaCDN Allocator, which (a) selects the optimal providers to deploy content to and (b) performs the actual physical deployment. The MetaCDN Allocator allows users to deploy files either directly (uploading a file from their local file system) or from an already publicly accessible origin Web site (sideloading the file, where the backend storage provider pulls the file).

The MetaCDN QoS monitor tracks the current and historical performance of participating storage providers, and The MetaCDN QoS Monitor tracks the performance of participating providers (and their available storage and delivery locations) periodically, monitoring and recording performance and reliability metrics from a variety of locations, which is used for QoS-optimized deployment matching.

The MetaCDN Manager tracks each user’s current deployment and performs various housekeeping tasks. The MetaCDN Manager has a number of housekeeping responsibilities. First, it ensures that all current deployments are meeting QoS targets of users, Second, it ensures that replicas are removed when no longer required, Third, for users that have made cost-optimized deployments, it ensures that a user’s budget has not been exceeded, by tracking usage

The MetaCDN Database stores crucial information needed by the MetaCDN portal, ensuring reliable and persistent operation of the system. It stores crucial information needed by the MetaCDN system, such as MetaCDN user details, their credentials for various storage cloud and other providers, and information tracking their (origin) content and any replicas made of such content.

The MetaCDN Load Redirector is responsible for directing MetaCDN end users (i.e., content consumers) to the most appropriate file replica, ensuring good performance at all times.

Integration of Geo-IP Services and Google Maps

The principle of cloud computing and storage is that you shouldn’t need to care where the

processing occurs or where your data are stored

Cloud storage users to know precisely where their data are stored.

This information is crucial for MetaCDN load balancing purposes, so end users are

redirected to the closest replica, to maximize their download speeds and minimize

latency.

To address this issue, MetaCDN offers its users the ability to pinpoint exactly where their

data are stored via geolocation services and Google Maps integration.

Load Balancing via DNS and HTTP

The MetaCDN Load Redirector is responsible for directing MetaCDN end users (i.e.,

content consumers) to the most appropriate file replica.

When a MetaCDN user deploys content, they are given a single URL, in the format

http://www. metacdn.org/MetaCDN/FileMapper?itemid5{item_id}, where item_id is a

unique key associated with the deployed content.

This provides a single namespace, which is more convenient for both MetaCDN users

(content deployers) and end users (content consumers), and offers automatic and totally

transparent load balancing for the latter.

Different load balancing and redirection policies can be utilized, like

simple random allocation, where end users are redirected to a random replica;

geographically aware redirection, where end users are redirected to their physically

closest replica;

least-cost redirection, where end users are directed to the cheapest replica from the

content deployer’s perspective; and

QoS-aware redirection, where end users are directed to replicas that meet certain

performance criteria, such as response time and throughput.

PERFORMANCE OF THE METACDN OVERLAY

Random (RAN): End users were directed to a random replica.

Geolocation (GEO): End users were directed to the closest physical replica

Cost (COST): End users were directed to the cheapest replica.

Utility aware (UTIL): End users were directed to the replica with the highest utility,

where utility depends on the weighted throughput for requests, the user-perceived

response times from direct replica access and via MetaCDN, the unit replication cost,

and the content size.