CLOUD COMPUTING: OPPORTUNITIES AND CHALLENGES

What is Cloud Computing? • A coherent, large-scale, publicly accessible collection of compute, storage, and networking resources

• Available via Web service calls through the Internet

• Short- or long-term access on a pay per use basis

Motivation for Cloud Computing •  Internet-based service provisioning

•  Provision is usually overestimated to deal with usage peaks •  Peaks are rare and relatively short •  Estimates indicate that data centers only use about 10-15% of their capacity

• Over-provisioning is expensive •  Electricity and cooling costs for low demand servers •  High personnel costs for managing and maintaining the physical

and hardware infrastructure

Motivation for Cloud Computing • Application deployment is non-trivial

•  Need to configure hardware, OS, network, storage devices, and install all the required application software (web server, application server, DBMS)

• Global financial crisis •  Cloud computing as a cost-effective alternative to cut down IT costs

Motivation for Cloud Computing • Global revenue for platform as a service (PaaS) will grow

consistently for five years from 2012 to reach $2.9bn in 2016, up from $900m in 2011, according to research firm Gartner.

• Of all the cloud technological aspects, infrastructure as a service (IaaS) and software as a service (SaaS) are the most mature and established from a competitive landscape perspective, while PaaS is the least evolved,

Cloud General Characteristics • Scaling up and scaling down of resource usage on an as-needed basis

• Economies of scale: The cloud provider can procure real estate, power, cooling, bandwidth, and hardware at the best possible prices

• Pay-as-you-go: Resource allocation decisions have an immediate effect on resource consumption and the level of overall costs

Cloud Business Characteristics • No up-front investment:

•  On-demand usage for resources, there’s no need to make a large one-time investment before actual demand occurs

•  Fixed costs become variable, CAPEX becomes OPEX: •  Resource consumption changes in real time

• Allocation is fine-grained: •  Minimal usage amounts for both time and resources •  For example: Hours of server usage, bytes of storage

Cloud Business Characteristics •  The business gains flexibility:

•  No long-term commitment to resources, the business is able to respond rapidly to changes in volume or the type of business

• Business focus of provider: •  The cloud provider has a strong incentive to supply services that

are reliable, applicable, and cost-effective

Cloud Technical Characteristics • Scaling is quick:

•  New hardware can be brought online in minutes to deal with unanticipated changes in demand, either internally (large compute jobs) or externally (traffic to a web site)

•  Infinite scalability illusion: •  Everyone can treat the cloud as if it offers near- infinite scalability •  There’s no need to provision ahead of time; dealing with surges

and growth in demand is a problem for the cloud provider

Cloud Technical Characteristics • Resources are abstract and undifferentiated:

•  Users focus on the relevant details—results and the observable performance

•  Underlying hardware will change and improve over time, but it’s the job of the provider to stay on top of the technology

• Clouds are building blocks: •  IT resources are individual, separately priced, atomic-level building

blocks •  The consumer can choose to use none, all, or some of the services

offered by the cloud

• Experimentation is cheap: •  You can access temporary resources to try out a new idea without

making long-term commitments to hardware

TIME

IT C

APA

CIT

Y

Actual Load

Allocated IT-capacities

“Waste“ IT Capacities

“Under-supply“ IT Capacities

Fixed Cost of IT-capacities

Load Forecast

Over-provisioning – Out of Cloud

Courtesy of Microsoft

Allocated IT capacities

Reduction of initial

investments

Reduction of “over-supply“

No “under-supply“

Possible reduction of IT-capacities

in case of reduced load

IT C

APA

CIT

Y

TIME

Actual Load

Load Forecast

Cloud-provisioning Reduce Costs

Improve Performance

Courtesy of Microsoft

Cloud Usage Patterns • Constant usage over time:

•  Common for internal applications where there’s little variation in usage or load from day to day or hour to hour

• Cyclic internal load: •  Characteristic for batch or data processing applications run on a

predictable cycle, such as close of business for the day or month •  The load, both in time and expected resource consumption, is

highly predictable

• Cyclic external load: •  Often applies to web sites that serve a particular market demand •  Sites related to entertainment and sporting events often fit this

pattern

Cloud Usage Patterns • Spiked internal load:

•  Typical in environments where researchers or analysts can submit large-scale, one-time jobs for processing

•  The demand is usually unpredictable

• Spiked external load: •  Seen on the Web when an unknown site suddenly becomes

popular, often for a very short time

• Steady growth over time: •  Usually for a mature application or web site •  As additional users are added, growth and resources track

accordingly

NIST Cloud Models

Service Models

Deployment Models

Service Models • Software as a Service:

•  On demand applications over the Internet •  Examples: Google Docs, Salesforce.com, and SAP Business ByDesign

• Platform as a Service: •  Platform layer resources, including operating system support and

software development frameworks •  Examples: Google App Engine, Microsoft Windows Azure, and

Force.com

•  Infrastructure as a Service: •  On-demand provisioning of infrastructural resources, usually in

terms of VMs •  Examples: Amazon EC2, GoGrid, and Flexiscale, Rackspace

SaaS Systems •  The application itself is provided by the service provider •  The first used service • Most known and most used •  The application is completely managed by the provider • Used in business applications

•  Customer Relationship Management (CRM) •  Enterprise Resource Planning (ERP) •  Human HRM

SaaS Systems

SaaS Systems •  Freemium model •  Typically offered with a subscription fee (i.e., monthly,

annual, per user, etc.) • Often implemented with a multi-tenant architecture

•  Single version of the application for all users

•  Typically used through browsers • Customers may specialize the application

•  Set configuration options •  Brands

SaaS Systems – Pros & Cons • Pros

•  Zero time configuration •  Automatic scaling •  Software maintenance (i.e., updates) is done by the provider

• Cons •  No flexibility •  High vendor lock-in

SaaS Systems – Major players

PaaS Systems - Pros & Cons • Pros:

•  No need to configure infrastructure •  Quick development •  Scaling is automatically managed •  Deploy is easier •  Finer cost granularity

• Cons: •  Tied to the specific framework offered by cloud provider •  Less flexibility

PaaS Systems

PaaS Systems – Major Players

IaaS Systems • Elements of the infrastructure are made available by

service providers •  Lowest level of abstraction •  For example:

•  Computing elements (i.e., Virtual Machines) •  Raw (block) storage •  Load balancers •  Firewalls •  ...

IaaS Systems – Pros & Cons • Pros:

•  Maximum flexibility •  Extreme customization (OS, frameworks, computing power, etc.) •  Exploit geographical location

• Cons: •  Higher configuration complexity

IaaS Systems – Major Players

Service Models

Storage

Server HW

Networking

Servers

Databases

Virtualization

Runtimes

Applications

Security & Integration

You

don’

t man

age

any

Storage

Server HW

Networking

Servers

Databases

Virtualization

Runtimes

Applications

Security & Integration Yo

u m

anag

e

Storage

Server HW

Networking

Servers

Databases

Virtualization

Runtimes

Applications

Security & Integration

You

man

age

Managed by vendor

Managed by vendor

Courtesy of Microsoft

Deployment Models •  Public clouds:

•  Service providers offer their resources as services to the general public •  Public clouds offer several key benefits to service providers •  Lack fine-grained control over data, network and security settings,

which hampers their effectiveness in many business scenarios

•  Private clouds: •  Also known as internal clouds, designed for exclusive use by a single

organization •  A private cloud may be built and managed by the organization or by

external providers •  A private cloud offers the highest degree of control over performance,

reliability and security •  Criticized for being similar to traditional proprietary server farms and do

not provide benefits such as no up-front capital costs

Deployment Models •  Hybrid clouds:

•  A combination of public and private cloud models that tries to address the limitations of each approach. Part of the service infrastructure runs in private clouds while the remaining part runs in public clouds

•  Offer more flexibility than both public and private clouds: •  Provide tighter control and security over application data compared to public

clouds •  Facilitate on-demand service expansion and contraction

•  Designing a hybrid cloud requires carefully determining the best split between public and private cloud components

•  Community Cloud: •  The cloud infrastructure is shared by several organizations and

supports a specific community that has shared concerns (e.g., mission, security requirements, policy, or compliance considerations)

•  Managed by the organizations or a third party and may exist on-premises or off-premises

Top 10 Obstacles for Cloud Computing •  Availability of a service:

•  Organizations worry about whether Cloud services will have adequate availability •  Very (very) high availability can be achieved by adopting multiple Cloud Computing providers •  Even if the Cloud provider has multiple data centers in different geographic regions, it may have common software infrastructure and accounting systems, or the company may even go out of business

•  Data lock-in: •  Software stacks have improved interoperability among platforms, but

the APIs for Cloud Computing itself are still essentially proprietary •  Customers cannot easily extract their data and programs from one site

to run on another

Top 10 Obstacles for Cloud Computing •  Data confidentiality and auditability:

•  My sensitive corporate data will never be in the cloud •  Current Cloud offerings are essentially public (rather than private)

networks, exposing the system to more attacks •  There are also requirements for auditability and privacy laws (many

Nations have laws requiring SaaS providers to keep customer data and copyrighted material within national boundaries)

•  There are no fundamental obstacles to making a Cloud Computing environment as secure as the vast majority of in-house IT environments, well-understood technologies (e,g., encrypted storage, VPN, firewalls,…)

•  Data Transfer Bottlenecks: •  Applications continue to become more data-intensive, significant costs

in the Cloud •  Avoid Internet transfers by shipping disks •  Data Storage for free, CPU cycles sustain the business •  WAN and LAN bandwidth are still bottlenecks

Top 10 Obstacles for Cloud Computing •  Performance unpredictability:

•  Multiple Virtual Machines can share CPUs and main memory surprisingly well in Cloud Computing, but I/O sharing is more problematic

•  Improve architectures and operating systems to efficiently virtualize interrupts and I/O channels

•  Flash memory adoption will decrease I/O interference •  Scalable storage:

•  Short-term usage, no up-front cost, and infinite capacity on-demand is more difficult to achieve with persistent storage with respect to computation

•  Bugs in large-scale distributed systems: •  Removing errors in very large scale distributed systems is very

challenging •  A common occurrence is that bugs cannot be reproduced in smaller

configurations, so the debugging must occur at scale in the production data centers

Top 10 Obstacles for Cloud Computing •  Scaling quickly while maintaining QoS:

•  Pay-as-you-go certainly applies to storage and to network bandwidth, both of which count bytes used

•  Computation is slightly different, depending on the virtualization level: •  Google AppEngine automatically scales in response to load increases and decreases,

and users are charged by the cycles used •  Amazon EC2 charges by the hour for the number of instances you occupy, even if your

machine is idle •  Automatically scale quickly up and down in response to load is important in

order to save money, but without violating service level agreements •  Reputation fate sharing:

•  Reputations do not virtualize well. One customer’s bad behavior can affect the reputation of the cloud as a whole

•  For instance, blacklisting of Amazon EC2 IP addresses by spam-prevention services may limit which applications can be effectively hosted

•  Legal issues •  Software licensing:

•  Current software licenses commonly restrict the computers on which the software can run

Case studies

Case studies • Scaling computational power

•  Animoto • Scaling data

•  Instagram • A hybrid cloud

•  Zynga

Animoto •  http://animoto.com/ • A web application that produces videos from photos,

music and video clips • About 4 million users •  Launched in 2007 •  Facebook application launched in April 2008

•  Mid-April about 750,000 people signed up in three days

•  iPhone app

Animoto • Big need to scale according to workload • Operations connected through queues (SQS)

•  One queue to fetch images from URLS •  One queue with the list of render jobs •  Decouples various parts of the site •  Scaling is decided looking at queues states

• Photos, videos, music stored in S3 • Producing video takes 8-9 minutes in average

Animoto

•  The system is managed using the Rightscale solution

•  Up to 4000 EC2 instances used simultaneously

•  Focus on CPU scalability •  Take advantage on new GPU-

based instances •  Better quality videos

Instagram

•  Free photo sharing social networking

• Allows to apply filters to photo •  Launched in 2010 • Bought by Facebook in April 2012 • Offered as iPhone and Android

apps

Instagram • About 30+ million users in less than 2 years • After the release of the Android app 40+ mln users in 10

days •  Initially deployed on a single machine less powerful than a

MacBook Pro •  First day they had 25K signups

•  Server melted! • Solution:

•  Migrate to Amazon!

Instagram

2010 •  2 engineers

2011 •  3 engineers

2012 •  5 engineers

(2.5 on backend)

Workforce @ Instagram

This includes iPhone and Android app developers

Instagram • Ubuntu Linux on Amazon EC2 •  Load balancing

•  Initially 2 nginx machines and DNS round robin •  Then moved to Amazon Elastic Load Balancer

• Amazon Route53 for DNS • Application server written in Django on Amazon High-CPU

machines •  Up to 25 EC2 instances for application server (scalable) •  AS is stateless

• Photos in S3 storage

Instagram - Database 1.  Initial database PostgreSQL with Django ORM

1.  A dedicated machine for database 2.  Hit the 68GB RAM limit of biggest machine in EC2

2.  Vertical partitioning of DB 1.  One DB for each kind of data (users, photos, etc.) 2.  Hit the 68GB RAM limit of biggest machine again!

3.  Horizontal partitioning (sharding) 1.  Divide data by user ID (logical shards) 2.  Map logical shards to physical shards

Instagram - Database

• Need to store the follow graph • Use the redis key value store •  Initial version of table (source id, target id, status) •  Fast inserts, fast subsets • Scalability

•  When an instance hit the 40K req/sec start another instance •  DBs stream to backup slaves which are saved to EBS

snapshots •  20 minutes operation

Zynga •  Founded in 2007 • Develops games for browsers, phones and stand-alone • Most of the games are offered through Facebook • About 250 million active users per month

Zynga • Problem

•  Games are risky: don’t know how many people will play •  How to do capacity planning? •  How will be the adoption rate?

• Games are initially launched in Amazon •  Experiment with configurations

• After the games mature it is moved to the a private cloud •  Z Cloud •  Hybrid cloud

Zynga – Hybrid cloud •  Z Cloud

•  Based on CloudStack and Rightscale

• After that the app is moved to Z Cloud, Amazon is used only for bursting

• CTO: “we view the public cloud and private cloud as one system. We architect and manage it as one system.”

•  They weren’t affected by Amazon outage • Key for scalability: degradable services

•  Isolate components