Market-Oriented Cloud Computing

Srikumar Venugopal

Grid Computing and Distributed Systems (GRIDS) LaboratoryDept. of Computer Science and Software EngineeringThe University of Melbourne, AustraliaE: srikumar@csse.unimelb.edu.au

2

Agenda

Motivation for Market-based Computing A vision for Market-based Clouds Current research in GRIDS Lab

Aneka: A resource provider Brokering of resources and negotiation

Conclusion and Open Questions

3

Salient Features of Cloud Computing

Abstracted Infrastructure Using Resources without reference to their location

Fully Virtualized Servers are virtual instances

Dynamic Can add, delete new instances dynamically

Pay by Consumption No fixed long-term contracts

Configurable Any application or OS can be provided

Forrester Research, “Is Cloud Computing Ready For The Enterprise?”,March 2008

4

Yet..

Cloud SLAs are still in their infancy Limited options for higher Quality of Service

Flat Pricing Model Amazon

Cloud provisioning is not a core product Amazon is an e-commerce company

Google is a search company

Need innovative business models

A larger marketplace

5

The Gridbus Project @ Melbourne:Enable Leasing of ICT Services on Demand

WWG

Pushes Grid computing intomainstream computing

Gridbus

6

The Gridbus Project @ GRIDS Lab, The University of Melbourne:Toolkit for Creating and Deploying e-* Applications on Utility Grids

The Gridbus Project @ GRIDS Lab, The University of Melbourne:The Gridbus Project @ GRIDS Lab, The University of Melbourne:

Toolkit for Creating and DeployingToolkit for Creating and Deploying ee--** Applications on Utility GridsApplications on Utility Grids

Gridbus

Distributed Data

http://www.gridbus.org

• Gridbus is a “open source” Grid R&D

project with focus on Grid Economy, Utility

Grids and Service Oriented Computing.

• Gridbus Middleware components include:

– Aneka: .NET-based Enterprise Grid

– Grid Market Directory and Web Services

– Grid Bank: Accounting and TransactionManagement

– Visual Tools for Creation of DistributedApplications

– Grid Service Broker and Scheduling

– Workflow Management Engine

– Libra: SLA-based Resource Allocation

– GridSim Toolkit

7

8

Participants, Goals, Requirements

Consumers: - minimize expenses, meet QoS How do I express QoS requirements ? How do I trade between timeframe & cost ? How do I discover services and map jobs to meet my QoS needs? How do I manage Grid dynamics and get my work done? …

Providers:– maximise ROI and profit How do I decide service pricing models ? How do I specify them ? How do I translate them into resource allocations ? How do I enforce them ? How do I advertise & attract consumers ? How do I do accounting and handle payments? …

They need mechanisms, tools and technologies that help them in valueexpression, value translation, and value enforcement. Service Level Agreements (SLAs)

9

Market Mechanisms for Clouds: Avision

StorageCloud

ComputeCloud

StorageCloud

ComputeCloud

Directory

Bank

Auctioneer

MARKET

EnterpriseResourceManager(Proxy)

Broker1

BrokerN

RequestResources

Negotiate/Bid

..

..

CloudConsumer

10

Aneka: a resource provider for paralleland distributed applications

Message Handler / Dispatcher

Communication Layer

Container

Security

Pe

rsisten

ce

Allocation Manager

ThreadModel

TaskModel

DataflowModel

MPIModel

MapReduce

OtherModels

Applications

SLANegotiation

11

Advance Reservations

Commitment of a guaranteed share of aresource ahead of usage time Resources : Nodes, Bandwidth, Storage

Advantages: Lowers risk for user

Easier capacity planning for provider

Assured income

Applications : workflow, multimedia applications, etc.

Are a form of SLA

12

Aneka’s SLA-View for Resource Allocation

Master Node

User/Broker

Enterprise Grid

Reservation

Store

Task

Store

Node

SelectionPolicy

PricingPolicy

Membership

Store

Execution Node

TaskStore

ReservationStore

Time SlotSelection

Policy

ExecutionNode

ExecutionNode

ExecutionService

MembershipService

SchedulingService

Reservation Service

Allocation Service

Negotiation Protocol Engine

Ack: C.S. Yeo

13

Pricing of Reservations

Dynamic pricing based on utilization level

Where p is the unit price,

x is the static component (base price), and

y = load factor * z, is the dynamic component

a and b are the relative weights

b can be set higher when resource availability is low and viceversa

Serves as a method of admission control

takes advantage of market conditions

p ax by

C.S.Yeo, S. Venugopal, X. Chu, and R. Buyya, Autonomic Metered Pricing for a UtilityService, Technical Report, GRIDS-TR-2008-16, GRIDS Laboratory.

14

Revenue vs Strategy

15

Cloud Provider Architecture

Dispatcher

VMMonitor

Service RequestMonitor

Pricing Accounting

Service Request Examiner andAdmission Control

- Customer-driven Service Management- Computational Risk Management

- Autonomic Resource Management

Users/

Brokers

SLAResource

Allocator

Virtual

Machines(VMs)

PhysicalMachines

16

Gridbus Broker: Abstracting ResourceAccess

Aneka

Gateway

UnicoreData Store

Access Technology

Grid FTPSRB

-PBS-Condor-SGE

Globus

Job manager

fork() batch()

Gridbusagent

Data Catalog

-PBS-Condor-SGE-XGrid

SSH

fork()

batch()

Gridbusagent

Credential RepositoryMyProxy

Home Node/Portal

GridbusBroker

fork()

batch() -PBS-Condor-SGE

Po

rtle

ts

17

Broker-Provider Negotiation

NegotiationModule

NegotiationModule

SchedulerResource Allocation

Manager

Broker Provider

AdvanceReservation

Job Submissionand Monitoring

Broker acts as a user agent Broker translates user requirements to resource requirements However, the negotiation process is invisible to the end user.

18

The Negotiation Protocol

19

Effect of deadline urgency

S. Venugopal, X. Chu, and R. Buyya, “A Negotiation Mechanism for Advance ResourceReservation using the Alternate Offers Protocol”, IWQoS 2008.

20

MetaCDN: Brokering Cloud StorageProviders

Dr. James Broberg, University of Melbourne, http://www.metacdn.org

21

Open Questions

How to commoditise cloud services ?

What would be the structure of the Cloudservices market ?

What are the accounting and paymentmechanisms available ?

How to monitor and enforce the SLAs arrived atby negotiation ? Who arbitrates the process ?

Thank You

Questions?

http://www.gridbus.org