cloud computing and service management day 5 10-25-10 · 2010-11-11 · cloud computing and service...

© 2009 IBM Corporation

Cloud Computing for a Smarter Planet

Prof. Dr. Kristof KloecknerVP Strategy & Enterprise Initiatives, Systems & Software, and CTO, Cloud ComputingIBM

October 25, 2010

Cloud Computing and Service Management Platform Services

© 2009 IBM Corporation2

Infrastructure-as-a-Service

Platform-as-a-Service

Application-as-a-Service

Servers Networking Storage

Middleware

Collaboration

Financials

CRM/ERP/HR

Industry Applications

Data Center Fabric

Shared virtualized, dynamic provisioning

Database

Web 2.0 ApplicationRuntime

JavaRuntime

DevelopmentTooling

Four major categories of Cloud Computing services are emergingExamples

Business Process-as-a-Service

Employee Benefits Mgmt.

Industry-specific Processes

Procurement

Business Travel


Ser

vice

Del

iver

y

Por

tal

BSS – Business Support Services

OSS – Operational Support Services

Reporting & AnalyticsMetering

Ser

vice

Dev

elop

men

t P

orta

l

Common Cloud Management Platform

Security & Resiliency

Cloud Service

Offerings

Service Provider Portal

Service Business Manager Service Operations Manager

AP

I

Use

r Int

erfa

ce

AP

I

The Common Cloud Platform

Virtualized Infrastructure – Server, Storage, Network

“Common Cloud Middleware Platform - PaaS”

Unified service management driving delivery economics

•Emerging and existing programming models•Hybrid Environments

Loosely Coupled Workloads

Analytics Workloads

Storage/Data Integration Workloads

Transactional Workloads

PaaS Tooling – Integrated Developer and Administrator Tools,

BSS plugin - PaaS specifc user roles OSS plugin - Service Templates, Management Plans


References

§ Articles downloaded from the Internet– Michael Armbrust et al., Above the Clouds: A Berkeley View of Cloud Computing, Feb. 2009

http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf– Cloud Computing: Platform as a Service. InformationWeek Analytics, October 2, 2009– Luiz André Barroso and Urs Hölzle, The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, Synthesis

Lectures on Computer Architecture, 2009, http://www.morganclaypool.com/doi/pdf/10.2200/S00193ED1V01Y200905CAC006?cookieSet=1– Scott Crowder, Introduction to Workload Optimized Approach & Workload Market Segmentation, IBM White Paper, December 2009– David Chappell, A short introduction to Cloud, http://www.davidchappell.com/CloudPlatforms--Chappell.pdf– David Chappell, Cloud Platforms Today: A Perspective, April 2009 http://www.davidchappell.com/CloudPlatformsToday--APerspective--Chappell.pdf– Jeffrey Dean and Sanjay Ghemawat, MapReduce: Simplified Data Processing on Large Clusters,

labs.google.com/papers/mapreduce-osdi04.pdf– DeCandia et al. Dynamo: Amazon’s highly available key-value store, SOSP 2007,

http://portal.acm.org/citation.cfm?id=1294281&dl=ACM&coll=ACM&CFID=47859964&CFTOKEN=98797782– European Network and Information Security Agency (ENISA), Cloud Computing, Benefits, risks and recommendations for information security, Nov

2009 (http://www.enisa.europa.eu)– Gregor Hohpe, Programming the Cloud, November 2009, http://www.enterpriseintegrationpatterns.com/docs/HohpeProgrammingCloudKeynote.pdf– Anna Liu, Architecting Cloud Applications – the essential checklist, AAF Keynote 2009, – National Institute of Standards and Technology, Definition of Cloud Computing, http://csrc.nist.gov/groups/SNS/cloud-computing/– Ning Duan et al., Tenant Behavior Analysis in Software as a Service Environment, ICSOC 2009– Daniel Nurmi et al., The Eucalyptus Open-source Cloud-computing System, http://www.cca08.org/papers/Paper32-Daniel-Nurmi.pdf– Open Cloud Manifesto, http://www.opencloudmanifesto.org/– OpenNebula.org – Various papers– B. Rochwerger et al., The Reservoir Model and Architecture for Open Federated Cloud Computing, IBM Journal of Research and Development,

April 2009 http://www8.cs.umu.se/~elmroth/papers/ibmjrd2009.pdf– Werner Vogels, Eventually Consistent, ACM Queue, October 2008– Ying Huang et al., A Framework for Building a Low Cost, Scalable and Secured Platform for Web-Delivered Business Services, IBM Systems

Journal, November 2009§ Company Web Sites: Amazon, Microsoft, Google, IBM, Salesforce.com§ Gregor Hohpe, Bobby Woolf, Enterprise Integration Patterns, Addison-Wesley 2004§ Kristof Kloeckner, Middleware for Distributed Systems, Lecture Notes 2004§ Kristof Kloeckner, The IBM Cloud Agenda, White Paper 2009§ George Reese: Cloud Application Architectures, O’Reilly 2009§ John W. Rittinghouse, James F. Ransome, Cloud Computing. Implementation, Management and Security, CRC Press 2009§ Andrew Tanenbaum, Maarten van Steen: Distributed Systems. Principles and Paradigms, Prentice-Hall 2009 § Rich Schiesser: IT Systems Management, Prentice-Hall 2002§ Jim Rymarczyk, Virtualization, Pre-Print 2009§ Tivoli Service Automation Manager Solution Guide


Agenda

§ Evolving Programming Models – Overview

§ Extensions to traditional programming models – patterns in the cloud

§ Loosely coupled, relaxed consistency– Amazon Web Services– Microsoft Azure– Google

§ Content centric– Hadoop

§ Database centric– Pangoo


Spectrum of Commercial Cloud Platforms

Client VMs

SharedServices

Hypervisor Hypervisor

Amazon Azure GoogleSalesforce.com

Standard VM Images

SharedServices

Client VMs

SharedServices

ClientServices

Degrees of Freedom


Security

Critical Pieces of a Cloud Architecture

Programming Model

Persistence Model

Mgmt Model

Tools

Connectivity + Integration

Infrastructure

Collaboration

Traditional Developer?

Non-traditional ISV’s? IT/CIO? Integrators

We will ignore this part in the study


Five Emerging Cloud Architectures

§ Virtualized Traditional - Extensions of Java Application Servers, Support for ‘Traditional’ Transactional Workloads– Moving existing workloads to the cloud– Requires best practices, patterns, tooling

§ Database Centric - data driven + small computation on small data– With multi-tenancy attractive for enterprise and service providers

§ Content Centric - computation needs to be close to data + large computation on large data– Data Mining, Analytics, Data Warehouse,

§ Loosely Coupled - computation and data are separate– Can be addressed by existing middleware, but ‘relaxed consistency’ models

emerging

§ Storage Analytics - Data and Storage Integration


Agenda







Virtualized InfrastructureVirtualized InfrastructureVirtualized Infrastructure

Application Runtimes in Virtual Machines(Middleware Images)

Application Runtimes in Virtual MachinesApplication Runtimes in Virtual Machines(Middleware Images)(Middleware Images)

Shared User Services(Persistence. Connectivity,

Composition, …)for multiple cloud programming

models(loosely coupled, content centric,

database centric)

Shared User ServicesShared User Services(Persistence. Connectivity, (Persistence. Connectivity,

Composition, Composition, ……))for multiple cloud programming for multiple cloud programming

modelsmodels(loosely coupled, content centric, (loosely coupled, content centric,

database centric)database centric)

Pattern Deployed Services(optimized multi image configurations)

Pattern Deployed ServicesPattern Deployed Services(optimized multi image configurations)(optimized multi image configurations)

Life Cycle ManagementLife Cycle Life Cycle

ManagementManagementPlatform

Web Interface

Platform Platform Web Web

InterfaceInterfaceRepository

(Parts/Patterns) Repository Repository

(Parts/Patterns)(Parts/Patterns) Service

Management Extensions

Service Service Management Management ExtensionsExtensions

Building Cloud Services from Cloud Patterns


Straw Man Scenario – Existing Application

§ Do Workload Analysis – select application candidates (simple tool exists)– Workload migration workbench

§ Do Application Analysis – potentially refactor (no tools exist yet)§ Deploy/Run

– Find recommended deployment pattern, use tools to deploy to selected infrastructure• Initially combination of (optimized) VMs (DB2, WAS etc.)• Increasingly a service (Messaging, Persistence/DB, Application Container…)

– Maintain an application (patch, update, manage the image…)§ Secure through Service Platform

– Identity, Compliance, Isolation§ Exchange Data through Service Platform

– SaaS application and enterprise– Bring data to the cloud

§ Connect back to enterprise business processes through Service Platform


The Application Model

§The model defines– The application– Requirements on libraries, security, data, and

services– Connections to other applications and systems

Based on a touchpoint model– Inbound Routing Rules for access to the application

§Model elements can be private to a user or shared (publicly or within a group)

§Model technology based on research work currently used in Rational Software Architect and WebSphere CloudBurst Appliance

The ApplicationThe Application

Routing RulesRouting Rules

LibrariesLibraries DataDataSecuritySecurity

ConnectionsConnections

ServicesServices


The Deployment Model

§ The Deployment Model maps an application to a particular, non-functional, execution topology

§ A single application can be deployed using different deployment models

§ Simple models include “development”, “test”, and “production”

§ Deployment models allow specification of desired QoS characteristics (such as high availability model or clustering structure) without specifically saying how to configure the middleware


Topology Model

§ The topology model defines the actual configuration of the application and the middleware to be executed on the cloud

§ Defined as a collection of virtual machine images arranged in a topology and customized to current usage

§ Topology model defines the activation of the system AND the lifecycle management of the components

§ Topology model based on Zephyr research work and work done by Andrew Trossman in Tivoli


Example of a Cluster Pattern

EAR DBTWO (Primary)

DBTWO (Standby)EAR

WXS Catalog

WXS Catalog

WXS Catalog

haproxy

haproxy

EIP1

EIP2

WAS

WAS

EAR

WAS

(For Session replication)


DBaaS vs DB2 on VMs (or BM: Bare Metal)

§ Ultra-simplicity: Little/no exposure of VM/BM instances, etc

§ Maintenance is automated/hidden as much as possible

§ Application developer is enabled to provision their own DB

Unit of Provisioning: DBOS Image + OS Image +HADR

Unit of provisioning: VM Instance

vs.

• 2 Operating system• OS configuration• OS maintenance

• 2 DB2 installations• DB2 configuration• DB2 maintenance

• HA topology

• Backup strategy• Security patches

• DB2 and OS• Manual • Failure recovery• Usage based on

DBA skills

DBaaS (ultra simple) Traditional DB

WebSphere

Client App

etc


Database Service Architecture

§ DBaaS– Provision a database in the cloud– Simplify admin/setup (limited dbm cfg)

§ DB2 MMT (multitenancy)– Provision/share a database schema– Support multitenant single-instance apps

IBM Confidential

Master App Metadata Repository

w/ context

REST

JDBC

Database Resource PoolDatabase Resource Pool

DB1DB1 DB3 (T1)DB3 (T1) DB4DB4

MMT REST Services

Single/Multi Tenant Application(w/ accountID, tenantID)

Single/Multi Tenant Application(w/ accountID, tenantID)

DB2 Common Client Driver

Admin Console

App

Admin Console

App

Data Node Data Node

RXA / JDBC

DB3 (T2)DB3 (T2) DB5DB5

• Accounts, tenants, offerings• logical/physical dbs• tables

DB ProxyService


Agenda



§ Loosely coupled, relaxed consistency– Amazon Web Services

Amazon material, best practices from A. Trossman, IBM– Microsoft Azure– Google




A “Loosely” Coupled infrastructure allows for a wide variety of programming and persistence models

Mgmt Model

Control, and mgmt flow

Mgmt Model

App1 App2

App3 App4

Mgmt Model

Loosely coupled here

Over time, loosely coupled here also

InfrastructurePersistenceProgramming


Critical elements of a loosely coupled model

20

Applications Services accessed via REST/SOAP messages• Storage services

• Data services

• Queuing/messaging Services

• Execution Services (virtualized hardware)

Design to minimize operational costs - up front• e.g. recognize some part of the platform will fail (Storage, DB, application) &

design into application

• Don’t debug - kill/freeze execution instant

Eventual Consistency for Data Handling & Replication: - sometimes data storage service or database service will sometimes return the wrong answer

Message queue - will deliver messages at least once, possibly more than once

Asynchronous - scale achieved by recognizing components that perform operate in parallel• Session/state information stored outside the application components

Commodity “parts” can come and go, the rest of the system does not fail• Both for infrastructure parts, as well as for application parts

Redundant (idempotent) execution is fine for infrastructure working AND for application semantics• Without that, very strict guarantees on application state will be required,

making the cost of execution very high


The ‘new ACID’ (Gregor Hohpe)

§ Old ACID – predictive and accurate– Atomic– Consistent– Isolated– Durable

§ New ACID – flexible and redundant– Associative– Commutative– Idempotent– Distributed


Amazon - AWS Evolution

AWS continues to evolve iteratively as Amazon learns what works for their business ecosystem

LOOSELY COUPLED STYLE


amazonaws.com


amazonaws.com

• Free developer account • REST and SOAP Cloud APIs• WSDL files• Voluminous documentation• Sample code • Usage-based billing• $0.10 - $0.17 / GB bandwidth


amazonaws.com

Message Queue

Message Queue

Message Queue


amazonaws.com

Message Queue

Message Queue

Message Queue

SQS

• Amazon Simple Queue Service• Scalable message queue• Elastic capacity• $.01 for 10,000 messages


amazonaws.com

Message Queue

Message Queue

Message Queue

Object Storage

Object Storage

Object Storage


amazonaws.com

Message Queue

Message Queue

Message Queue

Object Storage

Object Storage

Object Storage

S3

• Amazon Simple Storage Service• Scalable object storage• Distributed, redundant• 1B – 5 GB / object• Public or private• URL-addressable• $.15 / GB / month


amazonaws.com

Message Queue

Message Queue

Message Queue

Object Storage

Object Storage

Object Storage

IndexedStorage


amazonaws.com

Message Queue

Message Queue

Message Queue

Object Storage

Object Storage

Object Storage

IndexedStorage

SDB

• Amazon SimpleDB• Indexed data storage• Distributed, redundant• Schemaless• Elastic capacity• Query language• $1.50 / GB / month


amazonaws.com

Message Queue

Message Queue

Message Queue

Object Storage

Object Storage

Object Storage

IndexedStorage

ElasticComputing


amazonaws.com

Message Queue

Message Queue

Message Queue

Object Storage

Object Storage

Object Storage

IndexedStorage

ElasticComputingEC2

• Amazon Elastic Compute Cloud• Scalable processing power• Small, medium, or large• Linux or OpenSolaris• Root-level access• Management API• $0.10 - $0.80 / instance / hour


amazonaws.com

Message Queue

Message Queue

Message Queue

Object Storage

Object Storage

Object Storage

IndexedStorage

ElasticComputing

BlockStorage


amazonaws.com

Message Queue

Message Queue

Message Queue

Object Storage

Object Storage

Object Storage

IndexedStorage

ElasticComputing

BlockStoragePersistent Block Storage

• 1 GB – 1 TB allocation• High-performance• Mount on EC2 instance• Format as filesystem• Snapshot backup to S3


amazonaws.com

Message Queue

Message Queue

Message Queue

Object Storage

Object Storage

Object Storage

IndexedStorage

ElasticComputing

BlockStorage

IP Addresses


amazonaws.com

Message Queue

Message Queue

Message Queue

Object Storage

Object Storage

Object Storage

IndexedStorage

ElasticComputing

BlockStorage

IP Addresses

Elastic IP Addresses

• Allocate as needed• Attach to EC2 instance• Remap as needed


amazonaws.com

Message Queue

Message Queue

Message Queue

Object Storage

Object Storage

Object Storage

IndexedStorage

ElasticComputing

BlockStorage

IP Addresses

Metering

Billing


amazonaws.com

Message Queue

Message Queue

Message Queue

Object Storage

Object Storage

Object Storage

IndexedStorage

ElasticComputing

BlockStorage

IP Addresses

Metering

Billing

Metering & Billing

• Dynamic usage tracking• Monthly credit card billing• Usage reports• Portal


S3 – Storage What is it?•Uses standards-based REST and SOAP interfaces designed to work with any Internet-development toolkit.

•S3 is built on a distributed architecture - data stored redundantly

•Each object is stored in a bucket & retrieved via a unique, developer-assigned key.

•A bucket can be located in the United States or in Europe. All objects within the bucket will be stored in the bucket’s location, but the objects can be accessed from anywhere.

What’s different about it•S3 will fail on read/writes as a component - but system remains reliable.

• Apps expected to be designed “loosely coupled” to take into account

•Not a filesystem. Objects are not files

•Not for transaction processing

•Data redundancy takes minutes - cannot be assure an object you created/updated in S3 will be immediately available to other S3 applications

•Developer frameworks that take into account S3 & EC2 working together in the innovation phase

AWS ServicesLOOSELY COUPLED STYLE


EC2 - virtual computing environmentWhat it is?•Provide “instances” - virtual machines/hardware that run in EC2; based on XenSource

•Images can be shared - or rented out to others (Paid AMI thru DevPay)

What’s different about it•Application instances & data are coupled - EC2 does not automatically save data outside it’s environment

•Instance rebooted - transient data not lost. Instance shutdown or fails -data lost

•Can recycle images to avoid runtime bugs/problems such as memoryleaks, race conditions, etc.. - and freeze images for off-line debugging.

•From the beginning a developer needs factor long term persistence into their application design when apps fail for whatever reason (S3 down, network connection down, etc..)

•Automated management of EC2 images in early phase. Most applications have rolled their own



SQS - Simple Queue ServiceWhat is it?• Access to SQS thru SOAP services

• Highly scalable, distributed, hosted queue to reduce/eliminate app-to-app dependencies

• All messages are stored redundantly across multiple servers and data centers

• Developers can create an unlimited number of Amazon SQS queues, each of which can send & receive an unlimited messages.

• Message body can contain up to 8 KB of text in any format.

• A message is “locked” while a computer is processing it, keeping other computers from trying to process it simultaneously. If processing fails, the lock will expire and the message will again be available.

What’s different about it?• It’s more than a simple queue - applications interact by telling SQS estimated processing time = workflow

• Message may not be delivered immediately

• Load balancing model is asynchronous - lots of instances could be taking work off the queue, in different data centers

• Asynchronous - state/session information store in SQS where possible

• Messages will end up being delivered more than once in some cases - application to deal with it.

• Workloads, number of messages on the queue for an application - is done mathematically on sampled queues• Pricing still a drawback to broader adoption



Best Practices (Andrew Trossman)

§ Image management– Launch parameters– S3, CVS, SVN– Image Style Management

§Release upgrades§Cluster everything (redundancy)§Dynamically respond

– Faults– Demand

§Processing Pipeline of Loosely Coupled Services§Conclusions


Image Management

§Changes makes 100% images impractical§Boot Scripts combined with Homogenous Environment work§Image + Launch Parameters ~= Image

–Extremely repeatable and reliable–Less storage –Tolerates change better§Example template

–Builds server from script–Pulls content/code from repository


Image Style Management

§Avoid Heisenbugs – cycle VMs regularly§Simple patches update “image”

–Automatically rolled out via regular cycling§Never “fix” by hand§Always “replace” the image


Release Upgrades

§Completely rebuild parallel environment– Test– Cut over data– Change DNS– Decommission old when confident

§Cheaper to “replace than fix”§Traditional “fix” process with staging etc.

– GBS identified 2/3 human effort dedicated to this process


Cluster Everything

§Everything Fails – Applications must accommodate§Transparent redundancy§Seamless failover§Monitoring & Events


Scalr Dynamic Response to Demand & Availabiltiy


Always Respond By Cloning

§Resist urge to “fix” in place§Most bugs are application bugs§Traditional QA is good at removing all but the Hysenbugs§Clone instance brings a “fresh” server to replace the faulty

one.– This gets past hysenbugs– Enables “off-line” problem determination

§ “Roll Forward” in the cloud


Scalr Process Flow


Pipeline Loosely Coupled Services

S3End users submit videos to be transcoded to the website

Request message is placed in the Amazon SQS incoming queue with a pointer to the video and to the target video format in the message

SQS

EC2

The transcoding engine, running on a set of Amazon EC2 instances, reads the request message from the incoming queue

1 2

34The engine retrieves, transcodes, and returns the video to S3

5a

SimpleDB

Metadata about the video (e.g., format, date created and length) can be indexed into Amazon SimpleDB for easy query

A Simplified Example: Video Transcoding Web Site

Sources: Amazon.com, MI Analysis

Client assumed to be:

Web Application Layer

5b

Response message is placed in the outgoing queue and sent to user with a pointer to the converted video


ServiceOrientedPlatform ofAmazon’sArchitecture

http://s3.amazonaws.com/AllThingsDistributed/sosp/amazon-dynamo-sosp2007.pdf


Examples


Frontend servers (x 3)- Medium instance (IO/Memory)- App & Cache servers

MySQL servers (x 6)- Medium instance (IO/Memory)- MySQL 5.1 w/ replication- Backup to S3 every 4 hours

Index servers (x 2)- X-Large (CPU/IO)- EBS volumes for IO throughput- EBS snapshots for backup

Infrastructure servers (x 3)- Dist. Logger (Medium – IO)- Analytics Server (Medium – IO)- Messaging Server (Small)

Crawlers (x ~70)- Small instance (Network IO)- Automated build & boot

Staging (x 3)- Medium / Small instances- Scratch space for internal use


Soocial


Observations from startups on 6 AWS (12 – 100s of AMIs)

§ Everyone deployed monitoring § All but one used open source monitoring (the other used home grown)§ NONE have humans watching/waiting§ All use image & boot script for repeatable deployments§ All have scripted fault prevention / resolution§ All Throw Away, rather than Fix§ All redeploy entire production for release upgrades


Scaling a Single Application

Single System

Tiered System

Clustered Middleware, Tiered System

Loosely Coupled

Services

Dynamic Massively Parallel Application

Vertical Scaling

Vertical Scaling

Partitioned DB

Vertical Scaling

Vertical Scaling

Vertical

Horizontal

Horizontal

Horizontal

Horizontal Scaling

Horizontal Scaling

Development Discontinuity

(new application architecture)

Significant Development Required


Conclusions

§Divide Complex Monolith– Two simpler problems

§ IaaS simplifies self-managed apps§Cost of IaaS + Apps < Monolithic App§PaaS _is_ an Application§Storage _is_ an Application§Using Anant’s Analogy:

– We have lots of small problems (apps)– We have one big problem (IaaS)


Microsoft Azure™Microsoft Azure™

Azure™ Services PlatformAzure™ Services Platform

Windows’ AzureWindows’ Azure

Live Services

Live Services

.NET Services

.NET Services

SQL Services

SQL Services

SharePoint Services

SharePoint Services

Dynamics CRM OnlineDynamics

CRM Online

Windows Live

Windows Live

OfficeLive

OfficeLive

ExchangeOnline

ExchangeOnline

SharePointOnline

SharePointOnline

Dynamics CRM

Online

Dynamics CRM

Online

Operating Environment

(IaaS)

Operating Environment

(IaaS)

PaaSPaaS

SaaSSaaS

Your ApplicationsYour Applications

Similar design points as AWS...Applications Services accessed via REST/SOAP

messagesSQL Services for data & storageAzure OS has messaging serviceAzure OS platform for app deployment

Data & storage - eventual consistencyQueued messages may be delivered more than once

...with key differences•Applications deployed - not Images

• VMs baked into OS

•Application provides declarative description for scalability, reliability & availability of application components

• e.g. developer of service owner specifies how pieces are to be distributed under what circumstances

•System automatically replicates code & data• Queuing/messaging Services

•SQL Services• Like Amazon, expecting it to be priced (high) based on

operation costs.

Similar design points as AWS...Applications Services accessed via REST/SOAP

messagesSQL Services for data & storageAzure OS has messaging serviceAzure OS platform for app deployment

Data & storage - eventual consistencyQueued messages may be delivered more than once

...with key differences•Applications deployed - not Images

• VMs baked into OS

•Application provides declarative description for scalability, reliability & availability of application components

• e.g. developer of service owner specifies how pieces are to be distributed under what circumstances

•System automatically replicates code & data• Queuing/messaging Services

•SQL Services• Like Amazon, expecting it to be priced (high) based on

operation costs.


Azure Cloud Development


Google Software Stack: One View

Google File System• Non-virtualized storage component – specialized distributed file system designed for Google workloads• Two types of servers: masters (network coordinators) & workers (operating on data as requested)• Chunk size is 64 MB – not typical file system block size to reduce workers interacting with master

Bigtable• Distributed column oriented data store but not a relational DB on top of GFS (Covered in Google TT last year)

Work Queue• Distributed batch processing component & job scheduler

Map Reduce – details• Framework/library in C++ component• Utilizes Work Queue to distribute computations to clusters• ~10,000 Map Reduce programs today• In 2004 ran 29,000 jobs – 2007, 2,200,000 jobs• Google runs ~100,000 jobs per day crunching thru 20 petabytes• Runs across ~100,000 node servers• Indexing , AdWords, Analytics, etc..

Sawzall• Query language, type-safe scripting langauge• Factor of 10 simpler to code up (and shorter) then in C++

CONTENT-CENTRIC CLOUD


Agenda







‘Content-Centric’ is really about “Big Data” AND “New Analytics”

Text

Logs &Transactions

Clickstream Data

Statistical Model Building

Text Analytics

Biological Sequences


A “Content-Centric” model runs infrastructure, data and computation all on the same nodes

Mgmt Model

Mgmt Model

Mgmt Model


Loosely coupled starts here

Real innovation occurs here


Critical elements of a content centric model

“Restricted” programming model• Think Batch: Redux

• Enables parallelized, distributed, fault tolerant computations without programming complexity

• No new programming experience required; framework hides details of parallelization, fault tolerance, load balancing, etc. from developer

• Offers simplicity of deployment & scalability - no application knowledge of runtime or OS or cloud necessary

Can be deployed on native hardware or virtualized• Underlying map/reduce runtimes automatically parallelizes the

computation across large-scale clusters of (virtual) machines

Storage & data - Leverages “hybrid” distributed storage system & file systems designed to handle petabytes of data - i.e not to be confused with an OS file system• Data Handling & Replication: map/reduce implementations thru a

software framework that handles data distribution

Designed to minimize operational costs• The “master” pings every worker periodically. If no response in a

certain amount of time, the master marks the worker as failed. handles machine failures, and schedules inter-machine communication to make efficient use of the network and disks



Apache Project: Hadoop Core

Open source project to recreate Google’s capabilities (lead by Yahoo) with improvements•Portable – can run as a native or virtualized system•Additional pluggable runtime components for crawling (structured & unstructured data), query languages (Pig Latin, JAQL, Hive, etc..)

Provides a Java framework for large scale parallel processing map/reduce apps•Offers simplicity of “programming” - Looks like a simple single threaded app model for developers •Today - setting up, coding Hadoop jobs in Java, etc. is the domain of skilled Java engineers

Awareness & Adoption Growing•Could become foundation of new generation of easily customizable web analytic applications – at web scale•Yahoo – used in production for indexing content•Facebook – analyze logs, analytics•New Your Times


Not as scalable as Google – but does it need to be?


Hadoop, an open source implementation of map-reduce

Map-reduce runtime• Partitions input data• Schedules program’s execution across set of

machines• Manages inter-machine communication• And more

Programming using Map-reduce:• Users specify a map function that processes a

key/value pair to generate a set of intermediatekey/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key.

• Processes and generates large data sets• Automates program recovery in case of a

failure• Supports functional style programming• Parallelism is an inherent feature• Critical to keeping the company's costs down

© 2009 IBM Corporation78 78

Large Financial Institution wanting to do fraud analytics§ A platform that can cost effectively manage PB’s of data and support fraud and marketing

analytics§ Must be efficient for structured data§ Integration with enterprise assets: warehouse, BI tools

New Analytics

Platform

Models of normal and fraudulent card usage

Transactional Credit Card Risk Management System

(Decision to authorize charge)

Transactional System

Analytics

§ Requirement: analyze 7 years – total 250TB at a rate of 100M transactions a day (transaction rate expected to grow substantially)

§ Problem 1 (1 year of data):– Today, w/o MSA, takes > 1 week – With MSA – 3 hr!

§ Problem 2 (1 month of data):– Customer goal: 1 day = “a win”; 10

minutes = “great”; 1 minute = “awesome”MSA at “great” (~10 mins), moving to “awesome”

Data Sizes and Performance


Agenda





§ Database centric– Pangoo– Salesforce.com


A “Database-Centric” model runs infrastructure and database on the same nodes

Mgmt Model

Mgmt Model

Mgmt Model


Real innovation at this layer


Critical elements of a database centric model

• The database layer needs to multiplex multiple applications• Database model needs to be flexible if different apps share the database

• For cloud economics to work out, mgmt cost of database layer << #app x mgmt cost of a single database for an app

• Programming model• A focus on schema configuration as opposed to schema design

• Constrain enough to keep cloud economics yet not reduce the market significantly

• Higher bandwidth within a “group of nodes”• For scaling the database within an app (could use larger SMP’s)

• Database nodes are the “keystone”, they need “HA” in some form (so the previous two architectures are not exactly the right fit)


From Single-Tenant to Multi-Tenant Application

MMT common service provides:§ Support for cost-effective resource sharing, isolation, diverse

SLAs, etc., across different tenants§Management of database resource pool, lifecycle of

applications & tenant subscriptions, monitor, analyze, and optimize system operations

§Highly on-demand availability and scalability with the number of tenants & offerings

§Minimize application development or transformation effort for SaaS ISVs

§MT data access mockup package for local testingMMT Meta Repository

MMT Common Service

Operator

1 5…

Database Resource Pool

10,000

App

1

App

2

App …

Few shards in MT

user1 user100… user1 user100… user1 user100…

user1,1 user10000,100…

App1 10

…


Database Multi-Tenancy for the Cloud

Tenant A

Tenant B

App Server

Shared Tables

(economic)

Separate Instances/Databases

(deluxe/advanced)

Separate Tables

(intermediate)

Tenant A

Tenant B

Multi-tenant App

App Server

Multi-tenant App

Hig

her Q

uery

Opt

imiz

atio

n/ru

ntim

e C

ompl

exity

,

Hig

her S

ecur

ity W

orrie

s

Multi-tenant App

App Server

Higher Multitenancy, better resource utilization


Multi-tenancy Challenges

Isolation, Scalability, Performance, Customization, Resource Utilization,

Metering …

Virtual Multi-Tenant LayerVirtual Multi-Tenant LayerVirtual Multi-Tenant Layer

DB Multi-Tenant Layer


MT DB Tradeoffs

Isolated Databases Separate Schemas Shared Tables

Simplicity simple simple (but need mechanism to avoid name collisions (3-part name or mapping))

hard

Customizability(schema)

high high low (might require migration)

Rigorous Isolation (regulatory law)

best moderate lowest

Resource Cost/tenant high low lowest

#Tenants Low large Largest

Operational Cost/tenant (backup, patches, etc.)

high low (but point in time recovery not easily possible)

Lowest (but point in time recovery even harder)

Tools Need tools to deal w/ large number of instances/databases

Need tools to deal w/ large number of tables

n/a

DB implementation cost Lowest (qry routing and simple mapping layer)

Low (qry routing, simple mapping layer and qry mapping)

High (qry routing, simple mapping layer, qry mapping, row-level isolation)

Scalability Per tenant Need some data/load balancing w/ dynamic migration

Need some data/load balancing w/ dynamic migration

Query Optimization Less critical Less critical Critical (wrong plan over very large tables is disastrous)

Per Tenant Query Performance

As usual need qry governance Need qry governance and tenant-specific statistics


Get tenant id via Tenant Identity propagation (ThreadLocal).Retrieve tenant profile (database, username, password, etc.)

according to tenant id.Connect to underlying database based on tenant profile– If shared tables, set tenant id in connection; pass down the

sql to target db. – If separate tables, get tenant specific schema name (assigned during tenant onboard) from tenant profile, and

set current schema before each statement is created.– If separate db, pass down the sql to target db.

MMT Metadata Repository

Tenant info;Offering info;

Physical DB info; Catalog info;SLA…..etc…

Dynamic Routing

MMT JDBC Wrapper

Get tenant id

SaaS Application

REST Service MMT Master App

REST Client

Tenant DB

2

3

45

REST requestw/ tenant id

REST responsew/ tenant profile:

DB info, SLA

JDBC connectionw/ tenant id

6

Result set

JDBC

1

Only once

DB2MMT

Non-db2mmt

Request db connection

Cache

DB2 JDBC Driver

Tenant Identity propagation


Bringing an Application to MMT for DB2

MT App (Offering) development/transform

Operation Management

Runtime

Service ProviderMMT Admin Console

ISV

TenantUsers

Monitoring, Governance,

…..

MMT Sandbox

Multi-tenant application

IDE

Tenant management

Offering management

Resource management

MMT Meta Repository

MMT Common Service

Operator

1 5…

Database Resource Pool

Shards in MT

Multi-tenant App


ISV App

DB2

Application

MT Meta Repository

MMT Common Service

DB2

ISV Local Env. DB2 MMT Runtime Env.

On-boarding

Simulated Meta File

MT Database Pool

Operator

DB2 DB2

MMT Runtime Agent

MT Application development/transformation

Provide offering metadata file

(XML) of application

Configure/Modify the application to use DB2 MMT access package

Embed tenant identification

Develop & Transformation Local & Runtime Environment

Supported J2EE environments– JDBC, Spring, iBatis/Hibernate, JPA– WAS/Tomcat, DB2

MMT Local Sandbox

Example of offering transformation1. Embed tenant identification in application

– Modify Web.xml to include the Filter servletTenantID for propagation through thread local

2. Configure the application to use MMT data access package– Modify Spring data source config to use MMT data source3. Provide offering metadata file (XML) of application– Data source info, DDL, shared tables info, config info, …


Operation Management (MMT Admin Console)

2. Tenant on-boarding/subscription

3. Offering Upgrade1. Offering onboarding

4. Offering & tenant topological view


Architecture of MMT for DB2

MMT Master App(WAS Cluster for HA & LB)

MMT Metadata Repository

JDBC w/ tenant context

REST w/ tenant context

REST

JDBC

Database Resource

Pool

Database Resource

PoolT 1T 1 T 3T 3 T 4T 4

MMT REST Services

A J2EE SaaS ApplicationA J2EE SaaS Application

MMT JDBC WrapperMMT JDBC Wrapper

DB2 JDBC Driver MMT Admin Console AppMMT Admin Console App

T 2T 2 T 5T 5T 1T 1 T 3T 3 T 4T 4

Tenant Data Node

T 2T 2 T 5T 5

Tenant Data Node

RXA / JDBC


KingDee’s Exploitationof Pangoo

Multi-tenant Metadata

Repository

MT Runtime Data Access Service

(Runtime Resource Sharing/Isolation, Dynamic Routing, SLA tracking …)

MT Operational & Management Service

(HA, Scalability, SLA tracking, Optimization, OLC etc.)

RDB Model Adapter Object Model Adapter

Tenant Context

MT-JDBC DriverSQL

REST/SOAPObject Query (LinQ,

SOQL, GQL etc.)

JDBC SDO Hibernate Agent

Data Object

Data Model Mapping Module

High Available & Scalable Data

Resources Pool

Application

Virtual Data Store

static schema

dynamic schema

DB-CENTRIC CLOUD


While Salesforce started with CRM, it and its partners run 1000’s of other transactional apps on force.com

4-way Oracle RAC

Multitenant Optimization Layer

CRM



4-way Oracle RAC 4-way Oracle RAC

CRM HR Travel HR Mktg

~TB of managed DB

~40,000 tenants~400,000 custom objects

Total 10 Pods

Pod1 Pod2 Pod3

Take 20 Standard Objects (Accounts, Orders, …)Customize or Create new ones

Mileage Object

Add Workflow or Business Logic

Get App

Service Multiple Tenants

DB-CENTRIC CLOUD


A Critical Innovation is the Multi-Tenant Database Architecture

Organization_id Key_prefix Id Name,(Others)

Val0 Val1 … ValN

org1 a01 a01…1

org1 a01 a01…2

org1 a02 a02…1

org1 a02 a02…2

org2 a01 a01…3

org2 a01 a01…4

org2 a02 a02…3

…

…

Custom Objects are forced into a very limited number of Oracle Tables

•Key_prefix subsetting●Still partitioning by organization_id

•Smart primary keys (key prefix)●Re-use across organizations

•GUID primary keys•ValN flex fields

Opex at database and platform level dominated by #objects [backups, stats, tuning, schema evolution, app design] for most databases. SFDC reduces this by forcing all disparate objects into fixed set of tables (as rows) -- trading off opex for platform development costs. Consequently, it is able to store ~400,000 different objects in a couple of dozen tables

DB-CENTRIC CLOUD


Support of Application Types through different PlatformsPlatform Enterprise apps

moved to the cloudScalable Web Applications (loosely coupled)

Content Centric Applications(parallelizable)

Data-base Centric Apps (multi-tenant)

IBM Cloud Service Platform

Sweet SpotImages, Patterns, Life cycle Mgmt

Virtuoso (future),Some support through existing MW (WSX, WVE)

Open Hadoop, with IBM extensions (service asset)

Pangoo (in deployment in China)

Microsoft Azure .Net Azure platform services

No Support Multi-tenancy supported in Azure SQL Services

Amazon Web Services

Images supported by partners

SQS, SimpleDB, Elastic MapReduce No support

Google AppEngine No support Sweet Spot ? BigTable, GFS etc. No support

Force.com No support No Support No Support Sweet Spot


The next question is, “which workloads on which architecture?”

Programming Model

Persistence Model

Mgmt Model

Infrastructure

Loosely Coupled

W1 (LC)… Wn(LC)

Programming Model

Persistence Model

Mgmt Model

Infrastructure

Content-Centric

W1 (CC)… Wm(CC)

Programming Model

Persistence Model

Mgmt Model

Infrastructure

Database-Centric

W1 (DC)… Wk(DC)

Before we answer that question, we need to answer: “Which workloads will move to the cloud?”


Impact of CloudOn Consumer

(Pain)

In order to understand workloads, we need to look at the pain vs. gain equation for the consumer of Public Cloud Services

What gain does the consumer get from the provider?

What pain does the consumer incur to utilize those services?

Lower costs due to standardization and better utilization

Business Resiliency

“Pay as you go”benefits for bursty use

Better Data Security

Ecosystem Availability and Network Effects

Conformance to Standardization

Integration Complexity

Latency and Data Transfer

Data and Business Policies

For a given workload, if Gain > Pain (by some factor), then it makes sense to utilize cloud for its deliveryProvider can be “internal IT”

We assume that the workload ends up on the right architecture, otherwise the pain will be too high

Benefits of CloudDelivered By Provider

(Gain)


External Cloud Usage Patterns Will Depend on Relative Level of Gain vs. Pain of a Cloud Delivered Service

Lower Gain From Clouds

Higher Gain From Clouds

Small Scale [incl. Start-ups], Ephemeral or High Demand FluctuationsVariable Model Less Expensive Even With Premium; Cost Allows Basic Function Accessibility to SME

High Amount of Data Transfer RequiredNetwork BW charges or Latency wipe out benefits for external cloud Large Enterprise with Long-Running, Stable Demand

Lower Cost Possible With Optimized Traditional Model

Lower Pain of Cloud Delivery

Higher Pain of Cloud Delivery

Self-Contained Applications/ServicesLittle or No Integration Needed; Easy to Schedule

Legacy or Highly Complex Services;Heterogeneous Services

Not Service Oriented or High Benefit to specialized infrastructure for components of solution

Workloads Amenable to Standardization, Multiplexing and Conforms to Cloud Architecture

Significant Application Redesign to Conform to Cloud Architecture

Moderately Complex Servicese.g. Application Development & Test

Additional Wild Card:Level of Control Needed/Fear of Lock-In

Security & Business Resiliency Tradeoffs

Ecosystem Availability & Network Effects for Workload


Lower Gain From Clouds

Higher Gain From Clouds

Lower Pain of Cloud Delivery

Higher Pain of Cloud Delivery

For an Internal Cloud, Similar Approach Can Be UsedEliminates Most Security and Data Transfer BarriersCan Lower Service Integration Complexity

Small Scale, Ephemeral or High Demand Fluctuations

Large Enterprise with Long-Running, Stable DemandLower Cost Possible With Optimized Traditional Model

Self-Contained Applications/ServicesLittle or No Integration Needed; Easy to Schedule

Legacy or Highly Complex Services;Heterogeneous Services

Not Service Oriented or High Benefit to specialized infrastructure for components of solution

Workloads Amenable to Multiplexing & Standardized Infrastructure

Significant Application Redesign to Conform to Cloud Architecture

Moderately Complex Servicese.g. Application Development & Test


Cloud ServiceDeveloper

Cloud Service Provider

Common Cloud Management Platform

Virtualized Infrastructure – Server, Storage, Network, Facilities

Cloud ServiceConsumer

Partner Clouds

Customer In-house IT

Consumer Administrator

Consumer Business Manager

Developer

Service Business Manager Service Operations Manager

Cloud Services

User Interface

Consumer End user

API

Software-as-a-Service

Platform-as-a-Service

Infrastructure-as-a-Service

Business-Process-as-a-Service

Metering, Analytics & Reporting

Service Development

Tools

Service Definition Tools

Image Creation Tools

Change & ConfigurationManagement

Offering Mgmt

Order Mgmt

Accounting & Billing

Customer Mgmt

Entitlements

Contract Mgmt SLAReporting

Pricing & Rating

Peering & Settlement

Subscriber Mgmt

Service OfferingCatalog

Invoicing

Service Automation Management

Virtualization Mgmt

Provisioning

Monitoring &Event Management IT Asset & License Management

Service Request Management

IT Service Level Management

Image Lifecycle Management

Capacity &Performance Management

Incident & ProblemManagement

BSSBusinessSupportServices

API

Service Delivery Portal

OSSOperational

SupportServices

Service Transition Manager

Service Security Manager Security & Resiliency

Service Delivery Catalog

Service Templates

Service Developm

ent Portal

Service Provider Portal

CCMP Reference Architecture serves as a base


Business Support System (BSS)

§ Services:1. Offering Management & Service

Offering Catalog2. Customer & Subscriber Management3. Contract Management4. Entitlements5. Order Management6. Pricing & Rating7. Accounting, Billing & Invoicing8. Peering & Settlement9. Analytics & Reporting

§ Processes:

§ Business Support Systems (BSS) are the components that a ServiceProvider uses to run its business operations towards customer


CCMP R1.0 and R1.1 BSS Functionality§ Sales

– Face to face using ePricer/eConfig tools

§ Customer Management– Bulk import of customer onboarding information by

Business Office– UI for user management with various roles– Web Identity support

§ Subscriber Management– Map customer admin and users to a contract

§ Offering Management– Bulk upload of Catalog data with list price and cost

information§ Service Offering Catalog

– UI for display of catalog items details like Images, VM Sizes, 32/64Bit, Block Storage, Reserved IP Address, VLAN

– UI for submitting provisioning request for a VM on a public or private network with appropriate IP address and attaching a storage

§ Contract Entitlements– Service Catalog entitlement information by

customer and contract loaded by the Business Office

§ Reporting and Analytics– Display of usage via BIRT reports– Royalty Reports for Redhat and SuSe

§ Contract Pricing and Rating– Pricing information by customer and contract

loaded by the Business Office– Simple ETL based price x quantity based pricing

model§ Billing

– Usage based by the hour, monthly recurring and one time charge

– Flexible billing calendar (monthly, quarterly & yearly) for a Geo

– Billing adjustments, incidental charges– Generating CFT/S spread-sheet feed file– “Green Dollar” Revenue back to SWG Products

§ Metering– Rollup of VM, IP addresses, storage blocks usage

information via Data Stage§ Costing

– Usage based costing using offering wide (non-contract) cost rate

– Generating CIF/SSC spread-sheet feed file§ API

– APIs for Image, Instance and Key Management


Pricing Models§ One Time Setup Charges

– Setup– Enterprise Onboarding

§ Monthly Recurring Charges– Rate Buy Down– VPN/VLAN

§ Per Hour Usage-based Charges– Virtual Machines

Images (software stack)OSStandardized (BR, SL, GD, PT, 32, 64) Compute

– IP Address Reservation– Standardized (SM, MD, LG) Persistent Storage


Publ

ic A

PI (R

est &

SO

AP)

REST& SOAP

Web Browse

rJavaScript

& CSS

CustomerAdmin

CustomerUser

ImageProvider

Developer

EclipsePlug-in

Reporting(BIRT)

Data Warehouse(DB2)

Data Acquisition(DataStage)

Web IdentityLDAP

TAM

Web Seal

AAA

Order to Cash

Billing (CFT/S)

Costing (SSC)

CSV Files

Billing

Cost

Rylty

Invoicing (Geos, IOL)

Financials (CLS, CARS)

OfferingManager

Create Customer Users& set resource limits

Request & use VM, Storage, IP Address

Upload Catalog & List Prices

Onboard Customers, Billing, Adjustments

Enterprise User Mgmt

BSS Extensions

Resource Mapping

Audit & Compliance

OSS Adapter

CloudUI

CloudBSS

ECWDB

BSS Detailed Component Diagram

Pricing & Rating

Abs

trac

tion

Laye

r

Port

al

Image Meta-data & Scripts

Rational Asset Manager RAM

Event Messaging

Subscriber ManagementREST

Service Offering Catalog

WDP BSS

EntitlementsWDPBSS

BSS forDev Test

BusinessOffice

Create Images


Layered Architecture


Operational Model


Backup Slides


Public Cloud Providers Have Different Architectures Driven By Legacy and Target WorkloadsAmazon Model Has Lowest Barrier to Migrating Existing Code

Amazon Google / Hadoop Salesforce

PaaS“Middleware as a Service”

Amazon Web Services (AWS)‣ Internally Amazon runs a SOA like

model‣ Exposes the key middleware

services which Amazon uses to drive its core business apps

App Engine‣Exposes the key middleware services which Google uses for its search application‣Hadoop is an open source implementation of Google’s key middleware services

Force.com‣ Optimized for database front end style

applications‣ Large number of business applications

fit this model (iSeries and LAMP stack are optimized for the same type of application)

‣ Requires the use of their proprietary scripting language

‣ Fault tolerant system Allows them to take additional costs out by using cheaper less reliable hardware and software components

IaaS“Hardware as a Service”

§ Ec2: Renting Intel MIPS§ S3: Renting Storage

Architecture Design Born From Internal e-commerce site Internal Search Engine Hosted Business Applications

From hereon called Loosely-Coupled Content-Centric DB-centric

Leverage Existing Code

and Skills

Build New Code and

Skills


Hybrid Cloud Management, Security and Integration

§ From the Enterprise Client’s perspective:

§ Seamless integration of enterprise management with workload running off-premise on clouds

– Visibility of software applications and services (monitoring, events, availability, performance)

– Control of identity, data security, governance, and compliance

– Automation of service definitions, policy based workload offloading, P2C/V2C cloud conversion, elastic scaling of CCMP, availability and disaster recovery SLAs

§ Security for Hybrids– Control security and resilience of services (identity

management, compliance, isolation)§ Enterprise to Cloud Integration

– Secure and efficient data exchange across the enterprise and clouds

– Secure business application connectivity and governance

§ Application and Workload migration– Tools to support the migration of workloads to cloud

Enterprise Resources

Public Cloud

Touchpoint

Pipe

Transformation

Private Cloud

Federated Virtual Service Domains

Business Applications & Information

Enterprise Management of Cloud software, applications, workload

Secure Pipe

Off-premise shared

services

Private shared services


Hybrid Cloud Management, Security & Connectivity in a Picture

On-premise business applications & information

Enterprise Infrastructure & Private Cloud

Cloud Integrator: Secure Connector, Business

Application Integration, Information Brokering,

Monitoring & Management, Security Federation

Public Cloud [SaaS, IBM Cloud, other Public Cloud]

Off-premise shared services

Off-premise business applications & information

Governance

Management

Integration

SecurityPrivate shared services


Application Life Cycle in a Cloud Enterprise apps moved to the cloud

Scalable Web Applications (loosely coupled)

Content Centric Applications(parallelizable)

Data-base Centric Apps (multi-tenant)

Model/Build Rational Tools(RSA, BuildForge)

sMash and AppBuilder

IBM Hadoop Pangoo Tools, SaaS Maker

Deploy WCA /Image Dispenser plus RAFW

Virtuoso IBM Hadoop Pangoo Onboarding Tool?

Run Optimized WAS and DB2 Services

Virtuoso IBM Hadoop Pangoo

Connect (with Enterprise)

SilverLining, Secure Information Broker..

SilverLining Secure Information Broker?

Secure Information Broker?

Manage/Secure Monitoring Service, apps migration wb, TFIM, isolation, backup/recovery

Monitoring Service, identity, backup/recovery

Monitoring Service, identity

Monitoring Service, compliance, identity, backup/recovery, compliance,


PaaS Competitive ViewCompetitor Generalized from…

and platform styleWhat does the platform offer for the apps?

IBM’s Response

Google App Engine Search -- content-centric

Scalable Analytics and Storage

Leverage open source hadoop

Amazon Web Services Shopping Cart, Spare computation -- loosely coupled

Cheap MIPS, flexible and simple storage, queues, database

Compute Cloud and scalable full functionmiddleware services,Application life cycle, image life cycle

Microsoft Azure MSN and countless ISV .net applications

VisualStudio Development for standard and new apps

Enterprise class and Java apps, Rational application life cycle tools

Force.com Salesforce.com multi-tenant CRM app --database centric

Simple Multi-tenant transactional app building environment

Leverage China deployment of Pangoo

Note: All the strengths mentioned in the previous style hold across all rows


Straw Man Scenario – New Application

§ Choose intended architecture (loosely coupled etc…)§ Use integrated Platform tool to model/build – need to pull existing tools into a suite§ Deploy/Run

– Deploy to IBM Service Platform based on architecture model• Persistence Services, Queuing Services etc.

§ Secure through Service Platform– Identity, Compliance, Isolation

§ Connect to Enterprise through Service Platform (including Data Transfer)§ Integrate platform development with on-boarding of IVT 6 pilots and existing solutions (ISV,

IBM)


IBM Cloud Service Platform Architecture

IaaS AbstractionIaaS Abstraction

Cloud Application RuntimesCloud Application Runtimes

Cloud ServicesCloud Services

PaaS Web

Interface

PaaS Web

Interface

Storehouse(Software Bundles,

Apps, Metadata)

Storehouse(Software Bundles,

Apps, Metadata)

Pattern DeployedPattern Deployed Shared User (aka Stood Up)Shared User (aka Stood Up)

Infrastructure Cloud PluginInfrastructure Cloud Plugin

Hybrid C

onnectivityH

ybrid Connectivity

PaaS Management System(Image/Pattern/Activation Tools, Auto-scaling, Monitoring and Control,

Security, BSS, Routing, Clustering, HA, SLA Management, etc)




Situational/Mashup Class AppsSituational/Mashup Class AppsEnterprise Class AppsEnterprise Class Apps

IBM Cloud Service Platform Details

IaaS AbstractionIaaS Abstraction




Security, BSS, Routing, Clustering, HA, SLA Management, etc)PaaS Web InterfacePaaS Web Interface Storehouse(Software Bundles, Apps,

Metadata)

Storehouse(Software Bundles, Apps,

Metadata)

Hybrid C

onnectivityH

ybrid Connectivity

Infrastructure Cloud PluginInfrastructure Cloud Plugin


Cloud ServicesCloud Services

Pattern DeployedPattern Deployed Shared UserShared User

WAS- OR Frameworks- Global Context

WAS- OR Frameworks- Global Context

DB2DB2 MQMQSilverliningSilverlining


Cognos/SPSS/M-R- AAF- TextAnalytics- Data-as-a-Service

Cognos/SPSS/M-R- AAF- TextAnalytics- Data-as-a-Service

HadoopHadoop

Cleansing-as-a-

Service

Cleansing-as-a-

Service


WASWAS

DB2-as-a-ServiceDB2-as-a-Service

sMashsMash

IDSIDS

CassandraCassandra

BluedoveBluedove exTremeScaleexTremeScale

Xactional WorkloadsXactional Workloads Analytics WorkloadsAnalytics Workloads Loosely Coupled WorkloadsLoosely Coupled Workloads

MG

MT

MG

MT

ProcessProcess

Data

Data

cloud computing and service management day 5 10-25-10 · 2010-11-11 · cloud computing and service...

Documents