development model for the cloud

Development Model for the Cloud Paradigm Shift of the Same Old Same Old? Dr. Umit Yalcinalp, Salesforce.com Developer Evangelist

Computing History Reduce Complexity, Do More

  Turing Machines

  Assembly code

  Programming Languages

  Application Frameworks

  Application Platforms

  Application Servers

  Scripting and Dynamic Languages

  …

Striving Towards Less Complexity

  Punch Cards

  Cobol

  C++

  Platforms

  Application Servers

  Scripting and Dynamic Languages

  …

Take Application Servers as an Example…   Essential Built in services

–  Load Balancing

–  Memory Cache

–  Transaction Management

–  Connection management

  Packaging a full app (war, ear, …)   Security/Authentication

  Messaging Infrastructure simplifies tasks

  User management   Monitoring and Management of Apps

  Versioning   Tooling and Composition

  Integration APIs (WS, REST, …)

However Business App Dev Requirements stayed similar…

  Develop an application that has m abstract entities. Define relationship between entities.

  Display data subset j in a page and update data set if x is true.

  Implement and Deploy an application that has n users where a user can belong to one of k different roles

  Restrict access of data subset j to user role i when x is true.

  If user role has made change p, notify user roles s.

  Do {x, y, z, send email, fax, ….to m users} when f happens.

  Do f every {day, week, …} when s …   Generate report on … organized by …

  Make the application extensible, customizable…

  Deploy mobile client for app that maintain disconnected data for user using dataset x .

  Integrate app with Twitter, feed, …

  Integrate app with Payment System, …

Challenges in traditional way of developing apps remained…

Infrastructure Services: Hardware

Infrastructure Services: Operating System

Infrastructure Services: Database

Infrastructure Services: App Server

Application Services

Operational Services

Business Services

How do many developers make this happen?

Putting it together: Duct Tape!

Complexities Abound to Final Destination

Infrastructure Services

Network Storage

Operating System Database

App Server Web Server Data Center

Disaster Recovery

Application Services

Security Sharing

Integration Customization Web Services

API Multi-Language Multi-Currency

Integrated Workflow Analytics

Multi-Device Messaging

Search

Operational Services Authentication

Availability Monitoring

Patch Mgmt Upgrades Backup

NOC

Business Services

Ordering Provisioning

Licensing Billing

Renewal Upgrades Marketing

Sales

Application Ideas

Final App

Now comes along Cloud Computing Paradigm

The chatter and buzz on the street…

  About developing services, platforms and applications

  Don’t host but use hosted resources elsewhere   Develop with hosted services, software, platforms

  Pay as you go

  Consume as you go   Monitor and adopt

Applications Moving to the Cloud

Today Cloud Computing Applications

1960’s Mainframe

1980’s Client/server

Today Cloud Computing Platforms

1960’s Mainframe

1980’s Client/server

Platforms Moving to the Cloud

1980’s Work Group Computing

2000s Intranet

Computing

Collaboration Moving to the Cloud

Today Social

Computing

Lotus Notes Novell GroupWise

SharePoint Groove File Sharing

Throw in a lot of new technologies …

Map reduce

Column db

Worker task

Scheduler administration

Data service

What does this new paradigm offer to app developers?…

The Three AmigoS of Cloud Computing Cloud Computing Platforms

  Software –  Hosted apps, abstracting APIs, OS, Data storage…

  Platform –  Hosted collection of APIs, abstracting OS, data storage, …

  Infrastructure –  Hosting virtual servers, OS and some kind of data store

DAT

A

?!

?!

Abstractions and Services

They are all over the Map

Cloud Developer Concerns: Capabilities Managed Environment   Abstraction Level

–  Capabilities –  Constraints

  Managing the Managed Environment

  Cost Indirectly Affecting Development

Lets Look at these Cloud Computing Platforms

  Amazon WS

  Microsoft Azure

  Google App Engine (GAE)

  Force.com

Services Available:   Virtual OS instances EC2

  Data Services –  S3 –  RDS

  Additional Services: –  Load Balancing Servers –  Simple Queue Service –  Elastic MapReduce

Processing Large Data Sets •  Job Flows on top of Hadoop

–  Different granularity of accessing data (i.e. Pig) •  Monitoring Individual Task Progress

…

Developer Perspective

  Hosting Locally Built Infrastructure Deployed Elsewhere

  Hosting Locally Built Apps

  Building Data Management Platform

  Testing/Monitoring –  Great Console for Infrastructure Monitoring –  Deployed app still responsibility of developer

Amazon AWS Activity:   Not about OS and simple data store anymore

  Hosting –  Existing Infrastructure Components: JEE, DB2, …(composed images)

–  Traditionally built apps   What do you build with it

–  More infrastructure: DIY app server functions for scalability & composition

•  Load Balancing

•  Deciding when to take resources offline (in development testing)

–  Low Level Data Services •  Large Data Processing (must understand Hadoop…)

–  Infrastructure Management Logic

  Development Methodology: It Depends!

  Managing/Debugging Your App: –  Console (monitoring, debugging, management (upgrades, shutdown))

–  Remote scripts managing specific to installed components

Microsoft Azure Platform:   Hosting Azure applications on Microsoft Cloud   Services for backend development (Old and New)

–  Table (Data Storage) –  Asynchronous Worker Tasks –  SQL Azure

  Develop/Deploy –  Requires OS+Long list of Packages Installation Locally –  Built/Test on local development environment

  Entry Point for developers is Visual Studio –  .NET libraries –  Local debugging –  Local packaging(.cspkg)/Manual deployment to location based Microsoft cloud

  Management of Remote Servers/Services –  Requires Server Logs Messages for managing

  Very tailored to existing Microsoft developers –  Java developer targeted with Third Party tools porting APIs

Google App Engine   Services: Lean App Server Features

–  JPA API for hiding BigTable –  Connection support, Cron like jobs

–  User Administration (Google accounts/Roll your own)

  Backend development Basics –  BigTable/GFS –  Tasks Queues (experimental)

  Built/Test on local development environment –  Naturally independent of OS

–  JRE + GAE

  Managed Environment Limits

  Entry Point: Eclipse –  Develop Locally/Deploy war file Remotely

  Remote Monitoring –  System Logs

  Multilanguage (Java/PHP)

Commonalities & Differences Emerge

  Abstracted Large Data Set or SQL

  Managed Environment Limits

  Asynchronous Task/Process Management –  Deal with Data Sets

–  Working with Managed Environment Limits

  Delineation of Development & Production

  Art of “System Log” Recording and Monitoring

App Services and Application Requirements?

Where do the Application Requirements get addressed?

Next Level in PaaS:

–  Infrastructure –  Application Server Capabilities

Force.com Platform: Application Services + Integrated End2End Framework

Qua

rterly

Rev

enue

($M

)

Revenue through fiscal quarter ended 10/31/09

FY2005 FY2006 FY2007 FY2008 FY2009 FY2010

First Cloud Company to Exceed:

Annual Revenue Run Rate $1.3 Billion +

FY2006 FY2007 FY2008 FY2002 FY2003 FY2004 FY2005 FY2009 FY2010

Paying Customers

Strong Growth In New Customers

67,900+

Multi-tenant Automatic Upgrades Pay-as-you-go

Elastic

Application Development: 5X Faster At half the cost

The Cloud Computing Model

Force.com Platform for Building Applications

Three AmigoS and Force.com

  Software: Applications on Salesforce.com Cloud

  Platform

  Infrastructure:   Multitenant Kernel   Data Storage   ISO 20071 Certified Security   Loan Balancing   Replication & Recovery   Transaction Management   Sandboxed Application   Trust (Status Monitoring

  Customizable   Extensible

Integration

Dev

elop

men

t Met

hodo

logy

Logic

User Interface

Analytics

Distribution

Platform - Force.com Philosophy:

•  Common Things Simple Complex Things Possible

•  Development Activity • Declarative • Programmable

•  Metadata Driven with Shared Data Model

•  Built in: Customizable UI, Analytics, Distribution, Workflows

•  Iterative Development Paradigm: “See it as you build it”

•  Application Sharing/Exchange

•  Enforce Good Development Practices Development to Production: Testing

Data Model

Hello World?

Development Process

  Create an “org”: Private on SFDC cluster –  A collection of applications –  Built on top of shared metadata

–  Every org becomes live and pre-populated with metadata –  Can accommodate > 1 developers depending on license

  Develop/Test

  If Sandbox, Complete Tests to Production

  Register Custom URL/Expose as Website

  “Package” specific app for AppExchange

Two Development Approaches:

  Web Interface –  What you build is what you see –  Rapid Prototyping

  Eclipse Plugin

The App is alive on the cloud

Application Data Model

LineItem Invoice Merchandise

Name

IsDeleted

…

• Text • LongText • RichText • Boolean • Number • Picklist (single or Multivalue) • … • Computed Fields

• Formula • Rollup

• Lookup: 1-m • Master-Detail: Hierarchical

SOQL: Object Oriented Query Language Metadata

Business Logic

Field Requiredness/Uniqueness

Audit History Tracking

Workflows Rules & Approval Processes

Declarative Logic (point and click)

Formula-Based Logic (spreadsheet-like)

Procedural Logic (code)

Formula Fields

Data Validation Rules

Workflows Rules & Approval Processes

Apex Triggers

Apex Classes

Declarative Workflow: Visual Process Manager

Data Model Realized in Default UI Unlimited Real-Time Customization

Custom Pages

Sites

Saw that even BigFoot is moving to the Cloud…

Next Stop Wonderland: Collaboration for the Enterprise

• One can follow any Entity just like a User • Same Development Paradigm with Any Entity in the platform

Developer’s Toolbox   Language Runtime: Apex: Java, C# like language

–  For creating CRUD Logic on Data Model (triggers)

–  Executing Queries (SOQL and SOSL)

–  Developing Controllers in conjunction with VF Pages

–  Background Process Management •  Scheduled Tasks

•  Batch Processes for handling large data sets

  Eclipse Plugin –  Developed using WS APIs

–  Retains only metadata locally

–  Allows browsing metadata, dynamic SQL and code execution

–  Remote execution, commit of metadata, dynamic code blocks

–  Synchronizes live development with an org on the cloud

–  Enables versioning locally with external tools

Developer

  Integration Toolkits: –  Force.com WS API language bindings in target languages, environments

  Packaging Apps for Distribution –  Within an org selecting an app

–  Requires Test Implementation

  Sandboxing –  A snapshot of metadata (and data) of an existing org

–  Requires testing to move to production

  Integral Part of Development Process: –  Testing Framework

–  Governance Limits

Differences in Platforms

  Degrees of Abstraction and Tight Integration with Data Model

  Abstraction of Application Server Capabilities

  Exposing Application Framework Capabilities

  Data Model & Handling and Managing Large Data

  Background Process & Task/Queue

  Governance Limits

  Level of Integration Support

  Level of Management Needed from Developer

  Tangible and Intangible Cost of using a platform

The Duct Tape Index for Cloud Platforms

Cloud Development Landscape   Levels of Abstraction

  Application Composition

  Development Environment (local, remote, mixed)

  Workflow

  Development and Deployment Cycles

  Customizability / Extensibility

  Integration

  Being a Good Citizen

  Versioning

  Monitoring

  Cost of Trying out a Platform

The confusion should not be cloud developer’s epitaph

Where a developer wants to get…

Conclusion:

  Choose Your Abstraction Level Wisely: –  Flexibility(Roll Your Own) vs. Time to Develop(Managed) –  Think long term

  Being a Good Citizen is Not Optional

  Logs and Monitoring Rules in Clouds

Q & A!

http://developer.force.com!Twitter: umityalcinalp !

Typical Development Cycle

  Create your own image/Packaging –  Bundle App Server/Application

  Install/Manage Remotely

  Monitor Deployed Services to scale and use –  Are more servers needed? –  Is a bigger database needed?

–  Do I need to shut down my servers during development?

  Manual Redeployment

User Interface: MVC Model Programmable User Interface