cloudfoundry architecture

Cloud Foundry Architecture Ramnivas Laddad

@ramnivas

© 2012 SpringOne 2GX. All rights reserved. Do not distribute without permission.

@ramnivas •  Spring framework committer •  Cloud Foundry committer •  Main interests

–  Cloud computing –  Aspect-oriented programming –  Scala and functional programming

•  Author of books and articles –  AspectJ in Action (1st and 2nd edition)

•  Speaker at many professional conferences –  JavaOne, JavaPolis, SpringOne, Software Development, No Fluff Just Stuff, EclipseCon,

O’Reilly OSCON etc.

•  Active involvement in AspectJ, Spring, and Cloud Foundry since their early form

Technologies “Я” Us

RabbitMQ

Postgres

REST

MySQL JPA

Hibernate JDBC

Django

node.js Redis

JTA

JMS

Web Services

Spring

Security

Caching

Mongo AMQP

Neo4j

JSF

Sinatra

Testing Scripting

HTML5

Rails Lift

Batch Job

LDAP

Grails

Ruby

Java

Python

Akka Scala

Closure

Hadoop

Technologies “Я” Us

RabbitMQ

Postgres

REST

MySQL JPA

Hibernate JDBC

Django

node.js Redis

JTA

JMS

Web Services

Spring

Security

Caching

Mongo AMQP

Neo4j

JSF

Sinatra

Testing Scripting

HTML5

Rails Lift

Batch Job

LDAP

Grails

Ruby

Java

Python

Akka Scala

Closure

Hadoop

Technologies “Я” Us

Monitoring

Hardware failures

Backups

Rolling Updates

Patches

Security DoS

OS

DNS Storage Routing

DBA

Network

Load balancing

User management Auditing

Access logs

Technologies “Я” Us

Monitoring

Hardware failures

Backups

Rolling Updates

Patches

Security DoS

OS

DNS Storage Routing

DBA

Network

Load balancing

User management Auditing

Access logs

§ More functionality § Time to market pressure § Complex integration § Higher stake in quality

Facets of complexity: Product

§ Sound architecture: future proofing without overdoing § Unit and integration tests § Responding to changing business needs § Confusing technology landscape

Facets of complexity: Development

§ Choosing the right hardware, operating system, web server § Monitoring applications § Responding to scalability needs § Dealing with hardware- and system-level failures § Upgrading without substantial down time

Facets of complexity: Deployment and operation

Inherent vs. Apparent Complexity

Implementation

Inherent vs. Apparent Complexity

Implementation Functional logic

Inherent vs. Apparent Complexity

Implementation Functional logic Inherent

complexity

Inherent vs. Apparent Complexity

Implementation Functional logic

Implementation overhead

Inherent complexity

Inherent vs. Apparent Complexity

Implementation Functional logic

Implementation overhead

Inherent complexity

Apparent complexity

Inherent vs. Apparent Complexity

Implementation Functional logic

Implementation overhead

Inherent complexity

Apparent complexity

What can we do about this?

What can we do?

?

SaaS Software as a Service

PaaS Platform as a Service

IaaS Infrastructure as a Service

Three layers of Cloud Computing

Private    Clouds    

Public  Clouds  

Micro  Clouds  

Cloud Foundry open PaaS - Choice of clouds

Data Services

Other Services

Msg Services

Private    Clouds    

Public  Clouds  

Data Services

Other Services

Msg Services

Apache2 license

Micro  Clouds  

Cloud Foundry open PaaS - Choice of clouds

Private    Clouds    

Public  Clouds  

Data Services

Other Services

Msg Services

Partners  

Apache2 license

Micro  Clouds  

Cloud Foundry open PaaS - Choice of clouds

Cloud Foundry Architecture

•  No single point of failure •  Distributed state •  Self healing •  Horizontally scalable

Architectural Goals

•  Loose coupling •  Event-driven •  Asynchronous •  Non-blocking •  Idempotent •  Eventually consistent •  Language-independent communication

Architectural Principles

Cloud Foundry Layers

Hardware (CPU, Memory, Storage, Network)

Infrastructure as a Service

Outer Shell

Inner Shell

Clients

Cloud Foundry Core Components The Inner Shell

Cloud Foundry Inner Shell and Up

Routers

CloudControllers App App

HealthManager Execution Agents

(DEA) Pool

Messaging

Users Developers

Stagers

Services

Zooming in…

© 2012 VMware, Inc. All rights reserved

staging jobs

redis

cc - db

cloud controller stager

package cache

blobstore

uaa - db dea

health manager

staging logs uaa - AuthN

App meta data

dea dea dea dea dea

•  Dynamically discoverable components •  No inter-component dependencies

–  Launch in any order –  Scale up and down independently

•  Monitor using HTTP end points

Design Principles

Cloud Foundry Inner Shell and Up

Routers

CloudControllers App App

HealthManager Execution Agents

(DEA) Pool

Messaging

Users Developers

Stagers

Services

•  Interface with the clients –  VMC –  STS –  Portal

•  Provides REST interface to domain objects –  Apps –  Services –  Orgs –  Spaces

Cloud Controller

Demo

Cloud Controller REST API

Cloud Foundry Inner Shell and Up

Routers

CloudControllers App App

HealthManager Execution Agents

(DEA) Pool

Messaging

Users Developers

Stagers

Services

•  Responsible for morphing user app into executable •  Pluggable architecture

–  Each plugin understands a framework or a runtime •  Allow the DEA to view applications uniformly

Stager

Stager’s role

Spring Play Rails …

……………………………………………………………………………..

Stager ……………………………………………………………………………… ::::::::::

app.war

From bits to executable app

app.war

From bits to executable app

app.war

From bits to executable app

app.war

startup

stop

From bits to executable app

Cloud Foundry Inner Shell and Up

Routers

CloudControllers App App

HealthManager

Messaging

Users Developers

Stagers

Services

Cloud Foundry Inner Shell and Up

Routers

CloudControllers App App

HealthManager Execution Agents

(DEA) Pool

Messaging

Users Developers

Stagers

Services

app.war

startup

stop

From bits to running app

app.war

startup

stop

From bits to running app

app.war

startup

stop

A g e n t

From bits to running app

app.war

startup

stop

A g e n t

From bits to running app

•  Responsible for running all apps •  Monitors apps

–  Memory and disk quota –  Stage changes

•  Uniform view of all apps –  Runtime/framework differences sorted by the stager

•  Ensures app isolation

Droplet Execution Agent (DEA)

Application Isolation

Warden container

Private network

App

Private File System

Host network

DEA Container API

Cloud Foundry Inner Shell and Up

Routers

CloudControllers App App

HealthManager Execution Agents

(DEA) Pool

Messaging

Users Developers

Stagers

Services

•  Responsible to route requests to –  User apps –  External-facing components

•  Cloud Controller •  UAA

Router

Application Routing

Messaging

Router

Application Routing

Messaging

Router

Application Routing

Messaging

DEA url è host:port

Router

Application Routing HTTP

Messaging

DEA url è host:port

Router

Application Routing HTTP

Locator Services

Nginx (+Lua extension)

Messaging

DEA url è host:port

Router

Application Routing HTTP

Locator Services

Nginx (+Lua extension)

Messaging

DEA url è host:port

Cloud Foundry Inner Shell and Up

Routers

CloudControllers App App

Execution Agents (DEA) Pool

Messaging

Users Developers

Stagers

Services

Cloud Foundry Inner Shell and Up

Routers

CloudControllers App App

HealthManager Execution Agents

(DEA) Pool

Messaging

Users Developers

Stagers

Services

§ Expected state: •  Cloud Controller

§ Current state: •  DEAs

§ Current state ç Expected state

Health Manager

•  Centralized Identity Management –  Authenticates users from multiple sources –  Presents a single standard protocol for consumers

•  User Account Management •  Client Application Registration •  OpenID Connect and Oauth2 – delegated authorization

–  Uses Spring Security

UAA

Oauth2 for Cloud Foundry

User Resource owner

UAA Identity provider

CF Portal Client

Cloud controller Resource server

Oauth2 for Cloud Foundry

User Resource owner

UAA Identity provider

CF Portal Client

Cloud controller Resource server

Accesses the portal

Who is this user What is he/she requesting Do I have the necessary authorization

Oauth2 for Cloud Foundry

User Resource owner

UAA Identity provider

CF Portal Client

Cloud controller Resource server

Accesses the portal

Who is this user What is he/she requesting Do I have the necessary authorization

Authenticate me Assert - Portal can only read my apps

Authenticates user Issues authorization code

Oauth2 for Cloud Foundry

User Resource owner

UAA Identity provider

CF Portal Client

Cloud controller Resource server

Accesses the portal

Who is this user What is he/she requesting Do I have the necessary authorization

Authenticate me Assert - Portal can only read my apps

Authenticates user Issues authorization code

Redirect user along with the authcode

Oauth2 for Cloud Foundry

User Resource owner

UAA Identity provider

CF Portal Client

Cloud controller Resource server

Accesses the portal

Who is this user What is he/she requesting Do I have the necessary authorization

Authenticate me Assert - Portal can only read my apps

Authenticates user Issues authorization code

Exchange authcode for an access token

Issue access token scoped to cloud_controller.apps.read

Redirect user along with the authcode

Oauth2 for Cloud Foundry

User Resource owner

UAA Identity provider

CF Portal Client

Cloud controller Resource server

Accesses the portal

Who is this user What is he/she requesting Do I have the necessary authorization

Authenticate me Assert - Portal can only read my apps

Authenticates user Issues authorization code

Exchange authcode for an access token

Present token containing cloud_controller.apps.read

Issue access token scoped to cloud_controller.apps.read

Redirect user along with the authcode

Cloud Foundry Inner Shell and Up

Routers

CloudControllers App App

HealthManager Execution Agents

(DEA) Pool

Messaging

Users Developers

Stagers

Cloud Foundry Inner Shell and Up

Routers

CloudControllers App App

HealthManager Execution Agents

(DEA) Pool

Messaging

Users Developers

Stagers

Services

•  Service Gateway –  Provisions and unprovisions services –  Advertise service availability to CloudController

•  Service Nodes –  Runs actual services –  Scale independently

Service Provisioning

Access to services info VCAP_SERVICES: {! "postgresql-9.0": [{! "name": "env-postgresql",! "label": "postgresql-9.0",! "plan": "free",! "credentials": {! "name": "de24667f9344b4eeaad6b5a2326d52faa",! "host": "172.30.48.122",! "port": 5432,! "user": "u50ce600bba434bacbc99e034bb415644",! "password": "pf4dca5bd449d4732841f0c4ae3f299d0"! }! }]!}!

Operating Cloud Foundry The Outer Shell

§  500 – 5,000 VMs §  40+ unique node types §  75+ unique software packages §  2x/week cf.com updates §  24x7x365 non-stop operation §  No-downtime deployments §  Reliable, robust, repeatable

deployments, updates, capacity adjustments

§  Small teams manage many instances

Production Grade Cloud Foundry Clusters

production, staging, stress, qa, dev

cloudfoundry.com

Service Evolution Technology for Operating Cloud Foundry in Production Environments

Automated Virtual Infrastructure •  At Cloud Scale •  Abstracted using a Cloud Provider Interface (CPI) Software Deployment, Configuration and Updates •  Optimized to Minimize Downtime •  Support for multiple VM roles Repeatable Process •  Release Management with Versioning Active Monitoring and Alerting BOSH has been used to run CloudFoundry.com since launch

What is BOSH? github.com/cloudfoundry/bosh

Cloud Foundry “BOSH” – Concepts

"BOSH"

Cloud Foundry “BOSH” – Concepts Stemcell • Base OS • “BOSH” Agent

"BOSH"

Cloud Foundry “BOSH” – Concepts Stemcell • Base OS • “BOSH” Agent

Release • Name Jobs • Software Packages • Configuration Templates • Scripts

Software Packages • Externally developed s/w • Internally developed s/w

"BOSH"

Cloud Foundry “BOSH” – Concepts Stemcell • Base OS • “BOSH” Agent

Deployment Manifest • Release name and version • # VMs, params for each Job • Stemcells to use

Release • Name Jobs • Software Packages • Configuration Templates • Scripts

Software Packages • Externally developed s/w • Internally developed s/w

"BOSH"

Cloud Foundry “BOSH” – Concepts Stemcell • Base OS • “BOSH” Agent

Deployment Manifest • Release name and version • # VMs, params for each Job • Stemcells to use

Release • Name Jobs • Software Packages • Configuration Templates • Scripts

Software Packages • Externally developed s/w • Internally developed s/w

"BOSH"

Cloud Foundry “BOSH” – Concepts

Environment

Stemcell Software Packages

Configuration

Stemcell • Base OS • “BOSH” Agent

Deployment Manifest • Release name and version • # VMs, params for each Job • Stemcells to use

Release • Name Jobs • Software Packages • Configuration Templates • Scripts

Software Packages • Externally developed s/w • Internally developed s/w

"BOSH"

Rolling Update of a Stateless Component

Message Bus

Cloud Controller

v1

Starting with v1 VMs

Router

Incoming HTTP Requests

Message Bus

Cloud Controller

v1

Cloud Controller

v2

Deploy a v2 VM (canary)

Create

Router

Incoming HTTP Requests

Message Bus

Cloud Controller

v1

Cloud Controller

v2

If it works, add more v2 VMs...

Create

Router

Incoming HTTP Requests

Example: Rolling Update of a Stateless Component

End Result: We upgraded from v1 to v2 with no downtime by building new VMs and destroying old ones

Message Bus

Cloud Controller

v1

Cloud Controller

v2

… while removing v1 VMs…

Create Destroy

Router / LB Incoming HTTP Requests

Message Bus

Cloud Controller

v2

… until all VMs are v2

Router / LB Incoming HTTP Requests

§ Parallel § Canaries § Eventual consistency

Deployment

§ Components •  Director •  Agent •  CLI

§ Stemcell

Under the Hood

Code Change

VM Update

VM Update

Cloud Provider Interface •  Abstracts infrastructure

–  Stemcells •  VM template

–  VMs –  Disks –  Networks

IaaS neutral by design

CPI: code complete functional status: “work in progress”

vSphere: battle tested implementation, thousands of deployments vCloud Director: “work in progress”, 2H 2012

contribute: github.com/cloudfoundry/bosh

Cloud Provider Interface (CPI)

Cloud Foundry BOSH

CPI Providers

Twitter •  http://twitter.com/cloudfoundry •  http://twitter.com/ramnivas

Sign up online at cloudfoundry.com •  Website: http://cloudfoundry.org •  Blog: http://blog.cloudfoundry.com and http://blog.cloudfoundry.org •  GitHub: http://github.com/cloudfoundry •  YouTube: http://youtube.com/cloudfoundry

Learn More. Stay Connected.

Questions?

Cloud Foundry Architecture Ramnivas Laddad

@ramnivas

© 2012 SpringOne 2GX. All rights reserved. Do not distribute without permission.