modularity in the cloud a case study

Modularity in the cloud

A case study

Paul BakkerArchitect at Luminis Technologies

@pbakker

Marcel OffermansDirector at Luminis TechnologiesMember at Apache

@m4rr5

The case

Educational system focussed on personalized learning

Used in high schools in The Netherlands

Expand to other countries in the near future

Cross device HTML 5 + JS

RESTful web services

OSGi services

Mongo

Apache Felix

Amdatu

S3

Amdatu: OSGi Cloud components

JAX-RSMONGO DBBlobStoresMulti-tenancySearchOpenSocial...

Apache Licensed

open source

project

Some numbers

190bundles

in a deployment

120PulseOn

bundles

School B

Load

Deployment

Load Balancer

PulseOn nodePulseOn node

PulseOn node

Load Balancer

PulseOn node

PulseOn node

PulseOn node

School A

Mongo

Mongo

Mongo

MongoMongo

Mongo

Availability zones

Load Balancer

PulseOn nodePulseOn node

PulseOn node

PulseOn nodePulseOn node

PulseOn node

Availability Zone 1

MongoMongo

Mongo

MongoMongo

Mongo

Availability Zone 2

Horizontal scalability

Horizontal scaling requires stateless nodes

HTML5 clients need less server side state

Any state should go to some kind of store

Auto scaling

Considerable higher loads during school hours

Enoughcapacity

Without paying for idle servers

at night...

Load Balancer small node

Cluster per school

Always use a load balancer because we don’t want downtime during scaling

Load Balancer small node

large node

large node

Early morning...

End of the day...

Load Balancer small node

large node

large node

But how do we install

our software on a node?

Provisioning servers

ace screenshot

User Interface

REST API Shell scripting

Node

AWS Auto Scaling

1. Start Load Balancer

Apache ACE

2. register

3. register

4. provisiondeployment

package

Provisioning servers

./as-­‐create-­‐launch-­‐config  demo            -­‐-­‐image-­‐id  ami-­‐0ee8e07a            -­‐-­‐instance-­‐type  m1.small            -­‐-­‐region  eu-­‐west-­‐1            -­‐-­‐group  sg-­‐ce1420ba            -­‐-­‐user-­‐data-­‐file  userdata.txt

./as-­‐create-­‐auto-­‐scaling-­‐group  demo          -­‐-­‐launch-­‐configuration  demo          -­‐-­‐min-­‐size  1          -­‐-­‐max-­‐size  1          -­‐-­‐availability-­‐zones  eu-­‐west-­‐1a          -­‐-­‐load-­‐balancers  demo          -­‐-­‐tag  "k=Name,v=demo,p=true"

Build releases

CI server

Release OBR

-runbundles: org.apache.felix.http.jetty;version=2.2.0,\ org.apache.felix.gogo.command;version=0.10.0,\ org.apache.felix.gogo.runtime;version=0.10.0,\ org.apache.felix.gogo.shell;version=0.10.0,\ osgi.cmpn;version='(4.2.0,4.2.1]',\ org.amdatu.security.tokenprovider.api;version=latest,\ org.amdatu.security.tokenprovider.impl;version=latest,\ org.amdatu.web.rest.doc,\ com.lopexs.arl.opensocial.config;version=latest,\ jackson-core-asl,\

Apache ACE2. Baseline 3. Publish

4. Distribution

Targets

1. GIT tag

5. Provision

How do we know which

bundles to update?

Semantic versioning

is usually about

package versions

Export-Package: org.apache.felix.dm;version="3.1.0"

Semantic versioning

demystified

major.minor.micro(.qualifier)

Major: Backward incompatible change

Minor: Backward compatible change

Micro: Implementation change

Qualifier: Label, e.g. build number

1.0.0.x

Semantic versioning

on bundles

Bundle-Version: 1.0.0

Important for deployment!

Baselining

Bytecode analysisCompare build with latest releaseCheck if version numbers should be bumped

Continuous deployment

CI server

Release OBR

-runbundles: org.apache.felix.http.jetty;version=2.2.0,\ org.apache.felix.gogo.command;version=0.10.0,\ org.apache.felix.gogo.runtime;version=0.10.0,\ org.apache.felix.gogo.shell;version=0.10.0,\ osgi.cmpn;version='(4.2.0,4.2.1]',\ org.amdatu.security.tokenprovider.api;version=latest,\ org.amdatu.security.tokenprovider.impl;version=latest,\ org.amdatu.web.rest.doc,\ com.lopexs.arl.opensocial.config;version=latest,\ jackson-core-asl,\

Apache ACE2. Baseline 3. Publish

4. Distribution

CD server

1. GIT tag

5. Provision

Snapshot OBR

ACE scripting with GoGotargetrepourl = "$DEPLOYMENT_TARGETOBR_URL"releaserepourl = "$DEPLOYMENT_RELEASEOBR_URL"autoconfurl = "$DEPLOYMENT_AUTOCONF_URL"

sourceindex = (repo:index /tmp/ace-cdeploy)sourcerepo = (repo:repo R5 $sourceindex)targetrepo = (repo:repo OBR $targetrepourl)releaserepo = (repo:repo OBR $releaserepourl)

echo "Deploying release resources"deployed = repo:cd $releaserepo $sourcerepo $targetrepo

echo "Opening client workspace"workspace = (ace:cw)

echo "Removing old artifacts"artifacts = ($workspace la)each $artifacts { $workspace da $it}

echo "Creating new artifacts"each $deployed { identity = $it getIdentity version = $it getVersion name = "$identity - $version" url = $it getUrl mimetype = $it getMimetype if { $mimetype equals "application/xml:osgi-autoconf" } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" filename="$name" processorPid="org.osgi.deployment.rp.autoconf" ] [ cdartifact="true" ] } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" Bundle-SymbolicName="$identity" Bundle-Version="$version" ] [ cdartifact="true" ] }}

echo "Create artifact to feature association"if { (coll:first ($workspace la2f "(rightEndpoint=*name=cdfeature*)")) } { } { $workspace ca2f "(cdartifact=true)" "(name=cdfeature)" "N" "1"}

echo "Create feature"if { (coll:first ($workspace lf "(name=cdfeature)")) } { } { $workspace cf "cdfeature" }

echo "Create feature to distribution association"if { (coll:first ($workspace lf2d "(leftEndpoint=*name=cdfeature*)")) } { } { $workspace cf2d "(name=cdfeature)" "(name=cddistribution)" "1" "1" }

echo "Create distribution"if { (coll:first ($workspace ld "(name=cddistribution)")) } { } { $workspace cd "cddistribution" }

echo "Create distribution to target association"if { (coll:first ($workspace ld2t "(leftEndpoint=*name=cddistribution*)")) } { } { $workspace cd2t "(name=cddistribution)" "(id=*)" "1" "N" }

echo "Create target"target = (coll:first ($workspace lt "(id=cdtarget)"))if { $target } { } { target = ($workspace ct "cdtarget") }if { $target isRegistered } { } { $target register }

echo "Configure target properties"$target addTag "ip-address" "127.0.0.1" $target addTag "schoolid" "someschool"$target addTag "timezone" "Europe/Amsterdam"

echo "Committing workspace"$workspace commit

ACE scripting with GoGo

targetrepourl = "$DEPLOYMENT_TARGETOBR_URL"releaserepourl = "$DEPLOYMENT_RELEASEOBR_URL"autoconfurl = "$DEPLOYMENT_AUTOCONF_URL"

sourceindex = (repo:index /tmp/ace-cdeploy)sourcerepo = (repo:repo R5 $sourceindex)targetrepo = (repo:repo OBR $targetrepourl)releaserepo = (repo:repo OBR $releaserepourl)

echo "Deploying release resources"deployed = repo:cd $releaserepo $sourcerepo $targetrepo

echo "Opening client workspace"workspace = (ace:cw)

echo "Removing old artifacts"artifacts = ($workspace la)each $artifacts { $workspace da $it}

echo "Creating new artifacts"each $deployed { identity = $it getIdentity version = $it getVersion name = "$identity - $version" url = $it getUrl mimetype = $it getMimetype if { $mimetype equals "application/xml:osgi-autoconf" } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" filename="$name" processorPid="org.osgi.deployment.rp.autoconf" ] [ cdartifact="true" ] } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" Bundle-SymbolicName="$identity" Bundle-Version="$version" ] [ cdartifact="true" ] }}

echo "Create artifact to feature association"if { (coll:first ($workspace la2f "(rightEndpoint=*name=cdfeature*)")) } { } { $workspace ca2f "(cdartifact=true)" "(name=cdfeature)" "N" "1"}

echo "Create feature"if { (coll:first ($workspace lf "(name=cdfeature)")) } { } { $workspace cf "cdfeature" }

echo "Create feature to distribution association"if { (coll:first ($workspace lf2d "(leftEndpoint=*name=cdfeature*)")) } { } { $workspace cf2d "(name=cdfeature)" "(name=cddistribution)" "1" "1" }

echo "Create distribution"if { (coll:first ($workspace ld "(name=cddistribution)")) } { } { $workspace cd "cddistribution" }

echo "Create distribution to target association"if { (coll:first ($workspace ld2t "(leftEndpoint=*name=cddistribution*)")) } { } { $workspace cd2t "(name=cddistribution)" "(id=*)" "1" "N" }

echo "Create target"target = (coll:first ($workspace lt "(id=cdtarget)"))if { $target } { } { target = ($workspace ct "cdtarget") }if { $target isRegistered } { } { $target register }

echo "Configure target properties"$target addTag "ip-address" "127.0.0.1" $target addTag "schoolid" "someschool"$target addTag "timezone" "Europe/Amsterdam"

echo "Committing workspace"$workspace commit

ACE scripting with GoGo

targetrepourl = "$DEPLOYMENT_TARGETOBR_URL"releaserepourl = "$DEPLOYMENT_RELEASEOBR_URL"autoconfurl = "$DEPLOYMENT_AUTOCONF_URL"

sourceindex = (repo:index /tmp/ace-cdeploy)sourcerepo = (repo:repo R5 $sourceindex)targetrepo = (repo:repo OBR $targetrepourl)releaserepo = (repo:repo OBR $releaserepourl)

echo "Deploying release resources"deployed = repo:cd $releaserepo $sourcerepo $targetrepo

echo "Opening client workspace"workspace = (ace:cw)

echo "Removing old artifacts"artifacts = ($workspace la)each $artifacts { $workspace da $it}

echo "Creating new artifacts"each $deployed { identity = $it getIdentity version = $it getVersion name = "$identity - $version" url = $it getUrl mimetype = $it getMimetype if { $mimetype equals "application/xml:osgi-autoconf" } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" filename="$name" processorPid="org.osgi.deployment.rp.autoconf" ] [ cdartifact="true" ] } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" Bundle-SymbolicName="$identity" Bundle-Version="$version" ] [ cdartifact="true" ] }}

echo "Create artifact to feature association"if { (coll:first ($workspace la2f "(rightEndpoint=*name=cdfeature*)")) } { } { $workspace ca2f "(cdartifact=true)" "(name=cdfeature)" "N" "1"}

echo "Create feature"if { (coll:first ($workspace lf "(name=cdfeature)")) } { } { $workspace cf "cdfeature" }

echo "Create feature to distribution association"if { (coll:first ($workspace lf2d "(leftEndpoint=*name=cdfeature*)")) } { } { $workspace cf2d "(name=cdfeature)" "(name=cddistribution)" "1" "1" }

echo "Create distribution"if { (coll:first ($workspace ld "(name=cddistribution)")) } { } { $workspace cd "cddistribution" }

echo "Create distribution to target association"if { (coll:first ($workspace ld2t "(leftEndpoint=*name=cddistribution*)")) } { } { $workspace cd2t "(name=cddistribution)" "(id=*)" "1" "N" }

echo "Create target"target = (coll:first ($workspace lt "(id=cdtarget)"))if { $target } { } { target = ($workspace ct "cdtarget") }if { $target isRegistered } { } { $target register }

echo "Configure target properties"$target addTag "ip-address" "127.0.0.1" $target addTag "schoolid" "someschool"$target addTag "timezone" "Europe/Amsterdam"

echo "Committing workspace"$workspace commit

ACE scripting with GoGo

targetrepourl = "$DEPLOYMENT_TARGETOBR_URL"releaserepourl = "$DEPLOYMENT_RELEASEOBR_URL"autoconfurl = "$DEPLOYMENT_AUTOCONF_URL"

sourceindex = (repo:index /tmp/ace-cdeploy)sourcerepo = (repo:repo R5 $sourceindex)targetrepo = (repo:repo OBR $targetrepourl)releaserepo = (repo:repo OBR $releaserepourl)

echo "Deploying release resources"deployed = repo:cd $releaserepo $sourcerepo $targetrepo

echo "Opening client workspace"workspace = (ace:cw)

echo "Removing old artifacts"artifacts = ($workspace la)each $artifacts { $workspace da $it}

echo "Creating new artifacts"each $deployed { identity = $it getIdentity version = $it getVersion name = "$identity - $version" url = $it getUrl mimetype = $it getMimetype if { $mimetype equals "application/xml:osgi-autoconf" } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" filename="$name" processorPid="org.osgi.deployment.rp.autoconf" ] [ cdartifact="true" ] } { $workspace ca [ artifactName="$name" url="$url" mimetype="$mimetype" Bundle-SymbolicName="$identity" Bundle-Version="$version" ] [ cdartifact="true" ] }}

echo "Create artifact to feature association"if { (coll:first ($workspace la2f "(rightEndpoint=*name=cdfeature*)")) } { } { $workspace ca2f "(cdartifact=true)" "(name=cdfeature)" "N" "1"}

echo "Create feature"if { (coll:first ($workspace lf "(name=cdfeature)")) } { } { $workspace cf "cdfeature" }

echo "Create feature to distribution association"if { (coll:first ($workspace lf2d "(leftEndpoint=*name=cdfeature*)")) } { } { $workspace cf2d "(name=cdfeature)" "(name=cddistribution)" "1" "1" }

echo "Create distribution"if { (coll:first ($workspace ld "(name=cddistribution)")) } { } { $workspace cd "cddistribution" }

echo "Create distribution to target association"if { (coll:first ($workspace ld2t "(leftEndpoint=*name=cddistribution*)")) } { } { $workspace cd2t "(name=cddistribution)" "(id=*)" "1" "N" }

echo "Create target"target = (coll:first ($workspace lt "(id=cdtarget)"))if { $target } { } { target = ($workspace ct "cdtarget") }if { $target isRegistered } { } { $target register }

echo "Configure target properties"$target addTag "ip-address" "127.0.0.1" $target addTag "schoolid" "someschool"$target addTag "timezone" "Europe/Amsterdam"

echo "Committing workspace"$workspace commit

Cluster

OSGi Target

Load Balancer

MongoDB

MongoDB

MongoDB

OSGi Target

Infrastructure Management

Component Provisioning

Cluster Management

System Monitoring

Automatic Scaling

Starts/stops nodes

Deploys and updates

software components

Monit

ors Q

uality

of S

ervic

e

aspe

cts

Adds/removes nodes from

the cluster

Instantiates one or more template based clusters

Cluster TemplateCluster

TemplateCluster Template

Artifacts, Features &

Deployments

5

Cluster Configuration

1

Requests nodes based on provided requirements

2 3Assi

gns s

oftware

distrib

ution

s to t

argets

4

Gets configured with artifacts for deployments

Provides usage data for scaling algorithms

6

7

89

Tooling and frameworks are

now mature enough

How does this fit other applications?

The architecture fits most

modern web applications

Works in the cloud

(but not a requirement)

amdatu.org

Recommended reading ;)

Available NOW!

Book signing Tue 14.30

@ O’Reilly Booth (JavaOne Pavilion)

Free books for the early birds...

There is more...

Building Modular Cloud Applications in Java:

Lessons LearnedTue 10am - 11am

Modular JavaScriptWed 3pm - 4pm

Cloud provisioning

http://ace.apache.org/Cloud OSGi services

http://www.amdatu.org/

Eclipse OSGi pluginhttp://bndtools.org/

That’s ushttp://luminis.eu/

There is more...

Takk

Grazie

Thankyou

Obrigado

MahaloDankeDank U

Merci

Gracias