Cloud Integration with node.js

From enterprise integration to micro-services and iPaaS

Alboaie Sînică CEO Axiologic, http://www.axiologic.ro

● business aspects● security aspects● different

○ vendors○ teams○ products

● modularity/ product quality ● timing, new requests

What is Integration. Why?

● Real Time Integration○ Point to Point

■ custom code, exploding complexity, fragility○ Bus (ESB, MOM)

● Scheduled Integration○ export/import of files○ shared databases○ ETL scripts (Extract/Transform/Load)

Point to Point, Files, Databases

○ informational street○ all integration points learn how to “drive”

What is a Bus ?

ESB : Enterprise Service BUSMOM: Message Oriented MiddlewareEIP: Enterprise Integration Patterns● Benefits:

○ can easily add or remove integration points later○ fewer custom configs and simpler deployments (one point of

integration for everybody)○ bring all integration points to a common platform and protocol

Integration with Bus like systems

● messages / events● message channel (PUB/SUB, Point to Point)● message endpoint● pipes and filters, routing, translator● guaranteed delivery (duplicates)● load balancing/load distribution● logging● monitoring, throttler, SLA, etc

EIP concepts and concerns

● JMS (Java Massages Services) - MOM● Apache ActiveMQ ● Apache Camel (EIP)● Apache Service Mix (ESB)● ZeroMQ!● commercial (SAP, Oracle, Microsoft...)

Related Java/C technologies

Integration Platform as a Service

Many categories of iPaaS vendors● e-commerce and B2B integration: simple and limited solution● Enterprise Service Bus (ESB) for hybrid (cloud and on premise)● Service Oriented Architecture (SOA) infrastructure

iPaaS

Service Oriented Architecture● Orchestration

○ Business Process Management■ long living processes■ BPEL (standards)

○ service composition○ intra-organisation

● Choreography○ WS-CDL○ inter-organisation

● SOA v1 (orchestrated services centrally ● SOA v2 (event driven SOA)

BPM, SOA

● actors (Carl Hewitt)○ message queue○ adresses

● send asynchronous messages● behaviour

○ set the behaviour for the next message○ create new actors○ learn other address

● mobility, locality● intuitive, naive messages● inspired OOP

Concurrency model: Actors

pi-calculus, CSP, etc

Formal models for asynchronisms

● asynchronous● no threading● cluster (processes): vertical scalability● service routing, external balancer

Node.js Technologies

● micro-services● integration● easily change complex process● form of choreography?

SwarmESB - node.js

Inverted perspective on the actor model● messages have behaviour ● mobile code nod actors● decisions in messages not in actors● distributed in a network (horizontal

scalability)● Active messages

Swarm communication model

● adapters● swarm description● phases● variables● swarm primitives● swarms at runtime

Concepts

● swarms at runtime○ kind of message○ more a set of related messages ○ have behaviours attached ○ have a current phase○ can exist in multiple nodes ○ can have a single identity for all messages○ clone itself and visit adapters

Concepts

● adapters○ somehow like actors: target for messages○ adapt external APIs or micro-services○ basically: a collection of functions

Concepts

● swarm description○ script that describe the choreography○ a set of phases (functions) that should be

executed○ also contains

■ ctors (constructors)■ swarm variables

Concepts

● phases○ hints about what type of adapter (node) can be execute it○ contains a function (imperative code)○ can call adapter’s APIs and swarm, home, broadcast primitives○ modify swarm variables

Concepts

statusCheck:{ node:"*", code : function (){

this.status = currentAPI_status(); //set swarm variableif( this.status == “OK”){ //use swarm variable this.home("okStatusLogger");} else { this.swarm("errorStatusLogger");}

} }

● swarm variables○ variables with a type○ initialisation

● meta variables (swarm status, control)

Concepts

var: {

hello:”Hello World”

}

● swarm primitives○ swarm: send a clone of the current swarm in another phase

(another adapter)○ home: send a clone to the client that started the swarm○ broadcast: send multiple clones in all adapters of a specified type

Concepts

https://github.com/salboaie/SwarmESB/blob/master/swarms/NodeStart.js

Example hello world

BenchmarkSwarm:https://github.com/salboaie/SwarmESB/blob/master/testSwarms/BenchMark.js

Test:https://github.com/salboaie/SwarmESB/blob/master/test/BenchmarkTest.js

Exemple

Load distribution with swarmsAdapters:https://github.com/salboaie/SwarmESB/blob/master/adaptors/Balancer.jshttps://github.com/salboaie/SwarmESB/blob/master/adaptors/Worker.js

Swarms:https://github.com/salboaie/SwarmESB/blob/master/swarms/WorkerSwarm.jshttps://github.com/salboaie/SwarmESB/blob/master/swarms/WorkerManagement.jsTest:https://github.com/salboaie/SwarmESB/blob/master/test/LoadBalancerTest.js

Example