A NEW PLATFORM FOR A NEW ERA

© Copyright 2014 Pivotal. All rights reserved.© Copyright 2014 Pivotal. All rights reserved.

Data Microservices with Spring

2

Breaking the enterprise integration monolith

Fred Melo @fredmelo_br

© Copyright 2014 Pivotal. All rights reserved.© Copyright 2014 Pivotal. All rights reserved. 3

About Me

@fredmelo_br

Fred Melo

Mountain View, CA

https://www.linkedin.com/in/melofred

© Copyright 2014 Pivotal. All rights reserved.

© Copyright 2014 Pivotal. All rights reserved.

© Copyright 2014 Pivotal. All rights reserved.

User Mgmt

Order Mgmt

Shipping

Inventory

Payment

Payment

Order Mgmt

Inventory

User Mgmt

Shipping

Contract

Contract

Contract

ContractContract

Conway’s Law and Microservices

Going Agile - Breaking the monolith

Dev Team

© Copyright 2014 Pivotal. All rights reserved.

Deploy

> cf push

Cloud Controller

BBS (message bus)

RouterApp Deployed

Order Mgmt

Contract

Automating through the Platform

© Copyright 2014 Pivotal. All rights reserved.

Scale

Re-deploy

Migrate

Distribute

Upgrade UpdateOrder Mgmt

Contract

User Mgmt

Contract

Pivotal Cloud Foundry

© Copyright 2014 Pivotal. All rights reserved.

Integration Bus / ESB

System A

System BUser Mgmt

Order Mgmt

Shipping

Inventory

Payment

© Copyright 2014 Pivotal. All rights reserved.

Integration Bus / ESB

System A

System BOrder Mgmt

Contract

InventoryContract

ShippingContractUser

Mgmt

Contract

© Copyright 2014 Pivotal. All rights reserved.

Integration Bus / ESB

System A

System B

ETL

Data Warehouse

Order Mgmt

Contract

InventoryContract

ShippingContractUser

Mgmt

Contract

© Copyright 2014 Pivotal. All rights reserved.

Integration Bus / ESB

System A

System BOrder Mgmt

Contract

InventoryContract

ShippingContractUser

Mgmt

Contract

ETL

Data Warehouse

???

© Copyright 2014 Pivotal. All rights reserved.

Anatomy of a typical Integration Pipeline

Source

Pro

cess

ing

Ste

p

Pro

cess

ing

Ste

p

Pro

cess

ing

Ste

p

Pro

cess

ing

Ste

p

Data Data

Monolith

Destination

© Copyright 2014 Pivotal. All rights reserved.

Integration Pipeline

Scale

Re-deploy

Upgrade

Migrate

Distribute

Update

Source Destination

Pro

cess

ing

Ste

p

Pro

cess

ing

Ste

p

Pro

cess

ing

Ste

p

Pro

cess

ing

Ste

p

© Copyright 2014 Pivotal. All rights reserved.

Integration Pipeline

Source Destination

Pro

cess

ing

Ste

p

Pro

cess

ing

Ste

p

Pro

cess

ing

Ste

p

Pro

cess

ing

Ste

p

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

© Copyright 2014 Pivotal. All rights reserved.

Binding Abstraction Layer

Transport Options

Integration Pipeline

Data Messaging Microservices

Spring Cloud Stream

Data Data

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

© Copyright 2014 Pivotal. All rights reserved.

Backing Services

Transport Options

Pivotal Cloud Foundry

Auto Scaling

Auto Healing

Aggregated Logging

Integrated Metrics

Transport Transparency

Infrastructure Transparency

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

Data Pipeline Visual Design

Integrated Monitoring

© Copyright 2014 Pivotal. All rights reserved.

Backing Services

Transport Options

Pivotal Cloud Foundry

Auto Scaling

Auto Healing

Aggregated Logging

Integrated Metrics

Transport Transparency

Infrastructure Transparency

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

Data Pipeline Visual Design

Integrated Monitoring

© Copyright 2014 Pivotal. All rights reserved.

JSON Filter Transform Enrich

CustomHTTP

Sample pipeline

Deploy

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

Pivotal Cloud Foundry

Spring Cloud Data Flow

© Copyright 2014 Pivotal. All rights reserved.

DEMO

“Fast Data"

Machine Learning

Normal

High Risk

© Copyright 2014 Pivotal. All rights reserved.

Rest / HTTP

Transactions

Browser

Machine Learning

?GemFire

Greenplum

Analytics

© Copyright 2014 Pivotal. All rights reserved.

Machine Learning (Clustering)

GemFire

Greenplum

Analytics

Train

© Copyright 2014 Pivotal. All rights reserved.

Machine Learning (Clustering)

GemFire

Greenplum

Analytics

Traindistance from home location

tran

sact

ion

valu

e

Medium RiskLow Risk

High Risk

© Copyright 2014 Pivotal. All rights reserved.

Machine Learning (Clustering)

GemFire

Greenplum

Analytics

TrainPMML

+ Distance from home

ML Evaluation

high risk transactions

Suspects

© Copyright 2014 Pivotal. All rights reserved.

Rest / HTTP

Transactions

Browser

High Risk Alerts

GemFire

Greenplum

+ Distance from home

© Copyright 2014 Pivotal. All rights reserved.

Order Mgmt

Contract

InventoryContract

ShippingContract

PaymentsContract

Pivotal Cloud Foundry

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

SpringBoot app

Contract

Data Warehouse

PMML Model

External services

Actionable insightsRecommendation

Engine

Business Microservices Data Microservices

A NEW PLATFORM FOR A NEW ERA