microservices, yet another architectural style?

Tom De Wolf Technical Lead

tom.dewolf@aca-it.be

Stijn Van den Enden CTO

stijn.vandenenden@aca-it.be

www.aca-it.be

Thomas Borghs Solution Engineer

thomas.borghs@aca-it.be

MICRO SERVICESYet another architectural style?

Concepts - What are Micro services?

Patterns - How to solve the challenges?

Technology - Using what?

The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and

communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by

fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming

languages and use different data storage technologies. !

Martin Fowler

SERVICE ORIENTED ARCHITECTURE?Yes, it’s SOA … but different implementation approach:

Classic SOA integrates different applications as a set of services

Microservices architect a single application as a set of services

Classic SOA integrates different applications as a set of services

Microservices

Enterprise Service Bus

WS* WS* WS* WS* WS*

WS* WS* WS* WS* WS*

Workflow Engine

Intelligence

Orchestration

business platform

integrates different applications as a set of services

Microservices architect a single application as a set of services

accounting service contract

service

ordering service

logistics service

prospects service

capability X service

capability Y service

external integrationsbackends

{ API } { API }{ API }

{ API } { API }

{ API }{ API }

{ API } { API }

{ API } { API }

Classic SOA integrates different applications as a set of services

Microservices architect a single application as a set of services

Typical implementation solution differs!

Heavy-weight

ESBWS*/SOAP

Orchestration

License-drivenTarget problem:

Integrate (Legacy) Software

Intelligent Communication Layer

Light-weight

HTTP/REST/JSON

Choreography

Target problem: Architect new Business Platform

Dumb Communication Layer

Intelligent Services

WHY

SOAWHY - DIFFERENTIATE FROM SOA

App Server

WAR/EAR

Ordering

Inventory

Billing

UI

WHY - THE CURSE OF THE MONOLITH

Database

Simple to scaleSimple to develop Simple to deploy

WHY - THE CURSE OF THE MONOLITHApp Server

WAR/EAR

Ordering

Inventory

Billing

UI

Large Code Intimidates DevelopersHard to understand

and modify

Development slows down

Overloaded IDE

Overloaded web container

WHY - THE CURSE OF THE MONOLITH

Small Change - Big ImpactAny change requires

full rebuild, test and deploy

Impact analysis is huge effort and takes long

Obstacle for frequent changes and deployments

App Server

WAR/EAR

Ordering

Inventory

Billing

UI

WHY - THE CURSE OF THE MONOLITH

Big Risk for Re-Write

App Server

WAR/EAR

Ordering

Inventory

Billing

UI

No hard module boundaries quality and modularity breaks down over time this enforces eventual need for re-write

Re-write = complete re-write no partial re-write

Long term commitment to technology stack change or try-out new technology implies re-write

WHY - THE CURSE OF THE MONOLITHApp Server

WAR/EAR

Ordering

Inventory

Billing

UI Failure in monolith brings it down

Little Resilience to Failure

WHY - THE CURSE OF THE MONOLITHApp Server

WAR/EAR

Ordering

Inventory

Billing

UI

Mostly Horizontal scaling many load balanced instances

Scaling can be difficult

Hard to scale to data growth cope with all data

Different components have different resource needs

Scaling development implies coordination overhead

WHY - TYPES OF SCALING

App Server

WAR/EAR

Ordering

Inventory

Billing

UI

Database All data

App Server

WAR/EAR

Ordering

Inventory

Billing

UI

App Server

WAR/EAR

Ordering

Inventory

Billing

UI

Horizontal Scaling(monolith)

Vertical Scaling

App Server

WAR/EAR

Ordering

Inventory

Billing

UI

App Server

WAR/EAR

Ordering

Inventory

Billing

UI

Database All data

(monolith)Data Scaling

App Server

WAR/EAR

Ordering

Inventory

Billing

UI

Database segment

App Server

WAR/EAR

Ordering

Inventory

Billing

UI

App Server

WAR/EAR

Ordering

Inventory

Billing

UI

Database segment

Database segment

(monolith)

WHY - TYPES OF SCALING

Database Ordering

Container

UI

Functional Scaling

Container

Ordering

Container

Inventory

Container

Billing

Container

Ordering

Container

Ordering

Container

Billing

Database Inventory Database

Billing

(micro-services)Team Scaling

Container

Ordering

Container

Inventory

Container

Billing

(micro-services)

balances application and development complexity

• Small and focussed on 1 capability • easier to understand • IDE and deployment faster for 1 service

• Independent • Release and deployment • Scaling • Development

• Loosely Coupled • through lightweight communication

• Fault Isolation vs bring all down.

• Allows try-out of new technologies.

• Re-write can be limited to 1 service • Impact Analysis stops at boundary

• Provide firm module boundaries with explicit interface! • Less risk on re-write • Harder to violate boundary in development

• Decentralised choreography • vs central orchestration • vs central point of failure

• Decentralised data • polyglot persistence

WHY - ARCHITECTURAL BENEFITS

WHY - EVOLUTIONARY ARCHITECTURE

1 - Key (business) drivers guide architectural decisions

2 - Postpone decisions to Last Responsible Moment

3 - Architect and develop for Evolvability

Micro-services are organised around business capabilities

Micro-services allow delay of scaling and technological decisions

Micro-services support evolution in technology, scaling, and features

HOW TO

HOW TO

Approach - Key to success

Challenges - And ways to overcome them

Functional decomposition of the business domain

Functional decomposition of the business domain

Software Design Customer Satisfaction

Separation of ConcernsLow Coupling, High Cohesion

Reduce Impact by Encapsulating Source of Change

Predictable Cost of ChangeChanges are Business Driven

Source of Change = Business

Functional Modularisation

B AB

A

Functional decomposition of the business domain

• Change A impacts all modules = costly • Change B requires split of module = costly

• Change A only impacts other module if api change • Change B limited to module

side note: Domain Driven Design

“In order to create good software, you have to know what that software is all about. You cannot create a banking software system unless you have a good understanding

of what banking is all about, one must understand the domain of banking.”

From: Domain Driven Design by Eric Evans.

Tackling complexity by abstracting the business domain concepts and logic into a domain model and using this as a base for software development

Domain driven design deals with large complex models by dividing them into different functionally bounded subdomains and the explicitly describing the

interrelations between these subdomains.

Bounded contexts

Functional decomposition of the business domain

ONLINE STORE

Ordering Billing

Inventory Accounting

Functional decomposition of the business domain

AccountingInventory

BillingOrdering

customer

Invoice

balance

order

item

item

stock order

order

item

incoming cash

outgoing cash

stock

Benefits of functional decomposition

AccountingInventory

BillingOrdering

customer

Invoice

balance

order

item

item

stock order

order

item

incoming cash

outgoing cash

stock

Benefits of functional decomposition

Applying services to bounded contexts

Accounting ServiceInventory Service

Billing ServiceOrdering Service

customer

Invoice

balance

order

item

item

stock order

order

item

incoming cash

outgoing cash

stock

AccountingInventory

BillingOrdering

customer

Invoice

balance

order

item

item

stock order

order

item

incoming cash

outgoing cash

stock

Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.

!

-- Melvyn Conway, 1967

Side note: Conway’s Law

Side note: Conway's Law

Technology-based Team Composition Functional Team Composition

Ordering

InventoryBilling

Accounting

Container

Ordering

Container

Billing

Container

Inventory

Container

Accounting

UI

UI

Leads to higher coupling between services

PITFALL: Incorrect functional decomposition

- harder to functionally scale the application - errors will propagate through multiple services - graceful degradation will be harder to achieve

- team development overhead - T-scaling becomes harder

- large communicational overhead between services - large overhead in releasing/deploying business features - refactoring to correct decomposition is costly

most of the microservices architectural benefits are lost

PITFALL: Incorrect functional decomposition

Safer to start with a functionally well decomposed monolith and evolve it to a microservices architecture when the need arises

- rewriting failure scenario’s - decentralised data - service contract redesign

Refactoring the initial functional decomposition will be easier in a monolith

Approach - Key to success

Challenges - And ways to overcome them

- Keep Releases and deployments manageable - high level of automation is needed

- Service monitoring is required - Configuration management becomes more complex

CHALLENGE #1: Operational Complexity

Complex Runtime: many moving parts

- distributed architectural properties to consider: - decentralised data - communication between services - handling failures of components

- testing effort becomes greater

CHALLENGE #2: Distributed Development

Services are deployed on multiple instances

Decentralised data

Each service has its own database - loose coupling Might even be in another database technology

Data duplication between services might be required to ensure loose coupling

When implementing use cases spanning multiple services: distributed transactions vs eventual consistency

Distributed Transactions vs Eventual Consistency

DISTRIBUTED TRANSACTIONS

- data is always consistent !

- reduces system availability - services are more tightly coupled

- has fallen out of favor in modern stacks (REST, NoSQL)

Distributed Transactions vs Eventual Consistency

EVENT-DRIVEN ASYNCHRONOUS UPDATES

- use a message broker to publish use cases to other services - decouples producers and consumers (services) of events - improves availability

- tradeoff between availability and data consistenty - application needs to be able to handle eventually consistent data

Communication between services

Use cases can span multiple services What type of communication is best used to implement such a use case?

Network properties need to be taken into account

What type of Communication?

SYNCHRONOUS HTTP-BASED

- easy - firewall-friendly, works across the internet

!

- doesn’t support publisher-subscriber patterns - client and server must both be available simultaneously - client needs to know host and port of server

What type of Communication?

ASYNCHRONOUS NON-BLOCKING

- Client doesn’t block calling thread - allows for parallelism

!

- client and server still must both be available simultaneously - client still needs to know host and port of server

What type of Communication?

ASYNCHRONOUS MESSAGING

- For example through a Broker - decouples message producers from consumers - broker can buffer messages - supports a variety of communication patterns

!

- broker is another moving part - adds complexity

- request-reply communication pattern is not a natural fit

What type of Communication?

Accounting ServiceInventory Service

Billing ServiceOrdering Service

Use case: New Order Received

New order Create invoice

Update Incoming CashflowUpdate Stock

Handling Failures

Services can fail at any moment

Design services to handle these kind of failures

Reactive systems are: Responsive

Resilient Elastic

Message Driven !

-- Reactive Manifesto: september 16 2014

Side note: Reactive Programming

Handling Failures

FALLBACK MESSAGE QUEUE

Ordering Service Billing Servicenew order received failure

Fallback queue

Handling Failures

PER-SERVICE THREAD POOLS

Ordering Service Billing Servicenew order received

Thread pool 10 threads

- communication between services is reduced - when functional decomposition is done right - when service size isn’t too small

- reduce communication between clients and Services with an API gateway - executing service calls in parallel reduces impact of communication overhead - reduce unneeded network usage by using circuit breakers

CHALLENGE #3 Minimising Communicational Overhead

Avoid Chatty Communication

Minimising Communicational Overhead

API GATEWAY

Ordering

Inventory

Billing

Accounting

Monolith Micro Services

Container

Ordering

Container

Inventory

Container

Billing

Container

Accounting

Desktop client

Mobile client

Desktop client

Mobile client

Minimising Communicational Overhead

API GATEWAY

Container

Ordering

Container

Inventory

Container

Billing

Container

Accounting

Desktop client

Mobile client

Api gateway

Handling Failures in Communication

CIRCUIT BREAKER

- Wrap a protected function in a circuit breaker - Monitor protected function for failures - The circuit breaks when a predefined threshold of fails is reached - All future calls to the function go to fallback until the circuit is restored

Handling Failures in Communication

CIRCUIT BREAKER

Ordering Service Billing Servicenew order received

Circuit breaker Threshold = 10

Fallback queue

TECHNOLOGY

Microservice Implementation Stack

DROPWIZARD Make features not WAR

DROPW

IZARD

guava

jackson

metrics

Validator

YAML

JDBI

core

migrations

hibernate

jdbi…

java -jar ./target/profileservice-0.1.0-SNAPSHOT.jar server ./src/main/resources/userprofileservice.yml 

SPRING BOOT opinionated view of building production-ready Spring applications

•Convention over configuration approach

•Deployable as a Self-contained jar or war

•Tackles dependency-hell via pre-packaging

•Support for monitoring and metrics (actuator module)

Deployment

Compute, Storage, Network

Host OS

Hypervisor

VM1 VM2

MicroService MicroService

Guest OS

JVM

Guest OS

JVM

VM’s abstract underlying hardware, but limit resource utilisation

Compute, Storage, Network

Host OS

container1

container2

container3

container4

JVM JVM JVM

MicroService MicroService MicroService

JVM

MicroService

Containers have own isolated resources

Static website Web frontend User DB Queue Analytics DB

Development VM

QA server Public Cloud Contributor’s laptop

DOCKER IS A SHIPPING CONTAINER SYSTEM FOR CODE M

ultip

licity

of S

tack

sM

ultip

licity

of

hard

war

e en

viro

nmen

ts

Production Cluster

Customer Data Center

Do services and apps interact

appropriately?

Can I m

igrate sm

oothly and quickly

…that can be manipulated using standard operations and run consistently on virtually any hardware platform

An engine that enables any payload to be encapsulated as a lightweight, portable, self-sufficient container…

Static website Web frontend User DB Queue Analytics DB

Development VM

QA server Public Cloud Contributor’s laptop

DOCKER IS A SHIPPING CONTAINER SYSTEM FOR CODE M

ultip

licity

of S

tack

sM

ultip

licity

of

hard

war

e en

viro

nmen

ts

Production Cluster

Customer Data Center

Do services and apps interact

appropriately?

Can I m

igrate sm

oothly and quickly

Operator: Configure Once, Run Anything

Developer: Build Once, Run Anywhere

Static website

Web frontend

Background workers

User DB

Analytics DB

Queue

Development VM QA Server Single Prod

Server Onsite Cluster Public Cloud Contributor’s laptop

Customer Servers

DOCKER SOLVES THE NXN PROBLEM

• Isolation

• namespace

• pid mnt net uts ipc user

• resource usage

• (CPU, memory, disk I/O, etc.)

• Limited impact on Performance - http://ibm.co/V55Otq

• Daemon and CLICompute, Storage, Network

Host OS

container1

container2

container3

container4

JVM JVM JVM

MicroService MicroService MicroService

JVM

MicroService

Source Control System

DockerFile

Continuous Integration Infrastructure

Container Image Repository

Compute, Storage, Network

Host OS

daemon

container1

JVM

MicroService

pull

Slave Node

Host OS

push

build

provision

container1

JVM

MicroService

https://github.com/spotify/helios

http://mesos.apache.org/

https://github.com/openshift/geard

…

PublicHybridPrivate

Compute, Storage, Network

Host OS

Hypervisor

VM1 VM2

MicroService MicroService

Guest OS

JVM

Guest OS

JVM

Compute, Storage, Network

Host OS

container1

container2

container3

container4

JVM JVM JVM

MicroService MicroService MicroService

JVM

MicroService

VM’s abstract underlying hardware, but limit resource utilisation

Containers have own isolated resources

OSGi Runtime

JVM

MicroService

Compute, Storage, Network

Host OS

MicroService MicroService

Microservice run in their own isolated classloader and standbox in the JVM

Balancing Load

Static

Dynamic

Loadbalancer Loadbalancer

MicroService MicroService MicroService MicroService MicroService

Web Front End

Web Front End

Web Front End

Web Front End

Web Front End

MicroService MicroService MicroService MicroService MicroService

A

B

Midtier Service Registry

MicroService

register

renew

get registry

Static

Dynamic

Loadbalancer Loadbalancer

MicroService MicroService MicroService MicroService MicroService

Web Front End

Web Front End

Web Front End

Web Front End

Web Front End

MicroService MicroService MicroService MicroService MicroService

A

B

Midtier Service Registry

MicroService

register

renew

get registry

eureka

ribbon

https://github.com/Netflix/eureka

https://github.com/Netflix/ribbon

The Story

* based on Functional Programming the Netflix API - Ben Christensen

Netflix API

Dependency A

Dependency D

Dependency G

Dependency J

Dependency M

Dependency P

Dependency B

Dependency E

Dependency H

Dependency K

Dependency N

Dependency Q

Dependency C

Dependency F

Dependency I

Dependency L

Dependency O

Dependency R

onsdag den 6. marts 13

* based on Functional Programming the Netflix API - Ben Christensen

Discovery of Rx began with a re-architecture ...

onsdag den 6. marts 13

* based on Functional Programming the Netflix API - Ben Christensen

... that collapsed network traffic into coarse API calls ...

onsdag den 6. marts 13

* based on Functional Programming the Netflix API - Ben Christensen

Iterablepull

Observablepush

T next()throws Exception

returns;

onNext(T)onError(Exception)onCompleted()

!//"Iterable<String>"!//"that"contains"75"Strings!getDataFromLocalMemory()!!.skip(10)!!.take(5)!!.map({!s!%>!!!!return!s!+!"_transformed"})!!.forEach(.....{!println!"next!=>!"!+!it})

!//"Observable<String>"!//"that"emits"75"Strings!getDataFromNetwork()!!.skip(10)!!.take(5)!!.map({!s!%>!!!!return!s!+!"_transformed"})!!.subscribe(.....{!println!"onNext!=>!"!+!it})

onsdag den 6. marts 13

* based on Functional Programming the Netflix API - Ben Christensen

onsdag den 6. marts 13

* based on Functional Programming the Netflix API - Ben Christensen

onsdag den 6. marts 13

* based on Functional Programming the Netflix API - Ben Christensen

+ =

https://github.com/Netflix/Hystrix

HystrixCommand run()

getFallback()

run()

failure/circuit open

public class GetUserPerferencesCommand extends HystrixCommand<UserPreferences> {!// ..//!! @Override! protected UserPreferences run() {! ! ! // call the UserPreferencesService ! }!! @Override! protected UserPreferences getFallback() {! !! return UserPreference.empty();! }!}

Monitoring

MicroService MicroService MicroService MicroService MicroServiceA

Logstash

Log Aggregation

Graphite

Metrics and Monitoring

- Correct Functional decomposition is crucial - reflect it in a organisational structure (Conway’s law) - pretty hard to get right from the start

- A modular system can evolve to microservices - balance the needs (advantages) with the costs (tradeoffs)

Conclusion

Are they here to stay?who can tell? but the monolith is dead