20150609 - (libero events) microservices

7/25/2019 20150609 - (Libero Events) Microservices

1/125

@aahoogendoorn

DESIGNING, BUILDING, TESAND DEPLOYING MICROSERSander Hoogendoornditisagile.nlMentoring Consulting Training Agile Software architecture Code


2/125

MICROSERVICES? STAIRWAY TO HEAV2015 [email protected]

Sander Hoogendoorn

MeDadMentor, trainer, software architect, programmer

Books, articles, conferences

WorkOwner, ditisagile.nlCTO insurance company[PTO Capgemini][Global design authority agile Capgemini]

Webwww.sanderhoogendoorn.comwww.smartusecase.comwww.speedbird9.com@[email protected]


3/125



4/125


Agenda

An introduction into microservicesPros and cons of monolithic softwareSome principles of service orientation

Definitions of microservicesSome principles of microservicesPromises of microservicesChallenges in microservices

A microservice architectureEvolutionary architectureBuilding a landscape of small applications and servicesMicro-applicationsComponents and microservicesExamples of design patterns for microservicesPicking the best technology for every microservicePloyglot persistence


5/125


Agenda

How do microservices communicate with eachother?

Service interfacesSetting up communication between services

Communication via REST

Patterns in communication

Services and transactions

Designing microservicesFrom business needs and features to microservices

Modeling services

Smart use cases

Mapping domain driven design to microservices


6/125


Agenda

Deployment of microservices

The importance of the deployment pipeline

Setting up the deployment pipelineAgile, Kanban and microservices

Microservices and DevOps

Concluding

Some recommendations

Do microservices solve all challenges your ITdepartment has?

How to continue?


7/125

@aahoogendoorn

MONOLITHSHard to deliver, even harder to test and impossible to maintain


8/125


WHO OF YOU AND THAT YO


9/125


Monoliths

AdvantagesA single (layered) architecture

A single technology stackA single code base maintained by multiple teams

DisadvantagesAll parts are interconnectedMany other systems are connected to your systemHard to change, hard to maintain

Long time between releases, thereby increasing risksSlow innovation

Hard to move to newer technologiesDoesnt scale very well

ProduOrde

ProduAOrder


10/125


Dependencies will kill you


11/125

@aahoogendoorn

A BRIEF HISTORYOF COMPONENTS AND SERV


12/125


Client server


13/125


Component based development


14/125


Service oriented architecture


15/125


Microservices


16/125

@aahoogendoorn

MICROSERVICESBeyond the hype?


17/125

MICROSERVICES? STAIRWAY TO HEAV2015 ditisa

@aahoogendoornwww.ditisagile.nl

Microservices. Beyond the hype?


18/125



Gartner hype cycle


19/125

@aahoogendoorn

MICROSERVICESThe clear benefits


20/125



BUT FIRST DE


21/125



In short, the microservice architectueach running in its own process aThese services are built around busi

There is a bare minimum of centrali

and use different dataMartin Fowler


22/125



In short, the microservicearchitectural styleis an approach to deaeach running in its own process acommunicating with, ofteThese services arebuilt around business capabiliandindependently dby fully

There is a bare minimum of centrali

and usedifferent data stora.Martin Fowler

M li h S l bili


23/125



Monoliths. Scalability

Product Account

Order Customer

Product Account

Order Customer

Produ

Order

Mi i S l bilit


24/125



Microservices. Scalability

Product Account

Orderustomer

Mi i S l bilit


25/125



Microservices. Scalability

Product Account

Orderustomer

Product

Customerustomer Customer

AccounAc

Monoliths Persistence


26/125



Monoliths. Persistence

Product AccountOrder Customer

ProductsAccountsOrders Customers

Microservices Polyglot persistence


27/125



Microservices. Polyglot persistence

Product Account Ordustomer

MongoDBCustomers

MonOrd

Active DirectoryAccounts

OracleProducts

Microservices Promises


28/125



Microservices. Promises

Products not projects

Scalable

Decentralized governance

Replaceable parts

High performance

Technology independent

Polyglot persistence

Easy to build

Easy to test

Easier deployment than monoliths

Produc

Custo

Microservices But


29/125



Microservices. But

What is a microservice exactly?

How small is a microservice?

Requirements in a microservice world

Components or services

Who owns a microservice?

What technologies do you use?

What protocols do you apply?

How to define messages

How to test microservices

How to coordinate when business services runacross components?

How to build deployment pipelines?

Produc

Custo


30/125

@aahoogendoorn

ARE MICRO

A STAIRWAY TO


31/125

@aahoogendoorn

OR A HIGHWAY TO HELL?


32/125

@aahoogendoorn

ARCHITECTURE IN THEMICROSERVICE WORLD


33/125

MICROSERVICES? STAIRWAY TO HEAV

2015 ditisa


FOR THE THIBEFORE WEWE LEARNAristotle


34/125


2015 ditisa


MICROSERVICES

Evolutionary architecture


35/125


2015 ditisa


Evolutionary architecture

Microservices are autonomousServices need to be able to change independentlyChoose the right technology for the job

How many different stacks will you support?

New versions of a service can be deployed without hurtingthe big pictureBoundaries between services need to be really clear

Built around business capabilitiesAlways align to business goalsOptimize service autonomy, while keeping the bigger picturein mindHow to split up into zones or boxes?When to split or merge services

How free are individual service teams to decide ontechnology?Be liberal inside the boxes, but strict between them

Communication over simple protocols

APIs for services need to be well -defined

Multiple protocols make communication har

Choose the protocol that works for you (almREST)

How to design REST verbs and nouns, retur

Principles and practices

Set up a set of guiding principles

Create practices that support these principles

Create example implementations and service

Make it easy for the teams to do the right thi

In order to move to microservices, you will nbetter at architecture, testing, deployment thright now


36/125



MICROSERVIArchitecture at a Dutc

A major Dutch insurance company


37/125



A major Dutch insurance company

We haveMost functionality on an expensive mainframeA wide variety of large systems written in Java that are hard

to maintain and to test, and that are very hard to replaceIndividual systems that cover large areas of functionality,usually coupled to departmentsAging technology

No mobile strategy, allowing for new business or newservices to clients, and intermediaries

We need toGet rid of the mainframeShorten time-to-market

Lower TCOUphold a fully secure systems landscape

Questions, questions, questions


38/125



Q , q , q

Communication archHow do we define interfaces betwe

How do we glue together rapid

Application architEnd user facing Different user

Which technology is the bes

Component architeComponents and services are evolviHow do we deal with versioning? H

Our guiding principles


39/125



g g p p

We decided to go from hereClient thinks in business processes, so we implementbusiness processesWe move away from the mainframe, to a new systemslandscape, consisting of micro-applications and micro-componentsRequirements and documentation are modeled rather thanwrittenApplications implement a single (elementary) businessprocessComponents serve a single purpose and offer servicesApplications and components all have their own boundedcontext a domain modelApplications and components will have an similar internalsoftware architecture to facilitate ease of maintenance andallow for harvesting re-useCommunication between applications and components willuse a simple open protocol - REST


40/125

@aahoogendoorn

BUSINESS PROCESSES FIRST

Different levels of processes (and requirements)


41/125



p ( q )

A high-level business process


42/125



g p

Step 1 Step 2 Step 3

Split into work processes


43/125



Step 3.1 Step 3.2 SteStep 3

Split into elementary processes - OTOPOP


44/125



Step 3.3.1 Step 3.3.2 SteStep 3.3

S


45/125

@aahoogendoorn

APPLICATIONS

Micro-applications


46/125



Our principlesEach application serves a single purpose

It mostly targets a single audience, either client,

intermediaries, in-house or third partiesEach application implements a single processApplications are always human facing, they have agraphical user interfaceThese user interfaces can be responsive web, nativedevice or even desktop (not preferred) depending onthe target audienceEach application has a bounded contextOur current stack uses HTML5, JavaScript (client-side),Bootstrap (responsive UI) and JSF (server-side)

Applications do not communicate with storage

P i


47/125



Presentation

ProcessDomain

ServicesOutside world

DomEnums /

SI

elations Dossiers Intermediaries A


48/125

@aahoogendoorn

COMPONENTS

Components


49/125



Our principles

Components are the workers of our softwarearchitecture

Components are fine-grained and target at serving asingle business purpose, such as Accounts, Relationsor Rates

Components are designed following the SingleResponsibility Principle (SRP).

All components have a bounded context a domain

modelEach component exposes a set of services to thelandscape, which exposes representations of itsbounded context

Components can interact with their own storage

S i i t f


50/125



Service interface

ProcessDomain

Data / ServicesOutside world

DomEnums /

StI

elations Dossiers IntermediariesDB2 MongoDB


51/125

@aahoogendoorn

AND THE REST ISCOMMUNICATION

Communication


52/125



Our principles

Communication between applications and componentsneeds to be easy to develop, secure and fast

Communication will leverage simple, scalable and safeweb protocols REST over HTTP

Representations from a components bounded contextcan be passed in the form of JSON objects

Representations offered by producers do notnecessarily have the same structure as required by

their consumersMapping or wrapping is inevitable

Be aware that neither REST nor JSON is really fullystandardized


53/125

@aahoogendoorn

DESIGNING MICROSERVICE


54/125



DOING BIG UDOING NO DDave Thomas

Design guidelines


55/125



Single Responsibility Principle (SRP)

Group together things that change together

Separate things that change for different reason

Microservices size

The smaller your services get, the more extreme youexperience both the benefits and the drawbacks

A team should be able to rebuild a service in twoweeks

Different types of componentsApplications, tasks

Aggregates, references, mediators

Bounded contexts

Produc

Custo

Single responsibility principle (SRP)


56/125



SOLIDSingle Responsibility Principle

Open Closed Principle

Liskov Substituion Principle

Interface Segregation Principle

Dependency Inversion Principle

Single Responsibility PrincipleEvery module should have responsibility over a single part of thefunctionality provided by the software,

That responsibility should be entirely encapsulated by the class

All its services should be narrowly aligned with that responsibility

ThereforeGroup together things that change together

Separate things that change for different reason

Bounded context


57/125



Domain driven designThe paradigm of designing software based on modelsof the underlying domain

The domain model helps the business and thedevelopers to reason about the functionality

A model needs to be unified internally consistentwithout contradictions

Bounded contextThe bounded context is a central pattern in domaindriven design

When you model larger domains, it becomesprogressively harder to create this single unified modelSo, instead of creating a single unified model, youcreate several, all valid within their bounded context

The single unified domain model


58/125



ProductVendo

StockrderClient

Delivery

Payment

Bounded contexts


59/125



ProductV

SOrder

ClientDelivery

Payment

Product

Bounded contexts


60/125



ModelingModeled in a (single) domain model

Model contains entities, domain objects

Usually evolves around a single main entity (aggregate root)

ServicesInterrogate the bounded context

Resources usually follow the aggregate root

Services post and put representations

Representations are mapped to the bounded context

ValidationRepresentations can be validated before being mapped

Bounded context can be validated as a whole e.g. before storing

Business rules

Properties and property types

Produ

Properties and property types


61/125



Basic typesstring, integer, date, datetimeInclude nullable wrapping

EnumerationsSet up at design time, unchangeable at run-timeGenders, Categories

Value objectsNo specific instancesIsbn, Email, Url, Money

References (code tables)Changeable at run-time, such as contract types

AssociationsTo cached entities such as Country, NationalityTo first level citizens such as Customer, Product


62/125

@aahoogendoorn

MODELING

APPLICATIONS

Back to the elementary processes


63/125



Step 3.3.1 Step 3.3.2 SteStep 3.3

S

Smart use cases


64/125



Modeling applications

Each elementary process is implemented in asingle application

The requirements are modeled using smart usecases

Each application consists of a single sea-level usecase and a number of fish-level use cases

Additionally we add the services that are requiredto implement the applications to the model

Doing so, we can easily do impact mapping on ourservices

Also, the smart use cases form a strongfoundation for integration testing

Different levels of use cases


65/125



Traditionaluse cases

Smartuse cases

Format Textual Visual

Granularity Different Unified

Estimate Hard Easy

Unit of work Lousy Good

Reuse Incidental Normal

Traceability Possible Normal

Testability Poor Good

Smart use cases


66/125



Wireframes with use cases


67/125



Applicationbounded context


68/125




69/125

@aahoogendoorn

AND THE REST ISCOMMUNICATION (OVER HT

HTTPHTTP


70/125



HTTPWhen using HTTP as your transfer protocol you will have tobe very precise about how you use itConsider HTTP a resource (collection) based protocol

Start from host and port, possibly add component to URIsMake use of HTTP verbs rather than descriptive URIsAgree on a limited set of return codes in your communicationbetween servicesAgree when to use which return code

VerbsGETPOSTPUTDELETE

HTTP RETUR


71/125



HTTP RETUR1**. Hold on2**. Here you3**. Go awa

4**. You fuck5**. I fucked

HTTP GETGET E l


72/125



GET

Retrieve whatever information isidentified by the request URI in the formof an entity

The entity is usually a single or a list ofobjects (of the type provided by theservice, often JSON, or XML)

GET is safe (retrieval only) andidempotent

Unfortunately there are many ways ofcreating GET requests (see examplesbelow)

Returns 200 (found), possibly 400 (badrequest) or 404 (not found)

Examples

Get an entire collectionlocalhost:8080/countries

Find objects in the collectionlocalhost:8080/countries?name

Find an object in the collection by IDlocalhost:8080/countries/38

Find a sub-object in the collection by IDlocalhost:8080/countries/38/capital

Find object in the collection by ISOlocalhost:8080/countries?isocode

Find object in the collection by ISOlocalhost:8080/countries/isocode/GRC

Find object in the collection by ISOlocalhost:8080/countries/GRC

HTTP POSTPOST E l


73/125



POST

Request that the server accepts theentity enclosed in the request as a newsubordinate of the resource identified bythe request URIThe posted entity is subordinate to thatURI in the same way that a file issubordinate to a directory containing it

Returns the location header for the newresource, possibly with created entity in

the bodyReturns 201 (created), possibly 500(server error)

Examples

Post to the collection (with entity in localhost:8080/countries

Returning location headerlocalhost:8080/countries/38

HTTP PUTPUT E l


74/125



PUT

Requests that the enclosed entity isstored under the supplied request URI

If the request URI refers to an existingresource, the enclosed entity is anmodified version of the resource

POST identifies the resource that willhandle the enclosed entity

PUT identifies the entity enclosed withthe request

Returns 200 or 204 (when resource ismodified), possibly 201 (if a resource iscreated), possibly 500 (server error)

Examples

Put with ID (with entity in body)localhost:8080/countries/38

Update capital (with entity in body)localhost:8080/countries/38/capital

HTTP DELETEDELETE Examples


75/125



DELETE

Requests that the resource identified bythe request URI is deleted.

Returns 200 or 204 (when resource isdeleted), possibly 500 (server error)

Examples

Deletelocalhost:8080/countries/38


76/125



BE CONSERVBE LIBERALPostels Law


77/125

@aahoogendoorn

MODELING COMPONENTS

Modeling resourcesWhy?


78/125



Why?

Services need to be able to change independently, so APIsneed to be well-defined

Allows for testing of your resources

How?

Model resources and representations

Start with the root resource

Add the valid HTTP methods for that resource

Follow paths to sub-resources

Add the valid HTTP methods for that resource

Add the representations that are expected and will bereturned (often in JSON)

Representations reflect your components bounded context

Modeling resourcesRoot resources (component)


79/125



( p )/qa

/questionnairesGET

Questionnaire

Id,Name, Description

/qa

GET the collection, but only lim

representation (but with location/qa/questionnaires

/questionnaires

GET

/{id}QuestionnaireId,Name, Description

QuestionType, Name, Description

AnswerName, Value

GET a single item from the cowith representation/qa/questionnaires/334532

Resourcemodel


80/125



Service use cases


81/125



Componentbounded context


82/125




83/125

@aahoogendoorn

TESTING MICROSERVICES

Testing microservicesWhy is it even more important?


84/125



The architecture evolves

We would like to use different technology stacks

We want to be able to replace servicesServices versionReleases become much, much shorter

Integration becomes much harder in a microserviceslandscapeBottom line: testing microservices is much morecomplex than testing a monolith

Next questionsTest when?Test what?

A service development lifecycle


85/125



Code Developer Test TestPrepare & Design

What to test


86/125




DevelopersUnit tests

DevelopersQ A

TestersScenarios & PIs


ProduPro

What to test automated


87/125




DevelopersUnit tests

DevelopersQ A



ProduPro

Developer Q&A Sonar


88/125



Developer Q&A Sonar


89/125



Developer Q&A Sonar


90/125



Developer Q&A Sonar


91/125



Presentation IntegraBDD


92/125



Process

Domain

Services

Outside world

DomEnums /

S

Ielations Dossiers Intermediaries A

Unit tests

Unit tests

Unit tests

Service interface IntegratSOAP U


93/125



Process

Domain

Data / Services

Outside world

DomEnums /

St

Ielations Dossiers IntermediariesDB2 MongoDB

SOAP U

Unit testsUnit tests

Unit tests

Integration tests


94/125




95/125

@aahoogendoorn

DEPLOYING MICROSERVICEContinuous integration, build pipelines and continuous delivery

ContinuousContinuous integration (CI)


96/125



Mechanism for making sure new code is in sync withexisting code

Developers check-in new code only after merging itlocally with current trunk (or branch)It is good practice to have unit test to validate merge

Get feedback from tests as soon as possible, alreadyduring development

Continuous deployment (CD)

Each check-in becomes a potential release candidateModel the build pipeline

Integration tests provide feedback as soon as possible,immediately after moving to the next environment

A typical build pipeline


97/125




A typical build pipeline


98/125



ccepntegrationestevelopment


Build pipelines in Jenkins


99/125



Microservices. Building a deployment pipeline

Code Developer Test Test A


100/125




p



Microservices. Pipeline hell?

Code Developer Test Test


101/125



Code v.2 Developer Test v.2

p

Test v.2 Acceptance Test v.2 Accep

Developer Test Test Acceptance

Code

Live

Continuous delivery and microservicesVersioning

Services version


102/125



Services version

Allow for clear semantic versioning, such asMAJOR.MINOR.PATCH

Breaking changes

Try to avoid breaking changes

If there are breaking changes, make sure they appear assoon as possible

Automated test should signal breaking changes

Team should negotiate

Build pipelines

Create a build pipeline per application and component

In production consider Platform-as-a-Service (e.g. Docker)

Dealing with breaking changes


103/125



Customer

v1

App1 App2

Customer

v1

App1 App2

v2

Customer

v1

App1 App2

v2

C

Ap


104/125

@aahoogendoorn

DO WE REALLY NEED PROJFrom projects to releases to continuous delivery

Do we really need projects?


105/125



Planning


106/125



Or roadmap


107/125



Visualise your flow


108/125



And go with the flow


109/125



Minimal viable product


110/125



M i t

From projects to continuous delivery?

P j t


111/125



MaintenanceProject

MaintenanceVP

Maintenanceontinuous delivery

Do we really need projects?


112/125




113/125

@aahoogendoorn


114/125



WORDLYWISDOMTO FAIL CONVENTJohn Maynard Ke


115/125

@aahoogendoorn

IN RETROSPECTIVE?

Work from roadmaps


116/125

@aahoogendoornSCALING AGIL

2015 ditisa


117/125

@aahoogendoorn

GO WITH THE FLO

Minimal viable product


118/125




119/125

@aahoogendoorn

FIRST DO IT

THEN DO ITTHEN DO IT


120/125

@aahoogendoorn

ALLOW THE TEAM TO LEARNCONTINUOUSLY


121/125

@aahoogendoorn

COMMUN

VIA RESTAS EASY AS IT P

The hockey stick model


122/125



AND HAV


123/125

@aahoogendoorn

THIS IS


124/125

@aahoogendoorn

AGILEwww.createspace.com/47

Password: agilescrum

Discount code: KGNWKK


125/125

@aahoogendoorn

www.sanderhoogendoowww.smartusecase.comwww.speedbird9.com

[email protected]

@aahoogendoorn

REFERENCESAND QUESTIONS

20150609 - (libero events) microservices

Documents