@crichardson

Handling Eventual Consistency in JVM Microservices with Event

Sourcing

Chris Richardson Founder of Eventuate.io

@crichardson

http://eventuate.io

Copyright © 2015. Chris Richardson Consulting, Inc. All rights reserved

Kenny Bastani Spring Developer Advocate

@kennybastani

http://pivotal.io

@crichardson

Presentation goal

Show how event sourcing is a great foundation for a microservice

architecture

@crichardson

About Chris

http://learnmicroservices.io

@crichardson

About Kenny

@kennybastani

@crichardson

Agenda

Overview of event sourcing

The problem with microservices, transactions and queries

Using events to maintain consistency

Event sourcing in a microservice architecture

Implementing queries with CQRS

@crichardson

Traditional persistence

Order

id state ….

101 ACCEPTED

Order table

…

ID STATE …

102 … …

@crichardson

But how did we get here?

Who did what and when?

What was the state of the Order last Monday at 3:01pm?

@crichardson

Order example History

@crichardson

Task example State

HistoryAudit

@crichardson

Usually auditing, history and temporal

queries is additional code and/or

an after-thought

@crichardson

Event sourcing

For each domain object (i.e. DDD aggregate):

Identify (state changing) domain events, e.g. use Event Storming

Define Event classes

For example, Order events: OrderCreated, OrderCancelled, OrderApproved, OrderRejected, OrderShipped

@crichardson

Persists events NOT current state

Event table

Entity type Event id

Entity id

Event data

Order 902101 …OrderApproved

Order 903101 …OrderShipped

Event type

Order 901101 …OrderCreated

@crichardson

Replay events to recreate state

Order

state

OrderCreated(…) OrderAccepted(…) OrderShipped(…)

Events

Periodically snapshot to avoid loading all events

@crichardson

Benefits of event sourcing

Reifies state changes:

Built in, reliable audit log

temporal queries

Preserved history ⇒ More easily implement future requirements

Eliminates O/R mapping problem (mostly)

Reliable event publishing: publishes events needed by predictive analytics etc, user notifications,…

@crichardson

Drawbacks of event sourcing

Requires application rewrite

Weird and unfamiliar style of programming

Events live forever ⇒ carefully evolve schema

Querying the event store can be challenging

Current state is no longer directly available

Often need to maintain views for efficiency

@crichardson

Demo

@crichardson

Agenda

Overview of event sourcing

The problem with microservices, transactions and queries

Using events to maintain consistency

Event sourcing in a microservice architecture

Implementing queries with CQRS

@crichardson

The Microservice architecture tackles complexity through

modularization

@crichardson

Microservice architecture

Browser

Mobile Device

Store Front UI

API Gateway

Catalog Service

Review Service

Order Service

… Service

Catalog Database

Review Database

Order Database

… Database

HTML

REST

REST

Apply X-axis and Z-axis scaling to each service

independently

@crichardson

But there are challenges implementing

transactions and queries

@crichardson

ACID transactions cannot be usedBEGIN TRANSACTION … SELECT ORDER_TOTAL FROM ORDERS WHERE CUSTOMER_ID = ? … SELECT CREDIT_LIMIT FROM CUSTOMERS WHERE CUSTOMER_ID = ? … INSERT INTO ORDERS … … COMMIT TRANSACTION

Private to the Order Service

Private to the Customer Service

Requires 2PC

@crichardson

2PC is not a viable option

Guarantees consistency

BUT

2PC is best avoided

Not supported by many NoSQL databases etc.

CAP theorem ⇒ 2PC impacts availability

….

@crichardson

Queries can’t use joins

SELECT * FROM CUSTOMER c, ORDER o WHERE c.id = o.ID AND o.ORDER_TOTAL > 100000 AND o.STATE = 'SHIPPED' AND c.CREATION_DATE > ?

Customer Service

Order Service

@crichardson

Agenda

Overview of event sourcing

The problem with microservices, transactions and queries

Using events to maintain consistency

Event sourcing in a microservice architecture

Implementing queries with CQRS

@crichardson

Event-driven architecture

@crichardson

Order ManagementOrder

id : 4567 total: 343 state = CREATED

Customer Management

Customer creditLimit : 12000 creditReservations: {}

Customer creditLimit : 12000 creditReservations: { 4567 -> 343}

Order id : 4567 total: 343 state = APPROVED

Eventually consistent credit checking

Message Bus

createOrder()

Publishes:Subscribes to:

Subscribes to:

publishes:

OrderCreatedEvent

CreditReservedEvent

OrderCreatedEvent CreditReservedEvent

@crichardson

How atomically update database and publish an event

Order Service

Order Database

Message Broker

insert Order

publish OrderCreatedEvent

dual write problem

?

@crichardson

Failure = inconsistent system

Order Service

Order Database

Message Broker

insert Order

publish OrderCreatedEvent

X

@crichardson

2PC is not an option

@crichardson

How to reliably publish events when state changes?

@crichardson

Agenda

Overview of event sourcing

The problem with microservices, transactions and queries

Using events to maintain consistency

Event sourcing in a microservice architecture

Implementing queries with CQRS

@crichardson

Event sourcing = event-centric persistence

Application

Database

Event store

update

publish

X

@crichardson

Event Store

Application architecture

Order 123 Customer 456

OrderCreated OrderApproved …

CustomerCreated CustomerCreditReserved …

CreateOrder UpdateOrder GetOrder

Subscribe

Order Service

CreateCustomer UpdateCustomer GetCustomer

Subscribe

Customer Service

@crichardson

Request handling in an event sourced application

HTTP Handler

Event Store

pastEvents = findEvents(entityId)

Order

new()

applyEvents(pastEvents)

newEvents = processCmd(someCmd)

saveEvents(newEvents) (optimistic locking)

Order Service

applyEvents(newEvents)

@crichardson

Event Store publishes events consumed by other services

Event Store

Event Subscriber

subscribe(EventTypes)

publish(event)

publish(event)

Customer

update()

Customer Service

@crichardson

Event Store publishes events consumed by other services

Event Store

Event Subscriber

subscribe(EventTypes)

publish(event)

publish(event)

CQRS View

update()

Service Xyz

send notifications

…

Event store = database + message broker

Hybrid database and message broker

Implementations:

Home grown/DIY

geteventstore.com by Greg Young

http://eventuate.io (mine)

Event Store

Save aggregate

events

Get aggregate

events

Subscribe to events

@crichardson

Agenda

Overview of event sourcing

The problem with microservices, transactions and queries

Using events to maintain consistency

Event sourcing in a microservice architecture

Implementing queries with CQRS

@crichardson

Find recent, valuable customers

SELECT * FROM CUSTOMER c, ORDER o WHERE c.id = o.ID AND o.ORDER_TOTAL > 100000 AND o.STATE = 'SHIPPED' AND c.CREATION_DATE > ?

Customer Service

Order Service

What if event sourcing is

used?…. is no longer easy

@crichardson

Use Command Query Responsibility Segregation

(CQRS)

@crichardson

Command Query Responsibility Segregation (CQRS)

Application logic

Commands Queries

XPOST PUT DELETE

GET

@crichardson

Command Query Responsibility Segregation (CQRS)

Command side

Commands

Event Sourcing domain objects

Event Store

Events

Query side

Queries

(Materialized) View

Events

POST PUT DELETE

GET

@crichardson

Query side design

Event Store

Updater

View Updater Service

Events

Reader

HTTP GET Request

View Query Service

View Store

e.g. MongoDB

ElasticSearch Neo4J

update query

@crichardson

Persisting a customer and order history in MongoDB

{ "_id" : "0000014f9a45004b 0a00270000000000", "_class" : "net.chrisrichardson…..views.orderhistory.CustomerView", "version" : NumberLong(5), "orders" : { "0000014f9a450063 0a00270000000000" : { "state" : "APPROVED", "orderId" : "0000014f9a450063 0a00270000000000", "orderTotal" : { "amount" : "1234" } }, "0000014f9a450063 0a00270000000001" : { "state" : "REJECTED", "orderId" : "0000014f9a450063 0a00270000000001", "orderTotal" : { "amount" : "3000" } } }, "name" : "Fred", "creditLimit" : { "amount" : "2000" } }

Denormalized = efficient lookup

@crichardson

Persisting customers and order info using Spring Data for MongoDB...

@crichardson

Persisting customers and order using Spring Data for MongoDB...

Benefits and drawbacks of CQRS

Benefits

Necessary in an event sourced architecture

Separation of concerns = simpler command and query models

Supports multiple denormalized views

Improved scalability and performance

Drawbacks

Complexity

Potential code duplication

Replication lag/eventually consistent views

@crichardson

Demo

@crichardson

Summary

Microservice architecture functionally decomposes an application into services

Transactions and queries resist decomposition

Use an event-driven architecture based on event sourcing to maintain data consistency

Implement queries using CQRS

@crichardson

chris@chrisrichardson.net kbastani@pivotal.io

http://learnmicroservices.io http://pivotal.io

Questions?