how to cqrs in node: eventually consistent, distributed microservice systems

Matt Walters @mateodelnorte [email protected]

How to CQRS in Node

Eventually consistent architectures that scale and grow.

mailto:[email protected]?subject=

It’s real. It works!

Former TechStars Co. CTO. Now consultant.

Built two businesses’ platforms from scratch using CQRS in Node. Both large, distributed systems.

CQRS made both more maintainable and extendable.

Marketing platform crunched the Twitter firehose in realtime. 3 Engineers managed around 20 services. - GoChime.com

Bond Exchange with over $1.75B in trades. 6 engineers managed around 40 services. - Electronifie.com

Open Source my tooling and frameworks.

Other companies using them too!

(and I’ll tell you how)

That’s me ^^ !!

When to CQRS• Realtime, reactive systems • When preferring small, modular services • When aiming for learning and growth • When aiming to grow or eventually split teams • Want ability to scale different parts of your

system separately

When not to CQRS

• Standalone, static sites • Standalone, simple CRUD applications

What is CQRS?

First, a primer from history.

Bertrand Meyer, regarding object interfaces:

“Every method should either be a command that performs an action, or a query that returns data to the caller, but not both. In other words, Asking a question should not change the answer.”

Long before CQRS was CQS:

Command Query Separation.

this guy

Some Type

doSomething() : void

getSomeStuff() : Stuff

Either change stuff<——

Or get stuff<——

CQSCommand-Query

Separation

doAndGetStuff() : Stuff Never both!——————

Command-Query Responsibility Segregation

A system-wide architecture that states - externally facing subsystems (apps and apis) send commands to perform actions which update the system’s state and request queries to determine system’s state.

*Basically CQS on a system-wide scale. Calls between services should change stuff, or get stuff. Never both.

CQRS

Command-Query Responsibility Segregation

CQRS also denotes that queries and command processing are provided by different subsystems.

Queries are made against a data store. Commands are sent to and processed by services.

CQRSOne more thing!

What is CQRS?

denormalizer dbdenormalizer dbdenormalizer db

web-uiweb-uiweb-appweb client

denormalizer

web-uiweb-uisvc

unidirectional floweventually consistent

Queries

Commands

the dance!Events

How about a larger system?


web-uiweb-uiweb-app

web-uiweb-uiweb-api

web client

mobile client

denormalizer

svc-3

svc-2web-uiweb-uisvc-1

Queries

Commands


the dance!Events

What’s that dance you’re doing?


denormalizer

svc-3


the dance!

chain reaction of events which play out as a result of an incoming command.

each service subscribes to the events they care about

choreography!(not orchestration)

Events

commands tell services when an actor wants an action

clients send commands to instruct a service to do work

commands are sent asynchronously; fire and forget

commands are present-tense, directives: order.create

web app order-svc

order.create

commands are sent directly to a single receiving service

events tell the world when you’re done

services publish to inform other services of work / actions performed, and state updated

services publish (broadcast) events to any services that wish to subscribe

events past-tense, describe what happened: order.created

order-svc fulfillment-svc

order.createdorder.createdorder.created

order.created

Two types of services


web-uiweb-uiweb-app

web-uiweb-uiweb-api

web client

mobile client

denormalizer

svc-3


front end


web-uiweb-uiweb-app

web-uiweb-uiweb-api

web client

mobile client

denormalizer

svc-3


back end

Two types of services


denormalizer

web-uiweb-uisvc

front end


What’s different?

Let’s focus on:


web-uiweb-uisvc

front end (an app’s perspective)


What’s different?

Apps (and apis) still query a db to get the state of the system

Never directly modify the db they read from

Let’s focus on:


web-uiweb-uisvc


What’s different?

Instead, apps (and apis) send commands instructing services to perform an action

Apps expect their read only representation of system state will eventually be updated in the denormalizer db

Let’s focus on:front end (an app’s perspective)

Commands are sent over a reliable transport (rabbitmq, kafka, zeromq, etc) to ensure delivery and eventual consistency

Let’s pick a transport!

rabbitmqmessaging that just works

• direct send to queue • fanout / topic routing • highly available • highly performant • used in financial exchanges, industrial applications and more • open source • free

rabbitmqmessaging that just works

brew install rabbitmq

framework!minimalist

Let’s pick a

servicebussuper simple messaging in node

• direct send • pub / sub / fanout / topic -routing • simple to set up • highly performant • used in financial exchanges, online advertising and more • open source • free • perfect for creating microservices!

servicebussuper simple messaging in node

npm install servicebus —save


denormalizer

web-uiweb-uisvc

front end


What’s different?

Let’s focus on:

web-uiweb-uiweb-app

web-uiweb-uisvc

front end commands


How’s this work?

Let’s focus on:

Sending commands from the front end

// web-app, onButtonClick. instead of updating db. const bus = require('servicebus').bus();

bus.send(‘order.create', { order: {

userId: userId, orderItems: items

} });

web-uiweb-uiweb-appweb-uiweb-uisvc

// fire and forget.

command name

command itself

order.create command

Then what from the front end?

web-uiweb-uiweb-appdenormalizer

dbdenormalizer dbdenormalizer db

We wait.

• reactive / realtime: • mongo oplog tailing (meteor) • rethinkdb • redis notifications • couchdb • graphQL • polling

• non-realtime: • product design: thanks! we’re processing your

order! check back later for updates!

queries!

We wait.

For the backend.

Choreography.

It’s eventually consistent!


web-uiweb-uiweb-app

web-uiweb-uiweb-api

web client

mobile client

denormalizer

svc-3


front end

Let’s focus on:


web-uiweb-uiweb-app

web-uiweb-uiweb-api

web client

mobile client

denormalizer

svc-3


back end

Let’s focus on:

back end


web-uiweb-uiweb-app

web-uiweb-uiweb-api

denormalizer

svc-3


Let’s focus on:

back end (a service’s perspective)

web-uiweb-uiweb-app

svc

Commands

Events

• Listen for commands and subscribe to events • Performs business logic to process commands and events • Update local state (optionally) • Publish events to tell external services of updated state

Let’s focus on:

Sample service.


command name

command object

// order-svc index.js const bus = require(‘./bus’); const create = require(‘./lib/create’);

bus.listen(‘order.create', (event) => { create(event, (err, order) => { if (err) return event.handle.reject(); bus.publish(‘order.created’, order, () => { event.handle.ack();

}); }); });

service publishes to the world when it’s done!

order.create command order.created

event

Sample service.


listening for commandsperforming business logic & updating state// order-svc index.js

const bus = require(‘./bus’); const create = require(‘./lib/create’);


}); }); });

order.create command order.created

event

completing atomic transaction and allowing error handling

back end (a downstream service’s perspective)

web-uiweb-uiweb-app

svc

Commands

Events


svc-2

Even

ts

Events

Let’s focus on:

Same thing!

Sample downstream service.


// fulfillment-svc index.js const bus = require(‘./bus’); const fulfill = require(‘./lib/fulfill’);

bus.subscribe(‘order.created', (event) => { fulfill(event, (err, order) => { if (err) return event.handle.reject(); bus.publish(‘order.fulfilled’, order, () => { event.handle.ack();

}); }); });

order.created order.fulfilled

subscribe for events instead of listening for commands

no different, from any other service!

Multiple handlers?

svc

Commands

svc-2

Even

ts

svc-2

Events

servicebus-register-handlersconvention based event handler definition for

distributed services using servicebus.

automatically registers event & command handlers saved as modules in folder

initialize at startup

servicebus-register-handlers

const bus = require(‘./lib/bus'); // instantiate servicebus instance const config = require('cconfig')(); const log = require('llog'); const registerHandlers = require('servicebus-register-handlers');

registerHandlers({ bus: bus, handleError: function handleError (msg, err) {

log.error('error handling %s: %s. rejecting message w/ cid %s and correlationId %s.', msg.type, err, msg.cid, this.correlationId);

log.error(err);

msg.handle.reject(function () { throw err; });

}, path: './lib/handlers', queuePrefix: 'my-svc-name' });

initialize at startup:

provide initialized busdefine your error handling

path to your handlers

prefix to differentiate similar queues

servicebus-register-handlers

const log = require("llog");

module.exports.ack = true;

module.exports.queueName = 'my-service-name-order'; module.exports.routingKey = "order.create"; module.exports.listen = function (event, cb) {

log.info(`handling listened event of type ${event.type} with routingKey ${this.routingKey}`);

/*

do something with your event

*/

cb(); };

each handler is a file

no params marks success. pass back error to retry or fail.

callback based transactions!

differentiate queues for different services

specify which commands or events to listen or subscribe to

servicebus-register-handlerssuper simple messaging in node

npm install servicebus-register-handlers —save

What about the ‘work’ part?

const log = require("llog");

module.exports.ack = true;

module.exports.queueName = 'my-service-name-order'; module.exports.routingKey = "order.create"; module.exports.listen = function (event, cb) {

log.info(`handling listened event of type ${event.type} with routingKey ${this.routingKey}`);

/*

do something with your event

*/

cb(); };

// order-svc index.js const bus = require(‘./bus’); const create = require(‘./lib/create’);


}); }); });

these parts

What about the ‘work’ part?That’s up to you!

Need an audit trail? Targeting finance? Consider event sourcing. *and my framework, ‘sourced’

Depending on your problem, the right choice could be mongoose and mongodb, a graph database, an in-memory data structure, or even flat files.

CQRS makes no assertions about what technology you should use, and in fact frees you to make a different decision for each particular problem.

and depends on the problem you’re solving

But wait! There’s more!servicebus middleware!

middleware can inspect and modify incoming and outgoing messages

// ./lib/bus.js required as single bus instance used anywhere in service const config = require('cconfig')(); const servicebus = require('servicebus'); const retry = require('servicebus-retry');

const bus = servicebus.bus({ url: config.RABBITMQ_URL });

bus.use(bus.package()); bus.use(bus.correlate()); bus.use(retry({ store: new retry.RedisStore({ host: config.REDIS.HOST, port: config.REDIS.PORT }) }));

module.exports = bus;

bus.use() middleware into bus message pipeline. middleware can act on incoming and/or outgoing messages







packages outgoing message data and adds useful type, timestamp, and other properties







adds a correlationId for tracing related commands and events through your system







now, every failed message will retry 3 times if errors occur. after that, the message will automatically be put on an error queue for human inspection!

And more!distributed tracing middleware!

var trace = require('servicebus-trace'); bus.use(trace({ serviceName: 'my-service-name', store: new trace.RedisStore({ host: config.REDIS_HOST || 'localhost', port: config.REDIS_PORT || 6379 }) }));


web-uiweb-uiweb-app

web-uiweb-uiweb-api

web client

mobile client

denormalizer

svc-3


back end

Recapping Back End Services

back end services

web-uiweb-uiweb-app

svc

Commands

Events


Recapping:


web-uiweb-uiweb-app

web-uiweb-uiweb-api

web client

mobile client

denormalizer

svc-3


back end

Recapping Back End Services

wat wat watwat

Wat! is a

denormalizer?

back end

What’s a denormalizer?

front end

• Just another back end service • Has one job to do • Subscribe to all events that the UI cares about • Persist events in a format most efficient for the UI to view• Completes the eventually consistent, unidirectional flow


web-uiweb-uiweb-app

web client

denormalizer

Eventssvc


web-uiweb-uiweb-app

web client

denormalizer

back end

What’s a denormalizer?

Events

front end

svc

Recapping the big picture.


web-uiweb-uiweb-app

web-uiweb-uiweb-api

web client

mobile client

denormalizer

svc-3


Queries

Commands


the dance!Events

CQRS? That’s a myth!

Thanks!We’ll sneak preview even more tooling for

easily building these systems in a future talk!

Matt Walters github & twitter: @mateodelnorte email: [email protected] website: iammattwalters.com

rabbitmq.com

npmjs.com/package/servicebus

How to CQRS in Node

npmjs.com/package/servicebus-register-handlersnpmjs.com/package/servicebus-retrynpmjs.com/package/servicebus-trace

mailto:[email protected]?subject=

http://iammattwalters.com

http://rabbitmq.com

https://www.npmjs.com/package/servicebus

https://www.npmjs.com/package/servicebus-register-handlers

https://www.npmjs.com/package/servicebus-retry

https://www.npmjs.com/package/servicebus-trace

how to cqrs in node: eventually consistent, distributed microservice systems

Technology