primer on cqrs and event sourcing
TRANSCRIPT
FRONTENDBACKEND
COMPONENTS OVERVIEWApplications and data stores in play
Vanessa CMS
Oracle DB MySQL
Website
BI Reporting CRM Service
REPLICATION
MESSAGING
FRONTENDBACKEND
EASY, JUST ADD MORE HARDWAREOh, and stay clear from the CFO
Vanessa CMS
Oracle DB MySQL
Website
BI Reporting CRM Service
REPLICATION
MESSAGING
MySQLMySQL
Oracle DBOracle DB
FRONTENDBACKEND
SCALE READS BY ADDING SLAVESSeparate write operations from read operations
Vanessa CMS
Oracle DBMySQL
<master>
Website
BI Reporting CRM Service
REPLICATION
MESSAGING
MySQL<slave>
REPLICATION
WRITING READING
MySQL<slave>
FRONTEND
WRITING
BACKEND
SEPARATE CONCERNSA layer responsible for reading, a layer responsible for writing
Vanessa CMS
Oracle DB
Website
BI Reporting CRM Service
REPLICATION
MESSAGING
READING
CMS
Website
BI Reporting
MySQL<master>
MySQL<slave>
REPLICATION
WRITING READING
MySQL<slave>
• Designer of Eiffel programming language
• Coined ‘design by contract’
• Coined ‘open/closed principle’
• Introduced command query separation (CQS)
BERTRAND MEYER.
• Separate methods that change and read state
• Command methods for changing state
• Query methods to read state
• Most likely from the same datastore & object model
COMMAND QUERY SEPARATION.
class Product{ protected $active; /** * Command method activating a product; Changes state */ public function activate() { $this->active = true; } /** * Command method deactivating a product; Changes state */ public function deactivate() { $this->active = false; } /** * Query method which returns the active state of the product; Read state */ public function isActive() { return $this->active; } }
CQS Example - 1/1
• CQS extended
• Not separation by methods, but by classes
• Command objects change state
• Query objects read state
• Brings separation of concerns
COMMAND QUERY RESPONSIBILITY SEPARATION.
FRONTEND
WRITING
BACKEND
SEPARATED CONCERNSUse of commands and queries separates write and read concerns
Vanessa CMS
Oracle DB
Website
BI Reporting CRM Service
REPLICATION
MESSAGING
READING
CMS
Website
BI Reporting
MySQL<master>
MySQL<slave>
REPLICATION
COMMANDS QUERYING
MySQL<slave>
• Command is the message to change state
• Message handled by a CommandHandler
• Returns void (so nothing)
• Describes business intent
• Immutable
COMMAND CHARACTERISTICS.
FRONTEND
WRITING
BACKEND
SEPARATED READ AND WRITE CONCERNSLet those millions of visitors do their thing
Vanessa CMS
Oracle DB
Website
BI Reporting CRM Service
REPLICATION
MESSAGING
READING
CMS
Website
BI Reporting
MySQL<master>
MySQL<slave>
REPLICATION
COMMANDS QUERIES
MySQL<slave>
FRONTEND
WRITING
BACKEND
COMPONENTS WITH DIFFERENT NEEDSBut are using the same data model
Vanessa CMS
Oracle DB
Website
BI Reporting CRM Service
REPLICATION
MESSAGING
READING
CMS
Website
BI Reporting
MySQL<master>
MySQL<slave>
REPLICATION
WRITING READING
MySQL<slave>
USAGE PRODUCT ENTITY.
Vanessa BI Reporting Website
Goal: provide CRUD oriented interface for manipulating product information
Goal: Create a price report for our entire catalog which we can use for comparison against our competitors
Goal: Display product information to website visitors in the least amount of time
Product: - Product ID - Product Name - Published
ProductAvailability: - Availability ID - Stock Location - Stock amount
ProductSalesPrice: - ProductSalesPrice ID - Price excluding VAT - Price including VAT
Product: - Product ID - Product Name - Price excluding VAT - Price including VAT
Product: - Product ID - Product Name - Published - Stock amount - Price excluding VAT - Price including VAT
FRONTEND
WRITING
BACKEND
PROVIDE MULTIPLE DATA STORESUsing different data sets tailored on the component usage
Vanessa CMS
Oracle DB
Website
BI Reporting CRM Service
MESSAGING
READING
CMS
Website
BI Reporting
Data store (representation)
Data store (reporting)
Magic replication process
WRITING
MySQL<master>
REPLICATION
FRONTEND
WRITING
BACKEND
OPTIMIZE MODEL FOR QUERYINGRead models used for query results
Vanessa CMS
Oracle DB
Website
BI Reporting CRM Service
MESSAGING
READING
CMS
Website
BI Reporting
Datastore (representation)
Datastore (reporting)
READMODEL
WRITING
Magic replication process
WRITING
MySQL<master>
REPLICATION
• Model specialized for queries
• Data Transfer Object
• Can be used
• Dynamically
• Flattened persistence
READ MODEL.
FRONTEND
WRITING
BACKEND
DYNAMIC USE FOR READ MODELQuery prepares data to populate read model on run time
Vanessa CMS
Oracle DB
Website
BI Reporting CRM Service
MESSAGING
READING
CMS
Website
BI Reporting
Datastore (representation)
Datastore (reporting)
READMODEL
WRITING
Magic replication process
WRITING
MySQL<master>
REPLICATION
• Ideal for queries that aren’t heavily used
• Saves persisting flattened representation of model
• Queries do most of the work
DYNAMIC USE OF READ MODEL.
FRONTEND
WRITING
BACKEND
FLATTENED READ MODELPersist flattened representation in datastores
Vanessa CMS
Oracle DB
Website
BI Reporting CRM Service
MESSAGING
READING
CMS
Website
BI Reporting
Datastore (representation)
Datastore (reporting)
READ MODEL
WRITING
Magic replication process
WRITING
MySQL<master>
REPLICATION
• Ideal for queries that are heavily used
• Hard work done during persisting
• Querying will be as simple as `select * from TABLE`
FLATTENED READ MODEL.
FRONTEND
WRITING
BACKEND
BEST OF BOTH WORLDSHybrid model which uses flattened and dynamic model
Vanessa CMS
Oracle DB
Website
BI Reporting CRM Service
MESSAGING
READING
CMS
Website
BI Reporting
Datastore (representation)
Datastore (reporting)
READMODEL
WRITING
Magic replication process
WRITING
MySQL<master>
REPLICATION
READ MODEL
• CQRS does not enforce any mechanism
• Possible options:
• Database replication
• Messaging
• Eventing
REPLICATION: IT’S UP TO YOU.
SYNCHRONIZE VIA MESSAGINGUse message bus for updating all data stores
CMSData store
(representation)
Data store
Message bus
CreateProductCommandName: ‘Product NEW’
CommandHandler
CommandHandler
Data store (reporting)
CommandHandler
FRONTEND
WRITING
BACKEND
DEFINING THE MAGICAL REPLICATIONAdding command bus, commands and command handlers
Vanessa CMS
Oracle DB
Website
BI Reporting
CRM Service
MESSAGING
READING
CMS
Website
BI Reporting
Datastore (representation)
Datastore (reporting)
READMODEL
MySQL<master>
Command bus
Command Handlers Command Handlers
Command Handlers COMMANDSCOMMANDS
• Data that has been written should be available immediately
• For example
• Adding a product to the shopping basket
• Decreasing the stock of a product
IMMEDIATE CONSISTENCY.
• Data that has been written should be available whenever possible
• For example
• Adding a review for a product
• Altering product content
EVENTUAL CONSISTENCY.
• Eventually consistency impacts workflow
• Can the business handle this workflow
SYNCHRONOUS VS ASYNCHRONOUS WORKFLOW.
• List all users that have accessed order ‘X’ in 2014
• List all products that were removed from a shopping basket for shop ‘Z’ in period ‘Q’ to ‘R’
BUSINESS REQUESTS.
• Relates to most domains in the real world
• Account at the bank
• Lifecycle of a product
• No coupling between representation of state and storage
• Events can be replayed
• Append-only model scales easier
• Extra bonus: auditing trail
WHY USE EVENT SOURCING.
• Events describe something that happened in the past
• Represents action
• Record user intent
WHAT ARE EVENTS.
EVENTS LEAD TO OBJECT STATE.
ProductCreatedEvent Name: ‘Product X’
ProductPublishedEventActive: true
ProductRenamedEvent Name: ‘Product NEW’
ProductPricedEventPrice: EUR 600,-
Product:Name: Product NEWPrice: EUR 600,-Active: true
YOUR DATABASE WILL LOOK LIKE THIS.
UUID Sequence Event Payload Date Issuerdb75c660-ce28-11e4-83f2-0002a5d5c51b 1 CreatedProductEvent {
"name": "test" }
2015-03-19 12:45:03 User A
db75c660-ce28-11e4-83f2-0002a5d5c51b 2 RenamedProductEvent { "name": “Product X" }
2015-03-19 13:01:23 User B
82669760-ce29-11e4-8ae9-0002a5d5c51b 8 DeletedProductEvent {} 2015-03-21 18:03:19 User B
SYNCHRONIZE VIA EVENTINGUse events for updating data stores
CMS
Data store (representation) Event Store
Command bus
CreateProductCommandName: ‘Product NEW’
CommandHandler Event Listener Event ListenerCreatedProductEventName: ‘Product NEW’
Event bus
Data store (reporting)
FRONTEND
WRITING
BACKEND
EVENT SOURCING OUR DATAStop storing state, but persist intentions and events
Vanessa CMS
Event store
Website
BI Reporting
CRM Service
MESSAGING
READING
CMS
Website
BI Reporting
Datastore (representation)
Datastore (reporting)
READMODEL
Event store
Command bus
Command Handlers Command Handlers
Event listeners COMMANDSCOMMANDS
Event bus
EVENTS
• Aggregating events into structural representation
• Replays events applying business requirements
PROJECTIONS.
FRONTEND
WRITING
BACKEND
PROJECT EVENTS INTO DATA REPRESENTATIONAggregate events to structural representations
Vanessa CMS
Event store
Website
BI Reporting
CRM Service
MESSAGING
READING
CMS
Website
BI Reporting
Datastore (representation)
Datastore (reporting)
READMODEL
Event store
Command bus
Command Handlers Command Handlers
Event listeners
COMMANDSCOMMANDS
Event bus
EVENTS
Projections
DATA REQUIREMENTS OF COMPONENT.
Vanessa BI Reporting Website
Goal: provide CRUD oriented interface for manipulating product information
Goal: Create a price report for our entire catalog which we can use for comparison against our competitors
Goal: Display product information to website visitors in the least amount of time
Product: - Product ID - Product Name - Published
ProductAvailability: - Availability ID - Stock Location - Stock amount
ProductSalesPrice: - ProductSalesPrice ID - Price excluding VAT - Price including VAT
Product: - Product ID - Product Name - Price excluding VAT - Price including VAT
Product: - Product ID - Product Name - Published - Stock amount - Price excluding VAT - Price including VAT
PROJECTING PRODUCTS TO OUR DATA STORES.
ProductCreatedEventName: ‘Product X’
ProductPublishedEventActive: true
ProductRenamedEventName: ‘Product NEW’
ProductPricedEventPrice: EUR 600,-
Product:Name: Product NEWPrice: EUR 600,-Active: true
REPRESENTATION PROJECTION
ProductCreatedEventName: ‘Product X’
ProductRenamedEventName: ‘Product NEW’
ProductPricedEventPrice: EUR 600,-
Product:Name: Product NEWPrice: EUR 600,-
REPORTING PROJECTION
• List all users that accessed order ‘X’ in 2015
• List the times that product ‘Y’ has been removed from a shopping basket and been replaced by product ‘Z’
• List all changes between ‘Q’ and ‘R’ executed by user ‘Z’
COOLBLUE PROJECTIONS.
FRONTEND
WRITING
BACKEND
Vanessa CMS
Event store
Website
BI Reporting
CRM Service
MESSAGING
READING
CMS
Website
BI Reporting
Datastore (representation)
Datastore (reporting)
READMODEL
Event store
Command bus
Command Handlers Command Handlers
Event listeners
COMMANDSCOMMANDS
Event bus
EVENTS
Projections
OVERVIEW AFTER APPLYING PATTERNSCQRS and Event Sourcing helped to tackle our challenges
• Scale read operations differently from write operations
• Simplicity in domain model by separating read and writes
• Added auditing trail
• Added ability to capture intent over state
ACHIEVEMENTS.
• Powerful patterns, but do they apply to your needs?
• CQRS probably influences your UI and thus the business workflow (Synchronous workflow vs Asynchronous workflow)
REFLECTING.
• CQRS is not easy
• Event sourcing is not mandatory for CQRS
• Commands are not fire and forget
• CQRS does not eliminate thinking about concurrency
BUST SOME CQRS MYTHS.
• Look at Broadway framework
• Experiment with your own implementation
• Do not take things for granted, not even this presentation
FOOD FOR THOUGHT.
Paul de RaaijE-mail: [email protected]: @pderaaijSlideshare: http://www.slideshare.net/pderaaij
THANK YOU!