cqrs & event sourcing in the wild (scotlandphp 2016)
TRANSCRIPT
CQRS & EVENT SOURCING IN THE WILD
Michiel Rook - @michieltcs
➤ Java, PHP & Scala developer
➤ Consultant, trainer, speaker
➤ Dutch Web Alliance
➤ make.io
➤ Maintainer of Phing
➤ @michieltcs
YOU
RAISE YOUR HAND
IF YOU HAVE
heard about CQRS / Event Sourcing
RAISE YOUR HAND
IF YOU HAVE
heard about CQRS / Event Sourcing
followed a tutorial, built a hobby project
RAISE YOUR HAND
IF YOU HAVE
heard about CQRS / Event Sourcing
followed a tutorial, built a hobby project
used it in production
RAISE YOUR HAND
IF YOU HAVE
TOPICS
➤ Quick recap
➤ Replays and rebuilds
➤ Event versioning
➤ Concurrency
➤ Scale
QUICK RECAP
' Event Sourcing ensures that all changes to application state are stored as a sequence of events.
-Martin Fowler
ACTIVE RECORD VS. EVENT SOURCING
Account number
Balance12345678 �€ 50,00
... ...
Money WithdrawnAccount number 12345678
Amount �€ 50,00
Money DepositedAccount number 12345678
Amount �€ 100,00
Account CreatedAccount number 12345678
COMMANDS TO EVENTS
Deposit MoneyAccount number 12345678
Amount �€ 100,00
class DepositMoney { public $accountNumber; public $amount; }
COMMANDS TO EVENTS
Deposit MoneyAccount number 12345678
Amount �€ 100,00
function depositMoney(DepositMoney $command) { $this->apply(new MoneyDeposited( $command->accountNumber, $command->amount, date())); }
command handler
COMMANDS TO EVENTS
Deposit MoneyAccount number 12345678
Amount �€ 100,00
Money DepositedAccount number 12345678
Amount �€ 100,00
class MoneyDeposited { public $accountNumber; public $amount; public $timestamp; }
command handler
AGGREGATES
class BankAccount { public $accountNumber; public $balance; // ... public function applyMoneyDeposited( MoneyDeposited $event) { $this->balance += $event->amount; }
AGGREGATE STATE
Account number
Balance12345678 �€ 0,00
Money WithdrawnAccount number 12345678
Amount �€ 50,00
Money DepositedAccount number 12345678
Amount �€ 100,00
Account CreatedAccount number 12345678
Account number
Balance12345678 �€ 100,00
Account number
Balance12345678 �€ 50,00
CQRS + EVENT SOURCING
Domain
UI
Event Bus
Event Handlers
Command
Repository
Data Layer
Database Database
Event Store
commands
events
events
queries DTOs
Aggregates
PROS AND CONS
➤ Domain fit
➤ Testing
➤ Audit trail
➤ Scalability
➤ Complexity
➤ Library support / maturity
DISCLAIMER
REPLAYS AND REBUILDS
PROJECTIONS AND READ MODELS
User Registered
User Registered
User Unregistered Number of active users?
PROJECTIONS AND READ MODELS
User Registered
User Deactivated
User Reactivated
Number of active users?User Registered
User Unregistered
PROJECTIONS AND READ MODELS
User Registered Event HandlerNumber of
active users +1
User Unregistered Event HandlerNumber of
active users -1
PROJECTIONS AND READ MODELS
Events Event Handler(s) Storage
ELASTICSEARCH
class CompanyRegistered { public $companyId; public $name; public $street; public $city; }
ELASTICSEARCH
function handleCompanyRegistered( CompanyRegistered $event) { $this->elasticsearch->index([ 'index' => 'companies', 'type' => 'company', 'id' => $event->companyId, 'body' => [ 'name' => $event->name, 'address' => [ 'street' => $event->street, 'city' => $event->city ] ] ]); }
READ MODEL UPDATES
➤ New type
➤ New structure
➤ Based on existing events
➤ Generate from scratch?
REBUILDING
Stop application
Remove old read model
Loop over events
Apply to read model
Start application
ZERO DOWNTIME
Loop over existing events
Apply to new read
model
Apply queued events
Start using new read
model
New events Queue
CHALLENGE: LONG RUNNING REBUILDS
➤ Alternatives:
➤ In memory
➤ Distributed
➤ Partial
➤ Background
CHALLENGE: SIDE EFFECTS
User RegisteredUser Id 123abc
Email Address [email protected] Handler
Exclude during replays!
CHALLENGE: TRANSACTIONS
Event Handler
Event Handler
Event
Event Handler
Event Handler ?
CHALLENGE: EVENTUAL CONSISTENCY
➤ Asynchronous event handlers
➤ Reads eventually return the same value
➤ Compare with ACID
➤ UI?
EVENT VERSIONING
DILEMMA
➤ New business requirements
➤ Refactoring
➤ New view on events
NEW EVENTS / VERSIONS
➤ No longer relevant
➤ Renamed
➤ Additional or renamed field(s)
➤ Too coarse, too fine
SUPPORT YOUR LEGACY?
➤ Commands can be renamed
➤ Events are immutable
➤ Correct (incorrect) old events with new events
UPCASTING
Event Store
UserRegistered_V1
Upcaster
UserRegistered_V2
Event Handler
UPCASTING
class UserRegistered_V1 { public $userId; public $name; public $timestamp; }
UPCASTING
class UserRegistered_V2 { public $userId; public $name; public $date; }
UPCASTING
function upcast($event): array { if (!$event instanceof UserRegistered_V1) { return []; } return [ new UserRegistered_V2( $event->userId, $event->name, $event->timestamp->format("Y-m-d")) ]; }
REWRITING HISTORY
Load (subset of) events
Deserialize
Modify
Serialize
Save/replace
THINGS TO BE AWARE OF
Upcasting
➤ Performance
➤ Complexity
➤ Existing projections not automatically updated
Rewriting events
➤ Running code that depends on old structure
➤ Breaking serialization
➤ Changing wrong events
CONCURRENCY
CONCURRENT COMMANDS
Withdraw MoneyAccount number 12345678
Amount �€ 50,00
Deposit MoneyAccount number 12345678
Amount �€ 100,00
?
PESSIMISTIC LOCKING
Withdraw MoneyAccount number 12345678
Amount �€ 50,00
Deposit MoneyAccount number 12345678
Amount �€ 100,00
Account number
Balance12345678 �€ 100,00
Account number
Balance12345678 �€ 50,00
wait for lock
lock
OPTIMISTIC LOCKING
Withdraw MoneyAccount number 12345678
Amount �€ 50,00version 1
Deposit MoneyAccount number 12345678
Amount �€ 100,00version 1
Account number
Balance12345678 �€ 100,00
version 2
ConcurrencyException
SCALE
PERFORMANCE
➤ Server
➤ Database
➤ Framework
➤ Language
➤ Serializer
STORAGE
➤ #events
➤ #aggregates
➤ #events_per_aggregate
➤ Serializer
➤ Event payloads
➤ Costs
SNAPSHOTS
Events1 Account Created2 Money Deposited3 Money Withdrawn4 Money Deposited
SNAPSHOT5 Money Withdrawn
Events1 Account Created2 Money Deposited3 Money Withdrawn4 Money Deposited5 Money Withdrawn
SHARDING
➤ Aggregate Id, Type, Event Timestamp, ...
➤ Rebalancing
➤ Distribution
ARCHIVING EVENTS
➤ Reduce working set
➤ Inactive / deleted aggregates
➤ Historic / irrelevant events
➤ Cheaper storage
FRAMEWORK COMPARISON
Framework Upcasting Snapshots Replaying
Broadway (PHP) No (PR) No (PR) Not in core
Prooph (PHP) MessageFactory Yes, triggers on
event countExample code,
off line
Axon (Java/Scala)
Upcaster / UpcasterChain
Yes, triggers on event count
Yes, ReplayingCluster
Akka Persistence (Java/Scala)
Event Adapter Yes, decided by actor Yes
