reactive reference architecture
TRANSCRIPT
![Page 1: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/1.jpg)
Reactive Reference Architecture
Markus Jura @markusjura
![Page 2: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/2.jpg)
Big dataMassive Learning Platform
http://allthingsd.com/files/2013/07/minions-‐640x327.jpg
![Page 3: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/3.jpg)
3
Principles of Reactive Systems
![Page 4: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/4.jpg)
Architecture
4
Web (Play)
backend (Akka)
Journal (Cassandra)
Fast data processing (Spark)
![Page 5: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/5.jpg)
Problem: Splitting application
![Page 6: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/6.jpg)
Monolithic App
6
![Page 7: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/7.jpg)
Splitting App
7
Frontend nodes
Backend nodes
![Page 8: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/8.jpg)
Akka Distributed Pub/Sub• Location transparency
• Publish/Subscribe
• Dynamically add/remove
8
![Page 9: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/9.jpg)
Distributed Pub/Sub Mediator
9
Frontend nodes
Backend nodes
M M
M M M
![Page 10: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/10.jpg)
Code
![Page 11: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/11.jpg)
Frontend
11
// Starting up the mediatorval mediator = DistributedPubSubExtension(system).mediator
// Sending message through mediatormediator ? DistributedPubSubMediator.Send(“/user/course-aggregate”, CreateCourse(course), localAffinity = true)
![Page 12: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/12.jpg)
Backend
12
// Starting up the mediatorval mediator = DistributedPubSubExtension(system).mediator
// Setting up the servicesval courseAggregate = system.actorOf(CourseRepository.props, "course-aggregate")mediator ! DistributedPubSubMediator.Put(courseAggregate)val levelAggregate = system.actorOf(LevelRepository.props, "level-aggregate")mediator ! DistributedPubSubMediator.Put(levelAggregate)
![Page 13: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/13.jpg)
Problem: Distributed State
![Page 14: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/14.jpg)
State on single node
14
Frontend nodes
Backend nodes
![Page 15: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/15.jpg)
State across nodes
15
Frontend nodes
Backend nodes
![Page 16: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/16.jpg)
Akka Cluster Sharding
• Partition actors across Cluster
• Location transparency
• Rebalancing
16
![Page 17: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/17.jpg)
How does it work?
17
![Page 18: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/18.jpg)
How does it work?
18
![Page 19: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/19.jpg)
Code
![Page 20: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/20.jpg)
Backend: Create ShardRegion
20
// Backend: Start cluster sharing in non-proxy modeClusterSharding(system).start( "users", Some(User.props), UserSharding.idExtractor, UserSharding.shardResolver())
![Page 21: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/21.jpg)
Backend: Shard Resolver & Id Extractor
21
// Extract the user shard // Rule of thumb: 10 times the max numbers of sharding nodesdef shardResolver(shardCount: Int = 20) = { case UserService.Envelope(email, _) => email.hashCode % shardCount.toString}
// Extract the id and payload of the user actor val idExtractor: ShardRegion.IdExtractor = { case UserService.Envelope(email, payload) => (email, payload)}
![Page 22: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/22.jpg)
Frontend
22
// Send message via shard region shardRegion ? UserService.Envelope(email, GetUser(email))
// Start sharding in proxy mode on every frontend nodeval shardRegion: ActorRef = ClusterSharding(system).start( "users", None, UserSharding.idExtractor, UserSharding.shardResolver())
![Page 23: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/23.jpg)
Problem: Loosing State
![Page 24: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/24.jpg)
State in memory
24
Frontend nodes
Backend nodes
![Page 25: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/25.jpg)
Persist state
25
Frontend
Backend
![Page 26: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/26.jpg)
Akka Persistence
• Stores state in Journal DB
• Event Sourcing
• Restore state
26
![Page 27: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/27.jpg)
Code
![Page 28: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/28.jpg)
Write user state
28
class User extends PersistentActor {
// User id / sharding extractor id override def persistenceId: String = self.path.name
// Persist UserCreated event to Cassandra journal override def receiveCommand: Receive = { case UserService.CreateUser(userProgress) => persist(UserCreated(userProgress)) { userCreated => // Update user state onUserCreated(userCreated) sender() ! UserService.UserCreated } } ...
![Page 29: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/29.jpg)
Recover user state
29
...
// Replay UserCreated message to recover state override def receiveRecover: Receive = { case e: UserCreated => onUserCreated(e)}
![Page 30: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/30.jpg)
Problem: Replicate Cache
![Page 31: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/31.jpg)
Caching
31
Frontend
Backend
C? C?
![Page 32: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/32.jpg)
CRDT Replicator
32
Frontend nodes
Backend nodes
CR
R R R
RC
![Page 33: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/33.jpg)
What is CRDT?
33
Conflict-free replicated data type is a type of specially designed data structure used to achieve strong eventual consistency
Great talk on CRDT by Sean Cribbs: Eventually Consistent Data Structures
![Page 34: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/34.jpg)
Akka Data replication• CRDT for Akka cluster
• Replicated in-memory data store
• Supports many types: (GSet, ORSet, PNCounter…)
34
![Page 35: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/35.jpg)
Code
![Page 36: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/36.jpg)
Frontend
36
//starting up the replicatorval replicator = DataReplication(system).replicator
//get cache entryval f = replicator ? Get(key, ReadLocal, None)
![Page 37: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/37.jpg)
Backend
37
//starting up the replicatorval replicator = DataReplication(system).replicator
//updating the cachereplicator ! Update(Course.cacheKey, GSet.empty, WriteLocal)
(_ + courseCreated.course)
![Page 38: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/38.jpg)
Q & Option[A]
@markusjura
![Page 39: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/39.jpg)
39
React.js
frontend
REST API Actors
backend
Cache
PubSub CRDTPersistenceCluster Sharding
journal
![Page 40: Reactive reference architecture](https://reader034.vdocuments.net/reader034/viewer/2022042509/55b6e5b8bb61eb69688b4637/html5/thumbnails/40.jpg)
©Typesafe 2014 – All Rights Reserved