developing distributed applications with akka and akka cluster
TRANSCRIPT
Developing distributed applications
with Akka and Akka Cluster
Presented by K.Tsykulenko
Introduction
Agenda• What is Akka?• Concurrency paradigms overview.• Actors and actor model.• Live demo #1.• Akka remoting and clustering.• Live demo #2.• CRDTs and Akka Distributed Data.• Live demo #3.• Summary.• Q&A.
What is Akka?
http://akka.io
Akka is a toolkit and runtime for building highly concurrent, distributed, and fault tolerant event-
driven applications on the JVM.
What is Akka?
http://akka.io
Concurrency paradigms
Concurrency paradigms• Shared state and locks• Software Transactional Memory (STM)• Message-Passing Concurrency (Actors)• Dataflow Concurrency • and more…
Software transactional memory
V 1 V 2
V 1 V 12
V 11
V 3
V 2 V 22
successfulwrite
transaction
retriedwrite
transaction
V 1
Time
Dataflow Concurrency
X 1
X12=f(X1,X2)
X123=f(X12,X3)
X 2
X 3
Actors
Actors• Originate in a 1973 paper by Carl Hewitt• Implemented in Erlang• Encapsulate state and behavior• Closer to the definition of OO than classes
Actors
Sender ActorRef
Actor
Dispatcher
hasMessage
Message
Message
Mailbox
has
enqueue dequeue
schedule
send
Actorsuser ! User(“John Doe")
class UserActor extends Actor { def receive = { case User(name) => sender ! s"Hi $name" }}
Actorsval greeting = user ? User(“John Doe")
class UserActor extends Actor { def receive = { case User(name) => sender ! s"Hi $name" }}
Supervision and hierarchy
worker 1
worker 2
worker 3
worker 4
supervisor 1
supervisor 2
user
Building a web crawler1. Fetch a page2. Parse the page to get links3. Check if max crawl depth has been reached
and if yes, finish4. Go to 1 for all parsed links
Building a web crawler
Parser
CrawlMaster
user
Fetcher
pass urls
pass page content
pass parsed urls
Building a web crawler
Routerpass urls
CrawlMaster
initial url
UrlHandler
create UrlHandler
Fetcher Fetcher
Router
Parser Parser
Building a web crawlerclass FetcherActor(val parser: ActorRef) extends Actor with ActorLogging { import context.dispatcher val pipeline: HttpRequest => Future[HttpResponse] = sendReceive
override def receive: Receive = { case Url(link, depth) => pipeline(Get(link)).map(…).pipeTo(parser) }}
Building a web crawlerclass ParserActor extends Actor with ActorLogging with HtmlParser { override def receive: Receive = { case UrlContents(Url(link, depth), resp, requester) => val links = parseHtml(resp) .map(l => if (l.matches("^[\\/#].*")) link + l else l) .filter(l => Try(new Url(l)).isSuccess links.foreach(requester ! Url(_, depth + 1)) }}
Building a web crawlerakka { actor.deployment { /parsers { router = round-robin-pool nr-of-instances = 5 } /fetchers { router = round-robin-pool nr-of-instances = 5 } }}
Live demo #1
Going remote• Everything works using asynchronous
message passing which is good for remoting• Akka-remoting allows working with remote
actors just as if they were in the same JVM• Still need to handle additional issues like
serialization and handling potential networking problems
Akka cluster – pool routing
akka.tcp://localhost:2551/user/crawler
Routees
akka.tcp://localhost:2552/user/crawler
Cluster PoolRouter
localhost:2550
5. Gossip: localhost:2551
is Up
2. Gossip: localhost:2552
is Up
6. Deployroutee
3. Deployroutee
localhost:2551
localhost:2552
1. Joins cluster
4. Joins cluster
/remote/…/(routee)
/remote/…/(routee)
1. Joins cluster
Akka cluster – group routing
akka.tcp://localhost:2551/user/crawler
Routees
akka.tcp://localhost:2552/user/crawler
Cluster GroupRouter
localhost:2550
5. Gossip: localhost:2551
is Up
2. Gossip: localhost:2552
is Up
6. Routesmessages
3. Routesmessages
/user/crawler
localhost:2551
4. Joins cluster
/user/crawler
localhost:2552
Simple crawler clusterClient VM
CrawlClient
Router
Worker VM
CrawlMaster(s)
Worker VM
CrawlMaster(s)
Going remotecluster { seed-nodes = [ "akka.tcp://[email protected]:2551", "akka.tcp://[email protected]:2552"] auto-down-unreachable-after = 10s role { client.min-nr-of-members = 1 backend.min-nr-of-members = 2 }}
Going remoteactor { deployment { /workerRouter { router = consistent-hashing-group nr-of-instances = 100 routees.paths = ["/user/master"] cluster { enabled = on allow-local-routees = on use-role = backend } } } provider = "akka.cluster.ClusterActorRefProvider“}
Live demo #2
Distributed state
Distributed state
But what if we need to?
CRDTs• Good performance and scalability, the cost is
eventual consistency• Two main classes: operation based and state
based
CmRDTs
CvRDTs
Live demo #3
Summary• Actor model provides a concurrency
paradigm that is easier to reason about than traditional Java concurrency
• Designing actor systems is a lot like OO• You can easily make your actor systems
distributed and have referential transparency… to an extent
• Akka has many useful modules, like Akka distributed data, which allows to manage distributed state
• Try to build your own application and see how it works for you
References• http://akka.io/• http://
www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf
• https://vimeo.com/43903960
Q&A
Thank you!Konstantin Tsykulenko
@Tsykulenko_K