up, up and out - scaling software with akka henrik engstrÖm software engineer - typesafe @h3nk3
TRANSCRIPT
Up, up and out - Scaling Software
with Akka
HENRIK ENGSTRÖMSOFTWARE ENGINEER - TYPESAFE
@h3nk3
✦ Typesafe Console Tech Lead
✦ Akka Honorary Team Member
✦ Programming Scala since 2010
$ whoami
✦ Consultant since 1998 - mainly Java
✦ Been with Typesafe since 2011
✦ Arsenal Supporter + Martial Artist
About Typesafe
✦ The Typesafe Platform
‣ Play, Akka, Scala, Scala IDE, Slick, SBT, etc.
✦ Subscription
✦ Training and Consulting
Agenda
✦ Akka Introduction
✦ Core Operations
✦ Let’s Build Something with Actors
✦ Akka Cluster
✦ Typesafe Console
Copyright Ingeborg van Leeuwen
Selection of Akka Production Users
At the core - Actors✦ Carl Hewitt’s definition (1973)
‣ Processing + Storage + Communication
✦ 3 Axioms - an Actor can
- create new actors
- send messages to Actors
- designate how to handle next message
Proven Concept✦ Actors have successfully been
used in the Telecom industry since the 1980s
✦ Unbelievably stable:
‣ 9 nines uptime or ...
‣ 0.999999999% uptime or ...
‣ 32 ms downtime per year
Akka Actors
✦ Akka Actors are extremely lightweight
✦ Each Actor has its own mailbox
✦ Communication done via asynchronous message passing
✦ Very memory efficient
‣ You can have approx. 2.7 million actors/GB
ActorActor MailboxMailbox
Akka Actor in Action
Core Actor Operations
1)DEFINE
2)CREATE
3)SEND
4)SUPERVISE
case class Greeting(who: String)class GreetActor extend Actor with ActorLogging { var counter = 0 def receive = { case Greeting(who) => counter += 1 log.info("Hello %s, counter = %s". format(counter, who)) }}
1) DEFINE
val system = ActorSystem("MyFirstActorSystem")val greetActor: ActorRef = system.actorOf(Props[GreetingActor], "greeter")
2) CREATE
AA
BB
BarBarFooFoo
CC
BBEE
AA
DD
CC
Guardian System Actor
system.actorOf(Props[Foo], "Foo")
context.actorOf(Props[A], "A")
Actors can form hierarchies
AA
BB
BarBarFooFoo
CC
BBEE
AA
DD
CC
/Foo
/Foo/A
/Foo/A/B
/Foo/A/D
Guardian System Actor
Name resolution - like a file-system
Sending✦ Communication done via message
passing
✦ Fire and forget, i.e. asynchronous and non-blocking
✦ Reactive Programming
- messages are kinetic energy
- huge buffered potential energy at rest
Throughput on single box
with 48 cores
+50 million messages per second
greetActor ! Greeting("JavaOneShanghai")// orgreetActor.tell(Greeting("JavaOneShanghai"))
3) SEND
case class Greeting(who: String)class GreetActor extend Actor with ActorLogging { var counter = 0 def receive = { case Greeting(who) => counter += 1 log.info("Hello %s, counter = %s".format(counter, who)) }}val system = ActorSystem("MyFirstActorSystem")val greetActor: ActorRef = system.actorOf(Props[GreetingActor], "greeter")greetActor ! Greeting("JavaOneShanghai")
Full Example
Supervision✦ Manage another Actor’s failures
✦ Notification based
‣ if an Actor crashes a notification will be sent to the supervisor
✦ Clean separation of processing/business logic and error handling
ErrorKernel
class MyActor extends Actor { override val supervisionStrategy = OneForOneStrategy() { case _: ArithmeticException => Resume case _: NullPointerException => Restart case _: IllegalArgumentException => Stop case _: Exception => Escalate } // actor implementation }
4) SUPERVISE
class ActorA extends Actor { val b = context.actorOf(Props[ActorB], "b") context.watch(b) def receive = { case Terminated(a) => // take proper action }}
Death Watch
Akka Configuration
✦ All settings that are programmable are also configurable
‣ A simple restart can totally change the behavior of your application
‣ Will keep your DevOps happy
Let’s Build Something!
✦ Let’s create a hierarchy of words
‣ Each letter becomes one actor
‣ They form a parent child relationship
‣ "hi" becomes Actor("h") as parent to Actor("i")
‣ Each actor has an internal counter to keep track of number of occurrences
Example Hierarchyhhhh
ii22ii22
ee99ee99
llll
ll22ll22
oo44oo44
rooroott
rooroott
ii33ii33
ss1414ss
1414
case class CreateWord(word: String)class LetterActor extends Actor { var count = 0 def receive = { case CreateWord(word) => if(word.isEmpty) count += 1 else { val firstLetter = word.head.toString context.child(firstLetter). getOrElse(context.actorOf(Props[LetterActor], firstLetter)) ! CreateWord(word.tail) } } }
LetterActor
class WordActor extends Actor { def receive = { case CreateWord(word) => val firstLetter = word.head.toString context.child(firstLetter). getOrElse(context.actorOf(Props[LetterActor], firstLetter)) ! CreateWord(word.tail) }}
WordActor
class MainActor extends Actor { val root = context.actorOf(Props[WordActor], "wordActor") def receive = { case cw: CreateWord => root ! cw }}
MainActor
object WordDemo { def main(args: Array[String]) { val system = ActorSystem("WordCount") val main = system.actorOf(Props[MainActor], "mainActor") main ! CreateWord("hello") 1 to 3 foreach { c => main ! CreateWord("hi") } main ! CreateWord("hell") main ! CreateWord("helium") }}
Main Program
Resulting Hierarchy
hhhh eeeellll
ll11ll11
oo11oo11
rooroott
rooroott
ii33ii33
iiii
uuuu
mm11mm11
Akka Cluster
✦ Gossip-based cluster membership
✦ Leader determination
✦ Accrual failure detection
✦ Cluster death watch
✦ Cluster aware routers
Cluster Membership
✦ Node ring style inspired by Riak/Dynamo
✦ Gossip protocol - used for
- membership
- configuration data
- leader determination
✦ Vector clocks to detect convergence
Node Ring with Gossiping Members
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
Gossip
Phi Accrual Failure Detector
• B monitors A
• Sample inter-arrival time to expect next beat
• B measures continuum of deadness of A
AA BBregular messages
http://ddg.jaist.ac.jp/pub/HDY+04.pdf
Selective Failure Detection
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
Heartbeat
Selective Failure Detection
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
MembMemberer
NodeNode
Heartbeat
Cluster Death Watch
✦ Triggered by marking node <A> DOWN
‣ Tell parents of their lost children on <A>
‣ Delete all children of actors on <A>
‣ Send TERMINATED for actors on <A>
akka { actor { provider = "akka.cluster.ClusterActorRefProvider" ... } extensions = ["akka.cluster.Cluster"] cluster { seed-nodes = [ "akka://[email protected]:2551", "akka://[email protected]:2552" ] auto-down = on }}
Enable Clustering
Typesafe Console
✦ Monitoring tool for Akka (and soon Play) applications
✦ Low overhead - always turned on
✦ Traces everything in your Akka application
✦ UI or REST service to retrieve statistics
✦ Free to use for developers via Activator
Typesafe Console
Dataflow
Akka has much much moreFSM
Transactors
Pub/Sub
ZeroMQ
Microkernel
IO
TestKit
Agents
SLF4J
Durable Mailboxes
EventBus
Camel
Pooling
TypedActor
Extensions
Typed Channels
EOF@h3nk3