akka: distributed by design

Patrik Nordwall@patriknw

Distributed by Design

actorsremote actors

rise of the clusterwhen the cluster grows up

outline

What is an Actor?

• Akka's unit of computation is called an Actor

• Actors are purely reactive components:– a mailbox– behavior & state– scheduled to run when sent a message

• Each actor has a parent, handling its failures

Event-drivenThread

Event-drivenThread

Behavior

State

Actor

public class Greeting implements Serializable {public final String who;public Greeting(String who) { this.who = who; }

}

public class GreetingActor extends UntypedActor { LoggingAdapter log = Logging.getLogger(getContext().system(), this); int counter = 0;

public void onReceive(Object message) {if (message instanceof Greeting) { counter++;

log.info("Hello #" + counter + " " + ((Greeting) message).who);}

}}

Define ActorDefine the message(s) the Actor

should be able to respond to

Define the Actor class

Define the Actor’s behavior


}


public void onReceive(Object message) {if (message instanceof Greeting) {

counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who);

}}

}

ActorSystem system = ActorSystem.create("MySystem");ActorRef greeter = system.actorOf(new Props(GreetingActor.class), "greeter");

Create Actor

Create an Actor system

Create the Actor

Actor configuration

Give it a nameYou get an ActorRef back

A

B

BarFoo

C

B E

A

D

C

Guardian System Actor

system.actorOf( new Props(Foo.class), “Foo”);

getContext().actorOf( new Props(A.class), “A”);

Actors can form hierarchies

A

B

BarFoo

C

B E

A

D

C

/Foo

/Foo/A

/Foo/A/B

/Foo/A/D

Guardian System Actor

Name resolution - like a file-system


}


public void onReceive(Object message) {if (message instanceof Greeting) { counter++; log.info("Hello #" + counter + " " + ((Greeting) message).who);}

}}

ActorSystem system = ActorSystem.create("MySystem");ActorRef greeter = system.actorOf(new Props(GreetingActor.class), "greeter");greeter.tell(new Greeting("Charlie Parker"), null);

Send Message

Send the message


}


public void onReceive(Object message) {if (message instanceof Greeting) { counter++;

log.info("Hello #" + counter + " " + ((Greeting) message).who);} else { unhandled(message);}

}}

ActorSystem system = ActorSystem.create("MySystem");ActorRef greeter = system.actorOf(new Props(GreetingActor.class), "greeter");greeter.tell(new Greeting("Charlie Parker"), null);

Full example

Distributableby Design

ErrorKernel

Node 1 Node 2

Create Actor

ActorRef greeter = system.actorOf(new Props( GreetingActor.class), "greeter");

akka { actor { provider = akka.remote.RemoteActorRefProvider deployment { /greeter { remote = } } }}

Just feed the ActorSystem with this configuration

Zero code changes

Configure a Remote Provider

Define Remote Path Protocol Actor System Hostname Port

For the Greeter actor

akka://MySystem@machine1:2552

Remote Deployment

Remote Lookup

ActorRef greeter = system.actorFor( "akka://MySystem@machine1:2552/user/greeter" );

Can you see the problem?

Fixed Addresses

ActorRef greeter = system.actorFor( "akka://MySystem@machine1:2552/user/greeter" );

akka { actor { provider = akka.remote.RemoteActorRefProvider deployment { /greeter { remote = akka://MySystem@machine1:2552

} } }}

Akka Cluster

Akka Cluster 2.1

• Gossip-based Cluster Membership

• Failure Detector

• Cluster DeathWatch

• Cluster-Aware Routers

Cluster is experimental preview in 2.1

Cluster Membership

• Node ring à la Riak / Dynamo

• Gossip-protocol for state dissemination

• Vector Clocks to detect convergence

Node ring with gossiping members

MemberNode

MemberNode

MemberNode

MemberNode

MemberNode

MemberNode

MemberNode

MemberNode

MemberNode

MemberNode

Gossip

Vector Clock

• Vector Clocks are used to:

- Generate a partial ordering of events in a distributed system

- Detecting causality violations

• We use Vector Clocks to to reconcile and merge differences in cluster state

Gossiping Protocol

Used for :– Cluster Membership– Configuration data– Leader Determination– Partitioning data– Naming Service

Push/Pull Gossip

• Push– sender only sends versions (Vector Clock)

• Pull– receiver only asks for information for which it has

an outdated version

• Partly biased– send fraction of gossip to nodes with older state

Cluster Convergence

• When each Node has seen the same Vector Clock

• unreachable nodes will fail this

• mark nodes DOWN to proceed

– manual Ops intervention– automatic action

Member States

• JOINING

• UP

• LEAVING

• EXITING

• DOWN

• REMOVED

• Any node can be the leader

• Just takes the role of being a leader

• Is deterministically recognized by all nodes

– always the first member in the sorted membership ring

Leader

Cluster Events

public class Listener extends UntypedActor { public void onReceive(Object message) { if (message instanceof MemberUp) { // ... } }}

ActorRef listener = system.actorOf(new Props(Listener.class), "listener");

Cluster.get(system).subscribe(listener, MemberEvent.class);

Cluster Events

public class Listener extends UntypedActor { public void onReceive(Object message) { if (message instanceof MemberUp) { MemberUp mUp = (MemberUp) message; getContext().actorFor(mUp.address() + "/user/greeter").tell( new Greeting("Charlie Parker"), getSelf()); } }}

ActorRef listener = system.actorOf(new Props(Listener.class), "listener");

Cluster.get(system).subscribe(listener, MemberEvent.class);

Phi Accrual Failure Detector

• B monitors A

• Sample inter-arrival time to expect next beat

• B measures continuum of deadness of A

A Bregular messages

http://ddg.jaist.ac.jp/pub/HDY+04.pdf



Selective Failure Detection

MemberNode

MemberNode

MemberNode

MemberNode

MemberNode

MemberNode

MemberNode

MemberNode

MemberNode

MemberNode

Heartbeat

Cluster DeathWatch

• Triggered by marking node «A» DOWN– Tell parents of their lost children on «A»– Kill all children of actors on «A»– Send Terminated for actors on «A»

Enable clusteringakka { actor { provider = "akka.cluster.ClusterActorRefProvider" ... } extensions = ["akka.cluster.Cluster"] cluster { seed-nodes = [ "akka://[email protected]:2551", "akka://[email protected]:2552" ] }}

mailto:[email protected]




Load Balancing

Routers

ActorRef routerActor = getContext().actorOf( new Props(ExampleActor.class). withRouter(new RoundRobinRouter(nrOfInstances)) );

…or from config

akka.actor.deployment { /path/to/actor { router = round-robin nr-of-instances = 5 } }

Configure a clustered router

akka.actor.deployment { /statsService/workerRouter { router = consistent-hashing nr-of-instances = 100

cluster { enabled = on max-nr-of-instances-per-node = 3 allow-local-routees = on } }}

Multi Node Testingobject MultiNodeSampleConfig extends MultiNodeConfig { val node1 = role("node1") val node2 = role("node2")}

"A MultiNodeSample" must {

"wait for all nodes to enter a barrier" in { enterBarrier("startup") }

}

Multi Node Testing "send to and receive from a remote node" in { runOn(node1) { enterBarrier("deployed") val ponger = system.actorFor(node(node2) / "user" / "ponger") ponger ! "ping" expectMsg("pong") }

runOn(node2) { system.actorOf(Props(new Actor { def receive = { case "ping" => sender ! "pong" } }), "ponger") enterBarrier("deployed") }

… when the Cluster grows up

Adaptive Load Balancing

• Metrics collected and spread– Heap memory– CPU, system load

• Adaptive Router– Biased random with weights based on capacity

One tree to rule them all

• One Actor tree per node

• Cluster tree is mapped to local sub-trees

One tree to rule them all

The Magic Sauce

• User code only sees cluster://... names

• ActorRef becomes repointable– local– remote

• Can now move actors around transparently– Actor encapsulation makes it possible

What does this enable?

• Actor migration

• Actor replication

• Automatic cluster partitioning– later also based on runtime metrics

• Node fail-over– first for stateless actors– later for stateful actors using event sourcing

➾ Fault Tolerance & Distribution

doc.akka.io/docs/akka/snapshot/cluster/cluster.html

Cluster Specification

doc.akka.io/docs/akka/snapshot/cluster/cluster-usage-scala.htmldoc.akka.io/docs/akka/snapshot/cluster/cluster-usage-java.html

Cluster User Guide

github.com/akka/akka/tree/master/akka-cluster

Cluster Code

http://doc.akka.io/docs/akka/snapshot/cluster/cluster.html





https://github.com/akka/akka/tree/master/akka-cluster

https://github.com/akka/akka/tree/master/akka-cluster

get it and learn morehttp://akka.io

http://typesafe.com

http://letitcrash.com

http://jonasboner.com/






akka: distributed by design

Documents