designing for distributed systems with reactor and reactive streams

Designing  for  Distributed  Systems  with  Reactor  and  Reactive  Streams

Stephane  Maldini  @smaldini

1

WHY  DISTRIBUTED  SYSTEMS  ARE  THE  NEW  NORMAL  ?

SCALING  MONOLITHIC  APPS

Payments Adult

E-­‐Learning Entertainment

Music Search

Payments Adult

E-­‐Learning Entertainment

Music Search

Payments Adult

E-­‐Learning Entertainment

Music Search

Payments Adult

E-­‐Learning Entertainment

Music Search

HOW  YOU  SHOULD  SCALE  ?

HOW  YOU  SHOULD  SCALE  ?

Enternaiment

E-­‐Learning

Payments

Adult

Music

Search

Requests

MICROSERVICES

Dependencies ?

The  Speed  Of  Light

I wish I acted in Star Trek Teleporting anyone ?

There’s  latency  between  each  remote  call

Let’s  use  asynchronous  processing

There’s  latency  between  each  remote  call

Let’s  use  asynchronous  processing

I  shall  not  block  incoming  requests  to  keep  serving

There’s  latency  between  each  remote  call

Let’s  use  asynchronous  processing

I  shall  not  block  incoming  requests  to  keep  serving

Thread  Executor  !

There’s  latency  between  each  remote  call

private ExecutorService threadPool = Executors.newFixedThreadPool(2); final List<T> batches = new ArrayList<T>();

//…

Callable<T> t = new Callable<T>() {

public T run() { T result = callDatabase(); synchronized(batches) { batches.add(result); return result; } } };

Future<T> f = threadPool.submit(t); T result = f.get()

Vanilla  background-­‐processing

private ExecutorService threadPool = Executors.newFixedThreadPool(2); final List<T> batches = new ArrayList<T>();

//…

Callable<T> t = new Callable<T>() {

public T run() { T result = callDatabase(); synchronized(batches) { batches.add(result); return result; } } };

Future<T> f = threadPool.submit(t); T result = f.get()

New  Allocation  By  Request

Vanilla  background-­‐processing

private ExecutorService threadPool = Executors.newFixedThreadPool(2); final List<T> batches = new ArrayList<T>();

//…

Callable<T> t = new Callable<T>() {

public T run() { T result = callDatabase(); synchronized(batches) { batches.add(result); return result; } } };

Future<T> f = threadPool.submit(t); T result = f.get()

New  Allocation  By  Request

Queue-­‐based  message  passing

Vanilla  background-­‐processing

private ExecutorService threadPool = Executors.newFixedThreadPool(2); final List<T> batches = new ArrayList<T>();

//…

Callable<T> t = new Callable<T>() {

public T run() { T result = callDatabase(); synchronized(batches) { batches.add(result); return result; } } };

Future<T> f = threadPool.submit(t); T result = f.get()

New  Allocation  By  Request

Queue-­‐based  message  passing

Vanilla  background-­‐processing

What  if  this  message  fails  ?

private ExecutorService threadPool = Executors.newFixedThreadPool(2); final List<T> batches = new ArrayList<T>();

//…

Callable<T> t = new Callable<T>() {

public T run() { T result = callDatabase(); synchronized(batches) { batches.add(result); return result; } } };

Future<T> f = threadPool.submit(t); T result = f.get()

New  Allocation  By  Request

Queue-­‐based  message  passing

Vanilla  background-­‐processing

What  if  this  message  fails  ?

censored

Mixing  latency  with  queue  based  handoff

http://ferd.ca/queues-don-t-fix-overload.html

Mixing  latency  with  queue  based  handoff

http://ferd.ca/queues-don-t-fix-overload.html

Mixing  latency  with  queue  based  handoff

http://ferd.ca/queues-don-t-fix-overload.html

Queues  must  be  bounded

Passing  message  /w  little  overhead

Queues  must  be  bounded

Passing  message  /w  little  overhead

Tolerating  some  consuming  rate  difference

Queues  must  be  bounded

Passing  message  /w  little  overhead

Tolerating  some  consuming  rate  difference

Reactor  !

Queues  must  be  bounded

Pre-­‐allocating  Slots  -­‐>  The  Ring  Buffer

Publishing events to the right slot

dude!

Re-­‐using  Threads  too  :  Event  Loop

I’m an event loop, consuming messages in the right order

//RingBufferProcessor with 32 slots by default RingBufferProcessor<Integer> processor = RingBufferProcessor.create(); //Subscribe to receive events processor.subscribe( //Create a subscriber from a lambda/method ref SubscriberFactory.unbounded((data, s) ->

System.out.println(data) )

);

//Dispatch data asynchronously int i = 0; while(i++ < 100000) processor.onNext(i)

//Terminate the processor processor.shutdown();

//a second subscriber to receive the same events in a distinct thread processor.subscribe( SubscriberFactory.unbounded((data, s) -> {

//a slow callback returning false when not interested in data anymore if(!sometimeSlow(data)){ //shutdown the consumer thread s.cancel();

}

}) );

Hold  on  !  The  guy  said  bounded  number  of  slots

So  we  still  block  when  the  buffer  is  full!

Hold  on  !  The  guy  said  bounded  number  of  slots

So  we  still  block  when  the  buffer  is  full!

…So  why  sending  more  requests

Hold  on  !  The  guy  said  bounded  number  of  slots

So  we  still  block  when  the  buffer  is  full!

…So  why  sending  more  requests

Reactive  Streams!

Hold  on  !  The  guy  said  bounded  number  of  slots

What  is  defined  in  Reactive  Streams

Async non-blocking data sequence

What  is  defined  in  Reactive  Streams

Async non-blocking data sequence

Minimal resources requirement

What  is  defined  in  Reactive  Streams

Interoperable protocol (Threads, Nodes…)Async non-blocking data sequence

Minimal resources requirement

What  is  defined  in  Reactive  Streams

Async non-blocking flow-control

Interoperable protocol (Threads, Nodes…)Async non-blocking data sequence

Minimal resources requirement

What  is  defined  in  Reactive  Streams

Async non-blocking flow-control

Interoperable protocol (Threads, Nodes…)Async non-blocking data sequence

Minimal resources requirement

What  is  defined  in  Reactive  Streams

What  is  reactive  flow  control  ?

PUBLISHER

What  is  reactive  flow  control  ?

PUBLISHER

SUBSCRIBER

What  is  reactive  flow  control  ?

PUBLISHER

SUBSCRIBER

What  is  reactive  flow  control  ?

SUBSCRIPTION

PUBLISHER

SUBSCRIBER

Events

What  is  reactive  flow  control  ?

SUBSCRIPTION

PUBLISHER

SUBSCRIBER

Events

What  is  reactive  flow  control  ?

SUBSCRIPTION

PUBLISHER

SUBSCRIBER

Events

Demand

What  is  reactive  flow  control  ?

SUBSCRIPTION

All  of  that  in  4  interfaces!

And  there’s  a  TCK  to  verify  implementations

All  of  that  in  4  interfaces!

And  there’s  a  TCK  to  verify  implementations

No  More  Unwanted  Requests

All  of  that  in  4  interfaces!

And  there’s  a  TCK  to  verify  implementations

No  More  Unwanted  Requests

Need  to  propagate  that  demand  upstream!

All  of  that  in  4  interfaces!

ccv

Doug Lea – SUNY Oswego

Some  Smart  Guys  Involved

RingBufferProcessor<Integer> processor = RingBufferProcessor.create(); //Subscribe to receive event […]

//Data access gated by a Publisher with backpressure PublisherFactory.forEach( sub -> { if(sub.context().hasNext()) sub.onNext(sub.context().readInt());

else sub.onComplete();

}, sub -> sqlContext(), context -> context.close()

) .subscribe(processor);

//Terminate the processor [..]

RingBufferProcessor<Integer> processor = RingBufferProcessor.create(); //Subscribe to receive event […]

//Data access gated by a Publisher with backpressure PublisherFactory.forEach( sub -> { if(sub.context().hasNext()) sub.onNext(sub.context().readInt());

else sub.onComplete();

}, sub -> sqlContext(), context -> context.close()

) .subscribe(processor);

//Terminate the processor [..]Connect  processor  to  this  publisher  and  start  requesting

RingBufferProcessor<Integer> processor = RingBufferProcessor.create(); //Subscribe to receive event […]

//Data access gated by a Publisher with backpressure PublisherFactory.forEach( sub -> { if(sub.context().hasNext()) sub.onNext(sub.context().readInt());

else sub.onComplete();

}, sub -> sqlContext(), context -> context.close()

) .subscribe(processor);

//Terminate the processor [..]Connect  processor  to  this  publisher  and  start  requesting

For  the  new  connected  processor,  create  some  sql  

context

RingBufferProcessor<Integer> processor = RingBufferProcessor.create(); //Subscribe to receive event […]

//Data access gated by a Publisher with backpressure PublisherFactory.forEach( sub -> { if(sub.context().hasNext()) sub.onNext(sub.context().readInt());

else sub.onComplete();

}, sub -> sqlContext(), context -> context.close()

) .subscribe(processor);

//Terminate the processor [..]Connect  processor  to  this  publisher  and  start  requesting

For  the  new  connected  processor,  create  some  sql  

context

Keep  invoking  this  callback  until  there  is  no  more  

pending  request

What  about  combining  multiple  asynchronous  calls

And  everything  in  a  controlled  fashion

What  about  combining  multiple  asynchronous  calls

And  everything  in  a  controlled  fashion

Including  Errors  and  Completion  

What  about  combining  multiple  asynchronous  calls

And  everything  in  a  controlled  fashion

Including  Errors  and  Completion  

Reactive  Extensions  !

What  about  combining  multiple  asynchronous  calls

FlatMap  and  Monads..  Nooooo  Please  No

24

Streams.just(‘doge’).flatMap{ name -> Streams.just(name) .observe{ println 'so wow' } .map{ 'much monad'} }.consume{ assert it == 'much monad' }

FlatMap  and  Monads..  Nooooo  Please  No

24

Streams.just(‘doge’).flatMap{ name -> Streams.just(name) .observe{ println 'so wow' } .map{ 'much monad'} }.consume{ assert it == 'much monad' }

A publisher that only sends “doge” on request

FlatMap  and  Monads..  Nooooo  Please  No

24

Streams.just(‘doge’).flatMap{ name -> Streams.just(name) .observe{ println 'so wow' } .map{ 'much monad'} }.consume{ assert it == 'much monad' }

A publisher that only sends “doge” on request

Sub-Stream definition

FlatMap  and  Monads..  Nooooo  Please  No

24

Streams.just(‘doge’).flatMap{ name -> Streams.just(name) .observe{ println 'so wow' } .map{ 'much monad'} }.consume{ assert it == 'much monad' }

A publisher that only sends “doge” on request

Sub-Stream definition

All Sub-Streams are merged under a single sequence

Scatter  Gather  and  Fault  Tolerance

25

Streams.merge(      userService.filteredFind(“Rick"),  //  Stream  of  User      userService.filteredFind(“Morty")  //  Stream  of  User  )  .buffer()  //  Accumulate  all  results  in  a  List  .retryWhen(  errors  -­‐>  //Stream  of  Errors      errors      .zipWith(Streams.range(1,3),  t  -­‐>  t.getT2())          .flatMap(  tries  -­‐>  Streams.timer(tries)  )  )  .consume(System.out::println);

Scatter  Gather  and  Fault  Tolerance

25

Interleaved merge from 2 upstream publishers

Streams.merge(      userService.filteredFind(“Rick"),  //  Stream  of  User      userService.filteredFind(“Morty")  //  Stream  of  User  )  .buffer()  //  Accumulate  all  results  in  a  List  .retryWhen(  errors  -­‐>  //Stream  of  Errors      errors      .zipWith(Streams.range(1,3),  t  -­‐>  t.getT2())          .flatMap(  tries  -­‐>  Streams.timer(tries)  )  )  .consume(System.out::println);

Scatter  Gather  and  Fault  Tolerance

25

Interleaved merge from 2 upstream publishers

Up to 3 tries

Streams.merge(      userService.filteredFind(“Rick"),  //  Stream  of  User      userService.filteredFind(“Morty")  //  Stream  of  User  )  .buffer()  //  Accumulate  all  results  in  a  List  .retryWhen(  errors  -­‐>  //Stream  of  Errors      errors      .zipWith(Streams.range(1,3),  t  -­‐>  t.getT2())          .flatMap(  tries  -­‐>  Streams.timer(tries)  )  )  .consume(System.out::println);

Scatter  Gather  and  Fault  Tolerance

25

Interleaved merge from 2 upstream publishers

Up to 3 tries

All Sub-Streams are merged under a single sequence

Streams.merge(      userService.filteredFind(“Rick"),  //  Stream  of  User      userService.filteredFind(“Morty")  //  Stream  of  User  )  .buffer()  //  Accumulate  all  results  in  a  List  .retryWhen(  errors  -­‐>  //Stream  of  Errors      errors      .zipWith(Streams.range(1,3),  t  -­‐>  t.getT2())          .flatMap(  tries  -­‐>  Streams.timer(tries)  )  )  .consume(System.out::println);

Scatter  Gather  and  Fault  Tolerance

25

Interleaved merge from 2 upstream publishers

Up to 3 tries

All Sub-Streams are merged under a single sequence

Streams.merge(      userService.filteredFind(“Rick"),  //  Stream  of  User      userService.filteredFind(“Morty")  //  Stream  of  User  )  .buffer()  //  Accumulate  all  results  in  a  List  .retryWhen(  errors  -­‐>  //Stream  of  Errors      errors      .zipWith(Streams.range(1,3),  t  -­‐>  t.getT2())          .flatMap(  tries  -­‐>  Streams.timer(tries)  )  )  .consume(System.out::println);

Delay retry

REACTOR2™  =    REACTOR  +  REACTIVE  EXTENSIONS  +  REACTIVE  STREAMS

reactor-net

reactor-streams

reactor-bus

reactor-core

reactor-net

reactor-streams

reactor-bus

reactor-core

Event Bus

Core Dispatchers & Processors

Streams and Promises

reactor-net

reactor-streams

reactor-bus

reactor-coreFunctional artifacts

(SAM components, tuples, timers)

Event Bus

Core Dispatchers & Processors

Streams and Promises

reactor-net

reactor-streams

reactor-bus

reactor-coreFunctional artifacts

(SAM components, tuples, timers)

Event Bus

Core Dispatchers & Processors

Streams and Promises

NetStreams [ Clients/Server ] [TCP, UDP, HTTP]

reactor-net

reactor-streams

reactor-bus

reactor-coreFunctional artifacts

(SAM components, tuples, timers)

Event Bus

Core Dispatchers & Processors

Streams and Promises

NetStreams [ Clients/Server ] [TCP, UDP, HTTP]

Fast Data [buffer, codec]

reactor-net

reactor-streams

reactor-bus

reactor-coreFunctional artifacts

(SAM components, tuples, timers)

Event Bus

Core Dispatchers & Processors

Streams and Promises

NetStreams [ Clients/Server ] [TCP, UDP, HTTP]

Fast Data [buffer, codec]

reactor-net

reactor-streams

reactor-bus

reactor-coreFunctional artifacts

(SAM components, tuples, timers)

Event Bus

Core Dispatchers & Processors

Streams and Promises

NetStreams [ Clients/Server ] [TCP, UDP, HTTP]

Fast Data [buffer, codec]

ASYNC  IO  ?

30

NetStreams.<String,  String>httpServer(spec  -­‐>        spec.codec(StandardCodecs.STRING_CODEC).listen(3000)  ).     ws("/",  channel  -­‐>  {       System.out.println("Connected  a  websocket  client:  "  +  channel.remoteAddress());  

    return  somePublisher.         window(1000).         flatMap(s  -­‐>  channel.writeWith(  s.  

reduce(0f,  (prev,  trade)  -­‐>  (trade.getPrice()  +  prev)  /  2).  map(Object::toString)  )  

);     }).     start().     await();

30

NetStreams.<String,  String>httpServer(spec  -­‐>        spec.codec(StandardCodecs.STRING_CODEC).listen(3000)  ).     ws("/",  channel  -­‐>  {       System.out.println("Connected  a  websocket  client:  "  +  channel.remoteAddress());  

    return  somePublisher.         window(1000).         flatMap(s  -­‐>  channel.writeWith(  s.  

reduce(0f,  (prev,  trade)  -­‐>  (trade.getPrice()  +  prev)  /  2).  map(Object::toString)  )  

);     }).     start().     await();

Listen  on  port  3000  and  convert  bytes  into  String  inbound/

outbound

30

NetStreams.<String,  String>httpServer(spec  -­‐>        spec.codec(StandardCodecs.STRING_CODEC).listen(3000)  ).     ws("/",  channel  -­‐>  {       System.out.println("Connected  a  websocket  client:  "  +  channel.remoteAddress());  

    return  somePublisher.         window(1000).         flatMap(s  -­‐>  channel.writeWith(  s.  

reduce(0f,  (prev,  trade)  -­‐>  (trade.getPrice()  +  prev)  /  2).  map(Object::toString)  )  

);     }).     start().     await();

Listen  on  port  3000  and  convert  bytes  into  String  inbound/

outbound

Upgrade  clients  to  websocket  on  root  URI

30

NetStreams.<String,  String>httpServer(spec  -­‐>        spec.codec(StandardCodecs.STRING_CODEC).listen(3000)  ).     ws("/",  channel  -­‐>  {       System.out.println("Connected  a  websocket  client:  "  +  channel.remoteAddress());  

    return  somePublisher.         window(1000).         flatMap(s  -­‐>  channel.writeWith(  s.  

reduce(0f,  (prev,  trade)  -­‐>  (trade.getPrice()  +  prev)  /  2).  map(Object::toString)  )  

);     }).     start().     await();

Listen  on  port  3000  and  convert  bytes  into  String  inbound/

outbound

Upgrade  clients  to  websocket  on  root  URI

Flush  every  1000  data  some  data  with  writeWith  +  window

30

NetStreams.<String,  String>httpServer(spec  -­‐>        spec.codec(StandardCodecs.STRING_CODEC).listen(3000)  ).     ws("/",  channel  -­‐>  {       System.out.println("Connected  a  websocket  client:  "  +  channel.remoteAddress());  

    return  somePublisher.         window(1000).         flatMap(s  -­‐>  channel.writeWith(  s.  

reduce(0f,  (prev,  trade)  -­‐>  (trade.getPrice()  +  prev)  /  2).  map(Object::toString)  )  

);     }).     start().     await();

Listen  on  port  3000  and  convert  bytes  into  String  inbound/

outbound

Upgrade  clients  to  websocket  on  root  URI

Flush  every  1000  data  some  data  with  writeWith  +  windowClose  connection  when  

flatMap  completes,  which  is  when  all  Windows  are  done

THE  STATE  OF  THE  ART

Nowreactive-­‐streams.1.0.0  

50%  guide  complete  :  http://projectreactor.io/docs/reference    

reactor-­‐*.2.0.3.RELEASE  -­‐  2.0.4  around  the  corner  

reactor-­‐*.2.1.0.BUILD-­‐SNAPSHOT  -­‐  early  access,  no  breaking  changes  

Now

Initial  Reactor  2  Support  in  :  Spring  Integration  4.2  Spring  Messaging  4.2  Spring  Boot  1.3    Spring  XD  1.2  Grails  3.0

After  Now

Spring  Integration  DSL  +  Reactive  Streams  Dynamic  Subscribers  on  Predefined  Channels  !  Best  Tool  for  each  job:  SI  for  integrating  Reactor  for  scaling  up  too  fast  to  be  true

SI  Java  DSL  +  Reactive  Stream  Preview@Configuration @EnableIntegration public static class ContextConfiguration {

@Autowired private TaskScheduler taskScheduler;

@Bean public Publisher<Message<String>> reactiveFlow() { return IntegrationFlows .from(“inputChannel”) .split(String.class, p -> p.split(",")) .toReactiveStreamsPublisher(); }

@Bean public Publisher<Message<Integer>> pollableReactiveFlow() { return IntegrationFlows .from("inputChannel") .split(e -> e.get().getT2().setDelimiters(",")) .<String, Integer>transform(Integer::parseInt) .channel(Channels::queue) .toReactiveStreamsPublisher(this.taskScheduler); }

}

After  Now

Reactor  +  Spring  Cloud  •  Annotation  Driven  FastData  •  Async  IO  :  (proxy,  client)  •  Circuit  breaker,  Bus,  …  

Reactor  +  Spring  XD  •  Scale  up  any  XD  pipeline  •Reactive  Backpressure  in  XD

37

@EnableReactorModule(concurrency  =  5)  public  class  PongMessageProcessor  implements  ReactiveProcessor<Message,  Message>  {         @Override     public  void  accept(Stream<Message>  inputStream,  ReactiveOutput<Message>  output)  {       output.writeOutput(           inputStream               .map(simpleMap())               .observe(simpleMessage())       );     }  

  //…  }  

37

@EnableReactorModule(concurrency  =  5)  public  class  PongMessageProcessor  implements  ReactiveProcessor<Message,  Message>  {         @Override     public  void  accept(Stream<Message>  inputStream,  ReactiveOutput<Message>  output)  {       output.writeOutput(           inputStream               .map(simpleMap())               .observe(simpleMessage())       );     }  

  //…  }  

Split  input  XD  channel  in  5  threads  !  With  a  blazing  fast  Processor

37

@EnableReactorModule(concurrency  =  5)  public  class  PongMessageProcessor  implements  ReactiveProcessor<Message,  Message>  {         @Override     public  void  accept(Stream<Message>  inputStream,  ReactiveOutput<Message>  output)  {       output.writeOutput(           inputStream               .map(simpleMap())               .observe(simpleMessage())       );     }  

  //…  }  

Split  input  XD  channel  in  5  threads  !  With  a  blazing  fast  Processor

Register  the  sequence  to  write  on  the  output  channel  after  some  operations

After

<3  Spring  5  +  Reactor  &  Reactive  Streams  <3

After

Reactive  IPC  for  the  JVM  <3  RxNetty  +  reactor-­‐net  <3  

Reactor  2.0.3.+  already  previews  the  concept  and  API  flavor…  

because  REACTOR  DOESN’T  KNOW  WAITING  (LOL)

Future  ??

• RxJava  2.0  timeline  (2016  and  after?)  • Thanks  Interop,  Reactive  Extensions  and  Naming  conventions,  it  can  converge  with  reactor-­‐streams  

• https://github.com/ReactiveX/RxJava/wiki/Reactive-­‐Streams  

Take-­‐away

• Distributed  System  is  the  new  cool  and  comes  at  some  cost,  two  big  ones  are  Latency  and  Failure  Tolerance  

• Asynchronous  Processing  and  Error  Handling  by  Design  deal  with  these  two  problems  -­‐>  Reactor,  Reactive  Extensions  /  Streams  

• However  to  fully  operate,  Asynchronous  Processing  should  be  bounded  proactively  (stop-­‐read)  -­‐>  Reactive  Streams

REACTIVE

Extra

Take-­‐away  links

• http://reactive-­‐streams.org    • http://projectreactor.io    • https://github.com/spring-­‐projects/spring-­‐integration-­‐java-­‐dsl/pull/31  

• https://github.com/smaldini/spring-­‐xd/tree/refresh-­‐reactor-­‐module/spring-­‐xd-­‐reactor/    

• https://github.com/reactive-­‐ipc/reactive-­‐ipc-­‐jvm    • https://github.com/smaldini/ReactiveStreamsExamples