@crichardson
Consuming web services asynchronously with Futures
and Rx ObservablesChris Richardson
Author of POJOs in ActionFounder of the original CloudFoundry.com
@[email protected]://plainoldobjects.com
@crichardson
Presentation goal
Learn how to use (Scala) Futures and Rx
Observables to write simple yet robust and scalable
concurrent code
@crichardson
About Chris
@crichardson
(About Chris)
@crichardson
About Chris()
@crichardson
About Chris
@crichardson
About Chris
http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
@crichardson
About Chris
@crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
@crichardson
Let’s imagine you are building an online store
@crichardson
Shipping
Recomendations
ProductInfo
Reviews
@crichardson
Reviews
ProductInfo Sales
ranking
@crichardson
Related books
Viewing history
Forum
@crichardson
+ mobile apps
@crichardson
Application architecture
Product Info ServiceDesktop browser
Native mobile client
REST
REST
REST
Recomendation Service
Review Service
Front end server
API gatewayREST
Mobile browser
Web Application
HTML/JSON
JSON
@crichardson
Handling getProductDetails()
Front-end server
Product Info Service
RecommendationsService
ReviewService
getProductInfo()
getRecommendations()
getReviews()
getProductDetails()
@crichardson
Handling getProductDetails() - sequentially
Front-end server
Product Info Service
RecommendationsService
ReviewService
getProductInfo()
getRecommendations()
getReviews()
getProductDetails()
Higher response time :-(
@crichardson
Handling getProductDetails() - in parallel
Front-end server
Product Info Service
RecommendationsService
ReviewService
getProductInfo()
getRecommendations()
getReviews()
getProductDetails()
Lower response time :-)
@crichardson
Implementing a concurrent REST client
Thread-pool based approach
executorService.submit(new Callable(...))
Simpler but less scalable - lots of idle threads consuming memory
Event-driven approach
NIO with completion callbacks
More complex but more scalable
@crichardson
The front-end server must handle partial failure of backend services
Front-end server
Product Info Service
RecommendationsService
ReviewService
getProductInfo()
getRecommendations()
getReviews()
getProductDetails() XHow to provide a good user experience?
@crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
@crichardson
Futures are a great concurrency abstraction
http://en.wikipedia.org/wiki/Futures_and_promises
@crichardson
Worker threadMain threadHow futures work
Outcome
Future
Client
get
Asynchronous operation
set
initiates
@crichardson
Benefits
Client does not know how the asynchronous operation is implemented
Client can invoke multiple asynchronous operations and gets a Future for each one.
@crichardson
Handling ProductDetails request
ProductDetailsController
ProductDetailsService
getProductDetails()
ProductInfoService
getProductInfo()
ReviewService
getReviews()
RecommendationService
getRecommendations()
RestTemplate
@crichardson
REST client using Spring @Async
@Componentclass ProductInfoServiceImpl extends ProducInfoService { val restTemplate : RestTemplate = ...
@Async def getProductInfo(productId: Long) = { new AsyncResult(restTemplate.getForObject(....)...) }
}
Execute asynchronously in
thread pool
A fulfilled Future
trait ProductInfoService { def getProductInfo(productId: Long):
java.util.concurrent.Future[ProductInfo]}
@crichardson
ProductDetailsService@Componentclass ProductDetailsService @Autowired()(productInfoService: ProductInfoService, reviewService: ReviewService, recommendationService: RecommendationService) {
def getProductDetails(productId: Long): ProductDetails = { val productInfoFuture = productInfoService.getProductInfo(productId)
}
}
val recommendationsFuture = recommendationService.getRecommendations(productId)val reviewsFuture = reviewService.getReviews(productId)
val productInfo = productInfoFuture.get(300, TimeUnit.MILLISECONDS) val recommendations = recommendationsFuture.get(10, TimeUnit.MILLISECONDS) val reviews = reviewsFuture.get(10, TimeUnit.MILLISECONDS)
ProductDetails(productInfo, recommendations, reviews)
@crichardson
ProductController
@Controllerclass ProductController @Autowired()(productDetailsService : ProductDetailsService) {
@RequestMapping(Array("/productdetails/{productId}")) @ResponseBody def productDetails(@PathVariable productId: Long) =
productDetailsService.getProductDetails(productId)
@crichardson
Blocks Tomcat thread until Future completes
Not bad but...
class ProductDetailsService def getProductDetails(productId: Long): ProductDetails = {
val productInfo = productInfoFuture.get(300, TimeUnit.MILLISECONDS)
Not so scalable :-(
@crichardson
... and also...
Java Futures work well for a single-level of asynchronous execution
BUT #fail for more complex, scalable scenarios
Difficult to compose and coordinate multiple concurrent operations
See this blog post for more details:http://techblog.netflix.com/2013/02/rxjava-netflix-api.html
@crichardson
Better: Futures with callbacks ⇒ no blocking! val f : Future[Int] = Future { ... } f onSuccess { case x : Int => println(x) } f onFailure { case e : Exception => println("exception thrown") }
Guava ListenableFutures, Spring 4 ListenableFutureJava 8 CompletableFuture, Scala Futures
@crichardson
Even better: composable futures
def asyncOperation() = Future { ... ; 1 }
val r = asyncOperation().map{ _ * 2 }
assertEquals(4, Await.result(r, 1 second))
Scala, Java 8 CompletableFuture (partially)
Transforms Future
@crichardson
map() is asynchronous - uses callbacks
class Future[T] {
def map[S](f: T => S): Future[S] = { val p = Promise[S]()
onSuccess { case r => p success f(r) } onFailure { case t => p failure t }
p.future }
When ‘this’ completes propagate outcome to ‘p’
Return new Future
@crichardson
Chaining using flatmap()
def asyncOperation() = Future { ... ; 3 }
def asyncSquare(x : Int) : Future[Int] = Future { x * x}
val r = asyncOperation().flatMap { x => asyncSquare(x) }
assertEquals(9, Await.result(r, 1 second))
Scala, Java 8 CompletableFuture (partially)
Chaining asynchronous operations
@crichardson
Combining futuresval f1 = Future { ... ; 1}val f2 = Future { .... ; 2}
val fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second))
def asyncOp(x : Int) = Future { x * x}val f = Future.sequence((1 to 5).map { x => asyncOp(x) })assertEquals(List(1, 4, 9, 16, 25), Await.result(f, 1 second))
Scala, Java 8 CompletableFuture (partially)
Combines two futures
Transforms list of futures to a future containing a list
@crichardson
Scala futures are Monadsdef callB() : Future[...] = ...def callC() : Future[...] = ...def callD() : Future[...] = ...
val result = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d
result onSuccess { .... }
Two calls execute in parallel
And then invokes D
Get the result of D
@crichardson
‘for’ is shorthand for map() and flatMap()
(callB() zip callC()) flatMap { r1 => val (b, c) = r1 callD(b, c) map { d => d }}
for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d
@crichardson
Scala Future + RestTemplateimport scala.concurrent.Future
@Componentclass ProductInfoService {
def getProductInfo(productId: Long): Future[ProductInfo] = { Future { restTemplate.getForObject(....) } }
}
Executes in thread pool
Scala Future
@crichardson
Scala Future + RestTemplateclass ProductDetailsService @Autowired()(....) {
def getProductDetails(productId: Long) = { val productInfoFuture = productInfoService.getProductInfo(productId) val recommendationsFuture = recommendationService.getRecommendations(productId) val reviewsFuture = reviewService.getReviews(productId)
for (((productInfo, recommendations), reviews) <- productInfoFuture zip recommendationsFuture zip reviewsFuture) yield ProductDetails(productInfo, recommendations, reviews) }
}
Non-blocking!
@crichardson
Async Spring MVC + Scala Futures
@Controllerclass ProductController ... {
@RequestMapping(Array("/productdetails/{productId}")) @ResponseBody def productDetails(@PathVariable productId: Long) = { val productDetails = productDetailsService.getProductDetails(productId)
val result = new DeferredResult[ProductDetails] productDetails onSuccess { case r => result.setResult(r) } productDetails onFailure { case t => result.setErrorResult(t) } result }
Convert Scala Future to
Spring MVCDeferredResult
@crichardson
Servlet layer is asynchronous but the backend uses thread
pools⇒
Need event-driven REST client
@crichardson
New in Spring 4
Mirrors RestTemplate
Methods return a ListenableFuture
ListenableFuture = JDK 7 Future + callback methods
Can use HttpComponents NIO-based AsyncHttpClient
Spring AsyncRestTemplate
@crichardson
Using the AsyncRestTemplate
http://hc.apache.org/httpcomponents-asyncclient-dev/
val asyncRestTemplate = new AsyncRestTemplate( new HttpComponentsAsyncClientHttpRequestFactory())
override def getProductInfo(productId: Long) = {
val listenableFuture = asyncRestTemplate.getForEntity("{baseUrl}/productinfo/{productId}", classOf[ProductInfo], baseUrl, productId)
toScalaFuture(listenableFuture).map { _.getBody }}
Convert to Scala Future and get entity
@crichardson
Converting ListenableFuture to Scala Future
implicit def toScalaFuture[T](f : ListenableFuture[T]) : Future[T] = { val p = promise[T]()
f.addCallback(new ListenableFutureCallback[T] { def onSuccess(result: T) { p.success(result)} def onFailure(t: Throwable) { p.failure(t) } }) p.future }
The producer side of Scala Futures
Supply outcome
Return future
@crichardson
Now everything is non-blocking :-)
@crichardson
If recommendation service is down...
Never responds ⇒ front-end server waits indefinitely
Consumes valuable front-end server resources
Page never displayed and customer gives up
Returns an error
Error returned to the front-end server ⇒ error page is displayed
Customer gives up
@crichardson
Fault tolerance at NetflixNetwork timeouts and retries
Invoke remote services via a bounded thread pool
Use the Circuit Breaker pattern
On failure:
return default/cached data
return error to caller
Implementation: https://github.com/Netflix/Hystrix
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
@crichardson
How to handling partial failures?
productDetailsFuture zip recommendationsFuture zip reviewsFuture
Fails if any Future has failed
@crichardson
Handling partial failures
recommendationService. getRecommendations(userId,productId) .recover { case _ => Recommendations(List()) }
“catch-like” Maps Throwable to value
@crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
@crichardson
Why solve this problem for JavaScript?
Browser invokes web services
Implement front-end server/API gateway using NodeJS
@crichardson
What’s NodeJS?
Designed for DIRTy apps
@crichardson
NodeJS
JavaScript
Reactor pattern
Modules
@crichardson
Asynchronous JavaScript code = callback hell
Scenarios:
Sequential: A ⇒ B ⇒ C
Fork and join: A and B ⇒ C
Code quickly becomes very messy
@crichardson
Callback-based HTTP client
request = require("request") handler = (error, clientResponse, body) -> if clientResponse.statusCode != 200 // ERROR else // SUCCESS
request("http://.../productdetails/" + productId, handler)
@crichardson
Messy callback codegetProductDetails = (req, res) -> productId = req.params.productId result = {productId: productId} makeHandler = (key) -> (error, clientResponse, body) -> if clientResponse.statusCode != 200 res.status(clientResponse.statusCode) res.write(body) res.end() else result[key] = JSON.parse(body) if (result.productInfo and result.recommendations and result.reviews) res.json(result)
request("http://localhost:3000/productdetails/" + productId, makeHandler("productInfo")) request("http://localhost:3000/recommendations/" + productId,
makeHandler('recommendations')) request("http://localhost:3000/reviews/" + productId, makeHandler('reviews'))
app.get('/productinfo/:productId', getProductInfo)
Returns a callback function
Have all three requests completed?
@crichardson
Simplifying code with Promises (a.k.a. Futures)
Functions return a promise - no callback parameter
A promise represents an eventual outcome
Use a library of functions for transforming and composing promises
Promises/A+ specification - http://promises-aplus.github.io/promises-spec
@crichardson
Basic promise API
promise2 = promise1.then(fulfilledHandler, errorHandler, ...)
called when promise1 is fulfilledreturns a promise or value
resolved with outcome of callback
called when promise1 is rejected
About when.jsRich API = Promises spec + use full extensions
Creation:
when.defer()
when.reject()...
Combinators:
all, some, join, map, reduce
Other:
Calling non-promise code
timeout
....
https://github.com/cujojs/when
@crichardson
Simpler promise-based code
rest = require("rest")
@httpClient = rest.chain(mime).chain(errorCode);
getProductInfo : (productId) -> @httpClient path: "http://.../productinfo/" + productId
Returns a promise
No ugly callbacks
@crichardson
Simpler promise-based code class ProductDetailsService getProductDetails: (productId) -> makeProductDetails = (productInfo, recommendations, reviews) -> details = productId: productId productDetails: productInfo.entity recommendations: recommendations.entity reviews: reviews.entity details responses = [@getProductInfo(productId), @getRecommendations(productId),
@getReviews(productId)] all(responses).spread(makeProductDetails)
Array[Promise] ⇒ Promise[Array]
@crichardson
Simpler promise-based code: HTTP handler
getProductDetails = (req, res) -> productId = req.params.productId
succeed = (productDetails) -> res.json(productDetails)
fail = (something) -> res.send(500, JSON.stringify(something.entity || something))
productDetailsService. getDetails(productId). then(succeed, fail)
app.get('/productdetails/:productId', getProductDetails)
@crichardson
Writing robust client code
@crichardson
Recovering from failures
getRecommendations(productId) .otherwise( -> {})
Invoked to return default value if getRecommendations() fails
@crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
@crichardson
Let’s imagine you have a stream of trades
and you need to calculate the rolling
average price of each stock
@crichardson
Spring Integration + Complex event processing (CEP) engine is a good choice
@crichardson
But where is the high-level abstraction that solves this
problem?
@crichardson
Future[List[T]]
Not applicable to infinite streams
@crichardson
Introducing Reactive Extensions (Rx)
The Reactive Extensions (Rx) is a library for composing asynchronous and event-
based programs using observable sequences and LINQ-style query operators. Using Rx, developers
represent asynchronous data streams with Observables , query
asynchronous data streams using LINQ operators , and .....
https://rx.codeplex.com/
@crichardson
About RxJava
Reactive Extensions (Rx) for the JVM
Implemented in Java
Adaptors for Scala, Groovy and Clojure
https://github.com/Netflix/RxJava
@crichardson
RxJava core concepts
class Observable<T> { Subscription subscribe(Observer<T> observer) ...}
interface Observer<T> {void onNext(T args)void onCompleted()void onError(Throwable e)
}
Notifies
An asynchronous stream of items
Used to unsubscribe
@crichardson
Comparing Observable to...Observer pattern - similar but adds
Observer.onComplete()
Observer.onError()
Iterator pattern - mirror image
Push rather than pull
Future - similar but
Represents a stream of multiple values
@crichardson
So what?
@crichardson
Transforming Observables
class Observable<T> {
Observable<T> take(int n); Observable<T> skip(int n); <R> Observable<R> map(Func1<T,R> func) <R> Observable<R> flatMap(Func1<T,Observable<R>> func) Observable<T> filter(Func1<T,java.lang.Boolean> predicate)
...}
Scala-collection style methods
@crichardson
Transforming observables
class Observable<T> {
<K> Observable<GroupedObservable<K,T>> groupBy(Func1<T,K> keySelector) ...}
Similar to Scala groupBy except results are emitted as items arrive
Stream of streams!
@crichardson
Combining observables
class Observable<T> {
static <R,T1,T2> Observable<R> zip(Observable<T0> o1, Observable<T1> o2, Func2<T1,T2,R> reduceFunction)
...}
Invoked with pairs of items from o1 and o2
Stream of results from reduceFunction
@crichardson
Creating observables from data
class Observable<T> { static Observable<T> from(Iterable<T> iterable]); static Observable<T> from(T items ...]); static Observable<T> from(Future<T> future]); ...}
@crichardson
Arranges to call obs.onNext/onComplete/...
Creating observables from event sources
Observable<T> o = Observable.create( new SubscriberFunc<T> () { Subscription onSubscribe(Observer<T> obs) {
... connect to event source....
return new Subscriber () { void unsubscribe() { ... };
}; });
Called once for
each Observer
Called when unsubscribing
@crichardson
Using Rx Observables instead of Futures
@crichardson
Rx-based ProductInfoService@Componentclass ProductInfoService override def getProductInfo(productId: Long) = {
val baseUrl = ...
val responseEntity = asyncRestTemplate.getForEntity(baseUrl + "/productinfo/{productId}", classOf[ProductInfo], productId)
toRxObservable(responseEntity).map { (r : ResponseEntity[ProductInfo]) => r.getBody }
}
@crichardson
ListenableFuture ⇒ Observable
implicit def toRxObservable[T](future: ListenableFuture[T]) : Observable[T] = { def create(o: Observer[T]) = { future.addCallback(new ListenableFutureCallback[T] { def onSuccess(result: T) { o.onNext(result) o.onCompleted() }
def onFailure(t: Throwable) { o.onError(t) } }) new Subscription { def unsubscribe() {} } }
Observable.create(create _) }
Supply single value
Indicate failure
Do nothing
@crichardson
Rx - ProductDetailsService@Componentclass ProductDetailsService @Autowired() (productInfoService: ProductInfoService, reviewService: ReviewService, ecommendationService: RecommendationService) {
def getProductDetails(productId: Long) = { val productInfo = productInfoService.getProductInfo(productId) val recommendations =
recommendationService.getRecommendations(productId) val reviews = reviewService.getReviews(productId)
Observable.zip(productInfo, recommendations, reviews, (p : ProductInfo, r : Recommendations, rv : Reviews) =>
ProductDetails(p, r, rv)) }
}Just like Scala Futures
@crichardson
Rx - ProductController
@Controllerclass ProductController
@RequestMapping(Array("/productdetails/{productId}")) @ResponseBody def productDetails(@PathVariable productId: Long) = val pd = productDetailsService.getProductDetails(productId) toDeferredResult(pd)
@crichardson
Rx - Observable ⇒ DeferredResult
implicit def toDeferredResult[T](observable : Observable[T]) = { val result = new DeferredResult[T] observable.subscribe(new Observer[T] { def onCompleted() {}
def onError(e: Throwable) { result.setErrorResult(e) }
def onNext(r: T) { result.setResult(r.asInstanceOf[T]) } }) result }
@crichardson
RxObservables vs. Futures
Much better than JDK 7 futures
Comparable to Scala Futures
Rx Scala code is slightly more verbose
@crichardson
Making this code robust
Hystrix works with Observables (and thread pools)
But
For event-driven code we are on our own
@crichardson
Back to the stream of Trades averaging example...
@crichardson
Calculating averages
Observable<AveragePrice> calculateAverages(Observable<Trade> trades) { ...}
class Trade { private String symbol; private double price;
class AveragePrice { private String symbol; private double averagePrice;
@crichardson
Using groupBy()APPL : 401 IBM : 405 CAT : 405 APPL: 403
groupBy( (trade) => trade.symbol)
APPL : 401
IBM : 405
CAT : 405 ...
APPL: 403
Observable<GroupedObservable<String, Trade>>
Observable<Trade>
@crichardson
Using window()APPL : 401 APPL : 405 APPL : 405 ...
APPL : 401 APPL : 405 APPL : 405
APPL : 405 APPL : 405 APPL : 403
APPL : 405 ...
window(...)
Observable<Trade>
Observable<Observable<Trade>>
N secs
N secs
M secs
@crichardson
Using fold()
APPL : 402 APPL : 405 APPL : 405
APPL : 404
fold(0)(sumCalc) / length
Observable<Trade>
Observable<AveragePrice> Singleton
@crichardson
Using merge()
merge()
APPL : 401
IBM : 405
CAT : 405 ...
APPL: 403
APPL : 401 IBM : 405 CAT : 405 APPL: 403
Observable<Observable<AveragePrice>>
Observable<AveragePrice>
@crichardson
Calculating average pricesdef calculateAverages(trades: Observable[Trade]): Observable[AveragePrice] = {
trades.groupBy(_.symbol).map { symbolAndTrades => val (symbol, tradesForSymbol) = symbolAndTrades ...
tradesForSymbol.window(...).map { windowOfTradesForSymbol => windowOfTradesForSymbol.fold((0.0, 0, List[Double]())) { (soFar, trade) => val (sum, count, prices) = soFar (sum + trade.price, count + 1, trade.price +: prices) } map { x => val (sum, length, prices) = x AveragePrice(symbol, sum / length, prices) } }.flatMap(identity) }.flatMap(identity)}
@crichardson
RxJS / NodeJS examplevar rx = require("rx");
function tailFile (fileName) { var self = this;
var o = rx.Observable.create(function (observer) { var watcher = self.tail(fileName, function (line) { observer.onNext(line); }); return function () { watcher.close(); } }); return o.map(parseLogLine);}
function parseLogLine(line) { ... }
@crichardson
Rx in the browser - implementing completion
TextChanges(input) .DistinctUntilChanged() .Throttle(TimeSpan.FromMilliSeconds(10)) .Select(word=>Completions(word)) .Switch() .Subscribe(ObserveChanges(output));
Your Mouse is a Databasehttp://queue.acm.org/detail.cfm?id=2169076
Ajax call returning Observable
Change events ⇒ Observable
Display completions
@crichardson
Summary
Consuming web services asynchronously is essential
Scala-style are a powerful concurrency abstraction
Rx Observables are even more powerful
Rx Observables are a unifying abstraction for a wide variety of use cases
