A modular, polyglot architecture has many advantages but it also adds complexity since each incoming request typically fans out to multiple distributed services. For example, in an online store application the information on a product details page - description, price, recommendations, etc - comes from numerous services. To minimize response time and improve scalability, these services must be invoked concurrently. However, traditional concurrency mechanisms are low-level, painful to use and error-prone.
In this talk you will learn about some powerful yet easy to use abstractions for consuming web services asynchronously. We will compare the various implementations of futures that are available in Java, Scala and JavaScript. You will learn how to use reactive observables, which are asynchronous data streams, to access web services from both Java and JavaScript. We will describe how these mechanisms let you write asynchronous code in a very straightforward, declarative fashion.
Potential of AI (Generative AI) in Business: Learnings and Insights
Futures and Rx Observables: powerful abstractions for consuming web services asynchronously (#springone #s2gx)
1. @crichardson
Consuming web services asynchronously
with Futures
and Rx Observables
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
chris@chrisrichardson.net
http://plainoldobjects.com
9. @crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
15. @crichardson
Application architecture
Product Info Service
Desktop
browser
Native mobile
client
REST
REST
REST
Recomendation Service
Review Service
Front end server
API gatewayREST
Mobile
browser
Web Application
HTML
17. @crichardson
Handling getProductDetails() - sequentially
Front-end server
Product Info Service
Recommendations
Service
Review
Service
getProductInfo()
getRecommendations()
getReviews()
getProductDetails()
Higher response time :-(
18. @crichardson
Handling getProductDetails() - in parallel
Front-end server
Product Info Service
Recommendations
Service
Review
Service
getProductInfo()
getRecommendations()
getReviews()
getProductDetails()
Lower response time :-)
19. @crichardson
Implementing a concurrent REST client
Thread-pool based approach
executorService.submit(new Callable(...))
Simpler but less scalable - lots of idle threads consuming memory
Event-driven approach
NIO with completion callbacks
More complex but more scalable
20. @crichardson
The front-end server must handle partial
failure of backend services
Front-end server
Product Info Service
Recommendations
Service
Review
Service
getProductInfo()
getRecommendations()
getReviews()
getProductDetails()
X
How to provide a
good user
experience?
21. @crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
22. @crichardson
Futures are a great concurrency
abstraction
http://en.wikipedia.org/wiki/Futures_and_promises
24. @crichardson
Benefits
Client does not know how the asynchronous operation is implemented
Client can invoke multiple asynchronous operations and gets a Future for each
one.
25. @crichardson
REST client using Spring @Async
@Component
class ProductInfoServiceImpl extends ProducInfoService {
val restTemplate : RestTemplate = ...
@Async
def getProductInfo(productId: Long) = {
new AsyncResult(restTemplate.getForObject(....)...)
}
}
Execute asynchronously in thread pool
A Future
containing a value
trait ProductInfoService {
def getProductInfo(productId: Long): java.util.concurrent.Future[ProductInfo]
}
26. @crichardson
ProductDetailsService
@Component
class ProductDetailsService
@Autowired()(productInfoService: ProductInfoService,
reviewService: ReviewService,
recommendationService: RecommendationService) {
def getProductDetails(productId: Long): ProductDetails = {
val productInfoFuture = productInfoService.getProductInfo(productId)
}
}
val recommendationsFuture = recommendationService.getRecommendations(productId)
val reviewsFuture = reviewService.getReviews(productId)
val productInfo = productInfoFuture.get(300, TimeUnit.MILLISECONDS)
val recommendations =
recommendationsFuture.get(10, TimeUnit.MILLISECONDS)
val reviews = reviewsFuture.get(10, TimeUnit.MILLISECONDS)
ProductDetails(productInfo, recommendations, reviews)
28. @crichardson
Not bad but...
val productInfo =
productInfoFuture.get(300, TimeUnit.MILLISECONDS)
Blocks Tomcat thread until Future
completes
Not so scalable :-(
29. @crichardson
... and also...
Java Futures work well for a single-level of asynchronous execution
BUT
#fail for more complex, scalable scenarios
Difficult to compose and coordinate multiple concurrent operations
http://techblog.netflix.com/2013/02/rxjava-netflix-api.html
30. @crichardson
Better: Futures with callbacks no blocking!
val f : Future[Int] = Future { ... }
f onSuccess {
case x : Int => println(x)
}
f onFailure {
case e : Exception => println("exception thrown")
}
Guava ListenableFutures, Spring 4 ListenableFuture
Java 8 CompletableFuture, Scala Futures
31. @crichardson
Even better: Composable Futures
val f1 = Future { ... ; 1 }
val f2 = Future { ... ; 2 }
val f4 = f2.map(_ * 2)
assertEquals(4, Await.result(f4, 1 second))
val fzip = f1 zip f2
assertEquals((1, 2), Await.result(fzip, 1 second))
Transforms Future
Combines two futures
Scala, Java 8 CompletableFuture (partially)
32. @crichardson
Scala futures are Monads
def callB() : Future[...] = ...
def callC() : Future[...] = ...
def callD() : Future[...] = ...
val result = for {
(b, c) <- callB() zip callC();
d <- callD(b, c)
} yield d
result onSuccess { .... }
Two calls execute in parallel
And then invokes D
Get the result of D
33. @crichardson
Scala Future + RestTemplate
import scala.concurrent.Future
@Component
class ProductInfoService {
def getProductInfo(productId: Long): Future[ProductInfo] = {
Future { restTemplate.getForObject(....) }
}
}
Executes in thread pool
Scala Future
34. @crichardson
Scala Future + RestTemplate
class ProductDetailsService @Autowired()(....) {
def getProductDetails(productId: Long) = {
val productInfoFuture = productInfoService.getProductInfo(productId)
val recommendationsFuture = recommendationService.getRecommendations(productId)
val reviewsFuture = reviewService.getReviews(productId)
for (((productInfo, recommendations), reviews) <-
productInfoFuture zip recommendationsFuture zip reviewsFuture)
yield ProductDetails(productInfo, recommendations, reviews)
}
}
Non-blocking!
35. @crichardson
Async Spring MVC + Scala Futures
@Controller
class ProductController ... {
@RequestMapping(Array("/productdetails/{productId}"))
@ResponseBody
def productDetails(@PathVariable productId: Long) = {
val productDetails =
productDetailsService.getProductDetails(productId)
val result = new DeferredResult[ProductDetails]
productDetails onSuccess {
case r => result.setResult(r)
}
productDetails onFailure {
case t => result.setErrorResult(t)
}
result
}
Convert Scala Future to
Spring MVC
DeferredResult
37. @crichardson
About the Reactor pattern
Defined by Doug Schmidt in 1995
Pattern for writing scalable servers
Alternative to thread-per-connection model
Single threaded event loop dispatches events on handles (e.g. sockets, file
descriptors) to event handlers
38. @crichardson
Reactor pattern structure
Event Handler
handle_event(type)
get_handle()
Initiation Dispatcher
handle_events()
register_handler(h)
select(handlers)
for each h in handlers
h.handle_event(type)
end loop
handle
Synchronous Event
Demultiplexer
select()
owns
notifies
uses
handlers
Application
creates
40. @crichardson
New in Spring 4
Mirrors RestTemplate
Methods return a ListenableFuture = JDK 7 Future + callback methods
Can use HttpComponents NIO-based AsyncHttpClient
Spring AsyncRestTemplate
41. @crichardson
Using the AsyncRestTemplate
http://hc.apache.org/httpcomponents-asyncclient-dev/
val asyncRestTemplate = new AsyncRestTemplate(
new HttpComponentsAsyncClientHttpRequestFactory())
override def getProductInfo(productId: Long) = {
val listenableFuture =
asyncRestTemplate.getForEntity("{baseUrl}/productinfo/{productId}",
classOf[ProductInfo],
baseUrl, productId)
toScalaFuture(listenableFuture).map { _.getBody }
}
Convert to Scala Future and get entity
42. @crichardson
Converting ListenableFuture to Scala Future
implicit def toScalaFuture[T](f : ListenableFuture[T]) : Future[T] = {
val p = promise[T]
f.addCallback(new ListenableFutureCallback[T] {
def onSuccess(result: T) { p.success(result)}
def onFailure(t: Throwable) { p.failure(t) }
})
p.future
}
The producer side of Scala Futures
Supply outcome
Return future
44. @crichardson
If recommendation service is down...
Never responds front-end server waits indefinitely
Consumes valuable front-end server resources
Page never displayed and customer gives up
Returns an error
Error returned to the front-end server error page is displayed
Customer gives up
45. @crichardson
Fault tolerance at Netflix
Network timeouts and retries
Invoke remote services via a bounded thread pool
Use the Circuit Breaker pattern
On failure:
return default/cached data
return error to caller
Implementation: https://github.com/Netflix/Hystrix
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
46. @crichardson
Using Hystrix
@Component
class ProductInfoServiceImpl extends ProductInfoService {
val restTemplate = RestTemplateFactory.makeRestTemplate()
val baseUrl = ...
class GetProductInfoCommand(productId: Long)extends
HystrixCommand[ProductInfo](....) {
override def run() =
restTemplate.
getForEntity("{baseUrl}/productinfo/{productId}",
classOf[ProductInfo],
baseUrl, productId).getBody
}
def getProductInfoUsingHystrix(productId: Long) : Future[ProductInfo] = {
new GetProductInfoCommand(productId).queue()
}
}
Runs in thread pool
Returns JDK Future
48. @crichardson
How to handling partial failures?
productDetailsFuture zip
recommendationsFuture zip reviewsFuture
Fails if any Future has failed
49. @crichardson
Handling partial failures
val recommendationsFuture =
recommendationService.
getRecommendations(userId,productId).
recover {
case _ => Recommendations(List())
}
“catch-like” Maps Throwable to value
50. @crichardson
Implementing a Timeout
resultFuture onSuccess { case r => result.setResult(r) }
resultFuture onFailure { case t => result.setErrorResult(t) }
No timeout - callbacks might never be
invoked :-(
Await.result(resultFuture, timeout)
Blocks until timeout:-(
51. @crichardson
Non-blocking Timeout
http://eng.42go.com/future-safefuture-timeout-cancelable/
object TimeoutFuture {
def apply[T](future: Future[T], onTimeout: => Unit = Unit)
(implicit ec: ExecutionContext, after: Duration): Future[T] = {
val timer = new HashedWheelTimer(10, TimeUnit.MILLISECONDS)
val promise = Promise[T]()
val timeout = timer.newTimeout(new TimerTask {
def run(timeout: Timeout){
onTimeout
promise.failure(new TimeoutException(s"Future timed out after ${after.toMillis}ms"))
}
}, after.toNanos, TimeUnit.NANOSECONDS)
Future.firstCompletedOf(Seq(future, promise.future)).
tap(_.onComplete { case result => timeout.cancel() })
}
}
val future = ...
val timedoutFuture = TimeoutFuture(future)(executionContext, 200.milleseconds)
Timer fails
promise
Outcome of first
completed
52. @crichardson
Using the Akka Circuit Breaker
import akka.pattern.CircuitBreaker
val breaker =
new CircuitBreaker(actorSystem.scheduler,
maxFailures = 1,
callTimeout = 100.milliseconds,
resetTimeout = 1.minute)
val resultFuture = breaker.
withCircuitBreaker{ asynchronousOperationReturningFuture() }
53. @crichardson
Limiting # of simultaneous requests
val limiter =
new ConcurrencyLimiter(maxConcurrentRequests=10, maxQueueSize=30)
val resultFuture = limiter.withLimit {
asynchronousOperationReturningFuture()
}
class ConcurrencyLimiter ...
val concurrencyManager : ActorRef = ...
def withLimit[T](body : => Future[T])(...) =
(concurrencyManager ?
ConcurrentExecutionManager.Request { () => body }).mapTo[T]
54. @crichardson
Putting it all together
@Component
class ProductInfoServiceImpl @Autowired()(...) extends ProductService {
val limiter = new ConcurrencyLimiter(...)
val breaker = new CircuitBreaker(...)
override def getProductInfo(productId: Long) = {
...
breaker.withCircuitBreaker {
limiter.withLimit {
TimeoutFuture { ... AsyncRestTemplate.get ... }
}
}
}
55. @crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
62. @crichardson
Messy callback code
getProductDetails = (req, res) ->
productId = req.params.productId
result = {productId: productId}
makeHandler = (key) ->
(error, clientResponse, body) ->
if clientResponse.statusCode != 200
res.status(clientResponse.statusCode)
res.write(body)
res.end()
else
result[key] = JSON.parse(body)
if (result.productInfo and result.recommendations and result.reviews)
res.json(result)
request("http://localhost:3000/productdetails/" + productId, makeHandler("productInfo"))
request("http://localhost:3000/recommendations/" + productId, makeHandler('recommendations'))
request("http://localhost:3000/reviews/" + productId, makeHandler('reviews'))
app.get('/productinfo/:productId', getProductInfo)
Returns a callback
function
Have all three requests
completed?
63. @crichardson
Simplifying code with Promises (a.k.a.
Futures)
Functions return a promise - no callback parameter
A promise represents an eventual outcome
Use a library of functions for transforming and composing promises
Promises/A+ specification - http://promises-aplus.github.io/promises-spec
64. @crichardson
Basic promise API
promise2 = promise1.then(fulfilledHandler, errorHandler, ...)
called when promise1 is fulfilled
returns a promise or value
resolved with outcome of callback
called when promise1 is rejected
returns a promise or value
65. About when.js
Rich API = Promises spec + use full
extensions
Creation:
when.defer()
when.reject()...
Combinators:
all, some, join, map, reduce
Other:
Calling non-promise code
timeout
....
https://github.com/cujojs/when
75. @crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
76. @crichardson
Let’s imagine you have a stream of trades
and
you need to calculate the rolling average
price of each stock
80. @crichardson
Introducing Reactive Extensions (Rx)
The Reactive Extensions (Rx) is a library for composing
asynchronous and event-based programs using
observable sequences and LINQ-style query operators.
Using Rx, developers represent asynchronous data
streams with Observables , query asynchronous
data streams using LINQ operators , and .....
https://rx.codeplex.com/
82. @crichardson
RxJava core concepts
class Observable<T> {
Subscription subscribe(Observer<T> observer)
...
}
interface Observer<T> {
void onNext(T args)
void onCompleted()
void onError(Throwable e)
}
Notifies
An asynchronous stream of
items
Used to unsubscribe
83. @crichardson
Comparing Observable to...
Observer pattern - similar but adds
Observer.onComplete()
Observer.onError()
Iterator pattern - mirror image
Push rather than pull
Future - similar but
Represents a stream of multiple values
87. @crichardson
Combining observables
class Observable<T> {
static <R,T1,T2>
Observable<R> zip(Observable<T0> o1,
Observable<T1> o2,
Func2<T1,T2,R> reduceFunction)
...
}
Invoked with pairs of items from
o1 and o2
Stream of results from
reduceFunction
88. @crichardson
Creating observables from data
class Observable<T> {
static Observable<T> from(Iterable<T> iterable]);
static Observable<T> from(T items ...]);
static Observable<T> from(Future<T> future]);
...
}
89. @crichardson
Creating observables from event sources
Observable<T> o = Observable.create(
new SubscriberFunc<T> () {
Subscription onSubscribe(Observer<T> obs) {
... connect to event source....
return new Subscriber () {
void unsubscribe() { ... };
};
});
Called once for
each Observer
Called when
unsubscribing
Arranges to call obs.onNext/onComplete/...
105. @crichardson
Using merge()
merge()
APPL : 401
IBM : 405
CAT : 405 ...
APPL: 403
APPL : 401 IBM : 405 CAT : 405 APPL: 403
Observable<Observable<AveragePrice>>
Observable<AveragePrice>
106. @crichardson
RxJS / NodeJS example
var rx = require("rx");
function tailFile (fileName) {
var self = this;
var o = rx.Observable.create(function (observer) {
var watcher = self.tail(fileName, function (line) {
observer.onNext(line);
});
return function () {
watcher.close();
}
});
return o.map(parseLogLine);
}
function parseLogLine(line) { ... }
107. @crichardson
Rx in the browser - implementing
completion
TextChanges(input)
.DistinctUntilChanged()
.Throttle(TimeSpan.FromMilliSeconds(10))
.Select(word=>Completions(word))
.Switch()
.Subscribe(ObserveChanges(output));
Your Mouse is a Database
http://queue.acm.org/detail.cfm?id=2169076
Ajax call returning
Observable
Change events Observable
Display completions
108. @crichardson
Summary
Consuming web services asynchronously is essential
Scala-style composable Futures are a powerful concurrency abstraction
Rx Observables are even more powerful
Rx Observables are a unifying abstraction for a wide variety of use cases