In this presentation, Akka Team Lead and author Roland Kuhn presents the freshly released final specification for Reactive Streams on the JVM. This work was done in collaboration with engineers representing Netflix, Red Hat, Pivotal, Oracle, Typesafe and others to define a standard for passing streams of data between threads in an asynchronous and non-blocking fashion. This is a common need in Reactive systems, where handling streams of "live" data whose volume is not predetermined.
The most prominent issue facing the industry today is that resource consumption needs to be controlled such that a fast data source does not overwhelm the stream destination. Asynchrony is needed in order to enable the parallel use of computing resources, on collaborating network hosts or multiple CPU cores within a single machine.
Here we'll review the mechanisms employed by Reactive Streams, discuss the applicability of this technology to a variety of problems encountered in day to day work on the JVM, and give an overview of the tooling ecosystem that is emerging around this young standard.
3. Philosophical Background
3
“You cannot step twice into the same stream.
For as you are stepping in,
other waters are ever flowing on to you.”
— Heraclitus
4. What is a Stream?
• ephemeral flow of data
• possibly unbounded in size
• focused on describing transformation
• can be formed into processing networks
4
7. Enter Reactive Streams
7
“Reactive Streams is an initiative to provide a
standard for asynchronous stream processing with
non-blocking back pressure on the JVM.”
— reactive-streams.org
8. Participants
• Engineers from (among others):
• Netflix
• Pivotal
• Red Hat
• Typesafe
• Individuals like Doug Lea and Todd Montgomery
8
9. Motivation
• all participants face the same basic problem
• all are building tools for their community
• a common solution benefits everybody
• interoperability to make best use of efforts
• proposal to include in JDK9 (j.u.c.Flow)
9
10. Recipe for Success
• minimal interfaces
• rigorous specification of semantics
• full TCK for verification of implementation
• complete freedom for many idiomatic APIs
10
11. Supply and Demand
• data items flow downstream
• demand flows upstream
• data items flow only when there is demand
• recipient is in control of incoming data rate
• data in flight is bounded by signaled demand
11
Publisher Subscriber
data
demand
12. Dynamic Push–Pull
• “push” behavior when consumer is faster
• “pull” behavior when producer is faster
• switches automatically between these
• batching demand allows batching data
12
Publisher Subscriber
data
demand
22. Streams are Ubiquitous
• Streaming SIMD Extensions (SSE) — Intel,1999
• ingesting, transforming and emitting data
• requests & responses flowing through a system
• streams are graphical and intuitive
22
23. Moving Bulk Data over the Network
• content streaming (e.g. video or audio)
• data storage / backup
• data replication between data centers
23
24. Mobile Devices
• streams of updates from the server
• streams of user commands from the client
• bad connection quality—need for back-pressure
24
25. Internet of Things
• ingesting large numbers of low-rate streams
• conflating or extrapolating data
• aggregating many streams
• streaming towards the fleet of devices
25
26. Realtime Data Analysis
• real-time business intelligence
• complex event processing
• temporal correlations (e.g. credit card fraud)
26
30. Akka Streams
• compose any kind of stream processing topology
from immutable and reusable blueprints
• creates RS compliant Publisher/Subscriber when
materializing a flow graph into Actors
• high-level flow graph DSL
30
com.typesafe.akka:akka-stream-experimental:1.0-RC3
40. Streaming Database Results to the Client
40
import DefaultJsonProtocol._
implicit val denormOrderFormat = jsonFormat5(DenormalizedOrder.apply)
val db = Database.forConfig("reportingDB")
private def getFromDb(userId: Int): Publisher[DenormalizedOrder] =
db.stream(denormalizedOrders.filter(_.userId === userId).result)
Http().bindAndHandle(
(get & path("orders" / IntNumber)) { userId =>
val pub =
Source(getFromDb(userId))
.transform(() => new ToJsonArray)
complete(HttpEntity.Chunked.fromData(`application/json`, pub))
},
"localhost", 8080)
41. REACTIVE PLATFORM
Full Lifecycle Support for Play, Akka, Scala and Spark
Give your project a boost with Reactive Platform:
• Monitor Message-Driven Apps
• Resolve Network Partitions Decisively
• Integrate Easily with Legacy Systems
• Eliminate Incompatibility & Security Risks
• Protect Apps Against Abuse
• Expert Support from Dedicated Product Teams
Enjoy learning? See about the availability of
on-site training for Scala, Akka, Play and Spark!
Learn more about our offersCONTACT US