Akka Streams in Action @ ScalaDays Berlin 2016

Konrad `@ktosopl` Malawski // ScalaDays Berlin
pic: Armored Core for Answer, White Gilt
streams
and Reactive Streams in action

Konrad `ktoso` Malawski
Akka Team,
Reactive Streams TCK,
Persistence, HTTP

Konrad `@ktosopl` Malawski
akka.io
typesafe.com
geecon.org
Java.pl / KrakowScala.pl
sckrk.com
GDGKrakow.pl
lambdakrk.pl

… in Action
Yes, we’ll build stuff.

… in Action
Yes, we’ll build stuff.
With pre-release stuff,
because ScalaDays.

“Stream”
What does it mean?!

Akka Streams
Akka Streams && Reactive Streams

Reactive Streams - story: early FRP
http://blogs.msdn.com/b/rxteam/archive/2009/11/17/announcing-reactive-extensions-rx-for-net-silverlight.aspx
http://infoscience.epﬂ.ch/record/176887/ﬁles/DeprecatingObservers2012.pdf - Ingo Maier, Martin Odersky
https://github.com/ReactiveX/RxJava/graphs/contributors
https://github.com/reactor/reactor/graphs/contributors
https://medium.com/@viktorklang/reactive-streams-1-0-0-interview-faaca2c00bec#.69st3rndy
- .NETs’ Reactive Extensions
.NET 3.5

Reactive Streams - story: 2013’s impls
~2013:
Reactive Programming
becoming widely adopted on JVM.
- Play introduced “Iteratees”
- Akka (2009) had Akka-IO (TCP etc.)
- Ben starts work on RxJava
Teams discuss need for back-pressure
in simple user API.
Play’s Iteratee / Akka’s NACK in IO.
}

Play Iteratees – pull back-pressure, difﬁcult API
Akka-IO – NACK back-pressure; low-level IO (Bytes); messaging API
RxJava – no back-pressure, nice API

Reactive Streams - Play’s Iteratees
def fold[B](
done: (A, Input[E]) => Promise[B],
cont: (Input[E] => Iteratee[E, A]) => Promise[B],
error: (String, Input[E]) => Promise[B]
): Promise[B]
// an iteratee that consumes chunkes of String and produces an Int
Iteratee[String,Int]
https://www.playframework.com/documentation/2.0/Iteratees
Feb 2013
Iteratees solved the back-pressure problem,
but were hard to use.
Iteratee & Enumeratee – Haskell inspired.
Play / Akka teams looking for common concept.

Reactive Streams - expert group founded
October 2013
Roland Kuhn (Akka) and Erik Meijer (Rx .NET) meet in Lausanne,
while recording “Principles of Reactive Programming” Coursera Course.
Viktor Klang (Akka), Erik Meijer, Ben Christensen (RxJava)
and Marius Eriksen (Twitter) meet at Twitter HQ.
The term “reactive non-blocking asynchronous back-pressure” gets coined.

October 2013
Goals:
- asynchronous
- never block (waste)
- safe (back-threads pressured)
- purely local abstraction
- allow synchronous impls.
Also, for our examples today:
- compatible with TCP

October 2013
December 2013
Stephane Maldini & Jon Brisbin (Pivotal Reactor) contacted by Viktor.

October 2013
December 2013
Soon after, the “Reactive Streams” expert group is formed.
Also joining the efforts: Doug Lea (Oracle), EndreVarga (Akka), Johannes Rudolph &  
Mathias Doenitz (Spray), and many others, including myself join the effort soon after.

October 2013
December 2013
Soon after, the “Reactive Streams” expert group is formed.
Also joining the efforts: Doug Lea (Oracle), EndreVarga (Akka), Johannes Rudolph &  
Mathias Doenitz (Spray), and many others, including myself join the effort soon after.
I ended up implementing much of the TCK.
Please use it, let me know if it needs improvements :-)

2014–2015:
Reactive Streams Spec & TCK
development, and implementations.
1.0 released on April 28th 2015,
with 5+ accompanying implementations.
2015
Proposed to be included with JDK9 by Doug Lea
via JEP-266 “More Concurrency Updates”
http://hg.openjdk.java.net/jdk9/jdk9/jdk/ﬁle/6e50b992bef4/src/java.base/share/classes/java/util/concurrent/Flow.java

JEP-266 – soon…!
public final class Flow {
private Flow() {} // uninstantiable
@FunctionalInterface
public static interface Publisher<T> {
public void subscribe(Subscriber<? super T> subscriber);
}
public static interface Subscriber<T> {
public void onSubscribe(Subscription subscription);
public void onNext(T item);
public void onError(Throwable throwable);
public void onComplete();
}
public static interface Subscription {
public void request(long n);
public void cancel();
}
public static interface Processor<T,R> extends Subscriber<T>, Publisher<R> {
}
}

Reactive Streams: goals
1) Avoiding unbounded buffering across async boundaries
2) Inter-op interfaces between various libraries

2)Inter-op interfaces between various libraries

Argh, implementing a correct RS Publisher
or Subscriber is so hard!

Argh, implementing a correct
RS Publisher or Subscriber is so hard!

Argh, implementing a correct
RS Publisher or Subscriber is so hard!
Don’t implement RS interfaces
directly, use GraphStage instead!

Akka Streams
Streams complement Actors,
they do not replace them.
Actors – distribution (location transparency)
Streams – back-pressured + more rigid-blueprint

13番: Akka is a Toolkit, pick the right tools for the job.
Runar’s excellent talk @ Scala.World 2015
Asynchronous processing toolbox:
Power
Constraints

Asynchronous processing toolbox:
Constraints
Power

Single value, no streaming by deﬁnition.
Local abstraction. 
Execution contexts.
Power
Constraints

Mostly static processing layouts.
Well typed and Back-pressured!
Constraints
Power

Plain Actor’s younger brother, experimental.
Location transparent, well typed.
Technically unconstrained in actions performed
Constraints
Power

Runar’s excellent talk @ Scala.World 2015
Location transparent.
Various resilience mechanisms.
(watching, persistent recovering, migration, pools)
Untyped and unconstrained in actions performed.
Constraints
Power

Lightbend monitoring
Measure. Don’t guess.
http://monitoring.lightbend.com/docs/latest/home.html

Lightbend monitoring
Measure. Don’t guess.
Let’s find if & where
we have a bottleneck…
http://monitoring.lightbend.com/docs/latest/home.html
DEMO

Materialization
Gears from GeeCON.org, did I mention it’s an awesome conf?

What is “materialization” really?

Custom Stages
&
Understanding the Types

Custom stages, made simple (!)

Understanding the Types
Source.fromGraph(Graph[SourceShape[Status], M])
.via(Graph[FlowShape[Status, T], M])
.to(Graph[SinkShape[Status], _])

Custom != hard; custom == extensible
Your Stages “feel” native.
DEMOLet’s try “Hundred-ify”.
http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala
See also mapConcat / statefulMapConcat.

Akka HTTP ~ streaming all the way
Streaming all the way
with Akka HTTP

Streaming in Akka HTTP
DEMO
“Framed entity streaming” https://github.com/akka/akka/pull/20778
HttpServer as a:
Flow[HttpRequest, HttpResponse]

DEMO
HttpServer as a:
HTTP Entity as a:
Source[ByteString, _]

DEMO
HttpServer as a:
HTTP Entity as a:
Source[ByteString, _]
Websocket connection as a:
Flow[ws.Message, ws.Message]

Persistence Query (experimental)
“The Query Side”
of Akka Persistence

Persistence Query (experimental)
Persistence Query Journals
akka/akka-persistence-cassandra 0.16
akka/akka @ leveldb-journal 2.4.8
dnvriend/akka-persistence-jdbc 2.3.3
scullxbones/akka-persistence-mongo 1.2.5
…and more, that I likely forgot about.
DEMO
Implementation of “data-pump” pattern.

Akka + Kafka = BFF
Reactive Kafka
+
Started by Krzysiek Ciesielski & Adam Warski @ SofwareMill.com

“ACKnowladged streams”
happy ACKing!

Kafka + Akka = BFF
Akka is Arbitrary processing.
Kafka is somewhat more than a message queue,
but very focused on “the log”.
Spark shines with it’s data-science focus.

Streams talking to Actors
&&
Actors talking to Streams

Streams <=> Actors inter-op
Source.actorRef (no back-pressure)
Source.queue (safe)
Sink.actorRef (no back-pressure)
Sink.actorRefWithAck (safe)

Next steps for Akka
Completely new Akka Remoting (goal: 1M+ msg/s (!)),
(it is built using Akka Streams).
More integrations for Akka Streams stages,
also dynamic fan-in/out A.K.A.“the Hub”.
Reactive Kafka polishing and stable release with SoftwareMill.
“Conﬁrmed Streams” work from Reactive Kafka generalised.
Akka Typed likely to progress again.
Of course, continued maintenance of Cluster and others.

Happy hAkking!
Announcing that there will be an announcement.
hAkknowledge
(pilot name?)
“initiative” / “competition”

Akka <3 contributions
• Easy to contribute:
• https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy-to-contribute
• https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3A%22nice-to-have+%28low-prio%29%22
• Akka Stream Contrib
• https://github.com/akka/akka-stream-contrib
• Mailing list:
• https://groups.google.com/group/akka-user
• Public chat rooms:
• http://gitter.im/akka/dev developing Akka
• http://gitter.im/akka/akka using Akka

Thanks!
ktoso @ typesafe.com
twitter: ktosopl
github: ktoso
team blog: letitcrash.com
home: akka.io
Thus spake the Master Programmer:
“After three days without programming,
life becomes meaningless.”

Q/A
(Now’s the time to ask things!)
ktoso @ typesafe.com
twitter: ktosopl
github: ktoso
team blog: letitcrash.com
home: akka.io

Akka Streams in Action @ ScalaDays Berlin 2016

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (13)

Similaire à Akka Streams in Action @ ScalaDays Berlin 2016

Similaire à Akka Streams in Action @ ScalaDays Berlin 2016 (20)

Dernier

Dernier (20)

Akka Streams in Action @ ScalaDays Berlin 2016