Different JVM languages lead to different architectural styles. We all know the typical three-tiered architecture and its limitations. Akka and Scala offer event-sourcing. Event-sourced apps model all state changes explicitly and store them immutably. The actor model makes this horizontally scalable. Even better are the functional benefits: a provably correct auditlog and creating new views on past data. This session introduces the event-sourcing concepts. You’ll see how well they map onto actors. To prove this, we show an event-sourced application using Akka. The new Akka Persistence module provides excellent building blocks. Want to learn about the next generation of scalable architectures on the JVM? Check out event-sourcing with Akka!
26. Actors
‣ Mature and open source
‣ Scala & Java API
‣ Akka Cluster
27. Actors
"an island of consistency in a sea of concurrency"
Actor
!
!
!
mailbox
state
behavior
async message
send
Process message:
‣ update state
‣ send messages
‣ change behavior
Don't worry about
concurrency
28. Actors
A good fit for event-sourcing?
Actor
!
!
!
mailbox
state
behavior
mailbox is non-durable
(lost messages)
state is transient
29. Actors
Just store all incoming messages?
Actor
!
!
!
mailbox
state
behavior
async message
send
store in journal
Problems with
command-sourcing:
‣ side-effects
‣ poisonous
(failing) messages
30. Persistence
‣ Experimental Akka module
‣ Scala & Java API
‣ Actor state persistence based
on event-sourcing
31. Persistent Actor
PersistentActor
actor-id
!
!
state
!
event
async message
send (command)
‣ Derive events from
commands
‣ Store events
‣ Update state
‣ Perform side-effects
event
journal (actor-id)
32. Persistent Actor
PersistentActor
actor-id
!
!
state
!
event
!
Recover by replaying
events, that update the
state (no side-effects)
!
event
journal (actor-id)
33. Persistent Actor
!
case object Increment // command
case object Incremented // event
!
class CounterActor extends PersistentActor {
def persistenceId = "counter"
!
var state = 0
!
val receiveCommand: Receive = {
case Increment => persist(Incremented) { evt =>
state += 1
println("incremented")
}
}
!
val receiveRecover: Receive = {
case Incremented => state += 1
}
}
34. Persistent Actor
!
case object Increment // command
case object Incremented // event
!
class CounterActor extends PersistentActor {
def persistenceId = "counter"
!
var state = 0
!
val receiveCommand: Receive = {
case Increment => persist(Incremented) { evt =>
state += 1
println("incremented")
}
}
!
val receiveRecover: Receive = {
case Incremented => state += 1
}
}
35. Persistent Actor
!
case object Increment // command
case object Incremented // event
!
class CounterActor extends PersistentActor {
def persistenceId = "counter"
!
var state = 0
!
val receiveCommand: Receive = {
case Increment => persist(Incremented) { evt =>
state += 1
println("incremented")
}
}
!
val receiveRecover: Receive = {
case Incremented => state += 1
}
}
async callback
(but safe to close
over state)
36. Persistent Actor
!
case object Increment // command
case object Incremented // event
!
class CounterActor extends PersistentActor {
def persistenceId = "counter"
!
var state = 0
!
val receiveCommand: Receive = {
case Increment => persist(Incremented) { evt =>
state += 1
println("incremented")
}
}
!
val receiveRecover: Receive = {
case Incremented => state += 1
}
}
37. Persistent Actor
!
case object Increment // command
case object Incremented // event
!
class CounterActor extends PersistentActor {
def persistenceId = "counter"
!
var state = 0
!
val receiveCommand: Receive = {
case Increment => persist(Incremented) { evt =>
state += 1
println("incremented")
}
}
!
val receiveRecover: Receive = {
case Incremented => state += 1
}
}
Isn't recovery
with lots of events
slow?
38. Snapshots
class SnapshottingCounterActor extends PersistentActor {
def persistenceId = "snapshotting-counter"
!
var state = 0
!
val receiveCommand: Receive = {
case Increment => persist(Incremented) { evt =>
state += 1
println("incremented")
}
case "takesnapshot" => saveSnapshot(state)
}
!
val receiveRecover: Receive = {
case Incremented => state += 1
case SnapshotOffer(_, snapshotState: Int) => state = snapshotState
}
}
39. Snapshots
class SnapshottingCounterActor extends PersistentActor {
def persistenceId = "snapshotting-counter"
!
var state = 0
!
val receiveCommand: Receive = {
case Increment => persist(Incremented) { evt =>
state += 1
println("incremented")
}
case "takesnapshot" => saveSnapshot(state)
}
!
val receiveRecover: Receive = {
case Incremented => state += 1
case SnapshotOffer(_, snapshotState: Int) => state = snapshotState
}
}
42. Persistent View
Persistent
Actor
journal
Persistent
View
Persistent
View
Views poll the journal
‣ Eventually consistent
‣ Polling configurable
‣ Actor may be inactive
‣ Views track single
persistence-id
‣ Views can have own
snapshots
snapshot store
other
datastore
43. Persistent View
!
"The Database is a cache
of a subset of the log"
- Pat Helland
Persistent
Actor
journal
Persistent
View
Persistent
View
snapshot store
other
datastore
44. Persistent View
!
case object ComplexQuery
class CounterView extends PersistentView {
override def persistenceId: String = "counter"
override def viewId: String = "counter-view"
var queryState = 0
def receive: Receive = {
case Incremented if isPersistent => {
queryState = someVeryComplicatedCalculation(queryState)
// Or update a document/graph/relational database
}
case ComplexQuery => {
sender() ! queryState;
// Or perform specialized query on datastore
}
}
}
45. Persistent View
!
case object ComplexQuery
class CounterView extends PersistentView {
override def persistenceId: String = "counter"
override def viewId: String = "counter-view"
var queryState = 0
def receive: Receive = {
case Incremented if isPersistent => {
queryState = someVeryComplicatedCalculation(queryState)
// Or update a document/graph/relational database
}
case ComplexQuery => {
sender() ! queryState;
// Or perform specialized query on datastore
}
}
}
46. Persistent View
!
case object ComplexQuery
class CounterView extends PersistentView {
override def persistenceId: String = "counter"
override def viewId: String = "counter-view"
var queryState = 0
def receive: Receive = {
case Incremented if isPersistent => {
queryState = someVeryComplicatedCalculation(queryState)
// Or update a document/graph/relational database
}
case ComplexQuery => {
sender() ! queryState;
// Or perform specialized query on datastore
}
}
}
48. Scaling out: Akka Cluster
Single writer: persistent actor must be singleton, views may be anywhere
Cluster node Cluster node Cluster node
Persistent
Persistent
Actor
id = "1"
Actor
id = "1"
Persistent
Actor
id = "1"
Persistent
Actor
id = "2"
Persistent
Actor
id = "1"
Persistent
Actor
id = "3"
distributed journal
49. Scaling out: Akka Cluster
Single writer: persistent actor must be singleton, views may be anywhere
How are persistent actors distributed over cluster?
Cluster node Cluster node Cluster node
Persistent
Persistent
Actor
id = "1"
Actor
id = "1"
Persistent
Actor
id = "1"
Persistent
Actor
id = "2"
Persistent
Actor
id = "1"
Persistent
Actor
id = "3"
distributed journal
50. Scaling out: Akka Cluster
Sharding: Coordinator assigns ShardRegions to nodes (consistent hashing)
Actors in shard can be activated/passivated, rebalanced
Cluster node Cluster node Cluster node
Persistent
Persistent
Actor
id = "1"
Actor
id = "1"
Persistent
Actor
id = "1"
Persistent
Actor
id = "2"
Persistent
Actor
id = "1"
Persistent
Actor
id = "3"
Shard
Region
Shard
Region
Shard
Region
Coordinator
distributed journal
51. Scaling out: Sharding
val idExtractor: ShardRegion.IdExtractor = {
case cmd: Command => (cmd.concertId, cmd)
}
IdExtractor allows ShardRegion
to route commands to actors
52. Scaling out: Sharding
val idExtractor: ShardRegion.IdExtractor = {
case cmd: Command => (cmd.concertId, cmd)
}
IdExtractor allows ShardRegion
to route commands to actors
val shardResolver: ShardRegion.ShardResolver =
msg => msg match {
case cmd: Command => hash(cmd.concertId)
}
ShardResolver assigns new
actors to shards
53. Scaling out: Sharding
val idExtractor: ShardRegion.IdExtractor = {
case cmd: Command => (cmd.concertId, cmd)
}
IdExtractor allows ShardRegion
to route commands to actors
val shardResolver: ShardRegion.ShardResolver =
msg => msg match {
case cmd: Command => hash(cmd.concertId)
}
ShardResolver assigns new
actors to shards
ClusterSharding(system).start(
typeName = "Concert",
entryProps = Some(ConcertActor.props()),
idExtractor = ConcertActor.idExtractor,
shardResolver = ConcertActor.shardResolver)
Initialize ClusterSharding
extension
54. Scaling out: Sharding
val idExtractor: ShardRegion.IdExtractor = {
case cmd: Command => (cmd.concertId, cmd)
}
IdExtractor allows ShardRegion
to route commands to actors
val shardResolver: ShardRegion.ShardResolver =
msg => msg match {
case cmd: Command => hash(cmd.concertId)
}
ShardResolver assigns new
actors to shards
ClusterSharding(system).start(
typeName = "Concert",
entryProps = Some(ConcertActor.props()),
idExtractor = ConcertActor.idExtractor,
shardResolver = ConcertActor.shardResolver)
Initialize ClusterSharding
extension
val concertRegion: ActorRef = ClusterSharding(system).shardRegion("Concert")
concertRegion ! BuyTickets(concertId = 123, user = "Sander", quantity = 1)
58. DDD+CQRS+ES
DDD: Domain Driven Design
Fully consistent Fully consistent
Aggregate
!
Aggregate
!
Eventual
Consistency
Akka Persistence is not a DDD/CQRS framework
But it comes awfully close
Root entity
enteitnytity entity
Root entity
entity entity
Persistent
Actor
Persistent
Actor
Message
passing
60. Designing aggregates
Focus on events
Structural representation(s) follow ERD
Size matters. Faster replay, less write contention
Don't store derived info (use views)
61. Designing aggregates
Focus on events
Structural representation(s) follow ERD
Size matters. Faster replay, less write contention
Don't store derived info (use views)
With CQRS read-your-writes is not the default
When you need this, model it in a single Aggregate
62. Between aggregates
‣ Send commands to other aggregates by id
!
‣ No distributed transactions
‣ Use business level ack/nacks
‣ SendInvoice -> InvoiceSent/InvoiceCouldNotBeSent
‣ Compensating actions for failure
‣ No guaranteed delivery
‣ Alternative: AtLeastOnceDelivery
63. Between systems
System integration through event-sourcing
topic 1 topic 2 derived
Kafka
Application
Akka
Persistence
Spark
Streaming
External
Application
64. Designing commands
‣ Self-contained
‣ Unit of atomic change
‣ Granularity and intent
65. Designing commands
‣ Self-contained
‣ Unit of atomic change
‣ Granularity and intent
UpdateAddress
street = ...
city = ... vs
ChangeStreet
street = ...
ChangeCity
street = ...
66. Designing commands
‣ Self-contained
‣ Unit of atomic change
‣ Granularity and intent
UpdateAddress
street = ...
city = ... vs
ChangeStreet
street = ...
ChangeCity
street = ...
Move
street = ...
city = ... vs
78. In conclusion
Event-sourcing is...
Powerful
but unfamiliar
Combines well with
UI/Client
Command
Model
Journal
Query
Model
DaDtaastatosrtoere Datastore
Events
DDD/CQRS
79. In conclusion
Event-sourcing is...
Powerful
but unfamiliar
Combines well with
UI/Client
Command
Model
Journal
Query
Model
DaDtaastatosrtoere Datastore
Events
DDD/CQRS
Not a
80. In conclusion
Akka Actors & Akka Persistence
A good fit for
event-sourcing
PersistentActor
actor-id
!
!
state
!
async message
send
(command)
event
event
journal (actor-id)
Experimental, view
improvements needed