Erlang is successfully used for game servers. Lua is a friendly, small language that is big in game scripting. And VoltDB is a new high speed ACID database from PostgreSQL inventor Mike Stonebreaker. Add some JSON, Websockets and JavaScript and you have an exciting HTML5 game stack.
This talk is about research and results from the work on a very high throughput game server with embedded Lua for game logic and VoltDB as SQL database backend. The stack aims to maximize robustness, throughput and ease of scripting. Its special feature is linearly scaling true ACID with a goal of a million transactions per second, while being easy on the game logic scripter.
This stack is part of the development effort of games start-up Eonblast for the cross-media Science Fiction franchise Solar Tribes, which will feature film elements merged with cross-platform online games. The stack is created to allow for a unified, non-sharded game world experience, thus for a unified story line and also otherwise transcends current MMO conventions. Players will share the same stories at the same time and be empowered to leave a stronger imprint on their environment. Interaction with other players is taken beyond the limitations currently seen in social games. All of which are technical challenges, asking for fastest possible data access and better data integrity guarantees.
A triple asynchronous server side architecture is described that communicates non-blocking with a fat JS/HTML5 browser client. The related Open Source projects Erlvolt and Fleece are introduced, which were created for this stack. The embedded Lua driver Erlualib and Robert Virding's new project, the Lua VM emulator Luerl are compared and evaluated. A brief overview of capabilities of new databases is given to explain why VoltDB was chosen for the task.
(c) 2012 Eonblast Corporation
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
An Erlang Game Stack
1. Lua + Erlang + VoltDB
An Erlang Game Stack
An Erlang Game Stack
Erlang User Conference 2012
H. Diedrich
http://www.eonblast.com – twitter @hdiedrich
1 Eonblast
2. An Erlang Game Stack
Henning Diedrich
• Maintainer Emysql, Erlvolt, Erlualib
• Eonblast: game + film
• NewTracks: game + music
• Bricks and mortar
• Insurance tariff language
Octocat says: try markedoc!
2 Eonblast
4. An Erlang Game Stack
This Talk
• A game server architecture
• Its components
• The glue
4 Eonblast
5. An Erlang Game Stack
This Talk
• Work in progress
• Unproven concepts
• Lies and conjecture
andalso
• 3 genius tools
• 10,000 lines of glue code
5 Eonblast
6. An Erlang Game Stack
Questions
Please throw them in like Torben
Mail me at hdiedrich at eonblast.com
Erlang Mailing List
6 Eonblast
7. An Erlang Game Stack
Erlang Game Servers
Zynga: FarmVille via membase, Activision Blizzard: Call of Duty, Bigpoint: Battle Star Galactica, Wooga: Magic Land
7 Eonblast
8. An Erlang Game Stack
Erlang + MySQL
http://eonblast.github.com/Emysql
Electronic Arts contributed emysql.
8 Eonblast
9. An Erlang Game Stack
The Hurt
• Leaks, Deadlocks
• Bad Productivity
• Game Design Language Gap
• Less Features
• Less Players
• Lesser Product
9 Eonblast
10. An Erlang Game Stack
Target
„The perfect game server“
(makes no sense)
10 Eonblast
11. An Erlang Game Stack
Generals’ Problem
State
Two generals must agree on a time.
11 Eonblast
12. An Erlang Game Stack
Generals’ Problem
State
Send a messenger.
12 Eonblast
13. An Erlang Game Stack
Generals’ Problem
State
Acknowledge.
13 Eonblast
14. An Erlang Game Stack
Generals’ Problem
State
ACK the ACK. Etc.
14 Eonblast
15. An Erlang Game Stack
Generals’ Problem
State
The messenger may get lost.
15 Eonblast
16. An Erlang Game Stack
Byzantine Generals
State
The generals, too.
16 Eonblast
17. An Erlang Game Stack
The Hunt
Server Stack
• Language
• Database
• Protocol
• Game Logic
17 Eonblast
18. An Erlang Game Stack
Spec
“Table Game”
• 1 million active players
• 1 million transactions per second
• 1 second latency
• Linear scale
• 100% data integrity
18 Eonblast
19. An Erlang Game Stack
Client Architecture
• Fat JavaScript Browser Client
• COMET
Not discussed in this talk.
19 Eonblast
20. An Erlang Game Stack
Language
Wishlist
• Robustness
• Deadlock-free
• Multi-core
• Linear Scale
er?
20 Eonblast
21. An Erlang Game Stack
Database
CAP
• Distributed
• Consistent
• Highly-available
• Partition-tolerant
All of it!
Brewer on CAP 2012: http://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed
21 Eonblast
22. An Erlang Game Stack
Database
ACID
• Atomicity
• Consistency
• Isolation
• Durability
For Granted?
22 Eonblast
23. An Erlang Game Stack
Database
Double Bookkeeping
• Not every game needs it
• Requires ACID Transactions
• Neigh impossible to emulate
• Impossible with BASE /
Eventual Consistency
23 Eonblast
24. An Erlang Game Stack
Database
Upcoming Blog Post
• Looking at 14 databases
• Riak, Cassandra, Membase, H-Base, Voldemort, MySQL, MySQL Cluster, Redis, Redis
Cluster, Tokyo Cabinet, Memcached, CouchDB, Couchbase, VoltDB, MongoDB, Postgres
• In the light of what games need
• Unbiased comparison
• Twitter @hdiedrich
24 Eonblast
25. An Erlang Game Stack
Protocol
Wishlist
• Small footprint
• Fast to encode
• Human-readable
25 Eonblast
26. An Erlang Game Stack
Logic
Wishlist
• Human Readable
• Stable
• Fast
• Concise
• Small Footprint
26 Eonblast
27. An Erlang Game Stack
The Answers
• Client
• Protocol
• Server
• Logic
• Data
Client: HTML5/JS, Protocol: JSON, Server: Erlang, Logic: Lua, Data: VoltDB.
27 Eonblast
28. An Erlang Game Stack
“Embeddable scripting language”
28 Eonblast
29. An Erlang Game Stack
Lua
”Lua is strange.“
R. Virding
“It has mutable variables, what more need I say.”
29 Eonblast
30. An Erlang Game Stack
Lua
a = “Roberto”
if a ~= nil then
print “Hello ” .. a .. “!”
end
30 Eonblast
31. An Erlang Game Stack
Lua
• Created 1993
• Simple, imperative syntax
• Stable syntax, stable VMs
• Two main VMs: ISO C, JIT
• Game scripting champion
• Open Source, 100% dictatorial*
*benevolent, of course
Site: http://www.lua.org
List: lua-l@lists.lua.org
31 Eonblast
32. An Erlang Game Stack
Lua
• Made for non-programmers - imperative syntax
• Powerful for programmers - Scheme semantics
• Made for DSLs - extensible semantics
• Made to be extended - simple C API
• Made to be embedded - small VM footprint
• Now with a native Erlang VM!
32 Eonblast
33. An Erlang Game Stack
Lua Caveats
• Deceiving Looks
• Encourages magic
• Simple but not for beginners
• Syntax forked
• Only one collection type
33 Eonblast
34. An Erlang Game Stack
Lua VMs
• PUC Reference VM pure ISO C
Lua 5.2 super compatible
• JIT VM X86, ARM, PPC, MIPS
remains at Lua 5.1
super fast
adds extensions
• Luerl native Erlang VM
Lua 5.2 super stable
incomplete
34 Eonblast
36. An Erlang Game Stack
Perceived reaction to asking about benchmarks on the Erlang mailing list.
36 Eonblast
37. An Erlang Game Stack
World State
State
Where does the state go? Into the Logic core, for speed.
37 Eonblast
38. An Erlang Game Stack
World State
State State State State State
State
38 Eonblast
39. An Erlang Game Stack
World State
State State State State State
State
39 Eonblast
40. An Erlang Game Stack
World State
State State State State
State
State
40 Eonblast
41. An Erlang Game Stack
World State
State State State State State
State
41 Eonblast
42. An Erlang Game Stack
“High Velocity Database”
42 Eonblast
43. An Erlang Game Stack
VoltDB
• Created 2009
• Simple SQL syntax
• Stable 2.5
• Commercial developer, support
• High Velocity Secret Tip
• Open Source, 100% dictatorial*
*benevolent, of course
Site: http://www.voltdb.com
Help: http://community.voltdb.com/forum
43 Eonblast
44. An Erlang Game Stack
VoltDB
• ACID transactions – double bookkeeping
• SQL – subset of '92
• Linear scale - „unlimited“ data (but ...)
• Made for OLTP - fast, cheap writes, high throughput
• “More SQL than SQL” – invites clean sep. of data
• In-memory - 100x faster than MySQL
• Replication, Snapshots – 'hot backup built in'
44 Eonblast
45. An Erlang Game Stack
VoltDB Caveats
• Less agile than MySQL/NoSQL
• Need to understand partitions
• Must program SPs in Java
• Not elastic (yet)
• No online schema change (yet)
• There is always a bottleneck
• Simple but not for beginners
45 Eonblast
46. An Erlang Game Stack
Horizonzal Partitions
State
#
46 Eonblast
53. An Erlang Game Stack
The Packages
• Protocol Fleece Lua to JSON
• Logic Erlualib Lua port
• Logic Luerl Lua VM
• Data Erlvolt VoltDB driver
53 Eonblast
54. An Erlang Game Stack
Erlualib
• Lua embedding library for Erlang
• Lua state in original PUC Lua VM
• A fast way
• A not secure way: can crash Erlang
54 Eonblast
55. An Erlang Game Stack
Erlualib
hello() ->
% get handle of the Lua engine
{ok, L} = lua:new_s tate(),
% put " print" global on top of s tack
lua:getfield(L, global, " print" ),
% put hello on top
lua:pus hs tring(L, " Hello from Lua!" ),
% exec ute on top 2 values on s tac k
lua:c all(L, 1, 0).
55 Eonblast
57. An Erlang Game Stack
Luerl
• New Lua VM programmed in Erlang
• Perfectly secure way
• Precompiled Chunks
• Re-usable State
• Work in progress
57 Eonblast
58. An Erlang Game Stack
Luerl
hello() ->
L = luerl:init(),
luerl:do(" print('Hello from Lua!')" , L).
58 Eonblast
59. An Erlang Game Stack
Luerl
hello() ->
L = luerl:init(),
C hunk = luerl:eval(" print('Hello from Lua!')" , L),
luerl:call(C hunk, L)
% C hunk and L c an be re-us ed.
59 Eonblast
60. An Erlang Game Stack
Benchmark Luerl vs Erlualib
Erlualib / PUC Luerl
270 µs state Init 250 µs
10 µs parse 1 + 1 15 µs
2.5 µs execute 1 + 1 13 µs
23 µs parse formula 63 µs
2.5 µs execute formula 35 µs
2.5 µs call a function 28 µs
Formula: a = 7.33; b = 9000; c = (33 * a / b) ^ 15 * a + b
60 Eonblast
61. An Erlang Game Stack
Fast Lua to JSON encoder
61 Eonblast
62. An Erlang Game Stack
Fleece
• Fastest Lua JSON encoder
• C+ASM or ISO C
• Faster than Lua JIT
• Faster than concat
• 10x faster than other fastest
62 Eonblast
64. An Erlang Game Stack
Erlvolt
• Native Erlang VoltDB driver
• Async parallel data shuffling
• Coming next quarter
• Your help is wanted
• Connect over github
64 Eonblast
65. An Erlang Game Stack
Server Architecture
• Erlang • Luerl
• Lua • Erlualib
• JSON • Fleece
• VoltDB • Erlvolt
65 Eonblast
66. An Erlang Game Stack
Invitation
• All at github
• All w/samples & docs
• All maintained by Eonblast
• Luerl maintained by R. Virding
• Updates @hdiedrich, @rvirding
66 Eonblast
67. An Erlang Game Stack
Questions
Mail: hdiedrich ∂ eonblast.com
Skype: eonblast
Twitter: @hdiedrich
67 Eonblast