1. Distributed app
development
node.js + zeroMQ = awwyeah

2. ZeroMQ from a noob’s
viewpoint

3. RUBEN TAN
• NodeJS enthusiast
• Author of eazy
(https://github.com/soggie/eazy)
• Dota 2-er

4. DISCLAIMER
• I AM NOT A ZEROMQ EXPERT
• (not even remotely close)
• Case study by consulting a successful
implementer of ZMQ in an adserver

5. Objective
• 1,000,000,000 ad impressions per month
(on average)
• 80ms maximum response time for each ad
• Good luck, Apache

6. Webserver Choices
• Netty - Java can actually be fast
https://netty.io
• Gevent - For Python lovers
http://gevent.org/
• Node.js - The reason why you’re here
http://nodejs.org/

7. So... which one?
• All 3, but that’s not the point
• Speed isn’t the most important factor
• Reliability under load is more important
than pure performance
• 1 missed impression = 1 potential disaster
(imagine BMW ads showing up in ThePirateBay)
• (but secretly, I prefer node.js)

8. Welcome to the
distributed app
ecosystem

9. Definitions, definitions
• Distributed: distributed across multiple
geological “clouds”
• Apps: each app handles a small scope of
responsibility
• Ecosystem: apps talk to apps via a
standardized protocol

10. Distributed
• Route 53 -> HAProxy -> Webserver ->
Apps
• Scale webservers by creating new instances
behind HAProxy
• Scale apps by creating new instances in
their own clouds

11. Distributed
Route 53 (DNS)
HAProxy “LB layer”
Webserver Webserver Webserver
App cloud App cloud
(EU) (NA)

12. Apps
• Each app should perform ONE single
domain of function
• Write the app like you’re writing an API
• The simpler the app, the better
• App can be written in any language

13. User app
• List of exposed API:
• create new user
• edit user
• delete user
• merge user accounts
• authenticate user

14. Ecosystem
• Apps communicate using JSON-RPC
• Apps connect to each other using ZMQ
• “Controller” scripts and monit handles
lifecycle

15. Quick & dirty ZeroMQ
• ZMQ = socket library
• Types of socket:
• inproc
• IPC
• TCP
• pgm
• epgm

16. Socket Types
• Push-pull
• Req-rep
• Pub-sub (not covered)
• Dealer-router (not covered)
• Xreq-xrep (not covered)

17. Push-pull
• Push socket pushes data to pull socket(s)
• Pull socket pulls data from push socket(s)
Push Pull
Socket Socket

18. Push-pull
• Use cases:
• Sending email: apps pushes email data to email app
• Logging: apps pushes messages to a logging app
• Data crunching: apps pushes partials to processing app(s) to work on
• Analytics: apps pushes tracking data to analytics app to process

19. Push-pull
• Push socket can connect to multiple pull
sockets (fan-out)
• Pull sockets can listen on multiple push
sockets (fan-in)
• Best part: ZeroMQ handles the load
balancing (usually via a LRU)

20. Advanced Push-Pull
• Example: Log file processing
• Log file needs to be parsed
• Push app (ventilator) parses the log file
• Push app continuously fires partials to pull
apps (workers)
• Workers push into another pull app
(collector)

21. Advanced Push-pull
Worker
Worker
push push
Ventilator Collector
pull pull
Worker
Worker

22. How do we scale?

23. ADD MORE
WORKERS

24. Req-rep
• Req socket sends data to rep socket
• Rep socket receives data from req socket
• Rep socket replies to req socket
• NOTE: Req socket always expects
replies
(remember that girl that you waited 5 years for?)

25. Req-rep
• Use cases:
• Inter-app communications: apps that has dependencies between
each other uses a REQ-REP pattern
• Caching: apps req cached items from cache app, cache app rep with
cached item
• API: 3rd party developers issue req to your API app, which filters the
request and replies with the results

26. Req-rep
REQ REP
Socket Socket
req
REQ REP
Socket Socket
rep
REQ REP
Socket Socket

27. How to scale?

28. ADD MORE REP APPS

29. All together now...

30. Sample ZMQ code:
01 var zmq = require('zmq'),
02 push = zmq.socket('push'),
03 pullA = zmq.socket('pull'),
04 pullB = zmq.socket('pull');
05
06 pullA.on('message', function (msg) {
07 console.log('pullA received ' + msg);
08 });
09
10 pullB.on('message', function (msg) {
11 console.log('pullB received ' + msg);
12 });
13
14 pullA.bindSync('inproc://nodehack-rocks');
15 pullB.bindSync('inproc://oh-la-la');
16
17 push.connect('inproc://nodehack-rocks');
18 push.connect('inproc://oh-la-la');
19
20 for (var i = 0; i < 10; i++) {
21 push.send('this is message ' + i);
22 }

31. Notes
• Each app can define and use multiple ZMQ
sockets
• You can organize your app’s internals using
inproc ZMQ sockets (better MVC)
• Controller scripts are important
• There are advanced patterns for reliability
(read the ZMQ guide)

32. Libraries
• NodeJS ZMQ binding
(https://github.com/JustinTulloss/zeromq.node)
• ZMQ Guide
(http://zguide.zeromq.org/page:all)
• Eazy
(https://github.com/soggie/eazy)

33. - the end -