WebSocket is cool, and you probably already played with it. But it’s just a transport technology. If you have thousands of client connections you need to do lots of improvements to make it scalable, reliable and achieve high performance. You need to implement many things on top of it. We are building financial data streaming platform for thousands of traders using WebSocket. I’m going to share my experience and cover such techniques as delta delivery, conflation, dynamic throttling, bandwidth and frequency limitation and other. I will also do a live demo of how to build scalable WebSocket backend from scratch using Java and Spring.

1. 1
JEECONF 2018
BUILDING SCALABLE
WEBSOCKET BACKEND
KONSTANTIN SLISENKO
LEAD SOFTWARE ENGINEER
MAY 18-19, 2018

2. 2
kslisenko@gmail.com
Konstantin Slisenko
• Java Team Lead at EPAM
• I’m building trading software
• 8+ years with Java
• 3+ years with WebSocket
github.com/kslisenko

3. 3

4. 4

5. 5
WHAT IS WEBSOCKET
standard: tools.ietf.org/html/rfc6455

6. 6
HTTP
WHAT IS WEBSOCKET
GET /demo HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
standard: tools.ietf.org/html/rfc6455

7. 7
HTTP
WHAT IS WEBSOCKET
HTTP/1.1 101 Web Socket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
GET /demo HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
standard: tools.ietf.org/html/rfc6455

8. 8
TCP
HTTP
WHAT IS WEBSOCKET
HTTP/1.1 101 Web Socket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
GET /demo HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade
DATA DATA DATA DATA DATA
DATA DATA DATA DATA DATA
standard: tools.ietf.org/html/rfc6455

9. 9
• Permanent connection
WEBSOCKET PROVIDES

10. 10
• Permanent connection
• Bidirectional communication
WEBSOCKET PROVIDES

11. 11
• Permanent connection
• Bidirectional communication
• Sending text or binary data
WEBSOCKET PROVIDES

12. 12
• Permanent connection
• Bidirectional communication
• Sending text or binary data
• Guaranteed delivery (TCP/IP)
WEBSOCKET PROVIDES

13. 13
• API and application protocol
WEBSOCKET DOESN’T PROVIDE

14. 14
• API and application protocol
• Network failure handling*
WEBSOCKET DOESN’T PROVIDE
*Only basic handling

15. 15
• API and application protocol
• Network failure handling*
• Network slowness handling*
WEBSOCKET DOESN’T PROVIDE
*Only basic handling

16. 16
• API and application protocol
• Network failure handling*
• Network slowness handling*
• Bandwidth/frequency limitation
WEBSOCKET DOESN’T PROVIDE
*Only basic handling

17. 17
ONE DOES NOT SIMPLY
BUILD HIGH PERFORMANCE APPLICATION
USING ONLY STANDARD WEBSOCKET FEATURES

18. 18
High performance WebSocket backend
Basic WebSocket backend

19. 19
BACKEND
BASIC WEBSOCKET

20. 20
data
subscribe
WebSocket
data
subscribe
WebSocket

21. 21
data
subscribe
WebSocket
data
subscribe
WebSocket
any
protocol
data
subscribe

22. 22
data
subscribe
WebSocket
any
protocol
data
data
subscribe
WebSocket
subscribe
IBM
AAPL
IBM
AAPL

23. 23
AAPL
data
subscribe
WebSocket
AAPL
IBM
any
protocol
data
IBM
EPAM
data
subscribe
WebSocket
subscribe
IBM
TSLA

24. 24
BACKEND
BASIC WEBSOCKET
LIVE DEMO

25. 25

26. 26
TOO MUCH DATA?
SEND LESS DATA!

27. 27
High performance WebSocket backend
Max frequency limit
Basic WebSocket backend

28. 28
HOW DO WE
SEND LESS DATA?

29. 29
RESAMPLING
No limits (10 updates/s)

30. 30
RESAMPLING
No limits (10 updates/s)
2 updates/s

31. 31
No limits (10 updates/s)
200 300 400 500 700 900
2 updates/s (last available)
900 1000600
600
100
100
RESAMPLING + CONFLATION

32. 32
No limits (10 updates/s)
100 200 300 400 500 600 700 800
2 updates/s (average)
300
900 1000
RESAMPLING + CONFLATION
800
100 + 200 + 300 + 400 + 500
5
= 300
600 + 700 + 800 + 900 + 1000
5
= 800

33. 33
data
data
subscribe
data
subscribe
subscribe
AAPL
IBM
IBM
TSLA
WebSocket
WebSocket
any
protocol
RECEIVINGTHREAD(S)

34. 34
data
data
subscribe
data
subscribe
subscribe
AAPL
IBM
IBM
TSLA
WebSocket
WebSocket
any
protocol
SHAREDMEMORY
RECEIVINGTHREAD(S)

35. 35
data
data
subscribe
data
subscribe
subscribe
AAPL
IBM
IBM
TSLA
WebSocket
WebSocket
SENDINGTHREAD(S)
any
protocol
SHAREDMEMORY
RECEIVINGTHREAD(S)

36. 36
data
data
subscribe
data
subscribe
subscribeAAPL
IBM
IBM
TSLA
TSLA
data
IBM
data
AAPL
data

37. 37
data
data
subscribe
data
subscribe
subscribeAAPL
IBM
IBM
TSLA
TSLA
data
info
info
info
info
rate limit
rate limit
rate limit
rate limit
IBM
data
AAPL
data

38. 38
WEBSOCKET
FREQUENCY LIMIT
LIVE DEMO

39. 39
MAX FREQUENCY LIMIT
2 KB/s*
Max frequency limit
33 KB/s*
No limits
-95%
* for demo application including gzip compression

40. 40
WHAT ELSE, CAP?

41. 41
WHAT ELSE, CAP?
SEND EVEN LESS DATA!

42. 42
High performance WebSocket backend
Schema
Max frequency limit
Basic WebSocket backend

43. 43
AAPL
snapshot
BID=10
ASK=15
LAST=12
BID SIZE=20
ASK SIZE=30
SCHEMA

44. 44
AAPL
snapshot
BID=10
schema
ASK=15
LAST=12
BID SIZE=20
ASK SIZE=30
BID=10
ASK=15
SCHEMA
BID
ASK

45. 45
AAPL
snapshot
BID=10
schema
ASK=15
LAST=12
BID SIZE=20
ASK SIZE=30
BID=10
ASK=15
BID=10
ASK=15
LAST=12
BID SIZE=20
ASK SIZE=30
SCHEMA
BID
ASK
schema
*

46. 46
High performance WebSocket backend
Delta delivery
Schema
Max frequency limit
Basic WebSocket backend

47. 47
DELTA DELIVERY
BID=10
ASK=15
LAST=12
BID SIZE=20
ASK SIZE=30
BID=15
ASK=15
LAST=12
BID SIZE=20
ASK SIZE=30
BID=20
ASK=20
LAST=12
BID SIZE=20
ASK SIZE=30
BID=20
ASK=20
LAST=12
BID SIZE=50
ASK SIZE=30
message message message message

48. 48
DELTA DELIVERY
BID=10
ASK=15
LAST=12
BID SIZE=20
ASK SIZE=30
BID=15
ASK=15
LAST=12
BID SIZE=20
ASK SIZE=30
BID=20
ASK=20
LAST=12
BID SIZE=20
ASK SIZE=30
BID=20
ASK=20
LAST=12
BID SIZE=50
ASK SIZE=30
message message message message

49. 49
DELTA DELIVERY
BID=10
ASK=15
LAST=12
BID SIZE=20
ASK SIZE=30
BID=15
ASK=15
LAST=12
BID SIZE=20
ASK SIZE=30
BID=20
ASK=20
LAST=12
BID SIZE=20
ASK SIZE=30
BID=20
ASK=20
LAST=12
BID SIZE=50
ASK SIZE=30
message message message message
BID=10
ASK=15
LAST=12
BID SIZE=20
ASK SIZE=30 BID=15
BID=20
ASK=20 BID SIZE=50
snapshot update update update

50. 50
High performance WebSocket backend
Position-based protocolPosition-based protocol
Delta delivery
Schema
Max frequency limit
Basic WebSocket backend

51. 51
{
symbol: AAPL,
bid: 10,
ask: 12,
last: 11,
bidSize: 100,
askSize: 200
}
AAPL|10|12|11|100|200
? bytes ? bytes

52. 52
{
symbol: AAPL,
bid: 10,
ask: 12,
last: 11,
bidSize: 100,
askSize: 200
}
AAPL|10|12|11|100|200
85 bytes 21 bytes

53. 53
{
symbol: AAPL,
bid: 10,
ask: 12,
last: 11,
bidSize: 100,
askSize: 200
}
AAPL||12|||200
46 bytes 14 bytes

54. 54
BANDWIDTH
OPTIMIZATION
LIVE DEMO

55. 55
SCHEMA + DELTA DELIVERY + POSITION-BASED PROTOCOL
400 B/s*
Using schema
2 KB/s*
No schema
-80%
* for demo application including gzip compression
370 B/s*
Delta delivery
-8%
310 B/s*
Position-based
-15%

56. 56
GOOD JOB!

57. 57
High performance WebSocket backend
Dynamic bandwidth control
Position-based protocol
Delta delivery
Schema
Max frequency limit
Basic WebSocket backend

58. 58

59. 59
DYNAMIC BANDWIDTH CONTROL

60. 60
DYNAMIC BANDWIDTH CONTROL
Data

61. 61
DYNAMIC BANDWIDTH CONTROL
Data

62. 62
DYNAMIC BANDWIDTH CONTROL
DataData

63. 63
DYNAMIC BANDWIDTH CONTROL
DataData
Data Data
ACK ACK ACK ACK ACK

64. 64
DYNAMIC
LIVE DEMO
BANDWIDTH CONTROL

65. 65
DYNAMIC BANDWIDTH CONTROL
420 B/s*
Bandwidth control
310 B/s*
No control
+35%
* for demo application including gzip compression

66. 66
High performance WebSocket backend
Position-based protocol
Delta delivery
Schema
Max frequency limit
Basic WebSocket backend
Bandwidth limit per user
Dynamic bandwidth control

67. 67

68. 68
BANDWIDTH LIMIT PER USER
paying users
Data

69. 69
BANDWIDTH LIMIT PER USER
paying users
Data
Data
free users

70. 70
200 KB/s
BANDWIDTH LIMIT PER USER

71. 71
200 KB/s
100 KB/s
100 KB/s
BANDWIDTH LIMIT PER USER

72. 72
BANDWIDTH LIMIT
LIVE DEMO
PER USER

73. 73
ARE WE READY
FOR PRODUCTION?

74. 74

75. 75
http://lightstreamer.com

76. 76
LIGHTSTREAMER SERVER

77. 77
LIGHTSTREAMER SERVER
Data
adapter

78. 78
LIGHTSTREAMER SERVER
Data
adapter
Metadata
adapter

79. 79
CONCLUSION

80. 80
No improvements
2 KB/s*Max frequency limit
400 B/s*Schema
370 B/s*Delta delivery
310 B/s*Position-based
420 B/s*Bandwidth limit
-95%
-80%
-8%
-15%
+35%
33 KB/s*
* for demo application including gzip compression
BANDWIDTH OPTIMIZATION*

81. 81
READ
High Performance Browser Networking
What every web developer should know
about networking and web performance
Free reading: https://hpbn.co/
By Ilya Grigorik
Publisher: O'Reilly Media
Release Date: September 2013

82. 82
http://lightstreamer.com/docs

83. 83
More Than Just WebSockets for Real-Time
Multiplayer Games and Collaboration
On YouTube and Slideshare
By Alessandro Alinone (CEO of Lightstreamer)

84. 84
1. Improve user-experience
2. Reduce bandwidth
3. Handle network failures
TAKE AWAYS

85. 85
QUESTIONS?
THANK YOU!

86. 86
github.com/kslisenko/streaming