Yiftach Shoolman Adrian Colyer Laura Merling Reynold Xin

1. Salvatore Sanfilippo
Creator of Redis

2. Yiftach Shoolman
Co-Founder & CTO Redis Labs

3. 2 Hot Topics
+
1 New Trend

4. Redis on New
Memory Technologies

5. 0.10
250.00
0.10
500.00
0.10
1.00
10.00
100.00
1,000.00
$0
$1
$2
$3
$4
$5
$6
$7
$8
$9
$10
DRAM NV-DIMM/PM NVMe SSD SATA SSD
microsec
$/GB
1 GB cost vs. Read/Write Performance
1 GB cost Read latency Write latency
$9
$0.4
High DRAM Cost and Low Flash Performance

6. High DRAM Cost and Low Flash Performance
0.10
90.00
250.00
0.10
25.00
500.00
0.10
1.00
10.00
100.00
1,000.00
$0
$1
$2
$3
$4
$5
$6
$7
$8
$9
$10
DRAM NV-DIMM/PM NVMe SSD SATA SSD
microsec
$/GB
1 GB cost vs. Read/Write Performance
1 GB cost Read latency Write latency
$9
$0.4
$1

7. High DRAM Cost and Low Flash Performance
0.10
1.00
90.00
250.00
0.10
1.00
25.00
500.00
0.10
1.00
10.00
100.00
1,000.00
$0
$1
$2
$3
$4
$5
$6
$7
$8
$9
$10
DRAM NV-DIMM/PM NVMe SSD SATA SSD
microsec
$/GB
1 GB cost vs. Read/Write Performance
1 GB cost Read latency Write latency
$9
$0.4
$1
$2

8. Bring Redis Performance on Persistent Memories
Close to Redis Performance on DRAM
The Challenge

9. Before dealing with this challenge…

10. Is Persistent Memory Really Persistent ?
NVMe
NV-DIMM
Cloud Instance

11. NVMe
NV-DIMM
Failed Instance
Is Persistent Memory Really Persistent ?

12. Is Persistent Memory Really Persistent ?
NVMe
NV-DIMM
Failed Instance
NVMe
NV-DIMM
New Empty Instance

13. Data Loss
Is Persistent Memory Really Persistent ?
NVMe
NV-DIMM
Failed Instance
NVMe
NV-DIMM
New Empty Instance

14. Alternative Solution
Redundant
Flash Array
Cloud Instance

15. Alternative Solution
New non-empty Instance
Redundant
Flash Array
Failed Instance

16. Another Alternative
• Requires ultra high-speed/low-
latency network like Fiber-Channel
or Infiniband
• Latency is still at ~ 100microsec
• Expensive, complex and not
supported by cloud vendorsNew non-empty Instance
Redundant
Flash Array
Failed Instance

17. Persistent Memory = Ephemeral Memory
Persistent Storage
AOF, Snapshot
NVMe
NV-DIMM
Cloud Instance

18. Redis on Flash (RoF)
• Flash used as a RAM extender (not as a persistent storage)
• In RAM: dict & keys & hot values
• On Flash: cold values
• Values are either in RAM or on Flash
• Multi-shard; multi-threaded; asynchronous
• Pluggable storage engine (RocksDB and IBM’s ark)
• 100% compatible with the OSS Redis

19. Flexible RAM/Flash Ratios

20. Behind the Scene

21. Increase RAM ratio  Scale-out Cluster
2500 GB2500 GB
50%50%

22. RoF is Fast
Avg: 2.04M ops/sec
Max: 2.14M ops/sec
Avg: 0.91msec
Max: 0.98 msec
% below 1msec:100%
Avg: 313RMB / 9.4WMB
Max: 1.71RGB / 96WMB
Avg: 1.45Gbps (Tx) / 0.97Gbps (Rx)
Max: 1.6Gbps (Tx) / 1.2Gbps (Rx)
>2M Ops/sec, > 1GB disk bandwidth, @ <1 msec latency
read
write
read
write

23. Master-Master Replication

24. Customer Perspective
• Keep the speed of Redis
• Maintain all Redis commands - simple and complex
• Flexible configuration across regions
• Any # of regions, single/multi-az/rack, any combination of nodes/shards
• Fully-managed service

25. How it Should Work?
Start
Synced
Not synced

26. How it Should Work?
Write
Locally
Synced
Not synced
App
App

27. How it Should Work?
Distribute P2P
Asynchronously
Synced
Not synced
App
App

28. Partially
Converged
How it Should Work?
Synced
Not synced
App
App

29. App
App
Fully
Converged
How it Should Work?
Synced
Not synced

30. Our Approach to Master-Master
• Behaves like local Redis, performs like local Redis
• Based on CRDT (Conflict-free Replication Data Type)
• Supports Strong Eventual Consistency
• No consensus protocol; no need for R + W > N
• Any two nodes that have received the same (unordered) set of updates will be
in the same state
• More to come…

31. 1 New Trend

32. Foreword

33. If you are here …

34. You’re probably a geek

35. Or a geekette (the female geek)

36. • Dealing with simple problems
• Writing code in shitty languages
• Using silly npm packages
• Not refactoring your poor old architecture because of politics
You are tired of…

37. And you really want to solve real problems

38. So what are your options ?

39. Option #1 – Create your own startup

40. 50% of startups fail in the first year
and
95% percent fail within five years

41. Option #2 – Create a Redis Module

42. Get respect from the community

43. And (hopefully) get many stars on GitHub

44. How do you make $$$ ?

45. • A marketplace for OSS and closed source modules
• All modules are certified by Redis Labs for these platforms:
Module Hub
1M+ instances 350K+ instances 100+ F500 Companies
www.redismodules.com

46. Geo Clustering

47. Geo Clustering

48. Adding a Module to Module Hub
Upload
A few clicks

49. Adding a Module to Module Hub
Upload
Certification
Process
One monthA few clicks

50. Adding a Module to Module Hub
Upload Alpha
Certification
Process
One monthA few clicks Still not shown in
the marketplace.
Test by 10 selected
customers

51. Adding a Module to Module Hub
Upload Alpha Beta
Certification
Process
One month At least 1
month free
A few clicks Still not shown in
the marketplace.
Test by 10 selected
customers

52. Adding a Module to Module Hub
Upload Alpha Beta GACertification
Process
One month Still not shown in
the marketplace.
Test by 10 selected
customers
At least 1
month free
A few clicks

53. So – can you make $$$ here ?

54. 3.15
2.40
21.00
8.70
24.57
10.61
0.00
5.00
10.00
15.00
20.00
25.00
30.00
Full text search Prefix search
Average Latency (msec)
RLEC Elasticsearch Solr
20,045
6,831
690
3,686
621
3,133
0
5,000
10,000
15,000
20,000
25,000
Full text search Prefix search
Ops/sec
RLEC Elasticsearch Solr
85% higher
32x higher
7.8x faster 4.1x faster
redisearch
The world fastest text search engine

55. For only $0.114/hr
redisearch – the world fastest text search engine
3.15
2.40
21.00
8.70
24.57
10.61
0.00
5.00
10.00
15.00
20.00
25.00
30.00
Full text search Prefix search
Average Latency (msec)
RLEC Elasticsearch Solr
20,045
6,831
690
3,686
621
3,133
0
5,000
10,000
15,000
20,000
25,000
Full text search Prefix search
Ops/sec
RLEC Elasticsearch Solr
85% higher
32x higher
7.8x faster 4.1x faster

56. The Possibilities are
Truly Endless
Thank You
@yiftachsh

57. Salvatore Sanfilippo
Creator of Redis

58. Adrian Colyer
Venture Partner at Accel
Author of the Morning Paper Blog
Former CTO, VMWare, Pivotal and Spring Source

59. As I get older, my memory just keeps
getting better and better!

60. blog.acolyer.org
350
Foundations
Frontiers

61. A new era of hardware (and what it might mean
for Redis)
A new era of data structures?
Talk outline
a
b

62. Human
computers
at Dryden by NACA (NASA) -
Dryden Flight Research Center
Photo Collection
http://www.dfrc.nasa.gov/Gallery/Photo/Places/HT
ML/E49-54.html. Licensed under Public Domain
via Commons -
https://commons.wikimedia.org/wiki/File:Human_c
omputers_-
_Dryden.jpg#/media/File:Human_computers_-
_Dryden.jpg

63. Computing on a human scale
63
10ns
70ns
10ms
10s
1:10s
116d
Registers
& L1-L3
File on
desk
Main
memory
Office filing
cabinet
HDDTrip to the
warehouse

64. Compute
HTM
Persistent Memory NI
FPGA
GPUs
Memory
NVDIMMs
Persistent Memory
Networking
100GbE
RDMA
Storage
NVMe
Next-gen NVM
Next generation hardware
All change please
64

65. 2-10m
Computing on a human scale
65
10s
1:10s
116d
File on
desk
Office filing
cabinet
Trip to the
warehouse
4x capacity
fireproof local
filing cabinets
23-40m
Phone
another office
(RDMA)
3h20m
Next-gen
warehouse

66. The new ~numbers everyone should know
66
Latency Bandwidth Capacity/IOPS
Register 0.25ns
L1 cache 1ns
L2 cache 3ns 8MB
L3 cache 11ns 45MB
DRAM 62ns 120GBs 6TB - 4 socket
NVRAM’ DIMM 620ns 60GBs 24TB - 4 socket
1-sided RDMA in Data Center 1.4us 100GbE ~700K IOPS
RPC in Data Center 2.4us 100GbE ~400K IOPS
NVRAM’ NVMe 12us 6GBs 16TB/disk,~2M/600K
NVRAM’ NVMf 90us 5GBs 16TB/disk,
~700/600K

67. NVMe Example: Redis on Flash
67
• 2M IOPS
• Sub-ms
latency
• 1:4
DRAM:NVM
split
(40x SATA
SSD)

68. NVMf Example: E8 Storage
68
$12M Series B led by Accel
• 10M IOPS in single appliance, accessed via RDMA
• 100us/50us read/write latency
• ...and this is with NAND Flash!

69. Data tiering in heterogeneous memory systems
DDR-DRAM NVM
Capacity per CPU 100s of GBs Terabytes
Read latency 1x 2x to 4x
Write bandwidth 1x ⅛ x to ¼ x
Estimated cost 5x 1x
Endurance 1016 106 to 108
Dulloor et al. 2016 (EuroSys)

70. What will it mean?
70
this!
All DRAM
(baseline)
All NVM
tiered memory,
with increasing
NVM proportion
Impact on cost?
Impact on
performance?
Impact on
performance/$?

71. Key-value store results - performance impact
Proportion of
DRAM
Performance
relative to all
DRAM
NVM only 1.15 - 1.45x
1/8 1.14 - 1.4x
1/4 1.06 - 1.13x
1/2 1.05x
beware!

72. Key-value store results - cost impact
Proportion of
DRAM
Performance/$
relative to all
DRAM
NVM only 3.45 - 4.5x
1/8 3.1x
1/4 2.3x
1/2 1.6x

73. And don’t forget about capacity!
Proportion of
DRAM
Performance
relative to all
DRAM
Performance/$
relative to all
DRAM
Capacity relative
to all DRAM
NVM only 1.15 - 1.45x 3.45 - 4.5x 4 - 10x
1/8 1.14 - 1.4x 3.1x 3.6 -8.9x
1/4 1.06 - 1.13x 2.3x 3.3 - 7.8x
1/2 1.05x 1.6x 2.5 - 5.5x

74. 74
Ultra low latency, PBs of data
• The RAMCloud storage system - Ousterhout et al. 2015
• Implementing linearizability at large scale and low latency - Lee
et al. 2015
1000+
Servers
PB+
Memory
RDMA RIFL

75. 75
RAMCloud Performance
Operation End-to-end Latency
median (99.9%-ile)
Read 4.7 (9.2)μs
Write (durable) 13.4 (148)μs
Transaction 20μs
5-object transaction 27μs
TPC-C throughput (10 nodes) 35K tps

76. 76
No compromises
Distributed transactions with consistency, availability, and performance
Dragojević et al. 2015
“This paper demonstrates that new software in modern data
centers can eliminate the need to compromise. It describes the
transaction, replication, and recovery protocols in FaRM, a main
memory distributed computing platform. FaRM provides
distributed ACID transactions with strict serializability, high
availability, high throughput and low latency. These protocols
were designed from first principles to leverage two hardware
trends appearing in data centers: fast commodity networks
with RDMA and an inexpensive approach to providing non-
volatile DRAM.”

77. 77
FaRM Performance
Performance
TPC-C
tps (90 nodes) 4.5M
99%-ile latency 1.9ms
K-V Store*
qps (20 nodes) 146M
latency (at peak throughput) 35μs
(* FaRM: Fast Remote Memory - Dragojević et al. 2014)

78. 78
Exploiting HTM...
Persist.
Memory
RDMA HTM DrTM
Fast in-memory transaction processing using
RDMA & HTM - Wei et al. 2015
5.5M
tps
(TPC-C,
6 node cluster)

79. Redis is way more than just a ‘K-V store’...
b A new era of data structures?

80. Evolution of the humble data structure
80
Collection
classes
& stdlibs
?
single
thread/core
Concurrent
data types
multi-
thread/core
Distributed
data types
multi-server
Shared
concurrent
data types
shared-server
PACELC

81. 81
Libraries, Servers, & PACELC
App Instance App InstanceApp Instance
CRDT
Library
DS Server DS Server
Client Client Client Client Client
CRDT
Server

82. 82
CRDT Evolution
Operation Based
● prepare and effect
● smaller messages
● require reliable
exactly-once causal
broadcast
● membership
management
● no batching
State Based
● state shipping and
idempotent join
● larger messages
● support unreliable
communication
channels
● accommodate
batching
Delta-State Based
● local execution, ship
representation of the
effect & idempotently
join
● smaller messages
● support unreliable
communication
channels
● accommodate
batching

83. Data Structures! E.g. GCounter and PNCounter
{ replica_id value }
83
What are CRDTs made of?
{“A”->3, “B”->5}
HINCRBY gcounter A 1
{“A”->4, “B” -> 5}
HVALS gcounter… 4+5 = 9
{“A” -> 4, “B” -> 7}
{“A”->3, “B”->5}
HINCRBY gcounter B 1
HINCRBY gcounter B 1
{“A”->3, “B” -> 7}
{“A” -> 4, “B” -> 7}
delta B -> 7
delta A -> 4
A B

84. Causal Context X Dot Store
84
Causal ∂-CRDTs
version vector
(Redis hash) Dot.Event
(Redis set or hash)
Dot =
replica_id:sequence_no
INCR
+ a delta queue for propagation...
WATCH my_crdt:seq_no
next = INCR my_crdt:seq_no
MULTI
HSET my_crdt:ccontext replica_id next
SADD my_crdt:dotstore “replica_id:next:val”
RPUSH my_crdt:deltaq delta
EXEC

85. A new era of hardware (and what it might mean
for Redis)
A new era of data structures
Summary
a
b

86. Thank You
@adriancolyer

87. Salvatore Sanfilippo
Creator of Redis

88. Laura Merling
Former VP and GM of IoT Initiative at SAP

89. The Internet of Things
Real Time Reality

90. The IoT Landscape
PLATFORMS AND ENABLEMENT
SOFTWARE, FULL-STACK,
DEVELOPER, ANALYTICS,
OPEN SOURCE

91. Big Data with an IOT View
Volume
Velocity
Data streams and near-real time;
RFID tags, sensors and smart
metering.
Variety
Structured and unstructured.
Variability
Inconsistent; event triggered.
Complexity
Multiple sources; connect and correlate
relationships, hierarchies and multiple data
linkages.
Volume
Billions of sensors and machine to
machine transactions.

92. Analytics Continuum
DiagnosticDescriptive PrescriptivePredictive
What Happened Why Did it Happen What Will Happen Make it Happen
In Real Time

93. Geospatial
The Redis IOT Opportunity
PubSub DatabaseCaching
Time Series

94. Challenges
• Latency
• Automation
• Complexity
Oil and Gas

95. Challenges
• Transformer
• Distribution
Electric Vehicle Charging

96. Manufacturing
Challenges
• Unpredictable
• Adherence
• Unavailable

97. Construction
Challenges
• Awareness
• Congestion
• Slow

98. Connected Home
Challenges
• Latency
• Interaction
• Simplification

99. Retail
Challenges
• Latency
• Interaction/Engagement
• Inventory

100. The Data
MINING
IoT FOR THE UNFILTERED
TRUTH

101. Thank You
@magicmerl

102. Salvatore Sanfilippo
Creator of Redis

103. Reynold Xin
Chief Architect Spark at Databricks

104. Spark and Redis: In-situ analytics
beyond Hadoop
Reynold Xin
@rxin
2016-05-11 RedisConf

105. About Databricks
Founded by creators of Spark in 2013
Cloud enterprise data platform
- Managed Spark clusters
- Interactive data science
- Production pipelines
- Data governance, security, …

106. Please put up your hand
if you know what Spark is?

107. Please put up your hand
if you think your significant other
knows what Spark is?

108. open source data processing engine built around
speed, ease of use, and sophisticated analytics
largest open source data project with 1000+
contributors

113. “Spark is the Taylor Swift
of big data software.”
- Derrick Harris, Fortune

114. Analytics: a traditional approach
Hadoop is complex to operate
Data consistency is difficult to maintain

115. Analytics in-situ
SQL
ML
Streaming

116. Analytics in-situ
SQL
Streaming
MLEnable SQL analytics over Redis
Use Redis to store streaming data
Use Redis to serve results generated by Spark

117. Demo
by Richard Garris

118. Discount code: Meetup16SF

119. Thank You
@rxin

120. Salvatore Sanfilippo
Creator of Redis

121. Enjoy RedisConf
Updates
• Your morning break is located downstairs in the Sponsor Showcase
• Three concurrent Breakout Sessions will begin at 11:15am
• The Developer Workshop begins at 1:00pm in the Developer Café
• Save your Redis Community Raffle ticket for today’s drawing
for a chance to win a Phantom Drone

122. 11:15am-12:00pm New Redis Capabilities: Own, Make, Share
1:00pm-1:45pm SSH I/O Streaming Via Redis-based Persistent
Message Queue
2:00pm-2:45pm Walmart and IBM Revisit the Linear Road Benchmark
3:15pm-4:00pm Back your App with MySQL & Redis, the Cloud
Foundry Way
4:15pm-5:00pm Postgres and Redis Sitting in a Tree
122
BREAKOUTS: WEDNESDAY, MAY 11

123. Salvatore Sanfilippo
Creator of Redis