The document is a presentation from the Global NoSQL Benchmark Framework meetup, presented by Filipe Oliveira, Principal Performance Engineer at Redis, on January 10, 2024. It provides an insightful exploration into the pivotal role of latency in modern digital services. Oliveira's presentation underscores how the current market demand for rapid, low-latency solutions is redefining the benchmarks for database performance. Oliveira begins by delineating the importance of benchmarking in real-world scenarios, highlighting the often-overlooked gap between anticipated performance and actual outcomes. This segment of the presentation emphasizes that theoretical benchmarks often fail to capture the complexities and dynamic nature of real-life applications. Consequently, Oliveira advocates for a more nuanced and practical approach to benchmarking, one that mirrors the challenges and demands of contemporary service environments. The core of the presentation delves into the challenges of achieving consistent low-latency performance in distributed database systems. Through a series of case studies, Oliveira illustrates how these challenges manifest in different scenarios, ranging from small-scale operations to large, global systems. These case studies provide practical insights into optimizing performance, particularly focusing on the role of Redis in enabling efficient data management and retrieval. A significant portion of the presentation is dedicated to discussing the trade-offs between consistency, availability, and performance in world-scale systems. Oliveira explains the complexities involved in balancing these three critical aspects, particularly in distributed environments where data consistency and availability are paramount. He presents various strategies and architectural choices that can help in navigating these trade-offs, ensuring that systems remain both robust and agile. Oliveira's talk concludes with an emphasis on the importance of performance engineering in the modern digital landscape. He argues that performance is not merely a technical concern but a critical business strategy that directly impacts user experience and service delivery. The presentation, rich in technical detail and practical advice, serves as an invaluable resource for professionals involved in database management, system architecture, and performance optimization. Overall, the presentation offers a comprehensive view of the current state and future directions of NoSQL databases, particularly in the context of performance and scalability. It's a must-see for anyone interested in understanding how to optimize database systems for speed, reliability, and efficiency in an increasingly demanding digital world.

1. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Global NoSQL Benchmark Framework:
Embracing Real-World Performance
Filipe Oliveira
Jan 10, 2024

2. Filipe Oliveira
Principal Performance Engineer at Redis

3. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Agenda
Latency is the new outgage
Market Expectations for Low-Latency Services
Benchmarking Real-World Scenarios
Market Expectations vs. Real-World Performance: A Reality Check
KEY TAKEAWAYS

4. Latency is the new outage!

5. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
What latency rates are actually
required to support near-term and
long-term use cases?

6. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
The market
demands!
Source:
5G Unlocks a World of Opportunities, HUAWEI
“Not only will operators need
to provide services (... )
efficiently,
but they will need to
commercialize those
services rapidly and
effectively.”
Click or Scan for full story

7. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
The market
demands!
Source:
5G Unlocks a World of Opportunities, HUAWEI
“Not only will operators need
to provide services (... )
efficiently,
but they will need to
commercialize those
services rapidly and
effectively.”
Click or Scan for full story

8. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
The market
demands!
Source:
5G Unlocks a World of Opportunities, HUAWEI
“Not only will operators need
to provide services (... )
efficiently,
but they will need to
commercialize those
services rapidly and
effectively.”
Click or Scan for full story

9. Market Expectations for Low-Latency
Services

10. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
The market
demands!
Source:
“Cutting Through the
Edge Computing Hype” interview data of 150
enterprises in North America, STL
Click or Scan for full story
Enterprises are
emphasizing…
CONSISTENCY OF
LATENCY
…as the most important SLA.

11. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Latency corridors distribution
Single-digit latency is now a thing!

12. Benchmarking real world scenarios!

13. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Our common benchmark scenarios
Heavily rely on
single-region
or
single-zone
abstractions…

14. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Benchmarking Real-World* Scenarios
EU (Paris)
eu-west-3
SA (São Paulo)
sa-east-1
US West (N. California)
us-west-1
Asia Pacific (Sidney)
ap-southeast-2
Asia Pacific (Tokyo)
ap-northeast-1

15. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Benchmarking Real-World* Scenarios
EU (Paris)
eu-west-3
SA (São Paulo)
sa-east-1
US West (N. California)
us-west-1
Asia Pacific (Sidney)
ap-southeast-2
Asia Pacific (Tokyo)
ap-northeast-1

16. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Operations
● You can easily add a DB
● Designed to map the ycsb
Read/Insert Signatures
● Control on Read
Preference
● Control on Write
Replication
// DB is the layer to access the database to be benchmarked.
type DB interface {
// Read reads a record from the database and returns a map of each field/value pair.
// table: The name of the table.
// key: The record key of the record to read.
// readPreference: route read operations to the primaries (0) or to the secondaries/replicas (1)
// fields: The list of fields to read, nil|empty for reading all.
Read(ctx context.Context, table string, key string, readPreference int64, fields []string)
(map[string][]byte, error)
// Insert inserts a record in the database. Any field/value pairs will be written into the
// database.
// table: The name of the table.
// key: The record key of the record to insert.
// numReplicas: number of replicas that acknowledged the insert.
// numReplicasTimeout: timeout in milliseconds to ack.
// values: A map of field/value pairs to insert in the record.
Insert(ctx context.Context, table string, key string, numReplicas int64, numReplicasTimeout
int64, values map[string][]byte) error
}
Click or Scan for benchmark repo

17. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Operations
● You can easily add a DB
● Designed to map the ycsb
Read/Insert Signatures
● Control on Read
Preference
● Control on Write
Replication
PRIMARY
EU (Paris)
eu-west-3
US West (N. Cal.)
us-west-1
REPLICA
EU (Paris)
eu-west-3
CLIENT EU
WRITE
READ
Case 1: route read operations to
the primaries (0)
Click or Scan for benchmark repo

18. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Operations
● You can easily add a DB
● Designed to map the ycsb
Read/Insert Signatures
● Control on Read
Preference
● Control on Write
Replication
PRIMARY
EU (Paris)
eu-west-3
US West (N. Cal.)
us-west-1
REPLICA
EU (Paris)
eu-west-3
CLIENT EU
WRITE
READ
Case 2: route read operations to
the replicas (1)
Click or Scan for benchmark repo
ASYNC
REPLICATION

19. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Operations
● You can easily add a DB
● Designed to map the ycsb
Read/Insert Signatures
● Control on Read
Preference
● Control on Write
Replication
PRIMARY
EU (Paris)
eu-west-3
US West (N. Cal.)
us-west-1
REPLICA
EU (Paris)
eu-west-3
CLIENT EU
WRITE
READ
Case 2: route read operations to
the replicas (1)
Click or Scan for benchmark repo
ASYNC
REPLICATION
HAS LAG!

20. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Operations
● You can easily add a DB
● Designed to map the ycsb
Read/Insert Signatures
● Control on Read
Preference
● Control on Write
Replication
PRIMARY
EU (Paris)
eu-west-3
US West (N. Cal.)
us-west-1
REPLICA
EU (Paris)
eu-west-3
CLIENT EU
1 WRITE
READ
Case 2: Control number of
replicas that acknowledged the
insert
DEFAULT: NO REPLICA ACK
Click or Scan for benchmark repo
2 WRITE ACK
ASYNC
REPLICATION
HAS LAG!

21. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Operations
● You can easily add a DB
● Designed to map the ycsb
Read/Insert Signatures
● Control on Read
Preference
● Control on Write
Replication
PRIMARY
EU (Paris)
eu-west-3
US West (N. Cal.)
us-west-1
REPLICA
EU (Paris)
eu-west-3
CLIENT EU
1 WRITE
READ
2 REPLICATION
Click or Scan for benchmark repo
3 REPL ACK
4 WRITE ACK
Case 3: Control number of
replicas that acknowledged the
insert
1 REPLICA ACK

22. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Setup
5 DB Locations
● Primary: US West (California)
● 4 Replicas: Paris, Sao Paulo,
Sidney, Tokyo
● OSS Redis version 7.2.4
5 Benchmark Locations
● California, Paris, Sao Paulo,
Sidney, Tokyo
Traffic always stays on the global AWS
backbone and never traverses the
public internet.
Click or Scan for benchmark repo
Deploy Infrastructure:

23. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Setup
5 DB Locations
● Primary: US West (California)
● 4 Replicas: Paris, Sao Paulo,
Sidney, Tokyo
● OSS Redis version 7.2.4
5 Benchmark Locations
● California, Paris, Sao Paulo,
Sidney, Tokyo
Traffic always stays on the global AWS
backbone and never traverses the
public internet.
Click or Scan for benchmark repo
Benchmark utility installation:

24. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Setup
5 DB Locations
● Primary: US West (California)
● 4 Replicas: Paris, Sao Paulo,
Sidney, Tokyo
● OSS Redis version 7.2.4
5 Benchmark Locations
● California, Paris, Sao Paulo,
Sidney, Tokyo
Traffic always stays on the global AWS
backbone and never traverses the
public internet.
Click or Scan for benchmark repo
Benchmark utility prebuilt bins:
OS Arch Link
Linux amd64 (64-bit X86) global-nosql-benchmark-linux-amd64
Linux arm64 (64-bit ARM) global-nosql-benchmark-linux-arm64
Darwin amd64 (64-bit X86) global-nosql-benchmark-darwin-amd64
Darwin arm64 (64-bit ARM) global-nosql-benchmark-darwin-arm64
Windows amd64 (64-bit X86) global-nosql-benchmark-darwin-amd64
Windows arm64 (64-bit ARM) global-nosql-benchmark-darwin-arm64

25. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Setup
5 DB Locations
● Primary: US West (California)
● 4 Replicas: Paris, Sao Paulo,
Sidney, Tokyo
● OSS Redis version 7.2.4
5 Benchmark Locations
● California, Paris, Sao Paulo,
Sidney, Tokyo
Traffic always stays on the global AWS
backbone and never traverses the
public internet.
Click or Scan for benchmark repo
5x Benchmark Agents:

26. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Setup
5 DB Locations
● Primary: US West (California)
● 4 Replicas: Paris, São Paulo,
Sidney, Tokyo
● OSS Redis version 7.2.4
5 Benchmark Locations
● California, Paris, São Paulo,
Sidney, Tokyo
Traffic always stays on the global AWS
backbone and never traverses the
public internet.
Click or Scan for benchmark repo
1x Benchmark Coordinator:

27. Expectations vs Real World
Measurements: A Reality Check

28. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Case 1: route
read operations
to the primaries
● Latencies on primary are
single digit
● Primary vs Replica Regions
> 130X difference
○ ON global AWS
backbone
● P50 46% variance even on
Replica Regions

29. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Case 1: route
read operations
to the primaries
● Latencies on primary are
single digit
● Primary vs Replica Regions
> 130X difference
○ ON global AWS
backbone
● P50 46% variance even on
Replica Regions
● Physical distance does
NOT explain all 141.55 ms 173.74 ms 138.96 ms 108.55 ms
1 Month p50 Latency between
N.California Regio and..
Physical distance between
N.California and… 8,892 km 10,141 km 12,118 km 8,551 km

30. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Case 1: route
read operations
to the primaries
● Latencies on primary are
single digit
● Primary vs Replica Regions
> 130X difference
○ ON global AWS
backbone
● P50 46% variance even on
Replica Regions
● P99 14% variance even on
Replica Regions

31. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Case 1: route
read operations
to the primaries
● Latencies on primary are
single digit
● Primary vs Replica Regions
> 130X difference
○ ON global AWS
backbone
● P50 46% variance even on
Replica Regions
● P99 14% variance even on
Replica Regions
● READS similar to WRITES

32. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
46%
P50 variance
solely between Replica Regions
130x
Diff Primary vs Replica
Regions even on AWS
only backbones
READS =
WRITES
Case 1 by the numbers

33. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Case 2: route
read operations
to the replicas
● Reads Latencies are single
digit (*might be stall)
● READ vs WRITE > 130X
difference
○ ON global AWS
backbone

34. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Single Digit
Latency on
Local Reads
130x
Diff READ vs WRITE
even on AWS only
backbones
STALLED
READS
Case 2 by the numbers

35. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Case 3: route
read operations
to the replicas
and ACK Inserts
● Reads Latencies are single
digit (*might be stall)
● 0 ACK is same as case 2
● 1 ACK
○ we get 87% Insert
latency Increase
○ Primary WRITE
Latency == Replica
WRITE Latency

36. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Case 3: route
read operations
to the replicas
and ACK Inserts
● Reads Latencies are single
digit (*might be stall)
● 0 ACK is same as case 2
● 3 ACK
○ we get 2.2x Insert
latency Increase
○ Write Latencies
converge to worst
link performance

37. © 2024 Redis Ltd. All rights reserved. Confidential (Internal use only)
Consistent**
READS
>500x
Diff READ vs WRITE
even on AWS only
backbones
WORST LINK
DICTATES
WRITE
PERFORMANCE
Case 3 by the numbers

38. KEY TAKEAWAYS…

39. Is this a problem?

40. It depends…

41. Goal: Planet wide performance…
Read Consistency
vs Write Performance
READ:WRITE
patterns
Writes need
to be local?

42. We’ve discussed Planet wide Primary-Replica
performance…
Read Consistency
vs Write Performance
READ:WRITE
patterns
Writes need
to be local?

43. There are others
Planet wide
Consensus
Primary
Replica
Multi-Master
(with conflict
resolution)
NO DIVERGENCE ON PERFORMANCE but
towards worst case
Reads and Writes are Local but can be
divergence

44. In Summary

45. In Summary
Reliable world scale systems… require
trade-offs between consistency and
availability… and deeply affect
performance

46. Follow up links
● CRDTs and Redis
● Diving into CRDTs
● Active-Active Geo-Distribution
Now Generally Available in Azure
Cache for Redis Enterprise

47. Thank you.