An introduction to Redis for the SQL practitioner, covering data types and common use cases. The video of this session can be found at: https://www.youtube.com/watch?v=8Unaug_vmFI

1. Redis Use Patterns
An Introduction to the SQL Practitioner
@ItamarHaber #DevConTLV 2014

2. About
A Redis Geek and Chief Developers Advocate at .com
We provide Redis Cloud – an enterprise-class Redis service for
developers (infinitely scalable, highly-available with auto failover, top
performing hosted Redis off AWS, Google, Azure & IBM SoftLayer)
Get your free
t-shirt and
sticker outside!

3. What’s Redis? (REmote DIctionary Server)
• Open-source (BSD), in-memory, persist-able key-value advanced datastore
• Key-value means something like
CREATE TABLE redis (
k VARCHAR(512MB) NOT NULL,
v VARCHAR(512MB),
PRIMARY KEY (k)
);
• 6 data types, 160 commands, blazing fast
• Created in 2009 by @antirez
(a.k.a Salvatore Sanfilippo)
• Source: https://github.com/antirez/redis
• Website: http://redis.io

4. Why Redis? Because It Is Fun!
• Simplicity  rich functionality, great flexibility
• Performance  easily serves 100K’s of ops/sec
• Lightweight  ~ 2MB footprint
• Production proven (name dropping) 
Twitter Pintrest StackOverflow teowaki many more
…

5. Redis
Makes
You
.
.
.
.
THINK!
• about how data is stored
• about how data is accessed
• about efficiency
• about performance
• about the network
• …
• Redis is a database construction kit
• Beware of Maslow's "Golden" Gavel/Law
of Instrument:
"If all you have is a hammer,
everything looks like a nail"

6. Pattern: Caching Calls to the DB
Motivation: quick responses, reduce load on DBMS
How: keep the statement's results using the Redis STRING data type
def get_results(sql):
hash = md5.new(sql).digest()
result = redis.get(hash)
if result is None:
result = db.execute(sql)
redis.set(hash, result)
# or use redis.setex to set a TTL for the key
return result

7. STRINGs
• Are the most basic data type
• Are binary-safe
• Is used for storing:
• Strings (duh) – APPEND, GETRANGE,
SETRANGE, STRLEN
• Integers – INCR, INCRBY, DECR, DECRBY
• Floats – INCRBYFLOAT
• Bits – SETBIT, GETBIT, BITPOS, BITCOUNT,
BITOP
http://xkcd.com/171/

8. Pattern: Avoiding Calls to the DB
Motivation: server-side storage and sharing of data that doesn't need a
full-fledged RDBMS, e.g. sessions and shopping carts
How: depending on the case, use STRING or HASH to store data in Redis
def add_to_cart(session, product, quantity):
if quantity > 0:
redis.hset('cart:' + session, product, quantity)
else:
redis.hrem('cart:' + session, product)
def get_cart_contents(session):
return redis.hgetall('cart:' + session)

9. The HASH Data Type
• Acts as a Redis-within-Redis  contains key-value pairs
• Have their own commands: HINCRBY, HINCRBYFLOAT, HLEN, HKEYS,
HVALS…
• Usually used for aggregation, i.e. keeping related data together for
easy fetching/updating (remember that Redis is not a relational
database). Example:
Using separate keys Using hash aggregation
user:1:id  1 user:1 id  1
user:1:fname  Foo fname  Foo
user:1:lname  Bar lname  Bar
user:1:email  foo@acme.com email  foo@acme.com

10. Denormalization
• Non relational  no foreign keys, no
referential integrity constraints
• Thus, data normalization isn't practical
• Be prepared to have duplicated data, e.g.:
> HSET user:1 country Mordor
> HSET user:2 country Mordor
…
• Tradeoff:
Processing Complexity ↔ Data Volume

11. Pattern: Lists of Items
Motivation: keeping track of a sequence, e.g. last viewed profiles
How: use Redis' LIST data type
def view_product(uid, product):
redis.lpush('user:' + uid + ':viewed', product)
redis.ltrim('user:' + uid + ':viewed', 0, 9)
…
def get_last_viewed_products(uid):
return redis.lrange('user:' + uid + ':viewed', 0, -1)

12. Key Points About Key Names
• Key names are "limited" to 512MB (also the values btw)
• To conserve RAM & CPU, try avoid using
unnecessarily_longish_names_for_your_redis_keys
because they are more expensive to store and compare
(unlike an RDBMS's column names, key names are saved for
each key-value pair)
• On the other hand, don't be too stringent (e.g 'u:<uid>:r')
• Although not mandatory, the convention is to use colons
(':') to separate the parts of the key's name
• Your schema is your keys' names so keep them in order

13. Pattern: Queues (apropos the list data type)
Motivation: a producer-consumer use case, asynchronous job
management, e.g. processing photo uploads
def enqueue(queue, item):
redis.lpush(queue, item)
def dequeue(queue):
return redis.rpop(queue)
# or use brpop for blocking pop

14. Is Redis ACID? (mostly) Yes!
• Redis is (mostly) single threaded, hence every
operation is
• Atomic
• Consistent
• Isolated
• WATCH/MULTI/EXEC allow something like
transactions (no rollbacks)
• Server-side Lua scripts ("stored procedures")
also behave like transactions
• Durability is configurable and is a tradeoff
between efficiency and safety

15. Pattern: Searching
Motivation: finding keys in the database, for example all the users
How #1: use a LIST to store key names
How #2: the *SCAN commands
def do_something_with_all_users():
first = True
cursor = 0
while cursor != 0 or first:
first = False
cursor, data = redis.scan(cursor, 'user:*')
do_something(data)

16. Pattern: Indexing
Motivation: Redis doesn't have indices, you need to maintain them
How: the SET data type (a collection of unordered unique members)
def update_country_idx(country, uid):
redis.sadd('country:' + country, uid)
def get_users_in_country(country):
return redis.smembers('country:' + country)

17. Pattern: Relationships
Motivation: Redis doesn't have foreign keys, you need to maintain them
> SADD user:1:friends 3 4 5 // Foo is social and makes friends
> SCARD user:1:friends // How many friends does Foo have?
> SINTER user:1:friends user:2:friends // Common friends
> SDIFF user:1:friends user:2:friends // Exclusive friends
> SUNION user:1:friends user:2:friends // All the friends

18. ZSETs (Sorted Sets)
I HAVE
CDO
IT'S LIKE
OCD
BUT ALL THE LETTERS ARE
IN ALPHABETICAL ORDER
AS THEY SHOULD BE
• Are just like SETs:
• Members are unique
• ZADD, ZCARD, ZINCRBY, …
• ZSET members have a score that's used for
sorting
• ZCOUNT, ZRANGE, ZRANGEBYSCORE
• When the scores are identical, members
are sorted alphabetically
• Lexicographical ranges are also supported:
• ZLEXCOUNT, ZRANGEBYLEX

19. Pattern: Sorting
Motivation: anything that needs to be sorted
How: ZSETs
> ZADD friends_count 3 1 1 2 999 3
> ZREVRANGE friends_count 0 -1
3
1
2
Set members (uids)
Scores (friends count)

20. The SORT Command
• A command that sorts LISTs, SETs and SORTED SETs
• SORT's syntax is the most complex (comparatively) but SQLers should
feel right at home with it:
SORT key [BY pattern] [LIMIT offset count]
[GET pattern [GET pattern ...]]
[ASC|DESC] [ALPHA]
[STORE destination]
• SORT is also expensive in terms of complexity  O(N+M*log(M))
• BTW, SORT is perhaps the only ad-hoc-like command in Redis

21. Pattern: Counting Things
Motivation: statistics, real-time analytics, dashboards, throttling
How #1: use the *INCR commands
How #2: use a little bit of BIT*
def user_log_login(uid):
joined = redis.hget('user:' + uid, 'joined')
d0 = datetime.strptime(joined, '%Y-%m-$d')
d1 = datetime.date.today()
delta = d1 – d0
redis.setbit('user:' + uid + ':logins', delta, 1)
def user_logins_count(uid):
return redis.bitcount(
'user:' + uid + ':logins', 0, -1)
I love to
COUNT(*) data!
One datum,
two data,
three data!
HA HA HA HA!

22. Pattern: Counting Unique Items
How #1: SADD items and SCARD for the count
Problem: more unique items  more RAM 
How #2: the HyperLogLog data structure
> PFADD counter item1 item2 item3 …
• HLL is a probabilistic data structure that counts (PFCOUNT) unique items
• Sacrifices accuracy: standard error of 0.81%
• Gains: constant complexity and memory – 12KB per counter
• Bonus: HLLs are merge-able with PFMERGE


23. Wait, There's More!
• There are 107 additional commands that we didn't cover 
• Expiration and eviction policies
• Publish/Subscribe
• Data persistency and durability
• Server-side scripting with Lua
• Master-Slave(s) replication
• High availability with Sentinel
• Redis v3 == Cluster (currently in beta)
• …

24. Where Next?
• Try the interactive demo and get a free 25MB Redis database in the
cloud at http://redislabs.com
• Need help?
• RTFM: http://redis.io/documentation
• Ask the redis-db mailing list
• Visit #redis on Freenode IRC
• Email me: itamar@ .com