In this article we will learn the basics of Redis: what it is, how it stores data and what data structures it provides.

Then we will take a look at session framework in Django. First we will learn how it works and then we will implement our own session engine that stores session data in Redis. It will be a great example for learning how to use Redis with Python.

Redis Essentials

Redis is a non-relational database (NoSQL) that stores a mapping of keys to different kind of values. In other words, it’s a key-value store. There are five main types of values that you can store:

• Strings
• Lists
• Hashes
• Sets
• Sorted Sets

There are some other types like bit arrays or hyperloglogs, but they are not that important at the beginning.

Redis stores data in memory but it also supports disk persistence which means that it can write data out to the disk. It could be useful when your server crashes unexpectedly.

Databases in Redis are identified by numbers: 0, 1, 2 and etc. The default database in Redis is a database with number 0. So when you run redis-cli command in the terminal, it will open a database with number 0. But you can switch between databases using select:

$ redis-cli
127.0.0.1:6379> select 1  # switch to db1
127.0.0.1:6379[1]> select 0  # switch back to db0
127.0.0.1:6379>

Let’s talk a little bit about available data structures in Redis and some commands that you can use with them.

Strings

Strings are just regular strings similar to other languages:

127.0.0.1:6379> set hello world  # set the key `hello` to the value `world`
OK
127.0.0.1:6379> get hello  # get the key `hello`
"world"
127.0.0.1:6379> strlen hello  # get the length of `world` value
(integer) 5
127.0.0.1:6379> del hello  # delete the key `hello`
(integer) 1
127.0.0.1:6379> get hello
(nil)

There is one distinction though, Redis allows us to treat strings as numbers:

127.0.0.1:6379> set num 42
127.0.0.1:6379> get num
"42"
127.0.0.1:6379> incr num
(integer) 43

Lists

Lists let us store a sequence of strings. In Redis, lists are represented as linked lists. The advantage of linked lists is that adding or removing elements from head and tail of the list is fast, but disadvantage is that accessing an element by index is slow. Lists were implemented like this because in databases it’s crucial to be able to add elements very fast.

Some operations with lists:

127.0.0.1:6379> rpush names John  # push `John` to the back of `names` list
(integer) 1
127.0.0.1:6379> lpush names Bob  # push `Bob` to the front of `names` list
(integer) 2
127.0.0.1:6379> lrange names 0 -1  # fetch elements from first to the last
1) "Bob"
2) "John"
127.0.0.1:6379> lpop names  # pop from the back
"Bob"
127.0.0.1:6379> lrange names 0 -1
1) "John"

Hashes

Hashes store a mapping of keys to values. However, values should be strings because Redis doesn’t support nested data structures. You can’t store hashes inside hashes.

Some operations with hashes:

127.0.0.1:6379> hset whoami name Denis  # set `name` to `Denis` in `whoami` hash
(integer) 1
127.0.0.1:6379> hset whoami occupation "Web Developer"
(integer) 1
127.0.0.1:6379> hset whoami website "<https://apirobot.me>"
(integer) 1
127.0.0.1:6379> hgetall whoami  # fetch all keys and values
1) "name"
2) "Denis"
3) "occupation"
4) "Web Developer"
5) "website"
6) "<https://apirobot.me>"
127.0.0.1:6379> hdel whoami occupation  # delete `occupation` from `whoami` hash
(integer) 1
127.0.0.1:6379> hdel whoami name
(integer) 1
127.0.0.1:6379> hgetall whoami
1) "website"
2) "<https://apirobot.me>"

If you’re familiar with document databases like MongoDB or relational databases like PostgreSQL, you can think of hash as a document from document databases or as a row from relational databases.

Sets

Sets are similar to lists but the difference is that set only stores unique strings and these strings are unordered.

Some operations with sets:

127.0.0.1:6379> sadd letters a  # add `a` to the `letters` set
(integer) 1
127.0.0.1:6379> sadd letters b
(integer) 1
127.0.0.1:6379> sadd letters c
(integer) 1
127.0.0.1:6379> smembers letters  # fetch the entire set of elements
1) "b"
2) "c"
3) "a"
127.0.0.1:6379> sismember letters "c"  # check whether an item is in set or not
(integer) 1
127.0.0.1:6379> sismember letters "d"
(integer) 0
127.0.0.1:6379> srem letters "c"  # remove `c` from the `letters` set
(integer) 1
127.0.0.1:6379> smembers letters
1) "b"
2) "a"

Sorted Sets

Sorted sets also store unique strings, but each element of sorted set is associated with some score. Based on the score we can sort elements of our set.

Some operations with sorted sets:

127.0.0.1:6379> zadd movies 1 "The Dark Knight"  # add `The Dark Knight` with score `1` to the `movies` sorted set
(integer) 1
127.0.0.1:6379> zadd movies 5 "Fight Club"
(integer) 1
127.0.0.1:6379> zadd movies 10 "The Matrix"
(integer) 1
127.0.0.1:6379> zrange movies 0 -1  # fetch the entire set of elements
1) "The Dark Knight"
2) "Fight Club"
3) "The Matrix"
127.0.0.1:6379> zrange movies 0 -1 withscores  # fetch with scores
1) "The Dark Knight"
2) "1"
3) "Fight Club"
4) "5"
5) "The Matrix"
6) "10"
127.0.0.1:6379> zrangebyscore movies 0 5  # fetch subset
1) "The Dark Knight"
2) "Fight Club"
127.0.0.1:6379> zrem movies "Fight Club"  # remove `Fight Club` from `movies` set
(integer) 1
127.0.0.1:6379> zrange movies 0 -1
1) "The Dark Knight"
2) "The Matrix"

How sessions in Django work

If you’re interested in learning how sessions work by watching a video, then here it is:

Otherwise, let’s continue.

When you open a site, your browser sends an HTTP request to the server. HTTP was designed to be stateless, which means that each request is totally unaware of actions that previous requests did. But if it’s stateless, how are we going to keep track of the “state” between the site and a particular browser such as user’s preferred language or what theme to use and etc? To solve this problem cookies and sessions were born.

Imagine that we have a simple view that increments the number of user’s visits each time the user loads the page:

# views.py
from django.http import HttpResponse
from django.template import Template, Context


def index(request):
    request.session['visits'] = int(request.session.get('visits', 0)) + 1
    t = Template('<h1>Visits: {{ visits }}</h1>')
    c = Context({'visits': request.session['visits']})
    return HttpResponse(t.render(c))

When the browser makes a request the first time, Django creates a new session and stores it in the database. The data that it stores consists of session key, session data and expire date:

$ python manage.py dbshell
sqlite> select * from django_session;
session_key|session_data|expire_date
jts1d1sjixiq5ah8fpou17y1y55jwcaq|OTYzYzg1Nzk5ODNjMDRhM2JlMzk4YTM2ZDQyN2NhMjc0Y2Q3N2Q0YTp7InZpc2l0cyI6MX0=|2019-03-07 07:30:09.727187

Session key is just a random string that Django sends to the user in response headers:

HTTP/1.1 200 OK
Set-Cookie: sessionid=jts1d1sjixiq5ah8fpou17y1y55jwcaq; expires=Thu, 07 Mar 2019 07:30:09 GMT; HttpOnly; Max-Age=1209600; Path=/; SameSite=Lax
Vary: Cookie
X-Frame-Options: SAMEORIGIN
...

The browser saves this session key in cookies and then sends it back to the server each time you try to access the site:

GET / HTTP/1.1
Host: localhost:8000
Cookie: sessionid=jts1d1sjixiq5ah8fpou17y1y55jwcaq
...

Django gets this session key from Cookie header and uses it to make sure you’re the same person who accessed the site earlier.

In session data Django stores the data itself. In our case it’s the number of times user visited our site. Also, it stores data in encrypted state because, as you already guessed, OTYzYzg1Nzk5ODNjMDRhM2JlMzk4YTM2ZDQyN2NhMjc0Y2Q3N2Q0YTp7InZpc2l0cyI6MX0= doesn’t make any sense. But if we try to decode it, we will get exactly what we want:

>>> import base64
>>> base64.b64decode('OTYzYzg1Nzk5ODNjMDRhM2JlMzk4YTM2ZDQyN2NhMjc0Y2Q3N2Q0YTp7InZpc2l0cyI6MX0=')
b'963c8579983c04a3be398a36d427ca274cd77d4a:{"visits":1}'

How to store sessions in Redis

By default, Django stores sessions in your database. In typical Django application, it’s either a PostgreSQL or SQLite database. But our application would work faster if we stored data in the database like Redis. I will show you the difference in performance later. Now, let’s implement our own session engine that stores sessions in Redis:

# yourproject/sessions/redis.py
from django.contrib.sessions.backends.base import SessionBase
from django.utils.functional import cached_property
from redis import Redis


class SessionStore(SessionBase):

    @cached_property
    def _connection(self):
        return Redis(
            host='127.0.0.1',
            port='6379',
            db=0,
            decode_responses=True
        )

    def load(self):
        # Loads the session data by the session key.
        # Returns dictionary.
        return self._connection.hgetall(self.session_key)

    def exists(self, session_key):
        # Checks whether the session key already exists
        # in the database or not.
        return self._connection.exists(session_key)

    def create(self):
        # Creates a new session in the database.
        self._session_key = self._get_new_session_key()
        self.save(must_create=True)
        self.modified = True

    def save(self, must_create=False):
        # Saves the session data. If `must_create` is True,
        # creates a new session object. Otherwise, only updates
        # an existing object and doesn't create one.
        if self.session_key is None:
            return self.create()

        data = self._get_session(no_load=must_create)
        session_key = self._get_or_create_session_key()
        self._connection.hmset(session_key, data)
        self._connection.expire(session_key, self.get_expiry_age())

    def delete(self, session_key=None):
        # Deletes the session data under the session key.
        if session_key is None:
            if self.session_key is None:
                return
            session_key = self.session_key
        self._connection.delete(session_key)

    @classmethod
    def clear_expired(cls):
        # There is no need to remove expired sessions by hand
        # because Redis can do it automatically when
        # the session has expired.

        # We set expiration time in `save` method.
        pass

In order for it to work, we should update settings:

# settings.py
...

SESSION_ENGINE = 'yourproject.sessions.redis'

# Sets the age of session cookies, in seconds.
# When expiration time is 60 seconds,
# it's easier to test whether expiration is working or not.
SESSION_COOKIE_AGE = 60

Now, let’s test how faster our web application can respond to the user if we store sessions in Redis instead of SQLite.

We have a simple script that makes 100 requests to the site:

# performance.py
import requests
import time


def measure_requests_time(n=100):
    start = time.time()
    for _ in range(n):
        requests.get('http://127.0.0.1:8000/')
    end = time.time()
    return end - start

On my computer it took 2 seconds to make these requests when the sessions were stored in SQLite:

$ python -i performance.py
>>> measure_requests_time()
2.029003620147705

But when I switched to Redis, it took a lot less:

$ python -i performance.py
>>> measure_requests_time()
0.3979659080505371

Conclusion

Redis is a great database and it’s popular nowadays. The selling point of Redis is that it’s fast and easy to use. It can be used for different purposes such as storing sessions, caching, maintaining counters and even as message brokers. There are more thing to learn but you already can apply Redis in real world.