Redis: Relations in a NoSQL world

Posted: March 23rd, 2010 | Author: | Filed under: NoSQL, programming, Python, Redis | 17 Comments »

In the first article in our series on Redis we talked about how to get started and the basics of the simple data structures that are available in redis. The simple structures are good for basic operations like storing strings and keeping counters but, using it for anything more complex requires relating one set of data to another. At first glance this is a bit of a problem since redis by design is a flat dictionary with no relations but, with a bit of application code and adherence to some mental programming standards you can build some quite complex applications using redis.

We hinted at this style of structure in the first article when we got to talking about sets with our reddit like example of story voting.

r_server.sadd("story:5419:upvotes", "userid:9102")

Here we have a set in a redis called “story:5419:upvotes” and every element stored in the set is called “userid:xxxx”. This is an example of a basic relation.

Let’s make this example simpler and go with the concept of username / password storage for a website. Instead of using MySQL to store this information we will use redis.

Relations in a NoSQL world

We have to cheat in redis’s flat name space to make relations in our data. Redis isn’t going to be aware of these relations and unlike RDBMS (like MySQL), redis does nothing to help us out. No index’s, no nifty SQL syntax with WHERE or JOIN to do the work for us. We have to handle all of the relational logic in application code, which in turn means you (the developers) have to do extra documentation explaining just how everything fits together in redis or you are going to lose your data. The benefit you though is raw speed and flexibility during development. Don’t like the constraints one data model is giving you? Start over and just recode; no need to alter a DB schema or change a DB server; redis doesn’t care.

Our requirements

  • Users have a username
  • Usernames have a password in a one-to-one fashion
  • We must be able to create a user and assign a password
  • Users must be able to login (check username against password)
  • We must be able to delete a username and their associated password

Designing our redis namespace

This the format of the description of the redis namespace we’ll be using
[key] – {datatype} – (example values)

We also will use the notation *variable* to denote that in a key name is a variable relating to something else.

[users] – {set} – (“adam”, “bob”, “carol”)
[user:*username*:fullname] – {string} – (“Adam Smith”, “Bob Barker”, “Carol Burnett”)
[user:*username*:password] – {string} – (md5 hash password, no example)

That’s it. As long as we stick with this design pattern here we can keep track of all of the users in our system and handle logins.

Creating a new user

r = redis.Redis("localhost")
from hashlib import md5

def add_user(username, fullname, password):
    if r.sadd("users", username):
        r.set("user:%s:fullname" % username, fullname)
        r.set("user:%s:password" % username, md5(password).hexdigest() )
        return True
    else:
        return False

We’ve created a function called add_user where we are taking in a username, the full name of the user, and password and will return True if we successfully create a new user and False if we don’t. Remember the redis command SADD (Set Add) returns False if the object already exists in the set, so if a user with that username already exists, we can’t create a new user so we return False; otherwise we add it to the set and then we set the associated keys “fullname” and “password” to the appropriate vaues. Hashing the password with a md5 hash since we never store passwords anywhere in plain text.

Let’s take a look at what this looks like being called.

>>> add_user("bob", "Bob Barker", "priceisright")
True
>>> add_user("bob", "Bob Barker", "priceisright")
False

The first time we try to add the username “bob” we succeed but, the second time it fails since there is already a user with that username. Let’s look at the data stored in redis.

>>> r.smembers("users")
set(['bob'])
>>> r.get("user:bob:fullname")
'Bob Barker'
>>> r.get("user:bob:password")
'543f24fdc95f4e3f52fc7a4f2166167e'

We see our key “user” that has a set stored with only one element, our username “bob”. And now we have our related keys “user:*username*:fullname” and “user:*username*:password” which in practice are “user:bob:fullname” and “user:bob:password”. As long as we stick to the naming convention we should always know where everything is!

Let’s go ahead and create a few more users just for good measure

>>> add_user("adam", "Adam Smith", "wealthofnations")
True
>>> add_user("carol", "Carol Burnett", "eartug")
True
>>> r.smembers("users")
set(['carol', 'bob', 'adam'])

Logging a user in

Now that we have created our relational data store in redis and have a few users in there, lets try to log one in.

def authenticate_user(username, password):
    #if username in r.smembers("users"):
    if r.sismember("user", username):
        passhash = md5(password).hexdigest()
        if passhash == r.get("user:%s:password" % username):
            return True
        else:
            return False
    else:
        return False

First thing we do is check to see if the username is in the set of all of the elements in the key “users” using the method sismember (set – is a member). If the username isn’t there we are going to return False since, that user doesn’t even exist. If the user does exist, I take a hash of their submitted password just like we did when we added the user, then we fetch that password from the related key and see if they are the same, if they are, we return True!

Let’s take a look at this in action

>>> authenticate_user("ghost", "idontexist!")
False
>>> authenticate_user("adam", "keynes")
False
>>> authenticate_user("adam", "wealthofnations")
True
>>> authenticate_user("bob", "priceisright")
True

So we can see here, a user that doesn’t exist is not authenticated. A user that does exist but, has the wrong password is denied. Given the right username and the right password a user gets in!. And just to show we can handle multiple users like a proper relational system should “bob” authenticates properly as well

Deleting a user

Eventually we are going to need to delete a user from our system and that is as easy everything else we have done as long as we stick to the same namespace we laid out before

def delete_user(username):
    if username in r.smembers("users"):
        r.srem("users", username)
        r.delete("user:%s:fullname" % username)
        r.delete("user:%s:password" % username)
        return True
    else:
        return False

Again the first thing we do is check that the user exists first, since if they don’t no need to do everything else. After we know the user exists, we remove them from the set of “users” then remove their related fullname and password keys. Let’s check this out in action

>>> authenticate_user("adam", "wealthofnations")
True
>>> delete_user("adam")
True
>>> authenticate_user("adam", "wealthofnations")
False
>>> delete_user("adam")
False
>>> r.smembers("users")
set(['bob', 'carol'])

We can authenticate the user “adam” properly. Then we delete him, and now he can’t authenticate any more. If we try to delete him again we see that we can’t since he doesn’t exist.

Wrapping Up

We can make very complex relational data structures in redis as long as we document and properly use the name space. Keep everything as simple and as straight forward as possible and document everything throughly. Also make sure all your actions on redis can be keep as modular as possible by wrapping them in functions or classes, just like in this example. Then if for some reason you need to alter your “schema” you only have one place to change it in your application.

Footnote:

This is an example and is not a secure method for handling user submitted data and does no sanitization or sanity checks on the information. I am aware of this but, this post was not designed to be a comprehensive analysis of AAA methods just a quick example of how to do basic relations in a key-value store.


17 Comments on “Redis: Relations in a NoSQL world”

  1. 1 DeGizmo » Blog Archive » Redis: Relations in a NoSQL world: Using Hashes said at 8:37 am on March 24th, 2010:

    [...] just yesterday we posted a tutorial on how to use redis to store relational despite relations not being supported. Soon after we published the documentation on the new redis hash type went online. Now hashes by [...]

  2. 2 Adam said at 7:29 pm on March 24th, 2010:

    Bug correction from Brian: http://degizmo.com/2010/03/24/redis-relations-in-a-nosql-world-using-hashes/#comments

  3. 3 Damian Janowski said at 6:18 pm on March 26th, 2010:

    Good intro on how to structure data on Redis.

    Readers may be interested in how Ohm does it. Ohm is a Ruby library that maps objects to Redis back and forth. It supports the native Redis data types and higher level features like relationships, indexes, validations, etc.

    http://ohm.keyvalue.org

  4. 4 Daniel said at 5:27 pm on April 26th, 2010:

    As a quick note, on delete_user, we’re first retrieving the entire list of users to then check if the to-be-deleted user is present, which is fine for a small demo but not good in a production site. Instead, to check for the existence of the user-to-be-deleted, one can use r.sismember(“user”, username) which will return just a boolean (which is also the method the author used in authentication).

  5. 5 Gora Khargosh said at 9:37 pm on January 24th, 2011:

    Shouldn’t that be:

    r.sismember(‘users’, username)

    instead of

    r.sismember(‘user’, username)

    Cheers.

  6. 6 mibbtodihzlk said at 5:09 am on June 29th, 2012:

    Enter comment here.

  7. 7 ¥Ê¥¤¥­ ¥á¥ó¥º said at 3:58 am on October 18th, 2013:

    Have You will Learned Which Samsung Unveils Corby HandsetsSamsung offers intrdouced a pair of completely new cell phones that will a type of equipment Coby, typically the teens through Western world that should be adedd. The newest products incllude typically the Coprby Corby (TXT) and even Coprby (PRO), that happen to be supplied along with typically the originaal Corby S3650-Mobile, currently availlable. The particular Cotrby (TXT) is additionally knnown given that the B3210 and features an important 2. 2-inch LCD, 2MP camcorder with the help of videos guidance, designed FM tuner, adjustable goes over, and even 40 MB from on-ship storage expandable from a microSD business card port. The product harmonizes with SIDES GPRS systems Quad-band. The particular phone are going to be availabe through Western world so that the close from Sept. The particular Crby (PRO) handet boasts a QWERTY kyboard and even will work over the 900 2100MHz HSDPA 7. 2Mbps ‘network ‘. The particular phpone comes with a 2. 8-inch touchscreen LCD guidance and an important 3MP camcorder. The particular camcorder can file an important videos. Otehr attributes incorporate a good Radio with the help of RDS, extractible go over, Wi-Fi, Bluuetooth, Mirco UNIVERSAL SERIAL BUS, 100 MB storage using a microSD business card port and many more. The particular phone will certainly vessel through December. All of this Reasons for The particular LG KP500 Cellular phone PhoneMobile phonees really are just about solely transmission devices. Now, a lot of us desire to be modern thewse devices, gammes boards and even carrier from Online world products. Good to view which will thhese very small items have been completely systematically discover most of these vital featurers. The particular LG KC910 is usually a single intelligent cell phones which can be avaoilable over the portable marketing and sales communications markets. That pohne posseses an concept which will renders an important great imprint in your imagination from people. When you are fervent abotu typically the camcorder, which could touch an important couplpe from photographs from excessive subsequently the LG phne is usually necessary to suit your needs. It gives you 8-megapixel camcorder will be imzage solution from 3264 back button 2448 pixels. Morover, the machine in addition has attributes for instance automotive concentration, videos recordinng, and even vidoe camcorder name. Gamerts will definitely come to be happpy using this machine, seeing that the person gives you progressed efatures great online game. There is various video games that one can enjoy aytime and even at any place. About the toher present, the machine will come in black color coloor, which happens to be the most accepted shade around the globe. Just by getting the machine indicates that one can post and even are given e-maipls with the help of attachemnts at a much more rapiid process. That phoone are going to be a good choice for trades-people so, who generally want to investigate your email address whle they are really definitely not opposite your Computers. LG KP500 Biscuit an alternative style with LG is really a experienced manufacturer. That shape Biscuit cellular phone offers numerous attributes which will most certainly help to make typically the mpobile individuals really are dependnet about them. That attractive system will come in a lot of colours, for instance black color, Vandyke red, anodized silcver and even precious metal tasteful. You should buy the cellular phone through some of these colours. There’s a lot of individuals desire to obtain cell phones according to thheir favoruite colours and even horoscope. That communicationms machine boasts a camcorder 3. 2 the mega pixels whicch typically the snapshot reslution from 2048 back button 1536 pxels. That camcorder will take high-quality imaegs, and you’ll download and read tese photographs on the web, preserve using your pc or perhaps remove them from prrint and even, therefore, supplied them all. Inside standby option, a electric battery is usually 350 hrs and even have a discussion the machine.

  8. 8 YtaMoliCync said at 7:31 pm on November 20th, 2013:

    Enter comment here.

  9. 9 gojivita.net said at 2:22 pm on July 28th, 2014:

    Great post. I was checking continuously this blog
    and I am impressed! Very useful info specially the last part :) I care for
    such info much. I was seeking this particular info for a very long time.
    Thank you and best of luck.

    Here is my website :: goji vita (gojivita.net)

  10. 10 ArthurBori said at 1:35 pm on August 26th, 2014:

    Enter comment here.

  11. 11 Anthonyrend said at 5:40 pm on September 20th, 2014:

    Enter comment here.

  12. 12 CharlieCync said at 7:56 pm on September 20th, 2014:

    Enter comment here.

  13. 13 SanfordTiek said at 7:57 pm on September 20th, 2014:

    Enter comment here.

  14. 14 StevenMl said at 8:03 pm on September 20th, 2014:

    Enter comment here.

  15. 15 WilliamWamb said at 8:49 pm on September 20th, 2014:

    Enter comment here.

  16. 16 Emersonguix said at 11:12 pm on September 20th, 2014:

    Enter comment here.

  17. 17 StevenNeep said at 11:35 pm on September 20th, 2014:

    Enter comment here.


Leave a Reply