Simon Willison had a great presentation at NoSQL EU recently and published his slides and notes. It’s quite a comprehensive presentation that’s a good read for both beginners and advanced users. He presents some great use cases based off of real world examples and utilization.
http://simonwillison.net/2010/Apr/25/redis/
There’s a great reddit thread going with the Onion’s Tech team on their recent transition to Django and this gem is buried in the conversation
And the biggest performance boost of all: caching 404s and sending Cache-Control headers to the CDN on 404. Upwards of 66% of our server time is spent on serving 404s from spiders crawling invalid urls and from urls that exist out in the wild from 6-10 years ago. [Edit: We dropped our outgoing bandwidth by about 66% and our load average on our web server cluster by about 50% after implementing that change]
Amazing result for such a simple change. Bang for your buck optimizations are always the best ones.
Salvatore has posted “part 1″ of the internals of Redis’ Virtual Memory subsystem today. Below is the intro with the link to full documentation on Google Code below.
This document details the internals of the Redis Virtual Memory subsystem. The intended audience is not the final user but programmers willing to understand or modify the Virtual Memory implementation.
Keys vs Values: what is swapped out?
The goal of the VM subsystem is to free memory transferring Redis Objects from memory to disk. This is a very generic command, but specifically, Redis transfers only objects associated with values. In order to understand better this concept we’ll show, using the DEBUG command, how a key holding a value looks from the point of view of the Redis internals.
So just yesterday we posted a tutorial on how to use redis to store relational despite relations not being supported. Soon after we published the documentation on the new redis hash type went online. Now hashes by themselves aren’t exactly relations but, more so an object field store. Extending the same concepts from our first article in namespace utilization and using hashes we can accomplish the same thing in a more formal fashion.
We will repeat the same exercise from the first article, creating a username password store, using hashes.
In the first article in our series on Redis we talked about how to get started and the basics of the simple data structures that are available in redis. The simple structures are good for basic operations like storing strings and keeping counters but, using it for anything more complex requires relating one set of data to another. At first glance this is a bit of a problem since redis by design is a flat dictionary with no relations but, with a bit of application code and adherence to some mental programming standards you can build some quite complex applications using redis.
So if you have been following NoSQL movement, the migration of some types of data to non-relational datastores has recently picked up speed. For web (and other developers) this has lead to some impressive engineering resources developing some amazing tools being open sourced for the world to use. One that caught my eye recently has been Salvatore Sanfilippo’s redis which has been taken under the wing of VMWare, solidifying and validating the great work that Salavatore has done making redis an amazing tool in any developers arsenal.
A very simplified explanation of redis is that it is an in memory key-value store like memcached but, it is persistent on disk, unlike memcached which is volatile. Along with being disk-persistent redis also supports some basic data structures like lists, sets, ordered sets, hashes, and of course basic key-value string storage like memcached. Redis, with disk-persistence and basic data structures, remains blazing fast with published benchmarks of 110,000 SETs per second, about 81,000 GETs per second.
This post is the start of a series of articles on redis for Python programmers. A prerequisite for this is going to be some basic Python knowledge, which if you haven’t used before I highly recommend the free web book Dive Into Python. This is going to be a simple overview of the basic data types and usage for Python programmers and we will slowly progress into more complex usages, so if you haven’t done anything with redis before this is a perfect start.
So far in the tutorial we’ve started our project, setup the database, enabled the admin interface, created models for our blog, and we just did a brief tutorial on Django’s ORM and DB-API. Now let’s look at getting data out of our database and onto the web page using views.
This section will be a bit short because we won’t really be using templates, which we’ll cover in the next section. We will just be looking at how the urls.py file and the views.py file interact and how we can use what we learned in our ORM tutorial not in the shell but, in our application.
So far in the tutorial we’ve started our project, setup the database, enabled the admin interface, created models for our blog. We’re going to slow down on actually developing our blog and talk about exactly how we get to the data we’ve stored in the database so far. The Admin Interface is great but, it doesn’t provide good programtic access to the data. Let’s open up a python shell but, we’re going to do it a special way when working with Django. From the command line in our root project directory (blog/)
$ python manage.py shell Python 2.5 (r25:51918, Sep 19 2006, 08:49:13) [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin Type "help", "copyright", "credits" or "license" for more information. (InteractiveConsole) >>>
Now this might look like a normal Python shell because, well it is. We’ve just loaded up some stuff in the background like path information to make our lives a bit easier. Now we have two models that we created in the application “main”. Let’s load those up in the shell.
So far in the tutorial we’ve started our project, setup the database, enabled the admin interface and now we’re ready to start to define the database for our project, a simple blog.
This is where we will start working with the models.py file in the “main” application directory (blog/main/). The models.py file is a definition file where we tell Django what all of our database models are. Let’s start with some categories for our blog entry.
from django.db import models
from django.contrib import admin
class Category(models.Model):
Name = models.CharField(max_length=255)
def __unicode__(self):
return self.Name
admin.site.register(Category)
Let’s walk through this real quick since this was a lot of code real quickly. The first two lines we are first importing Django’s base class “models”. This is what we’ll subclass all of our models off of. The next line imports the Django admin object which we will use to include our model in Django’s admin interface.
So we’ve setup our application, initilized our database and now we’re ready to take our first look at one of the cooler features in Django, the admin interface.
Let’s open up the urls.py file in the root project (blog/) directory. I’ve removed the some of the commented out lines below for brevity
from django.conf.urls.defaults import *
# Uncomment the next two lines to enable the admin:
# from django.contrib import admin
# admin.autodiscover()
urlpatterns = patterns('',
# Uncomment the next line to enable the admin:
# (r'^admin/', include(admin.site.urls)),
)<
Recent Comments