Why you should upstream cache your 404′s

There’s a great reddit thread going with the Onion’s Tech team on their recent transition to Django and this gem is buried in the conversation

Deep link to post

And the biggest performance boost of all: caching 404s and sending Cache-Control headers to the CDN on 404. Upwards of 66% of our server time is spent on serving 404s from spiders crawling invalid urls and from urls that exist out in the wild from 6-10 years ago. [Edit: We dropped our outgoing bandwidth by about 66% and our load average on our web server cluster by about 50% after implementing that change]

Amazing result for such a simple change. Bang for your buck optimizations are always the best ones.

Redis: Relations in a NoSQL world: Using Hashes

So just yesterday we posted a tutorial on how to use redis to store relational despite relations not being supported. Soon after we published the documentation on the new redis hash type went online. Now hashes by themselves aren’t exactly relations but, more so an object field store. Extending the same concepts from our first article in namespace utilization and using hashes we can accomplish the same thing in a more formal fashion.

We will repeat the same exercise from the first article, creating a username password store, using hashes.

Read the rest of this entry »

Redis: Relations in a NoSQL world

In the first article in our series on Redis we talked about how to get started and the basics of the simple data structures that are available in redis. The simple structures are good for basic operations like storing strings and keeping counters but, using it for anything more complex requires relating one set of data to another. At first glance this is a bit of a problem since redis by design is a flat dictionary with no relations but, with a bit of application code and adherence to some mental programming standards you can build some quite complex applications using redis.

Read the rest of this entry »

Getting Started: Redis and Python

So if you have been following NoSQL movement, the migration of some types of data to non-relational datastores has recently picked up speed. For web (and other developers) this has lead to some impressive engineering resources developing some amazing tools being open sourced for the world to use. One that caught my eye recently has been Salvatore Sanfilippo’s redis which has been taken under the wing of VMWare, solidifying and validating the great work that Salavatore has done making redis an amazing tool in any developers arsenal.

A very simplified explanation of redis is that it is an in memory key-value store like memcached but, it is persistent on disk, unlike memcached which is volatile. Along with being disk-persistent redis also supports some basic data structures like lists, sets, ordered sets, hashes, and of course basic key-value string storage like memcached. Redis, with disk-persistence and basic data structures, remains blazing fast with published benchmarks of 110,000 SETs per second, about 81,000 GETs per second.

This post is the start of a series of articles on redis for Python programmers. A prerequisite for this is going to be some basic Python knowledge, which if you haven’t used before I highly recommend the free web book Dive Into Python. This is going to be a simple overview of the basic data types and usage for Python programmers and we will slowly progress into more complex usages, so if you haven’t done anything with redis before this is a perfect start.

Read the rest of this entry »

Tutorial: Writing our first views

So far in the tutorial we’ve started our project, setup the database, enabled the admin interface, created models for our blog, and we just did a brief tutorial on Django’s ORM and DB-API. Now let’s look at getting data out of our database and onto the web page using views.

This section will be a bit short because we won’t really be using templates, which we’ll cover in the next section. We will just be looking at how the urls.py file and the views.py file interact and how we can use what we learned in our ORM tutorial not in the shell but, in our application.

Read the rest of this entry »

Tutorial: Learning the DB-API and ORM

So far in the tutorial we’ve started our project, setup the database, enabled the admin interface, created models for our blog. We’re going to slow down on actually developing our blog and talk about exactly how we get to the data we’ve stored in the database so far. The Admin Interface is great but, it doesn’t provide good programtic access to the data. Let’s open up a python shell but, we’re going to do it a special way when working with Django. From the command line in our root project directory (blog/)

$ python manage.py shell
Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>>

Now this might look like a normal Python shell because, well it is. We’ve just loaded up some stuff in the background like path information to make our lives a bit easier. Now we have two models that we created in the application “main”. Let’s load those up in the shell.

Read the rest of this entry »

Tutorial: Create our models

So far in the tutorial we’ve started our project, setup the database, enabled the admin interface and now we’re ready to start to define the database for our project, a simple blog.

This is where we will start working with the models.py file in the “main” application directory (blog/main/). The models.py file is a definition file where we tell Django what all of our database models are. Let’s start with some categories for our blog entry.

from django.db import models
from django.contrib import admin

class Category(models.Model):
    Name    =   models.CharField(max_length=255)
    def __unicode__(self):
        return self.Name
admin.site.register(Category)

Let’s walk through this real quick since this was a lot of code real quickly. The first two lines we are first importing Django’s base class “models”. This is what we’ll subclass all of our models off of. The next line imports the Django admin object which we will use to include our model in Django’s admin interface.

Read the rest of this entry »

Tutorial: The Admin Interface

So we’ve setup our application, initilized our database and now we’re ready to take our first look at one of the cooler features in Django, the admin interface.

Let’s open up the urls.py file in the root project (blog/) directory. I’ve removed the some of the commented out lines below for brevity

from django.conf.urls.defaults import *

# Uncomment the next two lines to enable the admin:
# from django.contrib import admin
# admin.autodiscover()

urlpatterns = patterns('',

    # Uncomment the next line to enable the admin:
    # (r'^admin/', include(admin.site.urls)),
)<

Read the rest of this entry »

Tutorial: Setting up the database

So far we’ve started our project in our tutorial but, it really doesn’t do that much. The heart of most web applications is the database. This is where we will store all of the data for our application, in this case our blog posts for our blog.

In the main directory of our project (blog/) we have a file called settings.py. This is where all of the database settings are included. Django currently has built in support for postgre, mysql, sqlite3, oracle, mssql out of the box and you can use any database you’d like for this demo. It doesn’t matter. I’m going to be using sqlite since it’s very easy to use and requires very little configuration. When you open settings.py your find a section with DATABASE_ options. This is where we’ll start making changes

DATABASE_ENGINE = 'sqlite3'
DATABASE_NAME = './blog.db'
DATABASE_USER = ''
DATABASE_PASSWORD = ''
DATABASE_HOST = ''
DATABASE_PORT = ''

Read the rest of this entry »

Tutorial: Starting a Project

So now that you’ve completed the first part of the tutorial lets actually get started working with Django.

Let’s create folder called “django-projects” and from the command line in that folder

django-admin.py startproject blog

This command uses the built in django-admin command which creates a basic file structure for our project in a directory called “blog”. Let’s take a quick glance at the files that it made for us

• __init__.py - This file is a “magic” python file that we can ignore
• manage.py - This main file for working with your project from the command line. You won’t need to edit this file ever but, you’ll be using it a bit in a few minutes to run a development server, and sync up your database.
• settings.py - Your basic config file. Contains your DB connection data, directory to your templates, what apps and extra Django modules you want the framework to load
• urls.py - This is where we’ll define the URL’s (URI’s) for our application in a minute

This is the backbone for our project. Django define’s a project as a wrapper around many applications. So a single project can have lots of little applications inside of it. Right now we only have a project. Let’s create out application, again from the command line inside of the “blog” project directory

Read the rest of this entry »