Why you should upstream cache your 404′s

There’s a great reddit thread going with the Onion’s Tech team on their recent transition to Django and this gem is buried in the conversation

Deep link to post

And the biggest performance boost of all: caching 404s and sending Cache-Control headers to the CDN on 404. Upwards of 66% of our server time is spent on serving 404s from spiders crawling invalid urls and from urls that exist out in the wild from 6-10 years ago. [Edit: We dropped our outgoing bandwidth by about 66% and our load average on our web server cluster by about 50% after implementing that change]

Amazing result for such a simple change. Bang for your buck optimizations are always the best ones.