Welcome to the Django-Cache-Magic Documenation¶

Temporal Caching¶

This is like memoizing an expensive operation. Usually something that doesn’t need to be super-fresh, and aggregates many objects - making invalidation difficult or impossible.:

@cached
def my_expensive_call(arg1, arg2):
    return do_other_things(arg1) * arg2

Instance Caching¶

This is the practice of caching individual model instances. CacheMagic provides a CacheController that you can attach to models to cause automatic caching and invalidations.

class Model(django.models.Model):
    cache = cachemagic.CacheController()
    field = django.models.TextField()

Model.objects.get(pk=27)    # hits the database
Model.cache.get(27)         # Tries cache first

Related Objects Caching¶

Having fetched an instance of a model, a frequent database operation is to find all the instances of another model that are related to your instance via foreign keys. You can attach a RelatedCacheController to your model to enable automatic caching and invalidation of these relations.

instance = Model.cache.get(pk=27)
related_things = instance.things_set.all()  # hits the database
related_things = instance.cache.things_set  # Tries cache first

The RelatedCacheController will automatically detect and cache objects related to the model it resides on by ForeignKeys, ManyToManyFields, and OneToOneFields.

Thundering Herd Protection¶

Any caching will be useless when a cache key expires and thousands of requests try to recompute the value at the same time. CacheMagic provides a cache backend for redis that prevents this problem by designating only one client to recompute the value while others simply read the existing cache value.

CACHES['default'] = {
    'BACKEND': 'cachemagic.cache.RedisHerdCache',
    'LOCATION': ':'.join([REDIS_HOST, str(REDIS_PORT), '0']),
    'OPTIONS': {
        'PASSWORD': REDIS_PASSWORD,
    },
}

Using in model methods¶

In most cases you should use an object’s primary key as the cache key instead of serializing the entire object.:

from django.db import models
from cachemagic.decorators import cached

class Model(models.Model):
    field1 = IntegerField()
    field2 = TextField()

    @cached(key=lambda self: self.pk, timeout=180)
    def get_my_related_things(self):
        return [other(item) for item in self.related_things.select_related()]

Reading From Cache¶

You can use the cache controller like a very simple manager: currently only the .get(pk) operation is supported. This will try to get and return the model instance from cache. In the event that the key does not have a cache entry, the value is read from the database using the model’s default manager. The result is placed into the cache before being returned to the caller.

obj = Model.cache.get(pk=933)

Just like objects.get(), cache.get() may raise a Model.DoesNotExist exception. A DoesNotExist marker is placed in cache when an instance is deleted or an attempt to fetch a non-existent row is made, preventing subsequent requests against the cache from hitting DB or returning stale data.

Instance Cache Keys¶

The default CacheController creates keys based on your model’s app, name and primary key, separated by colons: app_label:model_name:primary_key. This should present you with a unique key for each object.

import hashlib

class HashCacheController(CacheController):
    """ Hashes the cache key. This creates keys that are difficult to type
        by hand, but can avoid problems related to key content and length.
    """
    def make_key(self, pk):
        key = super(HashCacheController, self).make_key(pk)
        return hashlib.sha256(key).hexdigest()

class ModelVersionCacheController(CacheController):
    """ Versions each cache key with the model's CACHE_VERSION attribute.
        Updating the model's version when altering it's schema will
        effectively invalidate all cached instances.
    """
    def make_key(self, pk):
        model_version = getattr(self.model, 'CACHE_VERSION', 0)
        key = ':'.join([super(HashCacheController, self), model_version])
        return key

Cache Timeouts¶

The default cache timeout is one hour. You can specify a number of seconds to timeout as the timeout parameter in the CacheController constructor. :

cache = CacheController(timeout=(60 * 60 * 24 * 7)) # timeout in one week

class LongCacheController(CacheController):
    # timeouts longer than 30 days are treated as absolute timestamps by
    # memcached; that makes 30 days the largest naive value we can use.
    DEFAULT_TIMEOUT = 60 * 60 * 24 * 30

Caveats¶

CacheMagic relies on the post_save and post_delete signals to keep your cache up to date. Performing operations that alter the database state without sending these signals will result in your cache becoming out of sync with your database.

Introduction¶

Given a model instance, one frequent database query is to get instances of another model that are related. This is commonly accomplished with the use of a Foreign Key.

We will be using the following models as examples throughout this document:

from django.db import models
from cachemagic.controllers import RelatedCacheController

class Person(models.Model):
    name = models.CharField(max_length=200)

    cache = RelatedCacheController()

class Book(models.Model):
    author = models.ForeignKey(Person, related_name='books')
    title = models.CharField(max_length=200)
    published_date = models.DateTimeField()

    class Meta:
        default_ordering = ('-published_date')

Suppose you have an authorship view, displaying all of the books that a given author has published. The view would typically look something like this:

def authorship(request, author_id):
    try:
        author = Person.objects.get(pk=author_id)
    except Person.DoesNotExist:
        raise Http404("No such person")
    books = author.books.all()
    return render(
        request,
        {'author': author, 'books': books},
        'authorship.html'
    )

This pattern will invoke two database queries: one to fetch a Person, and one to fetch the books with a foreign key relationship to the author. We can use the cachemagic features to try the cache first.

author = Person.objects.get(pk=author_id)   # database query
author = Person.cache.get(pk=author_id)     # cached query

books = author.books.all()                  # database query
books = author.cache.books                  # cached query

Reading From Cache¶

Given an instance of an object with a RelatedCacheController, all of the attributes on the instance to fetch related objects are mirrored on the controller. If the instance has a .thing_set and a RelatedCacheManager assigned to cache, then instance.cache.thing_set will return the same values as list(instance.thing_set.all()).

Cache Keys¶

A cache key for the instance is obtained by calling the same make_key(pk) function described in Instance Cache Keys. The key for the related objects is the instance key, appended with the related name of the collection.

author = Person.objects.get(pk=1)   # get an instance of a Person in the sample_app
author.cache.books                  # cache key is sample_app:Person:1:books

Cache Timeouts and Multicache¶

The RelatedCacheController accepts the same timeout and backend arguments as CacheController.

cache = RelatedCacheController(
        timeout=(60 * 60 * 24 * 7),     # timeout in one week
        backend='my_app_cache'),        # use the cache backend named
                                        # 'my_app_cache' in settings.py
    )

Usage¶

To setup the protection, simply use the RedisHerdCache backend provided. Here is an example configuration:

CACHES['default'] = {
    'BACKEND': 'cachemagic.cache.RedisHerdCache',
    'LOCATION': ':'.join([REDIS_HOST, str(REDIS_PORT), '0']),
    'OPTIONS': {
        'PASSWORD': REDIS_PASSWORD,
    },
    'VERSION': 0,
}