yourcelf / django-jimmypage

Generation-based whole page caching for Django

Jimmy Page alpha 0.1

Jimmy Page is a simple caching app for Django, inspired by Johnny Cache, but for whole pages rather than querysets. It is an automatic generation-based page cache.

If Johnny Cache (generational caching of querysets) is the first line of defense in caching, Jimmy Page is the last: it caches the output of views forever, but with conservative expiration on database writes. Jimmy uses a global "generation" number as part of the cache key, which is incremented whenever any model is saved or deleted, expiring the whole page cache. This technique is similar to that described by Open Logic from the Rails community.

This technique provides easy whole-page caching, with an assurance that no part of the site will ever contain stale content. The conservative approach to expiration allows Jimmy to function in a drop-in manner, without any domain-specific knowledge of how data updates might affect the output of views. It will greatly speed up slowly updated sites, especially when used in combination with Johnny Cache and carefully designed, more aggressive caching for particularly intensive views. This technique is not likely to be effective in sites that have a high ratio of database writes to reads.

Installation

This is the first, as yet largely untested alpha release. Some notes:

• In order to function properly, Johnny Cache should be installed and used. Johnny Cache patches the Django caching framework to allow caching with infinite timeouts, something that this app does not provide alone. If you don't want to use Johnny Cache, you should set the JIMMY_PAGE_CACHE_SECONDS setting to something other than 0.
• If you have any custom SQL that updates the database without emitting post_save or pre_delete signals, things might get screwy. At this stage, Jimmy Page works best with sites using vanilla ORM calls.

Install using pip:

pip install -e git://github.com/yourcelf/django-jimmypage.git#egg=django-jimmypage

or clone the git archive and use setup.py:

python setup.py install

Usage

To use, include jimmypage in your INSTALLED_APPS setting, and define JIMMY_PAGE_CACHE_PREFIX in your settings.py file:

# settings.py
INSTALLED_APPS = (
    ...
    "jimmypage",
    ...
)
JIMMY_PAGE_CACHE_PREFIX = "jp_mysite"

To cache a view, use the cache_page decorator:

from jimmypage.cache import cache_page

@cache_page
def myview(request):
    ...

Any update to any table will clear the cache (by incrementing the generation), unless the tables are included in the JIMMY_PAGE_EXPIRATION_WHITELIST. The defaults can be overridden by defining it in your settings.py. By default it includes:

JIMMY_PAGE_EXPIRATION_WHITELIST = [
    "django_session",
    "django_admin_log",
    "registration_registrationprofile",
    "auth_message",
    "auth_user",
]

Views are cached on a per-user, per-language, per-path basis. Anonymous users share a cache, but authenticated users get a separate cache, ensuring that no user will ever see another's user-specific content. The cache is only used if:

• The request method is GET
• There are no messages stored for the request
• The response status is 200
• The response does not contain a Pragma: no-cache header

Please contribute any bugs or improvements to help make this better!

TODO

Current TODOs include:

• Much more testing, analysis, and code review
• middleware to apply the caching to everything, and a decorator to exclude particular views
• Hooks into Django Debug Toolbar to make debugging easier

django-jimmypage's Issues

Cache is cleared on every launch

At the end of cache.py you call clear_cache()

This means the code runs on import and therefore everytime Django starts - even for management commands.

Is this necessary? I use several frequent cron tasks that run via management commands and I'm looking to minimise the number of times I start from a cold cache.

If I can safely remove this line then my cache will spend considerably longer warm.

Does not play well with CSRF middleware

CSRF middleware sets a "Vary: Cookie" header when {% csrf_token %} is used in a template. However, this happens at a middleware layer farther out than the decorator. Hence, the decorator has no way to observe that csrf_token has influenced the response, and the page gets cached when it shouldn't.

I run intro trouble with pip freeze and your repo

Hi thanks for creating this! It's really sweet for smaller sites.
I have a problem. Not sure if it is my ignorance of pip / git or your installation instructions?:

I install on my dev machine like this:
pip install -e git://github.com/yourcelf/django-jimmypage.git#egg=django-jimmypage

This works fine but when I want to move into prod I run into trouble.

I do pip freeze > requirements.txt which adds the following line:
-e git://github.com/yourcelf/django-jimmypage.git@7763e4d#egg=django_jimmypage-dev

on my prod machine i do pip install -r requirements.txt and run into the following error:

Updating /home/matti/.python_envs/welldone/src/django-jimmypage clone (to 7763e4d) You are not currently on a branch, so I cannot use any
'branch..merge' in your configuration file.
Please specify which remote branch you want to use on the command
line and try again (e.g. 'git pull ').
See git-pull(1) for details.
Complete output from command /usr/bin/git pull -q:

---

Command /usr/bin/git pull -q failed with error code 1

cache_page decorator does not work with class based views

Set up as per the Django documentation

from jimmypage.cache import cache_page

class VideoDetailView(DetailView):
    model = Video
    context_object_name = "video"
    template_name = "video/view.html"

    def get_context_data(self, **kwargs):
        # Call the base implementation first to get a context
        context = super(VideoDetailView, self).get_context_data(**kwargs)
        context['archive_posts'] = Video.objects.all()
        return context

    @method_decorator(cache_page(60*60))
    def dispatch(self, *args, **kwargs):
        return super(VideoDetailView, self).dispatch(*args, **kwargs)

First it fails with

'cache_page' object has no attribute '__name__'

When I added

__name__ = 'cache_page'

to the cache_page class in jimmypage.cache it fails with

The response content must be rendered before it can be accessed.

Am I doing it wrong?

Allow setting of HttpResponse parameters on response

Right now cached pages are always returned as text/html no matter what, even if the original response was text/xml, for example. It would be nice to be able to capture that as well, or allow the user to pass the HttpResponse params as part of the decorator.

I haven't thought this all the ay through yet, so I'm not sure what the most elegant solution might be.