vivekn / redis-simple-cache

Simple redis based cache for storing results of python function calls, json encoded strings or html.

redis-simple-cache

redis-simple-cache is a pythonic interface for creating a cache over redis.
It provides simple decorators that can be added to any function to cache its return values.

Requirements:

redis 2.6.2
redis-py 2.7.1 (see requirements.txt file)

Installation:

pip install redis-simple-cache

or to get the latest version

git clone git://github.com/vivekn/redis-simple-cache.git
cd redis-simple-cache
python setup.py install

Usage:

from redis_cache import cache_it_json

@cache_it_json(limit=1000, expire=60 * 60 * 24)
def fib(n):
    if n == 0:
        return 0
    elif n == 1:
        return 1
    else:
        return fib(n-1) + fib(n-2)

limit is the maximum number of keys, expire is the expire time in seconds.
It is always recommended to specify a expire time, since by default redis-server will only remove keys with an expire time set in a event of full memory. But if you wish your keys to never expire, set expire to None.
Note that function arguments and result must be pickleable, since cache_it uses the pickle module.

It is also possible to use redis-simple-cache as a object-oriented cache:

>> from redis_cache import SimpleCache
>> c = SimpleCache(10)  # cache that has a maximum limit of 10 keys
>> c.store("foo", "bar")
>> c.get("foo")
'bar'
>> "foo" in c  # efficient membership test, time-complexity O(1)
True
>> len(c)  # efficient cardinality calculation, time-complexity O(1)
1
>> c.keys()  # returns all keys, time-complexity O(N) with N being the cache c cardinality
set(['foo'])
>> c.flush()  # flushes the cache, time-complexity O(N) with N being the cache c cardinality
>> "foo" in c
False
>> len(c)
0

Check out more examples in the test_rediscache.py file.

Advanced:

Advanced users can customize the decorators even more by passing a SimpleCache object. For example:

my_cache = SimpleCache(limit=100, expire=60 * 60, hashkeys=True, host='localhost', port=6379, db=1, namespace='Fibonacci')
@cache_it(cache=my_cache)
def fib(n):
    # ...

hashkeys parameter makes the SimpleCache to store keys in md5 hash. It is True by default in decorators, but False by default in a new SimpleCache object.
host, port and db are the same redis config params used in StrictRedis class of redis-py. By default, the namespace is the name of the module from which the decorated function is called, but it can be overridden with the namespace parameter.

AUTHOR: Vivek Narayanan

CONTRIBUTORS:

Flávio Juvenal

Sam Zaydel

David Ng

DJ Gilcrease

Johannes Maximilian Toball

Robert Marshall

Ben Hayden

LICENSE: BSD

redis-simple-cache's Issues

Not used the parameter namespace on cache_it_json function

It seems a simple fix bug that the cache_it_json has the parameter namespace but it is passing always None to the inner function cache_it

The prefix depends on the id of the SimpleCache object.

I missed this earlier, but I found that the SimpleCache.make_key() method uses the id of the instance in the key's prefix. The id is determined at runtime, so effectively a new cache is created every time the program is run. If the program is restarted the earlier cache is inaccessible. Is this really a good idea?

@fjsj I would like to hear your views on it.

Ranges of keys

I think it would be nice to have ranges of keys, which are stored as members of a set. One should be able to define a range of any given name, and optionally add keys for any wrapped function to this set. Any number of sets should be possible. This would make it easier to selectively expire groups of keys prematurely, if necessary. I can see a use case for this being where a change made to some data elements will result in no-longer valid cached entries, but their expiration originally set on the wrapped function or functions has not yet been reached.

expire keys

It would be great to implement methods to expire not all keys in set but with specific args and specific functions.

Optional keyword to forcibly bypass cache

So, in case I'm not understanding things correctly, is there a way to handle the following use case: I have a function which I need to cache most of the time, but in a few very specific cases I need it run its course. The function is grabbing things from a web api, most of these do not change but some do, and that fact only matters in one use case. Can I provide an additional keyword just for that specific use case?

e.g.

get_data(dont_cache=True)

I'm thinking the best place to implement this (in case this doesn't already exist) would be here. Thoughts?

Installing package downgrades redis python library

This is because of direct version binding in requirements.txt:
redis==2.7.1

Infinite limit

It would be great to have infinite limit when 'limit' is set to, for example, 0

cache_it and cache_it_json will not properly cache methods that use keyword args

If you wish to cache a method that uses keyword args you need to do it manually instead of using the decorator, which is annoying since it should be a simple fix to add a use_kwargs argument to cache_it that defaults to False, can be True for using all kwargs or can be a comma separated string of kwarg keys to use in the generation of the key

SimpleCache.store() uses SETEX which cannot accept non-positive values

SETEX does not support this:

 https://groups.google.com/d/msg/redis-db/DmtcGkZuD10/OZU16j4ZUpkJ

store() could probably include a check for expire=None and simply use SET for those requests.

Have I understood this issue properly? Would you like a patch for this?

some ut failed

Hi,

Some ut cases failed, such as:
def test_decorator(self)

Impossible to select redis db

rediscache.py contains this code:

            self.connection = RedisConnect(host=self.host,
                                           port=self.port,
                                           db=0,
                                           password=password).connect()

so as you can see db is always 0

Support python 3

This code currently doesn't work under Python 3. The to_unicode() function, and perhaps other things too.

Cache Initial Conditions [Feature Request]

I am planning to implement initial conditions (seed values). Providing seed values can simplify the body of a recursive function significantly.

I image it looking something like this:

# Fibonacci
@cache_it(seed={0:0, 1:1})
def F(i): return F(i-1) + F(i-2)

Since the seed changes all the values of the function, the seed should be fed into the hash used for the function key.

Are there any other considerations I should take into account before implementation?
Does this sound like it falls within the scope of the goals of this repo?

Allow cache_it* decorators to provide a way to cache instance methods

That'd be a good option to have cache_it() and cache_it_json() either auto-detecting instance methods or being marked of such a use case by, for example, strip_first_arg boolean parameter that'd perform something like

pickle.dumps(args[1:])

There's a side effect in such a use case, I know, but could be an option for those who know what they do.

In my case, using instance methods eliminates a need to call getLogger every time I need to log something and provides an option to use one, instance-wide, logger (which certainly cannot be pickled because of file descriptors inside any logger object).