wushian / cachetower

An efficient multi-layered caching system for .NET

Cache Tower

An efficient multi-layered caching system for .NET

Overview

Computers have multiple layers of caching from L1/L2/L3 CPU caches to RAM or even disk caches, each with a different purpose and performance profile. Why don't we do this with our code?

Cache Tower isn't a single type of cache, its a multi-layer solution to caching with each layer on top of another.

Officially supported cache layers include:

• MemoryCacheLayer (built-in)
• JsonFileCacheLayer (via CacheTower.Providers.FileSystem.Json)
• ProtobufFileCacheLayer (via CacheTower.Providers.FileSystem.Protobuf)
• MongoDbCacheLayer (via CacheTower.Providers.Database.MongoDB)
• RedisCacheLayer (via CacheTower.Providers.Redis)

These various cache layers, configurable by you, are controlled through the CacheStack which allows a few other goodies.

• Local refresh locking (a single instance of CacheStack prevents multiple threads refreshing at the same time)
• Remote refresh locking (see details)
• Optional stale-time for a cache entry (serve stale data while a background task refreshes the cache)
• Remote eviction on refresh (see details)

For a comprehensive understanding of why a multi-layered cache is a good idea, have a read Nick Craver's blog post about caching. Nick's blog post was the inspiration behind Cache Tower and a number of the primary features including stale-time for cache entries.

Package Status

Package	NuGet	Downloads
CacheTower
CacheTower.Extensions.Redis
CacheTower.Providers.Database.MongoDB
CacheTower.Providers.FileSystem.Json
CacheTower.Providers.FileSystem.Protobuf
CacheTower.Providers.Redis

How do I choose what cache layers to use?

There are pros-and-cons to each of the types of cache layers though the idea is to use each layer to your advantage.

• In-memory caching provides the best performance though would use the most memory.
• File system caching can cache a lot of data but is extremely slow relative to in-memory caching.
• Protobuf file caching is faster than JSON file caching but requires defining various attributes across the model.
• MongoDB caching can be fast and convenient under certain workloads (eg. when you don't have Redis).
• Redis caching is about as fast as possible while not being in-memory in the same process (or potentially server).

For more details: Performance Figures & Cache Layer Comparisons

How does Cache Tower compare to other caching solutions?

Cache Tower has been built from the ground up for high performance and low memory consumption. Across a number of benchmarks against other caching solutions, Cache Tower performs similarly or better than the competition.

Where Cache Tower makes up in speed, it may lack a variety of features common amongst other caching solutions. It is important to weigh both the feature set and performance when deciding on a caching solution.

For more details: Comparisons to Cache Tower Alternatives

Extension System

To allow more flexibility, Cache Tower uses an extension system to enhance functionality. Some of these extensions rely on third party libraries and software to function correctly.

Built-in

Cache Tower comes with one extension by default but it is an important one: AutoCleanupExtension

The cache layers themselves, for the most part, don't directly manage the co-ordination of when they need to delete expired data. While the RedisCacheLayer does handle cache expiration directly, none of the other offical cache layers do. Unless you are only using the Redis cache layer, you will be wanting to include this extension.

CacheTower.Extensions.Redis

RedisLockExtension

This extension uses Redis as a shared lock between multiple instances of CacheStack, even across multiple web servers. Using Redis in this way can avoid cache stampedes where multiple different web servers are refreshing values at the same instant.

If you are only running one web server with one instance (singleton) of CacheStack, you won't need this extension.

RedisRemoteEvictionExtension

This extension uses the pub/sub feature of Redis to co-ordinate cache invalidation from other CacheStack instances. This works in the situation where one web server has refreshed a key and wants to let the other web servers know their data is now old.

While this can work independently of using RedisLockExtension or even the RedisCacheLayer, you most likely would be using these all at the same time.

Example Usage

Setup with DI

public void ConfigureServices(IServiceCollection services)
{
	services.AddCacheStack<SomeClassThatExtendsICacheContext>(
		new [] {
			new MemoryCacheLayer(),
			new ProtobufFileCacheLayer("directory/where/the/cache/can/write")
		}, 
		new [] {
			new AutoCleanupExtension(TimeSpan.FromMinutes(5))
		}
	);
}

Setup without DI

Setup and store a singleton of CacheStack

//Context is allowed to be null if you don't need it
var myContext = new SomeClassThatExtendsICacheContext();

var myCacheStack = new CacheStack<SomeClassThatExtendsICacheContext>(myContext, new [] {
	new MemoryCacheLayer(),
	new ProtobufFileCacheLayer("directory/where/the/cache/can/write")
}, new [] {
	new AutoCleanupExtension(TimeSpan.FromMinutes(5))
});

Accessing the Cache

Somewhere in your code base where you are wanting to optionally pull data from your cache.

await myCacheStack.GetOrSetAsync<MyTypeInTheCache>("MyCacheKey", async (old, context) => {
	//Here is where your heavy work code goes, maybe a call to the DB or API
	//Your returned value will be cached
	//"context" here is what you declared in the CacheStack constructor
	return await HoweverIGetMyData();
}, new CacheSettings(TimeSpan.FromDays(1), TimeSpan.FromMinutes(60));