Multiple cachelib instances or multiple pools about cachelib

[ss7pro]

I'm not part of Meta cachelib team but we did some benchmarking and noticed that as you add more threads to instance performance per thread goes down. This is mostly related to locks that needs to be held during evictions when you add a new item and there's not enough memory to accomodate that. So if you plan to put significant amount of load on your cache and run many threads you're better separating data types to different instances.

[haowu14]

Hi @wenhaocs, your understanding is correct that different pools will be using the same cache trait thus the same eviction policy.

On multiple instances vs. multiple pools: The benefit of multiple pools is that cachelib can help you balance the memory between pools. If you use multiple instances, you will need to make sure you size each instances and adjust them should your workload changes.

[wenhaocs]

Make sense. Thanks @haowu14 @ss7pro