Valkey support for memory locking (?) about valkey

[hpatro]

@ranshid One of the anecdote I would like to share here is some user(s) want predictable behavior throughout the lifetime of the process and when the engine enters swapping, that doesn't stay true. As everything slows down right from command processing to maintenance operations (defrag, lazy evictions, client evictions etc...), the engine is pretty much unresponsive and can lead to cascading effects like all clients getting disconnected and eventual connection storm. I think this would be a good to have configuration for such user(s).

[JimB123]

I think this idea may be misguided. Swap is never the problem. Swap is a SOLUTION to the problem of insufficient memory.

Swap provides a safety net for transient OOM situations. Without this safety net, the OS would kill the process. Eliminating swap, or crippling it by exempting the largest single process only affects the safety-net, without addressing the cause of the low-memory condition.

Please note - I'm well aware that active swapping kills performance. Nobody wants to be running on a system that's heavily in swap. I view swap ONLY as a transient safety net and I would never advocate for planned usage of swap as part of normal processing.

Attempting to lock Valkey into memory (making it exempt from swapping) is likely to be counter-productive. Valkey is likely to be the single largest user of memory on the system. What this would really be saying is: "I want the safety net of swapping, but I want to exempt the single largest user of memory from that function". This essentially REQUIRES the OS to swap out pages which are more necessary than than pages in Valkey. That can't be good.

If you don't want swap, that's pretty simple - just configure the host system without swap. When then system has low memory, Valkey (the largest process) will be killed. Another option may be to configure accounting limits on the Valkey process - and when Valkey exceeds those limits, it will be killed.

most linux distros come with swappiness level set to 60 which means the system will be aggressive swapping pages of inactive processes

This is not a completely true statement. There is a lack of good information regarding swappiness, and a lot of mis-information. Start up a linux system. Leave it idle. Check the swap - it will be zero. Clearly the system is not "aggressively swapping pages of inactive processes".

Swapping only occurs as the system is close to memory exhaustion, regardless of the swappiness setting. I don't claim to be an expert on this setting, but as best I can tell from the differing information I've found, this setting is primarily a preference to page out data vs paging out (discarding) file-backed memory (like code). A lower swappiness value (range 0-200) is more likely to cause code to be discarded than data to be written to disk. (At least to the best of my understanding.) I think this reference seems a little better than most: https://www.howtogeek.com/449691/what-is-swapiness-on-linux-and-how-to-change-it/ Also, that's mostly collaborated by this reference: https://www.baeldung.com/linux/swap-space-settings - but you have to read both references carefully. In some cases, the word "swapping" is used to only refer to paging out (writing to disk) data pages. The words "less swapping" sound good, however - the unstated alternative - discarding file-backed pages (like code) - is very similar in impact. In both cases, when an address fault occurs, the page (be it data or code) needs to be read back from disk.

[madolson]

Broadly I'm in agreement with @JimB123. I don't think we should be locking memory for performance reasons. The inverse is crashing, which I would argue is almost always worse than swapping. I think there might be some arguments about selectively locking pages, or collating data from the main dictionary into pages that are locked, to try to improve the performance of the engine while swapping, but I don't think that is a high priority.

With that said, I do think we should consider locking memory for security purposes. I would really like to lock any security related options (like masterauth) so that it doesn't ever get spilled to disk.

[zvi-code]

I think we could do something better here. If we separate the memory by usage we can have different lock policy for different memory usages. We than can, for example: lock all memory not in user dbs OR not user values memory. This will allow many non data intensive commands to execute, while leaving most of the memory for swap.

The above goal can be achieved by using separate memory regions for these allocations. Depends on what we want to lock, we can achieve this with any allocator, I believe. With jemalloc specifically the natural way would be to use user
-defined arena. There are many other benefits to this approach IMO (for example my comment here).