RateLimiterRedis: Block set with secDuration=0 is reset on a consume method call about node-rate-limiter-flexible

[vworld]

@animir Sounds good - will check both of these. Thanks again for all the help and time.

[animir]

@vworld Yes, until you change number of points for the blocked key, it will be blocked.

[vworld]

Sounds awesome, thank you for the support.

[animir]

@vworld Hi, thank you!

Unfortunately, there is no way to achieve that with a single consume call — it overwrites the duration for the key, so it will expire based on default limiter option.
You can call get(key) to check if key is blocked forever and if so, don't call consume.

[vworld]

@animir Thank you for the quick response.

I was thinking about using blacklist but the docs say Blacklisted keys are blocked on code level not in store/memory. So I assume this will not work in a distributed system, where if a block is set in one process/node/machine it is honored by all processes/node/machine. Is that accurate?

And if we use RedisRateLimiter to block a key, the same will be propagated to redis and the block will be available across all machines that use the same key and same redis server. Is that correct?

[animir]

@vworld Yes, array of blocked keys stored in blackList is accessible for the current single process only. You could store blocked keys in any store and check it with isBlackListed function a way suitable for you. It is more work to store list of blocked keys somewhere else in database, but you could easily use Redis for that.

As of RateLimiterRedis, yes. Blocked keys are blocked on the Redis server level making it blocked for any process in distributed environment.

[animir]

@vworld Just another idea in addition to the answer for the first question. To store list of blocked keys you could easily store it with another instance of RateLimiterFlexible with options {points: 1, duration: 0}. If key should be blocked you can use set method, then check it with get method inside isBlackListed function of RLWrapperBlackAndWhite. But it is still one more request to the store, which shouldn't any problem though.

[vworld]

Just a heads up - Just like you suggested, I ended up using get to check if the key is blocked before using consume.
The assumption here is that once a key is blocked with secDuration=0 the key is blocked forever with msBeforeNext = -1 until we try to consume the key again.

So something like this works for me while maintaining availability of the block thorough out the distributed setup. This basically goes with the paradigm of my codes that expect an error whenever a key is blocked.

    /**
     * Consumes the specified amount of points for a key
     * @throws {RateLimiterRes} - msBeforeNext > -1 if not blocked and msBeforeNext = -1 if key is blocked
     * @param {string} ip
     * @param {number} pointsToConsume
     * @returns {Promise<void>}
     * @private
     */
    private async _attemptToConsumePoints(ip: string, pointsToConsume: number = 1){
        const rLKey = this._getLimiterKey(ip);

        // check if key is blocked
        const res = await this.rateLimiter.get(rLKey);
        if (res && res.msBeforeNext === -1) {
            throw res;
        }
        await this.rateLimiter.consume(rLKey, pointsToConsume);
    }

While I have not tested the assumptions fully but initial tests look promising.

@animir is it safe to assume a blocked key will always return msBeforeNext = -1 until we consume