Comments (6)
Changes have been made in the @nestjs/throttler
package. If nestjs/throttler#1303 has been merged then I will adjust changes to this package and merge these changes to master and release a new version.
from nestjs-throttler-storage-redis.
Which version of nestjs-throttler-storage-redis are you currently using? This should not be a problem with the latest 0.2.2 version.
from nestjs-throttler-storage-redis.
the version is 0.2.2.
The issue is related to Redis storage. because without RedisStorage(in memory) the guard is working perfectly.
I think the logic written in addRecord and getRecord works very slowly, for it can't accumulate requests. @kkoomen
from nestjs-throttler-storage-redis.
This is the script which I use for testing.
`
const axios = require('axios');
for (let i = 0; i < 1000; i++) {
axios
.get('http://localhost:9000/v1/test')
.then(({ data }) => console.count())
.catch((err) => {
console.log(err.toString());
});
}
`
from nestjs-throttler-storage-redis.
Okay so, this is a tougher problem (to me) than it initially seemed. I found out that express-rate-limit
with rate-limit-redis
do not have this problem and I know why.
Have a look at the express-rate-limit
: https://github.com/express-rate-limit/express-rate-limit/blob/master/source/lib.ts#L230-L369
and also the rate-limit-redis
: https://github.com/wyattjoh/rate-limit-redis/blob/main/src/lib.ts
So, how the @nestjs/throttler
guard (see here) works is by first calling getRecord()
in order to get the current TTLs (partially based the req.ip
) and then if it did not exceed the limit, then it will call addRecord
.
The way express-rate-limit
works is by calling an increment
function (see here) - which is similar to the addRecord()
in the @nestjs/throttler
- where they increase the amount of requests being done (based on the req.ip
) and return the new amount along with the remaining time. So after running the above script to trigger 1000 queries, the value in redis will eventually be ±1000. Then if this is >= specified limit, then a 429 error will be thrown. The rate-limit-redis
does implement its own increment
by using EVALSHA with a custom lua script that will do this all in a single call. I believe this is the reason that they don't face this issue, because everything is being done in a single call.
To make it more clear: the express-rate-limit
gets triggered, does +1 in amount of requests being done in redis, but if there is another request then the other request will have the +1 from the previous request already included, because it is already in redis.
The problem with @nestjs/throttler
is that we first calls addRecord
, then the request after that will also call addRecord
almost at the same time, let's say that at this point we have 4 requests and there are 5 allowed, then only the first one should be allowed, but this isn't the case, because both requests will notice that there are 4 requests, and they will therefore both execute.
@jmcdo29 I need your input on this one.
@Zizitto @zhorakaroy if you guys have a possible solution that doesn't involve rewriting the @nestjs/throttler
logic and only requires changes in this package, please feel free to comment below.
EDIT: A solution has been implemented, see my next comment.
from nestjs-throttler-storage-redis.
This has been released in version v0.3.0 on NPM. Special thanks to @zhorakaroy for discovering this nasty bug. This was a good improvement for the nestjs throttler core package as well as for this package.
from nestjs-throttler-storage-redis.
Related Issues (20)
- That is a bad idea to use Redis SCAN command, it works not like you expects... HOT 6
- unable to implement custom guard HOT 1
- WRONGTYPE Operation against a key holding the wrong kind of value HOT 8
- Redis storage does is not working - no errors HOT 3
- Wrong `timeToExpire` unit HOT 2
- Dynamically update config values of limiter HOT 1
- Redis connection kept open after application shutdown
- @nestjs/throttler v5 ttl unit changed from second to milisecond HOT 4
- connecting with forRootAsync doesn't work HOT 11
- Not working for limit greater than 10_000 HOT 5
- nest 8 compatibility HOT 1
- Problems with Redis storage HOT 1
- Support redis cluster HOT 14
- Issue Fixed in 322 Still Exists HOT 2
- `Maximum call stack size exceeded` error while e2e test my NestJS app HOT 3
- update ioredis to v5
- suggestion: uninstall @types/ioredis
- Update peer dependencies on @nestjs/common and @nestjs/throttler to latest version compatible versions HOT 4
- connect aws ElastiCache with seperate read and write HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nestjs-throttler-storage-redis.