Coder Social home page Coder Social logo

ysocorp / koa2-ratelimit Goto Github PK

View Code? Open in Web Editor NEW
118.0 7.0 37.0 673 KB

Rate-limiting middleware for Koa2 ES6. Use to limit repeated requests to APIs and/or endpoints such as password reset.

License: MIT License

JavaScript 100.00%
koa2-ratelimit ratelimit-sequelize ratelimit koa2-rate-limit rate-limit brute bruteforce brute-force middleware koa2-bruteforce

koa2-ratelimit's Introduction

Koajs 2 Rate Limit (Bruteforce)

Build Status NPM version

Rate-limiting middleware for Koa2 with async await. Use to limit repeated requests to APIs and/or endpoints such as password reset.

Note: This module is based on express-rate-limit and adapted to koa2 ES6 with the async await capabilities.

Summary

Install

$ npm install --save koa2-ratelimit

Usage

For an API-only server where the rate-limiter should be applied to all requests:

const RateLimit = require('koa2-ratelimit').RateLimit;

const limiter = RateLimit.middleware({
  interval: { min: 15 }, // 15 minutes = 15*60*1000
  max: 100, // limit each IP to 100 requests per interval
});

//  apply to all requests
app.use(limiter);

Create multiple instances to apply different rules to different routes:

const RateLimit = require('koa2-ratelimit').RateLimit;
const KoaRouter = require('koa-router');
const router = new KoaRouter();

const getUserLimiter = RateLimit.middleware({
  interval: 15*60*1000, // 15 minutes
  max: 100,
  prefixKey: 'get/user/:id' // to allow the bdd to Differentiate the endpoint 
});
// add route with getUserLimiter middleware
router.get('/user/:id', getUserLimiter, (ctx) => {
  // Do your job
});

const createAccountLimiter = RateLimit.middleware({
  interval: { hour: 1, min: 30 }, // 1h30 window
  delayAfter: 1, // begin slowing down responses after the first request
  timeWait: 3*1000, // slow down subsequent responses by 3 seconds per request
  max: 5, // start blocking after 5 requests
  prefixKey: 'post/user', // to allow the bdd to Differentiate the endpoint 
  message: "Too many accounts created from this IP, please try again after an hour",
  messageKey: "message"
});
// add route  with createAccountLimiter middleware
router.post('/user', createAccountLimiter, (ctx) => {
  // Do your job
});

// mount routes
app.use(router.middleware())

Set default options to all your middleware:

const RateLimit = require('koa2-ratelimit').RateLimit;

RateLimit.defaultOptions({
    message: 'Get out.',
    // ...
});

const getUserLimiter = RateLimit.middleware({
  max: 100,
  // message: 'Get out.', will be added
});

const createAccountLimiter = RateLimit.middleware({
  max: 5, // start blocking after 5 requests
  // message: 'Get out.', will be added
});

Use with RedisStore

npm install redis@4
const RateLimit = require('koa2-ratelimit').RateLimit;
const Stores = require('koa2-ratelimit').Stores;
//Detailed Redis Configuration Reference: https://github.com/redis/node-redis/blob/master/docs/client-configuration.md
RateLimit.defaultOptions({
    message: 'Get out.',
    store: new Stores.Redis({
        socket: {
            host: 'redis_host',
            port: 'redis_port',
        },
        password: 'redis_password',
        database: 1
    })
});

const getUserLimiter = RateLimit.middleware({
    prefixKey: 'get/user/:id',
});
router.get('/user/:id', getUserLimiter, (ctx) => {});

const createAccountLimiter = RateLimit.middleware.middleware({
    prefixKey: 'post/user',
});
router.post('/user', createAccountLimiter, (ctx) => {});

// mount routes
app.use(router.middleware())

Use with SequelizeStore

npm install sequelize@5
const Sequelize = require('sequelize');
const RateLimit = require('koa2-ratelimit').RateLimit;
const Stores = require('koa2-ratelimit').Stores;

const sequelize = new Sequelize(/*your config to connected to bdd*/);

RateLimit.defaultOptions({
    message: 'Get out.',
    store: new Stores.Sequelize(sequelize, {
        tableName: 'ratelimits', // table to manage the middleware
        tableAbuseName: 'ratelimitsabuses', // table to store the history of abuses in.
    })
});

const getUserLimiter = RateLimit.middleware({
    prefixKey: 'get/user/:id',
});
router.get('/user/:id', getUserLimiter, (ctx) => {});

const createAccountLimiter = RateLimit.middleware.middleware({
    prefixKey: 'post/user',
});
router.post('/user', createAccountLimiter, (ctx) => {});

// mount routes
app.use(router.middleware())

Use with MongooseStore (Mongodb)

npm install mongoose@5
const mongoose = require('mongoose');
const RateLimit = require('koa2-ratelimit').RateLimit;
const Stores = require('koa2-ratelimit').Stores;

await mongoose.connect(/*your config to connected to bdd*/);

RateLimit.defaultOptions({
    message: 'Get out.',
    store: new Stores.Mongodb(mongoose.connection, {
        collectionName: 'ratelimits', // table to manage the middleware
        collectionAbuseName: 'ratelimitsabuses', // table to store the history of abuses in.
    }),
});

A ctx.state.rateLimit property is added to all requests with the limit, current, and remaining number of requests for usage in your application code.

Configuration

  • interval: Time Type - how long should records of requests be kept in memory. Defaults to 60000 (1 minute).

  • delayAfter: max number of connections during interval before starting to delay responses. Defaults to 1. Set to 0 to disable delaying.

  • timeWait: Time Type - how long to delay the response, multiplied by (number of recent hits - delayAfter). Defaults to 1000 (1 second). Set to 0 to disable delaying.

  • max: max number of connections during interval milliseconds before sending a 429 response code. Defaults to 5. Set to 0 to disable.

  • message: Error message returned when max is exceeded. Defaults to 'Too many requests, please try again later.'

  • statusCode: HTTP status code returned when max is exceeded. Defaults to 429.

  • headers: Enable headers for request limit (X-RateLimit-Limit) and current usage (X-RateLimit-Remaining) on all responses and time to wait before retrying (Retry-After) when max is exceeded.

  • skipFailedRequests: when true, failed requests (response status >= 400) won't be counted. Defaults to false.

  • whitelist: Array of whitelisted IPs/UserIds to not be rate limited.

  • getUserIdFromKey: Function that extracts from given key the userId. Defaults to (key) => key.split(options.prefixKeySeparator).

  • prefixKeySeparator: Separator string between the prefixKey and the userId. Defaults to ::. (Set it to | if you want whitelist userIds)

  • getUserId: Function used to get userId (if connected) to be added as key and saved in bdd, should an abuse case surface. Defaults:

    async function (ctx) {
        const whereFinds = [ctx.state.user, ctx.user, ctx.state.User, 
          ctx.User, ctx.state, ctx];
        const toFinds = ['id', 'userId', 'user_id', 'idUser', 'id_user'];
        for (const whereFind of whereFinds) {
          if (whereFind) {
            for (const toFind of toFinds) {
              if (whereFind[toFind]) {
                  return whereFind[toFind];
              }
            }
          }
        }
        return null;
    },
  • keyGenerator: Function used to generate keys. By default userID (if connected) or the user's IP address. Defaults:

    async function (ctx) {
        const userId = await this.options.getUserId(ctx);
        if (userId) {
            return `${this.options.prefixKey}|${userId}`;
        }
        return `${this.options.prefixKey}|${ctx.request.ip}`;
    }
  • skip: Function used to skip requests. Returning true from the function will skip limiting for that request. Defaults:

    async function (/*ctx*/) {
        return false;
    }
  • handler: The function to execute once the max limit has been exceeded. It receives the request and the response objects. The "next" param is available if you need to pass to the next middleware. Defaults:

    async function (ctx/*, next*/) {
        ctx.status = this.options.statusCode;
        ctx.body = { message: this.options.message };
        if (this.options.headers) {
            ctx.set('Retry-After', Math.ceil(this.options.interval / 1000));
        }
    }
  • onLimitReached: Function to listen each time the limit is reached. It call the store to save abuse, You can use it to debug/log. Defaults:

    async function (ctx) {
        this.store.saveAbuse({
            key: await this.options.keyGenerator(ctx),
            ip: ctx.request.ip,
            user_id: await this.options.getUserId(ctx),
        });
    }
  • weight: Function to set the incrementation of the counter depending on the request. Defaults:

    async function (/*ctx*/) {
        return 1;
    }
  • store: The storage to use when persisting rate limit attempts. By default, the MemoryStore is used.

    Avaliable data stores are:

    • MemoryStore: (default)Simple in-memory option. Does not share state when app has multiple processes or servers.
    • SequelizeStore: more suitable for large or demanding deployments.

The delayAfter and timeWait options were written for human-facing pages such as login and password reset forms. For public APIs, setting these to 0 (disabled) and relying on only interval and max for rate-limiting usually makes the most sense.

Time Type

Time type can be milliseconds or an object

    Times = {
        ms ?: number,
        sec ?: number,
        min ?: number,
        hour ?: number,
        day ?: number,
        week ?: number,
        month ?: number,
        year ?: number,
    };

Examples

    RateLimit.middleware({
        interval: { hour: 1, min: 30 }, // 1h30 window
        timeWait: { week: 2 }, // 2 weeks window
    });
    RateLimit.middleware({
        interval: { ms: 2000 }, // 2000 ms = 2 sec
        timeWait: 2000, // 2000 ms = 2 sec
    });

Upgrade

0.9.1 to 1.0.0

1.0.0 moves sequelize, mongoose and redis from dependencies to peerDependencies.

Install the one you use (see Use with RedisStore, Use with SequelizeStore or Use with MongooseStore (Mongodb)).

The rest did not change.

License

MIT ยฉ YSO Corp

koa2-ratelimit's People

Contributors

alexandrebodin avatar cckelly avatar drfaraday avatar jeanclaudeyalap avatar jmacpherson avatar jrexhmati avatar julienwilmet avatar mahamada-gy avatar nmeylan avatar viossat avatar yveskaufmann avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

koa2-ratelimit's Issues

Redis package

promise-redis breaks if redis@next installed along. promise-redis is not needed as node-redis v4 support natively.

Library is breaking tests: can't find module 'uuid/v1' in sequelize utilities

I'd love to use this library but it is breaking my tests with the following error:

Cannot find module 'uuid/v1' from 'node_modules/sequelize/lib/utils.js'

Require stack:
      node_modules/sequelize/lib/utils.js
      node_modules/sequelize/lib/sequelize.js
      node_modules/sequelize/index.js
      node_modules/koa2-ratelimit/src/SequelizeStore.js
      node_modules/koa2-ratelimit/src/index.js
      server/server.js
      server/heartbeat/__tests__/heartbeat-requests.test.js

      at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:306:11)
      at Object.<anonymous> (node_modules/sequelize/lib/utils.js:6:16)

This happens as soon as I require the module in my app:

const rateLimit = require('koa2-ratelimit').RateLimit

This is how I'm using it in my server.js file:

app.use(
  rateLimit.middleware({
    interval: { min: 15 }, // 15 minutes = 15*60*1000
    max: 100, // limit each IP to 100 requests per interval
  })
)

I am not using a store. For now I am conditionally adding the block for production but I'm wondering why sequelize is being added at all if I'm not using it.

Thanks for your time!

Update mongoose version

Unless the mongoose package dependency is removed, it should be updated to a newer version, since v.5.13.8 uses [email protected] which is marked as a critical vulnerability by npm.

Sequelize store broken with sequelize 5

Use of global.sequelize and of $lte operator.

I use this patch (with patch-package)

diff --git a/node_modules/koa2-ratelimit/src/SequelizeStore.js b/node_modules/koa2-ratelimit/src/SequelizeStore.js
index f5bceb4..ac3b23b 100644
--- a/node_modules/koa2-ratelimit/src/SequelizeStore.js
+++ b/node_modules/koa2-ratelimit/src/SequelizeStore.js
@@ -1,4 +1,5 @@
 const Sequelize = require('sequelize');
+const Op = Sequelize.Op;
 
 const Store = require('./Store.js');
 
@@ -116,7 +117,7 @@ class SequelizeStore extends Store {
     }
 
     async _increment(table, where, nb = 1, field) {
-        return table.update({ [field]: global.sequelize.literal(`${field} + ${nb}`) }, { where });
+        return table.update({ [field]: this.sequelize.literal(`${field} + ${nb}`) }, { where });
     }
 
     // remove all if time is passed
@@ -124,7 +125,7 @@ class SequelizeStore extends Store {
         const now = new Date();
         await table.destroy({
             where: {
-                date_end: { $lte: now.getTime() },
+                date_end: { [Op.lte]: now.getTime() },
             },
         });
     }

MongoStore deprecated methods

While running with the latest version of Mongoose, I get the following warnings:

(node:31651) DeprecationWarning: collection.remove is deprecated. Use deleteOne, deleteMany, or bulkWrite instead.
(node:31651) DeprecationWarning: collection.findAndModify is deprecated. Use findOneAndUpdate, findOneAndReplace or findOneAndDelete instead.
(node:31651) DeprecationWarning: Mongoose: `findOneAndUpdate()` and `findOneAndDelete()` without the `useFindAndModify` option set to false are deprecated. 

Moreover, is good to put into the examples the following options for the mongoose connection:

useCreateIndex: true

in order to avoid the deprecation warning:

DeprecationWarning: collection.ensureIndex is deprecated. Use createIndexes instead.

Is there any intent to fix these warnings?

MongoError: E11000 duplicate key error index

Hi

First of all, thank you for this great lib.

I am getting lots of errors for duplicate key error index when using Stores.Mongodb

db.ratelimits.$key_1 dup key: { : "global|{uid}" }

Here is how I have configured it:

try {
  RateLimit.defaultOptions({
    message: 'Get out.',
    store: new Stores.Mongodb(mongoose.connection, {
      collectionName: 'ratelimits', // table to manage the middleware
      collectionAbuseName: 'ratelimitsabuses' // table to store the history of abuses in.
    })
  });
} catch (error) {
  throw new Error(error);
}
limiter: RateLimit.middleware({
    skipFailedRequests: true,
    interval: { min: 1 }, // 1 second interval
    max: 300, // limit each IP to 300 requests per interval
    getUserId: async context => {
      try {
        let user = null;
        if (context.header['x-token'] && context.header['x-refresh-token']) {
          user = await validateToken(context.header['x-token'], context.header['x-refresh-token']) || null;
        }
        if (user) return user.uid;

        return null;
      } catch (error) {
        return null;
      }
    }
  })

Any ideas?

Why not stop requests?

The data is stored in the database, but the requests continue to go, not blocked
why?
code

setting ratelimit

import mongoose from '../connectors/mongo/ips'
import { RateLimit, Stores } from 'koa2-ratelimit'
 
RateLimit.defaultOptions({
    message: 'Get out.',
    store: new Stores.Mongodb(mongoose.connection, {
        collectionName: 'ratelimits',
        collectionAbuseName: 'ratelimitsabuses'
    })
})

const middlewareLimiter = RateLimit.middleware({
  	interval: { ms: 10000 },
  	timeWait: 3*1000,
  	message:'Error',
  	max: 10
})

export {
	RateLimit,
	middlewareLimiter
}

ROUTER

router.use(middlewareLimiter)
router.use(routesApiV1)

Mongodb Store does not clear expired docs in the ratelimits collection.

The following function is supposed to clean all the existing documents in the ratelimits collection where dateEnd <= now, but it does not. Therefore once a limit is reached, it stays there forever.

async _removeAll() { await this.Ratelimits.remove({ dateEnd: { $lte: Date.now() } }); }

Missing LICENSE file

Repo seems to be MIT-licensed (as referenced in the README), but lacks a standard LICENSE file

Store.Mongodb data error

If you use the Mongodb store to persist the rate limits, X-RateLimit-Remaining and X-RateLimit-Reset headers are set as NaN because of the bug with the function below.

async incr(key, options, weight) { await this._removeAll(); const data = await this.Ratelimits.findOrCreate({ where: { key }, defaults: { key, dateEnd: Date.now() + options.interval, }, }); await this._increment(this.Ratelimits, { key }, weight, 'counter'); return { counter: data.counter + weight, dateEnd: data.dateEnd, }; }

The data object has a key value where the counter and dateEnd are listed. The return object should return
return { counter: data.value.counter + weight, dateEnd: data.value.dateEnd, }
instead of
return { counter: data.counter + weight, dateEnd: data.dateEnd, }

Redis Store Feature request

Express version supports Redis Store.
I would highly appreciate if you can make this support Redis store as well.
Thanks!

Mongoose Store not working

      store: new Stores.MongodbStore(mongoose.Connection, {
        collectionName: "ratelimits",
        collectionAbuseName: "ratelimitabuses"
      })

throws an error:
UnhandledPromiseRejectionWarning: TypeError: Stores.MongodbStore is not a constructor

Marking peer dependencies as optional

๐Ÿš€ Feature request

When using this package with one of the providers you will always get peerDependencies warnings because you won't be using all possible providers at once.

With npm v7+ peerDependenciesMeta were introduced to mark peerDependencies as optional. In this package I think it would be a quick win to avoid the npm warnings and make everyone happy :)

Happy to make a PR if someone is ready to merge it and release :)

Parameterized Route not working

const getUserLimiter = RateLimit.middleware({
  interval: 15*60*1000, // 15 minutes
  max: 100,
  prefixKey: 'get/user/:id' // to allow the bdd to Differentiate the endpoint 
});
// add route with getUserLimiter middleware
router.get('/user/:id', getUserLimiter, (ctx) => {
  // Do your job
});

It is not differentiating according to actual value of id. /user/1 and /user/2 should be treated as 2 different prefixKey.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.