Coder Social home page Coder Social logo

joshbeckman / mongo-throttle Goto Github PK

View Code? Open in Web Editor NEW
8.0 5.0 6.0 44 KB

Basic IP rate-limiting middleware for Express stored in mongoDB. Use to throttle or limit incoming request rate.

Home Page: https://www.npmjs.com/package/mongo-throttle

License: MIT License

JavaScript 100.00%

mongo-throttle's Introduction

Mongo Throttle

Basic IP rate-limiting middleware for Express stored in MongoDB. Use to throttle or limit incoming request rate.

Let MongoDB do the heavy lifting and synchronization of throttling/rate-limiting requestors. This Express middleware registers and maintains a rate limit for each requesting IP address. It uses MongoDB's built-in expiration index to automatically sweep out expired Throttle records.

This rate-limiting implementation will scale across multiple servers with the same implementation.

Usage

var throttler = require('mongo-throttle');

// In Express-style app:
app.use(throttle());

// Optionally only rate-limit requests that begin with /api/
app.use('/api/', throttle());

// Optionally configure limits:
app.use(throttle({ rateLimit: { ttl: 600, max: 5 } }));

// Optionally configure a custom limit handler:
app.use(throttle(function(req, res, hits, remaining) {
    var until = new Date((new Date()).getTime() + remaining);
    res.statusCode = 420;
    res.send('You shall not pass ' + hits + ' until ' + until + '!');
}));

API

Headers

# Header fields set:
X-Rate-Limit-Limit      # Maximum requests within TTL
X-Rate-Limit-Remaining  # Requests remaining for IP
X-Rate-Limit-Reset      # msec until limit reset for IP

Configuration

// Configuration options/defaults:
{
    "rateLimit": {
        "ttl": 600      // TTL window for each IP limit
        "max": 600      // Max hits for IP within window
    },
    "response": {
        "code":    429, // Response code when limit is reached
        "message": "Rate Limit reached. Please wait and try again."
    },
    "mongoose":{
        "uri": false    // Optional Mongo URI connection string
    },
    "useCustomHeader": "x-custom-header" // Count hits by the distinct values found in this header, instead of by IP
}

// When using a custom limit handler:
/**
 * Custom Limit-reached handler
 *
 * @param {object} req          Express request
 * @param {object} res          Express response
 * @param {number} hits         Total hits for this IP within TTL
 * @param {number} remaining    msec reminaing before this IP resets TTL window
 */
function custom_limit(req, res, hits, remaining) {
    // handler response here
}

Models

// Upon registration, this middleware will create
// a Mongoose model collection with schema:
Throttle = {
    createdAt:   Date,
    ip:          String,
    hits:        Number
}

Install

npm install mongo-throttle [--save]

Acknowledgments/Notes

Mongo has a useful feature called a TTL index.

TTL collections make it possible to store data in MongoDB and have the mongod automatically remove data after a specified number of seconds or at a specific clock time.

You can tell Mongo to remove data for you! We will use this to remove expired request counts from our rate-limiting check. There are a couple important things to note about this feature:

  • As an index, it is set upon collection creation. If you want to change it, you'll have to do so manually.
  • The index-specific field, expireAfterSeconds, is in seconds. Unlike most other timestamps in your JavaScript code, don't divide this by 1000.

See Also

License

MIT

mongo-throttle's People

Contributors

avnersorek avatar greenpioneer avatar joshbeckman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

mongo-throttle's Issues

Fix readyState

I am using this in MeanStackJS and it just constantly warns me about how its skipping which tells me the ready State isn't flipping to 1. which begs the questions is it reliable or is there another way we need to be using it.

Note in the mean stack this is order of the startup below. Could it be that i have to re order to use ready state ? but then whats the point of that parameter if is not dynamic enough to updated after you connect

  1. Init Mongoose
  2. Configure Throttle() as middleware
  3. Configure all other model
  4. Connect to the db

Expected Behavior

To not skip over the throttle or log after the mean stack sets up.

Current Behavior

But even after the ready state should have turned to 1 it does not even though i can see the model. At this point the rest of the system works fine except here.

Possible Solution

None yet, need to investigate the reliability of this ready state function

Ip Info

Save the IP info for admins to review at a later date to potentially use. i think this is just a nice to have maybe not useful.

NPM - node-ipinfo

{
  "ip": "102.180.165.25",
  "hostname": "102.180.165.25.lightspeed.rcsntx.sbcglobal.net",
  "city": "Fort Worth",
  "region": "Texas",
  "country": "US",
  "loc": "32.2418,-97.3126",
  "org": "AT&T Services, Inc.",
  "postal": "76222"
}

Mongoose Uri & inMemory Check

Lets add the mongoose connection for those who need to pass it in because of delay in the mongoose ready state. Let make sure we add back in the readyState to alert usering of potentially needing to add in the uri. When we use the URI we should use the createConnection call .

Look into the inMemory check so that if they are over the limit we store them so that we dont have to go back to the DB

Ability to turn off logs

Expected Behavior

The ability to silence/turn off the log

Current Behavior

MongoDB connection not set. Skipping Mongo-Throttle

Possible Solution

Send in options that allows you to select if you want to log at all

IP mongoose Validation

I believe this to be a enterprise issue but still has merit to handle

Validation Error: IP (12.243.243.23).

That isnt the exact ip string because im not allowed to share that but you get the idea.

Basically my point is lets add some error handling for validation on our Model so that it does not break any one system

Configuration error

Expected Behavior

throttle({
    rateLimit: {
      ttl: 600,
      max: 10000
    }
})

to make sure that when you set configs they rep what is seen in backend

Current Behavior

I was implementing into my Mean Stack JS -#124 and it kept breaking down

Possible Solution

Use the configs instead of the default which revert back to init settings of 600

Steps to Reproduce (for bugs)

1.Send in configuration different to the init ones
2.Look at the response headers and you will see the default

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.