Coder Social home page Coder Social logo

seangarner / mongo-url-utils Goto Github PK

View Code? Open in Web Editor NEW
6.0 2.0 2.0 102 KB

utilities to parse url param objects into objects that can be passed to mongo functions

License: MIT License

JavaScript 95.28% Makefile 4.72%
mongo javascript parsing pegjs url-parsing nodejs

mongo-url-utils's Introduction

mongo-url-utils

Build Status

Utilities to parse url parameters into objects that can be passed to mongo functions.

compatibility

Currently depends on mongo 2.6 for the eq support.

Tested against node 4 LTS, 6 LTS and 8 LTS.

example

var mongoUrlUtils = require('mongo-url-utils');

var params = {
  sort: '-age,+firstName',
  fields: '-_id,+email,+firstName',
  limit: '10',
  skip: '0',
  query: 'or(gt(age,18),eq(married,false))'
};

var opts = mongoUrlUtils(params);

collection.find(opts.query, opts, function (err, docs) {
  console.dir(docs);
});

The above would yield this opts object:

{
  query: {
    $or: [
      { age: { '$gt': 18 } },
      { married: { '$eq': false } },
    ]
  },
  options: {
    sort: {
      age: -1.
      firstName: 1
    },
    fields: {
      _id: 0,
      email: 1,
      firstName: 1
    },
    limit: 10,
    skip: 0
  }
}

Or you can parse the query string directly:

mongoUrlUtils('query=gt(age,21)&sort=-age');

// {
//   query: { age: { '$gt': 21 } },
//   options: {
//     sort: { age: -1 }
//   }
// }

findIn

Sugar to parse a url or params and pass to mongo find method of a collection. Returns a cursor.

var people = db.collection('people');
mongoUrlUtils.findIn(people, 'query=eq(id,3)').toArray(function (err, docs) {
  // ...
});

findOneIn

Sugar to parse a url or params and pass to mongo findOne method of a collection.

var people = db.collection('people');
mongoUrlUtils.findOneIn(people, 'query=eq(id,3)', function (err, person) {
  // ...
});

find operators

A find string is made up of any of the query, sort, fields, limit and skip operators.

query

A query string which when parsed builds a query object for find. Coverage of the mongo query interface isn't 100% implemented yet. Here's what's available:

operator example
$eq eq(name,"West and Sons")
$gte gte(id,6)
$gt gt(id,6)
$lte lte(id,3)
$lt lt(id,3)
$ne ne(closed,true)
$size size(grades,4)
$in in(restaurant_id,["8165423","5827429"])
$nin nin(id,[1,2,3,4,5])
$all all(address.coord,["-47.9327","-82.6261"])
$and and(eq(grades.score,5),eq(borough,"Buckinghamshire"))
$or or(eq(id,1),eq(borough,"Buckinghamshire"))
$not not(size(grades,0))
$regex regex(address.street,".*Road.*")
$where where("parseInt(this.restaurant_id, 10) === 5827429")
$text text("y hijos", "es")
$mod mod(id,5,1)
$elemMatch elemMatch(grades,eq(score,2))
$exists exists(closed,false)
$type type(name,2) or type(name,String) (see Mongo Types)

Example; only return people who are widowed or age is greater than 50 and less than 70.

GET /people?query=or(eq(widowed,true),and(gt(age,50),lt(age,70)))

There are also extra operators that wrap $regex providing a more predictable query without the full power or danger associated with PCREs.

operator example
startsWith startsWith(name, "We")
endsWith endsWith(address.street, "Road")
contains contains(borough, "shire")

The extra operators also support $not. For example not(contains(borough, "shire")) would find the docs in which borough does not contain shire.

data type support

In addition to the string, number, boolean and null types, the query parser implements support for native Date parsing.

type format parsed as example usage
date Date Time String Date gt(grades.date,Date(2015-04-28T17:00Z))

Please submit pull requests for additional data types, particularly those that are found within the MongoDB Extended JSON specification.

case insensitive matching

Some operators support the i flag to denote that the operator should match the value case insensitively. This is useful if you want to enable case insensitive match without allowing full $regex powers (because $regex is the only way of achieving this in mongo).

  • eq(tags, 'NODE', i) matches Node, NODE, node, NoDe, etc

Also supported with ne, startsWith, endsWith and contains, but must be enabled using the disabledOperators query option as the default is to disable this feature.

var options = {
  query: {
    caseInsensitiveOperators: true
  }
};
mongoUrlUtils({query: 'regex(email,"[email protected]",i)'}, options);

regex safety checking

The regex query operator can be dangerous allowing attackers to create an expensive expression. Set safeRegex to true to pass all regexes through safe-regex and throw an error if an unsafe regex is sent.

var options = {
  query: {
    safeRegex: true
  }
};
mongoUrlUtils({query: 'regex(email,"(a+){10}y")'}, options);
// throws Error

mongo types

The type() query operator allows either integer identifiers as per the mongodb documentation. For convinience it also maps the following types to their ids: Double, String, Object, Array, Binary, Undefined, ObjectId, Boolean, Date, Null, RegExp, Javascript, Symbol, ScopedJavascript, Int32, Timestamp and Int64.

todo

  • full $not support
    • missing for regex, where, text, mod, elemMatch, exists, type
  • $nor
  • /regex/ (can't use $regex with $in/$nin)

sort

Comma separated field names prefixed with a + for an ascending sort or - for a descending sort. There is no default so either - or + must be provided.

Example; return people sorted by oldest age first.

GET /people?sort=-age

fields

A projection parameter which limits which fields are returned. Fields are comma separated.

You can either use an inclusive or exclusive projection. An inclusive (+) projection means give me back only these fields. An exclusive (-) projection means give me back all fields except these. Inclusive and exclusive cannot be mixed, with one exception for the _id field in an inclusive projection.

Example; only return the first name and country of people and exclude _id. This is the only time you can mix - and +.

GET /people?fields=-_id,+firstName,+address.country

The following projection operators are also supported:

operator example
elemMatch-projection elemMatch(students,eq(school,102))

Projection operators can be mixed with projection parameters, however it's difficult to prevent mixed exclusive and exclusive parameters with operators so it's recommended that operators come last in the fields string. e.g.

GET /courses?fields=-id,+title,elemMatch(students,eq(gender,"female"))

limit

Limit how many documents are returned.

Example; return at most the first 10 documents.

GET /people?limit=10

skip

How many documents to skip before returning the set. When combined with limit it can be used to page results.

Example; return all except the first 10 documents.

GET /people?skip=10

disabling query operators

Perhaps you don't want to allow all query operators for performance reasons. It's possible to disable operators at the parser level so the parser will throw an exception when a blacklisted operator is used.

Example; disable the regex operator.

var options = {
  query: {
    disabledOperators: ['regex', 'text']
  }
};
mongoUrlUtils({query: 'regex(email,".*\\\\.gmail\\\\.com")'}, options);

// Error: regex operator is disabled

a note on URL encoding

Browsers don't encode a literal + in the query string of a url but node will convert them into literal spaces when parsing the querystring. This is a little inconvenient for sort and fields which both prefix fields with +. Both parsers works around this by treating a literal space as it would a + at the beginning of the query value.

If this magic behavior concerns you it can be disabled by setting the {strictEncoding: true} option - but remember clients are now responsible for encoding + before making the request.

var options = {
  strictEncoding: true
};
mongoUrlUtils('fields=+id,-_id', options); // throws an Error
mongoUrlUtils('fields=%2Bid,-_id', options); // works as expected

mongo-url-utils's People

Contributors

davidgwking avatar greenkeeperio-bot avatar seangarner avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

mongo-url-utils's Issues

An in-range update of mongodb is breaking the build 🚨

Version 2.2.32 of mongodb was just published.

Branch Build failing 🚨
Dependency mongodb
Current Version 2.2.31
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

mongodb is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • ❌ continuous-integration/travis-ci/push The Travis CI build could not complete due to an error Details

Commits

The new version differs by 15 commits.

  • 116c603 2.2.32
  • 253a9a3 doc(changes): update changelog for v2.2.32
  • d2b0ff5 test(buffering): fix expectations of flakey tests
  • 776b418 chore(package): update mongodb-core dependency
  • 834a0e2 style(*): fix various minor typos throughout module
  • bf71053 doc(grid-fs): fixed spelling error
  • 75dd442 docs(server): improve docs re: autoReconnect
  • 2bb81ab style(collection): fix typo-ed internal fn name
  • d159eab fix(db): bubble up reconnectFailed event from Server topology
  • 2d1fcfe doc(bulk): add param to documentation for BulkWriteResult.getWriteError
  • 1f7954a chore(travis): remove 0.10/0.12 builds because of request semver
  • 7bd839b fix(collection): allow passing noCursorTimeout as an option to find()
  • 129d540 fix(cursor): hasNext should propagate errors when using callback
  • 7cb2b7d test(cursor): update cursor tests to check for errors
  • beccc83 fix(aggregation): ensure that the cursor key is always present

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of nodemon is breaking the build 🚨

Version 1.12.2 of nodemon was just published.

Branch Build failing 🚨
Dependency nodemon
Current Version 1.12.1
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

nodemon is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • ❌ continuous-integration/travis-ci/push The Travis CI build could not complete due to an error Details

Commits

The new version differs by 13 commits.

  • 24a4b84 feat: disable chokidar globbing on windows (#1140)
  • 6d57dac fix: bump deps and fix tests for local dev (#1139)
  • 5a89df6 fix: Support for .mjs es6 modules (#1138)
  • 9b6c786 fix: sync help.txt to available options
  • 0b1bf13 docs: update faq for -c option
  • f3e0c29 fix: read config file before defaulting script parameter (#1110)
  • d0c515a chore: Merge branch 'AverageMarcus-fix/810'
  • 5b990b2 chore: merge branch 'fix/810'
  • af54f64 docs: trying a new position
  • 29a9a44 fix: Add support for multi-level filenames
  • ff935ca test: Test support for multi-level filenames
  • 07e55cc docs: update FAQ with added platform documentation
  • 4391ede docs: Reposition Code Sponsor (#1090)

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

An in-range update of mocha is breaking the build 🚨

Version 3.4.1 of mocha just got published.

Branch Build failing 🚨
Dependency mocha
Current Version 3.4.0
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

As mocha is β€œonly” a devDependency of this project it might not break production or downstream projects, but β€œonly” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this πŸ’ͺ

Status Details
  • ❌ continuous-integration/travis-ci/push The Travis CI build could not complete due to an error Details

Release Notes Ohai CRLF...

Fixed a publishing mishap with git's autocrlf settings.

Commits

The new version differs by 3 commits0.

false

See the full diff

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.


Your Greenkeeper Bot 🌴

optional checking of regexes via substack/safe-regex

Consider if during parsing is a good time to check if regexes are potentially evil using substack/safe-regex.

After parsing is difficult because it requires the user walk the mongo query object looking for regexes to test, so parsing time does make sense from that POV.

An in-range update of nodemon is breaking the build 🚨

Version 1.11.1 of nodemon just got published.

Branch Build failing 🚨
Dependency nodemon
Current Version 1.11.0
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

As nodemon is β€œonly” a devDependency of this project it might not break production or downstream projects, but β€œonly” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this πŸ’ͺ

Status Details
  • ❌ continuous-integration/travis-ci/push The Travis CI build could not complete due to an error Details

Commits

The new version differs by 8 commits.

See the full diff

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.


Your Greenkeeper Bot 🌴

cleanup

  • rename q to query for consistency with mongo
  • prefix name of parser that threw an error as it throws

Limit must be an integer Code

Am trying to get my head around this code in the limit.js file

 if (n.toString() !== s){ 
    throw new Error('limit must be an integer');}

Why do you have it there?

An in-range update of nsp is breaking the build 🚨

Version 2.8.0 of nsp just got published.

Branch Build failing 🚨
Dependency nsp
Current Version 2.7.0
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

As nsp is β€œonly” a devDependency of this project it might not break production or downstream projects, but β€œonly” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this πŸ’ͺ

Status Details
  • ❌ continuous-integration/travis-ci/push The Travis CI build could not complete due to an error Details

Commits

The new version differs by 7 commits.

  • 93816b2 2.8.0
  • 6dfcca4 shrinkwrap shuffle apparently
  • 31029c2 Merge pull request #184 from nodesecurity/lock
  • fb48dd1 Merge branch 'master' of github.com:nodesecurity/nsp into lock
  • 82ee301 remove code don't comment it out
  • 1d0b7b5 adding package-lock.json support
  • 56f9848 Fix build status and add mention of nodesecurity.io

See the full diff

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.


Your Greenkeeper Bot 🌴

Element Match for simple array?

Is there a special syntax for running elemMatch that would work on a simple array (as opposed to an array of objects)?
The first example for $elemMatch on the mongo docs is:
db.scores.find( { results: { $elemMatch: { $gte: 80, $lt: 85 } } } )

which I imagine could be written as
elemMatch(results,gte(80),lt(85))

But doesn't work, I imagine, because the gte() and lt() operators are unary.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.