lloyd / jsonselect

CSS-like selectors for JSON

JSONSelect is EXPERIMENTAL, ALPHA, etc.

JSONSelect defines a selector language similar to CSS intended for JSON documents. For an introduction to the project see jsonselect.org or the documentation.

Project Overview

JSONSelect is an attempt to create a selector language similar to CSS for JSON objects. A couple key goals of the project's include:

• intuitive - JSONSelect is meant to feel like CSS, meaning a developers with an understanding of CSS can probably guess most of the syntax.
• expressive - As JSONSelect evolves, it will include more of the most popular constructs from the CSS spec and popular implementations (like sizzle). A successful result will be a good balance of simplicity and power.
• language independence - The project will avoid features which are unnecessarily tied to a particular implementation language.
• incremental adoption - JSONSelect features are broken in to conformance levels, to make it easier to build basic support and to allow incremental stabilization of the language.
• efficient - As many constructs of the language as possible will be able to be evaluated in a single document traversal. This allows for efficient stream filtering.

JSONSelect should make common operations easy, complex operations possible, but haughtily ignore weird shit.

What's Here

This repository is the home to many things related to JSONSelect:

• Documentation which describes the language
• The jsonselect.org site source
• A reference implementation in JavaScript

Related projects

Conformance tests are broken out into a separate repository and may be used by other implementations.

jsonselect's Issues

Not like operator

Is there already a way of doing a not like expression?

I can see it's really easy to add another binary operator for it, but it doesn't seem correct as if this was done in CSS it would use :not()

minified link is broken

http://jsonselect.org/js/jsonselect.min.js

Selector support for prefix ?

Just wondering do we have a support of prefix selector, any planning
e.g. in css/jq *[class^='fade'] can match to .fade1, .fade2

tags

Would it be possible to create tags for release > 0.2.1?

The reason being that we use bower to manage our front-end dependencies and bower in turn uses git tags to enumerate available versions.

PS. Adding a bower.json file might not be a bad idea either...

Update HTML based test runners

Basically, make sure the HTML based tests are still working and runnable

Enhancement: Excluding properties from traversal

The selected object tree needs to be acyclic, which is quite inconvenient if you are cross referencing branches. An indirection via an external map helps, but is not really elegant, IMO. I'd rather suggest a "special" key being excluded on traversal, e.g. "__meta" (should be customizable) which could serve as such.

In conjuction with issue #58 (parent access in forEach), this offers a lot of flexibility traversing an object tree.

What do you think?

Returning the matched paths instead of the values

I modified jsonselect to have the ability to return matched paths instead of just the values as part of a project I'm working on. Is this something that could be integrated into the main project?

Expression to negate has()?

Hey, I'm trying to filter objects that have one property, but don't have another. Is there a way to do that? Given this structure:

{
    people: [
        {
            name: "Betty",
            friend: { name: "Robert" }
        },
        {
            name: "Victoria",
            friend: { 
                name: "Chris", id: 1,
                people: [
                    {
                        friend: {
                            name: "Jon"     
                        }
                    }
                ]
            }
        },
    ]
}

I use this selector: object:has(.friend > .name) but I also want to include a condition - to get those friends that don't have an id, something like: object:has(.friend > .name):hasnot(.friend > .id) (to get objects that contain Robert and Jon, but not Chris)

Is this possible?

Selected values manipulator in forEach || JSONSelect.map() required.

I'm trying to validate nodejs app POST input with JSONSelect. It would be nice if I could also manipulate the selected data. Like in the following code:

JSONSelect.forEach(
    ".points_to_delete .id," +
    ".points_to_update .status," +
    ".points_to_update .task_id," +
    ".points_to_update .warning",
    args,
function(item,itemManipulator)
{
    if(require('validator').isNumeric(item))
    {
        itemManipulator.setItemValue(parseInt(item))
    }
    else
    {
        throw Error("Item is not numeric!")
    }
})

Probably, it would be better to create JSONSelect.map() for this purposes. Thank you.

Select single value, not an array

I've been trying to get this to work for a bit now and can't seem to figure it out. What I'm trying to do is simply return a value instead of an array that contains a single value. For example, given this json:

{this: "is", some: {test: "json"}}

and this JSONSelect query:

.some > .test

is there a way to get back simply "json", instead of ["json"]? I know I can index the array when the result comes out, but I'd really prefer to be able to state in my JSONSelect query that I just want a value and not an array.

Is this project still active?

Looks like an amazing piece of work. Has it been forked or continued elsewhere?

Protect against circular objects

Currently, feeding a circular object to JSONSelect means it'll recurse until the call stack is exceeded. Keeping track on which objects has been traversed and protecting against circular references (i.e. not traversing objects that have already been traversed) would prevent this from happening.

For an example of how it can be done, you can look at node's util.inspect (though you probably already know about it, I bet ;) ).

node.js module

As far as I can tell, this could easily be made into a node.js module by replacing the JSON parser with the native method in node.

site: fork link broken

https://github.com/lloyd/JSONSelect/blob/master/site/index.html#L130

Homepage needs some examples to get people started

Hi there,
just stumbled upon your amazing package, really awesome.

One small suggestion. It would be great, and improve adoption rates, if the homepage contains samples how to get started so that people don't have to go to github to do it. Like for example node.js does with the mini server example.

JQuery like API?

Hello,
This isn't as much of an issue as a feature, but was wondering if it makes sense provide a jQuery like API which:

• Allowes chaining
• Clings on to the jQuery object if available

So that (in the browser), one can do stuff like:

var names = JSONSelect("string.name", OBJECT).map(jQuery.trim).map(function(name) {
return "The name is: " + name;
});

names.toArray() returns the real array, but names would be a jQuery-like object.

Golang Version

Hey there @lloyd --

Working off of a python implementation of jsonselect that I came across, I've written up a go implementation that supports all three levels (per your own tests) excepting error messages.

Cheers!

Python version?

Is there a python version? Or does this already exist?

Wanted real path with dot-notation in JSONSelect.forEach callback

Hello there!
First of all, your project is awesome!

It would be great if you could add the real-path with dot-notation of matched values in place of second argument of callback in forEach-method.

For example, we have a JSON:

var data = {
a: {x: 1, y: 2},
b: ["one", "two"]
};

JSONSelect.forEach('.a > number, .b :nth-child(1n)', data, function( value, path ){
// Here I would love to see:
// value - 1, path - "a.x"
// value - 2, path - "a.y"
// value - "one", path - "b.0"
// value - "two", path - "b.1"
});

Is it possible?
Thank you in advance.

key name?

Is it possible to retrieve a key name as opposed to a value?

e.g

FROM:

"resources": [{
    "params": {
        "one": "optional",
        "two": "optional",
        "three": "optional"
    }
}, {
    "params": {
        "four": "optional",
        "five": "optional",
        "six": "optional"
    }
}]

TO:

[
    "one",
    "two",
    "three",
    "four",
    "five",
    "six"
]

JSONSelect get direct child ?

var obj = { name : 'li1", options :[ {name : "li2"},{name : "li3"}] }
JSONSelect.match('.name',obj) // ==> ["li1","li2","li3"]

but i want only ===> ["li1"]

Unable to select numbers

Hi. The file shown below is a valid JSON object. However, unless I wrap numbers with the double quote, JSONSelect isn't able to select an object.

So, using this file:

{
   "categories": [
      {
         "id": "1",
        "name": "Test"
      },
      {
         "id": 2,
         "name": "Boom"
      }
  ]
}

If I write

JSONSelect.match('.categories :has(.id:val("1"))', data);

it find the object. If I write

JSONSelect.match('.categories :has(.id:val("2"))', data);

it doesn't find the object (an empty array is returned).

In jsonselect in its current form is a huge ideological problem.

In jsonselect in its current form is a huge ideological problem.

In the query language jsonselect swapped up (or mixed up) the names of the elements and classes.
For example, on page http://jsonselect.org/#tryit selector string.favoriteColor
shows yellow although ideologically correct to favoriteColor.string cause favoriteColor - name of the element, and string - his class (type).
Instead of string can be used number, array, hash, language-pseudotypes etc.

Again, that key to json should act as the names of the and types - as classes.

Take, for example jQuery and HTML DOM.
HTML has elements (such as <a>, <div>, etc.) and attributes of these elements (such as class, rel, src, id)
jQuery uses CSS Selectors. For example, to get the tag <div class="foo2"> in the following code:

<html>
    <head></head>
    <body>
        <div class="bar1">
            <div class="foo1"></div>
        </div>
        <div class="bar2">
            <div class="foo2"></div>
        </div>
    </body>
</html>

We can use the following selectors: div.foo2 or . bar2>. foo2 or body. foo2, etc.

In json have no attributes, such as class. There is only the element names (tags), compared with json html dom.
It would be logical to use the types of classes as values.

In general, if we take the example of page http://jsonselect.org/#tryit, it should be so:

code
.languagesSpoken .lang => languagesSpoken lang
.drinkPreference :first-child => drinkPreference:first-child
.seatingPreference :nth-child(1) seatingPreference:nth-child(1)
."weight" => weight
.lang => lang
.favoriteColor => favoriteColor
string.favoriteColor => favoriteColor.string
string:last-child => .string:last-child
string:nth-child(-n+2) => .string:nth-child(-n+2)
string:nth-child(odd) => .string:nth-child(odd)
string:nth-last-child(1) => .string:nth-last-child(1)
:root => :root
number => .number
:has(:root > .preferred) => :has(:root > preferred)
.preferred ~ .lang => preferred ~ lang
:has(.lang:val("Spanish")) > .level => :has(lang:val("Spanish")) > level
.lang:val("Bulgarian") ~ .level => lang:val("Bulgarian") ~ level
.weight:expr(x<180) ~ .name .first => weight:expr(x<180) ~ name first

In the end, what we have now - like the css selectors with very confused logic.
Urge to release a new edition of the standard, which will take into account this remark and focus on the new version.
Because if taken as a basis jQuery / CSS selectors, you must comply with this idea.

psuedo typo

http://jsonselect.org/#docs

There are 8 occurrences of "psuedo" instead of "pseudo" in the doc :)

Thanks for that lib, it is really sexy!

implement :has() in reference implementation

:has() was recently added to the docs as a level 3 addition. The reference implementation should include it.

throw 42 makes debugging with "break on exception" unusable

var bail = function() { throw 42; };

Makes debugging with "break on exception" unusable. Please workaround this somehow if possible.

match exact path

Hi,
First of all, it is great utility and i was able to find a good use in my project.

one thing which i came across was, when i do

JSONSelect.match(path , obj), it return me all the sub object which matches the path at all levels. What i mean by that is

let us say :
{
"attribute1" : value1,
"subObject" : {
"attribute1" : value2
}
}

JSONSelect(".attribute1" , obj) will return me an array
[ "attribute1" : value1 , "attribute1" , "value2 ]

Is there a way to say, i want the exact match (like first level) and not any sub-levels......

any advise is appreciated.

Thanks.

Is this dead?

See title.

Usage is not clear.

Usage is not clear, I suggest adding it to the Readme.

ie6 compatibility

I know, it's a real jerk thing to ask for...

but I'd like to incorporate JSONSelect into the Modernizr test suite but it doesnt run in IE6 which is a dealbreaker for me.

Not exactly sure where it's choking.. :/

you think you could take a look?

package.json file is invalid

Per http://package-json-validator.com/

I'm seeing the following error when trying to validate the JSONSelect/package.json file:

{
  "valid": false,
  "errors": [
    "Value for field name, JSONSelect does not match format: /^[a-z0-9][a-z0-9\\.\\-_]+$/"
  ],
  "warnings": [
    "Missing recommended field: keywords",
    "Missing recommended field: bugs",
    "Missing recommended field: licenses"
  ],
  "recommendations": [
    "Missing optional field: contributors",
    "Missing optional field: files",
    "Missing optional field: bin",
    "Missing optional field: man",
    "Missing optional field: directories",
    "Missing optional field: config",
    "Missing optional field: bundledDependencies",
    "Missing optional field: optionalDependencies",
    "Missing optional field: engineStrict",
    "Missing optional field: os",
    "Missing optional field: cpu",
    "Missing optional field: preferGlobal",
    "Missing optional field: private",
    "Missing optional field: publishConfig"
  ]
}

I looked into it, and it appears that that package names should be lowercased (as of npm 1.3) [1]. It looks like Isaac is considering relaxing that restriction, but thought I'd post this as a placeholder issue in case we wanted to consider changing this to jsonselect in the future.

[1] https://groups.google.com/forum/#!msg/npm-/LXXgFyLZBPQ/2S-lixuuEwcJ

a means of matching values

NOTE: This is design exploration for JSONSelect level 3, which is the wild and crazy stuff, not the bread and butter

A major feature missing is the ability to match values, features related to this are by far the most requested for people interested in JSONSelect.

Examples of these types of features exist in different selector engines. Like sizzle where :contains("Lloyd") would find nodes that have textual content where the substring Lloyd is found.

Practical applications of this are all over. Like what if you wanted to get my work email (use case provided by mike hanson):

[
    {
        "type": "home",
        "email": "[email protected]"
    },
    {
        "type": "work",
        "email": "[email protected]"
    }
 ]

Without the ability to match values, you would have to programmatically iterate over each member, which is rather lame for such a simple task.

One very general solution to the problem that I'm considering is an expression pseudo function. Something that might look like this when attempting to get my work email:

.type:expr(x = "work") ~ .email

That is the email sibling of a type key whose value satisfies the expression x = "work". A similar way to accomplish the same thing would be:

:has(.type:expr(x = "work")) .email

That is, the value of an email property on a node which has a descendant property named "type" that has a value matching x = "work".

I lean toward this :expr() construct because it can be extremely flexible, and the only really unnatural bit is the fact that x is a placeholder that refers to value. I do like that there's precedent for this idea in the nth-*() family of pseudo classes.

There are other ways we could go, such as including more specific pseudo functions for different operations. This is very much what jsonschema does. For instance:

.type:val("work") .email

Rather than a complex expression, this shorthand is basically equivalent, but less flexible. This approach becomes unwieldy and clunky for more complex expressions. Compare:

.salary:expr(80000 < x < 140000)

That is, find the value of a "salary" property that is a number which is less than 140000 and greater than 80000. If you attempted to express this expression with more granular specific pseudo classes you'd have something like:

.salary:less-than(140000):greater-than(80000)

We should be in no rush to close this issue.

python jsonselect level 2 implementation

Would be great to get this library listed on jsonselect.org/#code

https://github.com/mwhooker/jsonselect

It's only conformant up to level 2, but I'm working on getting level 3 working.

Thanks.

Support CSS4 selectors

Three years go by and there's another set of CSS Selectors available!

Not all of these are relevant to JSONSelect, but the Subject of a selector with Child combinator would be phenomenally useful. It would allow you to select a value whose child has some property without using :has():

.foo .bar .baz  # selects .baz
.foo !.bar .baz # selects .bar

Break conformance tests out into their own repository

So others can include them with less fuss.

test_lex.html for '.' fails

Which part of the regex is this supposed to match?

Failed example:
JSONSelect._lex(".");
Expected:
[1, "."]

Homepage needs some examples to get people started

Hi there,
just stumbled upon your amazing package, really awesome.

One small suggestion. It would be great, and improve adoption rates, if the homepage contains samples how to get started so that people don't have to go to github to do it. Like for example node.js does with the mini server example.

Offer a minified version

Just a suggestion - Could be nice to have the .min.js version with the important comments intact right in the repo.

Is jsonSelect case sensitive?

From example:

.lang:val("Spanish") works.

.lang:val("spanish") not works.

order based pseudo classes shouldn't match object children.

Rob Sayre mentioned a flaw in the design of JSONSelect. Specifically that there in not a reliable and widely implemented method of normalizing property order in JSON objects.

The fallout of this is that the application of order dependant pseudo classes (like :first-child, :nth-child(), etc), can be unreliable and can vary depending on the serialization of the object.

This should result in two changes to JSONSelect:

1. order based pseudo classes shouldn't match object children -- I argue that arrays are typically used where order is important (like for drink preferences ;)
2. The depth first post-order traversal requirement I wanted to place on implementations should become a recommendation.

NOTE: Brendan Eich mentions that a tighter definition about canonical property order is in progress, but in order for JSONSelect to be immediately useful, I still think ripping this feature out is the right thing to do.

Gonna leave this bug open for a bit for thoughts....

An expression parser

Hi, while using JSON Select i had a requirement of being able to access a JSON object with expression like [uuid="123"].account_info.[full_name="Test"].city_state_zip which would select any node that had UUID as 123 and a object in that node with full_name as Test and then the value of city_state_zip in that object, so the sake of simplicity i took a simple example. For the above requirement i have written a parser, which will convert an expression as above to a JSON Select expression. Below is the code for the same.

var Parser = function () {
var SIMPLE_EXPRESSION = 0;
var COMPLEX_EXPRESSION = 1;
var SELECT_ALL_EXPRESSION = 2;
var ATTRIBUTE_SEPERATOR = "=";
var SQUARE_OPENER = "[";
var SQUARE_CLOSER = "]";
var EXPRESSION_SEPERATOR = ".";
var GLOBAL_SELECTOR = "*";

    function Expression(objName, attribute, exprType) {
        this.exprType = exprType;
        this.objName = objName;
        if(exprType != SIMPLE_EXPRESSION){
            var attributeArr = attribute.split(ATTRIBUTE_SEPERATOR);
            this.attributeName = attributeArr[0];
            this.attributeVal = attributeArr[1];
        }
        this.cssExpression = this.generateCSSExpression();
    }

    Expression.prototype = {
        generateCSSExpression: function(){
            if(this.cssExpression == undefined || this.cssExpression == null){
                var cssExpression = null;
                switch(this.exprType){
                    case COMPLEX_EXPRESSION:
                        cssExpression = "."+this.objName+" :has(."+this.attributeName+":expr(x="+this.attributeVal+"))";
                        break;
                    case SELECT_ALL_EXPRESSION:
                        cssExpression = ":has(:root > ."+ this.attributeName+ ":expr(x="+this.attributeVal+"))";
                        break;
                    case SIMPLE_EXPRESSION:
                        cssExpression =  "."+this.objName;
                        break;
                }
                this.cssExpression = cssExpression;
            }
            return this.cssExpression;
        }
    }

    function Parser() {

    }

    Parser.parse = function (expr) {
        return new Parser().parse(expr);
    };

    Parser.getExpressionStack = function (expr) {
        return new Parser().getExpressionStack(expr);
    };

    Parser.prototype = {
        getExpressionStack: function(expr){
            return this.parseExpression(expr, false);
        },
        parse: function (expr) {
            return this.parseExpression(expr, true);
        },
        parseExpression: function(expr, isString){
            this.expression = expr;
            this.expressionStack = [];
            this.pos = 0;
            var objectArray = this.expression.split(EXPRESSION_SEPERATOR);
            var self = this;
            /**
             * This function is used to parse a expression and generate a CSS expression out of it
             * Example expressions are
             * name_address[type='PRI'].email_address would generate a CSS Expression .name_address .type:val('PRI') ~ .email_address
             */
            $.each(objectArray, function(index, expr){
                var expression = null;
                var expressionType = self.getExpressionType(expr);
                switch(expressionType){
                    case COMPLEX_EXPRESSION:
                    case SELECT_ALL_EXPRESSION:
                        var exprArray = expr.replace(SQUARE_OPENER,"").split(SQUARE_CLOSER);
                        if(exprArray[1] === "" && exprArray.length == 2){
                            exprArray[1] = exprArray[0].replace("*","");
                        }
                        expression = new Expression(exprArray[0], exprArray[1], expressionType);
                        break;
                    case SIMPLE_EXPRESSION:
                        expression = new Expression(expr, null, expressionType);
                        break;
                }
                if(isString){
                    self.expressionStack.push(expression.cssExpression);
                } else {
                    self.expressionStack.push(expression);
                }

            });
            if(isString){
                return self.expressionStack.join(" ");
            }
            return this.expressionStack;
        },
        getExpressionType: function(expr){
            if(expr.indexOf(GLOBAL_SELECTOR) >= 0)
                return SELECT_ALL_EXPRESSION;
            if(expr.indexOf(SQUARE_OPENER) >= 0 && expr.indexOf(SQUARE_CLOSER) >= 0)
                return COMPLEX_EXPRESSION;
            return SIMPLE_EXPRESSION;
        }
    }
    return Parser;
}();

I am a newbie to Javascript, Please let me know if the above can be improved

Matching properties with reserved names

I am testing JSONSelect against some activity streams json. The activity streams specification defines an object property in the json.

When I try to match the selector .object .objectType I am getting string required after ..

It seems that object is a reserved word for type selectors. I'm hoping a change to this regular expression will solve the problem, but I'm not sure what?

implement ~ in reference implementation

~ lets you express sibling relationships, it's now a level 2 addition to jsonselect. we should include support for it in the reference implementation.

Can not use multiple val selectors with strings

Attempting to do something like

:has(:root > :first-child:val("string")) :has(:root > :first-child:val("string"))

Causes an invalid json string error.

Ran this with some additional logging and it appears that the parser is trying to evaluate this as a JSON string:

"string")) :has(:root > :first-child:val("string"

Can work around this issue by performing another evaluation loop in the javascript space.

How to setup the return array/json value ?

How to setup the return array/json value ?
How about grouping a new array with the return data ?

A means of filtering selections

Suggested by @dunkfordyce , implementation discussed here:
http://librelist.com/browser//jsonselect/2011/6/3/objects/#b8bb946b3b839bd5894d9f78a5c6fb2fh

What if you want to limit the amount of data that's selected by your selector? A concrete case is when selecting a subset of twitter's api which return HUGE JSON OBJECTS with lots of stuff that YOU PROBABLY DONT CARE ABOUT

The proposal is to add two functions, one which discards items matching a selector, and the other which discards items not matching a selector. Naming is up in the air a bit, here are options:

• :only() and :without()
• :include() and :exclude()
• :reduce() and :remove()
• :limit() and :filter()
• some permutation of the above...

These functions take a selector as an argument (like :has()), they do not affect the selection algorithm at all, but are applied after elements are selected and modify the return value.

for instance, given the document:

{
    "first": "Lloyd",
    "middle": "Trevor",
    "last": "Hilaiel"
}

both :root:only(.first,.last) and :root:remove(.middle) would return:

{ "first": "Lloyd", "last": "Hilaiel" }

decouple matcher from traversal - for streaming filter?

would it be possible to decouple the matcher from the traversal,
so that it is possible to use this on objects that are fully created finished yet?

basically, over here Floby/node-json-streams#4 we are working on a streaming JSON parser,

in that process, we are already traversing the object,
would it be possible to call part of this module to select what parts of the stream we are interested in?

I presume you would need something like the root object, the current value, and the current path?

cheers, Dominic

How to write an expression return a object by it's key

something like this:

var data = {
  "objects" : [ ..., {
    "name": "JSONSelect"
  }, ...]
}

// return the child it's name is "JSONSelect"
jsonSelect.match(expr, data);

How to use || and && in the selector

I would like to use || and && in my expression. Here is an example.

On http://jsonselect.org/#tryit , I would like to select "name": {} when .first equals to "Lloyd" OR .last equals to "Hilaiel".

I tried this expression, but it does not work.
:root :has(.first:val("Lloyd") || .last:val("Hilaiel"))

Could you please give me a suggestion? Thank you.

Docs - how to use library

I don't see any documentation how to use a library. Is source code (and tests) the only API documentation? It's seems there are functions: match compile and forEach and two pseudo hidden _lex and _parse. Are there docs for them?