jg-rp / python-jsonpath Goto Github PK
View Code? Open in Web Editor NEWA flexible JSONPath engine for Python with JSON Pointer and JSON Patch
Home Page: https://jg-rp.github.io/python-jsonpath/
License: MIT License
A flexible JSONPath engine for Python with JSON Pointer and JSON Patch
Home Page: https://jg-rp.github.io/python-jsonpath/
License: MIT License
Include a function extension that accepts two string arguments and returns true
if the second argument is a prefix of the first argument.
Existing match
and search
function extensions can achieve the same result, but carry less semantic meaning and are probably less readable.
Add a parent()
method to JSONPointer
, returning a new JSONPointer
pointing to the original pointer's parent.
We are missing documentation for the keys()
function extension.
python-jsonpath/jsonpath/function_extensions/keys.py
Lines 7 to 8 in 8e72725
The non-standard contains
and in
membership operators currently work for any sequence.
python-jsonpath/jsonpath/env.py
Lines 551 to 554 in 880c123
We should change that so they work for any container (any mapping is a container).
We currently evaluate filter sub-expressions for every preceding selector match, even if that sub-expression can't change. Here, _.names.size
is resolved once for each item in users
.
$.users[[email protected] > _.names.size].name
With the exception of custom mapping classes that generate dynamic values - which we probably don't care about within the scope of a single findall()
/finditer()
- the only filter expression type that needs to be reevaluated for every item is SelfPath
(default @
).
To make it clear where the error is coming from, wrap potential JSONDecodeError
s when giving findall()
, finditer()
etc. a string as a target document.
We don't enforce the spec's well-typeness rules when it comes to validating JSONPath queries containing filter function calls. This was a deliberate choice in an attempt at making user-defined function extension as simple as possible.
However, there is nothing stopping us from supporting well-typedness validation for those that do want it. That goes for standard filer functions and user-defined functions.
We should extend the class-based filter function interface to allow for adding parameter and return type information, and a JSONPathEnvironment
option to enable/disable well-typedness validation.
When a singular filter query appears as part of a logical expression, we are incorrectly using the node's value rather than treating the query as an existence test.
Given the query $.users[[email protected] && @.score > 1]
and the data:
{
"users": [
{
"name": "Sue",
"score": 100
},
{
"name": "John",
"score": 86,
"admin": true
},
{
"name": "Sally",
"score": 84,
"admin": false
},
{
"name": "Jane",
"score": 55
}
],
"moderator": "John"
}
We get
[
{
"name": "John",
"score": 86,
"admin": true
}
]
And we should get
[
{
"name": "John",
"score": 86,
"admin": true
},
{
"name": "Sally",
"score": 84,
"admin": false
}
]
The keys()
functions extension does not inherit from FilterFunction
and define its argument/return types.
We probably need to return a special "Nothing" value that plays nicely with contains
and in
, then return that if keys()
is passed a non-mapping argument.
When parsing JSONPath filter expressions, we are not giving the logical not operator (!
) a higher precedence than logical or (||
) and logical and (&&
).
>>> import jsonpath
>>> str(jsonpath.compile("$[[email protected] || [email protected]]"))
"$[?!(@['a'] || !@['b'])]"
We are erroneously raising a JSONPathSyntaxError
when non-singular filter queries appear in logical expressions.
Example:
import jsonpath
data = [
{
"a": "b",
"d": "e"
},
{
"b": "c",
"d": "f"
}
]
query = "$[?@..* && @.b]"
jsonpath.findall(query, data)
# JSONPathSyntaxError: non-singular query is not comparable, line 1, column 8
This would be the correct behaviour in a comparison expression, as non-singular queries can not be compared, but we are applying the same logic to logical expressions, which is bad.
Rather than messing around decoding, encoding then decoding string literals, we should pass them through json.loads()
. It will handle UTF-16 encoded surrogate pairs and unnecessary escape characters. The latter is something the spec requires, and that we don't currently do.
python-jsonpath/jsonpath/parse.py
Lines 394 to 401 in 6ce5495
Also, we're currently only doing this for quoted property selectors, and not string literal in comparison expressions or filter function arguments.
Commit the JSONPath Compliance Test Suite as a submodule. This will allow us to test against it in GitHub workflows and make it easier to report accurate test coverage.
We'll need to cover git submodule usage in a "contributing" guide.
Add an optional flags
argument to the built-in match()
and search()
filter functions.
This would be a deliberate deviation to the IETF JSONPath standard.
An empty JSONPath query (non-standard) or a query containing just the root identifier ($
) will return the target JSON document in its entirety.
from jsonpath import findall
rv = findall("$", {'a': [1, 2, 3], 'b': 42})
print(rv) # [{'a': [1, 2, 3], 'b': 42}]
But there is no way to conditionally return the target root (or an empty result/node list) using a filter selector. This is a known limitation in the IETF JSONPath Base. See ietf-wg-jsonpath/draft-ietf-jsonpath-base#390.
We can stray into Python and use jsonpath.match()
.
from jsonpath import match
data = {'a': [1, 2, 3], 'b': 42}
rv = [data] if match("$[@ < 50]", data) else []
# ...
But that is not great or always an option.
One solution is to add an additional root identifier. A root Identifier that is functionally equivalent to wrapping the target JSON value in a single-element array. I suggest the circumflex (U+005E ^
aka caret) character for this purpose. I've been calling this the "fake root identifier".
from jsonpath import findall
data = {'a': [1, 2, 3], 'b': 42}
# Implicitly wrap `data` in an array.
findall("^[[email protected] < 50]", data) # [{'a': [1, 2, 3], 'b': 42}]
findall("^[[email protected] > 50]", data) # []
# Same as explicitly wrapping `data`.
findall("$[[email protected] < 50]", [data]) # [{'a': [1, 2, 3], 'b': 42}]
findall("$[[email protected] > 50]", [data]) # []
Add a match()
function to complement findall()
and finditer()
.
jsonpath.match()
will take the same arguments as findall()
and finditer()
, but return a JSONPathMatch
for the first match found, or return None
if there were no matches.
Note that None
could be the legitimate result of a match, but a match()
result will always be wrapped in a JSONPathMatch
instance. So the matched object would be available as JSONPathMatch.obj
.
Add support for specifying a custom JSONDecoder
when calling findall()
or finditer()
with a string or file-like object containing a JSON document.
Add support for building JSONPointer
instances from and existing pointer and a relative JSON Pointer.
See https://www.ietf.org/id/draft-hha-relative-json-pointer-00.html.
Add _
as a handle for the current key when filtering a map-like object.
When filtering a map-like object, _
would be the key associated with @
, or undefined
if filtering a non-mapping object.
Add a non-standard "keys" or "property names" selector to retrieve keys/properties from an object/mapping/dict.
Some JSONPath implementations use ~
as a keys selector. In the absence of a better idea, we'll do the same.
We're getting a JSONPathSyntaxError
when trying to parse a JSONPath containing a filter context selector (default _
) as a filter function argument.
Add support for converting a JSONPathMatch
(as generated by finditer()
) to a JSON Pointer.
Our JSON Pointer class will define get
and set
methods for retrieving and updating a single value in a JSON document or equivalent Python object.
There's no immediate plan to evaluate JSON Pointers given as strings.
It has become clear that a complete implementation of rfc6901 is for the best, plus rfc6902 (JSON Patch) for modifying JSON documents, with a Python API mirroring JSON Patch operations.
Implement JSONPointer.__truediv__()
to build child pointers from an existing JSON Pointer.
Add a non-standard typeof()
or type()
filter function. Such a function would return the type of its argument as a string.
We would default to using JSON terminology - "object", "array", "number" etc. - but allow package users to select Python equivalent type names or define custom aliases.
isinstance()
or is()
would be similar, but return a Boolean if the type of the first argument matches the second argument. This has the benefit of matching against several possible aliases for a type in one function call.
I have a json like this
{
"task_tag": "blade_appearance_detect",
"image_type": "base64",
"extra_args": [
{
"model": "crack",
"param": {
"conf": 0.66,
"iou": 0.25
}
},
{
"model": "damaged",
"param": {
"conf": 0.3,
"iou": 0.25
}
}
]
}
I want param's all keys, expect ['conf', 'iou', 'conf', 'iou'],
I use jsonpath.findall('$..param~', ret)
to get right result, but other syntax is "$..param.*~", not compatible.
When given an empty parts iterable, the class method JSONPointer.from_parts()
currently returns a pointer with a string representation of /
. It should be an empty string.
There's a similar situation with JSONPointer.from_match()
, although there's technically no way to reference the document root using a valid JSONPath.
We have findall()
and finditer()
, add find_one
, find_n
/ limit
and skip
/ drop
helpers for managing the results generated by finditer()
.
Also alias findall
as find_all
and finditer
as find_iter
.
Add an option to disable decoding of UTF-16 escape sequences when parsing a JSONPath. If client code has already handled the possibility UTF-16 code points, including surrogate pairs, disabling it could improve parsing performance.
JSONPointer
and JSONPatch
already have arguments to control UTF-16 decoding.
Remember to:
When given an arbitrary string without any slashes, pointer.ressolve()
returns the document root. Only the empty string should resolve to the document root.
import jsonpath
print(jsonpath.pointer.resolve("noshuchthing", {"foo": [0, 1, 2]}))
Output:
{'foo': [0, 1, 2]}
We expect a JSONPointerKeyError
.
Add a command line interface for:
Add an exists()
method to JSONPointer
. exists()
takes a JSON-like "document" and returns true
if the document contains a value at the pointed to location, or false
otherwise.
Hi,
Just wanted to thank you for this awesome json path library. The best I've tried and the most pleasing to work with.
Keep up the good work!
The spec allows bracketed segments to contain an arbitrary number of filter selectors alongside other selectors, separated by commas. We currently only recognise filters that appear on their own inside square brackets.
$.foo[?<expression>]
but not
$.foo['bar', ?<expression>, ?<expression>]
We should follow the spec in this regard.
All commits up to this point, including the syntax defined in jsonpath.bnf, where done without reference to the IETF JSONPATH Working Group Internet-Draft. I have only just become aware of it.
With the goal of aligning with the aforementioned document, we need to ..
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.