thedevsaddam / gojsonq

A simple Go package to Query over JSON/YAML/XML/CSV Data

A simple Go package to Query over JSON Data. It provides simple, elegant and fast ODM like API to access, query JSON document

Installation

Install the package using

$ go get github.com/thedevsaddam/gojsonq/v2

Usage

To use the package import it in your *.go code

import "github.com/thedevsaddam/gojsonq/v2"

Let's see a quick example:

See in playground

package main

import gojsonq "github.com/thedevsaddam/gojsonq/v2"

func main() {
	const json = `{"name":{"first":"Tom","last":"Hanks"},"age":61}`
	name := gojsonq.New().FromString(json).Find("name.first")
	println(name.(string)) // Tom
}

Another example:

See in playground

package main

import (
	"fmt"

	gojsonq "github.com/thedevsaddam/gojsonq/v2"
)

func main() {
	const json = `{"city":"dhaka","type":"weekly","temperatures":[30,39.9,35.4,33.5,31.6,33.2,30.7]}`
	avg := gojsonq.New().FromString(json).From("temperatures").Avg()
	fmt.Printf("Average temperature: %.2f", avg) // 33.471428571428575
}

You can query your document using the various query methods such as Find, First, Nth, Pluck, Where, OrWhere, WhereIn, WhereStartsWith, WhereEndsWith, WhereContains, Sort, GroupBy, SortBy and so on. Also you can aggregate data after query using Avg, Count, Max, Min, Sum etc.

Find more query API in Wiki page

Bugs and Issues

If you encounter any bugs or issues, feel free to open an issue at github.

Also, you can shoot me an email to mailto:[email protected] for hugs or bugs.

Credit

Special thanks to Nahid Bin Azhar for the inspiration and guidance for the package. Thanks to Ahmed Shamim Hasan Shaon for his support from the very beginning.

Contributors

• Lenin Hasda
• Sadlil Rhythom
• See contributors list here

Contribution

If you are interested to make the package better please send pull requests or create an issue so that others can fix. Read the contribution guide here

License

The gojsonq is an open-source software licensed under the MIT License.

gojsonq's Issues

Can it handle 50 GB Files like Wikipedia?

and any plans for making WRITES possible?

How access array of objects using gojsonq?

const json = [{"city":"dhaka","type":"weekly","temperatures":[30,39.9,35.4,33.5,31.6,33.2,30.7]}]

avg := gojsonq.New().JSONString(json).From("[0].temperatures").Avg()
Above code does not work.

How can it be done?

Add support for writing the final output to io.Writer

gojsonq.New().File("./data.json").From("items").SortBy("price").Writer(os.Stdout)

Multiple nested array support

Is it possible to traverse multiple nested arrays in this library without knowing the array indexes? I have a scenario that works well with where clauses on the first array, but I haven't had any success with a second array below that.

This is a successful query using Find (but assumes I know the index to each array):

Find("data.items.[1].nestedA.[3].nestedB.value")

I've tried many variations of:
From("data.items").Where("itemValue", "=", "asdf").From("nestedA").Where("nestedBOtherValue", "eq", "fdsa").Find("nestedB.value") without any luck.

Selecting the average of a property of a group

Given the following data:

{
  "products": [
    { 
      "name": "foo",
      "price": 10
    },
    { 
      "name": "foo",
      "price": 20
    },
    { 
      "name": "bar",
      "price": 5
    }    
  ]
}

how would I try to query for the average price per distinct name? I've tried .From("products").GroupBy("name").Select("price").Avg(); and .From("products").GroupBy("name").Avg("price") but those both result in NaN.

Find map's element using Where query

Query: SELECT * FROM . WHERE age < 30

{
  "one":{
    "name":"John",
    "age":28
  },
  "two":{
    "name":"Jane",
    "age":30
  },
  "three":{
    "name":"Tom",
    "age":28
  }
}

Result:

{
  "one":{
    "name":"John",
    "age":28
  },
  "three":{
    "name":"Tom",
    "age":28
  }
}

How to map result to struct?

Hello, having following query
applications.WhereEqual("reporting", false).Select("applications").Only("id", "name")
and need to map it to struct.

Example of return value is
[map[id:1.94664978e+08] map[id:1.12640692e+08] map[id:1.94411185e+08]]

Could You suggest how? Having no idea :(

Support for unselect

I have an array of JSON objects which all contain a few common keys and then a whole bunch of keys that are different. I want to select a subset of this array that drops a single common key.

The only way to do that right now is to use a select or only with all the possible keys the object can contain minus the one I want to drop. It would be nice to support something like "unselect" or "drop" where I can drop a specific key from the result while keeping the rest.

Add support for nested level query using Where

Add nested level query using Where()

res := gojsonq.New().File("./data.json").From(".").Where("users.address.present", "=", "LA").Get()
println(res)

Support for `Result` method

Result will return static type instead of interface.

const json = `{"name":{"first":"Tom","last":"Hanks"},"age":61}`
result, err := gojsonq.New().JSONString(json).FindR("name.first")
if err != nil {
	log.Fatal(err)
}
name, _ := result.String()
fmt.Println(name)

Result may contains method like: String/Int/Int32/Int64/Uint/Uint32/Uint64/Float32/Float64/[]String/[]Int/[]Uint/[]Float32 etc method

Result method name may FindR/GetR etc

Add Date Support

Looks great. I'm eager to start trying it out.

One thing I'll need soon, though, is support for dates. Basically, in addition to int, string, bool or slice in Where expressions, I need to be able to use dates. Somehow (tbd), I would expect to specify the format of the date string which would apply to both the string I pass as val to Where as well as what I'm expecting to be found in the JSON. Ideally, this format could be any of the predefined layouts supported by time.Parse. The Where op arguments would need to be extended to support the equivalents of time.After, time.Before and time.Equal. Finally, I need a way to traverse a JSON collection in chronological order or at a minimum pull the first or last chronologically from a collection. E.g. date-aware variations on Sort and SortBy would suffice.

The type of value is wrong

func main()  {
	text := `{"id": 1827049}`
	j := gojsonq.New().JSONString(text).Find("id")
	fmt.Println("Num: ", j, "\nType: ", fmt.Sprintf("%T", j))
}

output:
Num:  1.827049e+06 
Type:  float64

Add option for enable Debug mode as well as custom logger

Add option for enable Debug mode as well as custom logger:

var customLogger *log.Logger
customLogger = log.New(os.Stderr, "", log.LstdFlags)
jq := gojsonq.New(gojsonq.SetDebug(true), gojsonq.SetLogger(&customLogger{}))

Map[string]interface{} root

Doesn't seem to work when you don't have nicely sliceable structures.

IE: https://www.iinet.net.au/status/event/events.json

I tried Where("DRW", "=", "Darwin") but kept getting nothing back

Offset and limit (Pagination)

Hi, do you have something that we can use to paginate like what we use in SQL (limit and offset) for example? I searched the docs but couldn't find anything.

Add support for nested level Select with alias

Add support for nested level Selection:

res := gojsonq.New().File("./data.json").From(".").Select("user.id as uid", "user.name.first", "user.posts").Where("users.address.present", "=", "LA").Get()
println(res)

Node names with "." characters in them, like foo.example.com

I'n looking at ansible inventory data containing hostnames, of the form "a192.example.com", and wonder how to express a query in gojsonq, for a datum such as "_meta.hostvars.a001.example.com.ZA_SERVER", which has the expected value of "zabbix.example.com"

Right now, it returns the empty string (;-))

Querying complex json

Hi,

I'm very new to Go and I stumbled across this module which I think is fantastic! I have a question more than issue which I'm hoping you can help with.

I have some very complicated and nested json (json output of aws describe instances) which I'm trying to query using gojsonq.

I've added an obfuscated version of the output below, essentially what I'm trying to do is query the Reservations.[].Instances[] to grab some details.

For example the payload below has details for two instances, I'm trying to get the InstanceIds as follows (without any luck).

instances := gojsonq.New().File("instances.json")
res := instances.From("Reservations.[].Instances.[]").Only("InstanceId")

What am I doing wrong? or is the complexity the issue?

{ "NextToken": null, "Reservations": [{ "Groups": null, "Instances": [{ "AmiLaunchIndex": 0, "Architecture": "x86_64", "BlockDeviceMappings": [{ "DeviceName": "/dev/sda1", "Ebs": { "AttachTime": "2017-01-10T12:59:09Z", "DeleteOnTermination": true, "Status": "attached", "VolumeId": "vol-085817e5af91e75ef" } }], "CapacityReservationId": null, "CapacityReservationSpecification": { "CapacityReservationPreference": "open", "CapacityReservationTarget": null }, "CpuOptions": { "CoreCount": 1, "ThreadsPerCore": 1 }, "EbsOptimized": false, "ElasticGpuAssociations": null, "ElasticInferenceAcceleratorAssociations": null, "EnaSupport": null, "HibernationOptions": { "Configured": false }, "Hypervisor": "xen", "ImageId": "ami-ae6259cd", "InstanceId": "i-abcdefg1234", "InstanceLifecycle": null, "InstanceType": "t2.small", "KernelId": null, "KeyName": "obfs", "LaunchTime": "2018-04-23T13:18:58Z", "Licenses": null, "Monitoring": { "State": "disabled" }, "NetworkInterfaces": [{ "Association": null, "Attachment": { "AttachTime": "2018-01-10T12:59:09Z", "AttachmentId": "eni-attach-e6303189", "DeleteOnTermination": true, "DeviceIndex": 0, "Status": "attached" }, "Description": "Primary network interface", "Groups": [{ "GroupId": "sg-obfs", "GroupName": "obfs" }], "InterfaceType": "interface", "Ipv6Addresses": null, "MacAddress": "02:1a:4b:29:c9:1f", "NetworkInterfaceId": "eni-abcdefg", "OwnerId": "0000000000", "PrivateDnsName": "ip-172-0-0-1.ap-southeast-1.compute.internal", "PrivateIpAddress": "172.0.0.1", "PrivateIpAddresses": [{ "Association": null, "Primary": true, "PrivateDnsName": "ip-172-0-0-1.ap-southeast-1.compute.internal", "PrivateIpAddress": "172.0.0.1" }], "SourceDestCheck": true, "Status": "in-use", "SubnetId": "subnet-obfs", "VpcId": "vpc-obfs" }], "Placement": { "Affinity": null, "AvailabilityZone": "ap-southeast-1a", "GroupName": "", "HostId": null, "PartitionNumber": null, "SpreadDomain": null, "Tenancy": "default" }, "Platform": null, "PrivateDnsName": "ip-172-0-0-1.ap-southeast-1.compute.internal", "PrivateIpAddress": "172.0.0.1", "ProductCodes": null, "PublicDnsName": "", "PublicIpAddress": null, "RamdiskId": null, "RootDeviceName": "/dev/sda1", "RootDeviceType": "ebs", "SecurityGroups": [{ "GroupId": "sg-obfs", "GroupName": "obfs" }], "SourceDestCheck": true, "SpotInstanceRequestId": null, "SriovNetSupport": null, "State": { "Code": 80, "Name": "stopped" }, "StateReason": { "Code": "Client.UserInitiatedShutdown", "Message": "Client.UserInitiatedShutdown: User initiated shutdown" }, "StateTransitionReason": "User initiated (2018-04-23 13:38:03 GMT)", "SubnetId": "subnet-b8a736dc", "Tags": [{ "Key": "new", "Value": "tag" }, { "Key": "Name", "Value": "obfs" }, { "Key": "environment", "Value": "DEV" }], "VirtualizationType": "hvm", "VpcId": "vpc-obfs" }], "OwnerId": "00000000000", "RequesterId": null, "ReservationId": "r-012456789" }, { "Groups": null, "Instances": [{ "AmiLaunchIndex": 0, "Architecture": "x86_64", "BlockDeviceMappings": [{ "DeviceName": "/dev/xvda", "Ebs": { "AttachTime": "2018-04-15T11:35:14Z", "DeleteOnTermination": true, "Status": "attached", "VolumeId": "vol-30234567890" } }], "CapacityReservationId": null, "CapacityReservationSpecification": { "CapacityReservationPreference": "open", "CapacityReservationTarget": null }, "ClientToken": "", "CpuOptions": { "CoreCount": 1, "ThreadsPerCore": 1 }, "EbsOptimized": false, "EnaSupport": true, "HibernationOptions": { "Configured": false }, "Hypervisor": "xen", "IamInstanceProfile": null, "ImageId": "ami-001122334455aa1bbb", "InstanceId": "i-9876543210", "InstanceLifecycle": null, "InstanceType": "t2.micro", "KernelId": null, "KeyName": "obfs", "LaunchTime": "2019-04-15T11:35:13Z", "Licenses": null, "Monitoring": { "State": "disabled" }, "NetworkInterfaces": [{ "Association": null, "Attachment": { "AttachTime": "2019-04-15T11:35:13Z", "AttachmentId": "eni-attach-01331224f2b57b180", "DeleteOnTermination": true, "DeviceIndex": 0, "Status": "attached" }, "Description": "", "Groups": [{ "GroupId": "sg-obfs", "GroupName": "obfs1" }], "InterfaceType": "interface", "Ipv6Addresses": null, "MacAddress": "02:8a:6f:31:7c:dc", "NetworkInterfaceId": "eni-obfs", "OwnerId": "0000000000", "PrivateDnsName": "ip-172-0-0-1.ap-southeast-1.compute.internal", "PrivateIpAddress": "172.0.0.1", "PrivateIpAddresses": [{ "Association": null, "Primary": true, "PrivateDnsName": "ip-172-0-0-1.ap-southeast-1.compute.internal", "PrivateIpAddress": "172.0.0.1" }], "SourceDestCheck": true, "Status": "in-use", "SubnetId": "subnet-obfs", "VpcId": "vpc-obfs" }], "Placement": { "Affinity": null, "AvailabilityZone": "ap-southeast-1a", "GroupName": "", "HostId": null, "PartitionNumber": null, "SpreadDomain": null, "Tenancy": "default" }, "Platform": null, "PrivateDnsName": "ip-172-0-0-1.ap-southeast-1.compute.internal", "PrivateIpAddress": "172.0.0.1", "ProductCodes": null, "PublicDnsName": "", "PublicIpAddress": null, "RamdiskId": null, "RootDeviceName": "/dev/xvda", "RootDeviceType": "ebs", "SecurityGroups": [{ "GroupId": "sg-obfs", "GroupName": "obfs1" }], "SourceDestCheck": true, "SpotInstanceRequestId": null, "SriovNetSupport": null, "State": { "Code": 80, "Name": "stopped" }, "StateReason": { "Code": "Client.UserInitiatedShutdown", "Message": "Client.UserInitiatedShutdown: User initiated shutdown" }, "StateTransitionReason": "User initiated (2019-04-15 20:29:38 GMT)", "SubnetId": "subnet-obfs", "Tags": [{ "Key": "purpose", "Value": "dev" }, { "Key": "created", "Value": "2019-04-15" }, { "Key": "Name", "Value": "obfs1" }], "VirtualizationType": "hvm", "VpcId": "vpc-obfs" }], "OwnerId": "00000000000", "RequesterId": null, "ReservationId": "r-obfs" }] }

Retrieving a subtree based on a key

Hi

Suppose I want to filter a JSON data string from a path based on an intermediate key which doesn't have value (it is just a key). Is that possible?

Optimize searching algorithm

Instead of Linear search we should use a better way to find element when querying large amount of data.

Querying number fields appears to force float64

From the example on the readme:

package main

import "github.com/thedevsaddam/gojsonq"

func main() {
	const json = `{"name":{"first":"Tom","last":"Hanks"},"age":61}`
	age := gojsonq.New().JSONString(json).Find("age")
	println(age.(float64))
}

Why does age get set as a float64 instead of an int or int64? Not very intuitive because looking at the JSON there are no decimal places and the number is small. What's the rationale? Is it always float64?

jsonq.go:395: undefined: sort.Slice

As sort.Slice introduced in go 1.8, it'll fail in the previous version of go 1.8.Need to fix the issue

Custom CSV file, with header on 2 lines

Hi,

I need to parse a csv file that has first general info about data, on 2 lines, and after, a list of measures ( timestamp --> value )

Is it possible to parse it with gojsonq ? I don't really need to query it, accessing all data with a loop is enough for me.

Here is a sample of how the csv begins:

Identifiant PRM;Type de donnees;Date de debut;Date de fin;Grandeur physique;Grandeur metier;Etape metier;Unite;Pas en minutes
23545612340000;Courbe de charge;06/08/2019;08/08/2019;Energie active;Consommation;Comptage Brut;W;
Horodate;Valeur;;;;;;;
2019-08-06T00:30:00+02:00;282;;;;;;;
2019-08-06T01:00:00+02:00;88;;;;;;;
2019-08-06T01:30:00+02:00;124;;;;;;;
2019-08-06T02:00:00+02:00;94;;;;;;;
2019-08-06T02:30:00+02:00;138;;;;;;;
2019-08-06T03:00:00+02:00;94;;;;;;;
2019-08-06T03:30:00+02:00;122;;;;;;;
2019-08-06T04:00:00+02:00;146;;;;;;;
2019-08-06T04:30:00+02:00;184;;;;;;;
2019-08-06T05:00:00+02:00;150;;;;;;;
2019-08-06T05:30:00+02:00;120;;;;;;;

How should I do it ?

Query json for array size.

Hello,

This is an "info request".

I have the following json and I would like to check if the input contains a father (fid) with two children.

{
  "data": [
    {
      "fid": 10,
      "children": [
        {
          "cid": 123
        },
        {
          "cid": 124
        }
       
      ]
    }
  ]
}

The code so far is

package main

import (
	"fmt"

	"github.com/thedevsaddam/gojsonq"
)

const json = `{
	"data": [
	  {
		"fid": 10,
		"children": [
		  {
			"cid": 123
		  },
		  {
			"cid": 124
		  }
		 
		]
	  }
	]
  }`

func main() {

	q := gojsonq.New().JSONString(json).From("data").
		Where("fid", "=", 10)
	fmt.Println(q.Get())
}

How can I modify the source code to decide if fid=10 AND children.size() = 2?

In principle how can I use the "array size" to query the json file/input?

Thanks

FromInterface custom decoder

Hi there,
Is there any way to load JSON document from map[string]interface{}?
Simplest use case is querying MongoDB, which returns result documents in that format(without using schema struct). I am willing to make a PR if little help is supplied.
Thanks

Code review, refactoring and doc block

Review code and refactoring. Add missing doc block to method and functions.

Add support for nested level query using GroupBy

Add support for the nested level in GroupBy()

res := gojsonq.New().File("./data.json").From(".").GroupBy("post.year").Get()
println(res)

NOT an issue, more a touch of help. Using parameters with gojsonq package

I am trying to build a server that can fetch certain json entries based on a parameter.
So for example if a user enters localhost:8080/store/1 It returns the json entry with the ID 1.

The code i currently have works if i hardcode the id field into the query, but trying to put in parameters i have so far had no luck.
Trying this:

func getGame(w http.ResponseWriter, r *http.Request) {
	params := mux.Vars(r)
	jq := gojsonq.New().File("./data/games.json")
	res := jq.From("games").Where("id", "=", params["id"]).Get()
	fmt.Println(res)
}

Suggestions please, before i throw something out the window.

multilevel slice

package main

import (
	"fmt"

	"github.com/thedevsaddam/gojsonq"
)

func main() {
	jq := gojsonq.New().JSONString(jsonStr).From("users").Where("addresses.name", "=", "home")
	fmt.Printf("%+v%v", jq.Get(), jq.Error())

}

var jsonStr = `
{
   "users": [
       {
           "name": "jhon",
           "addresses": [
               {"name": "home"},
               {"name": "work"}
           ]
        }
   ]
}
`

➜  jsontest go run main.go
[]gojsonq: invalid node name name%

is there anyway to do it?

Pass dynamic i.th value in for loop to Find method

how can I pass i value in

gojsonq.New().FromString(data).From("tab_info.[0].product_info.products.[i]").

the above code will certainly not work.

Go-critic warnings

go-critic linter found some issues , here corresponding log:

check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS
witchVar: case 0 can benefit from type switch with assignment
check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS
witchVar: case 1 can benefit from type switch with assignment
check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS
witchVar: case 2 can benefit from type switch with assignment
check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS
witchVar: case 3 can benefit from type switch with assignment
check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS
witchVar: case 4 can benefit from type switch with assignment
check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS
witchVar: case 5 can benefit from type switch with assignment
check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS
witchVar: case 6 can benefit from type switch with assignment
check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\jsonq.go:271:3: range
ValCopy: each iteration copies 48 bytes (consider pointers or indexing)
check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\jsonq.go:45:18: typeU
nparen: could simplify to [][]query

In #18 I propose fixing for them

Why replacerange with for loop? You must create an issue regarding this refactor before sending PR.

Thanks, good point. The way that I made in my PR not very accurate and should be replaced into this one as written in description to rangeValCopy.

How to get the queried data to user defined custom type?

How can I get the output data to a custom struct like:

Input document:

{
  "items":[
    {
      "id":1,
      "name":"MacBook Pro 13 inch retina",
      "price":1350
    },
    {
      "id":2,
      "name":"MacBook Pro 15 inch retina",
      "price":1700
    }
  ]
}

Code:

type item struct {
	ID    int    `json:"id"`
	Name  string `json:"name"`
	Price int    `json:"price"`
}

func main() {
	v := item{}
	output := gojsonq.New().File("./data.json").From("items.[0]").Get()
        // how to assign the output value to v
}

What I want:

package main

import (
	"github.com/davecgh/go-spew/spew"
	"github.com/thedevsaddam/gojsonq"
)

type item struct {
	ID    int     `json:"id"`
	Name  string  `json:"name"`
	Price float64 `json:"price"`
}

func main() {
	v := item{}
	jq := gojsonq.New().File("./data.json").From("items.[0]")
	jq.Out(&v)
	if jq.Error() != nil {
		panic(jq.Error())
	}
	spew.Dump(v)
}

// Output:
// (main.item) {
//  ID: (int) 1,
//  Name: (string) (len=26) "MacBook Pro 13 inch retina",
//  Price: (float64) 1350
// }

From should support a variable-length argument to describe the path

Thanks for a great package! :D

I am parsing JSON that uses a dot (".") for some key names. I've discovered that I could redefine the separator using gojsonq.SetSeparator("·") to a middle-dot, and then use fmt.Sprintf to define the path input to gojsonq.From.

What would be even better would be to avoid all of this, and just have From support multiple arguments, with each argument describing components of the path to the desired node. In this case, the separator can be ignored, and I don't have to use the middle dot setup as above.

For example, q.From("first.level.key.name", "data") would select the path:

• first.level.key.name
  • data

where the first path component has dots in it, and the second path component does not.

At the moment, I have to do the following:

q.From(fmt.Sprintf("%s·%s", "first.level.key.name", "data"))

Add support for nested level Sorting using SortBy

Add support for nested level sorting using SortBy()

res := gojsonq.New().File("./data.json").From(".").SortBy("users.name.first").Get()
println(res)

Hi everyone, who's using gojsonq in production?

Hello everyone, I'm just curious any companies or individual using gojsonq in production? If you see the post please share with us, I'd like to include their list in Readme. It'll be inspiring to hear from you.

how to extract child element after using find

package main

import (
"fmt"
"github.com/thedevsaddam/gojsonq"
"log"
)
func main(){
jsonStr := {"code":0,"data":{"tick":{"bids":[[0.5,0.335],[0.45,0.235],[0.4,0.35]],"asks":[[0.6,22.01],[0.61,0.52],[0.63,0.35]]}}}
grapArg := GrapArg{path:"data.tick",keyBids:"bids",keyAsks:"asks",keyPrice:"0",keyAmount:"1"}
getDepthByPath(jsonStr,grapArg)
}

func getDepthByPath(jsonStr string,grapArg GrapArg)([][]float32,[][]float32){
jq := gojsonq.New().FromString(jsonStr)
path := grapArg.path
if len(path)>0{
data := jq.Find(path)
if jq.Error() != nil {
log.Fatal(jq.Errors())
}
fmt.Println(data)

}
return [][]float32{{0.0000000000000001},{0.0001}}, [][]float32{{1000000},{0.0001}}

}

type GrapArg struct {
path string
keyBids string
keyAsks string
keyPrice string
keyAmount string
}

// I want to extract elements "bids" and "asks" , how to do ? help me if convenient please!

Provide option for custom JSON decoder

Option for user to provide custom JSON Decoder , like below:

package main

import (
	"fmt"

	"github.com/davecgh/go-spew/spew"
	jsoniter "github.com/json-iterator/go"
	"github.com/pquerna/ffjson/ffjson"
	"github.com/thedevsaddam/gojsonq"
)

func main() {
	jq := gojsonq.New(gojsonq.SetDecoder(&iteratorDecoder{})).
		File("./data.json").
		From("vendor.items").
		Where("id", "=", 1).OrWhere("id", "=", 3)

	spew.Dump("Result: ", jq.Only("name", "id"), jq.Error())
}

type iteratorDecoder struct {
}

func (i *iteratorDecoder) Decode(data []byte, v interface{}) error {
	var json = jsoniter.ConfigCompatibleWithStandardLibrary
	return json.Unmarshal(data, &v)
}

type ffDecoder struct {
}

func (f *ffDecoder) Decode(data []byte, v interface{}) error {
	return ffjson.Unmarshal(data, &v)
}

Add support for JSON stream loaded

Without loading the full json into memory there should be an option to process json in a stream for large documents.

How do I Find() or Get() from which key has dots?

Hi, I have the following issue:

package main

import (
	"fmt"

	"github.com/thedevsaddam/gojsonq"
)

var (
	jsonstring = `
		{
			"labels": {
				"foo": "bar",
				"io.kubernetes.pod.name": "mytest-sandbox",
				"io.kubernetes.pod.namespace": "default",
				"io.kubernetes.pod.uid": "c27e82eb-785a-4532-8121-e0bb0f8f2461"
			}
		}`
)

func main() {
	jq := gojsonq.New().JSONString(jsonstring)
	fmt.Println(jq.Find(`labels.foo`))
	fmt.Println(jq.Find(`labels.io.kubernetes.pod.name`))
	fmt.Println(jq.Find(`labels."io.kubernetes.pod.name"`))
	fmt.Println(jq.Find(`labels.[io.kubernetes.pod.name]`))
	fmt.Println(jq.Find(`labels.["io.kubernetes.pod.name"]`))
	fmt.Println(jq.Find(`labels[io.kubernetes.pod.name]`))
	fmt.Println(jq.Find(`labels["io.kubernetes.pod.name"]`))
	fmt.Println(jq.From("labels").Select("io.kubernetes.pod.name").Get())
}

which outputs:

bar
<nil>
<nil>
<nil>
<nil>
<nil>
<nil>
[]

Any idea what I could do to be able to select the value of a key with dots?

Multiple calls to `Find` returns nil

I'm trying to use gojsonq to make some queries to JSON data. I was setting up some tests that were returning nil when I tried to validate the data inside of the JSON object. Eventually, I noticed if I do multiple Finds, it starts returning nil.

Here's an example of the behavior: https://play.golang.org/p/qxiooDmflIe

Is this expected? I couldn't find a way in the documentation to run the same query multiple times and get the same data back.

Cannot find all elements in array

sample

{
  "items" : [ {
    "downloadUrl" : "http://localhost:8081/repository/maven-releases/file2/file2/1.0.0/file2-1.0.0.jar",
    "path" : "file2/file2/1.0.0/file2-1.0.0.jar",
    "id" : "bWF2ZW4tcmVsZWFzZXM6ZDQ4MTE3NTQxZGNiODllYzQzMTI5MDdhYzdiOWUyMDM",
    "repository" : "maven-releases",
    "format" : "maven2",
    "checksum" : {
      "sha1" : "d9f877a857a7e9928eac04d09a59f25967624155",
      "sha256" : "0a8cac771ca188eacc57e2c96c31f5611925c5ecedccb16b8c236d6c0d325112",
      "sha512" : "e4694dce7937894bba2b5b4ad33dfd1893d8494fcd4eaac7ca3392c49ddba2ef0497b2a0307c81580611a785a0244a6003b4bae0d978c8d1544f4f691e6c7130",
      "md5" : "ad60407c083b4ecc372614b8fcd9f305"
    }
  }, {
    "downloadUrl" : "http://localhost:8081/repository/maven-releases/file47/file47/1.0.0/file47-1.0.0.pom.sha1",
    "path" : "file47/file47/1.0.0/file47-1.0.0.pom.sha1",
    "id" : "bWF2ZW4tcmVsZWFzZXM6ODUxMzU2NTJhOTc4YmU5YTQyNGYxMzQ3NDYwYmM0ODI",
    "repository" : "maven-releases",
    "format" : "maven2",
    "checksum" : {
      "sha1" : "0126c00d391aebeddcd55ef8d1bd99c26a0fdb57",
      "md5" : "d579a682f677fa6b22e054c8a69494d2"
    }
  } ],
  "continuationToken" : "35303a6235633862633138616131326331613030356565393061336664653966613733"
} 

Attempts

gojsonq.New().JSONString(json).From("items").Find("downloadUrl")

and

name := gojsonq.New().JSONString(json).Find("items.downloadUrl")

return

nil

while jq .items[].downloadUrl return all downloadUrl

The following jq statement returns all downloadUrls:

curl -X GET http://localhost:8081/service/rest/v1/search/assets?repository=maven-releases | jq .items[].downloadUrl

returns:

"http://localhost:8081/repository/maven-releases/file2/file2/1.0.0/file2-1.0.0.jar"
"http://localhost:8081/repository/maven-releases/file2/file2/1.0.0/file2-1.0.0.jar.md5"
"http://localhost:8081/repository/maven-releases/file2/file2/1.0.0/file2-1.0.0.jar.sha1"
"http://localhost:8081/repository/maven-releases/file2/file2/1.0.0/file2-1.0.0.pom"
"http://localhost:8081/repository/maven-releases/file2/file2/1.0.0/file2-1.0.0.pom.md5"
"http://localhost:8081/repository/maven-releases/file2/file2/1.0.0/file2-1.0.0.pom.sha1"
"http://localhost:8081/repository/maven-releases/file10/file10/1.0.0/file10-1.0.0.jar"
"http://localhost:8081/repository/maven-releases/file10/file10/1.0.0/file10-1.0.0.jar.md5"
"http://localhost:8081/repository/maven-releases/file10/file10/1.0.0/file10-1.0.0.jar.sha1"
"http://localhost:8081/repository/maven-releases/file10/file10/1.0.0/file10-1.0.0.pom"
"http://localhost:8081/repository/maven-releases/file10/file10/1.0.0/file10-1.0.0.pom.md5"
"http://localhost:8081/repository/maven-releases/file10/file10/1.0.0/file10-1.0.0.pom.sha1"
"http://localhost:8081/repository/maven-releases/file11/file11/1.0.0/file11-1.0.0.jar"
"http://localhost:8081/repository/maven-releases/file11/file11/1.0.0/file11-1.0.0.jar.md5"
"http://localhost:8081/repository/maven-releases/file11/file11/1.0.0/file11-1.0.0.jar.sha1"
"http://localhost:8081/repository/maven-releases/file11/file11/1.0.0/file11-1.0.0.pom"
"http://localhost:8081/repository/maven-releases/file11/file11/1.0.0/file11-1.0.0.pom.md5"
"http://localhost:8081/repository/maven-releases/file11/file11/1.0.0/file11-1.0.0.pom.sha1"
"http://localhost:8081/repository/maven-releases/file7/file7/1.0.0/file7-1.0.0.jar"
"http://localhost:8081/repository/maven-releases/file7/file7/1.0.0/file7-1.0.0.jar.md5"
"http://localhost:8081/repository/maven-releases/file7/file7/1.0.0/file7-1.0.0.jar.sha1"
"http://localhost:8081/repository/maven-releases/file7/file7/1.0.0/file7-1.0.0.pom"
"http://localhost:8081/repository/maven-releases/file7/file7/1.0.0/file7-1.0.0.pom.md5"
"http://localhost:8081/repository/maven-releases/file7/file7/1.0.0/file7-1.0.0.pom.sha1"
"http://localhost:8081/repository/maven-releases/file8/file8/1.0.0/file8-1.0.0.jar"
"http://localhost:8081/repository/maven-releases/file8/file8/1.0.0/file8-1.0.0.jar.md5"
"http://localhost:8081/repository/maven-releases/file8/file8/1.0.0/file8-1.0.0.jar.sha1"
"http://localhost:8081/repository/maven-releases/file8/file8/1.0.0/file8-1.0.0.pom"
"http://localhost:8081/repository/maven-releases/file8/file8/1.0.0/file8-1.0.0.pom.md5"
"http://localhost:8081/repository/maven-releases/file8/file8/1.0.0/file8-1.0.0.pom.sha1"
"http://localhost:8081/repository/maven-releases/file4/file4/1.0.0/file4-1.0.0.jar"
"http://localhost:8081/repository/maven-releases/file4/file4/1.0.0/file4-1.0.0.jar.md5"

Why is it nil?

package main

import (
	"fmt"
	"github.com/thedevsaddam/gojsonq"
)

func main()  {
	const json  = `{"name": "BeanWei", "Age": "23", "foucus": ["python", "golang", "Kotlin"]}`
	dom := gojsonq.New().JSONString(json)
	name := dom.Find("name")
	age := dom.Find("Age")
	ist := dom.Find("foucus.[1]")
	fmt.Println(name, age, ist)
}

>>>output: BeanWei <nil> <nil>

this way aren't supported?

FromInterface undefined

Getting this error when trying to use the FromInterface method:
gojsonq.New().FromInterface undefined (type *gojsonq.JSONQ has no field or method FromInterface)

Versions newer than v2.3.0 unavailable via go mod

I'm unable to download versions newer than v2.3.0 seem to be unavailable. Per https://pkg.go.dev/github.com/thedevsaddam/gojsonq/?tab=doc v2.3.0 is the latest version.

Please suggest what I should do to get get v2.5.0.

Pluck should return []interface{}

As stated in its description 'Pluck build(s) an array of values form a property of a list of objects' it returns an array (slice) so why not making it clear in the function signature as well.

Edit: This should not lead to incompatibilities as []interface{} is a subset of interface{}. This follows the rule of returning the most specific type possible.
If you still concerned a method Plucks could be added as incremental additions is covered by the Go compatibility guarantee.

The change requires literally adding two characters.

No need to convert y in neq function

// eq checks whether x, y are deeply eq
func eq(x, y interface{}) (bool, error) {
	// if the y value is numeric (int/int8-int64/float32/float64) then convert to float64
	if fv, ok := toFloat64(y); ok {
		y = fv
	}
	return reflect.DeepEqual(x, y), nil
}

// neq checks whether x, y are deeply not equal
func neq(x, y interface{}) (bool, error) {
	// if the y value is numeric (int/int8-int64/float32/float64) then convert to float64
	if fv, ok := toFloat64(y); ok {
		y = fv
	}
	b, err := eq(x, y)
	return !b, err
}

Since y is converted in eq function, there's no need to be converted in neq function, right?

---

another question: why not let notIn function reuse in function?

Package has no way to clear errors

Greetings,

When using the package to fetch various fields (user supplied paths) in sequence, I noticed that after the first error, ALL Find()'s fail with the same error. Looking through the code, it seems j.errors gets appended, but never reset.

Should it not be part of Reset()? Or if that is not appropriate, have it as a side effect of the Error() or Errors() function?

Why interfaces?

Thanks for sharing this project. One question. Why do you do this:

// strStrictContains checks if x contains y
// This is case sensitive search
func strStrictContains(x, y interface{}) (bool, error) {
	xv, okX := x.(string)
	if !okX {
		return false, fmt.Errorf("%v must be string", x)
	}
	yv, okY := y.(string)
	if !okY {
		return false, fmt.Errorf("%v must be string", y)
	}
	return strings.Contains(xv, yv), nil
}

...and not this:

// strStrictContains checks if x contains y
// This is case sensitive search
func strStrictContains(x, y string) (bool) {
	return strings.Contains(xv, yv)
}

...or in this case, simply strings.Contains(xv, yv) without a wrapper.

In this example, you are only allowing string inputs anyway, so why not use strongly typed strings as input params instead of interface{}?

json reflect type err

{
	"employees": [{
			"firstName": "John",
			"lastName": "Doe"
		},
		{
			"firstName": "Anna",
			"lastName": "Smith"
		},
		{
			"firstName": "Peter",
			"lastName": "Jones"
		}
	],
	"name": {
		"first": "Tom",
		"last": "Hanks"
	},
	"age": 61.9,
	"high": 175,
	"iswoman": true,
	"sgggg": "ajsdfjlj",
	"nick": {
		"first": "Tom",
		"last": "Hanks"
	}
}

JJ := gojsonq.New().JSONString(n).Get()
if reflect.ValueOf(v).Kind() == reflect.Float64 {
				//jsonstruct[k] = v
				//unmarshal(v)
				base = true
				jsonstruct[k] = v
			}

get this high type is float64

only support map,slice,float64,string,bool ?

Find where subquery

I h've a problem when try so search condition with subquery.

SELECT * FROM coupons WHERE ((start_date >= '2018-12-01 00:00:00' AND end_date <= '2018-12-01 23:59:59') OR(start_date >= '2018-12-10 00:00:00' AND end_date <= '2018-12-10 23:59:59'))

If i want to search with example query like this . Can you please help how to filter condition like this?