keenerd / jshon Goto Github PK

Describe the bug
Given the input, which is apparently is wrong, I would have expected an error message.

To Reproduce

$ echo '{ "atribute1": { "attribute2": "name1", "attribute3": { "attribute2": "name2" }, "attribute3": { "attribute2": "name3" }, "attribute3": { "attribute2": "name4" } } }' | jshon '.'
{
  "atribute1": {
    "attribute2": "name1",
    "attribute3": {
      "attribute2": "name4"
    }
  }
}

Expected behavior

$ cat expected_output.json 
{ 
 "atribute1": {
  "attribute2": "name1",
  "attribute3": {
   "attribute2": "name2"
  },
  "attribute3": {
   "attribute2": "name3"
  },
  "attribute3": {
   "attribute2": "name4"
  }
 }
}
$

Environment (please complete the following information):

• OS and Version: Linux Ubuntu 18.04.2
• jshon version:

$ jshon --version
20130901

Additional context
I suspect this to be a language problem, as the competition (jq) does the same. An error message would be nice, though.

I get the following error since 4150532 when doing cat FILE | jshon for large files.

json read error: line 1 column 65536: premature end of input

Jansson: 2.5
OS X: 10.9

This works:

$ curl -s 'http://public-api.wordpress.com/rest/v1/sites/en.blog.wordpress.com/posts/' | jshon -t
object

This works:

$ curl -s -o test.json 'http://public-api.wordpress.com/rest/v1/sites/en.blog.wordpress.com/posts/'
$ jshon -F test.json -t
object

This does not work:

$ curl -s -o test.json 'http://public-api.wordpress.com/rest/v1/sites/en.blog.wordpress.com/posts/'
$ cat test.json | jshon -t
json read error: line 1 column 65536: premature end of input

Actually, it does work, but only very occasionally: about one time out of ten.

Note that, at the moment, that URL returns 222127 bytes:

$ cat test.json | wc -c
  222127

Which is clearly not 65536, a very suspicious 2^16 :)

gcc -std=c99 -Wall -pedantic -Wextra -Werror -march=nocona -msahf -mtune=generic -O2 -pipe   -c -o jshon.o jshon.c
In file included from jshon.c:2:
jshon.c: In function ‘remove_jsonp_callback’:
jshon.c:425:44: error: array subscript has type ‘char’ [-Werror=char-subscripts]
  425 |     while (first < last && JSON_IDENTIFIER(*first))
      |                                            ^~~~~~
jshon.c:425:28: note: in expansion of macro ‘JSON_IDENTIFIER’
  425 |     while (first < last && JSON_IDENTIFIER(*first))
      |                            ^~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make: *** [<builtin>: jshon.o] Error 1
==> ERROR: A failure occurred in build().

build if i remove the -Werror flag, i'm not sure why that's part of the makefile tbh

The -S option sorts keys of course, and works well.

However, the order of the items in the following, remains unchanged with -S :

"flintstones": [ "bambam", "dino", "pebbles", "fred" ]

It'd be great if it were bambam, dino, fred, pebbles.

While jshon could benefit from another command line option to sort such unkeyed string arrays, there's also no reason why -S couldn't cover that permutation too.

Thanks :)

Super cool, would be a way to handle a list of files. Relates to request #7 of course. :)

It would be very helpful for me in updating the jshon package in Debian if the repo had tagged releases. Github allows downloading tarballs of tagged releases. With a tarball, I can use existing Debian tools like uscan for updating the package.

Hey, great job on this.

I'm trying to create what I believe would be a pretty simple json with nested objects very similar to Youtube's "add to playlist":

{
'snippet': {
'playlistId': '{PLAYLIST_ID}',
'resourceId': {
'kind': 'youtube#video',
'videoId': '{VIDEO_ID}'
}
'position': 0
}
}

However, I for the life of me can't get "playlistId" to be inside of "snippet". Is this possible with jshon? What am I missing?

Thanks!

I created a script to parse availability reports from Nagios / Icinga but under some circumstances due to wrong downtime calculation, a huge integer appears as value. jshon then fails to parse the entire json output with an error "too big integer".

# curl -s -u user:pw "http://icinga.example.com/cgi-bin/icinga2-classicui/avail.cgi?show_log_entries=&host=internet&service=HTTP+mysite.example.com&assumeinitialstates=yes&assumestateretention=yes&assumestatesduringnotrunning=yes&includesoftstates=no&initialassumedhoststate=0&initialassumedservicestate=0&timeperiod=lastmonth&backtrack=8&jsonoutput" | jshon -k
json read error: line 31 column 1350: too big integer near '18446744073709548794'

As you can see from the json output below, the problem comes from the value of field time_critical_unscheduled inside the avail/service_availability/services array.

{ "cgi_json_version": "1.11.0",
"icinga_status": {
"status_data_age": 15,
"status_update_interval": 10,
"reading_status_data_ok": true,
"program_version": "r2.6.1-1",
"icinga_pid": 18010,
"timezone": "CEST",
"date_format": "us",
"program_start": 1522738805,
"total_running_time": "0d 0h 13m 32s",
"last_external_command_check": 0,
"last_log_file_rotation": 0,
"notifications_enabled": true,
"disable_notifications_expire_time": 0,
"service_checks_being_executed": true,
"passive_service_checks_being_accepted": true,
"host_checks_being_executed": true,
"passive_host_checks_being_accepted": true,
"obsessing_over_services": false,
"obsessing_over_hosts": false,
"check_service_freshness": true,
"check_host_freshness": true,
"event_handlers_enabled": true,
"flap_detection_enabled": true,
"performance_data_being_processed": true
},
"avail": {
"service_availability": {
"services": [
{ "host_name": "internet",  "host_display_name": "internet",  "service_description": "HTTP mysite.example.com",  "service_display_name": "HTTP mysite.example.com",  "time_ok_scheduled": 134221,  "percent_time_ok_scheduled": 5.018,  "percent_known_time_ok_scheduled": 5.081,  "time_ok_unscheduled": 2504799,  "percent_time_ok_unscheduled": 93.644,  "percent_known_time_ok_unscheduled": 94.815,  "total_time_ok": 2639020,  "percent_total_time_ok": 98.662,  "percent_known_time_ok": 99.895,  "time_warning_scheduled": 0,  "percent_time_warning_scheduled": 0.000,  "percent_known_time_warning_scheduled": 0.000,  "time_warning_unscheduled": 0,  "percent_time_warning_unscheduled": 0.000,  "percent_known_time_warning_unscheduled": 0.000,  "total_time_warning": 0,  "percent_total_time_warning": 0.000,  "percent_known_time_warning": 0.000,  "time_unknown_scheduled": 0,  "percent_time_unknown_scheduled": 0.000,  "percent_known_time_unknown_scheduled": 0.000,  "time_unknown_unscheduled": 0,  "percent_time_unknown_unscheduled": 0.000,  "percent_known_time_unknown_unscheduled": 0.000,  "total_time_unknown": 0,  "percent_total_time_unknown": 0.000,  "percent_known_time_unknown": 0.000,  "time_critical_scheduled": 5590,  "percent_time_critical_scheduled": 0.209,  "percent_known_time_critical_scheduled": 0.212,  "time_critical_unscheduled": 18446744073709548794,  "percent_time_critical_unscheduled": -0.106,  "percent_known_time_critical_unscheduled": -0.107,  "total_time_critical": 2768,  "percent_total_time_critical": 0.103,  "percent_known_time_critical": 0.105,  "time_undetermined_not_running": 0,  "percent_time_undetermined_not_running": 0.000,  "time_undetermined_no_data": 33011,  "percent_time_undetermined_no_data": 1.234,  "total_time_undetermined": 33011,  "percent_total_time_undetermined": 1.234,  "total_time_all": 2674799,  "percent_total_time_all": 100.000,  "percent_known_time_all": 100.000,
"log_entries": [
{ "start_time_string": "03-01-2018 00:59:04",  "start_time_timestamp": 1519862344,  "end_time_string": "03-01-2018 02:32:40",  "end_time_timestamp": 1519867960,  "duration_string": "0d 1h 33m 36s",  "duration_timestamp": 5616,  "entry_type": "SERVICE DOWNTIME START",  "state_type": "",  "state_information": "Start of scheduled downtime"},
{ "start_time_string": "03-29-2018 17:00:00",  "start_time_timestamp": 1522335600,  "end_time_string": "03-31-2018 23:59:59",  "end_time_timestamp": 1522533599,  "duration_string": "2d 6h 59m 59s+",  "duration_timestamp": 197999,  "entry_type": "SERVICE OK",  "state_type": "HARD",  "state_information": "HTTP OK: HTTP/1.1 200 OK - 8697 bytes in 0.304 second response time"}
]

}
]
}
}
}

I came across similar bug reports across other json projects (tc39/proposal-bigint#24, akheron/jansson#8), so this seems to be a json parsing problem in general. Is there a way to fix this in jshon or to simply ignore such a value, e.g. by returning 'int_too_big' ?

That way, Github creates a zip/tar file for easy downloading. Useful for packaging.

These flags allow you to call json_dumps() and json_loads() on any value (not only arrays and objects), and to escape slashes when encoding. They have been added in Jansson 2.1, 2.3 and 2.4 respectively.

I am using jshon to generate (as opposed to convert) JSON at the moment, and am presently having to do things like:

echo {} | jshon ...

because otherwise jshon borks when there is no input. Perhaps I am just unaware of how to remove the need for input, but if not, then it would be nice to add a flag which just tells jshon to not read any input, and to instead start with an empty JSON object (i.e. "{}").

Would be great having an option to use four spaces instead of one. This would improve readability.

Other request like this are:
https://en.wikipedia.org/wiki/JSON (Four specific characters are considered whitespace for this purpose)
https://docs.openstack.org/doc-contrib-guide/json-conv.html
dotnet/runtime#1174

Thx!

Hi,

Could you please help me create a nested object with JSHON? For example, how would I do to create the following object:

{
 "device": "xx-xx",
 "data" : {
    "action": "shutdown",
    "count": 4
  }
}

Thanks!

This is the JSON snippet I'm working on (as returned by aws cli):

{
  "Outputs": [
   {
    "OutputValue": "value1",
    "OutputKey": "key1"
   },
   {
    "OutputValue": "value2",
    "OutputKey": "key2"
   }
  ]
}

I want jshon to get the OutputValue where the OutputKey is 'key2'. 'value2' that is. Currently, I don't see a way to do this, so I'd like to suggest an optional value to the -e parameter:

jshon -e Outputs -a -e OutputKey=key2 -p -e OutputValue -u

In man for -d we have Negative array indexes will wrap around.
Trying $ echo '[1,2,3]' | jshon -d -1

[
 1,
 2,
 3
]

Did I misinterpreted man?

Floats are rounded very poorly in jshon.

E.g., echo [0.3] | jshon outputs:

[
 0.29999999999999999
]

grep -i springs to mind.

It's not a biggie as one can use pipes and moretools' sponge command

Is it possible to extract values from one object and insert them into another?

Source:

{
"packid": "1234",
"clientid": "4321",
"activation_date": "0",
"lastrenew": "1488355200",
"renewdate": "1491030000",
"balance": "0.00",
"parentpack": "0",
"servername": "example.com",
"ip_address": "127.0.0.1",
"cp_host": "0",
"active": "1",
"quantity": "1.00",
"created": "1487813969",
}

Expected result:

{"clientid":4321,"packid":1234}

The Mac's dominant package manager, Homebrew, grabs the single .tar.gz you have hosted :

https://github.com/mxcl/homebrew/blob/207f18efe1f83acbc79527350759fb033b610cd6/Library/Formula/jshon.rb

If you put a version number in the file name it'd make for a greater stability for those of us beginning to use jshon in solutions :)

Thx,

• Paul

Just started to look into this.

[root@nikita24 jshon]# make
gcc -c -std=c99 -Wall -pedantic -Wextra -Werror jshon.c
jshon.c:8:21: error: jansson.h: No such file or directory
cc1: warnings being treated as errors
jshon.c:72: warning: implicit declaration of function 'JSON_INDENT'
jshon.c:72: error: initializer element is not constant
jshon.c:84: error: expected '=', ',', ';', 'asm' or 'attribute' before '' token
jshon.c:85: error: expected '=', ',', ';', 'asm' or 'attribute' before '' token
jshon.c:112: error: expected ')' before '' token
jshon.c:124: error: expected '=', ',', ';', 'asm' or 'attribute' before '' token
jshon.c:138: error: expected '=', ',', ';', 'asm' or 'attribute' before '*' token
jshon.c:148: error: expected specifier-qualifier-list before 'json_t'
jshon.c: In function 'MAPPUSH':
jshon.c:169: error: 'mapping' has no member named 'stk'
jshon.c:169: warning: implicit declaration of function 'stack_safe_peek'

wget http://www.reddit.com/r/test/comments/pb42s/test_title.json
cat test_title.json | jshon -d body_html

... chokes with ....

parse error: illegal index on arg 2, "body_html"

%zd is unknown on MingW. I had to replace all of its occurences by %Iu (unsigned) and %Id (signed).

Hi,

the manpage says:

If you care about extremely short one liners, arguments can be condensed when it does not cause ambiguity.

But condensed arguments are not handled the same way.

Example:

$ echo '{"a":1,"b":[true,false,null,"none"],"c":{"d":4,"e":5}}' > sample.json
$ sample.json ./jshon -a -t
number
array
object

$ <sample.json jshon -at
number
bool
parse error: type not mappable

Regards.

Hi there,

Do you have an example of creating some JSON from some shell output?

For example I would like to replace json_encode with jshon here:
https://github.com/kaihendry/Greptweet/blob/master/grep.php

Many thanks!

Using newline delimiters is unsafe when content can legitimately contain newlines.

Streams with NUL delimiters can be easily and safely read from bash (IFS= read -r -d ''), and unambiguously separates syntax from data.

Of course, unixland is split between --version and -version.

jshon has no self-reported version string at all though :-(

foo:bar paul$ jshon
warning: nothing to read on stdin
foo:bar paul$ jshon -v
jshon: illegal option -- v
Valid: -[P|S|Q|V|C] -[t|l|k|u|p|a] -[s|n] value -[e|i|d] index
foo:bar paul$ jshon -version
jshon: illegal option -- v
Valid: -[P|S|Q|V|C] -[t|l|k|u|p|a] -[s|n] value -[e|i|d] index
foo:bar paul$ jshon --version
jshon: illegal option -- -
Valid: -[P|S|Q|V|C] -[t|l|k|u|p|a] -[s|n] value -[e|i|d] index
foo:bar paul$

Hello ,
I have this error while parsing an array of more than 100 entries.

If I use:
command | jshon -e invoices -e 100
I have the error:
parse error: type 'array' index out of bounds (arg 4)

If I use
command | jshon -e invoices -e 99
Everything is ok

Is there a max size of array that jshon can manage?

Thanks

Hi,

I'm try to build a RPM for jshon.

The provided .spec reference a patch jshon-makefile-enable-install.patch which does not exists in the repo.

May you provide the file, or remove the reference in the .spec file?

bests

Not really sure what's going on here. I built jshon from this release and get this result when testing it:

$ jshon -e foo <<< '{"foo":"ay4d748edadf7"}'
json read error: line 2 column 1: end of file expected near '_'

however if the string is just one character shorter, it works fine:

$ jshon -e foo <<< '{"foo":"ay4d748edadf"}'
"ay4d748edadf"

another experiment yields a different error:

$ echo -n '{"foo":"ay4d748df7"}' | jshon -e foo
json read error: line 1 column 20: unable to decode byte 0xff

Any idea what might be going wrong?

It would be nice to have a new release that includes the ZSH completions for linux distribution packagers

when special chars are on the input, jshon complains telling:

json read error: line 1 column 12627: unable to decode byte 0xd1

(line, column and specific code vary of course).

Hi...

I just expect -k on ["a","b","c"] to output:
0
1
2
instead of error... it will make loops in bash easy... now i have to get "3" using -l, subtract 1 to get 2 and use "seq 0 2" to drive for loop properly...

Thx

I have a file of size 232830522671 bytes, that is 232GB.

When I try jshon -d sometag < large_file.json, it fails with

error: failed to allocate 232830522672 bytes
error: failed to read input

When i try cat large_file.json | jshon -d something cat fails writing to the pipe:

cat: write error: Broken pipe

{
"bar": "hello",
"baz": 2,
"blort":
{
"aaa": "hello",
"bbb": 2
}
}

jshon -d aaa
(does not remove aaa)

According to https://tools.ietf.org/html/rfc7159.html (which supersedes https://tools.ietf.org/html/rfc4627.html) the root JSON entity can be not only an array or an object, but also a value(string, int etc.). For example, "hw" is a valid root JSON entity according to new rfc. So far, I've not found a way to decode such JSON objects with jshon. For example, echo '"hw"' | jshon -u spits out this: json read error: line 1 column 4: '[' or '{' expected near '"hw"'.

Does this tool support rfc7159?

Unless I'm missing something, I see no easy way to feed in lines of JSON objects, one per line. This format is used in some programs, like MongoDB, and it would be awesome if this utility could be extended to provide that functionality. I am willing to donate a couple cases of beer to your cause (or at least an equivalent sum of cash!).

Here's a sample of the input, notice the lack of comma at the end, each line is a separate object:

    {"foo" : "bar", "apple" : "tree", "foods" : [ "chips", "cake" ]}
    {"foo" : "bar", "apple" : "tree", "foods" : [ "chips", "cake" ]}
    {"foo" : "bar", "apple" : "tree", "foods" : [ "chips", "cake" ]}
    {"foo" : "bar", "apple" : "tree", "foods" : [ "chips", "cake" ]}
    {"foo" : "bar", "apple" : "tree", "foods" : [ "chips", "cake" ]}

please help. i am in love with this library, yet i am unable to marry it

Didn't see a read me - ran make and got this :

$ make
cc -std=c99 -Wall -pedantic -Wextra -Werror    -c -o jshon.o jshon.c
jshon.c:9:21: fatal error: jansson.h: No such file or directory
 #include <jansson.h>
                     ^
compilation terminated.
make: *** [jshon.o] Error 1

Please fix working with floating point numbers... maybe use long instead of float? modify printf formating string? Really best thing that you can do is to internally store numbers as strings unless you really need to compute with them. (AFAIK there's no support for adding ar multiplying numbers, etc... jshon is just expected to take nuber on one side and print exact same number on the other end... so just check if it's number and if it is, then just store it as string).

Here's sample problem:

jshon -e e <<<'{"e":200.123456789}'
200.123457

I've also managed to get something like 22.37 translated into 22.370001 or 22.369999, but can't find the exact number which caused the problem right now...

THX for jshon. I've been planning to create something simillar till i find jshon...