dominictarr / json.sh

a pipeable JSON parser written in Bash

JSON.sh

yo, so it's a json parser written in shell, compatible with ash, bash, dash and zsh

pipe json to it, and it traverses the json objects and prints out the path to the current object (as a JSON array) and then the object, without whitespace.

$ json_parse < package.json
["name"]  "JSON.sh"
["version"]  "0.0.0"
["description"]  ""
["homepage"]  "http://github.com/dominictarr/JSON.sh"
["repository","type"]  "git"
["repository","url"]  "https://github.com/dominictarr/JSON.sh.git"
["repository"]  {"type":"git","url":"https://github.com/dominictarr/JSON.sh.git"}
["bin","json_parse"]  "./JSON.sh"
["bin"]  {"json_parse":"./JSON.sh"}
["dependencies"]  {}
#  ... etc

a more complex example:

curl registry.npmjs.org/express | ./JSON.sh | egrep '\["versions","[^"]*"\]'
... try it and see

Options

-b

Brief output. Combines 'Leaf only' and 'Prune empty' options.

-l

Leaf only. Only show leaf nodes, which stops data duplication.

-p

Prune empty. Exclude fields with empty values.

-n

No-head. Don't show nodes that have no path. Normally these output a leading '[]', which you can't use in a bash array.

-s

Remove escaping of the solidus symbol (stright slash).

-h

Show help text.

Cool Links

• step-/JSON.awk JSON.sh ported to awk
• kristopolous/TickTick Object Oriented BASH
• archan937/jsonv.sh
• mclarkson/JSONPath.sh JSONPath based on JSON.sh

Installation

Install with npm, pip or from AUR on Archlinux:

• npm install -g JSON.sh
• pip install git+https://github.com/dominictarr/JSON.sh#egg=JSON.sh
• yaourt -Sy json-sh (json-sh on aur thanks to kremlin-)

License

This software is available under the following licenses:

• MIT
• Apache 2

json.sh's Issues

Forward slash escaping

Hi all,

Jettison escapes forward slashes by default. JSON does not mandate forward slashes to be escaped. But escaping forward slahes is perfectly valid.
Using json.sh, when I try to parse JSON payloads that contain escaped forward slahes (/) I encounter issues.
Does json.sh handle escaped forward slashes?
Payload example

{
"ressources": {
"statut": "http://localhost:8080/mapb-abl-dua/batch/179/statut" }
}

Any help would be much appreciated.

error

Parse failure triggering error: EXPECTED EOF GOT {

I'm trying to use JSON.sh to parse the JSON here:
http://pastebin.com/F2sbcjPX

and its dying without parsing the entire thing:

./JSON.sh < /tmp/disk_usage.linux.json
["cuda-farm-ljf0.nvidia.com","/dev","available"] "1530608"
["cuda-farm-ljf0.nvidia.com","/dev","1K-blocks"] "1530608"
["cuda-farm-ljf0.nvidia.com","/dev","used"] "0"
["cuda-farm-ljf0.nvidia.com","/dev","capacity"] "0%"
["cuda-farm-ljf0.nvidia.com","/dev","filesystem"] "devtmpfs"
["cuda-farm-ljf0.nvidia.com","/dev"] {"available":"1530608","1K-blocks":"1530608","used":"0","capacity":"0%","filesystem":"devtmpfs"}
["cuda-farm-ljf0.nvidia.com","/boot","available"] "61894"
["cuda-farm-ljf0.nvidia.com","/boot","1K-blocks"] "194241"
["cuda-farm-ljf0.nvidia.com","/boot","used"] "122107"
["cuda-farm-ljf0.nvidia.com","/boot","capacity"] "67%"
["cuda-farm-ljf0.nvidia.com","/boot","filesystem"] "/dev/sda1"
["cuda-farm-ljf0.nvidia.com","/boot"] {"available":"61894","1K-blocks":"194241","used":"122107","capacity":"67%","filesystem":"/dev/sda1"}
["cuda-farm-ljf0.nvidia.com","/sys/fs/cgroup","available"] "1541960"
["cuda-farm-ljf0.nvidia.com","/sys/fs/cgroup","1K-blocks"] "1541960"
["cuda-farm-ljf0.nvidia.com","/sys/fs/cgroup","used"] "0"
["cuda-farm-ljf0.nvidia.com","/sys/fs/cgroup","capacity"] "0%"
["cuda-farm-ljf0.nvidia.com","/sys/fs/cgroup","filesystem"] "tmpfs"
["cuda-farm-ljf0.nvidia.com","/sys/fs/cgroup"] {"available":"1541960","1K-blocks":"1541960","used":"0","capacity":"0%","filesystem":"tmpfs"}
["cuda-farm-ljf0.nvidia.com","/","available"] "77139440"
["cuda-farm-ljf0.nvidia.com","/","1K-blocks"] "82104148"
["cuda-farm-ljf0.nvidia.com","/","used"] "4964708"
["cuda-farm-ljf0.nvidia.com","/","capacity"] "7%"
["cuda-farm-ljf0.nvidia.com","/","filesystem"] "/dev/sda2"
["cuda-farm-ljf0.nvidia.com","/"] {"available":"77139440","1K-blocks":"82104148","used":"4964708","capacity":"7%","filesystem":"/dev/sda2"}
["cuda-farm-ljf0.nvidia.com","/run","available"] "1538312"
["cuda-farm-ljf0.nvidia.com","/run","1K-blocks"] "1541960"
["cuda-farm-ljf0.nvidia.com","/run","used"] "3648"
["cuda-farm-ljf0.nvidia.com","/run","capacity"] "1%"
["cuda-farm-ljf0.nvidia.com","/run","filesystem"] "tmpfs"
["cuda-farm-ljf0.nvidia.com","/run"] {"available":"1538312","1K-blocks":"1541960","used":"3648","capacity":"1%","filesystem":"tmpfs"}
["cuda-farm-ljf0.nvidia.com","/dev/shm","available"] "1541960"
["cuda-farm-ljf0.nvidia.com","/dev/shm","1K-blocks"] "1541960"
["cuda-farm-ljf0.nvidia.com","/dev/shm","used"] "0"
["cuda-farm-ljf0.nvidia.com","/dev/shm","capacity"] "0%"
["cuda-farm-ljf0.nvidia.com","/dev/shm","filesystem"] "tmpfs"
["cuda-farm-ljf0.nvidia.com","/dev/shm"] {"available":"1541960","1K-blocks":"1541960","used":"0","capacity":"0%","filesystem":"tmpfs"}
["cuda-farm-ljf0.nvidia.com","/mnt/sdb1","available"] "8609104"
["cuda-farm-ljf0.nvidia.com","/mnt/sdb1","1K-blocks"] "15717884"
["cuda-farm-ljf0.nvidia.com","/mnt/sdb1","used"] "7108780"
["cuda-farm-ljf0.nvidia.com","/mnt/sdb1","capacity"] "46%"
["cuda-farm-ljf0.nvidia.com","/mnt/sdb1","filesystem"] "/dev/drbd0"
["cuda-farm-ljf0.nvidia.com","/mnt/sdb1"] {"available":"8609104","1K-blocks":"15717884","used":"7108780","capacity":"46%","filesystem":"/dev/drbd0"}
["cuda-farm-ljf0.nvidia.com","/media","available"] "1541960"
["cuda-farm-ljf0.nvidia.com","/media","1K-blocks"] "1541960"
["cuda-farm-ljf0.nvidia.com","/media","used"] "0"
["cuda-farm-ljf0.nvidia.com","/media","capacity"] "0%"
["cuda-farm-ljf0.nvidia.com","/media","filesystem"] "tmpfs"
["cuda-farm-ljf0.nvidia.com","/media"] {"available":"1541960","1K-blocks":"1541960","used":"0","capacity":"0%","filesystem":"tmpfs"}
["cuda-farm-ljf0.nvidia.com"] {"/dev":{"available":"1530608","1K-blocks":"1530608","used":"0","capacity":"0%","filesystem":"devtmpfs"},"/boot":{"available":"61894","1K-blocks":"194241","used":"122107","capacity":"67%","filesystem":"/dev/sda1"},"/sys/fs/cgroup":{"available":"1541960","1K-blocks":"1541960","used":"0","capacity":"0%","filesystem":"tmpfs"},"/":{"available":"77139440","1K-blocks":"82104148","used":"4964708","capacity":"7%","filesystem":"/dev/sda2"},"/run":{"available":"1538312","1K-blocks":"1541960","used":"3648","capacity":"1%","filesystem":"tmpfs"},"/dev/shm":{"available":"1541960","1K-blocks":"1541960","used":"0","capacity":"0%","filesystem":"tmpfs"},"/mnt/sdb1":{"available":"8609104","1K-blocks":"15717884","used":"7108780","capacity":"46%","filesystem":"/dev/drbd0"},"/media":{"available":"1541960","1K-blocks":"1541960","used":"0","capacity":"0%","filesystem":"tmpfs"}}
[] {"cuda-farm-ljf0.nvidia.com":{"/dev":{"available":"1530608","1K-blocks":"1530608","used":"0","capacity":"0%","filesystem":"devtmpfs"},"/boot":{"available":"61894","1K-blocks":"194241","used":"122107","capacity":"67%","filesystem":"/dev/sda1"},"/sys/fs/cgroup":{"available":"1541960","1K-blocks":"1541960","used":"0","capacity":"0%","filesystem":"tmpfs"},"/":{"available":"77139440","1K-blocks":"82104148","used":"4964708","capacity":"7%","filesystem":"/dev/sda2"},"/run":{"available":"1538312","1K-blocks":"1541960","used":"3648","capacity":"1%","filesystem":"tmpfs"},"/dev/shm":{"available":"1541960","1K-blocks":"1541960","used":"0","capacity":"0%","filesystem":"tmpfs"},"/mnt/sdb1":{"available":"8609104","1K-blocks":"15717884","used":"7108780","capacity":"46%","filesystem":"/dev/drbd0"},"/media":{"available":"1541960","1K-blocks":"1541960","used":"0","capacity":"0%","filesystem":"tmpfs"}}}
EXPECTED EOF GOT {

Is this a bug?

Does Not Work on CircleCI

Trying to use this with CircleCI scripts and the JSON file parses and blows up with this error

SyntaxError: Unexpected token \ in JSON at position 35<br> &nbsp; &nbsp;at JSON.parse (&lt;anonymous&gt;)<br> &nbsp; &nbsp;at parse (/home/node/data/node_modules/body-parser/lib/types/json.js:89:19)<br> &nbsp; &nbsp;at /home/node/data/node_modules/body-parser/lib/read.js:121:18<br> &nbsp; &nbsp;at invokeCallback (/home/node/data/node_modules/raw-body/index.js:224:16)<br> &nbsp; &nbsp;at done (/home/node/data/node_modules/raw-body/index.js:213:7)<br> &nbsp; &nbsp;at IncomingMessage.onEnd (/home/node/data/node_modules/raw-body/index.js:273:7)<br> &nbsp; &nbsp;at IncomingMessage.emit (events.js:203:15)<br> &nbsp; &nbsp;at IncomingMessage.EventEmitter.emit (domain.js:466:23)<br> &nbsp; &nbsp;at endReadableNT (_stream_readable.js:1145:12)<br> &nbsp; &nbsp;at process._tickCallback (internal/process/next_tick.js:63:19)</pre>

Also it does not work in windows shell same script different error at the call to the JSON parser

DATA=$URL| ./scripts/JSON.sh | egrep '\["upload_url","[^"]*"\]'

get this error

EXPECTED value GOT EOF

return as environment variable

i use JSON.sh as curl 'http://conf.com/conf/config_20150920_test.json' | ./JSON.sh -l -p -b -s and it work greet and it return result like

["storage","mysql","u_db","database"]   "dev"
["storage","mysql","u_db","username"]   "user"
["storage","mysql","u_db","password"]   "123@1"
["storage","mysql","u_db","internal_ip"]    "10.176.192.120"
["storage","mysql","u_db","internal_port"]  3306
["storage","mysql","a","database"]  "a"
["storage","mysql","a","username"]  "user"
["storage","mysql","a","password"]  "122a@1"

but what if add config param to determine how output look like and go with exporting this variables as env variables like

storage_mysql_u_db_internal_port=3306
storage_mysql_a_database=a
storage_mysql_a_username=user
storage_mysql_a_password=122a@1

which make it easy to be evaluated with eval as environment variable.

Parsing owncloud logfile, EXPECTED EOF GOT {

Hi, sorry if it's trivial or obvious, I'm new in JSON and shell script
I try to parse some ownCloud log file and I've got an EXPECTED EOF GOT { error after the end of the first json line.

freezed@machine ~ # script/JSON.sh < oc-log.json
["reqId"]   "mwpZIyDDtQG3gZhKZODF"
["remoteAddr"]  "192.168.1.2"
["app"] "caldav"
["message"] "Exception: {\"Message\":\"HTTP\\\/1.1 404 Node with name 'calendar1_shared_by_user_4' could not be found\",\"Exception\":\"Sabre\\\\DAV\\\\Exception\\\\NotFound\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Tree.php(76): OCA\\\\DAV\\\\CalDAV\\\\CalendarHome->getChild('calendar1_shared_b...')\\n#1 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(903): Sabre\\\\DAV\\\\Tree->getNodeForPath('calendars\\\/user_3...')\\n#2 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(334): Sabre\\\\DAV\\\\Server->getPropertiesForPath('calendars\\\/user_3...', Array, 0)\\n#3 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpPropFind(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#4 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#5 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(459): Sabre\\\\Event\\\\EventEmitter->emit('method:PROPFIND', Array)\\n#6 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(248): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#7 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/caldav.php(81): Sabre\\\\DAV\\\\Server->exec()\\n#8 \\\/var\\\/www\\\/owncloud\\\/remote.php(164): require_once('\\\/var\\\/www\\\/ownclo...')\\n#9 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/CalDAV\\\/CalendarHome.php\",\"Line\":104,\"User\":\"user_3\"}"
["level"]   0
["time"]    "2016-08-10T04:18:21+00:00"
["method"]  "PROPFIND"
["url"] "\/remote.php\/caldav\/calendars\/user_3\/calendar1_shared_by_user_4\/"
["user"]    "user_3"
[]  {"reqId":"mwpZIyDDtQG3gZhKZODF","remoteAddr":"192.168.1.2","app":"caldav","message":"Exception: {\"Message\":\"HTTP\\\/1.1 404 Node with name 'calendar1_shared_by_user_4' could not be found\",\"Exception\":\"Sabre\\\\DAV\\\\Exception\\\\NotFound\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Tree.php(76): OCA\\\\DAV\\\\CalDAV\\\\CalendarHome->getChild('calendar1_shared_b...')\\n#1 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(903): Sabre\\\\DAV\\\\Tree->getNodeForPath('calendars\\\/user_3...')\\n#2 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(334): Sabre\\\\DAV\\\\Server->getPropertiesForPath('calendars\\\/user_3...', Array, 0)\\n#3 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpPropFind(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#4 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#5 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(459): Sabre\\\\Event\\\\EventEmitter->emit('method:PROPFIND', Array)\\n#6 \\\/var\\\/www\\\/owncloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(248): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#7 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/caldav.php(81): Sabre\\\\DAV\\\\Server->exec()\\n#8 \\\/var\\\/www\\\/owncloud\\\/remote.php(164): require_once('\\\/var\\\/www\\\/ownclo...')\\n#9 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/CalDAV\\\/CalendarHome.php\",\"Line\":104,\"User\":\"user_3\"}","level":0,"time":"2016-08-10T04:18:21+00:00","method":"PROPFIND","url":"\/remote.php\/caldav\/calendars\/user_3\/calendar1_shared_by_user_4\/","user":"user_3"}
EXPECTED EOF GOT {
zsh: exit 1     script/JSON.sh < oc-log.json

You can found my sample log file here

Getting "EXPECTED value GOT EOF" for any kind of JSON in Cygwin

I'm using JSON.sh in a Cygwin environment and I'm getting an error when trying to use it for any kind of JSON:

EXPECTED value GOT EOF

• Tried with your example: curl registry.npmjs.org/express | ./JSON.sh | egrep '\["versions","[^"]*"\]'
• Also tried with: echo "{\"foo\":\"bar\"}" | ./JSON.sh

Maybe related to #26? Or maybe related to CRLF vs. LF on Windows vs. Linux (note I'm using Cygwin on Windows)?

How to speed up json parsing ?

Dear contributors

This is more à challenge than an issue...!

I m currently using this code in different project I maintain and for example the main one is a bash library of ~3500 lines : fbx-delta-nba_bash_api.sh .

Json parsing is working fine but when an api call reply a complexe json which is more than 80kb or 100kb, parsing became very slow, too slow for a dynamic usage...

I need a way to throttle by 10 or more those functions (certains json parsing takes several minutes)...

I did try to systematically cache all parsed values and I get a global throttle of x3 on the frontend functions of the library which are using this project as the underlying engine. But it's still not enough for a proper dynamic usage (dynamic question/reply).

Maybe there is something I'm doing wrong, but as I'm not a developer, I don't know what to do so I would ask for help here (and i cannot use 'jq' instead).

Does someone has an idea of how speeding up the exec of these function ?

Is there some small change which permit execution throttling ?

Thanks and kind regards
nbanba

JSON.sh cannot parse strings with the character DEL 0x7F inside

Step to reproduce

$ echo $'["\x7F"]' | ./JSON.sh

Expected behaviour

According to RFC 7159, character DEL 0x7F is perfectly valid unescaped.

JSON.sh should parse the string and print:

[0] "\x7F"
[]  ["\x7F"]

Actual behaviour

JSON.sh cannot parse the string and prints an error:

EXPECTED value GOT "

The issue comes from the fact that JSON.sh uses cntrl to recognize control characters in regular expressions, and cntrl includes DEL 0x7F, while JSON control characters (the one that MUST be escaped) only go from U+0000 to U+001F and don't include U+007F.

Busyboxy / ash support

Hi there,

I am very interested in using JSON.sh. However, I am having a problem when running it on OpenWRT --> a distro that uses Busybox / ash (not bash).

I added this at the top of the file:

!/bin/sh

But when I try to run it, I get this:

./JSON.sh

sh: =: argument expected

Any ideas? Please advise? Thanks!

escaping and sign quote

The option -s does not work :

 echo '{"str_1":"\"ab\"" , "str_2" : "\\b" }' | ./lib/JSON.sh -s

still see the \" and \\

Is there a bug in line 156 to l65

The code:
case "$token" in
'{') parse_object "$jpath" ;;
'[') parse_array "$jpath" ;;
# At this point, the only valid single-character tokens are digits.
''|[!0-9]) throw "EXPECTED value GOT ${token:-EOF}" ;;
*) value=$token
isleaf=1
[ "$value" = '""' ] && isempty=1
;;
esac

I think there is one issues:
'{' should change to '^{$' ,and '[' should change to '^[$'
this is to prevent that user have a string value "abcd[efg" which will mismatch the pattern.

Dev missed out on legendary opportunity...

...to own a project named JSON Bourne

No License

There's no license on this project, so nobody is legally allowed to use it.

JSON reading by using shell script with out jq

Hi Team,

Can anyone have an idea of how to read the JSON field values from a JSON file through Shell script without jq command?

Any help would be appreciated.

Thanks.

Remove all global variables?

Any objection to converting all variables to being local? That would make sourcing the file a lot safer.

At first glance, I think this would mostly amount to passing token to a bunch of functions as an argument, and having them echo it back, though maybe you know of a better way.

I guess the other option is to just declare it as local in parse() and leave it to be set and read by everything else as done today...

Brilliant!!!

That's all. Thanks!

Output values for keys directly

What is the best way to output the corresponding value for any line of the output of this script, given a key?

use grep -E

JSON.sh uses the non-standard egrep command and also non-standard gawk to replace it if it is unavailable. You should use grep -E instead.

New line in json does not work

When I try to use json.sh with json string containing new line, parsing will fail.

Example:
printf '{"a":"b\nc"}' | ./json.sh

Result:
EXPECTED value GOT "

wrong value in brief mode

Consider these two tests

# bash ./JSON.sh
{"k1":[{"k11":11,"k12":12}]}

["k1",0,"k11"]  11
["k1",0,"k12"]  12
["k1",0]    {"k11":11,"k12":12}
["k1"]  [{"k11":11,"k12":12}]
[]  {"k1":[{"k11":11,"k12":12}]}

# bash ./JSON.sh -b
{"k1":[{"k11":11,"k12":12}]}

["k1",0,"k11"]  11
["k1",0,"k12"]  12
["k1",0]    12
["k1"]  12

The last two output lines in brief mode seem wrong, as if the value carried over from the last deeper recursion.
Tested with unmodified source, ash and bash.

As an enhancement request, please I would like an option to omit printing non-leaf nodes, so in the above case output would look like this

["k1",0,"k11"]  11
["k1",0,"k12"]  12

Thank you

more busybox ash support

I just tested it with busybox ash and it runs very smoothly, except for one thing, and two minor issues. Let me start with how I source JSON.sh from an ash script:

#! /bin/busybox ash
alias egrep="/bin/busybox egrep"
export BASH_SOURCE=1
. JSON.sh -b
tokenize | parse

Omit the export and tokenize lines if your script needs to call tokenize and parse separately. Omit -b if you don't want terse output.

Ok, one issue I found is early exit when BRIEF=1 and the shell runs with the -e flag. This is to be expected, as -e makes the script exit at the first error (non-zero exit of a command or function). However, parse_array() and parse_object() always return non-zero when BRIEF=1. So add true as the last command of those functions, it won't hurt when BRIEF=0 and it will make the script continue when BRIEF=1 and ash runs with option -e. Perhaps this point applies to bash and other shells, too. For consistency, I've also added true at the end of parse_value(), although it shouldn't be necessary.

Which brings me to a bit more serious issue. Let me show the code, and a possible fix. This is at the end of function parse_value():

  esac
#ME  ! ([ $BRIEF -eq 1 ] && ([ -z "$jpath" ] || [ $value = '""' ])) \
  ! ([ $BRIEF -eq 1 ] && ([ -z "$jpath" ] || [[ "$value" = '""' ]])) \
      && printf "[%s]\t%s\n" "$jpath" "$value"
  true #ME shouldn't be needed but just in case.
}

I had to replace [ $value = '""' ] with [[ "$value" = '""' ]] because when $value includes white space ash complains about an 'unknown operand'.

The last minor point is about egrep in tokenize(). I had to comment out --color=never simply because busybox egrep doesn't support option --color.

HTH, thanks for this script!

step

Update npm package

Could you update the version of JSON.sh that's published to npm?

JSON.sh does crash when parsing numerous nested structures

Step to reproduce

$ python -c "print('['*100000)" | ./JSON.sh 

Expected behaviour

JSON.sh should either parse the string, or return an error.

Actual behaviour

./JSON.sh: line 206: 40694 Done                    tokenize
40695 Segmentation fault: 11  | parse

JSON.sh does crash because it lacks a nesting limit.

Problem with valid JSON

EVENT='{
"event": {
"timestamp":"2011-06-26T21:26:26.679Z",
"beacon":"urn:example:xyz:web:11234",
"context":"http://www.example.org/",
"object":"http://www.example.org/about/Pages/default.aspx",
"subject":"urn:example:tid:745623",
"action":"urn:examplepage-view",
"platform":"urn:example:plat:web:Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)",
"client_ip":"123.45.67.8",
"request": {
"method":"GET",
"url":"http://www.example.org/about/Pages/default.aspx",
"headers": {
"user-agent":"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)",
"accept-encoding":"gzip",
"accept-language":"en-US,en"
}
},
"response": {
"status":"201"
}
}
}
'

EVENT_PARSED=$(echo $"EVENT" | JSON.sh)
echo $EVENT_PARSED

Throws me: "EXPECTED value GOT E"

Parse error

Running the package.json file as currently available results in an error after 9 lines - EXPECTED , or } GOT "JSON.sh". Looks like the comma is missing at LINE 10 COLUMN 2 of the sample json file. Adding the comma allows the script to run successfully.

Strings with spaces within arrays don't parse properly

echo {\"foo\":[\"string with space\"]} | JSON.sh

["foo",0]   "string with space"
["foo"] ["string][with][space"]
[]  {"foo":["string][with][space"]}

Nice work on JSON.sh though, I really like it.

Parse files with byte-order-mark

When parsing a JSON file with BOM such as:

https://github.com/dotnet/toolset/blob/40cc5860e2ef311b9aca733b1d2eccaa681bd422/TestAssets/InstallationScriptTests/InstallationScriptTests.json

JSON.sh gives the following error:

EXPECTED EOF GOT {

Current workaround is to strip these characters using tool like awk: awk 'NR==1{sub(/^\xef\xbb\xbf/,"")}1' "$json_file" | sh JSON.sh | .....

It would be nice if parser skip these BOM characters and start parsing after the offset. It will save consumers from stripping these characters.

Tokenize fails with busybox 1.11.2

Running on an embedded system with Busybox 1.11.2, any input fails:

$ echo {} | ./JSON.sh
EXPECTED string GOT EOF

$ egrep --version
egrep: unrecognized option `--version'
BusyBox v1.11.2 (2016-05-23 16:56:16 EEST) multi-call binary

No help available.
$ busybox --help
BusyBox v1.11.2 (2016-05-23 16:56:16 EEST) multi-call binary
Copyright (C) 1998-2008 Erik Andersen, Rob Landley, Denys Vlasenko
and others. Licensed under GPLv2.
See source distribution for full notice.

Usage: busybox [function] [arguments]...
   or: function [arguments]...

	BusyBox is a multi-call binary that combines many common Unix
	utilities into a single executable.  Most people will create a
	link to busybox for each function they wish to use and BusyBox
	will act like whatever it was invoked as!

Currently defined functions:
	[, [[, adjtimex, arping, ash, awk, basename, brctl, bunzip2, bzcat, cat, chgrp, chmod, chown, chroot, clear, cp, crond, crontab, cut, date, dd, df, diff, dirname,
	dmesg, du, echo, egrep, env, expr, false, fgrep, find, free, garp, getty, grep, gunzip, gzip, halt, head, hexdump, hostid, hwclock, id, ifconfig, init, insmod,
	kill, killall, killall5, klogd, length, less, ln, lock, logger, login, logread, ls, lsmod, md5sum, mesg, mkdir, mkfifo, mknod, mktemp, mount, mv, nc, netmsg,
	netstat, nice, nslookup, passwd, pgrep, pidof, ping, ping6, pivot_root, pkill, poweroff, printf, ps, pwd, pwdog, rdate, realpath, reboot, reset, rm, rmdir,
	rmmod, route, sed, seq, sh, sleep, sort, start-stop-daemon, strings, switch_root, sync, sysctl, syslogd, tail, tar, tee, telnet, telnetd, test, tftp, time,
	top, touch, tr, traceroute, true, udhcpc, umount, uname, uniq, uptime, uudecode, uuencode, vconfig, vi, watchdog, wc, wget, which, xargs, yes, zcat, zcip

Then I looked at tokenization only. Sourcing JSON.sh doesn't work (still expecting input), so I changed the last lines like this:

#if ([ "$0" = "$BASH_SOURCE" ] || ! [ -n "$BASH_SOURCE" ]);
#then
#  parse_options "$@"
#  tokenize | parse
#fi
parse_options "$@"
tokenize

What then happens, is:

$ echo {} | shinc/JSON.sh
{
$ echo '{"foo": 1234, "bar":{"something": "hi there"}}' | shinc/JSON.sh
{

Running all tests indeed shows failures.