ohler55 / compare-go-json

A comparison of several go JSON packages.

Compare Go JSON

Not all JSON tools cover the same features which make it difficult to select a set of tools for a project. Here is an attempt to compare features and benchmarks for a few of the JSON tools for Go.

Features

Feature	go/json	fastjson	jsoniter	OjG	simdjson	gjson
Parse []byte to simple go types	✅	❌	✅	✅	✅	✅💥
Validate	✅	✅	✅	✅	✅	✅
Parse - io.Reader (large file)	✅	❌	✅	✅	❌	❌
Parse from file	✅	❌	✅	✅	✅	❌
Parse to structs	✅	❌	✅	✅	❌	❌
Parse to interface types	❌	❌	❌	✅	❌	❌
Multiple JSON file/stream	✅	❌	✅	✅	❌	❌
ndjson (newline separated)	✅	❌	✅	✅	✅	❌
Marshal/Write	✅	❌	✅	✅	❌	❌
JSON Builder	❌	❌	❌	✅	❌	❌
JSONPath	❌	❌	❌	✅	❌	❌🔹
Data type converters	❌	❌	❌	✅	❌	❌
Simple Encoding Notation	❌	❌	❌	✅	❌	❌
Test coverage	--	93%	21%	100%	57.4%	91.5%

💥 gjson does not validate while parsing (try a number of 1.2e3e4) although it does catch that error in validation.

🔹 gjson has an alternative search feature

Details of each feature listed are at the bottom of the page

Benchmarks

Parse string/[]byte to simple go types ([]interface{}, int64, string, etc)
     json.Unmarshal          50142 ns/op        17776 B/op          334 allocs/op
       oj.Parse              19453 ns/op         5691 B/op          364 allocs/op
 fastjson >>> not supported <<<
 jsoniter.Unmarshal          40849 ns/op        19656 B/op          451 allocs/op
 simdjson.Parse             119196 ns/op       141345 B/op          370 allocs/op
    gjson.ParseBytes         53440 ns/op        20039 B/op          175 allocs/op

       oj ██████████████████  2.58
 jsoniter ████████▌ 1.23
     json ▓▓▓▓▓▓▓ 1.00
    gjson ██████▌ 0.94
 simdjson ██▉ 0.42
 fastjson >>> not supported <<<

Validate string/[]byte
     json.Valid              12034 ns/op            0 B/op            0 allocs/op
       oj.Validate            3802 ns/op            0 B/op            0 allocs/op
 fastjson.Validate            4521 ns/op            0 B/op            0 allocs/op
 jsoniter.Valid               8699 ns/op         2184 B/op          100 allocs/op
 simdjson.Validate           26498 ns/op       118842 B/op           20 allocs/op
    gjson.Validate            3904 ns/op            0 B/op            0 allocs/op

       oj ██████████████████████▏ 3.17
    gjson █████████████████████▌ 3.08
 fastjson ██████████████████▋ 2.66
 jsoniter █████████▋ 1.38
     json ▓▓▓▓▓▓▓ 1.00
 simdjson ███▏ 0.45

Iterate tokens in a string/[]byte
     json.Decode             90142 ns/op        22600 B/op         1175 allocs/op
       oj.Tokenize            8049 ns/op         1976 B/op          156 allocs/op
 fastjson >>> not supported <<<
 jsoniter.Decode             48138 ns/op        20360 B/op          456 allocs/op
 simdjson >>> not supported <<<
    gjson >>> not supported <<<

       oj ██████████████████████████████████████████████████████████████████████████████▍ 11.20
 jsoniter █████████████  1.87
     json ▓▓▓▓▓▓▓ 1.00
 fastjson >>> not supported <<<
 simdjson >>> not supported <<<
    gjson >>> not supported <<<

Unmarshal string/[]byte to a struct
     json.Unmarshal          39226 ns/op         2560 B/op           73 allocs/op
       oj.Unmarshal          41857 ns/op         9355 B/op          456 allocs/op
 fastjson >>> not supported <<<
 jsoniter.Unmarshal          13005 ns/op         3137 B/op          170 allocs/op
 simdjson >>> not supported <<<
    gjson >>> not supported <<<

 jsoniter █████████████████████  3.02
     json ▓▓▓▓▓▓▓ 1.00
       oj ██████▌ 0.94
 fastjson >>> not supported <<<
 simdjson >>> not supported <<<
    gjson >>> not supported <<<

Marshal simple types to string/[]byte
     json.Marshal            51471 ns/op        17559 B/op          345 allocs/op
       oj.JSON                6615 ns/op            0 B/op            0 allocs/op
 fastjson >>> not supported <<<
 jsoniter.Marshal            18855 ns/op         6299 B/op           63 allocs/op
 simdjson >>> not supported <<<
    gjson >>> not supported <<<

       oj ██████████████████████████████████████████████████████▍ 7.78
 jsoniter ███████████████████  2.73
     json ▓▓▓▓▓▓▓ 1.00
 fastjson >>> not supported <<<
 simdjson >>> not supported <<<
    gjson >>> not supported <<<

Marshal a struct to string/[]byte
     json.Marshal            11353 ns/op         3457 B/op            1 allocs/op
       oj.Marshal             8453 ns/op         1712 B/op           44 allocs/op
 fastjson >>> not supported <<<
 jsoniter.Marshal            11315 ns/op         3466 B/op            2 allocs/op
 simdjson >>> not supported <<<
    gjson >>> not supported <<<

       oj █████████▍ 1.34
 jsoniter ███████  1.00
     json ▓▓▓▓▓▓▓ 1.00
 fastjson >>> not supported <<<
 simdjson >>> not supported <<<
    gjson >>> not supported <<<

Read from single JSON file
     json.Decode             85118 ns/op        32416 B/op          342 allocs/op
       oj.ParseReader        28191 ns/op         9787 B/op          365 allocs/op
 fastjson >>> not supported <<<
 jsoniter.Decode             59635 ns/op        20327 B/op          456 allocs/op
 simdjson >>> not supported <<<
    gjson >>> not supported <<<

       oj █████████████████████▏ 3.02
 jsoniter █████████▉ 1.43
     json ▓▓▓▓▓▓▓ 1.00
 fastjson >>> not supported <<<
 simdjson >>> not supported <<<
    gjson >>> not supported <<<

Read multiple JSON in a small log file (100MB)
     json.Decode        1354784239 ns/op   1102470120 B/op     14810436 allocs/op
       oj.ParseReader    668270061 ns/op    761294040 B/op     15402846 allocs/op
 fastjson >>> not supported <<<
 jsoniter.Decode        1131730647 ns/op   1178775816 B/op     19390099 allocs/op
 simdjson.ParseReader    679189142 ns/op   1398403912 B/op     13639254 allocs/op
    gjson >>> not supported <<<

       oj ██████████████▏ 2.03
 simdjson █████████████▉ 1.99
 jsoniter ████████▍ 1.20
     json ▓▓▓▓▓▓▓ 1.00
 fastjson >>> not supported <<<
    gjson >>> not supported <<<

Read multiple JSON in a semi large log file (5GB)
     json.Decode       88433748107 ns/op  28649383232 B/op    740520188 allocs/op
       oj.ParseReader  38353089960 ns/op  11590577208 B/op    770140750 allocs/op
 fastjson >>> not supported <<<
 jsoniter.Decode       86478665854 ns/op  32465480800 B/op    969507711 allocs/op
 simdjson.ParseReader >>> out of memory <<<
    gjson >>> not supported <<<

       oj ████████████████▏ 2.31
 jsoniter ███████▏ 1.02
     json ▓▓▓▓▓▓▓ 1.00
 fastjson >>> not supported <<<
 simdjson >>> out of memory <<<
    gjson >>> not supported <<<

 Higher values (longer bars) are better in all cases. The bar graph compares the
 parsing performance. The lighter colored bar is the reference, the go json
 package.

Tests run on:
 OS:              Ubuntu 20.04.2 LTS
 Processor:       Intel(R) Core(TM) i7-8700 CPU
 Cores:           12
 Processor Speed: 3.20GHz
 Memory:          16 GB

Feature Explanations

• Parse parse a string to []byte slice in simple go types of []interface, map[string]interface{}, string, float64, int64, bool, or nil. This support the use case of extracting data from a JSON suitable for natigating as well as handing off to other packages such as a database for storage.

• Validate a string or []byte slice without extracting values.

• Read from io.Reader indicates a source such as a socket or file larger than will fit into memory can be parsed.

• Read from file indicates a parser can read from a file if not directly then using ioutils.

• Parse to structs is the ability to reconstitute a struct type from JSON.

• Parse to interface types is the ability to reconstitutes types even if they are included as interfaces in a containing struct or slice.

• Multiple JSON indicates a file or stream with multiple JSON documents can be parsed. This is no restricted to the limited case of exactly one JSON element per line. Encountered in database dumps and load files.

• ndjson is a multiple document JSON where each JSON document must be on exactly one line. Found in log files.

• Marshal/Write is the ability of the package to marshal go types in JSON.

• JSON Builder is the ability to create new data structures suitable for JSON encoding.

• JSONPath is the ability to navigate data using JSONPath expressions.

• Data type converters tools for converting from type to simple data types. Basically marshalling and unmarshalling to simple types instead of to JSON.

• Simple Encoding Notation is a lazy JSON format where quotes and commas are optional in most cases. A merge of JSON and GraphQL formats for those of us that don't want to be bothered with strict syntax checking.

• Parser Test coverage percent unit test coverage of the parser package. It does not include coverage of other package in the offering.