hjson / hjson-rust Goto Github PK

View Code? Open in Web Editor NEW

96.0 7.0 30.0 760 KB

Hjson for Rust

Home Page: https://hjson.github.io/

License: MIT License

Rust 100.00%

hjson rust serde maintainer-wanted

hjson-rust's Introduction

hjson-rust for serde

{
  # specify rate in requests/second (because comments are helpful!)
  rate: 1000

  // prefer c-style comments?
  /* feeling old fashioned? */

  # did you notice that rate doesn't need quotes?
  hey: look ma, no quotes for strings either!

  # best of all
  notice: []
  anything: ?

  # yes, commas are optional!
}

The Rust implementation of Hjson is based on the Serde JSON Serialization Library. For other platforms see hjson.github.io.

This crate is a Rust library for parsing and generating Human JSON Hjson. It is built upon Serde, a high performance generic serialization framework.

Install

This crate works with Cargo and can be found on crates.io with a Cargo.toml like:

[dependencies]
serde = "*"
serde-hjson = "*"

From the Commandline

Install with cargo install hjson

Hjson, the Human JSON.

Usage:
  hjson [options]
  hjson [options] <input>
  hjson (-h | --help)
  hjson (-V | --version)

Options:
  -h --help     Show this screen.
  -j            Output as formatted JSON.
  -c            Output as JSON.
  -V --version  Show version.

Sample:

run hjson test.json > test.hjson to convert to Hjson
run hjson -j test.hjson > test.json to convert to JSON

Usage

extern crate serde;
extern crate serde_hjson;

use serde_hjson::{Map,Value};

fn main() {

    // Now let's look at decoding Hjson data

    let sample_text=r#"
    {
        # specify rate in requests/second
        rate: 1000
        array:
        [
            foo
            bar
        ]
    }"#;

    // Decode and unwrap.
    let mut sample: Map<String, Value> = serde_hjson::from_str(&sample_text).unwrap();

    // scope to control lifetime of borrow
    {
        // Extract the rate
        let rate = sample.get("rate").unwrap().as_f64().unwrap();
        println!("rate: {}", rate);

        // Extract the array
        let array : &mut Vec<Value> = sample.get_mut("array").unwrap().as_array_mut().unwrap();
        println!("first: {}", array.get(0).unwrap());

        // Add a value
        array.push(Value::String("tak".to_string()));
    }

    // Encode to Hjson
    let sample2 = serde_hjson::to_string(&sample).unwrap();
    println!("Hjson:\n{}", sample2);
}

API

see Rust doc

History

see history.md

hjson-rust's People

Contributors

Stargazers

Watchers

hjson-rust's Issues

Numbers are always be parsed to Value::F64

{
  # specify rate in requests/second (because comments are helpful!)
  rate: 1000
}

This is the example taken from README. And we have to use the value of rate by as_f64 like this:

let rate = sample.get("rate").unwrap().as_f64().unwrap();

1000 should be Value::U64 or at least Value::I64.

Maybe we can represent number values as Value::Number like serde-json does?

serde_hjson::from_slice panics on ParseIntError

more fuzzing gave me this error:

thread '' panicked at 'called Result::unwrap() on an Err value: ParseIntError { kind: Overflow }', /home/capitol/project/hjson-rust/hjson/src/util.rs:208:67

full stacktrace:

    #0 0x561f6f275731 in __sanitizer_print_stack_trace /rustc/llvm/src/llvm-project/compiler-rt/lib/asan/asan_stack.cpp:86:3
    #1 0x561f6f936660 in fuzzer::PrintStackTrace() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x996660)
    #2 0x561f6f95299a in fuzzer::Fuzzer::CrashCallback() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9b299a)
    #3 0x7f153a3933bf  (/lib/x86_64-linux-gnu/libpthread.so.0+0x153bf)
    #4 0x7f153a1b718a in __libc_signal_restore_set /build/glibc-YYA7BZ/glibc-2.31/signal/../sysdeps/unix/sysv/linux/internal-signals.h:86:3
    #5 0x7f153a1b718a in raise /build/glibc-YYA7BZ/glibc-2.31/signal/../sysdeps/unix/sysv/linux/raise.c:48:3
    #6 0x7f153a196858 in abort /build/glibc-YYA7BZ/glibc-2.31/stdlib/abort.c:79:7
    #7 0x561f6f9aeb36 in std::sys::unix::abort_internal::h5c8b2a90c624abaf /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/sys/unix/mod.rs:167:14
    #8 0x561f6f997bc5 in std::process::abort::hb13208ae9f5b7133 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/process.rs:1623:5
    #9 0x561f6f9201b6 in libfuzzer_sys::initialize::_$u7b$$u7b$closure$u7d$$u7d$::h2ef829035805c4e9 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9801b6)
    #10 0x561f6f99eed7 in std::panicking::rust_panic_with_hook::h2f4c96dfd8ba524a /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:581:17
    #11 0x561f6f99ea88 in std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::h7740abbe2875cb4d /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:484:9
    #12 0x561f6f999ebb in std::sys_common::backtrace::__rust_end_short_backtrace::hcad001df0a36db28 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/sys_common/backtrace.rs:153:18
    #13 0x561f6f99ea48 in rust_begin_unwind /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:483:5
    #14 0x561f6fa04460 in core::panicking::panic_fmt::hb15d6f55e8472f62 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/core/src/panicking.rs:85:14
    #15 0x561f6fa040d2 in core::result::unwrap_failed::h110828a80aba3eec /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/core/src/option.rs:1221:5
    #16 0x561f6f2d923d in serde_hjson::util::ParseNumber$LT$Iter$GT$::parse::hba5da05d298b23ff (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x33923d)
    #17 0x561f6f2a58ad in serde_hjson::de::Deserializer$LT$Iter$GT$::parse_tfnns::h7d7fb93a4d3df50b (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x3058ad)
    #18 0x561f6f2abb92 in serde_hjson::de::Deserializer$LT$Iter$GT$::parse_value::h09e41fbc88e5efe4 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x30bb92)
    #19 0x561f6f301ff1 in serde::de::MapVisitor::visit::hf1a5b50f97f17367 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x361ff1)
    #20 0x561f6f310966 in _$LT$linked_hash_map..serde..LinkedHashMapVisitor$LT$K$C$V$GT$$u20$as$u20$serde..de..Visitor$GT$::visit_map::h730c4bb087010ef0 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x370966)
    #21 0x561f6f2aece8 in serde_hjson::de::Deserializer$LT$Iter$GT$::parse_value::h9b2c32860cf298a3 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x30ece8)
    #22 0x561f6f2cd8ba in serde_hjson::de::from_iter::hc227fa3539b40986 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x32d8ba)
    #23 0x561f6f317118 in rust_fuzzer_test_input (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x377118)
    #24 0x561f6f9201e0 in __rust_try (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9801e0)
    #25 0x561f6f91fe3f in LLVMFuzzerTestOneInput (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x97fe3f)
    #26 0x561f6f952edc in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9b2edc)
    #27 0x561f6f95aec0 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9baec0)
    #28 0x561f6f95b87c in fuzzer::Fuzzer::MutateAndTestOne() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9bb87c)
    #29 0x561f6f95dc7f in fuzzer::Fuzzer::Loop(std::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9bdc7f)
    #30 0x561f6f92e239 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x98e239)
    #31 0x561f6f1f22e6 in main (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x2522e6)
    #32 0x7f153a1980b2 in __libc_start_main /build/glibc-YYA7BZ/glibc-2.31/csu/../csu/libc-start.c:308:16
    #33 0x561f6f1f248d in _start (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x25248d)

Can be reproduced with the following unit test:

#[cfg(test)]
mod test {
    use crate::{Map,Value};
    use crate::error::Result;

    #[test]
    pub fn parse_int_error() {
        let data: Vec<u8> = vec![47, 97, 47, 65, 58, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 0, 0, 0, 54, 35, 54, 54, 54, 54, 54, 54, 54, 44, 35, 58, 45, 85, 85, 85, 35, 116, 45, 35, 35, 58, 47];

        let mut sample: Result<Map<String, Value>> = crate::from_slice(&data);
    }
}

Support for single quoted strings

See hjson/hjson#74

Strips leading slashes from key names and string fields

Example:

{
    foo: {
        /bar/: /baz/bak
        foo: /fnord
    }
}

becomes:

{
  "foo": {
    "bar/": "baz/bak",
    "foo": "fnord"
  }
}

Upgrade to Serde 1.0.x using Serde's JSON implementation

Since Hjson is an extension of JSON, and the Serde team keeps their JSON implementation up to date, I think it would make sense to keep hjson-rust as a fork of serde-rs/json. That way any changes the Serde team makes can be merged or modified to apply to Hjson. I've already been working on the deserialization part for a bit, and I'll upload my progress on Monday.

Is this crate maintained?

Asking because dependencies are outdated and there seems to be little progress overall.

@dqsully ?

Parse error on deserialization of enum value

Trying to deserialize an struct that contains an enum on Rust nightly. However using hjson i get a parse error while the example seems to work when using json instead.

thread 'hjson_enum_serialize_deserialize_works' panicked at 'called `Result::unwrap()` on an `Err` value: Syntax(Found a punctuator where a key name was expected (check your syntax or use quotes if the key name includes {}[],: or whitespace), 1, 1)', /Users/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-mac/build/src/libcore/result.rs:845

Example test-case:

#![feature(proc_macro)]

#[macro_use]
extern crate serde_derive;
extern crate serde_hjson;

#[derive(Serialize, Deserialize, Debug, PartialEq)]
struct Config {
    color: Color,
}

#[derive(Serialize, Deserialize, Debug, PartialEq)]
enum Color {
    Green,
}

#[test]
fn hjson_enum_serialize_deserialize_works() {
    let config = Config { color: Color::Green };
    let serialized = serde_hjson::to_string(&config).unwrap();
    println!("serialized:\n{}", serialized);
    let deserialized: Config = serde_hjson::from_str(&serialized).unwrap();
    println!("deserialized:\n{:?}", deserialized);
    assert_eq!(config, deserialized);
}

Best regards,
Sander Hahn

Depends on outdated linked-hash-map

The linked-hash-map crate release 0.5.3 contained a critical bugfix; older releases might stop working when the compiler is able to better detect misuses of uninitialized memory (rust-lang/rust#71274). Would be great if you could update the dependency of your crate to use the latest version. :)

tests fail to compile due to quasi

When running "cargo test" in hjson_tests/, it fails to compile with:

error[E0432]: unresolved import `syntax::parse::parse_tts_from_source_str`
  --> /home/jvalleroy/.cargo/registry/src/github.com-1ecc6299db9ec823/quasi-0.20.1/src/lib.rs:36:37
   |
36 | use syntax::parse::{self, classify, parse_tts_from_source_str, token};
   |                                     ^^^^^^^^^^^^^^^^^^^^^^^^^ no `parse_tts_from_source_str` in `parse`. Did you mean to use `parse_item_from_source_str`?

and many other errors.

quasi crate is no longer maintained. Should it be replaced with something else?

from_str method failing with integer types.

Error occured after I passed a string that contains integer/float in the body.

Error

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Syntax("trailing characters", 1, 3)', src/libcore/result.rs:1188:5

Reproducible code

extern crate serde_hjson;

use serde_hjson::{from_str,Value};

fn main (){
    let json:Value = from_str("123").unwrap();
}

Reason

hjson-rust/hjson/src/de.rs

Line 130 in 37c4be8

Why this method is not implemented yet? Can you please implement this?

Update to Serde 1.0

Serde 0.9.0 release notes: https://github.com/serde-rs/serde/releases/tag/v0.9.0
Serde 1.0.0 release notes: https://github.com/serde-rs/serde/releases/tag/v1.0.0

How to use Serialize/Deserialize derives with this crate?

Hello,

I am a bit new to Rust (I have to start with that), but I am also very confused how to use the custom derives of serde with this crate?

From your docs it would seem like it should be:

extern crate serde;
extern crate serde_hjson;

use serde_hjson::{Map,Value};

#[derive(Serialize, Deserialize, Debug)]
struct Point {
    x: i32,
    y: i32,
}

fn main() {
let sample_text=r#"[{ x: 1, y: 2 },{ x: 3, y: 4 }]"#;
    let sample: Value = serde_hjson::from_str(&sample_text).unwrap();
    {
        let sample2 = serde_hjson::to_string(sample.as_array().unwrap().get(0).unwrap().as_object().unwrap()).unwrap();

        let testData: Point = serde_hjson::from_str(&sample2).unwrap();
    }
}

However this returns the following error:

error[E0277]: the trait bound 'Point: serde::de::Deserialize' is not satisfied
   --> src/lib.rs:36:30
    |
36  |         let testData: Point = serde_hjson::from_str(&sample2).unwrap();
    |                              ^^^^^^^^^^^^^^^^^^^^^ the trait 'serde::de::Deserialize' is not implemented for 'Point'
    | 
    |
851 |     where T: de::Deserialize
    |              --------------- required by this bound in 'serde_hjson::from_str'

Edition is 2018 and my Cargo.toml deps are:

[dependencies]
serde = { version = "0.9", features = ["derive"] }
serde-hjson = "*"

If you could point me towards the right direction of what I am missing here I would be most grateful.

hjson-rust fails with Rust 1.48

We've been using hjson in Nushell, and it's been great. Definitely an improvement to have a more-flexible json format.

With the latest Rust 1.48, though, we ran into an issue where one of the dependencies is leaving memory uninitialized.

Here's the full backtrace:

---- commands::math::compound_where_paren stdout ----
=== stderr
thread 'main' panicked at 'attempted to leave type `linked_hash_map::Node<std::string::String, value::Value>` uninitialized, which is invalid', /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/mem/mod.rs:658:9
stack backtrace:
   0: rust_begin_unwind
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:483
   1: core::panicking::panic_fmt
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/panicking.rs:85
   2: core::panicking::panic
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/panicking.rs:50
   3: core::mem::uninitialized
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/mem/mod.rs:658
   4: linked_hash_map::LinkedHashMap<K,V,S>::insert
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/linked-hash-map-0.3.0/src/lib.rs:203
   5: <linked_hash_map::serde::LinkedHashMapVisitor<K,V> as serde::de::Visitor>::visit_map
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/linked-hash-map-0.3.0/src/serde.rs:72
   6: <<serde_hjson::value::Value as serde::de::Deserialize>::deserialize::ValueVisitor as serde::de::Visitor>::visit_map
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/value.rs:441
   7: serde_hjson::de::Deserializer<Iter>::parse_value
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:151
   8: <serde_hjson::de::Deserializer<Iter> as serde::de::Deserializer>::deserialize
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:443
   9: <serde_hjson::value::Value as serde::de::Deserialize>::deserialize
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/value.rs:446
  10: <serde_hjson::de::SeqVisitor<Iter> as serde::de::SeqVisitor>::visit
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:537
  11: <serde::de::impls::VecVisitor<T> as serde::de::Visitor>::visit_seq
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-0.8.23/src/de/impls.rs:433
  12: <<serde_hjson::value::Value as serde::de::Deserialize>::deserialize::ValueVisitor as serde::de::Visitor>::visit_seq
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/value.rs:432
  13: serde_hjson::de::Deserializer<Iter>::parse_value
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:147
  14: <serde_hjson::de::Deserializer<Iter> as serde::de::Deserializer>::deserialize
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:443
  15: <serde_hjson::value::Value as serde::de::Deserialize>::deserialize
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/value.rs:446
  16: serde_hjson::de::from_iter
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:813
  17: serde_hjson::de::from_slice
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:846
  18: serde_hjson::de::from_str
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-hjson-0.9.1/src/de.rs:853
  19: nu_cli::commands::from_json::from_json_string_to_value
             at /home/jonathan/Source/nushell/crates/nu-cli/src/commands/from_json.rs:71
  20: nu_cli::commands::from_json::from_json::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/commands/from_json.rs:114
  21: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  22: <nu_cli::commands::from_json::FromJSON as nu_cli::commands::command::WholeStreamCommand>::run::__run::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/commands/from_json.rs:36
  23: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  24: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/future.rs:119
  25: nu_cli::commands::command::Command::run::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/commands/command.rs:321
  26: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  27: nu_cli::evaluation_context::EvaluationContext::run_command::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/evaluation_context.rs:159
  28: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  29: nu_cli::commands::classified::internal::run_internal_command::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/commands/classified/internal.rs:29
  30: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  31: nu_cli::commands::classified::block::run_pipeline::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/commands/classified/block.rs:82
  32: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  33: nu_cli::commands::classified::block::run_block::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/commands/classified/block.rs:55
  34: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  35: nu_cli::cli::process_line::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/cli.rs:1036
  36: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  37: nu_cli::cli::cli::{{closure}}
             at /home/jonathan/Source/nushell/crates/nu-cli/src/cli.rs:479
  38: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
  39: futures_executor::local_pool::block_on::{{closure}}
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-executor-0.3.5/src/local_pool.rs:317
  40: futures_executor::local_pool::run_executor::{{closure}}
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-executor-0.3.5/src/local_pool.rs:87
  41: std::thread::local::LocalKey<T>::try_with
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:272
  42: std::thread::local::LocalKey<T>::with
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:248
  43: futures_executor::local_pool::run_executor
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-executor-0.3.5/src/local_pool.rs:83
  44: futures_executor::local_pool::block_on
             at /home/jonathan/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-executor-0.3.5/src/local_pool.rs:317
  45: nu::main
             at /home/jonathan/Source/nushell/src/main.rs:171
  46: core::ops::function::FnOnce::call_once
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

thread 'commands::math::compound_where_paren' panicked at 'assertion failed: `(left == right)`
  left: `""`,
 right: `"[{\"a\":2,\"b\":1},{\"a\":2,\"b\":2}]"`', crates/nu-cli/tests/commands/math/mod.rs:257:5
stack backtrace:
   0: rust_begin_unwind
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:483
   1: std::panicking::begin_panic_fmt
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:437
   2: main::commands::math::compound_where_paren
             at ./tests/commands/math/mod.rs:257
   3: main::commands::math::compound_where_paren::{{closure}}
             at ./tests/commands/math/mod.rs:249
   4: core::ops::function::FnOnce::call_once
             at /home/jonathan/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:227
   5: core::ops::function::FnOnce::call_once
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/ops/function.rs:227
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Assuming I'm reading that correctly, the issue is in linked_hash_map. I looked on their repo and it looks like it's no longer maintained.

Any ideas if there's a fix for this? We'd definitely love to keep using this package.

serde_hjson::from_slice panics on decoding invalid utf8

I found that the from_slice function panics if the input isn't a valid utf8 string instead of returning an error value.

Can be reproduced with this unit test:


#[cfg(test)]
mod test {
    use crate::{Map,Value};
    use crate::error::Result;

    #[test]
    pub fn invalid_utf8() {
        let data: Vec<u8> = vec![155];

        let mut sample: Result<Map<String, Value>> = crate::from_slice(&data);
    }
}

serde_hjson::from_slice panics at 'removal index (is 0) should be < len (is 0)'

more testing of the from_slice function with valid utf8 gave me this error:

thread '' panicked at 'removal index (is 0) should be < len (is 0)', library/alloc/src/vec.rs:1057:13

complete stacktrace is:

==1202250== ERROR: libFuzzer: deadly signal
    #0 0x55c30665a731 in __sanitizer_print_stack_trace /rustc/llvm/src/llvm-project/compiler-rt/lib/asan/asan_stack.cpp:86:3
    #1 0x55c306d1b660 in fuzzer::PrintStackTrace() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x996660)
    #2 0x55c306d3799a in fuzzer::Fuzzer::CrashCallback() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9b299a)
    #3 0x7f714cb733bf  (/lib/x86_64-linux-gnu/libpthread.so.0+0x153bf)
    #4 0x7f714c99718a in __libc_signal_restore_set /build/glibc-YYA7BZ/glibc-2.31/signal/../sysdeps/unix/sysv/linux/internal-signals.h:86:3
    #5 0x7f714c99718a in raise /build/glibc-YYA7BZ/glibc-2.31/signal/../sysdeps/unix/sysv/linux/raise.c:48:3
    #6 0x7f714c976858 in abort /build/glibc-YYA7BZ/glibc-2.31/stdlib/abort.c:79:7
    #7 0x55c306d93b36 in std::sys::unix::abort_internal::h5c8b2a90c624abaf /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/sys/unix/mod.rs:167:14
    #8 0x55c306d7cbc5 in std::process::abort::hb13208ae9f5b7133 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/process.rs:1623:5
    #9 0x55c306d051b6 in libfuzzer_sys::initialize::_$u7b$$u7b$closure$u7d$$u7d$::h2ef829035805c4e9 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9801b6)
    #10 0x55c306d83ed7 in std::panicking::rust_panic_with_hook::h2f4c96dfd8ba524a /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:581:17
    #11 0x55c306d83a88 in std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::h7740abbe2875cb4d /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:484:9
    #12 0x55c306d7eebb in std::sys_common::backtrace::__rust_end_short_backtrace::hcad001df0a36db28 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/sys_common/backtrace.rs:153:18
    #13 0x55c306d83a48 in rust_begin_unwind /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:483:5
    #14 0x55c306de9460 in core::panicking::panic_fmt::hb15d6f55e8472f62 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/core/src/panicking.rs:85:14
    #15 0x55c306de12a5 in alloc::vec::Vec$LT$T$GT$::remove::assert_failed::h2c0d56e327999482 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/alloc/src/vec.rs:1057:13
    #16 0x55c3066c55fd in serde_hjson::util::StringReader$LT$Iter$GT$::parse_whitespace::hf7084d7b3ccb8e69 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x3405fd)
    #17 0x55c306693981 in serde_hjson::de::Deserializer$LT$Iter$GT$::parse_value::h9b2c32860cf298a3 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x30e981)
    #18 0x55c3066b28ba in serde_hjson::de::from_iter::hc227fa3539b40986 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x32d8ba)
    #19 0x55c3066fc118 in rust_fuzzer_test_input (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x377118)
    #20 0x55c306d051e0 in __rust_try (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9801e0)
    #21 0x55c306d04e3f in LLVMFuzzerTestOneInput (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x97fe3f)
    #22 0x55c306d37edc in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9b2edc)
    #23 0x55c306d3fec0 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9baec0)
    #24 0x55c306d4087c in fuzzer::Fuzzer::MutateAndTestOne() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9bb87c)
    #25 0x55c306d42c7f in fuzzer::Fuzzer::Loop(std::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9bdc7f)
    #26 0x55c306d13239 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x98e239)
    #27 0x55c3065d72e6 in main (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x2522e6)
    #28 0x7f714c9780b2 in __libc_start_main /build/glibc-YYA7BZ/glibc-2.31/csu/../csu/libc-start.c:308:16
    #29 0x55c3065d748d in _start (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x25248d)

Can be reproduced with this unit test

#[cfg(test)]
mod test {
    use crate::{Map,Value};
    use crate::error::Result;

    #[test]
    pub fn removal_index() {
        let data: Vec<u8> = vec![47, 42, 44, 45];

        let mut sample: Result<Map<String, Value>> = crate::from_slice(&data);
    }
}

serde_hjson::from_slice panics on subtract with overflow

testing the from_slice function with valid utf8 gave me this error:

thread '' panicked at 'attempt to subtract with overflow', /home/capitol/project/hjson-rust/hjson/src/de.rs:294:22

complete stacktrace:

    #0 0x55a627176731 in __sanitizer_print_stack_trace /rustc/llvm/src/llvm-project/compiler-rt/lib/asan/asan_stack.cpp:86:3
    #1 0x55a627837660 in fuzzer::PrintStackTrace() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x996660)
    #2 0x55a62785399a in fuzzer::Fuzzer::CrashCallback() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9b299a)
    #3 0x7fb82afe03bf  (/lib/x86_64-linux-gnu/libpthread.so.0+0x153bf)
    #4 0x7fb82ae0418a in __libc_signal_restore_set /build/glibc-YYA7BZ/glibc-2.31/signal/../sysdeps/unix/sysv/linux/internal-signals.h:86:3
    #5 0x7fb82ae0418a in raise /build/glibc-YYA7BZ/glibc-2.31/signal/../sysdeps/unix/sysv/linux/raise.c:48:3
    #6 0x7fb82ade3858 in abort /build/glibc-YYA7BZ/glibc-2.31/stdlib/abort.c:79:7
    #7 0x55a6278afb36 in std::sys::unix::abort_internal::h5c8b2a90c624abaf /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/sys/unix/mod.rs:167:14
    #8 0x55a627898bc5 in std::process::abort::hb13208ae9f5b7133 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/process.rs:1623:5
    #9 0x55a6278211b6 in libfuzzer_sys::initialize::_$u7b$$u7b$closure$u7d$$u7d$::h2ef829035805c4e9 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9801b6)
    #10 0x55a62789fed7 in std::panicking::rust_panic_with_hook::h2f4c96dfd8ba524a /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:581:17
    #11 0x55a62789fa88 in std::panicking::begin_panic_handler::_$u7b$$u7b$closure$u7d$$u7d$::h7740abbe2875cb4d /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:484:9
    #12 0x55a62789aebb in std::sys_common::backtrace::__rust_end_short_backtrace::hcad001df0a36db28 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/sys_common/backtrace.rs:153:18
    #13 0x55a62789fa48 in rust_begin_unwind /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/std/src/panicking.rs:483:5
    #14 0x55a627905460 in core::panicking::panic_fmt::hb15d6f55e8472f62 /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/core/src/panicking.rs:85:14
    #15 0x55a6279053ac in core::panicking::panic::h5d1c61fed2502a5f /rustc/d006f5734f49625c34d6fc33bf6b9967243abca8/library/core/src/panicking.rs:50:5
    #16 0x55a6271c0f46 in serde_hjson::de::Deserializer$LT$Iter$GT$::parse_ml_string::h6d7ce8a9cfd6f9b5 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x31ff46)
    #17 0x55a6271a0d95 in serde_hjson::de::Deserializer$LT$Iter$GT$::parse_tfnns::h150292d6fccc2cd0 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x2ffd95)
    #18 0x55a6271b2891 in serde_hjson::de::Deserializer$LT$Iter$GT$::parse_value::hefb12d656723d8ef (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x311891)
    #19 0x55a6271cef2d in serde_hjson::de::from_iter::hc227fa3539b40986 (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x32df2d)
    #20 0x55a627218118 in rust_fuzzer_test_input (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x377118)
    #21 0x55a6278211e0 in __rust_try (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9801e0)
    #22 0x55a627820e3f in LLVMFuzzerTestOneInput (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x97fe3f)
    #23 0x55a627853edc in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9b2edc)
    #24 0x55a62785bec0 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9baec0)
    #25 0x55a62785c87c in fuzzer::Fuzzer::MutateAndTestOne() (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9bb87c)
    #26 0x55a62785ec7f in fuzzer::Fuzzer::Loop(std::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x9bdc7f)
    #27 0x55a62782f239 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x98e239)
    #28 0x55a6270f32e6 in main (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x2522e6)
    #29 0x7fb82ade50b2 in __libc_start_main /build/glibc-YYA7BZ/glibc-2.31/csu/../csu/libc-start.c:308:16
    #30 0x55a6270f348d in _start (/home/capitol/project/hjson-rust/hjson/fuzz/target/x86_64-unknown-linux-gnu/release/fuzz_target_1+0x25248d)