dprint / jsonc-parser

JSONC parser for Rust

dprint

Monorepo for dprint—a pluggable and configurable code formatting platform.

Links

• Overview
• Getting Started
• Playground

Plugins

• dprint-plugin-typescript - TypeScript/JavaScript code formatter.
• dprint-plugin-json - JSON/JSONC code formatter.
• dprint-plugin-markdown - Markdown code formatter.
• dprint-plugin-toml - TOML code formatter.
• dprint-plugin-jupyter - Jupyter notebook code block formatter.
• dprint-plugin-dockerfile - Dockerfile code formatter.
• dprint-plugin-biome - Biome (JS/TS) wrapper plugin.
• dprint-plugin-prettier - Prettier wrapper plugin.
• dprint-plugin-roslyn - Roslyn (C#/VB) wrapper plugin.
• dprint-plugin-ruff - Ruff (Python) wrapper plugin.
• dprint-plugin-exec - Formats code with any CLI executable.

Notes

This repo is under active early development.

1. The interface between the CLI and plugins might change often. You may need to keep updating to the latest version of both the CLI and plugins (the CLI will let you know what to do).
   • An upgrade path will be outlined in the release notes when this occurs.
2. I do a lot of this development in my spare time. Please consider sponsoring if you are a commercial company using this.

jsonc-parser's Issues

JSONC parser fails to correctly parse non-BMP escape sequences

In accordance with RFC 8258 § 7, the non-BMP character 𝄞 (U+1D11E) should be escaped as the escaped surrogate pair \uD834\uDD1E. Therefore, I expect the following Rust code to compile and run successfully:

use jsonc_parser::JsonValue;
use jsonc_parser::parse_to_value;

fn main() {
    let src = r#""\uD834\uDD1E""#;
    let v = parse_to_value(src, &Default::default()).unwrap().unwrap();
    if let JsonValue::String(s) = v {
        assert_eq!("\u{1D11E}", s)
    }
    else {
        panic!();
    }
}

However, on the latest version of jsonc-parser (as of writing, this is version 0.21.0), this code panics at the unwrap on line 6 with the message "Invalid unicode escape sequence. 'D834' is not a valid UTF8 character".

Implement std::error::Error for ParseError

Using the lib I encountered a bit surprising behavior where ParseError doesn't implement std::error::Error. It makes handling a bit unwieldy.

error[E0277]: the trait bound `jsonc_parser::errors::ParseError: std::error::Error` is not satisfied
   --> cli/tsc_config.rs:147:15
    |
147 |   let jsonc = jsonc_parser::parse_to_value(config_text.as_ref()).map_err(ErrBox::from)?.unwrap();
    |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `std::error::Error` is not implemented for `jsonc_parser::errors::ParseError`

Also, why is ok return value from parse_to_value an Option<JsonValue> instead of just JsonValue?

cannot find function parse_to_serde_value

I'm trying it use parse_to_serde_value but I get this error:
error[E0425]: cannot find function parse_to_serde_value in crate jsonc_parser
It's really weird because the function is clearly there and all I'm doing is just calling it using jsonc_parser::parse_to_serde_value.
Another weird thing is, when I start typing parse_to_serde_value inside use jsonc_parser::{}, vscode starts recommending it but then after I accept it, it just says no parse_to_serde_value in the root!
What's going on?

Reference the original string as much as possible

This should be changed to point at the original string as much as possible. Before doing this though, benchmarks should be created in #9.

Add first class manipulation API

Adding properties should be as easy as calling obj.add_property. Maybe consider converting to a CST

Investigate range on unexpected word errors

For example:

Line 1, column 1: Unexpected word

  class Test {
  ~~~~~~

Or

Line 2, column 13: Unexpected word

      "test": asdf
              ~~~~~

For:

{
    "test": asdf
}

Fails to parse number with `+` prefix

While this is a JSON5 feature rather than JSONC, the parser seems to implement support but fails?:

let parsed = jsonc_parser::parse_to_value(r#"{ "test": +42 }"#, &Default::default());
println!("{:#?}", parsed);

Outputs error:

Err(
    ParseError {
        range: Range {
            start: 10,
            end: 11,
        },
        message: "Unexpected token",
        display_message: "Unexpected token on line 1 column 11.",
    },
)

jsonc-parser/src/scanner.rs

Lines 272 to 293 in abd65a8

	match self.current_char() {
	Some('e') | Some('E') => {
	match self.move_next_char() {
	Some('-') | Some('+') => {
	self.move_next_char();
	if !self.is_digit() {
	return Err(self.create_error_for_current_char("Expected a digit"));
	}
	}
	_ => {
	if !self.is_digit() {
	return Err(self.create_error_for_current_char("Expected plus, minus, or digit in number literal"));
	}
	}
	}
	while self.is_digit() {
	self.move_next_char();
	}
	}
	_ => {}
	}

---

Or is this observation a misunderstanding? (it was first noted here)

It does seem accurate that this crate is more of a JSON5 parser than a JSONC one?

Add Benchmarks

Need memory usage and speed.

Looks like this might be useful: https://github.com/marketplace/actions/continuous-benchmark

common::Ranged provides method to convert to line/col from byte index

Like Ranged::text(), taking the original text, it would be great to be able to return a (usize, usize) that returns a line and column variant, which is more used when displaying positions to users or editors.

Investigate how serde handles invalid utf8 escape sequences

Try out something like "\uDF06" in serde. Might be best to just have some way of representing this anyway in jsonc-parser.

Add helper methods for taking values from JsonObject

Example, should be able to do json_object.take_string(key_name)?;.

Deserialize with error line/column

I'd really like a way to deserialize into a struct while seeing the line/column of errors when there's an invalid type.

With serde_json::from_str I parse a string directly into a struct:

let settings_str = fs::read_to_string(&file_path).unwrap();
serde_json::from_str(&settings_str).unwrap()

And that shows the line/column. But with parse_to_serde_value() & serde_json::from_value, you lose that:

thread 'main' panicked at 'called Result::unwrap() on an Err value: Error("invalid type: boolean true, expected struct Example", line: 0, column: 0)', src/settings.rs:32:40

Make it easy to convert ast::Value to serde_json::Value

When walking the AST it is useful to be able to convert an AST value to a serde_json value for interoperability with lots of other libraries.

Fails to parse multi unit unicode escape sequences

jsonc_parser::parse_to_value("A\uDF06").unwrap().unwrap()

This results in:

panicked at 'called `Result::unwrap()` on an `Err` value: ParseError { range: Range { start: 847, end: 853, start_line: 29, end_line: 29 }, message: "Error converting hex of DF06 to u8. ParseIntError { kind: Overflow }", display_message: "Error converting hex of DF06 to u8. ParseIntError { kind: Overflow } on line 30 column 46." }'

Disregard the line and column numbers in the error above.

Do not store line numbers in ranges

It is slightly faster not to store them and uses less memory. Instead offer ways to retrieve them, but people should use https://github.com/dsherret/text_lines

I already have the code done for this, I'm just going to upgrade dprint-plugin-json first to see how much it impacts that.

Method on ast::Value to get the range

All of the variants of ast::Value contain a range: common::Range, but consumers currently have to pattern match all the variants to be able to extract the range, where a .range() methods could make that easy.

Implement TryFrom<JsonValue> for serde::Value

As per title, it'd be great to have this built into the library.

Change the repository name to json5-parser

jsonc-parser/src/scanner.rs

Lines 317 to 334 in 6ef505d

	match self.current_char() {
	Some('e') | Some('E') => match self.move_next_char() {
	Some('-') | Some('+') => {
	self.move_next_char();
	if !self.is_digit() {
	return Err(self.create_error_for_current_char("Expected a digit"));
	}
	while self.is_digit() {
	self.move_next_char();
	}
	}
	_ => {
	return Err(self.create_error_for_current_char(
	"Expected plus or minus symbol in number literal",
	));
	}
	},
	_ => {}

according to the implementation, this parser could parse not only json with comments but also json5
Change the repository name or add extra keyword to get better SEO