tomlj / tomlj

A Java parser for Tom's Obvious, Minimal Language (TOML).

TomlJ: A Java parser for Tom's Obvious, Minimal Language (TOML)

TomlJ is a complete TOML parser with the following attributes:

• Supports the latest TOML specification version (1.0.0).
• Provides detailed error reporting, including error position.
• Performs error recovery, allowing parsing to continue after an error.

It uses the ANTLR parser-generator and runtime library.

Usage

Parsing is straightforward:

Path source = Paths.get("/path/to/file.toml");
TomlParseResult result = Toml.parse(source);
result.errors().forEach(error -> System.err.println(error.toString()));

String value = result.getString("a. dotted . key");

Getting TomlJ

TomlJ is published to a Maven Central.

To include using Maven:

<dependency>
  <groupId>org.tomlj</groupId>
  <artifactId>tomlj</artifactId>
  <version>1.1.1</version>
</dependency>

To include using Gradle: implementation 'org.tomlj:tomlj:1.1.1'

Links

• GitHub project
• Online Java documentation
• Issue tracker: Report a defect or feature request
• StackOverflow: Ask "how-to" and "why-didn't-it-work" questions

tomlj's Issues

Ability to index through tables inside of a table

For a project I'm working, it's within scope for there to be a large amount of tables nested inside of tables. It would be really good to be able to fetch a List object containing all tables found within a result, array, or table object.

Weird StackOverflowError

Following lines cause stack overflow with no adequate call site information:

[group.child]
b="B"

[group]
a="A"

Yet if I declare [group] prior to [group.child] - everything works fine.

Toml Serializer?

I want to replace my toml impl with tomlj, but how can I create an instance of TomlTable? MutableTomlTable is package private.

toJson() produces invalid JSON if a value is a JSON string

Given the toml document:
jsonval = '{"msg":"This is invalid json"}'

An invalid JSON document will be produced by a toJson() call.
Generated JSON from example toml:

{
  "jsonval" : "{"msg":"This is invalid json"}"
}

Add Ability To Parse Out Of Range Longs/Doubles As BigInteger/BigDecimal

I am using this project in one of my libraries to read configs, but when I try to parse a large long (eg: 9223372036854775808), it fails with an Integer is too large error. While this issue is not bound to occur in the case of a properly formed TOML file, it will in the case of one that is improperly formed. To better deal with such issues of exceptionally large numbers being treated as a long/double as opposed to a BigInteger or BigDecimal and to increase the stability of the parser, an option to cast large numbers to BigInteger or BigDecimal or even a plain string would be a great feature, as in the case of json.org taking the same number and treating it a string or SnakeYAML treating it as a BigInteger. Please consider adding this as a feature or I can fork the library and attempt it myself. Thanks for making such a useful library. Much appreciated.

Remove JSR305 dependency

JSR305 causes problems on JDK 9+. This Guava issue explains it pretty well:
google/guava#2960

Tomlj has com.google.findbugs:jsr305 as a dependency, and it is added to dependent projects transitively.

After last commit, toJson throws exception

Hi @cleishm
After the last commit, I'm facing issue with toJson(). Here's what I did:

TOML:

[[fruit]]
name = "apple"

[[fruit.physical]]
color = "red"
shape = "round"

[[fruit.variety]]
name = "red delicious"

[[fruit.variety]]
  name = "granny smith"

[[fruit]]
name = "banana"

[[fruit.variety]]
name = "plantain"

Code:

Path   TOML = Paths.get("D:/test/test.toml");
TomlParseResult toml = Toml.parse(TOML);
String result = toml.toJson();
System.out.println(result);

Exception:

java.util.NoSuchElementException
	at java.util.ArrayList$Itr.next(Unknown Source)
	at org.tomlj.JsonSerializer.toJson(JsonSerializer.java:73)
	at org.tomlj.JsonSerializer.appendTomlValue(JsonSerializer.java:110)
	at org.tomlj.JsonSerializer.toJson(JsonSerializer.java:46)
	at org.tomlj.JsonSerializer.toJson(JsonSerializer.java:30)
	at org.tomlj.TomlTable.toJson(TomlTable.java:1240)
	at org.tomlj.TomlTable.toJson(TomlTable.java:1225)
	at Main.main(Main.java:29)

Malformed output when serializing TOML tables

When serializing nested table data, e. g. the TomlParseResult
obtained from deserializing this TOML file:

[buggy]
a = { foo = "bar", baz = ["xyzzy"] }
b = { foo = "bar", baz = ["xyzzy"] }
c = "quux"
d = "quux"
e = { foo = "bar", baz = ["xyzzy"] }
f = "quux"

tomlj incorrectly emits simple and nested values alongside each other:

[buggy]
  [buggy.a]
    foo="bar"
    baz=[
      "xyzzy"
    ]
  [buggy.b]
    foo="bar"
    baz=[
      "xyzzy"
    ]
  c="quux"
  d="quux"
  [buggy.e]
    foo="bar"
    baz=[
      "xyzzy"
    ]
  f="quux"

Obviously in this output the key f will be parsed as part of
[buggy.e], and keys c and d to buggy.b instead of
buggy where they belong in the original file.

What the serializer should do is output the nested ones as individual
sections like so:

[buggy]
  c="quux"
  d="quux"
  f="quux"

[buggy.a]
  foo="bar"
  baz=[
    "xyzzy"
  ]
[buggy.b]
  foo="bar"
  baz=[
    "xyzzy"
  ]
[buggy.e]
  foo="bar"
  baz=[
    "xyzzy"
  ]

Cherry-pick this commit for a test case based on a real world Rust crate.

Changelog for v1.0.1?

Hi @cleishm I noticed that there is a v1.0.1 on bintray: https://bintray.com/tomlj/tomlj/tomlj/1.0.1 released on May 2, 2020 however there only appears to be a v1.0.0 on GitHub here: https://github.com/tomlj/tomlj/releases/tag/1.0.0 and here: https://github.com/tomlj/tomlj/blob/main/build.gradle#L22

Is there a changelog or any details for v1.0.1?

tomlj not found in maven central

This morning our builds started failing spuriously due to gradle being unable to resolve org.toml:tomlj:1.0.0 (spring-boot gradle plugin depends on it)
The artifact does not appear in maven central presently. Could you please re-publish it?
https://search.maven.org/search?q=org.toml

Status of project

It would be nice to clarify the overall status of this project.
Is it feature complete? Will there be future development or support?
Does it / will it support TOML 1.0.0 stable?

Map to TomlParseResult parser

Hello there!

I'm developing library that supports multiple configuration systems.
I'm willing to integrate it with this repo, cause it uses the most recent version of TOML

But for that, I need the method to parse Map<String, Object> to TomlParseResult.

Thanks in advance :)

tomlj-all missing on maven central

With 1.1.0 tomlj switched its dependencies from "compile" to "runtime". Some build systems, like ours, only bundle compile scope dependencies by default. We need to explicitly add tomlj dependencies to our build now. This means we need to take care of dependency management for tomlj now manually. Using tomlj-all instead would be an option, but it it is not deployed to maven as of now.

Would it be possible to deploy tomlj-all to maven central as well?

Thanks for providing this great library!

parsing issue

foo = 2bar this will be parsed as number 2.

I looked at the toml wiki, did not find explicit indicating of this is invalid or not

I am also asking the same question in here

Datetime array issues

Case 1 :-

Code:

	public static void main(String[] args) {

		TomlParseResult toml = null;
		try {
			Path source = Paths.get("D:/test/example.toml");
			toml = Toml.parse(source);
			TomlArray array = toml.getTable("array").getArray("key1");
			System.out.println("Count: " + array.size());
			for (int i = 0; i < array.size(); i++) {
				System.out.println("Value " + (i+1) + ": " + array.get(i));
			}
		} catch (Exception e) {
			e.printStackTrace();
		}
	}

Data:

[array]
key1 = [1979-05-27T00:32:00.999999-07:00, 1979-05-28T00:32:00.999999-07:00, 1979-05-31T07:32:00]

Result:

Count: 2
Value 1: 1979-05-27T00:32:00.999999-07:00
Value 2: 1979-05-28T00:32:00.999999-07:00

Debug shows error: "Unexpected '1979-05-31T07', expected ] or a newline" But no exception thrown.
Expected: Successful Execution

Got the below stacktrace:

java.lang.IllegalArgumentException: Error occured during parsing: Unexpected ']', expected ], a comma, or a newline
	at org.tomlj.Parser.parse(Parser.java:41)
	at org.tomlj.Toml.parse(Toml.java:80)
	at org.tomlj.Toml.parse(Toml.java:67)
	at Main.main(Main.java:16)
Caused by: Unexpected ']', expected ], a comma, or a newline (line 188, column 102)
	at org.tomlj.AccumulatingErrorListener.reportError(AccumulatingErrorListener.java:67)
	at org.tomlj.AccumulatingErrorListener.syntaxError(AccumulatingErrorListener.java:54)
	at org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:41)
	at org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:544)
	at org.antlr.v4.runtime.DefaultErrorStrategy.reportUnwantedToken(DefaultErrorStrategy.java:377)
	at org.antlr.v4.runtime.DefaultErrorStrategy.singleTokenDeletion(DefaultErrorStrategy.java:548)
	at org.antlr.v4.runtime.DefaultErrorStrategy.sync(DefaultErrorStrategy.java:266)
	at org.tomlj.internal.TomlParser.array(TomlParser.java:3011)
	at org.tomlj.internal.TomlParser.val(TomlParser.java:754)
	at org.tomlj.internal.TomlParser.keyval(TomlParser.java:409)
	at org.tomlj.internal.TomlParser.expression(TomlParser.java:298)
	at org.tomlj.internal.TomlParser.toml(TomlParser.java:219)
	at org.tomlj.Parser.parse(Parser.java:37)
	... 3 more

Case 2 :- - UPDATE: I understood how error handling is done. Consider Case 2 as SOLVED!

Code: Same as Case 1

Data:

[array]
key1 = [s1979-05-27T00:32:00.999999-07:00, s1979-05-28T00:32:00.999999-07:00, s1979-05-31T07:32:00]

Result:

Count: 1
Value 1: 1979-05-27T00:32:00.999999-07:00

Expected: ParserException

Case 3 :-

Code:

	public static void main(String[] args) {

		TomlParseResult toml = null;
		try {
			Path source = Paths.get("D:/test/example-v0.4.0.toml");
			toml = Toml.parse(source);
			TomlArray array = toml.getTable("array").getArray("key1");
			System.out.println("Count: " + array.size());
			System.out.println(array.containsLongs());
			System.out.println(array.containsStrings());
			System.out.println(array.containsBooleans());
			System.out.println(array.containsOffsetDateTimes());
			System.out.println(array.containsLocalDateTimes());
			System.out.println(array.containsLocalDates());
			System.out.println(array.containsLocalTimes());
			System.out.println(array.containsDoubles());
			for (int i = 0; i < array.size(); i++) {
				System.out.println("Value " + (i+1) + ": " + array.get(i));// + " -> " + TomlType.typeNameFor(array.get(i)));
			}
		} catch (Exception e) {
			e.printStackTrace();
		}
	}

Data:

[array]
key1 = [s1979-0x5-27T00:32:00.999999-07:00, s1979-05-28T00:32:00.999999-07:00, s1979-05-31T07:32:00]

Result:

Count: 0
true
true
true
true
true
true
true
true

Expected: ParserException. Boolean values to be false?

Can you please have a look?
Also, #3 was raised by me. I tweaked the source a bit and they seem to work as expected.
Can you please have a look at #3 and make/merge necessary changes?

Again, thank you for this lib! I'm not a java developer, nor an expert in Java. Found TOML and decided to play with it. If you get time, please take the below requests into consideration:

Requests:

(i) The reason I asked for maintaining sequence while creating JSON - want to implement JSON schema validation on this. In future, may be TOLS would be reality to validate the structure!
(ii) How about a toToml() similar to toJson()? Converts JSON to TOML.

Hope to see v1.0 implemented as well 👍

Version 1.0.0 doesn't support parsing of arrays that contain multiple data types

As per https://toml.io/en/v1.0.0-rc.1#array, this is valid TOML 1.0.0-rc.1:

[example]
contributors = [
  "Foo Bar <[email protected]>",
  { name = "Baz Qux", email = "[email protected]", url = "https://example.com/bazqux" }
]

But when parsing using "org.tomlj:tomlj:1.0.0", this array is not parsed:

TomlParseResult result = ...

println(result.get("example"))

What gets printed is a MutableTomlTable instance of size 0.

Another query:

println(result.get("example.contributors"))

This one just prints null.

java.lang.StackOverflowError: stack size 1039KB

hi,I met a problem,when toml file is too big!
`
java.lang.StackOverflowError: stack size 1039KB
at java.util.stream.Sink$ChainedReference.(Sink.java:250)
at java.util.stream.SortedOps$AbstractRefSortingSink.(SortedOps.java:310)
at java.util.stream.SortedOps$SizedRefSortingSink.(SortedOps.java:335)
at java.util.stream.SortedOps$OfRef.opWrapSink(SortedOps.java:141)
at java.util.stream.AbstractPipeline.wrapSink(AbstractPipeline.java:523)
at java.util.stream.StreamSpliterators$WrappingSpliterator.initPartialTraversalState(StreamSpliterators.java:293)
at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:158)
at java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:300)
at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
at org.tomlj.JsonSerializer.toJson(JsonSerializer.java:39)
at org.tomlj.JsonSerializer.appendTomlValue(JsonSerializer.java:112)
at org.tomlj.JsonSerializer.toJson(JsonSerializer.java:44)
...
at org.tomlj.JsonSerializer.toJson(JsonSerializer.java:29)
at org.tomlj.TomlTable.toJson(TomlTable.java:1173)
at org.tomlj.TomlTable.toJson(TomlTable.java:1158)

//my code:

 Path source = Paths.get(tomlPath);
    TomlParseResult result = null;
    try {
        result = Toml.parse(source);
    } catch (IOException e) {
        LogUtils.e(e.getMessage());
        addLoadErrorMsg("parse toml fial: "+e.getMessage());
        throw new TomlParseException(e.getMessage());
    }
    return result.toJson();

`
I,m want to fix it

Update Gradle Wrapper Version / Support Gradle 6

Building the project on Gradle 6 fails due to:

> Task :spotlessGroovyGradle FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':spotlessGroovyGradle'.
> You must declare outputs or use `TaskOutputs.upToDateWhen()` when using the incremental task API

Understandably, the Gradle wrapper exists for this reason. However, the Gradle wrapper used by tomlj is on Gradle 5.4

Gradle 5.4 doesn't work with JDK 15, so contributors / possible contributors have to switch to an older JDK version in order to build.

Publish to Maven Central?

With bintray going away on May 1, is there any chance of publishing tomlj to Maven Central?

ANTLR Version mismatch

Hello, I'm working on a project that uses ANTLR runtime v4.9.1, while Tomlj uses v4.7.2. This gives me the following error:

ANTLR Tool version 4.7.2 used for code generation does not match the current runtime version 4.9.1
ANTLR Runtime version 4.7.2 used for parser compilation does not match the current runtime version 4.9.1
ANTLR Tool version 4.7.2 used for code generation does not match the current runtime version 4.9.1
ANTLR Runtime version 4.7.2 used for parser compilation does not match the current runtime version 4.9.1

Note that the Tomlk code still runs, so perhaps the solution is to turn off this error somehow.

I have two options for a quick fix:

1. I could downgrade the ANTLR version in my project to match.
2. I could fork Tomlj and compile it locally with v4.9.1.

However, I'm wondering if there's a way to make this nicer for the future: perhaps the ANTLR dependency could be separated? I'm not quite sure how you've set your project up.

Project maintained?

It looks like development stopped some time ago.
Is there still an active maintainer? @cleishm?

toJson() tries to escape single quotes producing invalid JSON

The toJson() method escapes single quotes which don't need to be escaped in JSON.

TomlParseResult config = Toml.parse("key = \"this is 'a test' with single quotes\"");
System.out.println(config.toJson());

Actual result:

{
  "key" : "this is \'a test\' with single quotes"
}

Expected result:

{
  "key" : "this is 'a test' with single quotes"
}

toJson() produces invalid and out of sequence result

File used for testing: example-v0.4.0.toml one of your files in test resources

Code:

public static void main(String[] args) {

	TomlParseResult result = null;
	try {
		Path source = Paths.get("D:/test/example-v0.4.0.toml");
		result = Toml.parse(source);
		String value = result.toJson();
		System.out.println(value);
	} catch (Exception e) {
		System.err.println(e.getMessage());
	}
}

Request: I noticed the JSON generated is not in sequence with the file, rather it's generating in alphabetical order. Can we get that in the sequence we created the toml?
UPDATE: I made some minor changes in the JsonSerializer.java and MutableTomlTable.java to achieve the sequence. Changes I did:

JsonSerializer: Removed .stream().sorted() from Iterator<String> iterator.
MutableTomlTable: Changed properties = new LinkedHashMap<>();

Issue 1: singe quote "'" having escape character "'" which breaks the JSON validation. (line no. 111). I'm using online JSON validator
Just had a quick look at your JsonSerializer.java. May be this is causing the issue? I tried removing this part and it worked as expected.

if (ch == '\'') {
    out.append("\\'");
    continue;
}

Issue 2: double quote " not having escape character "". JSON validation failed. Same line number. Maybe you missed something like this? (Just a guess). I added the below code, it worked as expected.

if (ch == '\"') {
    out.append('\\');
    out.append('\"');
    continue;
}

Issue 3: Line no. 116 "regex2": "I [dw]on't need \d{2} apples" seems to be wrong. Maybe it would be "regex2": "I [dw]on't need \\d{2} apples" ? Same issue with single "" noticed in Line 119, 120, 221. Line 121 is "winpath2" : "\\ServerX\admin$\system32\", maybe it would be "winpath2" : "\\\\ServerX\\admin$\\system32\\" I added the below code, it worked as expected.

if (ch == '\\') {
    out.append('\\');
    out.append('\\');
    continue;
}

Thanks in advance for this! I'm not a java coder, pardon me for mistakes!
I found toml recently and decided to explore.

Will the version v1.0.0-rc.1 be implemented?

Parser failure when parsing inline tables with empty arrays

The parser fails to parse the following (valid) toml:

key = { value1 = 1, value2 = [] }

If i express this as a standard table instead of using an inline table, it works:

[key]
value1 = 1
value2 = []

Sample code snippet that fails:

import org.tomlj.Toml;
import org.tomlj.TomlParseResult;

public class TomlTest {
  public static void main(String[] args) throws Exception {
    String s = "key = { value1 = 1, value2 = [] }";
    TomlParseResult result = Toml.parse(s);
    if (result.hasErrors()) {
      System.out.println(result.errors().toString());
    } else {
      System.out.println("no errors");
    }
  }
}

The output is:

[Unexpected '}', expected } or a comma (line 1, column 33)]

I tried several combinations, but any time i included an empty array in an inline table, the parser fails with a similar error message.

tomlj gets confused with last comma in arrays

The following TOML document can't be parsed.

base64 = { version = "0.21", default-features = false, test = ["std", ], optional = true }

(Note the comma in the ["std" ,], taken from arrow-rs/parquet/Cargo.toml ).

Leading zeroes in exponent parts of floats

Hi @cleishm here's another case I found
Code:

String TOML = "key = 5e+22";
TomlParseResult toml = Toml.parse(TOML, throwiferror);
String result = toml.toJson();
System.out.println(result);

Result (as expected):

{  "key": 4.9999999999999996E22 }

But..

String TOML = "key = 5e+022";

Error: java.lang.IllegalArgumentException: A parsing error has occured: Unexpected '22', expected a newline or end-of-input

I saw the testcase, and for key = 5e22 the result is same as key = 5e+22
As per discussion here, key = 5e+022 should also be valid?

Noticed change in Lexer.g4:
fragment Exp : [eE] '0'* DecInt;
Should this be as below?
fragment Exp : [eE] [-+]? '0'* (Digit | Digit1_9 ('_'? Digit)+);

Toml serializer

I have a dependent configuration library which supports multiple configuration formats depending on the caller's choice. Toml has been requested a few times, but I still need the ability to write comments on the toml values at serialization time. Unfortunately, TomlTable#entrySet does not use any kind of "node" type, which makes it harder to provide additional metadata to toml values.

In the original issue for toml serialization, #16, it was suggested to add a type hierarchy for toml values to enable us to write toml effectively. Such a suggestion could be worked into a version 2 of this library. Comments could be attached to the objects mentioned:

Exposing something similar to Jackson/Gson/Hjson/Jankson's type hierarchy (JsonElement, JsonObject, JsonPrimitive) would be great too.

Throw an error when duplicate keys are encountered

According to the TOML spec, duplicate keys are not allowed, but this library silently ignores when there are duplicate keys, rather than producing an error. I think it would be more correct to throw an error.

Context: Gradle uses TOML for its version catalog files, but cannot produce an error when there are duplicate keys because this library does not allow detecting that.

How to WRITE something to a specific toml file?

Hello, I am looking for how to write a section (all data types of toml file) in to a specific .toml file, but I didn't find it. Are there any functions in this package that provide it?

tomlj incorrectly marked as a pom artifact

In the pom xml for the tomlj maven artifact, it is marked as a pom package, which is incorrect since tomj is not an aggregator artifact and has a jar. This is causing our maven resolving tool to skip the jar when resolving this artifact.

https://repo1.maven.org/maven2/org/tomlj/tomlj/1.0.0/tomlj-1.0.0.pom

<packaging>pom</packaging>

Maintainers wanted

I wrote TOMLJ at a time when I was using it fairly actively. I'd love to see it maintained, but that needs maintainers who are willing to submit pull requests. Seems a lot of people like the library, so please feel free to get involved 😀

TOML writer

Besides being able to read TOML files, it would be really useful to be able to write them as well. Exposing something similar to Jackson/Gson/Hjson/Jankson's type hierarchy (JsonElement, JsonObject, JsonPrimitive) would be great too.

Getting tables with spaces in their key doesn't seem to work

Given the following TOML:

[columns]

    [columns."Website 1"]
        module = "Person"
        target = "group"
        group = "PerContactGrp"
        field = "ValueTxt"
        qualifier = 1

Trying to do:

def toml = Toml.parse(tomlFile.toPath())
println toml.getTable("columns").getTable("Website 1").keySet()

Throws an error:

java.lang.IllegalArgumentException: Invalid key: Unexpected '1', expected end-of-input
	at org.tomlj.Parser.parseDottedKey(Parser.java:96)
	at org.tomlj.TomlTable.getTable(TomlTable.java:1085)
Caused by: Unexpected '1', expected end-of-input (line 1, column 9)
	at org.tomlj.AccumulatingErrorListener.reportError(AccumulatingErrorListener.java:67)
	at org.tomlj.AccumulatingErrorListener.syntaxError(AccumulatingErrorListener.java:54)
	at org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:41)
	at org.antlr.v4.runtime.Parser.notifyErrorListeners(Parser.java:544)
	at org.antlr.v4.runtime.DefaultErrorStrategy.reportUnwantedToken(DefaultErrorStrategy.java:377)
	at org.antlr.v4.runtime.DefaultErrorStrategy.singleTokenDeletion(DefaultErrorStrategy.java:548)
	at org.antlr.v4.runtime.DefaultErrorStrategy.recoverInline(DefaultErrorStrategy.java:467)
	at org.antlr.v4.runtime.Parser.match(Parser.java:206)
	at org.tomlj.internal.TomlParser.tomlKey(TomlParser.java:357)
	at org.tomlj.Parser.parseDottedKey(Parser.java:92)
	... 3 more

If I remove the space in the table name (i.e. "Website1"), it works.