Coder Social home page Coder Social logo

tomli's Introduction

Build Status codecov.io PyPI version

Tomli

A lil' TOML parser

Table of Contents generated with mdformat-toc

Intro

Tomli is a Python library for parsing TOML. It is fully compatible with TOML v1.0.0.

A version of Tomli, the tomllib module, was added to the standard library in Python 3.11 via PEP 680. Tomli continues to provide a backport on PyPI for Python versions where the standard library module is not available and that have not yet reached their end-of-life.

Installation

pip install tomli

Usage

Parse a TOML string

import tomli

toml_str = """
[[players]]
name = "Lehtinen"
number = 26

[[players]]
name = "Numminen"
number = 27
"""

toml_dict = tomli.loads(toml_str)
assert toml_dict == {
    "players": [{"name": "Lehtinen", "number": 26}, {"name": "Numminen", "number": 27}]
}

Parse a TOML file

import tomli

with open("path_to_file/conf.toml", "rb") as f:
    toml_dict = tomli.load(f)

The file must be opened in binary mode (with the "rb" flag). Binary mode will enforce decoding the file as UTF-8 with universal newlines disabled, both of which are required to correctly parse TOML.

Handle invalid TOML

import tomli

try:
    toml_dict = tomli.loads("]] this is invalid TOML [[")
except tomli.TOMLDecodeError:
    print("Yep, definitely not valid.")

Note that error messages are considered informational only. They should not be assumed to stay constant across Tomli versions.

Construct decimal.Decimals from TOML floats

from decimal import Decimal
import tomli

toml_dict = tomli.loads("precision-matters = 0.982492", parse_float=Decimal)
assert isinstance(toml_dict["precision-matters"], Decimal)
assert toml_dict["precision-matters"] == Decimal("0.982492")

Note that decimal.Decimal can be replaced with another callable that converts a TOML float from string to a Python type. The decimal.Decimal is, however, a practical choice for use cases where float inaccuracies can not be tolerated.

Illegal types are dict and list, and their subtypes. A ValueError will be raised if parse_float produces illegal types.

Building a tomli/tomllib compatibility layer

Python versions 3.11+ ship with a version of Tomli: the tomllib standard library module. To build code that uses the standard library if available, but still works seamlessly with Python 3.6+, do the following.

Instead of a hard Tomli dependency, use the following dependency specifier to only require Tomli when the standard library module is not available:

tomli >= 1.1.0 ; python_version < "3.11"

Then, in your code, import a TOML parser using the following fallback mechanism:

import sys

if sys.version_info >= (3, 11):
    import tomllib
else:
    import tomli as tomllib

tomllib.loads("['This parses fine with Python 3.6+']")

FAQ

Why this parser?

  • it's lil'
  • pure Python with zero dependencies
  • the fastest pure Python parser *: 16x as fast as tomlkit, 2.3x as fast as toml
  • outputs basic data types only
  • 100% spec compliant: passes all tests in BurntSushi/toml-test test suite
  • thoroughly tested: 100% branch coverage

Is comment preserving round-trip parsing supported?

No.

The tomli.loads function returns a plain dict that is populated with builtin types and types from the standard library only. Preserving comments requires a custom type to be returned so will not be supported, at least not by the tomli.loads and tomli.load functions.

Look into TOML Kit if preservation of style is what you need.

Is there a dumps, write or encode function?

Tomli-W is the write-only counterpart of Tomli, providing dump and dumps functions.

The core library does not include write capability, as most TOML use cases are read-only, and Tomli intends to be minimal.

How do TOML types map into Python types?

TOML type Python type Details
Document Root dict
Key str
String str
Integer int
Float float
Boolean bool
Offset Date-Time datetime.datetime tzinfo attribute set to an instance of datetime.timezone
Local Date-Time datetime.datetime tzinfo attribute set to None
Local Date datetime.date
Local Time datetime.time
Array list
Table dict
Inline Table dict

Performance

The benchmark/ folder in this repository contains a performance benchmark for comparing the various Python TOML parsers. The benchmark can be run with tox -e benchmark-pypi. Running the benchmark on my personal computer output the following:

foo@bar:~/dev/tomli$ tox -e benchmark-pypi
benchmark-pypi installed: attrs==21.4.0,click==8.0.3,pytomlpp==1.0.10,qtoml==0.3.1,rtoml==0.7.1,toml==0.10.2,tomli==2.0.1,tomlkit==0.9.2
benchmark-pypi run-test-pre: PYTHONHASHSEED='3088452573'
benchmark-pypi run-test: commands[0] | python -c 'import datetime; print(datetime.date.today())'
2022-02-09
benchmark-pypi run-test: commands[1] | python --version
Python 3.8.10
benchmark-pypi run-test: commands[2] | python benchmark/run.py
Parsing data.toml 5000 times:
------------------------------------------------------
    parser |  exec time | performance (more is better)
-----------+------------+-----------------------------
     rtoml |    0.891 s | baseline (100%)
  pytomlpp |    0.969 s | 91.90%
     tomli |        4 s | 22.25%
      toml |     9.01 s | 9.88%
     qtoml |     11.1 s | 8.05%
   tomlkit |       63 s | 1.41%

The parsers are ordered from fastest to slowest, using the fastest parser as baseline. Tomli performed the best out of all pure Python TOML parsers, losing only to pytomlpp (wraps C++) and rtoml (wraps Rust).

tomli's People

Contributors

dimitripapadopoulos avatar goodmami avatar hauntsaninja avatar hugovk avatar hukkin avatar hukkinj1 avatar layday avatar ofek avatar oittaa avatar pre-commit-ci[bot] avatar takluyver avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tomli's Issues

`tox -e benchmark` ModuleNotFoundError: No module named 'setuptools_rust'

This might be an issue with rtoml, but the benchmark fails with the following error on MacOS:

ModuleNotFoundError: No module named 'setuptools_rust'

Workaround is to remove rtoml from benchmark/requirements.txt and install it after setuptools-rust.

$ git diff
diff --git a/benchmark/requirements.txt b/benchmark/requirements.txt
index 2433985..4bff435 100644
--- a/benchmark/requirements.txt
+++ b/benchmark/requirements.txt
@@ -6,4 +6,3 @@ pytomlpp
 toml
 tomlkit
 qtoml
-rtoml
diff --git a/pyproject.toml b/pyproject.toml
index 0aa803f..4c9d2fb 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -96,6 +96,8 @@ description = run the benchmark script against a local Tomli version
 deps =
     -r benchmark/requirements.txt
 commands =
+    pip3 install setuptools-rust
+    pip3 install rtoml
     python -c 'import datetime; print(datetime.date.today())'
     python --version
     python benchmark/run.py

A test case to add

Add this test case once the spec clarifies expected outcome:

[[tab.arr]]
[tab]
arr.val1=1

Question: tomli.load doesn't work with pathlib.read_bytes?

It seems that I can do this:

with open("./config.toml", "rb") as f:
    tomli.load(f)   

But not this:

f = Path("./config.toml").read_bytes()
tomli.load(f)   

Error message:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/var/folders/jc/rnc4j5c90sx162px6x2gl33h0000gp/T/ipykernel_86569/1174330575.py in <module>
      1 f = Path("./config.toml").read_bytes()
----> 2 tomli.load(f)

/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/tomli/_parser.py in load(__fp, parse_float)
     57 def load(__fp: BinaryIO, *, parse_float: ParseFloat = float) -> dict[str, Any]:
     58     """Parse TOML from a binary file object."""
---> 59     b = __fp.read()
     60     try:
     61         s = b.decode()

AttributeError: 'bytes' object has no attribute 'read'

I assume the pathlib.read_bytes method would be equivalent to with open(path, "rb"). Any idea why the pathlib method doesn't work with tomli?

The latest recommendations from Python is to use pathlib over the with open method, so would be nice if this works. Thanks!

Strange behavior of single quotes

When parsing the follow file:
a = "b\nc" d = 'e\nf' "g\nh" = 1 'i\nj' = 2
i have the result:
{'a': 'b\nc', 'd': 'e\\nf', 'g\nh': 1, 'i\\nj': 2}

  1. In the documentation, I did not find that the value of the string type can be wrapped in single quotes.In the documentation, I did not find that the value of the string type can be wrapped in single quotes.
  2. Why is the result of escaping characters in different quotes different?

Better error messages

I was just testing this on a malformed toml file:

[tool.a]
hi = 1
ho = 2
[tool.*]
ho = 2
he = 3

toml gave a nice error message:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/henryschreiner/git/software/cibuildwheel/venv/lib/python3.9/site-packages/toml/decoder.py", line 134, in load
    return loads(ffile.read(), _dict, decoder)
  File "/Users/henryschreiner/git/software/cibuildwheel/venv/lib/python3.9/site-packages/toml/decoder.py", line 455, in loads
    raise TomlDecodeError("Invalid group name '" +
toml.decoder.TomlDecodeError: Invalid group name '*'. Try quoting it. (line 4 column 1 char 23)

But tomli just gave:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/henryschreiner/git/software/cibuildwheel/venv/lib/python3.9/site-packages/tomli/_parser.py", line 53, in load
    return loads(s, parse_float=parse_float)
  File "/Users/henryschreiner/git/software/cibuildwheel/venv/lib/python3.9/site-packages/tomli/_parser.py", line 91, in loads
    create_dict_rule(state)
  File "/Users/henryschreiner/git/software/cibuildwheel/venv/lib/python3.9/site-packages/tomli/_parser.py", line 229, in create_dict_rule
    key = parse_key(state)
  File "/Users/henryschreiner/git/software/cibuildwheel/venv/lib/python3.9/site-packages/tomli/_parser.py", line 309, in parse_key
    key.append(parse_key_part(state))
  File "/Users/henryschreiner/git/software/cibuildwheel/venv/lib/python3.9/site-packages/tomli/_parser.py", line 328, in parse_key_part
    raise TOMLDecodeError("Invalid key definition")
tomli._parser.TOMLDecodeError: Invalid key definition

Unlike toml, there's no location for the error, and no indication of what failed either.

TOMLDecodeError: Invalid initial character for a key part

I noticed tomli is stricter than toml when it comes to whitespaces:

$ cat sample.toml
digests = [
  {
    key = "value",
    key2 = "value2"
  }
]
$ In [4]: toml.load('sample.toml')
Out[4]: {'digests': [{'key': 'value', 'key2': 'value2'}]}
$ In [5]: tomli.load(open('sample.toml', 'rb'))
...
    403     return parse_one_line_basic_str(src, pos)
--> 404 raise suffixed_err(src, pos, "Invalid initial character for a key part")

TOMLDecodeError: Invalid initial character for a key part (at line 2, column 4)

Is it something that can be relaxed, please?

AttributeError: when running black.

Python Version: 3.10.0

tomli==2.0.0

Traceback (most recent call last):
  File "/Users/XYZ/venv/bin/black", line 8, in <module>
    sys.exit(patched_main())
  File "/Users/XYZ/venv/lib/python3.10/site-packages/black/__init__.py", line 1130, in patched_main
    main()
  File "/Users/XYZ/venv/lib/python3.10/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/Users/XYZ/venv/lib/python3.10/site-packages/click/core.py", line 1061, in main
    with self.make_context(prog_name, args, **extra) as ctx:
  File "/Users/XYZ/venv/lib/python3.10/site-packages/click/core.py", line 923, in make_context
    self.parse_args(ctx, args)
  File "/Users/XYZ/venv/lib/python3.10/site-packages/click/core.py", line 1379, in parse_args
    value, args = param.handle_parse_result(ctx, opts, args)
  File "/Users/XYZ/venv/lib/python3.10/site-packages/click/core.py", line 2364, in handle_parse_result
    value = self.process_value(ctx, value)
  File "/Users/XYZ/venv/lib/python3.10/site-packages/click/core.py", line 2326, in process_value
    value = self.callback(ctx, self, value)
  File "/Users/XYZ/venv/lib/python3.10/site-packages/black/__init__.py", line 105, in read_pyproject_toml
    config = parse_pyproject_toml(value)
  File "/Users/XYZ/venv/lib/python3.10/site-packages/black/files.py", line 95, in parse_pyproject_toml
    pyproject_toml = tomli.load(f)
  File "/Users/XYZ/venv/lib/python3.10/site-packages/tomli/_parser.py", line 55, in load
    s = __fp.read().decode()
AttributeError: 'str' object has no attribute 'decode'. Did you mean: 'encode'?

mypy errors with suggested `tomli`/`tomllib` compatibility layer

Hello, I'm attempting to write forward-looking code which uses the documented tomli/tomllib compatibility layer:

try:
    import tomllib
except ModuleNotFoundError:
    import tomli as tomllib

result = tomllib.loads("")
reveal_type(result)

When I check this code with mypy, I am met with the following errors:
(Python 3.10.9, mypy 0.991)

$ mypy example.py
example.py:2: error: Cannot find implementation or library stub for module named "tomllib"  [import]
example.py:2: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
example.py:4: error: Name "tomllib" already defined (possibly by an import)  [no-redef]
example.py:7: note: Revealed type is "Any"
Found 2 errors in 1 file (checked 1 source file
  • Line 2: The tomllib import error is expected since I am running Python 3.10 (and it was added to the standard lib in 3.11). This is resolved by ignoring the import error.

  • Line 4: The Name "tomllib" already defined error is resolved by upgrading to mypy 1.2.0 or ingoring the redef error.

try:
    import tomllib  # type: ignore[import]
except ModuleNotFoundError:
    import tomli as tomllib  # type: ignore[no-redef]

result = tomllib.loads("")
reveal_type(result)
$ mypy example.py
example.py:7: note: Revealed type is "Any"
Success: no issues found in 1 source file
  • Line 7: This is the unfortunate part. When using the prescribed pattern, tomllib is Any, so all functions and their return values are Any as well.

When the compatibility layer is not used, the typing is correctly provided (presumably via typeshed, although I can't find it?)

import tomli
reveal_type(tomli)

result = tomli.loads("")
reveal_type(result)
$ mypy example_nolayer.py
example_nolayer.py:2: note: Revealed type is "types.ModuleType"
example_nolayer.py:5: note: Revealed type is "builtins.dict[builtins.str, Any]"

Is this a limitation of mypy that we have to live with, or can the pattern be somehow updated to resolve these issues?

Why splitting the library into two?

Hi! I like the idea of having a simple pure Python library for TOML! And I understand the motivation of keeping it minimal, however, I am not sure if splitting the package into two is the optimal approach. I would like to share my thoughts on that.

  1. If I am a user that does NOT need dump(s), what is the benefit of NOT having dump(s)? By looking at tomli-w, it seems that the benefit is the absence of ~150 lines of Python code. If it is true, I personally do not see use cases where this benefit can be important. I understand the goal of "theoretically minimal library". Are you sure that this goal is practical?
  2. If I am a user that needs dump(s), I have to install two packages instead of one and import two different packages instead of one. This setup does not feel minimal to me (unfortunately, this is exactly my case).
  3. In fact, tomli is not minimal: loads can be replaced by using StringIO and load :) While this is rather a joke, I would like to say that the "minimalism", when not absolute, becomes a subjective thing.

Taking all this into account, merging the two libraries into one looks good to me. What do you think?

Python 3.6 is not supported anymore

The doc states:

To build code that uses the standard library if available, but still works seamlessly with Python 3.6+, do the following.

but you bumped requirements to Python 3.7+ in 6a93a19

I wished this could still run on Python 3.6 as advertized which is still supported on many Linux distros.

Parse `decimal.Decimal`

Add a use_decimal flag or some other way to parse TOML floats into Decimals instead of floats. Make sure that no lossy conversion to float ever happens. Go straight from str to Decimal.

Slackware script

I'd like to build a package for slackware64-current, but I need the setup.py file.

Include setup.py in sdist

I noticed setup.py was removed here, however since tomli is a required dependency for bootstrapping flit this should be included to make installing flit easier for distros that need it for setting up a PEP-517 build environment.

toml compatibility - load('some.toml') fails

I am migrating from toml. And tomli claims full compatibility. But with toml I was able to:

load('some.toml')

but with tomli I get:

tomli.load('some.toml')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.11/site-packages/tomli/_parser.py", line 59, in load
    b = __fp.read()
        ^^^^^^^^^
AttributeError: 'str' object has no attribute 'read'

Bug: cyclic dependency with `tomli==1.2.1` and `flit_core==3.2.0`

I can not download tomli because there is a cyclic dependency with flit_core.

Reproducer:

python3.9 -m venv test
./test/bin/pip download --no-binary=":all:"  tomli  # fails
./test/bin/pip download --no-binary=":all:" flit_core # fails

Example output:

Collecting tomli
  File was already downloaded XXX/tomli-1.2.1.tar.gz
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 2:
   command: XXX/test/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-91g152mw/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'flit_core>=3.2.0,<4'
       cwd: None
  Complete output (31 lines):
  Collecting flit_core<4,>=3.2.0
    Downloading flit_core-3.4.0.tar.gz (27 kB)
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
      Preparing wheel metadata: started
      Preparing wheel metadata: finished with status 'done'
  Collecting tomli
    Downloading tomli-1.2.1.tar.gz (14 kB)
  ERROR: Exception:
  Traceback (most recent call last):
    File "XXX/test/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
      status = self.run(options, args)
    File "XXX/test/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 357, in run
      resolver.resolve(requirement_set)
    File "XXX/test/lib/python3.9/site-packages/pip/_internal/legacy_resolve.py", line 177, in resolve
      discovered_reqs.extend(self._resolve_one(requirement_set, req))
    File "XXX/test/lib/python3.9/site-packages/pip/_internal/legacy_resolve.py", line 333, in _resolve_one
      abstract_dist = self._get_abstract_dist_for(req_to_install)
    File "XXX/test/lib/python3.9/site-packages/pip/_internal/legacy_resolve.py", line 282, in _get_abstract_dist_for
      abstract_dist = self.preparer.prepare_linked_requirement(req)
    File "XXX/test/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 515, in prepare_linked_requirement
      abstract_dist = _get_prepared_distribution(
    File "XXX/test/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 94, in _get_prepared_distribution
      with req_tracker.track(req):
    File "/usr/lib/python3.9/contextlib.py", line 117, in __enter__
      return next(self.gen)
    File "XXX/test/lib/python3.9/site-packages/pip/_internal/req/req_tracker.py", line 148, in track
      self.add(req)
    File "XXX/test/lib/python3.9/site-packages/pip/_internal/req/req_tracker.py", line 115, in add
      raise LookupError(message)
  LookupError: https://files.pythonhosted.org/packages/75/50/973397c5ba854445bcc396b593b5db1958da6ab8d665b27397daa1497018/tomli-1.2.1.tar.gz#sha256=a5b75cb6f3968abb47af1b40c1819dc519ea82bcc065776a866e8d74c5ca9442 (from https://pypi.org/simple/tomli/) (requires-python:>=3.6) is already being built: tomli from https://files.pythonhosted.org/packages/75/50/973397c5ba854445bcc396b593b5db1958da6ab8d665b27397daa1497018/tomli-1.2.1.tar.gz#sha256=a5b75cb6f3968abb47af1b40c1819dc519ea82bcc065776a866e8d74c5ca9442
  ----------------------------------------
ERROR: Command errored out with exit status 2: XXX/test/bin/python3.9 XXX/test/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-91g152mw/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'flit_core>=3.2.0,<4' Check the logs for full command output.

Type stub incompatiblity with tarfile.ExFileObject (i.e. tarfile.TarFile.extractfile() result)

Please consider the following snippet:

import tomli
import tarfile

with tarfile.open("test.tar") as tar:
    f = tar.extractfile("test.toml")
    assert f is not None
    tomli.load(f)

When I run mypy on it, I get:

test.py:7: error: Argument 1 to "load" has incompatible type "IO[bytes]"; expected "BinaryIO"  [arg-type]
Found 1 error in 1 file (checked 1 source file)

To be honest, typing docs don't really say what the difference between IO[bytes] and BinaryIO is. Besides type stubs, the code works. FWICS, tomllib would suffer from the same problem if not for the fact that mypy overrides its type stub to:

def load(__fp: SupportsRead[bytes], *, parse_float: Callable[[str], Any] = ...) -> dict[str, Any]: ...

Custom `datetime` parsers?

datetime in standard library has a microsecond precision, but for example numpy.datetime64 has a nanosecond precision. It would be pretty easy to add another parameter parse_datetime like the current parse_float. Even something a bit more complex like the custom JSON encoders and decoders might be useful. Or do you think custom parsers would complicate things too much?

Lower `z` in datetime should be valid

import rtoml
>>> rtoml.loads('t = 1987-07-05t17:45:00z')
{'t': datetime.datetime(1987, 7, 5, 17, 45, tzinfo=<builtins.TzClass object at 0x7f1892986310>)}
import tomli
>>> tomli.loads('t = 1987-07-05t17:45:00z')
Traceback (most recent call last):
  File "<input>", line 1, in <module>
    tomli.loads('t = 1987-07-05t17:45:00z')
  File "/path/to/python3.9/site-packages/tomli/_parser.py", line 136, in loads
    raise suffixed_err(
tomli._parser.TOMLDecodeError: Expected newline or end of document after a statement (at line 1, c
olumn 15)

References:

tomli violates PEP517 Build Requirements

The PEP517 Build Requirements specifically disallow dependency cycles:

  • Project build requirements will define a directed graph of requirements (project A needs B to build, B needs C and D, etc.) This graph MUST NOT contain cycles. If (due to lack of co-ordination between projects, for example) a cycle is present, front ends MAY refuse to build the project.

  • Where build requirements are available as wheels, front ends SHOULD use these where practical, to avoid deeply nested builds. However front ends MAY have modes where they do not consider wheels when locating build requirements, and so projects MUST NOT assume that publishing wheels is sufficient to break a requirement cycle.

Support reading TOML from a filename.

Both uiri/toml and samuelcolvin/rtoml support filenames in their load functions. They are then responsible for opening the file with the correct encoding and reading its contents, before calling loads.

However, tomli only supports file-like objects, leaving it to the user to open the file and, inevitably, forgetting to use UTF-8 as the encoding (e.g. https://github.com/pulls?q=is%3Apr+author%3Adomdfcoding+UTF-8).

I propose copying toml and rtoml and adding support for strings and os.PathLike objects. That way the user can write:

config = tomli.load("pyproject.toml")

or

config = tomli.load(pathlib.Path("pyproject.toml"))

without having to worry about the encoding, and saving a line over the with open approach.


What do you think? If you're happy with the idea I'll submit a PR

Not allowing `open(file, "r", encoding=...)` is problematic for `tomli.load()`

# Create a file with unicode content
with open(datafile, "w", encoding="GBK") as f:
    f.write('"你好" = "世界"\n')

# Use `tomli.load()` to load the file
with open(datafile, "r", encoding="GBK") as f:
    loaded = tomli.load(f)
# TypeError: File must be opened in binary mode

with open(datafile, "rb") as f:
    loaded = tomli.load(f)
# UnicodeDecodeError('utf-8', b'"\xc4\xe3\xba\xc3" = "\xca\xc0\xbd\xe7"\n', 1, 2, 'invalid continuation byte')

We can only use tomli.loads() as a workaround.

See:
https://github.com/pwwang/toml-bench#testloadunicode

cannot install tomli by pip-compile

I used to manage dependency by pip-compile and requirements.in .
But I can not install tomli via pip-compile.
$ pip-compile
Could not find a version that matches tomli (from -r requirements.in (line 4))
No versions found
Was https://pypi.org/simple reachable?
$ python3
Python 3.9.6 (default, Aug 15 2021, 21:24:28) `

Incorrect error message

I ran across an incorrect error message while using tomli.loads. See the snippet below

import tomli
>>> tomli_str = b'test="a"'
>>> tomli.loads(tomli_str)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "venv\lib\site-packages\tomli\_parser.py", line 74, in loads
    src = __s.replace("\r\n", "\n")
TypeError: a bytes-like object is required, not 'str'

Note the actual text of of the type error. It indicates it wants "a bytes-like object is required, not 'str'". Except it was passed a bytes like object and consulting the documentation loads actually wants a str object. So error message text is wrong and in fact opposite of what the function actually wants.

I generated this error using tomli 2.0.1

MicroPython Support

I've made a fork of tomli that supports micropython:

https://github.com/BrianPugh/micropython-tomli

A lot of the changes make the codebase worse from a cpython perspective (removed type hints, uglier regex, etc etc), so I'm not looking to make a PR or anything. I was wondering if you'd be interested in linking my fork from your readme? If you think micropython may be too niche, I also understand not wanting to clutter the readme.

Thanks!

Looking to help

Tomli seems really well put together. I want to make sure it gets maintained. Let me know if there's anything I can do 🙂

Alias for TOMLDecodeError → TomlDecodeError

Hello.

We are writing tool called micropipenv (https://github.com/thoth-station/micropipenv) and we are trying to make it compatible with various toml backends.

Would it make sense to create an alias for TOMLDecodeError - TomlDecodeError to make it the same as it is in this implementation: https://github.com/uiri/toml/blob/3f637dba5f68db63d4b30967fedda51c82459471/toml/__init__.py#L15 ?

After that, it'd be possible to import toml/tomli and use the same exception name no matter what implementation is available.

What do you think about it?

ModuleNotFoundError: No module named 'tomli'

As the title says. I have triple checked that I have install tomli, but it still doesn't work.

Traceback:

Traceback (most recent call last):
  File "C:\Users\pitch\Desktop\Undefined-bot\general-commands.py", line 2, in <module>
    import tomllib
ModuleNotFoundError: No module named 'tomllib'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\pitch\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\discord\ext\commands\bot.py", line 606, in _load_from_module_spec
    spec.loader.exec_module(lib)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\Users\pitch\Desktop\Undefined-bot\general-commands.py", line 4, in <module>
    import tomli as tomllib
ModuleNotFoundError: No module named 'tomli'
Traceback (most recent call last):
  File "C:\Users\pitch\Desktop\Undefined-bot\main.py", line 31, in <module>
    bot.load_extension(extension)
  File "C:\Users\pitch\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\discord\ext\commands\bot.py", line 678, in load_extension
    self._load_from_module_spec(spec, name)
  File "C:\Users\pitch\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\discord\ext\commands\bot.py", line 609, in _load_from_ionmodule_spec
    raise errors.ExtensionFailed(key, e) from e                                                                                                                                                          module_spec
discord.ext.commands.errors.ExtensionFailed: Extension 'general-commands' raised an error: ModuleNotFoundError: No module named 'tomli'

I am also using the default python virtual environment (python -m venv project-venv)

Please include tests in the pypi tarball

On Gentoo we really want to let our users run tests before installing the package. Normally, we just use the autogenerated GitHub archives for that. However, since tomli uses pyproject.toml, this will effectively create a cyclic dependency between flit and itself once flit switches over from toml (and it already creates one for the Gentoo build tool).

Could you please include the necessary test files in the generated pypi tarballs? Since they include a backwards compatible setup.py script, they would really help us avoid cyclic dependencies.

On Windows only: TOML parsing error: "Couldn't read config file pyproject.toml: Unescaped '\' in a string"

I wrote out the details at nedbat/coveragepy#1481, as the problem was encountered during use of coveragepy in a Windows CI run. I don't have a local Windows environment handy to test with.

The meat of it is, I have this entry in a TOML file, and I think it's valid, and it parses fine on Linux and MacOS, but not Windows, using tomli 2.0.1:

install = "if [ $VIRTUAL_ENV ]; then pip install -r local-requirements.txt; else printf '%s\n' 'Please activate a venv first'; return 1; fi"

Result on Windows:

Couldn't read config file pyproject.toml: Unescaped '\' in a string (at line 62, column 21)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.