pyproject.toml: validation error since setuptools 61.2.0 about soupsieve

[priv-kweihmann]

Thanks again for the feedback - I played around a bit with hatchling and was able to come up with the needed code to make it work on the yocto end of things - so there's nothing to be done here.

I personally believe it was a bad idea to strictly require packages to pass validate-pyproject, especially when it is still noted as an experimental project that currently has not widened their scope to fully encompass PEP compliant backends such as Hatchling. If we are talking about more complex packages that utilize C modules and such, there is no way that they will chose flit-core, they will chose something like Hatchling. Yocto will be left behind in such cases.

I'm totally with you on this one - basically this move makes a ton of packages fail for no real reason.

I will close the issue

[facelessuser]

What do you mean by impossible to build a wheel? The project current builds just fine.

We currently have setuptools-62.3.2 installed. More importantly, we have build-0.7.0 installed, which we use to build wheels.

➜  soupsieve git:(main) ✗ python3 -m build
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (hatchling>=0.21.1)
* Getting dependencies for sdist...
* Building sdist...
* Building wheel from sdist
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (hatchling>=0.21.1)
* Getting dependencies for wheel...
* Building wheel...
Successfully built soupsieve-2.3.2.post1.tar.gz and soupsieve-2.3.2.post1-py3-none-any.whl

Installing directly from PyPI results in no issues:

➜  github python3 -m pip install -U soupsieve
Collecting soupsieve
  Using cached soupsieve-2.3.2.post1-py3-none-any.whl (37 kB)
Installing collected packages: soupsieve
  Attempting uninstall: soupsieve
    Found existing installation: soupsieve 2.3.1
    Uninstalling soupsieve-2.3.1:
      Successfully uninstalled soupsieve-2.3.1
Successfully installed soupsieve-2.3.2.post1

There is nothing "requiring" that the tool should be run on the toml to build the wheel. There is nothing stopping wheel building or installation except a user manually running this tool.

Additionally, the tool is very unhelpful with its error. What does this even mean?

Invalid file: pyproject.toml
[ERROR] `project.license` must be valid exactly by one definition (2 matches found):

    - keys:
        'file': {type: string}
      required: ['file']
    - keys:
        'text': {type: string}
      required: ['text']

We only specify our license once.

license = "MIT"

So, what is it complaining about? No line number, no info pointing to an actual invalid reference, nothing extremely helpful to guide compliance.

[facelessuser]

Just to reiterate, wheel building seems to work just fine without any actual breakage, even with the latest setuptools installed. I see nothing here requiring any actual changes.

As far as compliance goes, if you can point out what validate_pyproject doesn't like, because it cannot seem to do so itself in a helpful manner, I would at least consider making the change or accepting a pull request.

[facelessuser]

I've double-checked the documentation. It seems validate-pyproject is a requirement for tomli, which they seem to be using, but nowhere in the documentation does it claim a project's toml must manually run validate-pyproject on their toml. And since installing via pip still works, and the suggested building of wheels via PYPA's own latest build still works, without issues, I see nothing required for us in this request.

I'd still like to understand why it complains with some cryptic error message when validate-pyproject is run on our toml, but I see no reason to elevate this urgency.

[facelessuser]

You know, I think I know what the issue is. validate-pyproject is most likely validating pyproject.toml files from the perspective of setuptools, but we do not use setuptools as a build backed anymore.

We use https://github.com/pypa/hatch as our build backend now, so it is not helpful to validate things for setuptools. Please discontinue using validate-pyproject on our toml unless you are able to show that it works for and is required for non-setuptools based build backends.

[facelessuser]

Here's the other telling thing from validate-pyproject's own readme:

This project is experimental and under active development. Issue reports and contributions are very welcome.

If this is experimental, I have no intention of going out of my way to adhere to it especially when it is not enforced.

[priv-kweihmann]

First of all thank you for your elaborate feedback.
I had deeper look into the issue.

I first encountered the issue with the Yocto/bitbake build system, after there was an upgrade from setuptools 58.x to 62.x.
As of now yocto only supports flit-core as a buildsystem (not sure if hatchling will be supported in the future).

It turns out that flit enforces the new PEP621 metadata standard quite strictly (maybe just in the yocto setup, I'm not entirely sure, will have to check that).
The mentioned error could be easily fixed with the following patch

-license = "MIT"
+license = { file = "LICENSE.md" }

which works with hatchling and the yocto implementation.
Sadly in addition I get the following error after applying the patch

| distutils.errors.DistutilsOptionError: No configuration found for dynamic 'classifiers'.
| Some dynamic fields need to be specified via `tool.setuptools.dynamic`

Not sure what to make out of this, as I lack the insight of what hatchling does here differently than other build tools.
https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html#dynamic-metadata suggests to use an extra multiline file to point to the classifiers - but I honestly don't really like that.

Easily I could come up with the needed patch for the license field, but not sure what to make out of the dynamic classifiers.
What would be your preferred way of solving this issue?

[facelessuser]

Here's the thing. We are trying to fix issues that are not issues here. I use hatchling instead of flit-core because of the ability to use dynamic classifiers and such. Also, Hatchling is compliant with many of the recent PEPs regarding packaging:

Project metadata is based entirely on PEP 621/PEP 631, the build system is compatible with PEP 517/PEP 660, versioning uses the scheme specified by PEP 440, dependencies are defined with PEP 508 strings, and environments use virtualenv.

Not only that, Hatchling is maintained by the PYPA (Python Packaging Authority) just like flit-core. They both aim to solve different issues. If you have a very simplistic, non-dynamic setup, then flit-core is a fine choice: https://flit.pypa.io/en/latest/rationale.html#why-use-flit. But, if you have a more complex environment that requires more dynamic options or you are building a non-pure Python package, such as a package that links with C code, flit-core cannot be used.

Even other PYPA projects have started to use Hatchling using the same notation I do with Licenses, see pipx as an example: https://github.com/pypa/pipx/blob/main/pyproject.toml#L9.

I support numerous packages. I selected Hatchling because some of my projects have some complex steps. SoupSieve is less complex, but I do use some dynamic classifiers to make my life easier, but packages like Rummage require me to build language files for translations during installation, and flit-core doesn't cut it. I have no intention to use more than one build backend on various projects either, so I've chosen Hatchling for all of my projects.

I first encountered the issue with the Yocto/bitbake build system, after there was an upgrade from setuptools 58.x to 62.x.
As of now yocto only supports flit-core as a buildsystem (not sure if hatchling will be supported in the future).

I understand, but yocto currently has a very narrow view of the packaging ecosystem right now. I admit that hatchling is kind of new, so I don't expect that every system is prepared for Hatchling right now, but there will be more packages migrating to Hatchling outside of the packages I maintain. Big packages like Tox are started to move towards it as well. If yocto doesn't support it now, I would advise that they strongly consider it moving forward.

-license = "MIT"
+license = { file = "LICENSE.md" }

which works with hatchling and the yocto implementation.

Hatchling's documentation on licenses is found here: https://hatch.pypa.io/latest/config/metadata/#license. While what you've suggested may not cause the validator to fail, I'm not 100% sure that it'll package and note things appropriately via Hatchling if done this way.

I personally believe it was a bad idea to strictly require packages to pass validate-pyproject, especially when it is still noted as an experimental project that currently has not widened their scope to fully encompass PEP compliant backends such as Hatchling. If we are talking about more complex packages that utilize C modules and such, there is no way that they will chose flit-core, they will chose something like Hatchling. Yocto will be left behind in such cases.