pastelmind / d2animdata Goto Github PK

Let's add support for more file formats. I can leverage the extras_require feature of setuptools (or the requires-extra option of flit, if I switch over to it) to add conditional dependencies. I can then detect which optional dependencies are available, and make related features selectively available.

File Formats

• JSON5: Minor improvements to JSON. Since we already support JSON, migration would be quite easy.
  • Dependency: dpranke/pyjson5 AKA json5.
  • PyJSON5 requires VC++ to build on Windows and provides no wheels, so no go.
• YAML: A popular configuration format. It is quite complex, though, so I might not use it.
  • Dependency: PyYAML or ruamel.yaml
• TOML: An increasingly popular alternative to YAML with less lenient syntax. Since I am already using this in one of my projects, adding support for this should be trivial.
  • Dependency: qtoml

The goal is to reduce the number of config files as much as possible.

• tox: Currently, only the legacy_tox_ini format is supported. This is an ugly trick that stores the entire INI as a TOML string. I'd rather store the config using a proper TOML syntax. That said, the documentation states that "there is a plan to add a pure TOML one soon". I can sit and wait.
• Pylint: Pylint 2.5 will support pyproject.toml. I am looking forward to its release, since it also includes several desirable fixes. Done in #13

Currently, the entire CI/CD Workflow (see .github/workflows/build.yml) takes about 2 minutes, not including the publish Job). This is too long for a simple one-file distribution.

A significant amount of time is spent on installing dependencies. Windows runners in particular spend inordinate amounts of time on pip install. See the stats for run 52035558, which ran on the current HEAD (79e4191):

Solutions

Install needed dependencies only

Only install dependencies required for each job. The lint jobs don't need tox. The test jobs don't need Pylint. No job needs pipdeptree, as I use it for examining dependencies.

To do so, split requirements-dev.txt into multiple files: linting, building-and-testing, and miscellaneous development tools. More fine-grained splitting is possible (requirements-lint-black.txt, requirements-lint-pylint.txt, etc.), but it would make managing dependencies a nightmare.

Unfortunately, the dependencies for building wheels (setuptools, wheel, etc.) must be installed on both the build and test jobs. This is because I want to test both sdists and wheels.

(Rejected) Merge build and test jobs

Since building sdists and wheels take a trivial amount of time, we could reduce the overall time spent by building them directly in each test job. This eliminates the need for a separate build job.

However, this means that make each test job would uses sdists and wheels built on its own platform and Python runtime, not the ones published to PyPI. The status quo ensures that I am actually testing what is published. Hence, this idea is rejected.