martinnj / distro-dumper Goto Github PK

Building, linting and typechecking are not implemented.

https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/

Replace the console exclusive printing with a proper logger implementation.
For now we're still just logging to console, but it makes it easier to also add file logging in the future.

https://archlinux.org/releng/releases/

Create a Jenkinsfile that supports testing in several Python version, linting and similar goodies.

Find absolute URLs in DOM:
https://www.raspberrypi.com/software/operating-systems/

Add a linting environment to tox.ini.

Settings can be tweaked later.

Moving them from the module into the an inherited class should make it easier to type-check everything.

Talking about:

• verify_config
• generate_from_environment
• create_worker

If they're defined as static method we can ignore actually constructing the class.
It feels dirty, but if it works.

From here: https://www.debian.org/CD/torrent-cd/
Looks like a URL format can be created from the architecture and media.

Instead of having a blob doing everything, create a way for distro (or just source) specific handling to be compartmentalised but handled with a consistent API.

For instance a module for Arch Linux and one for Debian, then the program can load them when they're configured and have them do all the specific work of getting the files from the websites.

Especially the module ones.

Ideally it would be a table with the following columns:

• Variable name
• Required (✓ or empty)
• Default value
• Description

Add to tox, then run it in the Jenkinsfile.

https://github.com/martinnj/distrowatch-dumper/blob/11ef3d3ca2db7c39f84d4166b16b3cfd196c1836/distrodumper/modules/debian.py#L51

This check only allows either mac or edu, not both. Documentation says it's a CSV containing either or both.

Manjaro changed their urls/page layout, module seems to be broken.

Search DOM for absolute torrent URLs:
https://manjaro.org/download/