internetarchive / arklet Goto Github PK

Hello,

It would be great to add a testing framework to the project. Beyond Django unittests I suggest a behaviour-driven (BDD) component : Behave. There's an available Django integration, and has as main advantage that tests can be easily written, understood, and enhances code-reuse (which is a way to avoid test code rot).

What a simple test might look like :

Feature : Minting ark identifiers
  Scenario: Simple minting request
    Given we have a running arklet service
      When we mint an id for url "http://example.com/this_test_is_ok"
        Then we obtain an ARK id
        And it forwards to "http://example.com/this_test_is_ok"

(see behave documentation for more information)

If the suggestion is validated, I can work on a PR that implements testing (be it behave or unittests).

Arklet was made as a standalone service, but it might make sense to package it as a reusable Django app too. We'll be creating some "about" content for Internet Archive. This content doesn't belong in the public repo; it wouldn't be relevant to other groups.

I'm considering creating a separate repo for Internet Archive's deployment which pulls in a package built from this repo as a reusable app. Even if we do that, I think it's useful to keep this repo as something that runs as-is. Developers should have a choice between running this repo as is, or adding the ark app to their new or existing Django site.

https://docs.djangoproject.com/en/4.0/intro/reusable-apps/

Having used pip, poetry and pipenv & even contributed to some, I'd like to suggest to move to Poetry for packaging & version management.

1. It uses pyproject.toml, as recommended by PEP518, along with it's own lockfile (poetry.lock) which doesn't get in the way
2. The tool evolves pretty well, simple to use, and appreciated by the community. Having used it for the pas 2 years, I have yet to find a problem with it
3. Easy local venv management
4. Integrates well with docker : RUN poetry config virtualenvs.create false && poetry install -no-dev --no-interaction --no-ansi

Along with poetry can be added code cleanup, formatting & linting tools : isort, flake8, pylint, black, ... we just have to agree on the basic set. Most use pyproject.toml for settings, except for flake8 which needs his own file (or pyproject.toml with some monkey patching).

Is Poetry OK ? And which tools could be added to the repo (I'd take all of them, but that may be overkill) ?

Storing the Key.key as plaintext is a bad idea. There are implementations of token auth in Django Rest Framework, but one of the design goals for arklet is to have only standard lib and Django as production dependencies. We want it to be really inexpensive to stay with the latest versions of Python and Django.

Maybe we could use the built-in Django password functionality? https://docs.djangoproject.com/en/4.0/topics/auth/passwords/#module-django.contrib.auth.hashers

I don't want to use a User model for API authentication because it makes fleet secret management a little harder (have to coordinate password changes with the secrets getting updated across the fleet). The access key model works a little bit better for secret rotation. We can create a new access key, start distributing it, wait for everything to update, and deactivate the old key.

The Key model should not use the key UUIDField as the primary key on the table. We should add a regular integer primary key, and then create a separate charfield to hold the hash.

https://docs.djangoproject.com/en/4.0/topics/auth/passwords/#django.contrib.auth.hashers.check_password
https://docs.djangoproject.com/en/4.0/topics/auth/passwords/#django.contrib.auth.hashers.make_password

check_password can be used against the authorization header
make_password can be used with a UUID4 input to create the hash text to store in the Key.key field.

This would also necessitate a new view for one-time display of newly created keys.

https://github.com/topics/arks

to improve SNR for archival resource key stuff versus some blockchain stuff on github.