julian / virtue

A modern, extensible, unittest-compliant test runner.

Virtue

virtue is a modern, extensible, unittest compliant test runner.

It is not a test framework (it doesn't contain a TestCase subclass and it never will).

Usage

Running a unittest-based suite works essentially as it does for twisted's trial, i.e.:

$ python -m virtue mypackage.tests

will run the tests subpackage of a given importable package.

More docs are coming. Sorry.

Contributing

I'm Julian Berman.

virtue is on GitHub.

Get in touch, via GitHub or otherwise, if you've got something to contribute, it'd be most welcome!

You can also generally find me on Libera (nick: Julian) in various channels, including #python.

If you feel overwhelmingly grateful, you can also sponsor me.

virtue's Issues

Enable better warning defaults

A test runner is a different environment than "normal" Python execution.

Even though Python's warnings defaults have long-since erred on the side of silencing warnings (so that end-users don't see warnings they can't address), test writers generally should see and address warnings in their code as part of suite execution.

Virtue should by default cause warnings to trigger errors during test execution.

Fix unittest.expectedFailure

Do stuff with test docstrings

Benchmark "time until first test starts executing"

Integrate with ward

https://github.com/darrenburns/ward

Skipped tests should output before errors and failures

Bit easier to address failures if they appear at the end of terminal scrollback.

Dynamically generate suite fixture files in Virtue's own test suite

Rather than continuing to litter virtue's package with fixture/sample files, there should be a simple interface like:

with virtue.tests.sample_suite(test_foo=FAIL, test_bar=FAIL, test_baz=SUCCEED)

or the like, to dynamically plop these down for the test that needs them.

--reporter should search for plugins

As trial does, to e.g. support https://github.com/Julian/trial-osxnotifier

Consider exiting unsuccessfully when no tests run

virtue fails trial doesn't oh no

⊙  t jsonschema.tests.test_jsonschema_test_suite.TestDraft6.test_ipv6_validation_of_IPv6_addresses_zone_id_is_not_a_part_of_ipv6_address                                                          julian@Air
Traceback (most recent call last):
  File "/Users/julian/.dotfiles/.local/share/virtualenvs/jsonschema/lib/python3.9/site-packages/virtue/locators.py", line 98, in locate_by_name
    return self.locate_in(obj)
  File "/Users/julian/.dotfiles/.local/share/virtualenvs/jsonschema/lib/python3.9/site-packages/virtue/locators.py", line 122, in locate_in
    raise ValueError(
ValueError: Can't determine the appropriate way to load <function _Test.to_unittest_method.<locals>.fn at 0x1074fbdc0>

(where that test method is dynamically generated in jsonschema's test suite)

But trial loads that just fine.

Shorten the build output / steal from Ivoire

Genericize loading tests into groups and subtests

This needs more thinking but rather than explicitly hardcoding module > case > method, it should be nicer to just have "group > tests" and then encode module as being a group which has tests, a case as also being a group which has tests, and then methods as tests with no subelements.

This'd allow collecting other nicer groupings transparently.

Tab-completable test cases