jayvdb / pytest-fastest

Use Git and coverage data to run only needed tests

pytest-fastest

Use SCM and coverage to run only needed tests

---

This pytest plugin was generated with Cookiecutter along with @hackebrot's cookiecutter-pytest-plugin template.

Features

• Gathers coverage data from tests to track which tests call functions from which modules.
• Uses Git to track changes from a given commit to find the minimum set of tests that need to run to test new changes, then skips everything else.

Requirements

• Python 3.5+
• pytest 3.4.0+

Installation

You can install "pytest-fastest" via pip from PyPI:

$ pip install pytest-fastest

Usage

pytest-fastest can be set to run only tests:

• That test modules that have changed in Git,
• Tests that we don't already have coverage data for, and
• Tests that we've added or changed.

In most common development workflows where you make short-lived branches off a main "master" or "dev" branch, the amount of code that actually changes while fixing a bug or writing a feature is usually just a small portion of the whole codebase. Instead of running thousands of tests after each change, pytest-fastest can identify the relevant ones that thoroughly test your work but skip all the things you haven't changed.

To use it:

• In pytest.ini, set fastest_commit to the name of a Git commit to compare your current work against. (You can also set or override it on the comment line with --fastest-commit). This is required if you want to skip tests, which is the main reason for using this plugin.
• Use the command line argument --fastest-mode to choice the appropriate running mode:
  • all (default): Run all tests without collecting coverage data. This emulates normal pytest behavior and has no effect on performance.
  • skip: Skip tests that don't need to be run, but update coverage data on the ones that do run. This will usually be faster than all, but because collecting coverage information takes some time, as the number of un-skippable tests grows very large it may actually become slower.
  • gather: Don't skip any tests, but do gather coverage data. This is slower than all but can be used to seed the coverage cache.
  • cache: This is a fast mode for fixing existing tests. It skips tests but doesn't update the coverage cache. It will never be slower than all and will always be faster than skip.

Contributing

Contributions are very welcome. Tests can be run with tox, please ensure the coverage at least stays the same before you submit a pull request.

License

Distributed under the terms of the MIT license, "pytest-fastest" is free and open source software

Issues

If you encounter any problems, please file an issue along with a detailed description.