kr1 / pytest-wish

Test-Driven no-Development plugin for pytest

pytest-wish

Search all installed modules for functions that pass a given feature-specification test suite. See the package documentation for the gory details.

Intended audience: python developers who've got better things to do than reinvent wheels.

Warning

Development status: almost beta (but not quite there yet).

Source code search by tests or Test-Driven no-Development

"Have a look at this piece of code that I’m writing--I’m sure it has been written before. I wouldn't be surprised to find it verbatim somewhere on GitHub." - @kr1

pytest-wish is a pytest plugin that enables a software development strategy called search by tests or Test-Driven no-Development, that is an extension of the Test-Driven Development paradigm.

The idea is that once the developer has written the tests that define the behaviour of a new function to a degree sufficient to validate the implementation they are going to write it is good enough to validate any implementation. Running the tests on a large set of functions may result in a hit, that is a function that already implements their feature.

Due to its nature the approach is better suited for discovering smaller functions with a generic signature.

Installation

You can install the latest version of "pytest-wish" via the pip package manager:

$ pip install pytest-wish

First test run

Let's create a our first test file with the simple specification of the factorial function using the wish fixture:

# content of test_factoral.py
def test_factorial(wish):
    factorial = wish
    assert factorial(0) == 1
    assert factorial(1) == 1
    assert factorial(21) == 51090942171709440000

That's it. You can now execute the test function on all functions in the Python standard library:

$ py.test --wish-from-stdlib
======================= test session starts ==========================
platform darwin -- Python 3.5.0, pytest-2.8.7, py-1.4.31, pluggy-0.3.1
rootdir: /tmp, inifile: setup.cfg
plugins: wish-0.9.0, timeout-1.0.0
collected 3259 items

test_factorial.py xxxxx[...]xxxxxXxxxxx[...]xxxxx

============================== 1 hit =================================

test_factorial.py::test_factorial[math:factorial] HIT

==== 3258 xfailed, 1 xpassed, 27 pytest-warnings in 45.07 seconds ====

We just found that the factorial function in the math module is the only HIT, that is it passes our specification test, and is the function we were looking for.

Usage

The plugin adds the following options to pytest command line:

wish:
  --wish-from-stdlib    Collects objects form the Python standard library.
  --wish-from-installed
                        Collects objects form all installed packages.
  --wish-from-all       Collects objects form the Python standard library and
                        all installed packages.
  --wish-from-specs=WISH_FROM_SPECS=[WISH_FROM_SPECS=...]
                        Collects objects from installed packages. Space
                        separated list of `pip` specs.
  --wish-from-modules=WISH_FROM_MODULES=[WISH_FROM_MODULES=...]
                        Collects objects from installed modules. Space
                        separated list of module names.
  --wish-includes=WISH_INCLUDES=[WISH_INCLUDES=...]
                        Space separated list of regexs matching full object
                        names to include, defaults to include all objects
                        collected via `--wish-from-*`.
  --wish-excludes=WISH_EXCLUDES=[WISH_EXCLUDES=...]
                        Space separated list of regexs matching full object
                        names to exclude, defaults to match 'internal use'
                        names '_|.*[.:]_'
  --wish-objects-from=WISH_OBJECTS_FROM
                        File name of full object names to include.
  --wish-predicate=WISH_PREDICATE
                        Full name of the predicate passed to
                        `inspect.getmembers`, defaults to `callable`.
  --wish-timeout=WISH_TIMEOUT
                        Test timeout.
  --wish-fail           Show wish failures.

Another example, find a function that decomposes a URL into individual rfc3986 components:

$ py.test examples/test_rfc3986_parse.py --wish-from-modules urllib.parse
[...]
examples/test_rfc3986_parse.py::test_rfc3986_parse_basic[urllib.parse:urlparse] HIT
examples/test_rfc3986_parse.py::test_rfc3986_parse_basic[urllib.parse:urlsplit] HIT
[...]

the two functions urlparse and urlsplit pass the basic rfc3986 parsing test, but do not pass the more complex test_rfc3986_parse_full test.

More advanced functions are available on PyPI:

$ pip install urllib3
$ py.test examples/test_rfc3986_parse.py --wish-from-modules urllib3
[...]
examples/test_rfc3986_parse.py::test_rfc3986_parse_basic[urllib3.util.url:parse_url] HIT
examples/test_rfc3986_parse.py::test_rfc3986_parse_full[urllib3.util.url:parse_url] HIT
[...]

now the function parse_url in the module urllib3.util.url passes both tests.

Help

We have the following support channels:

• questions on stackoverflow

Issues

If you encounter any problems, please file an issue along with a detailed description.

Contributing

Contributions are very welcome. Issues and pull requests on the pytest-wish GitHub repository. Please see the CONTRIBUTING document for development guidelines.

Authors:

• Alessandro Amici - @alexamici

Contributors:

• @kr1

Sponsors:

License

Distributed under the terms of the MIT license, "pytest-wish" is free and open source software