anys,jwodder

Any ideas how to improve readability of test output on failed matches?

First of all thanks for this nice projects, look very useful and well written!

I'm wondering if you are aware of any tricks to make the output of a failed match within pytest easier to read. Here is a contrived example, simulating a large-ish payload that gets matched against some matcher.

from anys import AnyMatch

LONG_TEXT_EXAMPLE = """
Example of a long multi-line text.
Example of a long multi-line text.
Example of a long multi-line text.
"""

def test_example():
    payload = {
        "foo": LONG_TEXT_EXAMPLE,
        "bar": LONG_TEXT_EXAMPLE,
        "baz": LONG_TEXT_EXAMPLE,
    }
    assert payload == {
        "foo": LONG_TEXT_EXAMPLE,
        "bar": AnyMatch("(.*)looong(.*)"), # doesn't match; mistake on purpose
        "baz": LONG_TEXT_EXAMPLE,
    }

Running a plain pytest shows that the matching has failed, but the output doesn't make it obvious that it was this bar matcher:

>       assert payload == {"foo": "asjdfhlaksfdh", "bar": AnyMatch("(.*)looong(.*)")}
E       AssertionError: assert {'bar': '\nEx...sjdfhlaksfdh'} == {'bar': AnyMa...sjdfhlaksfdh'}
E         Omitting 1 identical items, use -vv to show
E         Differing items:
E         {'bar': '\nExample of a long multi-line text.\nExample of a long multi-line text.\nExample of a long multi-line text.\n'} != {'bar': AnyMatch('(.*)looong(.*)')}
E         Left contains 1 more item:
E         {'baz': '\n'
E                 'Example of a long multi-line text.\n'
E                 'Example of a long multi-line text.\n'...
E         
E         ...Full output truncated (3 lines hidden), use '-vv' to show

test_test.py:17: AssertionError
=============================================================================================================================================== short test summary info ===============================================================================================================================================
FAILED test_test.py::test_example - AssertionError: assert {'bar': '\nEx...sjdfhlaksfdh'} == {'bar': AnyMa...sjdfhlaksfdh'}
================================================================================================================================================== 1 failed in 0.06s ==================================================================================================================================================

Using pytest -vv contains the information in a sense, but the usage of the + and - in the diff still confuse me a little bit.

>       assert payload == {"foo": "asjdfhlaksfdh", "bar": AnyMatch("(.*)looong(.*)")}
E       AssertionError: assert {'foo': 'asjdfhlaksfdh', 'bar': '\nExample of a long multi-line text.\nExample of a long multi-line text.\nExample of a long multi-line text.\n', 'baz': '\nExample of a long multi-line text.\nExample of a long multi-line text.\nExample of a long multi-line text.\n'} == {'foo': 'asjdfhlaksfdh', 'bar': AnyMatch('(.*)looong(.*)')}
E         Common items:
E         {'foo': 'asjdfhlaksfdh'}
E         Differing items:
E         {'bar': '\nExample of a long multi-line text.\nExample of a long multi-line text.\nExample of a long multi-line text.\n'} != {'bar': AnyMatch('(.*)looong(.*)')}
E         Left contains 1 more item:
E         {'baz': '\n'
E                 'Example of a long multi-line text.\n'
E                 'Example of a long multi-line text.\n'
E                 'Example of a long multi-line text.\n'}
E         Full diff:
E           {
E         -  'bar': AnyMatch('(.*)looong(.*)'),
E         +  'bar': '\n'
E         +         'Example of a long multi-line text.\n'
E         +         'Example of a long multi-line text.\n'
E         +         'Example of a long multi-line text.\n',
E         +  'baz': '\n'
E         +         'Example of a long multi-line text.\n'
E         +         'Example of a long multi-line text.\n'
E         +         'Example of a long multi-line text.\n',
E            'foo': 'asjdfhlaksfdh',
E           }

test_test.py:17: AssertionError
=============================================================================================================================================== short test summary info ===============================================================================================================================================
FAILED test_test.py::test_example - AssertionError: assert {'foo': 'asjdfhlaksfdh', 'bar': '\nExample of a long multi-line text.\nExample of a long multi-line text.\nExample of a long multi-line text.\n', 'baz': '\nExample of a long multi-line text.\nExample of a long multi-line text.\nExample of a long multi-line text.\n'} == {'foo': 'asjd...
================================================================================================================================================== 1 failed in 0.06s ==================================================================================================================================================

In the end this is probably just how pytest works after an == comparison has failed, but maybe you have some other ideas how to improve the readability.

What if the library would offer its own assert_eq(a, b) helper (drawing some inspiration here from the great rust-pretty-assertions in Rust)? Then it could either:

format the diff itself, with an understanding of the matchers, but that could get tough with all the pretty printing.
the __eq__ implementation of the matchers could -- as a side effect (dirty...) -- collect all failed matchers in a local context. Then the assert_eq function could simply run a == b and after the comparison is done, it could look into that "local matcher errors collection" if it contains errors and display a more specific explanation why certain matchers have failed.

Support varargs for AnyIn

It would be cool to allow users to write assert foo == AnyIn(2,2) instead of assert foo == AnyIn([2,2])

jwodder / anys Goto Github PK

anys's People

Contributors

Stargazers

Watchers

anys's Issues

Any ideas how to improve readability of test output on failed matches?

Support varargs for AnyIn

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent