darrenburns / pytest-clarity Goto Github PK
View Code? Open in Web Editor NEWA plugin to improve the output of pytest with colourful unified diffs
License: MIT License
A plugin to improve the output of pytest with colourful unified diffs
License: MIT License
Hi, it seems this plugin cause xdist to be unable to discover the tests:
$ py.test tests --runslow -x -n auto
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.6.1, pytest-3.8.0, py-1.5.4, pluggy-0.7.1
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/emmanuel/projects/parsec-cloud/.hypothesis/examples')
rootdir: /home/emmanuel/projects/parsec-cloud, inifile: setup.cfg
plugins: xdist-1.23.2, trio-0.5.0, profiling-1.3.0, forked-0.2, cov-2.5.1, hypothesis-3.74.0
gw0 [374] / gw1 [374] / gw2 [374] / gw3 [374]
scheduling tests via LoadScheduling
...................................................................s................................s.....................................................................x..............................x... [ 54%]
..........................................................s......xssss.............x.................x............ssssss...................ssssssssss.sssssssss.......... [100%]
================================================================================ 337 passed, 32 skipped, 5 xfailed in 41.08 seconds =================================================================================
$ pip install pytest-clarity
Collecting pytest-clarity
Requirement already satisfied: pytest>=3.5.0 in ./venv/lib/python3.6/site-packages (from pytest-clarity) (3.8.0)
Requirement already satisfied: termcolor==1.1.0 in ./venv/lib/python3.6/site-packages (from pytest-clarity) (1.1.0)
Requirement already satisfied: atomicwrites>=1.0 in ./venv/lib/python3.6/site-packages (from pytest>=3.5.0->pytest-clarity) (1.1.5)
Requirement already satisfied: attrs>=17.4.0 in ./venv/lib/python3.6/site-packages (from pytest>=3.5.0->pytest-clarity) (18.1.0)
Requirement already satisfied: pluggy>=0.7 in ./venv/lib/python3.6/site-packages (from pytest>=3.5.0->pytest-clarity) (0.7.1)
Requirement already satisfied: more-itertools>=4.0.0 in ./venv/lib/python3.6/site-packages (from pytest>=3.5.0->pytest-clarity) (4.3.0)
Requirement already satisfied: six>=1.10.0 in ./venv/lib/python3.6/site-packages (from pytest>=3.5.0->pytest-clarity) (1.11.0)
Requirement already satisfied: setuptools in ./venv/lib/python3.6/site-packages (from pytest>=3.5.0->pytest-clarity) (40.0.0)
Requirement already satisfied: py>=1.5.0 in ./venv/lib/python3.6/site-packages (from pytest>=3.5.0->pytest-clarity) (1.5.4)
Installing collected packages: pytest-clarity
Successfully installed pytest-clarity-0.1.0a1
$ py.test tests --runslow -x -n auto
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.6.1, pytest-3.8.0, py-1.5.4, pluggy-0.7.1 -- /home/emmanuel/projects/parsec-cloud/venv/bin/python3.6
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/emmanuel/projects/parsec-cloud/.hypothesis/examples')
rootdir: /home/emmanuel/projects/parsec-cloud, inifile: setup.cfg
plugins: xdist-1.23.2, trio-0.5.0, profiling-1.3.0, forked-0.2, cov-2.5.1, clarity-0.1.0a1, hypothesis-3.74.0
[gw0] linux Python 3.6.1 cwd: /home/emmanuel/projects/parsec-cloud
[gw1] linux Python 3.6.1 cwd: /home/emmanuel/projects/parsec-cloud
[gw2] linux Python 3.6.1 cwd: /home/emmanuel/projects/parsec-cloud
[gw3] linux Python 3.6.1 cwd: /home/emmanuel/projects/parsec-cloud
[gw0] Python 3.6.1 (default, Jul 16 2017, 14:02:57) -- [GCC 5.4.0 20160609]
[gw1] Python 3.6.1 (default, Jul 16 2017, 14:02:57) -- [GCC 5.4.0 20160609]
[gw2] Python 3.6.1 (default, Jul 16 2017, 14:02:57) -- [GCC 5.4.0 20160609]
[gw3] Python 3.6.1 (default, Jul 16 2017, 14:02:57) -- [GCC 5.4.0 20160609]
gw0 [0] / gw1 [0] / gw2 [0] / gw3 [0]
scheduling tests via LoadScheduling
=========================================================================================== no tests ran in 1.95 seconds ============================================================================================
$
It's possible that this is on purpose, but it's jarring to me.
I have found the plugin most useful when paired with -qq to turn off automatically setting -v or -vv, not sure which.
For example, normally running pytest with no flags results in lots of dots, and with -v you get the test list and pass/fail and percentages.
With pytest-clarity, you get all that without the -v.
That's fun for a few tests, but annoying with thousands.
Diffing in unittest.mock is not supported.
Note: Sample output is manageable at the level below but common tests contain more than the sample data.
Sample test:
from unittest import mock
def test_clarity():
mocked_object = mock.MagicMock()
mocked_object.save({"data": "wrong data"})
mocked_object.save.assert_called_with({"data": "expected data"})
Sample output:
Hi,
Thanks for a great plugin!
While the diffs are super helpful, it looks like we lose the very handy drill-down output:
$ cat test.py
a = [1, 2, 3]
b = [1, 2]
def test_a():
assert len(a) == len(b)
without plugin
def test_a():
> assert len(a) == len(b)
E assert 3 == 2
E + where 3 = len([1, 2, 3]) <-- i miss these
E + and 2 = len([1, 2])
with plugin
def test_a():
> assert len(a) == len(b)
E assert left == right failed.
E Showing split diff:
E
E left: 3
E right: 2 <-- really nice!
perhaps @nicoddemus has any advice/ideas on how to make this possible?
Assertions raised from outside the test file are not diffed. Here's an example.
Is this a failure in pytest_assertrepr_compare maybe?
Baseline - beautiful.
def test_diff():
assert {"1": 1} == {"2": 1}
E LHS vs RHS shown below
E
E {'1': 1}
E {'2': 1}
Assert in non-test function, still beautiful.
def over_here():
assert {"1": 1} == {"2": 1}
def test_diff():
over_here()
E LHS vs RHS shown below
E
E {'1': 1}
E {'2': 1}
Assert in another file, dang.
from tests.utils import over_here
def test_diff():
over_here()
# tests/utils.py
def over_here():
assert {"1": 1} == {"2": 1}
def over_here():
> assert {"1": 1} == {"2": 1}
E AssertionError
tests/utils/__init__.py:120: AssertionError
Using pytest-clarity 0.3.0a0
with python 3.6
leads to the following error
File ".tox/py3/lib/python3.6/site-packages/pytest_clarity/util.py", line 3, in <module>
import six
ModuleNotFoundError: No module named 'six'
pytest-clarity
reduces clarity of the diff if two similar strings only have \n
at the end as the difference.
This is the normal pytest
output (without any verbosity):
> assert 'Not enough operands\n' == 'Not enough operands'
E AssertionError: assert 'Not enough operands\n' == 'Not enough operands'
E - Not enough operands
E + Not enough operands
E ? +
That +
at the end means that the first string has an LF at the end and the second one does not.
This is the pytest-clarity
output (with triple vebosity):
> assert 'Not enough operands\n' == 'Not enough operands'
E assert == failed. [pytest-clarity diff shown]
E
E LHS vs RHS shown below
E
E Not enough operands
E
> assert 'Not enough operands' == 'Not enough operands\n'
E assert == failed. [pytest-clarity diff shown]
E
E LHS vs RHS shown below
E
E Not enough operands
E
It would be great to add support for unittest
's asserts.
For example:
import unittest
class TestClarity(unittest.TestCase):
def test_clarity_assert(self):
left = [1, 2, 3]
right = [1, 2, 4]
assert left == right
def test_clarity_self_assert(self):
left = [1, 2, 3]
right = [1, 2, 4]
self.assertEqual(left, right)
if __name__ == "__main__":
unittest.main()
assert
is nicely coloured:
unittest.assertEqual
isn't:
Python 3.7 is showing the following
/usr/lib/python3.7/site-packages/pytest_clarity/hints.py:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
from collections import Sequence
I guess that means it is broken on py38
The contributing guidelines say, "please ensure the coverage at least stays the same before you submit a pull request," but there is no coverage reporting when running the tests with "tox". Looks like "coverage" was removed from the requirements in 0bdbaa8.
(also, btw: the flake8 tox environment fails because flake8 isn't installed.)
Assertions where one side of the comparison is a float("nan")
render as a ?
with pytest-clarity.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.