Coder Social home page Coder Social logo

tophat / syrupy Goto Github PK

View Code? Open in Web Editor NEW
507.0 5.0 33.0 12.13 MB

:pancakes: The sweeter pytest snapshot plugin

Home Page: https://syrupy-project.github.io/syrupy/

License: Apache License 2.0

Python 97.07% JavaScript 0.46% AppleScript 1.75% Dockerfile 0.72%
python pytest pytest-plugin snapshot-testing testing snapshot snapshot-tests snapshot-plugin snapshottest

syrupy's Introduction

syrupy

Logo

All Contributors Stage codecov

Pytest>=5.1.0,<9.0.0 Pypi Wheel PyPI - Python Version PyPI - Downloads PyPI - License

Overview

Syrupy is a zero-dependency pytest snapshot plugin. It enables developers to write tests which assert immutability of computed results.

Motivation

The most popular snapshot test plugin compatible with pytest has some core limitations which this package attempts to address by upholding some key values:

  • Extensible: If a particular data type is not supported, users should be able to easily and quickly add support.
  • Idiomatic: Snapshot testing should fit naturally among other test cases in pytest, e.g. assert x == snapshot vs. snapshot.assert_match(x).
  • Soundness: Snapshot tests should uncover even the most minute issues. Unlike other snapshot libraries, Syrupy will fail a test suite if a snapshot does not exist, not just on snapshot differences.

Installation

python -m pip install syrupy

Migration from snapshottest

You cannot use syrupy alongside snapshottest due to argument conflicts. To ease migration, we've made syrupy aware of snapshottest call syntax. Simply uninstall snapshottest and remove old snapshots:

pip uninstall snapshottest -y;
find . -type d ! -path '*/\.*' -name 'snapshots' | xargs rm -r

Pytest and Python Compatibility

Syrupy will always be compatible with the latest version of Python and Pytest. If you're running an old version of Python or Pytest, you will need to use an older major version of Syrupy:

Syrupy Version Python Support Pytest Support
4.x.x >3.8.1 >=7
3.x.x >=3.7, <4 >=5.1, <8
2.x.x >=3.6, <4 >=5.1, <8

Usage

Basic Usage

In a pytest test file test_file.py:

def test_foo(snapshot):
    actual = "Some computed value!"
    assert actual == snapshot

when you run pytest, the above test should fail due to a missing snapshot. Re-run pytest with the update snapshots flag like so:

pytest --snapshot-update

A snapshot file should be generated under a __snapshots__ directory in the same directory as test_file.py. The __snapshots__ directory and all its children should be committed along with your test code.

Custom Objects

The default serializer supports all python built-in types and provides a sensible default for custom objects.

Representation

If you need to customise your object snapshot, it is as easy as overriding the default __repr__ implementation.

def __repr__(self) -> str:
    return "MyCustomClass(...)"

If you need bypass a custom object representation to use the amber standard, it is easy using the following helpers.

def test_object_as_named_tuple(snapshot):
    assert snapshot == AmberDataSerializer.object_as_named_tuple(obj_with_custom_repr)

See test_snapshot_object_as_named_tuple_class for an example on automatically doing this for all nested properties

Attributes

If you want to limit what properties are serialized at a class type level you could either:

A. Provide a filter function to the snapshot exclude configuration option.

def limit_foo_attrs(prop, path):
    allowed_foo_attrs = {"do", "not", "serialize", "these", "attrs"}
    return isinstance(path[-1][1], Foo) and prop in allowed_foo_attrs

def test_bar(snapshot):
    actual = Foo(...)
    assert actual == snapshot(exclude=limit_foo_attrs)

B. Provide a filter function to the snapshot include configuration option.

def limit_foo_attrs(prop, path):
    allowed_foo_attrs = {"only", "serialize", "these", "attrs"}
    return isinstance(path[-1][1], Foo) and prop in allowed_foo_attrs

def test_bar(snapshot):
    actual = Foo(...)
    assert actual == snapshot(include=limit_foo_attrs)

C. Or override the __dir__ implementation to control the attribute list.

class Foo:
    def __dir__(self):
        return ["only", "serialize", "these", "attrs"]

def test_bar(snapshot):
    actual = Foo(...)
    assert actual == snapshot

Both options will generate equivalent snapshots but the latter is only viable when you have control over the class implementation and do not need to share the exclusion logic with other objects.

CLI Options

These are the cli options exposed to pytest by the plugin.

Option Description Default
--snapshot-update Snapshots will be updated to match assertions and unused snapshots will be deleted. False
--snapshot-details Includes details of unused snapshots (test name and snapshot location) in the final report. False
--snapshot-warn-unused Prints a warning on unused snapshots rather than fail the test suite. False
--snapshot-default-extension Use to change the default snapshot extension class. AmberSnapshotExtension
--snapshot-no-colors Disable test results output highlighting. Equivalent to setting the environment variables ANSI_COLORS_DISABLED or NO_COLOR Disabled by default if not in terminal.
--snapshot-patch-pycharm-diff Override PyCharm's default diffs viewer when looking at snapshot diffs. See IDE Integrations False

Assertion Options

These are the options available on the snapshot assertion fixture. Use of these options are one shot and do not persist across assertions. For more persistent options see advanced usage.

matcher

This allows you to match on a property path and value to control how specific object shapes are serialized.

The matcher is a function that takes two keyword arguments. It should return the replacement value to be serialized or the original unmutated value.

Argument Description
data Current serializable value being matched on
path Ordered path traversed to the current value e.g. (("a", dict), ("b", dict)) from { "a": { "b": { "c": 1 } } }}

NOTE: Do not mutate the value received as it could cause unintended side effects.

Built-In Matchers

Syrupy comes with built-in helpers that can be used to make easy work of using property matchers.

path_type(mapping=None, *, types=(), strict=True, regex=False)

Easy way to build a matcher that uses the path and value type to replace serialized data. When strict, this will raise a ValueError if the types specified are not matched.

Argument Description
mapping Dict of path string to tuples of class types, including primitives e.g. (MyClass, UUID, datetime, int, str)
types Tuple of class types used if none of the path strings from the mapping are matched
strict If a path is matched but the value at the path does not match one of the class types in the tuple then a PathTypeError is raised
regex If true, the mapping key is treated as a regular expression when matching paths
replacer Called with any matched value and result is used as the replacement that is serialized. Defaults to the object type when not given
from syrupy.matchers import path_type

def test_bar(snapshot):
    actual = {
      "date_created": datetime.now(),
      "value": "Some computed value!!",
    }
    assert actual == snapshot(matcher=path_type({
      "date_created": (datetime,),
      "nested.path.id": (int,),
    }))
# name: test_bar
  dict({
    'date_created': datetime,
    'value': 'Some computed value!!',
  })
# ---

NOTE: When regex is True all matcher mappings are treated as regex patterns

path_value(mapping=None, *, **kwargs)

Shares the same kwargs as path_type matcher, with the exception of the mapping argument type. Only runs replacement for objects at a matching path where the value of the mapping also matches the object data string repr.

Argument Description
mapping Dict of path string to object value string representations

See test_regex_matcher_str_value for example usage.

exclude

This allows you to filter out object properties from the serialized snapshot.

The exclude parameter takes a filter function that accepts two keyword arguments. It should return true if the property should be excluded, or false if the property should be included.

Argument Description
prop Current property on the object, could be any hashable value that can be used to retrieve a value e.g. 1, "prop_str", SomeHashableObject
path Ordered path traversed to the current value e.g. (("a", dict), ("b", dict)) from { "a": { "b": { "c": 1 } } }}
Built-In Filters

Syrupy comes with built-in helpers that can be used to make easy work of using the filter options.

props(prop_name, *prop_name)

Easy way to build a filter that excludes based on string based property names.

Takes an argument list of property names, with support for indexed iterables.

from syrupy.filters import props

def test_bar(snapshot):
    actual = {
      "id": uuid.uuid4(),
      "list": [1,2,3],
    }
    assert actual == snapshot(exclude=props("id", "1"))
# name: test_bar
  dict({
    'list': list([
      1,
      3,
    ]),
  })
# ---
paths(path_string, *path_strings)

Easy way to build a filter that uses full path strings delimited with ..

Takes an argument list of path strings.

from syrupy.filters import paths

def test_bar(snapshot):
    actual = {
      "date": datetime.now(),
      "list": [1,2,3],
    }
    assert actual == snapshot(exclude=paths("date", "list.1"))
# name: test_bar
  dict({
    'list': list([
      1,
      3,
    ]),
  })
# ---

include

This allows you filter an object's properties to a subset using a predicate. This is the opposite of exclude. All the same property filters supporterd by exclude are supported for include.

The include parameter takes a filter function that accepts two keyword arguments. It should return true if the property should be include, or false if the property should not be included.

Argument Description
prop Current property on the object, could be any hashable value that can be used to retrieve a value e.g. 1, "prop_str", SomeHashableObject
path Ordered path traversed to the current value e.g. (("a", dict), ("b", dict)) from { "a": { "b": { "c": 1 } } }}

Note that include has some caveats which make it a bit more difficult to use than exclude. Both include and exclude are evaluated for each key of an object before traversing down nested paths. This means if you want to include a nested path, you must include all parents of the nested path, otherwise the nested child will never be reached to be evaluated against the include predicate. For example:

obj = {
    "nested": { "key": True }
}
assert obj == snapshot(include=paths("nested", "nested.key"))

The extra "nested" is required, otherwise the nested dictionary will never be searched -- it'd get pruned too early.

To avoid adding duplicate path parts, we provide a convenient paths_include which supports a list/tuple instead of a string for each path to match:

obj = {
    "other": False,
    "nested": { "key": True }
}
assert obj == snapshot(include=paths_include(["other"], ["nested", "key"]))

extension_class

This is a way to modify how the snapshot matches and serializes your data in a single assertion.

def test_foo(snapshot):
    actual_svg = "<svg></svg>"
    assert actual_svg == snapshot(extension_class=SVGImageSnapshotExtension)

diff

This is an option to snapshot only the diff between the actual object and a previous snapshot, with the diff argument being the previous snapshot index/name.

def test_diff(snapshot):
    actual0 = [1,2,3,4]
    actual1 = [0,1,3,4]

    assert actual0 == snapshot
    assert actual1 == snapshot(diff=0)
    # This is equivalent to the lines above
    # Must use the index name to diff when given
    assert actual0 == snapshot(name='snap_name')
    assert actual1 == snapshot(diff='snap_name')
Built-In Extensions

Syrupy comes with a few built-in preset configurations for you to choose from. You should also feel free to extend the AbstractSyrupyExtension if your project has a need not captured by one our built-ins.

  • AmberSnapshotExtension: This is the default extension which generates .ambr files. Serialization of most data types are supported.
    • Line control characters are normalised when snapshots are generated i.e. \r and \n characters are all written as \n. This is to allow interoperability of snapshots between operating systems that use disparate line control characters.
  • SingleFileSnapshotExtension: Unlike the AmberSnapshotExtension, which groups all tests within a single test file into a singular snapshot file, this extension creates one .raw file per test case.
  • PNGImageSnapshotExtension: An extension of single file, this should be used to produce .png files from a byte string.
  • SVGImageSnapshotExtension: Another extension of single file. This produces .svg files from an svg string.
  • JSONSnapshotExtension: Another extension of single file. This produces .json files from dictionaries and lists.

name

By default, if you make multiple snapshot assertions within a single test case, an auto-increment identifier will be used to index the snapshots. You can override this behaviour by specifying a custom snapshot name to use in place of the auto-increment number.

def test_case(snapshot):
    assert "actual" == snapshot(name="case_a")
    assert "other" == snapshot(name="case_b")

Warning: If you use a custom name, you must make sure the name is not re-used within a test case.

Advanced Usage

By overriding the provided AbstractSyrupyExtension you can implement varied custom behaviours.

See examples of how syrupy can be used and extended in the test examples.

Overriding defaults

It is possible to override include, exclude, matchers and extension_class on a more global level just once, instead of every time per test. By default, after every assertion the modified values per snapshot assert are reverted to their default values. However, it is possible to override those default values with ones you would like persisted, which will be treated as the new defaults.

To achieve that you can use snapshot.with_defaults, which will create new instance of SnapshotAssertion with the provided values.

snapshot.use_extension is retained for compatibility. However, it is limited to only overriding the default extension class.

JSONSnapshotExtension

This extension can be useful when testing API responses, or when you have to deal with long dictionaries that are cumbersome to validate inside a test. For example:

import pytest

from syrupy.extensions.json import JSONSnapshotExtension

@pytest.fixture
def snapshot_json(snapshot):
    return snapshot.with_defaults(extension_class=JSONSnapshotExtension)
    # or return snapshot.use_extension(JSONSnapshotExtension)


def test_api_call(client, snapshot_json):
    resp = client.post("/endpoint")
    assert resp.status_code == 200
    assert snapshot_json == resp.json()

API responses often contain dynamic data, like IDs or dates. You can still validate and store other data of a response by leveraging syrupy matchers. For example:

from datetime import datetime

from syrupy.matchers import path_type

def test_api_call(client, snapshot_json):
    resp = client.post("/user", json={"name": "Jane"})
    assert resp.status_code == 201

    matcher = path_type({
      "id": (int,),
      "registeredAt": (datetime,)
    })
    assert snapshot_json(matcher=matcher) == resp.json()

The generated snapshot:

{
  "id": "<class 'int'>",
  "registeredAt": "<class 'datetime'>",
  "name": "Jane"
}

Or a case where the value needs to be replaced using a condition e.g. file path string

import re

from syrupy.matchers import path_type

def test_matches_generated_string_value(snapshot, tmp_file):
    matcher = path_value(
        mapping={"file_path": r"\w+://(.*/)+dir/filename.txt"},
        replacer=lambda _, match: match[0].replace(match[1], "<tmp-file-path>/"),
        types=(str,),
    )

    assert snapshot(matcher=matcher) == tmp_file

The generated snapshot:

{
  "name": "Temp Files",
  "file_path": "scheme://<tmp-file-path>/dir/filename.txt"
}

Extending Syrupy

Inline Snapshots

Syrupy does not support inline snapshots. For inline snapshots, we recommend checking out the inline-snapshot library.

IDE Integrations

PyCharm

The PyCharm IDE comes with a built-in tool for visualizing differences between expected and actual results in a test. To properly render Syrupy snapshots in the PyCharm diff viewer, we need to apply a patch to the diff viewer library. To do this, use the --snapshot-patch-pycharm-diff flag, e.g.:

In your pytest.ini:

[pytest]
addopts = --snapshot-patch-pycharm-diff

See #675 for the original issue.

Known Limitations

  • pytest-xdist support only partially exists. There is no issue when it comes to reads however when you attempt to run pytest --snapshot-update, if running with more than 1 process, the ability to detect unused snapshots is disabled. See #535 for more information.

We welcome contributions to patch these known limitations.

Uninstalling

pip uninstall syrupy

If you have decided not to use Syrupy for your project after giving us a try, we'd love to get your feedback. Please create a GitHub issue if applicable.

Contributing

Feel free to open a PR or GitHub issue. Contributions welcome!

To develop locally, clone this repository and run . script/bootstrap to install test dependencies. You can then use invoke --list to see available commands.

See contributing guide

Contributors

Noah
Noah

🚇 🤔 💻 📖 ⚠️
Emmanuel Ogbizi
Emmanuel Ogbizi

💻 🎨 🚇 📖 ⚠️
Adam Lazzarato
Adam Lazzarato

📖
Marc Cataford
Marc Cataford

💻 ⚠️
Michael Rose
Michael Rose

💻 ⚠️
Jimmy Jia
Jimmy Jia

💻 ⚠️
Steven Loria
Steven Loria

🚇
Artur Balabanov
Artur Balabanov

💻
Huon Wilson
Huon Wilson

💻 🐛
Elizabeth Culbertson
Elizabeth Culbertson

💻 ⚠️
Joakim Nordling
Joakim Nordling

🐛
Ouail
Ouail

💻
Denis
Denis

💻
N0124
N0124

💻
dtczest
dtczest

🐛
Eddie Darling
Eddie Darling

📖
darrenburns
darrenburns

📖
Magnus Heskestad Waage
Magnus Heskestad Waage

🐛
Herbert Ho
Herbert Ho

🐛
Tolga Eren
Tolga Eren

🐛
John Kurkowski
John Kurkowski

🐛
Atharva Arya
Atharva Arya

💻
Michał Jelonek
Michał Jelonek

💻
ManiacDC
ManiacDC

💻
Dmitry Dygalo
Dmitry Dygalo

📖
Allan Chain
Allan Chain

🐛
Nir Schulman
Nir Schulman

💻

This section is automatically generated via tagging the all-contributors bot in a PR:

@all-contributors please add <username> for <contribution type>

License

Syrupy is licensed under Apache License Version 2.0.

syrupy's People

Contributors

adamlazz avatar allanchain avatar allcontributors[bot] avatar arturbalabanov avatar atharva-2001 avatar darrenburns avatar dependabot-preview[bot] avatar dependabot[bot] avatar eaculb avatar huonw avatar iamogbz avatar joakimnordling avatar john-kurkowski avatar maniacdc avatar mcataford avatar mhwaage avatar michaljelonek avatar msrose avatar noahnu avatar renovate[bot] avatar semantic-release-bot avatar sloria avatar spagh-eddie avatar stranger6667 avatar taion avatar tolgaeren avatar tommasoamici avatar tophat-opensource-bot avatar ultimatelobster avatar umar-ahmed avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

syrupy's Issues

Snapshotting set does not support all hashable types

Describe the bug

Frozen sets are the only custom types allowed in the test, others fail with the error message

TypeError: '<' not supported between instances of 'str' and '...'

To Reproduce
Steps to reproduce the behavior:

  1. git checkout unsupported_sort
  2. inv test -d

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots

https://github.com/tophat/syrupy/blob/9e8dea8337425f0f31ed2951c7ebd2e7d236f92a/tests/test_extension_amber.py#L76-L77

Support ignoring fields when serializing

Is your feature request related to a problem? Please describe.

When asserting on snapshots support filtering out properties that should not be matched on. This would allow deterministic snapshots when matching on objects that generate realtime values in tests without needing to mock.

Describe the solution you'd like

def test_my_dict(snapshot):
	my_dict = {
		"match_me": True,
		"do_not_match_me": time.time(),
		"nested_do_not_match": {
			"do_not_match_me": time.time(),
		},
	}
	assert my_dict == snapshot(exclude=("do_not_match_me",))
# name: test_my_dict
  <class 'dict'> {
    'match_me': True,
	'nested_do_not_match': <class 'dict'> {
	},
  }
---

Describe alternatives you've considered

This is already possible but is cumbersome to implement, and cannot easily be extended to other serialization types.

class OmitDataSerializer(DataSerializer):
	@classmethod
	def __filter_data(cls, data) -> Dict[str, Any]:
		return {
			key: data[key]
			for key in data
			if key not in ("do_not_match_me",)
		}
	
    @classmethod
    def serialize_dict(cls, data, **kwargs) -> str:
        return super().serialize_dict(cls.__filter_data(data), **kwargs)


class OmitDataExtension(AmberSnapshotExtension):
    def serialize(self, data):
        return OmitDataSerializer.serialize(data)


def test_my_dict(snapshot):
	my_dict = {
		"match_me": True,
		"do_not_match_me": time.time(),
		"nested_do_not_match": {
			"do_not_match_me": time.time(),
		},
	}
	assert my_dict == snapshot(extension=OmitDataExtension)

Additional context

Syrupy assertion diff does not show missing carriage return

Describe the bug

Syrupy assertion diff does not show missing carriage return.

To Reproduce

Add this test.

def test_example(snapshot):
    assert snapshot == "line 1\r\nline 2"

Run pytest --snapshot-update.

Remove the \r from the string in the test case so you get:

def test_example(snapshot):
    assert snapshot == "line 1\nline 2"

Run pytest. The test will fail with a useless diff of "...".

Expected behavior

Syrupy should output each line with the missing carriage return, with some indication of what's missing.

Additional context

Syrupy v0.3.1

#113 fixed the bug where carriage returns were not being serialized. This issue addresses the missing functionality in the snapshot assertion diff reporter.

Argument conflict when trying to replace snapshottest

Describe the bug

Pytest raises an error from argparse because of a conflicting arguments: both snapshottest and syrupy declare a --snapshot-update argument, so you have to trash your virtualenv or explicitly uninstall snapshottest to get syrupy working.

To Reproduce

Steps to reproduce the behavior:

  1. Create a project with snapshottest and pytest as a dependency (e.g. install it in a venv)
  2. Install syrupy and remove snapshottest from your dependencies
  3. Running pytest shows an error because of the duplicate argument
  4. Delete your virutalenv and reinstall, or explicitly uninstall snapshottest
  5. pytest works a-ok

Expected behavior

It would be nice if you didn't get this confusing error when migrating, it would make syrupy an easier drop-in replacement for snapshottest, but it seems like this may be out of control of plugins. Could be worth calling it out explicitly in the docs for other people who encounter it, if a programmatic solution is not available.

Desktop (please complete the following information):

  • OS: MacOS
  • Version 10.14.5
  • Python 3.6.6, pip 18.1

Update migration instructions from snapshottest

Is your feature request related to a problem? Please describe.

The current migration instructions are misleading since syrupy uses a different default snapshot directory than snapshottest.

Describe the solution you'd like

Include full instructions on how to migrate from snapshottest, including a shell command to delete snapshottest snapshots.

Describe alternatives you've considered

We could add a migrate command, e.g. python -m syrupy migrate which could look for snapshottest files and directories and delete them.

syrupy does not work with pytest 5.0.1

Describe the bug

Running syrupy with pytest 5.0.1 produces:

AttributeError: 'Config' object has no attribute 'invocation_params'

To Reproduce

pip install pytest==5.0.1 syrupy==0.3.5
echo "def test_example(snapshot):\nassert snapshot == 'hi'" > test_file.py
pytest

If we don't support 5.0.1, we should be explicit about min versions.

Pretty diff of snapshot comparisons

Is your feature request related to a problem? Please describe.

Diffing complex objects on assertion failure does not show only the change but the entire snapshot

Describe the solution you'd like

Should show the exact line and characters (tokens) that differ between snapshots

Describe alternatives you've considered

N/A

Additional context

N/A

Make pyyaml dependency optional

Is your feature request related to a problem? Please describe.

Syrupy supports multiple serializes including custom serializes. If a codebase using syrupy selects a non-yaml serializes, there is no reason to install the unused dependency.

Describe the solution you'd like

List pyyaml as an optional dependency. Throw error if the seralizer is used if pyyaml is not present in system.

Describe alternatives you've considered

  • Continue installing unused dependencies.

Additional context

Python dependency management is awful 😄

Syrupy removes other snapshot files in the same snapshot directory

Describe the bug
Syrupy removes other snapshot files in the same snapshot directory

To Reproduce
Steps to reproduce the behavior:

  1. Have two test files (test_file1.py, test_file2.py) in the same directory where each one has a snapshot assertion
  2. Generate snapshots pytest --snapshot-update see two snapshot files
  3. pytest test_file1.py --snapshot-update
  4. See only one snapshot file

Expected behavior
Syrupy should update the file for the file under test and not remove the other snapshot file

Desktop (please complete the following information):

  • OS: macOS
  • Version 10.15.2 (Catalina)

Filter out unused snapshots when specifying test nodes

Describe the bug

Specifying test cases in a test module does not filter out unused snapshots from reporting as unused

To Reproduce

  1. pytest --pyargs tests.test_extension_amber::TestClass
    a. 7 snapshots passed. 38 snapshots unused.
  2. pytest tests/test_extension_amber.py::TestClass
    a. 7 snapshots passed. 38 snapshots unused.

Expected behavior

1a. 7 snapshots passed.
2a. 7 snapshots passed.

Screenshots

Screen Shot 2020-02-28 at 9 57 59 PM

Desktop (please complete the following information):

  • OS: macOS Catalina
  • Version 10.15.3

Additional context
N/A

Make serialized representation of empty iterable more concise

Is your feature request related to a problem? Please describe.

An empty list takes up 3 lines in an ambr file.

<class 'list'> [
]

Describe the solution you'd like

Represent an empty iterable in a more concise format:

<class 'list'> []

or

<class 'list'> [ ]

Describe alternatives you've considered

There's an argument that the more verbose format produces smaller git diffs if adding an element, since we don't change the line indicating a list is being used.

Syrupy cannot handle carriage returns in strings

Describe the bug

If a carriage return is present in a string, Syrupy will strip out the carriage return when serializing the data thus making the assertion persistently fail even after a snapshot-update.

To Reproduce

def test_carriage(snapshot):
    assert snapshot == "line 1\r\nline 2"
pytest --snapshot-update; pytest

Expected behavior

Syrupy should be able to serialize any control character.

Additional context

Syrupy v0.3.0

Support nested test classes

Describe the bug

Syrupy does not support nested test blocks

To Reproduce
Steps to reproduce the behavior:

Given

# test_str.py
class TestClass:
    class TestSubClass:
        def test_sub_class_has_parent(self, snapshot):
            assert "this should be in a nested class" == snapshot

Result

# name: TestSubClass.test_sub_class_has_parent
  'this should be in a nested class'
---

Expected behavior

# name: TestClass.TestSubClass.test_sub_class_has_parent
  'this should be in a nested class'
---

Screenshots

N/A

Desktop (please complete the following information):

  • Version 0.1.0

Additional context

N/A

Configuration option to specify default serializer plugin

Is your feature request related to a problem? Please describe.

Currently you have to override the snapshot fixture in your project's root conftest.py in order to specify the default serializer class. It's potentially not clear what the final serializer is (order of shadowing).

Describe the solution you'd like

A configuration option to specify the python module path to load for the default serializer.

--snapshot-default-plugin syrupy.extensions.image.SVGImageExtension

Describe alternatives you've considered

Adding documentation for overriding the default fixture.

More tests!

Is your feature request related to a problem? Please describe.

No integration tests mean wide ranging can introduce bugs and regressions not caught.

Describe the solution you'd like

💯 coverage!

Describe alternatives you've considered

No bragging rights for 💯 coverage

Additional context

N/A

Unused snapshots reported for deselected tests

Describe the bug
image
Notice how in the screenshot that 5/6 tests are deselected, but the test run reports 2 unused snapshots.

To Reproduce
Steps to reproduce the behavior:

  1. pip install pytest pytest-watch syrupy
  2. Add this test file:
    import pytest
    
    def test_thing_1(snapshot):
        assert snapshot == "123"
    
    def test_thing_2(snapshot):
        assert snapshot == "456"
    
    def test_thing_11(snapshot):
        assert snapshot == "123"
    
    def test_thing_22(snapshot):
        assert snapshot == "456"
    
    @pytest.mark.wip
    def test_thing_111(snapshot):
        assert snapshot == "123"
    
    def test_thing_222(snapshot):
        assert snapshot == "456"
  3. Run pytest --snapshot-update
  4. Run ptw -- -m wip to watch test files and only run tests with the wip mark
  5. The pytest run reports 2 unused snapshots
  6. It seems like this behaviour is the same without using ptw, e.g. running pytest -m wip reports two unused snapshots.

Expected behavior
Here I'd expect no unused snapshots to be reported -- seems like deselected tests are sometimes being considered not having used their snapshot, which is technically true, but given that the test didn't run I'd say there's no way to assess whether or not the snapshot is out of date.

Screenshots
Included above

Desktop (please complete the following information):

  • MacOS 10.14.5 with python 3.6.2 and syrupy 0.3.2

Additional context
The behaviour doesn't occur with two tests, i.e. the following test file does report an unused snapshot when run with pytest -m wip:

import pytest

def test_thing_1(snapshot):
    assert snapshot == "123"

@pytest.mark.wip
def test_thing_2(snapshot):
    assert snapshot == "456"

SerializerClass vs IOClass

Discussion

Would it be more idiomatic to have the current SerializerClass be replaced in name by IOClass but retain the same functionality.

Arguments

For replacing with AbstractSnapshotIO For keeping Serializer nomenclature
IO handles the file system, the serialization is just a step in the process Serializer handles writing and reading from file system
IO (input, output) is more encompassing allowing for extensions to functionality like preparing assertion diffs Serializer is a more familiar concept to most people making the naming appropriate for its function

Example

https://github.com/tophat/syrupy/blob/44b0f6ccdc1036c531a76dfac87ed5ca75d6be51/README.md#L71
https://github.com/tophat/syrupy/blob/44b0f6ccdc1036c531a76dfac87ed5ca75d6be51/src/syrupy/__init__.py#L66-L71
https://github.com/tophat/syrupy/blob/44b0f6ccdc1036c531a76dfac87ed5ca75d6be51/src/syrupy/assertion.py#L71-L75
https://github.com/tophat/syrupy/blob/44b0f6ccdc1036c531a76dfac87ed5ca75d6be51/src/syrupy/assertion.py#L135-L138

Deletes file when running only subset of tests in test file with update snapshots flag

Describe the bug

The snapshot file holding the serialized assertions gets deleted when running in partial mode and update snapshots enabled

To Reproduce
Steps to reproduce the behavior:

  1. Clone repo and . script/bootstrap
  2. inv test -du -t=test_multiple_snapshots
  3. This deletes the entire file fixed in #20
  4. This deletes all files and snapshots not touched by the subset of tests

Expected behavior

  1. Only deletes unused snapshots and preserves newly written snapshots
  2. Only modifies files to be touched by a subset of the test

Additional context
Add any other context about the problem here.

Benchmark syrupy performance on test runs in CI

Is your feature request related to a problem? Please describe.

It'd be nice to see the impact a diff has on the plugin performance

Describe the solution you'd like

  • Benchmark code performance
  • Compare against head (master)
  • Commit or stash the generated profile
  • Github status/checks on branches to show effect

Describe alternatives you've considered

N/A

Additional context

Support snapshottest --snapshot-update flag

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Compare snapshots without deserializing

Is your feature request related to a problem? Please describe.

Snapshots are currently deserialized into python objects before comparison is done, this is unnecessary

Describe the solution you'd like

No deserialization is done and comparison is done on the serialized representation

Describe alternatives you've considered

N/A

Additional context

Unlocks using yaml.dumps in place of yaml.safe_dumps because loading yaml data is needless

Add pylint to precommit and CI

Is your feature request related to a problem? Please describe.

Black is just a formatter and does not catch things like unused imports.

Describe the solution you'd like

pylint/flake8 to precommit hook and CI.

Add pytest as a dependency

Describe the bug

Syrupy relies on pytest's APIs, specifically its hooks, and yet it doesn't specify what version of pytest is supported.

Expected behavior

Syrupy should list pytest in the setup.py with the maximum version range supported.

Additional context

syrupy v0.2.0

Add code of conduct

Is your feature request related to a problem? Please describe.

To help build a positive community around Python snapshot testing we should agree on an official code of conduct and hold ourselves accountable.

This is part of GitHub's community checklist.

Clarify test methods for integration tests and testing in general

Current tests are starting to be hard to reason about. Some clarification about the behaviour each test is attempting to verify is getting more essential.

Describe the solution you'd like

Make each test case group a single file testing all the behaviours with module and test doc string explaining the purpose

Describe alternatives you've considered
N/A

Additional context
The tests/test_integration_default.py file is an example of this complexity.

Show snapshot report after failing tests

Describe the bug

Snapshot report should be shown after test failures are reported not before

To Reproduce
Steps to reproduce the behavior:

  1. echo 'def test_example(snapshot): assert "abc" == snapshot' > tests /test_example.py
  2. pytest tests/test_example.py
(syrupy.venv) (iamogbz-patch-1) λ pytest tests/test_example.py
========================= test session starts =========================
platform darwin -- Python 3.8.1, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /Users/Emmanuel/Sources/syrupy, inifile: pytest.ini
plugins: syrupy-2020.3.1.111512023105
collected 1 item                                                      

tests/test_example.py F                                         [100%]

1 snapshot failed.

============================== FAILURES ===============================
____________________________ test_example _____________________________

snapshot = SnapshotAssertion(name='snapshot', num_executions=1)

    def test_example(snapshot):
>       assert 'abc' == snapshot
E       assert received == snapshot
E         Snapshot does not exist!

tests/test_example.py:2: AssertionError
========================== 1 failed in 0.05s =========================

Expected behavior

Snapshot report should show after the failing tests

Screenshots
N/A

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 1.0]

Additional context
Add any other context about the problem here.

Syrupy is deleting newly created snapshots when parametrizing a class method

Describe the bug

Snapshots are being deleted immediately after being written when syrupy is run with --snapshot-update.

To Reproduce

class TestMyClass:
    @pytest.mark.parametrize(
        "some_arg",
        [
            "abc",
            "def",
        ],
    )
    def test_param(self, some_arg, snapshot):
        assert some_arg == snapshot

Expected behavior

Snapshots should not be deleted if they're being used.

Additional Context

Syrupy v0.0.14 using the Image serializer.

Account for accessibility and readability in snapshot outputs

This is a general issue to track notes about readability and accessibility in diffs. See this jest blog post for more information: https://jestjs.io/blog/2020/01/21/jest-25, and this jest PR jestjs/jest#9132

They seem to have put a lot of effort into improving diff outputs recently so it's definitely something to consider and take inspiration from.

Sorry for the lack of specificity on this issue, mostly created it for tracking purposes with the idea of triggering an investigation.

Named tuple serialisation should include parameter names

Describe the bug

Named tuples when serialised loose argument information

To Reproduce
Steps to reproduce the behavior:

  1. See current example or named tuple serialisation in the test amber snapshots

Expected behavior
Parameter names should be sorted and preserved like a dictionary or class

Screenshots
N/A

Desktop (please complete the following information):

  • v0.2.0

Additional context
Add any other context about the problem here.

Replacing `os.path` with `pathlib`

Is your feature request related to a problem? Please describe.
This is mainly a refactor concern: in Python 3.4+, pathlib can serve as a more powerful (and built-in) replacement for os.path, dealing with flexible pathlike objects compatible anywhere where a string path may be used. It offers extra features over just dealing with paths, such as navigating filetrees with ease using object methods. Succinctly put, it provides a cleaner experience when dealing with file structures.

Describe the solution you'd like
Refactor syrupy so that it uses pathlib wherever it deals with paths instead of os.path.

Describe alternatives you've considered
Continuing with os.path and accepting increased complexity.

Additional context
Since Syrupy already requires python >= 3.6, the version requirement for pathlib isn't a concern.

Add syrupy test example usecases

Is your feature request related to a problem? Please describe.

It useful to know how syrupy can be used especially in contrast with other snapshot test libraries

Describe the solution you'd like

Add documentation on the various ways syrupy can be used and extended, with example test code for each.

Describe alternatives you've considered

Just documentation no examples

Additional context

A couple for starters

  • Custom snapshots directory
  • Custom snapshot names
  • Nested test support (pytest-describe)
  • JSON support
  • Ignore fields in dictionary snapshot

Partial test run snapshot update does not support pytest parametrize

Describe the bug

When running a subset of all tests with the pytest keyword flag -k, snapshots in tests matching the keyword should be updated and deleted appropriately. This does not work for pytest.mark.parametrize tests

To Reproduce
Steps to reproduce the behavior:

  1. Checkout master
  2. Remove a parametrize test case e.g.
    https://github.com/tophat/syrupy/blob/7cde8a7b0773d3ecbf6b8e1a8d677ba8c982dfa2/tests/test_snapshots.py#L37
  3. Run inv test -d -u -t=<test_name>
  4. Test case does not get removed

Expected behavior

Test case removed from the parametrized definition gets deleted

Screenshots
N/A

Desktop (please complete the following information):

  • OS: macos catalina
  • Version: a15276c

Additional context
N/A

Automate releases on GitHub

Is your feature request related to a problem? Please describe.

Package releases are currently manual. We should automate these releases via semantic commits.

This means we need to enforce semantic commits as part of precommit hook and CI, and parse the commits to determine version bump.

Alternatives

Continuing manual releases w/ thoroughly documented process.

Snapshot diffing in test assertion

Is your feature request related to a problem? Please describe.

Snapshots should be helpful in indicating at a glance what has changed.

Describe the solution you'd like

def test_my_dict(snapshot):
	my_dict = {
		"field_0": True,
		"field_1": "no_value",
		"nested": {
			"field_0": 1,
		},
	}
	assert my_dict == snapshot(id="case1")
	my_dict["field_1"] = "yes_value"
	assert my_dict == snapshot(diff="case1")
	my_dict["nested"]["field_0"] = 2
	assert my_dict == snapshot(id="case3", diff=1)
# name: test_my_dict.case_1
  <class 'dict'> {
    'field_0': True,
    'field_1': 'no_value',
	'nested': <class 'dict'> {
        'field_0': 1
	},
  }
---
# name: test_my_dict.1
    'field_1': 'yes_value',
---
# name: test_my_dict.case_3
        'field_0': 2
---

Describe alternatives you've considered

N/A

Additional context

Show number of snapshots asserted in the report

Is your feature request related to a problem? Please describe.

Show how many snapshots were used in the test session

Describe the solution you'd like

Showing the number summary e.g.

1 snapshot failed | 5 snapshots passed | 1 snapshot updated | 3 snapshots generated | 2 snapshots unused

Describe alternatives you've considered

N/A

Additional context

This requires an overhaul of how snapshot assertions are tracked

Serializer should fail on max depth exceeded to break cycles

Describe the bug

If you serialize an object containing a cycle, it will recurse until call stack overflows / python can no longer handle it, rather than a graceful application error.

To Reproduce

cycle = { "a": {} }
cycle["a"]["b"] = cycle

def test_cycle(snapshot):
    assert snapshot == cycle

Expected behaviour

Error should be thrown if a cycle is detected.

Additional context

We already track "indent". We could add some logic to throw an error if indent exceeds some level as it's essentially equivalent to depth.

Provide better default repr for classes without overridden repr

Describe the bug

Snapshotting a class instance which does not explicitly define a serializable repr or str, will use the default object repr which includes the memory address of the class instance. This changes every test run.

To Reproduce

Add to a test file:

class MyClass:
    pass

def test_class(snapshot):
    assert snapshot = MyClass()

then

inv test -d -u

Expected behavior

Snapshots should not change between test runs if the code is deterministic.

Additional context

It's possible we don't want to try support this, as it clearly implies the data type is not meant to be serialized.

Remove assertion call syntax

Is your feature request related to a problem? Please describe.

One of the project's motivations is to provide a uniform, idiomatic way to assert snapshots. We're keeping snapshot.assert_match for backwards compatibility with snapshottest for the moment, however no one uses the snapshot(actual) syntax. It's a remnant of testing out different approaches.

Describe the solution you'd like

Remove the snapshot call syntax: https://github.com/tophat/syrupy/blob/7cc40476556cbb7ddb3f60bab81668341550a742/src/syrupy/assertion.py#L123

Fail when there are unused snapshots

Is your feature request related to a problem? Please describe.

When there are unused snapshots there should be a behaviour to fail the tests especially for running in ci.

Describe the solution you'd like

Fail when unused snapshots are detected.

Describe alternatives you've considered

Flag to fail in ci --ci.

Additional context
Add any other context or screenshots about the feature request here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.