verifytests / verify

Verify is a snapshot tool that simplifies the assertion of complex data models and documents.

Verify

Verify is a snapshot tool that simplifies the assertion of complex data models and documents.

Verify is called on the test result during the assertion phase. It serializes that result and stores it in a file that matches the test name. On the next test execution, the result is again serialized and compared to the existing file. The test will fail if the two snapshots do not match: either the change is unexpected, or the reference snapshot needs to be updated to the new result.

See Milestones for release notes.

Sponsors

A HUGE Thank-You to AWS for sponsoring this project in September 2023 as part of the AWS Open Source Software Fund.

Thanks to DataDog for the generous monthly sponsorship.

Requirements

• Supported runtimes: net472, net48, net481, net6, net8, and net9.
• Supported SDK: 8.0.300 and up

Getting started wizard

Get customized instructions for the specific combination of Operating System, IDE, Test Framework, and Build Server.

Start wizard.

NuGet packages

• https://nuget.org/packages/Verify.NUnit/
• https://nuget.org/packages/Verify.Xunit/
• https://nuget.org/packages/Verify.Fixie/
• https://nuget.org/packages/Verify.Expecto/
• https://nuget.org/packages/Verify.MSTest/

Snapshot management

Accepting or declining a snapshot file is part of the core workflow of Verify. There are several ways to do this and the approach(s) selected is a personal preference.

• In the Windows Tray via DiffEngineTray
• ReSharper test runner plugin (Source)
• Rider test runner plugin (Source)
• Via the clipboard.
• Manually making the change in the launched diff tool. Either with a copy paste, or some tools have commands to automate this via a shortcut or a button.
• Manually on the file system. By renaming the .received. file to .verified.. This can be automated via a scripted to bulk accept all (by matching a pattern) .received. files.
• Using the dotnet tool Verify.Terminal.

Usage

ImplicitUsings

All examples use Implicit Usings. Ensure <ImplicitUsings> is set to enable to ensure examples compile correctly.

<ImplicitUsings>enable</ImplicitUsings>

If ImplicitUsings are not enabled, substitute usages of Verify() with Verifier.Verify().

Class being tested

Given a class to be tested:

public static class ClassBeingTested
{
    public static Person FindPerson() =>
        new()
        {
            Id = new("ebced679-45d3-4653-8791-3d969c4a986c"),
            Title = Title.Mr,
            GivenNames = "John",
            FamilyName = "Smith",
            Spouse = "Jill",
            Children =
            [
                "Sam",
                "Mary"
            ],
            Address = new()
            {
                Street = "4 Puddle Lane",
                Country = "USA"
            }
        };
}

^{snippet source | anchor}

NUnit

Support for NUnit

[TestFixture]
public class Sample
{
    [Test]
    public Task Test()
    {
        var person = ClassBeingTested.FindPerson();
        return Verify(person);
    }
}

^{snippet source | anchor}

xUnit

Support for xUnit

public class Sample
{
    [Fact]
    public Task Test()
    {
        var person = ClassBeingTested.FindPerson();
        return Verify(person);
    }
}

^{snippet source | anchor}

Fixie

Support for Fixie

public class Sample
{
    public Task Test()
    {
        var person = ClassBeingTested.FindPerson();
        return Verify(person);
    }
}

^{snippet source | anchor}

Expecto

Support for Expecto

open Expecto
open VerifyTests
open VerifyExpecto

[<Tests>]
let tests =
    testTask "findPerson" {
        let person = ClassBeingTested.FindPerson()
        do! Verifier.Verify("findPerson", person)
    }

^{snippet source | anchor}

Caveats

Due to the nature of the Expecto implementation, the following APIs in Verify are not supported.

• settings.UseParameters()
• settings.UseTextForParameters()

MSTest

Support for MSTest

[TestClass]
public partial class Sample
{
    [TestMethod]
    public Task Test()
    {
        var person = ClassBeingTested.FindPerson();
        return Verify(person);
    }
}

^{snippet source | anchor}

Marking tests as 'Using Verify'

The MSTest implementation leverages a Source Generator and requires test classes to opt in to being processed by the Source Generator.

Add the UsesVerifyAttribute.

For all test classes in an assembly:

[assembly: UsesVerify]

For a specific a test class:

[UsesVerify]

Or inherit from VerifyBase:

[TestClass]
public class VerifyBaseUsage :
    VerifyBase
{
    [TestMethod]
    public Task Simple() =>
        Verify("The content");
}

^{snippet source | anchor}

Initial Verification

No existing .verified. file.

graph TD
run(Run test and<br/>create Received file)
failTest(Fail Test<br/>and show Diff)
closeDiff(Close Diff)
run-->failTest
shouldAccept{Accept ?}
failTest-->shouldAccept
accept(Move Received<br/>to Verified)
shouldAccept-- Yes -->accept
discard(Discard<br/>Received)
shouldAccept-- No -->discard
accept-->closeDiff
discard-->closeDiff

Loading

When the test is initially run will fail. If a Diff Tool is detected it will display the diff.

To verify the result:

• Execute the command from the Clipboard, or
• Accept with DiffEngineTray tool,
• Accept with ReSharper Addin or Rider Addin
• Use the diff tool to accept the changes, or
• Manually copy the text to the new file

Verified result

This will result in the Sample.Test.verified.txt being created:

{
  GivenNames: John,
  FamilyName: Smith,
  Spouse: Jill,
  Address: {
    Street: 4 Puddle Lane,
    Country: USA
  },
  Children: [
    Sam,
    Mary
  ],
  Id: Guid_1
}

^{snippet source | anchor}

Subsequent Verification

Existing .verified. file.

graph TD
run(Run test and<br/>create Received file)
closeDiff(Close Diff)
failTest(Fail Test<br/>and show Diff)
run-->isSame
shouldAccept{Accept ?}
failTest-->shouldAccept
accept(Move Received<br/>to Verified)
shouldAccept-- Yes -->accept
discard(Discard<br/>Received)
shouldAccept-- No -->discard

isSame{Compare<br/>Verified +<br/>Received}
passTest(Pass Test and<br/>discard Received)
isSame-- Same --> passTest
isSame-- Different --> failTest
accept-->closeDiff
discard-->closeDiff

Loading

If the implementation of ClassBeingTested changes:

public static class ClassBeingTested
{
    public static Person FindPerson() =>
        new()
        {
            Id = new("ebced679-45d3-4653-8791-3d969c4a986c"),
            Title = Title.Mr,
            // Middle name added
            GivenNames = "John James",
            FamilyName = "Smith",
            Spouse = "Jill",
            Children =
            [
                "Sam",
                "Mary"
            ],
            Address = new()
            {
                // Address changed
                Street = "64 Barnett Street",
                Country = "USA"
            }
        };
}

^{snippet source | anchor}

And the test is re run it will fail.

The Diff Tool will display the diff:

The same approach can be used to verify the results and the change to Sample.Test.verified.txt is committed to source control along with the change to ClassBeingTested.

Async

Verify() has overloads that accept Task<T>, ValueTask<T>, and IAsyncEnumerable<T>. These are awaited before verification.

There is also an overload that accepts Func<Task<T>>, which works well with async lambda expressions:

await Verify(
    async () => new
    {
        Foo = await repo.GetFoo(id),
        Bars = await repo.GetBars(id)
    });

^{snippet source | anchor}

VerifyJson

VerifyJson performs the following actions

• Convert to JToken (if necessary).
• Apply ignore member by name for keys.
• PrettyPrint the resulting text.

[Fact]
public Task VerifyJsonString()
{
    var json = "{'key': {'msg': 'No action taken'}}";
    return VerifyJson(json);
}

[Fact]
public Task VerifyJsonStream()
{
    var json = "{'key': {'msg': 'No action taken'}}";
    var stream = new MemoryStream(Encoding.UTF8.GetBytes(json));
    return VerifyJson(stream);
}

[Fact]
public Task VerifyJsonJToken()
{
    var json = "{'key': {'msg': 'No action taken'}}";
    var target = JToken.Parse(json);
    return Verify(target);
}

^{snippet source | anchor}

Results in a .txt file:

{
  key: {
    msg: No action taken
  }
}

^{snippet source | anchor}

Source control: Received and Verified files

Includes/Excludes

• All *.received.* files should be excluded from source control.

eg. add the following to .gitignore

*.received.*

If using UseSplitModeForUniqueDirectory also include:

*.received/

All *.verified.* files should be committed to source control.

Text file settings

Text variants of verified and received have the following characteristics:

• UTF8 with a Byte order mark (BOM)
• Newlines as line-feed (lf)
• No trailing newline

This manifests in several ways:

Source control settings

All text extensions of *.verified.* should have:

• eol set to lf
• working-tree-encoding set to UTF-8

eg add the following to .gitattributes

*.verified.txt text eol=lf working-tree-encoding=UTF-8
*.verified.xml text eol=lf working-tree-encoding=UTF-8
*.verified.json text eol=lf working-tree-encoding=UTF-8

EditorConfig settings

If modifying text verified/received files in an editor, it is desirable for the editor to respect the above conventions. For EditorConfig enabled the following can be used:

# Verify settings
[*.{received,verified}.{txt,xml,json}]
charset = "utf-8-bom"
end_of_line = lf
indent_size = unset
indent_style = unset
insert_final_newline = false
tab_width = unset
trim_trailing_whitespace = false

Note that the above are suggested for subset of text extension. Add others as required based on the text file types being verified.

Static settings

Most settings are available at the both global level and at the instance level.

When modifying settings at the both global level it should be done using a Module Initializer:

public class StaticSettings
{
    [Fact]
    public Task Test() =>
        Verify("String to verify");
}

public static class StaticSettingsUsage
{
    [ModuleInitializer]
    public static void Initialize() =>
        VerifierSettings.AddScrubber(_ => _.Replace("String to verify", "new value"));
}

^{snippet source | anchor}

VerifyResult

In some scenarios it can be helpful to get access to the resulting *.verified.* files after a successful run. For example to do an explicit check for contains or not-contains in the resulting text. To allow this all Verify methods return a VerifyResult.

var result = await Verify(
    new
    {
        Property = "Value To Check"
    });
Assert.Contains("Value To Check", result.Text);

^{snippet source | anchor}

If using Verifier.Throws, the resulting Exception will also be accessible

var result = await Throws(MethodThatThrows);
Assert.NotNull(result.Exception);

^{snippet source | anchor}

CurrentFile

Utility for finding paths based on the current file.

using IOPath = System.IO.Path;

namespace VerifyTests;

public static class CurrentFile
{
    public static string Path([CallerFilePath] string file = "") =>
        file;

    public static string Directory([CallerFilePath] string file = "") =>
        IOPath.GetDirectoryName(file)!;

    public static string Relative(string relative, [CallerFilePath] string file = "")
    {
        var directory = IOPath.GetDirectoryName(file)!;
        return IOPath.Combine(directory, relative);
    }
}

^{snippet source | anchor}

Versioning

Verify follows Semantic Versioning. The same applies for extensions to Verify. Small changes in the resulting snapshot files may be deployed in a minor version. As such nuget updates to Verify.* should be done as follows:

• Updates all Verify.*packages in isolation
• Re-run all tests.
• If there are changes, ensure they look correct given the release notes. If the changes do not look correct, raise an issue.
• Accept those changes.

Snapshot changes do not trigger a major version change to avoid causing Diamond dependency issues for downstream extensions.

Unit testing inside virtualized environment

Unit tests referencing Verify (including unit tests within this repository as well as any other code referencing Verify) can be run and debugged on a local virtualized environment supported by Visual Studio Remote Testing. Initial configurations have been added for WSL and net 7.0 linux docker via testenvironments.json (for third party code, the file needs to be copied or recreated next to the .sln solution file for solution to leverage the functionality).

Upon opening the Tests Explorer the advanced environments are available in the GUI:

This readme will not discuss definitive list of details for proper setup of the environments instead refer the following information sources and warn about particular gotchas:

• WSL runs
  • Install WSL.
  • Install a distribution.
  • Install .NET Runtime
• Docker runs
  • Install Docker Desktop
  • First run of docker scenario might need elevation (Test project does not reference any .NET NuGet adapter error)
• Third party test runners might not support this feature. Use Visual Studio Test Explorer.

Media

• Snapshot testing in C# with Verify - Tim Deschryver (21 Feb 2024)
• Snapshot Testing in C# - William Rees (13 Feb 2024)
• I want to do a snapshot test with C# (26 Dec 2023)
• Introduction to Snapshot testing and using Verify.Xunit - Betatalks (11 Oct 2023)
• I REGRET Not Telling Dave Farley THIS about Approval Testing - Emily Bache (27 Sep 2023)
• The Way to Test Legacy Code in C# - Gui Ferreira (19 Sep 2023)
• Today's random F# code: Using Verify to prevent breaking changes in stored data - Urs Enzler (31 Mar 2023)
• Compare object values in xUnit C# with Verify - Pierre Belin (18 Nov 2022)
• Unit testing assertions are now easier than ever with Verify Snapshot tool - Rana Krishnrajsinh (5 Nov 2022)
• The easiest way to Unit Test with Verify in C# - Tomasz Juszczak (6 Sep 2022)
• Testing C# code reliably by freezing it in time - Nick Chapsas (1 Aug 2022)
• Snapshot Testing in .NET with Verify - Dan Clarke (21 Jul 2022)
• Testing an incremental generator with snapshot testing (14 Dec 2021)
• Snapshot Testing with Verify - Dan Clarke (10 Dec 2021)
• OSS Power-Ups: Verify (14 Jul 2021)
• Unhandled Exception podcast: Snapshot Testing (26 Nov 2021)
• 5 helpful Nuget package for Unit Testing in .NET (16 Oct 2021)
• 5 open source .NET projects that deserve more attention (9 Sep 2021)
• Snapshot Testing with Verify: Carl Franklin's Blazor Train (3 Apr 2020)
• Verify: Snapshot Testing for C# (23 Nov 2020)
• Verify Xunit Intro (26 Apr 2020)

Extensions

• Verify.AngleSharp: Html verification utilities via AngleSharp.
• Verify.AspNetCore: Verification of AspNetCore bits.
• Verify.Aspose: Verification of documents (pdf, docx, xlsx, and pptx) via Aspose.
• Verify.AustralianProtectiveMarkings: Verification of AustralianProtectiveMarkings.
• Verify.Avalonia: Verification of Avalonia UIs.
• Verify.Blazor: Verification of Blazor Component via bunit or via raw Blazor rendering.
• Verify.Brighter: Verification of Brighter bits.
• Verify.CommunityToolkit.Mvvm: Verification of CommunityToolkit.Mvvm.
• Verify.Cosmos: Verification of Azure CosmosDB.
• Bunsen-Burner support: BunsenBurner.Verify.NUnit / BunsenBurner.Verify.Xunit.
• Verify.DiffPlex: Comparison of text via DiffPlex.
• Verify.DocNet: Verification of pdfs via DocNet.
• Verify.EntityFramework: Verification of EntityFramework bits.
• Verify.FakeItEasy: Verification of FakeItEasy bits.
• Verify.Flurl: Verification of Flurl bits.
• Verify.HeadlessBrowsers: Verification of Web UIs using Playwright, Puppeteer Sharp, or Selenium.
• Verify.Http: Verification of Http bits.
• Verify.ICSharpCode.Decompiler: Comparison of assemblies and types via ICSharpCode.Decompiler.
• Verify.ImageHash: Comparison of images via ImageHash.
• Verify.ImageSharp.Compare: Verification and comparison of images via Codeuctivity.ImageSharp.Compare.
• Verify.ImageMagick: Verification and comparison of images via Magick.NET.
• Verify.ImageSharp: Verification of images via ImageSharp.
• Verify.MailMessage: Verification of MailMessage and related types.
• Verify.MassTransit: Support for MassTransit test helpers.
• Verify.MicrosoftLogging: Verify MicrosoftLogging.
• Verify.MongoDB: Verification of MongoDB bits.
• Verify.Moq: Verification of Moq bits.
• Verify.NodaTime: Support for NodaTime.
• Verify.NewtonsoftJson: Support for converting Newtonsoft.Json types (JObject and JArray).
• Verify.NServiceBus: Verify NServiceBus Test Contexts.
• Verify.NSubstitute: Support for NSubstitute types.
• Verify.Nupkg: Verification of NuGet .nupkg files.
• Verify.PdfPig: Verification of pdfs via PdfPig.
• Verify.Phash: Comparison of images via Phash.
• Verify.Quibble: Comparison of objects via Quibble.
• Verify.QuestPDF: Verification of pdf documents via QuestPDF.
• Verify.RavenDb: Verification of RavenDb bits.
• Verify.SendGrid: Verification of SendGrid.
• Verify.Serilog: Verification of Serilog bits.
• Verify.SqlServer: Verification of SqlServer bits.
• Verify.SourceGenerators: Verification of C# Source Generators.
• Verify.SystemJson: Support for converting System.Text.Json types.
• Verify.Terminal: A dotnet tool for managing Verify snapshots.
• Verify.WinForms: Verification of WinForms UIs.
• Verify.Xamarin: Verification of Xamarin UIs.
• Verify.Xaml: Verification of Xaml UIs.
• Spectre.Verify.Extensions: Add an attribute driven file naming convention to Verify.
• Verify.Syncfusion: Verification of documents (pdf, docx, xlsx, and pptx) via Syncfusion File Formats.
• Verify.Wolverine: Verifiable test context for Wolverine.
• Verify.ZeroLog: Verifiable test context for ZeroLog.

More Documentation

• Clipboard
• Compared to assertions
• Verify options
• VerifyDirectory
• VerifyZip
• VerifyFile
• VerifyXml
• Serializer Settings
• Guid behavior
• Date behavior
• File naming
• AppendFile
• Parameterised tests
• Named Tuples
• Scrubbers
• Diff Engine
• Diff Tools
• Diff Tool Order
• Custom Diff Tool
• Using anonymous types
• Verifying binary data
• Build server
• Comparers
• Converters
• Recording
• Explicit Targets
• FSharp Usage
• Compared to ApprovalTests

Icon

Helmet designed by Leonidas Ikonomou from The Noun Project.

verify's Issues

Some settings not respected when serializing converter info

Issue with Verify creating empty file

Preamble

• Visual Studio - 16.5.4
• dotnet - 3.1.latest
• Verify.Xunit

Full refs on the test project:

Describe the bug

Running the test fails.

• Seems to create the *.received.txt file
• Interestingly, if I rename the *.received.txt to *.verified.txt, the test the passes.

So my guess is it's choking trying to create the empty verified one, which the stack trace certainly attests to.

• Note, I went looking into that .nuget packages path, and what struck me as odd is the path it can't find a part of is:
• C:\Users\Developer\.nuget\packages\emptyfiles\2.0.1\lib\netstandard2.0\EmptyFiles
  But the file it could be looking for is:
• C:\Users\Developer\.nuget\packages\emptyfiles\2.0.1\lib\netstandard2.0\EmptyFiles.dll

i.e. it's a dll not a folder.

    System.TypeInitializationException : The type initializer for 'EmptyFiles.AllFiles' threw an exception.
    ---- System.IO.DirectoryNotFoundException : Could not find a part of the path 'C:\Users\Developer\.nuget\packages\emptyfiles\2.0.1\lib\netstandard2.0\EmptyFiles'.
  Stack Trace: 
    AllFiles.TryCreateFile(String path, Boolean useEmptyStringForTextFiles) line 209
    DiffRunner.Launch(ResolvedDiffTool diffTool, String tempFile, String targetFile) line 79
    DiffRunner.Launch(String tempFile, String targetFile) line 65
    VerifyEngine.ProcessMissing(StringBuilder builder, FilePair item) line 278
    VerifyEngine.ProcessMissing(StringBuilder builder) line 238
    VerifyEngine.ThrowIfRequired(String message) line 123
    InnerVerifier.Verify(String input, VerifySettings settings) line 25
    InnerVerifier.Verify[T](T input, VerifySettings settings) line 43
    --- End of stack trace from previous location where exception was thrown ---
    ----- Inner Stack Trace -----
    FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound)
    FileSystemEnumerator`1.ctor(String directory, EnumerationOptions options)
    FileSystemEnumerable`1.ctor(String directory, FindTransform transform, EnumerationOptions options)
    FileSystemEnumerableFactory.UserFiles(String directory, String expression, EnumerationOptions options)
    Directory.InternalEnumeratePaths(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
    Directory.EnumerateFiles(String path, String searchPattern, SearchOption searchOption)
    AllFiles.cctor() line 63

Minimal Repro

Incoming when I get a moment.

Submit a PR that fixes the bug

Incoming when I get a moment.

Add xml docs to DiffEngine

Add DisableDiff and DisableClipboard to VerifySettings

• https://github.com/SimonCropp/Verify#disable-clipboard
• https://github.com/SimonCropp/Verify/blob/master/docs/diff-tool.md#disable-diff

Add rider diff support

Add AutoVerify

https://github.com/SimonCropp/Verify#autoverify

Add ability to customise build tool order via code

use resulting file type for diff engine

add DisableNewLineEscaping

https://github.com/SimonCropp/Verify/blob/master/docs/serializer-settings.md#disablenewlineescaping

Fix guids/datetimes in dictionary keys not being scrubbed

Fix suppress deterministic for mutli target

Add cleanup support to converters

rename SharedVerifySettings to VerifierSettings

fix EmptyFiles transient files not included

add RequiresTarget to diffengine

allows diff tools to op out of creating an empty file for comparison purposes

Tests pass when they should fail

Describe the bug

Running tests, the comparison tool is launched and the diff clearly shows that the verified vs received are different, but for some reason the tests pass.

• Visual Studio - 16.5.1
• dotnet --version = 3.1.200
• Project file is netcoreapp3.1 with:
• xunit 2.4.1 (and vs runner)
• Verify.xunit 1.27.0

Minimal Repro

Raising for now, will do when I can.

Submit a PR that fixes the bug

Raising for now, will do when I can.

Add OnHandlers

https://github.com/SimonCropp/Verify/#onhandlers

Allow converters to have different extensions per stream

https://github.com/VerifyTests/Verify/blob/master/docs/converter.md

Before

VerifierSettings.RegisterFileConverter<Image>(
    toExtension: "png",
    canConvert: target => Equals(target.RawFormat, ImageFormat.Tiff),
    conversion: (image, settings) =>
    {
        var pages = image.GetFrameCount(FrameDimension.Page);

        var streams = new List<Stream>();
        for (var index = 0; index < pages; index++)
        {
            image.SelectActiveFrame(FrameDimension.Page, index);

            var page = new MemoryStream();
            image.Save(page, ImageFormat.Png);
            streams.Add(page);
        }

        return new ConversionResult(
            info: new
            {
                image.PixelFormat,
                image.Size
            },
            streams);
    });

VerifierSettings.RegisterFileConverter(
    fromExtension: "tif",
    toExtension: "png",
    conversion: (stream, settings) =>
    {
        using Image image = Image.FromStream(stream);
        var pages = image.GetFrameCount(FrameDimension.Page);

        var streams = new List<Stream>();
        for (var index = 0; index < pages; index++)
        {
            image.SelectActiveFrame(FrameDimension.Page, index);

            var page = new MemoryStream();
            image.Save(page, ImageFormat.Png);
            streams.Add(page);
        }

        return new ConversionResult(
            info: new
            {
                image.PixelFormat,
                image.Size
            },
            streams);
    });

After

VerifierSettings.RegisterFileConverter<Image>(
    canConvert: target => Equals(target.RawFormat, ImageFormat.Tiff),
    conversion: (image, settings) =>
    {
        var pages = image.GetFrameCount(FrameDimension.Page);

        var streams = new List<ConversionStream>();
        for (var index = 0; index < pages; index++)
        {
            image.SelectActiveFrame(FrameDimension.Page, index);

            var page = new MemoryStream();
            image.Save(page, ImageFormat.Png);
            streams.Add(new ConversionStream("png",page));
        }

        return new ConversionResult(
            info: new
            {
                image.PixelFormat,
                image.Size
            },
            streams);
    });

VerifierSettings.RegisterFileConverter(
    fromExtension: "tif",
    conversion: (stream, settings) =>
    {
        using Image image = Image.FromStream(stream);
        var pages = image.GetFrameCount(FrameDimension.Page);

        var streams = new List<ConversionStream>();
        for (var index = 0; index < pages; index++)
        {
            image.SelectActiveFrame(FrameDimension.Page, index);

            var page = new MemoryStream();
            image.Save(page, ImageFormat.Png);
            streams.Add(new ConversionStream("png",page));
        }

        return new ConversionResult(
            info: new
            {
                image.PixelFormat,
                image.Size
            },
            streams);
    });

Add custom diff tool support

https://github.com/SimonCropp/Verify/blob/master/docs/diff-tool.custom.md

File comparison should not lock file

Update Diffengine to 4.5

Explanation on how to get actual Json Format

I've spent a good 15 mins trying to figure out which api changes the JsonTextWriter.

Also I think if you have something called "Json" then it should be json otherwise make a new format. The problem is you might "think" it looks better but if it's not json then other json based tooling will not work with it.

Also while we're at it where can I find how to change the source directory for all the received files?

incoming stream should be disposed

42a54e3

Make Verify.Xunit static

Inheriting from VerifyBase is no longer required.

Instead use the static Verifier and add [UsesVerifyAttribute] to all xunit tests that use Verify

UseParameters should allow nulls

43ccd63

Use semantic HTML comparison instead of raw text/string comparison

Feature

HTML can change from a raw markup perspective, without changing the semantic meaning of it. Most common is changes to whitespace. This causes tests to break, when they do not need to.

Describe the solution

If Verify uses a semantic HTML comparison library, I created AngleSharp Diffing for this exact purpose, instead of just doing text comparison, then tests that uses Verify will be much more stable. There are some additional options that can be used in to modify the diffing, e.g. switch to case insensitive comparison, or ignore certain attributes (generated IDs), that are also very helpful.

I will be happy to help integrate AngleSharp Diffing into Verify. Just let me know if you have any questions.

Additional context

The context is primarily with the bUnit integration.

Here is a quick sample, that shows how Verify fails a test because of insignificant whitespace, and the bUnit tests passes, because it uses MarkupMatches to verify the rendered markup, using AngleSharp Diffing.

sync on snapshot/approvals/verification testing

@theramis @isidore @michaelstaib @nscheibe @claremacrae

hey peoples. Given our project overlaps i though it might be helpful for us to be aware of each other and have a possible sync so we can share ideas, lessons learnt, code etc.

• ApprovalTests
• Snapshooter
• Snapper
• Verify

Some discussion points i can think of

• the trade offs between stack trace walking and using [CallerFilePath]
• how diff tools work, which i think Snapper and Snapshooter dont have?
• verification of files/binary data. eg image or office document verification
• the possibility of sharing code. i have already created a stand alone nuget for EmptyFiles (which makes comparison of files easier). i was also considering splitting out the whole diff tool detection+management into a nuget.

So would people like to get on a call? or perhaps discuss things here? or some other medium?

Is there anyone else I should have brought in? or tools i should have listed?

Comparers not applying to text

Add more string scrubbing

https://github.com/SimonCropp/Verify/blob/master/docs/scrubbers.md#available-scrubbers

Partial namespace included in verified file name

for multipart namespaces (eg Foo.Bar.Bat) part of the namespace is being correctly included in the verified file name. Once upgrading to 1.27.0 all existing files, with incorrect names, will be renames to the correct file name.

pass settings to comparer

cache ncrunch env variable

Add SharedVerifySettings.NameForParameter

https://github.com/VerifyTests/Verify/blob/master/docs/parameterised.md#complex-memberdata

Add more diff tool support

• KDiff3
• TkDiff
• TortoiseIDiff
• TortoiseMerge
• DiffMerge
• Kaleidoscope

Fix dangling verified file

when verifying multiple streams, if the umber of files reduces, the test should fail and the clipboard should contain a command to remove extra files

Rename some input params to "target"

add TeamCity detection

Expose Namer `Runtime` or `RuntimeAndVersion` strings

https://github.com/SimonCropp/Verify/blob/master/docs/naming.md#namerruntimeandversion

release process testing

Remove non-async stream compare

missing tiff and jpeg file types for diff tool support

93064f1

Add MaxInstancesToLaunch

https://github.com/SimonCropp/Verify/blob/master/docs/diff-tool.md#maxinstancestolaunch

Move types in the Verify to VerifyTests

to unblock c# using static feature

Update EmptyFiles to V1

Add TortoiseGitMerge support

Update DiffEngine to fix teamcity detection

fe4e8e8

Add support for IAsyncEnumerable

New file name convention logic when Verify is called in base classes

Is the feature request related to a problem

Recent changes wrt to [UsesVerify] and how Verifier.Verify (the method) uses [CallerFilePath] string sourceFile = "" have meant that when upgrading some of my projects the file names generated overlapped each other.

The issue is because when calling Verify from the base class, the CallerFilePath is the base class' file path.

Essentially I had:

[UsesVerify]
public class TestsBase
{
    protected async Task UtilityAssertMethod(...)
    {
        //... A bunch of other assertions / mock verifications, etc.
       
        await Verifier.Verify(actual, settings);
    }

    protected async Task Some_Theory_Should_Work_Impl(bool someParameter)
    {
        //...arrange
        //...act
        settings.UseParameter(someParameter);
        await Verifier.Verify(actual, settings);
    }
}

public class SomethingTests : TestsBase
{
    [Fact]
    public async Task Some_Test_Should_Work()
    {
        // ...arrange
        // ...act
        base.UtilityAssertMethod(...);
    } 

    [Theory]
    [InlineData(true)
    public async Task Some_Theory_Should_Work(bool someParameter)
        => await base.Some_Theory_Should_Work_Impl(someParameter);
}

The initial way around it I found was to have the base class define an abstract method for it's utility method to call Verify from:

public abstract Task Verify(string target, VerifySettings settings);

And then each leaf test class actually implement that, so their file path would be used:

public override Task Verify(string target, VerifySettings settings)
            => Verifier.Verify(target, settings);

Potentially another workaround would be to pass that value explicitly, rather than let the runtime pass it through. Here's a dotnetfiddle that seems to show that should work: https://dotnetfiddle.net/BT27ry

But I'd need to supply the full path of the file, and I guess getting that is just as challenging. Actually I could just update my protected async Task UtilityAssertMethod(...) to accept [CallerFilePath] string callerFilePath = "" and pass that on to Verify. Will give that a go, as it's a lot cleaner. One issue might be that then the verify assertions on UseParameters might want that to be a parameter...

Describe the solution

Don't really have a solution, was just talking through the issue at the moment, but potentially we can just immediately close this and leave it as an example solution for others.

Describe alternatives considered

Two solutions above.

Additional context

None at this stage.

Fix more null references

using nullable attributes from net5

Should use ToString on Expression