Coder Social home page Coder Social logo

writethemfirst / approvals-java Goto Github PK

View Code? Open in Web Editor NEW
8.0 5.0 4.0 873 KB

Approval testing library for Java. Alleviates the burden of hand-writing assertions.

License: GNU General Public License v3.0

Java 98.78% Shell 1.22%
approval java verify assertions tests

approvals-java's Introduction

Write Them First!


Approvals-Java

Approvals-Java is a lightweight open source assertion/verification library to facilitate unit testing. It alleviates the burden of hand-writing assertions.

What's new?

Just have a look at our Releases Notes!

Get Approvals-Java

Approvals-Java is released on which means you don't need any particular Maven/Gradle configuration to retrieve it.

Also, it is written in pure Java and has no additional dependencies.

Maven

In your pom.xml, add this dependency:

<dependency>
    <groupId>com.github.writethemfirst</groupId>
    <artifactId>approvals-java</artifactId>
    <version>0.13.0</version>
    <scope>test</scope>
</dependency>

Gradle

In your build.gradle, add this dependency:

testCompile 'com.github.writethemfirst:approvals-java:0.13.0'

Sbt (Scala users)

In your build.sbt, add this dependency:

libraryDependencies += "com.github.writethemfirst" % "approvals-java" % "0.13.0"

Nightly Builds

Our SNAPSHOT versions are released on oss.jfrog.org. To use them, simply add this repository to your pom.xml or to your settings.xml:

<repositories>
    <repository>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
        <id>oss-jfrog-snapshot</id>
        <name>oss-jfrog-snapshot</name>
        <url>https://oss.jfrog.org/artifactory/oss-snapshot-local</url>
    </repository>
</repositories>

And then you can simply rely on the latest SNAPSHOT:

<dependency>
    <groupId>com.github.writethemfirst</groupId>
    <artifactId>approvals-java</artifactId>
    <version>0.12.1-SNAPSHOT</version>
    <scope>test</scope>
</dependency>

Requirements

Approvals-Java has been tested to work efficiently with:

  • Windows 7+,
  • Linux,
  • Java 8,
  • JUnit 5.

If you use it in other contexts, do not hesitate to let us know!

Why using Approvals-Java?

Approval testing basics

Traditional unit testing is based on hand-writing assertions on the output of your method. This might sound boring for some people, or even sometimes really hard in case of working on some legacy source code.

Approval Testing is a way of approching assertions with the following principle:

  1. You first execute the source code you'd like to test and let it produce its usual output,
  2. You review it manually, and say if it's producing the results you expect,
  3. All future test executions will actually compare the produced results with what has been previsouly approved.

Which means you no longer write assertions... You just approve the data which will be used by assertions computer by the framework.

Approvals-Java basics

Approvals-Java is a simple Java framework allowing you to compute verifications of what your source code is doing, relying on Approval Testing principles.

Instead of writing tons of assertions, you simply call approvals.verify(result);.

  1. The first time verify is called, a received file is generated with a representation of its argument,
  2. You review the content and approve it by renaming the file, (this step is usually facilitated by a merge tool detected and launched by Approvals-Java)
  3. You commit the approved file, it is now part of the unit test and specifies the behaviour of your code,
  4. Now each time verify is called, the argument is compared with the approved file.

This replaces the calls to traditional assert methods.

What about other testing libraries?

Approvals-Java is compatible with most unit test frameworks and libraries such as JUnit, AssertJ, Mockito, etc. Since it's actually doing another job.

Is it Java only?

Approvals-Java should be able to work fine while being called from Scala or Kotlin, at least we're working on that topic. There might be a few things to take in consideration while calling the framework though. Refer to our wiki to get some details.

Usage examples?

Approvals-Java can be used to verify objects which would usually require several hand-written assertions, such as:

  • HashMaps & Collections,
  • Long Strings,
  • Files and folders,
  • Anything with a proper toString method...

And for sure lots of other usages you will find out!

How to use Approvals-Java?

Please note that most of our code samples are based on the Gilded Rose Kata. Do not hesitate to check it out ;)

Sample project

First, if you'd just want a sample project to see it in action, we have one for you!

Verify a simple object

package com.examples;

import com.github.writethemfirst.approvals.Approvals;

public class GildedRoseApprovalTest {
    private Approvals approvals = new Approvals();

    @Test
    void approvalSwordShouldDeteriorate() {
        final Item sword = new Item("basic sword", 10, 8);
        approvals.verify(GildedRose.nextDay(sword));
    }
}

The toString() of sword is used for representing the data to be stored in the approved file.

Verify each file in a folder

package com.examples;

import com.github.writethemfirst.approvals.Approvals;

public class GildedRoseApprovalTests {
    @Test
    void approvalCopySrcFolder() {
        final Approvals approvals = new Approvals();

        final Path output = Files.createTempDirectory("src");
        FolderCopy.copyFrom(Paths.get("."), output);
        approvals.verifyAgainstMasterFolder(output);
    }
}

Each file in output is checked against the master directory.

Verify a method with combinations of arguments

This can save you a lot of time instead of manual assertions, and still cover for limit cases like those which mutation testing detected.

package com.examples;

import com.github.writethemfirst.approvals.Approvals;

import static java.util.Arrays.asList;
import static java.util.Collections.singletonList;

class GildedRoseApprovalTest {

    private Approvals approvals = new Approvals();

    @Test
    void updateQuality_pass_shouldEvolve() {
        approvals.verifyAll(
            singletonList("Backstage passes"),
            asList(-1, 0, 1, 5, 6, 10, 11),
            asList(-1, 0, 1, 10),
            this::doTest);
    }

    private Item doTest(final String name, final int sellIn, final int quality) {
        final Item[] items = new Item[]{new Item(name, sellIn, quality)};
        final GildedRose app = new GildedRose(items);
        app.updateQuality();
        return app.items[0];
    }
}

Each of the 28 (1x7x4) combinations of name, sellIn, quality is used to call doTest(name, sellIn, quality).

The 28 results are stored in the received text file and compared with the approved text file, which should look like:

(Backstage passes, -1, -1) => Backstage passes, -2, 0
(Backstage passes, -1, 0) => Backstage passes, -2, 0
(Backstage passes, -1, 1) => Backstage passes, -2, 0
(Backstage passes, -1, 10) => Backstage passes, -2, 0
(Backstage passes, 0, -1) => Backstage passes, -1, 0
(Backstage passes, 0, 0) => Backstage passes, -1, 0
(Backstage passes, 0, 1) => Backstage passes, -1, 0
(Backstage passes, 0, 10) => Backstage passes, -1, 0
(Backstage passes, 1, -1) => Backstage passes, 0, 2
(Backstage passes, 1, 0) => Backstage passes, 0, 3
(Backstage passes, 1, 1) => Backstage passes, 0, 4
(Backstage passes, 1, 10) => Backstage passes, 0, 13
(Backstage passes, 5, -1) => Backstage passes, 4, 2
(Backstage passes, 5, 0) => Backstage passes, 4, 3
(Backstage passes, 5, 1) => Backstage passes, 4, 4
(Backstage passes, 5, 10) => Backstage passes, 4, 13
(Backstage passes, 6, -1) => Backstage passes, 5, 1
(Backstage passes, 6, 0) => Backstage passes, 5, 2
(Backstage passes, 6, 1) => Backstage passes, 5, 3
(Backstage passes, 6, 10) => Backstage passes, 5, 12
(Backstage passes, 10, -1) => Backstage passes, 9, 1
(Backstage passes, 10, 0) => Backstage passes, 9, 2
(Backstage passes, 10, 1) => Backstage passes, 9, 3
(Backstage passes, 10, 10) => Backstage passes, 9, 12
(Backstage passes, 11, -1) => Backstage passes, 10, 0
(Backstage passes, 11, 0) => Backstage passes, 10, 1
(Backstage passes, 11, 1) => Backstage passes, 10, 2
(Backstage passes, 11, 10) => Backstage passes, 10, 11

Advanced documentation

If you can't find the information you're searching for in our documentation or in our code sample, then don't hesitate to have a look at our FAQ or Javadoc.

Frequently Asked Questions

Don't hesitate to have a quick look at our Frequently Asked Questions before submitting an issue.

Help/Contribute

This project is completely open to any contributions! (and remember: feedback is a valuable contribution!)

Do not hesitate to:

  1. Submit issues about any feedbacks you may have about the library,
  2. Send us a Pull Request with any contribution you think about,
  3. Have a look at open issues if you want to find a topic to work on,
  4. Do not hesitate to have a look at good first issues or help wanted issues if you search for something to start with!
  5. Get in touch with us to discuss about what you'd like to contribute if you don't feel like starting alone ;)

Before contributing though, please have a look at our Code of Conduct (because we value humans and their differences) and to our Contribution Guide (because we think that a few rules allow to work faster and safer).

Do not hesitate to discuss anything from those documents if you feel they need any modification though.

Thanks/Inspiration

Approvals-Java is inspired by ApprovalTests.

We really liked the idea of approval testing but not so much the Java implementation (Github).

Our main concerns were that:

  • it is not published on Maven Central, so you need to add the jar manually to your project,
  • it is not actively maintained (Pull Requests are not actively merged),
  • the code style is not up to Java standards (developer is mainly working with .Net).

So we decided to implement quickly a subset of the initial features and deploy the dependency on Maven Central!

Thanks a lot to all the people behind Approvals, because we got the inspiration from their work!

Thanks also to all people who created those tools we love:

The team?

Write Them First! is just a bunch of french developers who strongly believe that automated tests are extremely important in software development.

Since they also value TDD or BDD, they decided to create a few (at least one) tools to make those activities easier!

License

Our code is released under GNU General Public License v3.0.

approvals-java's People

Contributors

aneveux avatar dependabot-preview[bot] avatar dependabot-support avatar dependabot[bot] avatar gbogard avatar tyrcho avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

approvals-java's Issues

Allow to customize the approval files location from annotations in the code

Instead of putting all the approved files in src/test/resources in folders named after the FQN of the class and creating files named after each method, annotations should allow the user to customize the location of generated files:

@ApprovalFiles("src/approvals/files")
class TestFoo {
   @ApprovalFile("barFile")
   @Test
   void bar() {
      Approvals.verify(...);
   }
}

Should allow me to have the generated approved and received files located in src/approvals/files/barFile.

Throws an exception on ubuntu due to recursive symbolic link to /usr/bin/X11

On Ubuntu, due to backwards compatibility reasons, there is symbolic link between /usr/bin/X11 and /usr/bin. Programs expect X11 binaries to be in former but Ubuntu places them in the latter.

This is causing the library to fail on Ubuntu with a java.nio.file.FileSystemLoopException.
This causes the library to fail on Ubuntu bases CI servers such as Semaphore CI too.

I'll publish a fix soon. Meanwhile, here's a complete stack trace :

lambda.infrastructure.code.SSPTemplateEngineSpec *** ABORTED ***
   java.lang.ExceptionInInitializerError:
   at com.github.writethemfirst.approvals.reporters.SupportedOs.<clinit>(SupportedOs.java:45)
   at com.github.writethemfirst.approvals.Reporter.<clinit>(Reporter.java:53)
   at com.github.writethemfirst.approvals.approvers.Approver.<init>(Approver.java:78)
   at lambda.infrastructure.TestUtils$Approbation.$init$(TestUtils.scala:27)
   at lambda.infrastructure.code.SSPTemplateEngineSpec.<init>(SSPTemplateEngineSpec.scala:9)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
   at java.lang.Class.newInstance(Class.java:442)
   ...
   Cause: java.io.UncheckedIOException: java.nio.file.FileSystemLoopException: /usr/bin/X11
   at java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:88)
   at java.nio.file.FileTreeIterator.hasNext(FileTreeIterator.java:104)
   at java.util.Iterator.forEachRemaining(Iterator.java:115)
   at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
   at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
   at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
   at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
   at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
   at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
   at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
   ...
   Cause: java.nio.file.FileSystemLoopException: /usr/bin/X11
   at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:294)
   at java.nio.file.FileTreeWalker.next(FileTreeWalker.java:372)
   at java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:84)
   at java.nio.file.FileTreeIterator.hasNext(FileTreeIterator.java:104)
   at java.util.Iterator.forEachRemaining(Iterator.java:115)
   at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
   at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
   at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
   at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
   at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
   ...

document / write an example of approving a website

I think we could use approvals-java together with something like selenium to extend the concept of approval testing to UI testing.

This might be an Epic rather than a Story ... Possible steps:

  • check that approving an image works (probably need to integrate new reporters which support image diff like this one). We'll also need to define what kind of files a reporter supports (Image / Text / ???).
  • check it also works on CI (with no software installed, how to do image diff in pure java ?)
  • document approving an image
  • document how to get an image from a website (using Selenium for instance - SO)
  • optionnally document how to get an image from a Swing application (using Java Robot)
  • give a full example explaining the approach

Inspiration:

Allow to compare full directories instead of only files

When computing approval tests, the framework should allow not only to compare single files, but to compare full directories containing several files/folders.

Given a batch method, generating directories and files,
When running the approval tests,
Then the framework should compare all the generated data (files and folders hierarchy) with a reference one, committed along with the source code.

Use VSCode for diff

Hello,

In order to use VSCode diff, you can use this config in approvals.java

/usr/local/bin/code //// --diff %received% %approved%

Detect diff & merge softwares installed on the developer's workstation and execute them for the approval merge

When computing the approval tests and facing an empty or different approved file, the framework should allow merge tools to be executed automatically to review and validate the outputs of the program.

This feature should work on any OS, and here's the first list of supported tools to look for:

Windows support

  • IntelliJ IDEA diff tools
  • GVim
  • KDiff 3
  • Beyond Compare 3
  • Beyond Compare 4
  • TortoiseIDiff
  • TortoiseMerge
  • WinMergeU
  • Araxis Merge
  • Code Compare

Linux support

  • IntelliJ Ultimate IDEA diff tools
  • Beyond Compare
  • IntelliJ Community IDEA diff tools
  • GVim
  • KDiff 3

Mac OS X support

  • IntelliJ IDEA diff tools
  • KDiff 3
  • GVim
  • DiffMerge
  • Beyond Compare
  • Kaleidoscope
  • p4merge
  • tkdiff

See legacy implementation

Allow to use a specific formatter while computing verification of a particular object

As per now, the framework relies on a toString call from the object under verification so it can be written in a file.

In some cases, that might not be convenient (if the toString method is already used for something else, or not providing the information you'd like). The framework should allow to specify a formatter function for that.

For example:

Approvals.verify(anObject, { o -> o.aField + " - " + o.anotherField });

Should allow to format the output not calling toString from the provided object, but calling the formatter function instead.

  • format Map (order keys, one key/value per line)
  • format collections (one entry per line)

Continuous Deployment of SNAPSHOT versions on Bintray

All commits should lead to Continuous Deployment builds to be triggered and SNAPSHOT versions to be pushed on bintray.

Note:

  • The repository location should be added to the documentation for developers to test upcoming releases

Approval Files formatter should include arguments' name

When computing tests for methods with several arguments, approval files should be formatted so all the arguments' names are kept in the file.

Given a method foo(a,b,c) which is tested by our framework, and producing result when called,
When approval tests are computed and the approval files are generated,
Then the produced file should be formatted as follows:

a=x; b=y; c=z; output=result;

So the file remains readable by a human.

Support kdiff3 on Linux

Since kdiff3 is a multi-platform program, it would make sense to add it to the list of supported linux reporters, so we have a consistent behaviour across platforms.

I could make a PR

When comparing directories, allow to ignore some specified files

Requirement: #11

When comparing a generated directory with a reference one, it should be possible for the developer to specify files to be ignored during the comparison.

For a first step, the files to be ignored could be specified as arguments to the test method which is called. Later on, a file similar to .gitignore could be used to simplify the management of those ignored files.

See Legacy implementation (not sure we want to do it the same way)

Framework binaries should be deployed to Maven Central

For easing the framework's usage, the framework's jar (along with source jars, and javadoc jars) should be deployed to Maven Central.

As a first step, we'd like them to be deployed to a Bintray repository, and to JCenter. But Maven Central is a must as well.

Allow to test a method with a set of arguments

When writting tests for a method with multiple arguments, the framework should provide a way to compute all combinations of possible arguments from values provided by the developer.

Given a method foo(a,b,c) which is tested by our framework,
When a developer is calling Approvals.verifyAll(::foo, aValues, bValues, cValues),
Then the framework should compute all combinations of provided values for all parameters, and run the verification for all those cases.

For example:

Testing a+b with a={1,10,100} and b={2,3} should run all following verifications:

a=1; b=2
a=10; b=2
a=100; b=2
a=1; b=3
a=10; b=3
a=100; b=3

provide more details about the position at which the difference happens

For instance, this error is difficult to fix:

[info]   java.lang.AssertionError: expected: <Success(
[info]   Event(
[info]     "T\ufffdrmilihus Winterthur",
[info]     "https://www.mtgtop8.com/event?e=18427",
[info]     8,
[info]     Sun Jan 28 00:00:00 CET 2018,
[info]     Highlander,
[info]     None
[info]   )
[info] )
[info] > but was: <Success(
[info]   Event(
[info]     "T\ufffdrmilihus Winterthur",
[info]     "https://www.mtgtop8.com/event?e=18427",
[info]     8,
[info]     Sun Jan 28 00:00:00 UTC 2018,
[info]     Highlander,
[info]     None
[info]   )
[info] )>

Refactoring of all approvers and Approvals entry point

The Approvals object and all the Approver objects are to be refactored.

Here are some thoughts after reading again the source code:

  • Documentation should be updated to state that Approvals is an easy entry point for accessing only default configuration,
  • Approvers can be used for advanced configuration of your tests and validations,
  • The custom extension should disappear. Using the csv extension might sound appealing (because of opening the files in other editors), but it actually is really confusing and might lead to other issues (ignoring csv in SCMs, etc.) - I can elaborate on that topic, but my opinion is: we can format the file as if it is a csv file, but it needs to remain a .approved file,
  • The custom file name isn't a feature we planned yet, and there is more to think about it than just an entry in the builder object, so it should disappear as well (I'm in favor of an agile approach for that: we implement if and only if we need it (on a real use case))
  • As far as possible, the source code itself should allow to understand that Approvers are actually just advanced entry points for approvals. Maybe putting them in an advanced package could do the trick?
  • Global refactoring (variable names, functions, etc.)
  • Javadoc the whole thing
  • Review the error messages and align them to all the other error messages
  • Remove from the ApprovalFiles class all the things linked to the extensions

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.