sqisher / java-object-diff

Library to diff and merge Java objects with ease

Introduction

java-object-diff is a simple, yet powerful library to find differences between Java objects. It takes two objects and generates a tree structure that represents any differences between the objects and their children. This tree can then be traversed to extract more information or apply changes to the underlying data structures.

Features

• Works out-of-the-box with with almost any kind of object and arbitrarily deep nesting
• Finds the differences between two objects
• Returns the differences in shape of an easily traversable tree structure
• Tells you everything there is to know about the detected changes
• Provides read and write access to the underlying objects, allowing you not only to extract the changed values but even to apply the diff as a patch
• Requires no changes to your existing classes (in most cases)
• Provides a very flexible configuration API to tailor everything to your needs
• Tiny, straightforward, yet very powerful API
• Detects and handles circular references in the object graph
• No runtime dependencies except for SLF4J
• Compatible with Java 1.5 and above

Support this Project

If you like this project, there are a few things you can do to show your support:

• Follow me on Twitter (@SQiShER)
• Surprise me with something from my Amazon Wishlist
• Contribute code, documentation, ideas, or insights into your use-case
• Star this repository (stars make me very happy!)
• Talk about it, write about it, recommend it to others

But most importantly: don't ever hesitate to ask me for help, if you're having trouble getting this library to work. The only way to make it better is by hearing about your use-cases and pushing the limits!

Getting Started

To learn how to use Java Object Diff have a look at the Getting Started Guide.

Using with Maven

<dependency>
    <groupId>de.danielbechler</groupId>
    <artifactId>java-object-diff</artifactId>
    <version>0.95</version>
</dependency>

Using with Gradle

compile 'de.danielbechler:java-object-diff:0.95'

Documentation

The documentation can be found over at ReadTheDocs.

Caveats

• Introspection of values other than primitives and collection types is curently done via standard JavaBean introspection, which requires your objects to provide getters and setters for their properties. However, you don't need to provide setters, if you don't need write access to the properties (e.g. you don't want to apply the diff as a patch.)

  If this does not work for you, don't worry: you can easily write your own introspectors and just plug them in via configuration API.

• Ordered lists are currently not properly supported (they are just treated as Sets). While this is something I definitely want to add before version 1.0 comes out, its a pretty big task and will be very time consuming. So far there have been quite a few people who needed this feature, but not as many as I imagined. So your call to action: if you need to diff and/or merge collection types like ArrayList, perhaps even with multiple occurence of the same value, please let me know. The more I'm aware of the demand and about the use-cases, the more likely it is, that I start working on it.

Why would you need this?

Sometimes you need to figure out, how one version of an object differs from another one. One of the simplest solutions that'll cross your mind is most certainly to use reflection to scan the object for fields or getters and use them to compare the values of the different object instances. In many cases this is a perfectly valid strategy and the way to go. After all, we want to keep things simple, don't we?

However, there are some cases that can increase the complexity dramatically. What if you need to find differences in collections or maps? What if you have to deal with nested objects that also need to be compared on a per-property basis? Or even worse: what if you need to merge such objects?

You suddenly realize that you need to scan the objects recursively, figure out which collection items have been added, removed or changed; find a way to return your results in a way that allows you to easily access the information you are looking for and provide accessors to apply changes.

While all this isn't exactly rocket science, it is complex enough to add quite a lot of extra code to your project. Code that needs to be tested and maintained. Since the best code is the code you didn't write, this library aims to help you with all things related to diffing and merging of Java objects by providing a robust foundation and a simple, yet powerful API.

This library will hide all the complexities of deep object comparison behind one line of code:

DiffNode root = ObjectDifferBuilder.buildDefault().compare(workingObject, baseObject);

This generates a tree structure of the given object type and lets you traverse its nodes via visitors. Each node represents one property (or collection item) of the underlying object and tells you exactly if and how the value differs from the base version. It also provides accessors to read, write and remove the value from or to any given instance. This way, all you need to worry about is how to treat changes and not how to find them.

This library has been battle-tested in a rather big project of mine, where I use it to generate activity streams, resolve database update conflics, display change logs and limit the scope of entity updates to only a subset of properties, based on the context or user permissions. It didn't let me down so far and I hope it can help you too!

Contribute

You discovered a bug or have an idea for a new feature? Great, why don't you send me a Pull Request so everyone can benefit from it? To help you getting started, here is a brief guide with everyting you need to know to get involved!

---

Thanks to JetBrains for supporting this project with a free open source license for their amazing IDE IntelliJ IDEA.

java-object-diff's Issues

The PrintingVisitor should print the RootNode in case nothing changed

The PrintingVisitor only prints nodes, that have changed in some way and do not contain any children. This way the output will not be flooded with nodes that have implicit changes, because one of their children has changed. While this is pretty neat, I think it makes sense to print the root node, in case it didn't change and doesn't contain any children. I could be wrong, but right now this seems reasonable to me.

Native types are not being compared properly

If I have a bean with native types, the comparison is not being performed properly in some cases.
For what I could understand, it does not work if the value in the "working" bean is the default value for the given data type (e.g 0 for int or false for boolean).

Odd as it may seem, it all works if the given bean class has private access (inner class), but does not work if is public.

Check the following test case where I show three examples.

By the way, this is a nice piece of software!

import de.danielbechler.diff.ObjectDiffer;
import de.danielbechler.diff.ObjectDifferFactory;
import de.danielbechler.diff.node.Node;
import de.danielbechler.diff.visitor.PrintingVisitor;

public class TestCase {

    // class has public access
    public static class TestIntBeanPublic {

        private int value;

        public int getValue() {
            return this.value;
        }

        public void setValue(final int value) {
            this.value = value;
        }

        @Override
        public String toString() {
            return "TestIntBeanPublic [value=" + this.value + "]";
        }
    }

    // class has private access
    private static class TestIntBeanPrivate {

        private int value;

        public int getValue() {
            return this.value;
        }

        public void setValue(final int value) {
            this.value = value;
        }

        @Override
        public String toString() {
            return "TestIntBeanPrivate [value=" + this.value + "]";
        }
    }

    public static void testIntegerFailsPublic() {

        TestIntBeanPublic working = new TestIntBeanPublic();
        working.setValue(0);

        TestIntBeanPublic base = new TestIntBeanPublic();
        base.setValue(1);

        ObjectDiffer differ = ObjectDifferFactory.getInstance();
        final Node root = differ.compare(working, base);

        root.visit(new PrintingVisitor(working, base));
    }

    public static void testIntegerWorksPrivate() {

        TestIntBeanPrivate working = new TestIntBeanPrivate();
        working.setValue(0);

        TestIntBeanPrivate base = new TestIntBeanPrivate();
        base.setValue(1);

        ObjectDiffer differ = ObjectDifferFactory.getInstance();
        final Node root = differ.compare(working, base);

        root.visit(new PrintingVisitor(working, base));
    }

    public static void testIntegerWorksPublic() {

        TestIntBeanPublic working = new TestIntBeanPublic();
        working.setValue(2);

        TestIntBeanPublic base = new TestIntBeanPublic();
        base.setValue(1);

        ObjectDiffer differ = ObjectDifferFactory.getInstance();
        final Node root = differ.compare(working, base);

        root.visit(new PrintingVisitor(working, base));
    }

    public static void main(final String[] args) {

        System.out.println("The following comparison works properly (detects change)");
        TestCase.testIntegerWorksPublic();
        System.out.println("The following comparison does not work properly (detects removal)");
        TestCase.testIntegerFailsPublic();
        System.out
                .println("The following comparison works properly bewcause the class has private access (detects change)");
        TestCase.testIntegerWorksPrivate();
    }
}

bi-directional relationships

Hello,

I have some questions about dealing with bi-directional relationships (and circular relationships in general). Please drop me a mail so that we can discuss things.

Thanx, Oliver

The introspector should cache its results

On a per class basis; ideally in an LRU cache. Maybe with soft references?

[question] can node.isAdded() be more granular

Hi,

I don't know how to explain this other than with an example:

class A {

  private Class B;

  // getters, setters etc
}

class B {
   private String c1;
   private String c2;

   // getters, setters etc
}

and then compare a1 against a2:

A a1 = new A();
A a2 = new A();
B b1 = new B();
b1.setC1(sometext)
b1.setC2(othertext)
a2.setB(B)

I will get one Node in the visitor that shows me that a complex type B has been added to a2. What i would be interested in actually is to receive 3 nodes: one for the addition of B to a2, then one for the addition of C1 property to b and one for the addition of C2 property to b.

Does this make sense ?

How to ignore certain fields for comparison?

I want to compare 2 objects but wanted to specify some fields that should be ignored for comparison. Can it be done?

Allow changing of the equivalence mechanism used for collection diffing.

At the moment the CollectionDiffer relies on the equals to check if the items refer to the same entity. It would be awesome to be able to parameterize this. e.g (pseudocode for brevity).

First Scenario:

A = {collection: [{id: 1, name: "walter", age: 49}]}
B = {collection: [{id: 1, name: "walter", age: 50}]}
equals: A.id == B.id

Result: CHANGED /collection/1/age of 49 => 50

Second Scenario:

In a different context, the equivalence of A and B could be different e.g.:
A = {collection: [{id: 1, name: "walter", age: 49}]}
B = {collection: [{id: 1, name: "walter", age: 50}]}
equals: A.id == B.id && A.name == B.name && A.age == B.age

Result: REMOVED /collection/1
ADDED /collection/1

In my situation, it is important that I retain the Object.equals defined in the second scenario, but would like to diff based on the equivalence in the first. Be able to parameterize the equivalence mechanism used by java-object-diff I believe would be a solution for this.

Diffing sets of complex objects while using equals only properties

We noticed that while trying to diff two objects that have sets of complex objects after having configured equals only properties, differences in the objects in the set weren't being diffed:

public class DiffTest {


    private class Thing {
        private String a;
        private String b;

        public Thing(String a, String b) {
            this.a = a;
            this.b = b;
        }
        // Equals, hash code, getters
    }

    private class ThingHolder {
        private Set<Thing> things;
        private String ignore;
        private String include;


        private ThingHolder(Set<Thing> things, String ignore, String include) {
            this.things = things;
            this.ignore = ignore;
            this.include = include;
        }
        // Equals, hash code, getters
    }


    @Test
    public void shouldDiffThings() {
        List<String> propertyNames = listWith("things", "include");
        Configuration configuration = new Configuration();
        for (String name : propertyNames) {
            PropertyPath propertyPath = PropertyPath.buildWith(name);
            configuration = configuration.withEqualsOnlyProperty(propertyPath).withPropertyPath(propertyPath);
        }

        Thing thingOne = new Thing("a", "b");
        Thing thingTwo = new Thing("aa", "bb");

        ThingHolder first = new ThingHolder(setWith(thingOne), "ignore", "include");
        ThingHolder second = new ThingHolder(setWith(thingTwo), "ignore this change", "include");
        Node compareResults = ObjectDifferFactory.getInstance(configuration).compare(first, second);

        assertThat(compareResults.isChanged(), is(true));
    }
}

I think I've tracked it down to the nodes created for the items in the sets. Their property paths include a collection element meaning they won't pass the includedProperties.contains(node.getPropertyPath()) check called by isIgnored(node) in BeanDiffer line 59.

The whole test is at https://gist.github.com/4122745

Create a Visitor to Print the Entire Object Graph

The ToStringPrintingVisitor already gets the job done, but could use a better name and could use a little face lift.

[question] support for diffing different collection types

Hi,

One thing we often do in a domain class is to declare a member collection as for example Sets.newHashSet() and in the getter we return a defensive copy like ImmutableSet.copy(this.myset) to prevent callers from inadvertently modifying the domain data.

This leads java-object-diff to throw an exception about incompatible object types. Will this case be supported in a future version ?

Thanks !
Jorg

Issue with circular

In BeanDiffer i've got a property node with a toString of:

DefaultNode(state=CIRCULAR, type=java.util.Collection, no children, accessed via property 'structureDiagram')

In print visitor it says:
Property at path '/xxxxx' has already been processed at another position. (Circular reference!) (Circular reference detected: The property has already been processed at another position.)

What I would expect here instead is for it to have the previous diff reported or some reference to the node that had previously been processed.

Is that possible?

Improve property path elements by enforcing proper implementation of equals, hashCode and toString instead of proxy methods

As explained here: http://stackoverflow.com/a/1612299/1137104

It should be possible to merge Arrays

Array handling is currently not supported.

How to diff Lists?

First of all, thanks for this great library!

I was wondering what is your idea about comparing collections? The DelegatingObjectDifferImpl uses the CollectionDiffer to compare both Sets and Lists. While List order is relevant, the differ does not detect changes in the order of the items.

Consider this code:

List<String> left = Arrays.asList("aaa", "bbb", "ccc");
List<String> right = Arrays.asList("aaa", "ccc", "bbb");
ObjectDiffer objectDiffer = ObjectDifferFactory.getInstance();
final Node root = objectDiffer.compare(left, right);
System.out.println("Equals: " + left.equals(right));
root.visit(new PrintingVisitor(left, right));

Results in:

Equals: false
Property at path '/' has not changed

The items 1 and 2 were swapped so the equals says 'false', but this was not detected as a change. Is this behaviour as intended? I would say that the List deserves a ListDiffer.

From the diff node I need access to the original node objects that were diffed

I have a UI were in addition to knowing the diff I want to display the other properties of the original object. Hence I need access to the original objects from the node. I know I can set the differ to also show the nodes that haven't changed but this isn't what I need.

Add convenience methods to access node states

Currently the only way to determine the state of a node is to compare the enum constant returned by node.getState() agains the Node.State enum. It would be better to hide the state flag behind some convenient boolean getters. E.g.:

boolean isUntouched();
boolean isIgnored();
boolean isAdded();
boolean isChanged();
boolean isRemoved();

Need ability to access annotations on objects

Hi.. Can you please add ability to access annotations on objects

in de.danielbechler.diff.accessor.Accessor add
Annotation getNodeAnnotation() throws Exception;
in de.danielbechler.diff.accessor.AbstractAccessor
@OverRide
public Annotation getNodeAnnotation() throws Exception {
return null;
}
in de.danielbechler.diff.accessor.PropertyAccessor
@OverRide
public Annotation getNodeAnnotation() throws Exception {
return readMethod.getAnnotations()[0];
}

This way in visitor I can do node.getNodeAnnotation() that potentially can have more info than just the type, value etc..

@ObjectDiffProperty(ignore = true) is not working in 0.10 version

diff looks up for properties even if @ObjectDiffProperty(ignore = true) annotation is stated. Worked for me in 0.9 version, but after update to 0.10 I have this problem.

collection of complex objects

Hi,

Is it possible that the change detection algorithm of collections is not working properly ?

Consider following case

import org.apache.commons.lang.builder.EqualsBuilder;
import org.apache.commons.lang.builder.HashCodeBuilder;

public class C2 {

    private String contents1;

    private String contents2;

    private String contents3;

    public String getContents1() {
        return contents1;
    }

    public void setContents1(String contents1) {
        this.contents1 = contents1;
    }

    public String getContents2() {
        return contents2;
    }

    public void setContents2(String contents2) {
        this.contents2 = contents2;
    }

    public String getContents3() {
        return contents3;
    }

    public void setContents3(String contents3) {
        this.contents3 = contents3;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o)
            return true;
        if (o == null || getClass() != o.getClass())
            return false;

        C2 c2 = (C2) o;
        return new EqualsBuilder().append(this.contents1, c2.getContents1()).append(this.contents2, c2.getContents2())
                .append(this.contents3, c2.getContents3()).isEquals();
    }

    @Override
    public int hashCode() {
        return new HashCodeBuilder(17, 59).append(this.contents1).append(this.contents2).append(this.contents3).toHashCode();
    }
}

import de.danielbechler.diff.visitor.PrintingVisitor;
import org.testng.annotations.Test;

import com.google.common.collect.Lists;

import de.danielbechler.diff.ObjectDifferFactory;
import de.danielbechler.diff.node.Node;

@Test
public class CompareTest {

    public void test0() {

        C2 c2 = new C2();
        c2.setContents1("aaa");
        c2.setContents2("bbb");
        c2.setContents3("ccc");

        C2 modifiedc2 = new C2();
        modifiedc2.setContents1("aaa");
        modifiedc2.setContents2("bbb");
        modifiedc2.setContents3("ccc111");

        List<C2> a = Lists.newArrayList(c2);
        List<C2> b = Lists.newArrayList(modifiedc2);

        Node root = ObjectDifferFactory.getInstance().compare(b, a);
        final Node.Visitor visitor = new PrintingVisitor(b, a);
        root.visit(visitor);
    }
}

As you can see the C2 object was modified in setContents3. The PrintingVisitor does detect something, but it's listed as an add and remove whereas logically the same object was modified.

Property at path '/[C2@14b78eb0]' has been added => [ C2@14b78eb0 ]
Property at path '/[C2@148a8e25]' with value [ C2@148a8e25 ] has been removed

When i diff the C2 object modifications by itself it does list the property paths that have been changed, just not when it's embedded in a collection.

JPA Entities

I have tried your framework with JPA Entities and it does not work properly. Do I have to do something special?

Ignoring properties from within a map difference

Hi and thanks for sharing this great tool.
I am trying to diff an object with circular dependency -

class ToCompare
{
    Map<Integer, A> myMap;
}

class A
{
    .... some interesting properties to comapre...

    private MyList listObject;
}

class MyList
{
    MyList next;
    MyList prev;
}

The problem is, I'm not diffing two objects of type A but two MAPs of objects of type A.

the messages I get from object-diff look like this:
Detected circular reference in node at path /myMap{39}/listObject/prev/prev/next. Going deeper would cause an infinite loop, so I'll stop looking at this instance along the current path.
I get a lot of lines like this.

I try to use configuration.withoutProperty(PropertyPath.buildWith("myMap", "listObject"));
and it doesn't work. since listObject is not a property of myMap - I need to gain access to the actual values of myMap.

Do you happen to have any solution to this?

Thanks.

Support for returning String from PrintingVisitor

The PrintingVisitor's output is fantastic. We have found use for java-object-diff in our project while writing unit tests.

Instead of requiring intricate coding to determine the diff or letting the developer infer the problem from the sys.out having the PrintingVisitor's output really helps.

Typically while writing units tests, the error messages are printed using:

Assert.equals(expected, actual, "message")

It will be great if we can pass PrintingVisitor's output to Assert.

Map keys should be checked for differences, too

Currently they only serve as identifiers for map values (via equals())

Integrate (optional) longest common subsequence detection for text nodes

The API could look like this:

if (node.isTextNode())
{
    TextNode textNode = node.asTextNode();
    LCS lcs = textNode.getLongestCommonSubsequences(working, base);
    ...
}

Since this is an optional post-comparison feature, it's an addition without sacrifices.

please deploy to maven central

Makes it much easier to use in a corporate environment, it's not easy to request addition of someones private repository to the corporate nexus instance.

For now i'm building myself and add it manually to our nexus instance but having this published to central would be better.

@ObjectDiffProperty(ignore = true) for interfaces

if I define @ObjectDiffProperty(ignore = true) for interface method, object diff will not respect this annotation for instances of interface and call methods of instance.

Move Maven Repository away from GitHub

I rather host the repository on my own server instead of GitHub, so I can guarantee the URL never changes.

"logback.xml" from test package is included into JAR file

The "logback.xml" configuration file from "src/test/ressources" directory is included into the JAR file due to the use of the Maven Resources plugin.

<plugin>
    <artifactId>maven-resources-plugin</artifactId>
    <configuration>
    <outputDirectory>${basedir}/target/classes</outputDirectory>
        <encoding>UTF-8</encoding>
    </configuration>
</plugin>

The "logback.xml" configuration file from java-object-diff is used instead of webapp dedicated file on Tomcat:

INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [jar:file:/tomcat/webapps/my-web-app/WEB-INF/lib/java-object-diff-0.9.jar!/logback.xml]
WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs multiple times on the classpath.
WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [file:/tomcat/resources/logback.xml]
WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/tomcat/webapps/my-web-app/WEB-INF/lib/java-object-diff-0.9.jar!/logback.xml]

Release Version 0.9

Make Library OSGi-compatible

Here are some useful resources:

• http://www.stoyanr.com/2012/09/making-plain-old-java-osgi-compatible.html
• http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html
• http://www.aqute.biz/Bnd/Format

NullPointerException when comparing Map values that are null in base and working instance

The MapDiffer delegates the comparison of those values, even though it could easily detect that the value is unchanged.

Create fluent Configuration API

Tweak output format of printing visitors

Write a Wiki Page about Circular Reference Handling

Make type detection for collections more reliable, by handing the runtime value type to the nodes

Switch to Apache License

Implement toString method of Accessors

May be useful for debugging.

Prevent construction of invalid PropertyPaths

Create a fluent Builder API to guarantee PropertyPath sanity.

Improve format of PropertyPath.toString()

Currently, it's pretty hard to see which element in the path represents what. Switching the '.' format to a more recognizable and commonly known '/' format should improve readability. It would also be nice to express the relationship between collections and their elements.

Release Notes for Version 0.9

This version contains some game changers and API breaks. The most significant are:

Circular Reference Detection

When a circular reference is detected (e.g. A > B > C > A), only the first occurrence of an instance along a property path will be scanned for changes. Subsequent occurrences of the same instance will return nodes marked as circular. These cases should have resulted in infinite loops before, but I rather point it out, so everyone is aware of this rather subtile change.

Property Path Builder API

There were many ways to construct a PropertyPath. This provided a lot of room to create invalid paths. In order to guarantee validity of every PropertyPath instance, construction is now exclusively handled by the PropertyPathBuilder, which has been converted to a guiding fluent API. This requires some adjustments in the client code.

Additional Changes

https://github.com/SQiShER/java-object-diff/issues?milestone=2&page=1&state=closed

BeanDiffer accesses ignored properties before they get dropped

This should not happen because some methods are marked as ignored because they perform heavy calculations or depend on a certain object state. The only reason why ignored accessors are not discarded right away during introspection is to be able to add them as ignored nodes to the result tree.

Get value of MapElement

Not really sure if this is the right place for a question, but I saw you answer here, so I'll give it a try.
First off, amazing library!!

I am trying to figure something and I can't find any example in your code on how to do it.

I created a custom visitor.
Using the accept function I am able to traverse all changes.

I am trying to get the actual element behind a node (element is in a collection).
Once I reach such an element I wrote the following code that works:

Map:
Integer key = (Integer) ((MapElement) node.getPathElement()).getKey();

List/Set:
SomeObject object = (SomeObject) ((CollectionElement) node.getPathElement()).getItem();

These seem to work, but I wonder...

1. Is there an easier way to perform this task?
2. How can I get the value in a mapElement?

Thanks for this great library!

Create fluent Node Assertion API for easier Testing

Verifying node states is pretty inconvenient, so a nice little fluent API could do wonders to improve the readability of the unit tests.

Prevent nodes to be added as their own children

The biggest pitfall during the tree population is to accidentally add the wrong node. Failing early is a nice way to discover those bugs quickly.

About Object Diff api

I am finding hard time using objectDiff functionality for comparing two similar type objects with collections as properties.
I need your assistance in using this api. My requirement is to compare two objects and return all the changes/additions/deletions between two versions of the similar object.

As far as my understanding, Printing visitor just prints differences between two simple data types. How should I use your api to get all the changes including collection objects? Thanks for your wonderful API. Its just that I could not use your api correctly.

Bring back NamedPropertyAccessor#getPropertyName()

Java 1.5

Hi,
This project is exactly what I need... except the Java version. Do you know if your classes would compile and function properly in java 1.5?

Thanks for the help.

David.

No example on how to merge objects

please provide some examples on how to merge objects if possbile.

Rename {is,to}{Map,Collection}Difference to {is,to}{Map,Collection}Node

Title says it all.

Printing the differences if the root has children

PrintingVisitor prints the differences in string if the node has no children. Do we have any methods which prints the differences in object if node has children