• Use standardized method to read and write config
• nice to have: Create config if missing (store defaults in Java classes, write if file missing, allows us to remove this file from repository)

jacop

https://github.com/wwu-pi/muggl/tree/jacop

980d0c3

The InsertionSort.java file contains among others, two dummy methods:
sillyFunction(int[] array, int position)
sillyFunction(int[] array, Integer position)

The latter works fine, the former throws a ClassCastException.

Muggl-SWT will not start on MacOS.

Given the following Java method (can be found in IntegerDivision: 9fe4389 ):

    public int simpleArray(int x) {
        int[] numbers = {x,0};
        return numbers[1];
    }

The javac compiler (openjdk: javac 1.6.0_24) generates the following bytecode:

00:  iconst_2
; Array initialization:
01:  newarray  10
03:  dup
04:  iconst_0
05:  iload_1
06:  iastore
; Return numbers[1]:
07:  astore_2
08:  aload_2
09:  iconst_1
10:  iaload
11:  ireturn

Only the zero-th position of the array is initialized. The first one is not, as JVM's newarray specification specifies the following: Each of the elements of the new array is initialized to the default initial value (§2.3, §2.4) for the element type of the array type.
[Taken from: http://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.html#jvms-6.5.newarray]

So, there is no need to initialize numbers[1] explicitly in the JVM code. However, Muggl initializes numbers[1] to null. As a consequence, null is returned in a method that should return an int. This generates the following test case, which is ill-typed:

        this.reference0 = null;
                ...
        assertEquals(this.reference0, this.testedClass.simpleArray(this.int1));

When selecting a .jar file on the left pane of the main window, the middle pane is always deactivated and empty.

This implies that classes at the root of a jar cannot be tested. This is not a major problem since this is bad practice, but it is also not documented anywhere. Therefore, this behaviour should be either explicated or fixed.

Herbert reported a crash in the GCD example.

Please provide us with some log or steps to repeat the behavior!

When build is triggered on Travis-CI by Github. The build runs so slow, that even simple test case run into org.junit.runners.model.TestTimedOutException and finally the whole build will timeout

This is called in the initializer of the IntegerCache of the java8-runtime. The IntegerCache is used for comparing integers in [-128, 127].

When calling VM.getSavedProperty("java.lang.Integer.IntegerCache.high"); the getSavedProperty will test if the system's properties are not empty.

Currently they are. Our VM has neither properties like java.runtime.name: OpenJDK Runtime Environment.

How to proceed?

There is no spec for initialization of the Runtime Lib.
The Hotspot VM does call some sort of special initializer when it sets up a thread
hotspot/src/share/vm/runtime/thread.cpp:1048: JavaCalls::call_static(&result, klass, vmSymbols::initializeSystemClass_name(), , where it effectively calls java.lang.System.initializeSystemClass()

Possible Short-Term Mitigations

Either one of these:

• Patch a special rt.jar that does not test if the properties are empty.
• Implement proper initialization of the java/lang/System.java, has anyone attempted that?

"open test case file" - does not open test case file

In Java 8 some interfaces have a default method, that has a default implementation of that method. Consequently, this method is not abstract.

Before Java 8 every method in an interface was abstract. Muggl had a check for this in de.wwu.muggl.vm.classfile.structures.Method#parseAccessFlags:

if (this.classFile.isAccInterface()) {
  ...
  if (!this.accAbstract)
    throw new ClassFileException("An interface method must have its ACC_ABSTRACT flag set.");
  ...

executing this MWE compiled function

public static void convert() {
    Collection<String> myList = Arrays.asList("Hello", "Java");
    myList.iterator();
}

in Normal VM execution mode, leads to an error (shortened log)

Parsing class /usr/lib/jvm/java-6-openjdk-amd64/jre/lib/rt.jar|java/util/List.class
...
Executing invokeinterface 0 27 1 0 // = java.util.Collection iterator iterator
...
The executed application threw an exception but no suitable exception handler was found. Uncaught exception: java.lang.IncompatibleClassChangeError (java.util.Arrays$ArrayList does not implement interface java.util.Collection.)

$java -version
java version "1.6.0_39"
OpenJDK Runtime Environment (IcedTea6 1.13.11) (6b39-1.13.11-0ubuntu0.14.04.1)
OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)
$ cat /etc/issue
Ubuntu 14.04.4 LTS

Add support for EAR file format.

Seriously

[ ] Unit Test Cases for core components

If eliminating via Flow Coverage, we get a multitude of test cases, where actually one test case would be sufficient (and correct).

This should be adapted to actually eliminate test cases based upon coverage.

The following method is given with two double parameters as input:

public void doSimple(double d1, double d2) {
  SimpleEntity se = new SimpleEntity(d2, d2);
  entityManager.persist(se);
}

The Java Instructions are:

  0: new           #20
  3: dup
  4: dload_3
  5: dload_3
  6: invokespecial #22
  9: astore        5
 11: aload_0
 12: getfield      #24
 15: aload         5
 17: invokeinterface #26,  2
 22: return

Since a double value takes two slots (on the operand stack, in the local variable table, etc.), the local variable table is as follows:

LocalVariableTable:
  Start  Length  Slot  Name   Signature
      0      23     0  this   Lde/wwu/pi/muggl/services/SimpleService;
      0      23     1    d1   D
      0      23     3    d2   D
     11      12     5    se   Lde/wwu/pi/muggl/services/entities/SimpleEntity;

Notice that d2 is at index 3, not at 2 (even though it is the second method argument...).

However, the Load instruction in Muggl does not consider that fact:

java.lang.ArrayIndexOutOfBoundsException: 3
    at de.wwu.muggl.vm.classfile.structures.Method.setVariable(Method.java:770)
    at de.wwu.muggl.instructions.general.Load.executeSymbolically(Load.java:192)
    at de.wwu.muggl.instructions.general.Load.executeSymbolically(Load.java:111)
    at de.wwu.muggl.vm.impl.symbolic.SymbolicVirtualMachine.executeInstruction(SymbolicVirtualMachine.java:455)
    at de.wwu.muggl.vm.impl.jpa.JPAVirtualMachine.executeInstruction(JPAVirtualMachine.java:38)
    at de.wwu.muggl.vm.VirtualMachine.executeFrame(VirtualMachine.java:503)
    at de.wwu.muggl.vm.impl.symbolic.SymbolicVirtualMachine.executeFrame(SymbolicVirtualMachine.java:377)
    at de.wwu.muggl.vm.impl.jpa.JPAVirtualMachine.executeFrame(JPAVirtualMachine.java:25)
    at de.wwu.muggl.vm.VirtualMachine.runMainLoop(VirtualMachine.java:427)
    at de.wwu.muggl.vm.impl.symbolic.SymbolicVirtualMachine.runMainLoop(SymbolicVirtualMachine.java:272)
    at de.wwu.muggl.vm.VirtualMachine.run(VirtualMachine.java:239)

Last line here throws error:

    public void setVariable(int index, Variable variable) {
        // Initialize the array, if needed.
        if (this.variables == null) {
            if (this.codeAttribute == null) {
                throw new IllegalStateException("The method has no code!");
            }
            this.variables = new Variable[getNumberOfParameters()];
        }

        // Set the new one.
        this.variables[index] = variable;
    }

java-6-openjdk
compiler level 1.6

Muggl's method parameter indexing differs from that of the JVM.

In the JVM, parameters are indexed by the position of their first byte.
For example, the method test.preconditions.Average#division(double a, int b) stores three parameters (index - name - type):
0 - this - Ltest/preconditions/Average
1 - a - double
3 - b - int,
where b has the index 3 because a (double) occupies 2 bytes of space.

Muggl internally manages parameters in an array, that is indexed only by the parameters. That is, b has the index 2.

Consequently, trying to get the parameter b during symbolic execution using 3 as an identifier results in a java.lang.ArrayIndexOutOfBoundsException exception.

DU chain generation throws an exception for certain programs under test.

For example, starting test generation for the sample program test.misc.JavaMemoryPuzzleImpolite#main(String[]) results in

NullPointerException in ObjectAttribute, line 39,
essentially caused by the call to getObjectRefAtCurrentPos() at DUGenerator, line 550. Its return value is null, because for Newarray no value was pushed to the stack.

Commit id: 8498b7d

public int simpleArraySwap(int[] array) {
   swap array[0] and array[1]
   if (array[0] > array[1]) return 1;
   return 0;
}

This generates a test case (also uploaded) that passes the array {0,-1} to the function, and it expects to return 1, while it actually returns 0.

Use the Byte Code Engineering Library (Apache Commons BCEL™)

Creates inequalities comparing IntVariables to null (instead of expected IntConstants)

Only one condition is tested in a simple if-else construct.

Tried with different Coverage options, resulting all in same generated test case (see below).

Code to test:

public class Foobar {
    public boolean fooTheBar(int x) {
        if(x > 1000) {
            return true;
        } else {
            return false;
        }
    }
}

Generated Test Case:

public class TestClass12 {
    // Fields for test parameters and expected return values.
    private de.wwu.pi.dbtest.sampleapp.Foobar testedClass;
    private boolean booleanfalse;
    private int intm50;

    /**
     * Set up the unit test by initializing the fields to the desired values.
     */
    @Before public void setUp() {
        this.testedClass = new de.wwu.pi.dbtest.sampleapp.Foobar();
        this.booleanfalse = false;
        this.intm50 = -50;
    }

    /**
     * Run the tests on de.wwu.pi.dbtest.sampleapp.Foobar.fooTheBar(int x).
     */
    @Test public void testIt() {
        assertEquals(this.booleanfalse, this.testedClass.fooTheBar(this.intm50));
    }

    /**
     * Invoke JUnit to run the unit tests.
     * @param args Command line arguments, which are ignored here. Just supply null.
     */
    public static void main(String args[]) {
        org.junit.runner.JUnitCore.main("test.unitTests.TestClass12");
    }
}

With this example:
public class VerySimple {

public static int fib(int n) {
    int r = 1;
    if (n > 1)
        r = n * fib(n - 1);
    return r;
}

}

An EmptyStack error is obtained. It seems there is some race condition: if a breakpoint is introduced before the error occurs, it dissapears.

It seems that the tool only generates test-cases properly if the methods are static.

Related to #10

In the same example, we might only be seeing on test case, though for a complete coverage of the DU Chains, additional test cases are required.

From the description, I'd deduct that the specific code could be adapted to employ a different library for the purpose of RGB color representation, so that this specific dependency may be omited.

In the example of GCD, the DUGenerator might not add all DU chains existing.

Consider the following example

while(n != m) {
        if (n > m) {
            n -= m;
        } else {
            m -= n;
        }
    }
    return n;

we might miss DU chains, since finish the branch "n-=m" the moment we found it.

Currently, we can only terminate, if an additional solution is not found in a defined amount of instructions executed.

We should improve this to automatically terminate upon the algorithm reaching a predefined search-depths.

A method to test with following signature:
public boolean doesUserExist(long key)

The de.wwu.muggl.solvers.jacop.JaCoPSolverManager creates a NumericConstant.INT in method getSolution, because the found solution (e.g., -50) is of type IntDomain.

In de.wwu.muggl.symbolic.testCases.SoltionProcessor#generateTestCases the following if-clause is used, that is never true:

if (parameterTypes[a - startAt].equals("long") && parameters[a] instanceof LongConstant)

This is because parametersTyes[x] is long , but parameters[a] is of type IntConstant , and not LongConstant.

Could not load local variable #0: Expected java.lang.Integer, got class java.lang.Boolean.

see 2.3.4 The boolean Type. boolean should be treated as int internally.

    public static Boolean BooleanBoxing(boolean in) {
        return Boolean.valueOf(in);
    }

bytecode:

public static java.lang.Boolean BooleanBoxing(boolean);
    descriptor: (Z)Ljava/lang/Boolean;
    flags: ACC_PUBLIC, ACC_STATIC
    Code:
      stack=1, locals=1, args_size=1
         0: iload_0
         1: invokestatic  #82                 // Method java/lang/Boolean.valueOf:(Z)Ljava/lang/Boolean;
         4: areturn