Resulting JUnit execution reports duplicate failures for same test case about randoop

[GoogleCodeExporter]

When I say all test cases pass... no errors and no failures.

Original comment by [email protected] on 21 Oct 2014 at 4:46

[GoogleCodeExporter]

Can you provide concrete steps to reproduce the error?
You have given only vague descriptions, so I am not sure exactly what you are 
doing.  If the developers cannot reproduce your exact problem, then they cannot 
correct it.

Original comment by [email protected] on 22 Oct 2014 at 7:27

[GoogleCodeExporter]

There is an issue over at JUnit for this as well:
https://github.com/junit-team/junit/issues/1003#issuecomment-59958853

So far, we haven't been able to reproduce the problem, either.

Original comment by [email protected] on 22 Oct 2014 at 5:42

[GoogleCodeExporter]

Yes, that is the issue I started over there... so everyone is investigating the 
same issue.

So I downloaded the project:
https://github.com/mattm401/DnD-AttributeGen

Ran Randoop on it using the default options. I open one of the tests and change 
the string text in a var1.equals("Warlord") [Assert True] and change it to 
"Warlord1" which will always return false.

When I re-run the JUnit test suite, the suite reports 2 errors instead of 1 and 
I am trying to understand why this is happening.

Original comment by [email protected] on 22 Oct 2014 at 5:47

[GoogleCodeExporter]

Can you give exact commands that can be pasted into a command shell?
This should include exactly how Randoop is run, exactly which test is changed, 
exactly how JUnit is run, and any other necessary details.
This will take you only a moment, but can save significant guessing and wasted 
time for the developers.
(For example, there is no indication in this bug report that you are running 
JUnit from Eclipse.  If you cannot reproduce the problem from the command line, 
then Eclipse or its plugins are more likely to be at fault.)

If you can give other help such as minimizing the test suite, that would be 
extremely helpful too.
For example, it can help to indicate whether the problem is with Randoop or 
JUnit or something else.

Original comment by [email protected] on 23 Oct 2014 at 12:43

[GoogleCodeExporter]

1) Download the project https://github.com/mattm401/DnD-AttributeGen and 
install it in eclipse ( I use ADT Build: v22.0.0-675183) with JUnit 4.11 and 
the Randoop Plugin installed.
2) Run Randoop Test Input on SRC.
3) Results: 27438 Tests Generated; 0 Failures.
4) Open TEST > Open RANDOOP > Open RANDOOPTEST10 > TEST1()
5) Change:

 assertTrue("'" + var2 + "' != '" + "Bard"+ "'", var2.equals("Bard"));

to 

 assertTrue("'" + var2 + "' != '" + "Bard"+ "'", var2.equals("Bard1"));

6) Run RANDOOPTEST10 as JUnit Test Case (Use configuration specific settings: 
Eclipse JUnit Launcher)
7) Result: 1 Failure (As expected)

8) Go back up to root TEST or RANDOOP an [happens with both] d run as JUnit 
Test Case (Use configuration specific settings: Eclipse JUnit Launcher)
9) Result: 2 Failures

... so my question is, why is this happening? Shouldn't I only see 1 Failure? 
Why is the test case getting run twice?

Original comment by [email protected] on 23 Oct 2014 at 3:31

[GoogleCodeExporter]

Regarding step 5) "Bard" may be "Warlock" or something else; line 27 
approximately.

Original comment by [email protected] on 23 Oct 2014 at 3:32

[GoogleCodeExporter]

I asked for a way to reproduce the problem using the command line.  The purpose 
was to help understand whether the problem is in Randoop, in JUnit, or in 
Eclipse.  (The latter seems most likely to me, but of course we don't know for 
sure and I don't have the time to debug Eclipse.)

I'm closing the issue since the person who reported the bug has not provided 
this information.  I'm happy to reopen it if a proper bug report is filed.

Original comment by [email protected] on 26 Oct 2014 at 10:15

• Changed state: Invalid