Event Listener and Report output incorrect or nunit2 test runs about nunit-v2-framework-driver

[CharliePoole]

I edited your description so your XML displays.

[Almewty]

I ran into the same problems and decided to look at the code.
I fixed it in my fork here but I am not too sure if it does what it should do because I'm not that familiar with all the States of NUnit.
Feedback would be appreciated!

[CharliePoole]

Looking at the filename in your last example, I gather these are xamarin tests. If that's the case, the NUnit engine (and hence the console runner) would not know how to run them. Can you clarify how you are running the tests?

[owenniblock]

These are just standard tests (I'm working at Xamarin hence the namespace). For example:

        [Test]
        public void SimpleSuccessTest()
        {
            Console.WriteLine("SimpleSuccessTest BEFORE WAIT:: " + DateTime.Now.ToString("yyyy-MM-dd hh:mm:ss"));

            Thread.Sleep(10000);

            Console.WriteLine("SimpleSuccessTest AFTER WAIT:: " + DateTime.Now.ToString("yyyy-MM-dd hh:mm:ss"));

            Assert.IsTrue(true);
        }

I'm running the tests using the console:

nunit3-console "D:\\MS\\test-apps\\Xamarin.NUnit2.Tests\\bin\\Debug\\Xamarin.NUnit2.Tests.dll"  --workers=0 --process=Single

[CharliePoole]

The results are interesting. The nunit2 format output is actually created from the nunit3 output. The nunit3 output is generated by the extension from the v2 object model. That suggests an error in creating the nunit3 output followed by a compensating error in reformatting to nunit2. Like a perfect storm of errors.

[owenniblock]

Interesting - so the correct data must be in there somewhere :) Sounds like it's just not getting mapped correctly.

EDIT: Looking at the code in the v2 report writer - it's mapping based on the "result" attribute on each test case. I'll have a look and see how it's totting up the values in the console.

[CharliePoole]

Rather than checking the full ResultState you should probably just look at the TestStatus component. Its values map directly to the four attributes that need to be set.

[Almewty]

Thanks for the advice!
I didn't really find a way to get access to the TestStatus component so I at least used the same logic to determine the state here. The only way I found was via TestContext.Result.State. But I could not find a way to get a TestContext.
But now that I know that there is a place to get this information from I would rather find a proper way.
Any advice?

[CharliePoole]

AH! Apologies... I forgot you are dealing with a V2 ResultState, which is nothing but an enum. In NUnit 3, that becomes a class with Status, Label and Site properties.

I think you should handle them consistent with how we translate to NUnit 3 result states in the GetTranslatedResultState method. I think you have done that with the sole exception of NotRunnable, which NUnit 3 treats as failed.

It would also be cool if you could avoid recursing through all the children four times and do it just once, but I don't see that as essential to the PR.

[CharliePoole]

Fixed by PR #23