Release 1.30 all test results are duplicated about junit-plugin

[oosterholt]

Here are the test report files in the 'upstream job' (see parallel-test-executor).

reports.zip

[timja]

Do you have an example project this can be with reproduced with? / can you provide one?

and what type of project are you using? maven, freestyle, pipeline?

[oosterholt]

Do you have an example project this can be with reproduced with? / can you provide one?

I'm trying to create a demo situation for reproduction. I'll let you know. It has definitely something to do with the Parallel Test Executor plugin.

what type of project are you using? maven, freestyle, pipeline?

I'm using a freestyle project.

[timja]

If you could bisect what change introduced this it would help otherwise I’ll try to get to it in the next couple of weeks

[oosterholt]

@timja What is the best way of doing this. When I clone this repo I do not even see the 'src' directory for example.

[timja]

if you run mvn package -P quick-build you will get an hpi that you can load into your Jenkins.

or you can run: mvn hpi:run -P quick-build which will start up a Jenkins server based on your commit

[oosterholt]

Cool thanks. That's fast. Since I do not see the sources, how can I bisect?

[timja]

is this in your IDE you don't see them? when you checkout the code it's in the src folder

[oosterholt]

Revision 3a90162 is causing this issue.

[timja]

Thanks and do you have an example project that you can reproduce this with? and a pipeline?

(it'll make it much easier for me or others to look at a fix if we have that)

[rpjesivac]

@timja @oosterholt
Any progress on this? We have upgraded to the 1.43 version and now we are seeing this issue as well.

[oosterholt]

@rpjesivac do you have the same job setup; are you also using the Parallel Test Executor plugin?

[zbynek]

@rpjesivac in your folder all the XMLs are duplicated: see
reports\2\report\test\TEST-our.package.activiti.connector.ActivitiPTBConnectorTest.xml
vs
reports\2\report\test\TEST-out.package.activiti.connector.ActivitiPTBConnectorTest.xml

Any idea what causes that?

Also, the following pipeline downloads your test results and runs them through the plugin, but I couldn't reproduce the bug:

pipeline {
    agent any
    stages {
        stage('Example') {
            steps {
            sh '''
            wget https://github.com/jenkinsci/junit-plugin/files/5318011/reports.zip
            rm -rf reports
            unzip reports.zip
            find reports | sed 's/.*/sleep .1 \\&\\& touch &/' | sh
            '''
            junit '**/test/*.xml'
            }
        }
    }
}

[oosterholt]

The 'Test-out.*' files are faulty. Files got duplicated while 'obfuscating' test reports. I've updated the reports.zip with the duplicated files removed. These are not the reason of the duplicated test results.

[oosterholt]

Still no clue what is going on. I've added a archive artefacts post build step with the same pattern as the publish test result. The artefacts do not show any duplicated test report files. A text search on any of the duplicate reported test only results in one match.

There are no changes in the archived test report file artefacts in a test run with the 1.29 version of the plugin and a test run with any of the newer plugin versions.

[oosterholt]

Additional information to clarify the situation.
All tests are reported twice; see test result chart:

Also see any of the test suites; all test methods are reported twice:

All test are reported once with their normal name:
...lastCompletedBuild/testReport/our.package.activiti/DocumentApprovalQuickTest/cancelWorkflow
and once with a nummer appended:
...lastCompletedBuild/testReport/our.package.activiti/DocumentApprovalQuickTest/cancelWorkflow_2