michaelleeallen / mocha-junit-reporter Goto Github PK

I would like to contribute a PR.
But first we may need to talk about how you would like to have it.

TL;DR Make classname and name switchable via config

Problem

Regarding <testcase name="" classname="" entries:

• Since I use junit.xml results generated by java-surefire together with the ones generated by this reporter I want classname to be switched with name.

Example from Java:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<testsuites failures="0" name="Custom" tests="14" time="0.8200000000000001">
  <testsuite errors="0" failures="0"
             name="ut.io.codeclou.customfield.editor.MyComponentUnitTest"
             skipped="0" tests="1" time="0">
    <testcase
      time="0.359"
      classname="ut.io.codeclou.customfield.editor.rest.CustomFieldHelperTest" 
      name="testReorderOptionsMoveTopMiddleToBottom"/>
    <testcase
      time="0.018"
      classname="ut.io.codeclou.customfield.editor.rest.CustomFieldHelperTest" 
      name="testDoBothListsHaveCommonValue"/>
  </testsuite>
</testsuites>

As you can see classname is really used for the className and name for the testMethodName.

The mocha-junit-reporter generates it the other way. When I say

describe('Global Permissions Test', () => {
  it('should be possible for admin to lookup bob by using his lastname in userSearch', (done) => {
   ...
  })
})

It generates

...
  <testsuite name="Global Permissions Test" timestamp="2017-04-08T07:04:13" tests="3"    
             file="/work-private/tests/rest-api/1/test-local.js" failures="0" time="0.244">
    <testcase
      time="0.146"
      classname="should be possible for admin to lookup bob by using his lastname in userSearch"
      name="suiteName a.s.o">
    </testcase>
  </testsuite>
...

And I want it to look like:

...
  <testsuite name="Global Permissions Test" timestamp="2017-04-08T07:04:13" tests="3"    
             file="/work-private/tests/rest-api/1/test-local.js" failures="0" time="0.244">
    <testcase
      time="0.146"
      classname="suiteName a.s.o"
      name="should be possible for admin to lookup bob by using his lastname in userSearch">
    </testcase>
  </testsuite>
...

Proposal

Add reporterOptions property named testCaseSwitchClassnameWithName that defaults to false.

So when you call:

TEST_CASE_SWITCH_CLASSNAME_WITH_NAME=true mocha test --reporter mocha-junit-reporter

Then it would generate it with switched name/classname for all testcase-entries.

What do you say?

The generated JUnit report files do not include any test output.
It would be awesome that they do.

/home/aditya/Desktop/pycom/node_modules/mocha-junit-reporter/index.js:106
return testsuites[testsuites.length - 1].testsuite;
^

TypeError: Cannot read property 'testsuite' of undefined
at lastSuite (/home/aditya/Desktop/pycom/node_modules/mocha-junit-reporter/index.js:106:46)
at MochaJUnitReporter. (/home/aditya/Desktop/pycom/node_modules/mocha-junit-reporter/index.js:131:5)
at emitTwo (events.js:131:20)
at Runner.emit (events.js:214:7)
at Runner.fail (/home/aditya/Desktop/pycom/pybytes-react/node_modules/mocha/lib/runner.js:239:8)
at Runner.uncaught (/home/aditya/Desktop/pycom/pybytes-react/node_modules/mocha/lib/runner.js:716:12)
at process.uncaught (/home/aditya/Desktop/pycom/pybytes-react/node_modules/mocha/lib/runner.js:821:10)
at emitOne (events.js:116:13)
at process.emit (events.js:211:7)
at process._fatalException (bootstrap_node.js:378:26)

Hi,

can you add mocha ver.5 into package.json?

npm ERR! peerinvalid The package [email protected] does not satisfy its siblings' peerDependencies requirements!
npm ERR! peerinvalid Peer [email protected] wants mocha@^4 || ^3 || ^2.2.5

Thank you

I'm trying to set an attachments with absolute path to a screenshot in case test failed in AfterEach hook.

this.test.attachments = [absolutePath] works great from test. But when I moved that line to AfterEach hook it stop working.
I've tried to use those and attachment wasn't added:
this.test.attachments = [absolutePath]; this.currentTest.attachments = [absolutePath]; this.currentTest.ctx.attachments = [absolutePath]; this.test.ctx.attachments = [absolutePath];

Is there any way to attach a screenshot path from AfterEach hook?

Regarding the enhancements provided in #51, is there any way to make the classname unique the testsuite? Here is an example mocha test:

describe('Get all endpoints', () => {
    it('should render the map page', (done) => {
...
    })
    it('should render faq page', (done) => {
...
})

When setting testCaseSwitchClassnameWithName = true This is the result:

<testsuite name="Get all endpoints" timestamp="2018-11-20T19:59:09" tests="12"  failures="4" time="0.4750000000000001">
    <testcase name="should render the map page" time="0.07" classname="Get all endpoints should render the map page">
    </testcase>
    <testcase name="should render faq page" time="0.02" classname="Get all endpoints should render faq page">
    </testcase>
...

It seems that the classname returns a concatenation of the testsuite name and the testcase name. Is there anyway to set the classname to the testsuite name in order to group the rest of the tests so that they all follow under one classname?

#95

package.json test script
"mocha test/**/*test.js --reporter mocha-junit-reporter

Error shown

12:00:21.803 > Error while importing {}
12:00:21.858 > Error while importing {}

However I could able to run the tests by switching to another reporter mocha-jenkins-reporter

I run the following command on linux (in a docker container):
mocha --timeout 6000 --reporter mocha-circleci-reporter --compilers ts:ts-node/register integration-tests/**/*.test.ts

And the process just stops at: komed-test-integration | 24 passing (1s)

on windows it works perfectly...

I'm not even sure if this is an issue anymore. Can anyone verify if this issue still exists?

It appears that a skipped test (using it.skip in mocha) does not appear in the xml file. I see the number of skipped tests but not the test case itself.

For example, this test:

const expect = require('expect');

describe('test', () => {
  it.skip('works', () => {
    expect(1).toBe(1);
  });
});

generates this result file:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Mocha Tests" time="0" tests="1" failures="0" skipped="1">
  <testsuite name="Root Suite" timestamp="2016-08-05T20:37:12" tests="0" failures="0" time="0">
  </testsuite>
  <testsuite name="test" timestamp="2016-08-05T20:37:13" tests="1" failures="0" time="0">
  </testsuite>
</testsuites>

but it's missing

    <testcase name="Root Suite test works" time="0" classname="works">
      <skipped/>
    </testcase>

I guess the real question is how do you turn on the includePending option

Struggling to work out how to debug this.
I have a set of nested folders with tests in them, if I run each subfolder individually, each one will generate a test-result.xml file without any issues.

If I run the whole folder, I get no test-results.xml file.
For example if I run this in package.json:
mocha --reporter mocha-junit-reporter --reporter-options useFullSuiteTitle=true 'path/to/test/**/*_SlowTest.js'; exit 0
It will work, however this will not:
mocha --reporter mocha-junit-reporter --reporter-options useFullSuiteTitle=true 'path/to/**/*_SlowTest.js'; exit 0

Running the same command without the --reporter shows that all tests are running (some are failing but that shouldnt matter)
My guess that there is some error preventing the .xml getting generated, but I have no idea where to start with finding out where, since each individual set of tests works without issue.

Hi,
i am using dsl to create jenkins jobs,
Can you please add dsl publisher groovy file for mocha-juint-reporter

sample dsl Publishers : https://github.com/jenkinsci/job-dsl-plugin/tree/master/job-dsl-core/src/main/docs/examples/javaposse/jobdsl/dsl/helpers/publisher/PublisherContext

Hi,

I have issue with cypress when the project is stored in a different folder then root, for example:

cypress run --project=apps/site-e2e

Deps installed:

"mocha": "6.1.4",
"mocha-junit-reporter": "1.23.0",

I presume the problem is because the packages json path start with ./

In fact just add fs.readFileSync(__dirname + "/../node_modules/mocha/package.json", "utf8") seems to work, but of course there are many way to aim that.

npm WARN [email protected] requires a peer of mocha@^3 || ^2.2.5 but none is installed. You must install peer dependencies yourself
https://github.com/mochajs/mocha/releases

Thanks for writing this reporter, it works great.

I would like to be able to see test results live as they are happening. I know about toConsole, but this just outputs XML at the end. If I have a long running test suite it would be nice to see live results while tests are running, like with mochawesome.

Happy to PR if interested.

Tried to set up an rc file like so:

module.exports = {
    reporter: 'mocha-junit-reporter',
    reporterOptions: {
        mochaFile: './path_to_your/file.xml'
    }
}

And didn't see the xml file in the path.

If I remove reporterOptions, I notice that test-result.xml gets spit out in the root of my project.

I decided to poke into the mocha-junit-reporter code and dropped a single console.log in configureDefaults and noticed that reporterOptions is always undefined.

I am able to get it to work if I set it up as a script in package.json and via command line, but I'd rather have it as a config file. I'm not sure if this is a problem with Mocha or this library, so I figured I would start here.

Using versions:

    "mocha": "^6.2.0",
    "mocha-junit-reporter": "^1.23.1",

Thanks in advance.

I have problem with export mocha test results to xml file. I need this to create report on Hiptest in test run for external tests.

I'm using mocha-junit-reporter, but there is a problem with nested describes. I will try to explain this on example:

describe("Tests", function(){
  describe("Describe 1", function(){
    it("Test 1", function(){
      //Some code
    }); 
    describe("Describe 2", function(){   
      it("Test 2", function(){
        //Some code
      });
      it("Test 3", function(){
        //Some code
      });
    });
    describe("Describe 3", function(){   
      it("Test 4", function(){
        //Some code
      });
      it("Test 5", function(){
        //Some code
      });
    });
  });
});

Mocha result:

Tests
   Describe 1
   1) Test 1
      Describe 2
         2) Test 2
         3) Test 3
      Describe 3
         4) Test 4
         5) Test 5

As a result in xml file I recieve:

Describe 1:
   - Test 1
Describe 2:
   - Test 2
   - Test 3
Describe 3:
   - Test 4
   - Test 5

Each describe is analyzed as a separate test suite. So in result top describe is ignored, beacuse there is no tests in it. Describe 2 and Describe 3 are not nested in Describe 1.

I don't know how to achive result i need.

As far as I can tell, the "testsuites" element (usually the root element) shouldn't have a timestamp attribute.

Sources:

• unofficial XSD for junit test https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsd (found via http://stackoverflow.com/questions/442556/spec-for-junit-xml-output)
• ran a unit test on the xunit-jenkins-plugin with the junit type. used the output from mocha-junit-reporter. similar to test in https://github.com/jenkinsci/xunit-plugin/blob/master/src/test/java/org/jenkinsci/plugins/xunit/types/JUnitTypeTest.java

Similar to #16

If exception is thrown in describe-s then reporter mixes their test suites. A live demo (xml is written to console): http://plnkr.co/edit/qGWdaXfQZF207lzWUgTM?p=preview

Hi there,

Can you please update your npm repository? The bug fixed in 41615f2 caused me quite a bit of trouble until I realised that npm was out of sync with GitHub.

Thanks!

Explain how to debug this reporter

Starting 1.23.2 after tests have passed/failed in Jenkins the following error is printed:

 139 passing (16m)


=============================== Coverage summary ===============================
Statements   : 89.09% ( 1053/1182 )
Branches     : 79.29% ( 356/449 )
Functions    : 89.82% ( 150/167 )
Lines        : 89.05% ( 1041/1169 )
================================================================================
[Pipeline] junit
Recording test results
No test report files were found. Configuration error?

If going back to 1.23.1, everything works as expected, so looks like one of these changes created a regression: v1.23.1...master

@clayreimann You seem to be the one who pushed code most recently.
@michaelleeallen FYI.

Let's say the javascript code I'm testing throws an uncaught error during execution. This causes the test to fail, as expected, and it's logged as a failure in the reporter's generated file.
However, it seems other reporters (I'm looking at xunit, specifically) include the details of the failure in the failure xml (example below) while mocha-junit-reporter leaves "failure" blanks:

xunit

<testcase classname="TheName _hasNonGlobalServerErrors" name="ignores global errors" time="NaN" message=""><failure classname="TheName _hasNonGlobalServerErrors" name="ignores global errors" time="NaN" message=""><![CDATA[Error
at React.createClass._hasNonGlobalServerErrors (/snipped/fileName.js:128:17)
at Context.&lt;anonymous&gt; (/snipped/fileName.js:82:24)
at Test.Runnable.run (/snipped/node_modules/mocha/lib/runnable.js:213:32)
at Runner.runTest (/snipped/node_modules/mocha/lib/runner.js:344:10)
at /snipped/node_modules/mocha/lib/runner.js:390:12
at next (/snipped/node_modules/mocha/lib/runner.js:270:14)
at /snipped/node_modules/mocha/lib/runner.js:279:7
at next (/snipped/node_modules/mocha/lib/runner.js:227:23)
at Immediate._onImmediate (/snipped/node_modules/mocha/lib/runner.js:247:5)
at processImmediate [as _immediateCallback] (timers.js:358:17)]]></failure></testcase>

mocha-junit-reporter

<testcase name="TheName _hasNonGlobalServerErrors ignores global errors" time="0" className="ignores global errors">
  <failure></failure>
</testcase>

Hi i'm using mocha with istanbul but i always get this error

: Warning: Could not find any test files matching pattern: mocha-junit-reporter

How it's possible?

Here the command line:

MOCHA_FILE="./report.xml" istanbul cover --dir ./coverage --report lcov --report cobertura --config ./.istanbul.yml ./node_modules/.bin/_mocha ./test/js/**/ --colors --reporter mocha-junit-reporter

Thanks.

As far as I can tell, the "classname" attribute for the "testcase" element should be all lowercase, i.e. "classname" instead of "className".

Sources:

• unofficial XSD for junit test https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsd (found via http://stackoverflow.com/questions/442556/spec-for-junit-xml-output)
• source code at http://svn.apache.org/repos/asf/ant/core/trunk/src/main/org/apache/tools/ant/taskdefs/optional/junit/XMLConstants.java (found via same stackoverflow thread above)
• ran a unit test on the xunit-jenkins-plugin with the junit type. used the output from mocha-junit-reporter. similar to test in https://github.com/jenkinsci/xunit-plugin/blob/master/src/test/java/org/jenkinsci/plugins/xunit/types/JUnitTypeTest.java

This is similar to #107. When the mochaFile option is provided via the .mocharc.js configuration file, it is not respected.

This is the case on the latest, 1.23.3 and Mocha 7.1.0.

reporterOptions: {
  mochaFile: '../reports/contract.xml'
},

A generic test-results.xml file is created every time.

I upgraded from version 1.18.0 to 1.23.2, and the mochaFile option didn't work anymore. Instead writing to the designated folder and filename, a generic "test-results.xml" file was created in the root.

I use the mocha file option like this in my package.json file: "--reporter-options mochaFile=./path_to_your/file.xml"

I am using your reporter with my project written in es6, and using following command to generate report in xml:

mocha --compilers js:babel-core/register --reporter mocha-junit-reporter

But I think report is not well formatted, see below:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Mocha Tests" time="0.022" tests="1" failures="0">
  <testsuite name="Root Suite" timestamp="2016-04-22T05:09:46" tests="0" failures="0" time="0">
  </testsuite>
  <testsuite name="DockerCloud class" timestamp="2016-04-22T05:09:46" tests="0" failures="0" time="0">
  </testsuite>
  <testsuite name="extractUuid()" timestamp="2016-04-22T05:09:46" tests="1" failures="0" time="0.022">
    <testcase name="Root Suite DockerCloud class extractUuid() should extract uuid from resource uri" time="0.022" classname="should extract uuid from resource uri">
    </testcase>
  </testsuite>
</testsuites>

Can you advice if I am doing something wrong?

Link to my repo if needed: https://github.com/CityofSurrey/dockercloud

The npm version seemed to be out of sync with github, due to missing the adding attachments for test. Could we consider publishing new version to npm soon?

Cause I want use the attachment feature with codeceptjs, but then realised it's not there yet in npm

Thanks!

It was suggested, and would be a good idea, to actually invoke mocha from inside our tests to ensure that we conform to mocha's API.

Ironically, it may be necessary to use a separate test driver in order to actually accomplish this.

Test result
I have some test cases with describe('mocked tests') and some with describe('unmocked test').

./node_modules/mocha/bin/mocha --invert --grep 'unmocked' -R mocha-junit-reporter
runs unmocked test cases.

./node_modules/mocha/bin/mocha --invert --grep '^unmocked' -R mocha-junit-reporter
runs both unmocked and mocked test cases.

What should be expected?

./node_modules/mocha/bin/mocha --invert --grep '^unmocked' -R mocha-junit-reporter
should run 'unmocked' test cases only.

Im not exactly sure what the root suite is useful for and would be very happy if I could disable it so that its not included in the output xml. I am using this tool to view my mocha tests in the xunit-viewer but have no need for any root suite thats always a success.

What exactly is it meant for?

Whenever attempting to use this reporter with the latest mocha, mocha fails. With previous releases of mocha, it did not.

it("\n1, test Cases...", function (done) {

            .get('https://google.com')
            .end(function (err, res) {
                if (!err) {                    
		      chai.expect(res.statusCode, res.error).to.have.property('data', 'Value', 'This is an error message when fails.');
                  done();
                }
                else
                    throw err;
            });
    });


The XML report:

 failure![CDATA[AssertionError [ERR_ASSERTION]: This is an error message when fails.
    at Test\test-apis\RaaS-V2\Authentication-Apis\Access-Token-Validation-Apis\GetCheckTokenValidity.js:58:24
    at pass_cb (Test\utility-modules\request-controller.js:76:17)
    at Test.<anonymous> (Test\utility-modules\request-controller.js:79:13)
    at Test.assert (node_modules\supertest\lib\test.js:179:6)
    at assert (node_modules\supertest\lib\test.js:131:12)
    at node_modules\supertest\lib\test.js:128:5
    at Test.Request.callback (node_modules\superagent\lib\node\index.js:718:3)
    at parser (node_modules\superagent\lib\node\index.js:906:18)
    at Stream.res.on (node_modules\superagent\lib\node\parsers\json.js:19:7)
    at Unzip.unzip.on (node_modules\superagent\lib\node\unzip.js:55:12)
    at endReadableNT (_stream_readable.js:1055:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickDomainCallback (internal/process/next_tick.js:218:9)]]/failure

The error message "This is an error message when fails." should show in attribute of failure tag.

failure message="This is an error message when fails." stack trace failure

I'm currently using this package to generate junit-style test results to be uploaded to Microsoft Visual Studio Team Services. The vsts team said that they currently validate junit results against this schema. However, the output from this package currently fails to validate against that schema. Is there a different schema that you're currently validating against? Is it possible for output to be altered to pass validation against the one that the vsts team is using?

Hi! Hope I'm reporting this at the right place.

I'm using cypress 3.4.1 with mocha 6.1.4 and mocha-junit-reporter 1.23.1 to generate junit test output, with the following parameters in cypress.json:

"reporterOptions": {
"mochaFile": "test-result/JUnit-[hash].xml",
"toConsole": false,
"outputs": true,
"antMode": true,
"jenkinsMode": true
}

I'm trying to publish them on jenkins with the xUnit plugin, but received the following message in the build log:
WARNING: At line 11 of file:/.../test-result/JUnit-e78c63fec54eadbedb7d94d2b95e557f.xml:cvc-pattern-valid: Value '5.007000000000001' is not facet-valid with respect to pattern '(([0-9]{0,3},)*[0-9]{3}|[0-9]{0,3})*(\.[0-9]{0,3})?' for type 'SUREFIRE_TIME'.

Said 11th line:
<testsuite name="Root Suite.Issues test" timestamp="2019-09-16T16:08:13" tests="3" package="Root Suite.Issues test" hostname="undefined" id="1" errors="0" failures="0" time="5.007000000000001">

As you can see the time attribute has too many fractional digits, failing the xsd of xUnit (which is hopefully Ant compliant). Can you round/truncate it?

It would make sense to be able to set the report output from CLI flags, ie. --report=unit-tests.xml, this would make fe. npm scripts cleaner and more portable.

Another option would be able to supress generated output altogether and do it the way jshint & jscs do it, with stdout redirect ie. --no-report > test-output/mocha-tests.xml.

Currently the latest version in npm is 1.6.1

It would be useful if the currently hard-coded value "Mocha Tests" for the "name" attribute in "testsuite" is configurable.

Jenkins for example groups test reports by this value. When having multiple test result files these could be grouped.

I can't seem to find any documentation on what the difference is between this reporter and the XUnit reporter built-into Mocha?

@junminstorage asked a similar question but appears to have gotten no answer.

How do you recommend implementing a screenshot feature next to the failed test ?

I am trying to set up my CI to use this reporter, and we run our tests in mocha-headless-chrome. Unfortunately it doesn't seem to work because the mocha-junit-reporter uses node-specific APIs such as "fs".

I'm trying to set an attachments with absolute path to a screenshot in case if a test failed in AfterEach hook.

this.test.attachments = [absolutePath] works great from test. But when I moved that line to AfterEach hook it stopped working.
I've tried to use the following and an attachment wasn't added:
this.test.attachments = [absolutePath];
this.currentTest.attachments = [absolutePath];
this.currentTest.ctx.attachments = [absolutePath];
this.test.ctx.attachments = [absolutePath];

Is there any way to attach a screenshot path from AfterEach hook?

Hi,

I have configured some projects using the reporter and it works perfectly when it does not fail any tests. It creates an XML report and everything works, but when it fails a test, it does not create the report so I do not receive the failing test.

I am using version 1.16.0, mocha 4.1.0 and chai 4.1.2.

Right now this reporter only generates a single testsuite element, but should support multiple testsuite elements.

Not sure if this a problem on the npm side or not, but thought you should know.

$ npm install mocha-junit-reporter
npm ERR! fetch failed https://registry.npmjs.org/mocha-junit-reporter/-/mocha-junit-reporter-1.11.0.tgz
npm WARN retry will retry, error on last attempt: Error: fetch failed with status code 404
npm ERR! fetch failed https://registry.npmjs.org/mocha-junit-reporter/-/mocha-junit-reporter-1.11.0.tgz
npm WARN retry will retry, error on last attempt: Error: fetch failed with status code 404

I am trying to run this command --reporter=mocha-junit-reporter mochaFile=./TestResults/TEST-RESULT.xml
and then it through error TypeError: runner.on is not a function

D:\............\node_modules\nyc\node_modules\yargs\yargs.js:1133
      else throw err
           ^

TypeError: runner.on is not a function
    at MochaJUnitReporter.Base (D:\............\node_modules\mocha\lib\reporters\base.js:267:10)
    at new MochaJUnitReporter (D:\............\node_modules\mocha-junit-reporter\index.js:110:8)
    at Object.create (D:\............\node_modules\nyc\node_modules\istanbul-reports\index.js:17:16)
    at reporter.forEach (D:\............\node_modules\nyc\index.js:459:24)
    at Array.forEach (<anonymous>)
    at NYC.report (D:\............\node_modules\nyc\index.js:458:17)
    at Object.exports.handler (D:\............\node_modules\nyc\lib\commands\report.js:44:7)
    at Object.runCommand (D:\............\node_modules\nyc\node_modules\yargs\lib\command.js:235:44)
    at Object.parseArgs [as _parseArgs] (D:\............\node_modules\nyc\node_modules\yargs\yargs.js:1046:30)
    at Object.parse (D:\............\node_modules\nyc\node_modules\yargs\yargs.js:551:25)
npm ERR! Test failed.  See above for more details.

Edit 1
"mocha": "^4.1.0",
"mocha-junit-reporter": "^1.18.0"

I get this error when running npm install mocha-junit-reporter:

You've said in pull #3 that you've set up travis-ci to do auto deployments to npm, maybe something goes bad there?

It seems impossible to install 1.3 - but 1.2 works.