karma-runner / karma-junit-reporter Goto Github PK

system-out and system-err are appended to the <suite> element. Log output would be significantly easier to read if it was appended to the corresponding <test> element instead.

having: test suite that provides the following mocha output:

  client
    ✔ is cool
    ✔ is awsome
    client utility
      ✔ has absolutely no bugs
  Sender
    ✔ should be a module object
    ✔ should have API post(objRequest)
    using it
      get request
        ✖  should not fail
      POST request
        ✔ should not fail

expected: testcase name to include the context, like:
<testcase name="Sender using it get request should not fail" time="0.816" classname="Android_4_1_2_(Android_4_1_2).Sender">
but got:
<testcase name="should not fail" time="0.816" classname="Android_4_1_2_(Android_4_1_2).Sender">

or, alternatively - if the format allows (im not sure what the JUnit xml spec sais) - then let suite nest context suite elements.
my personal hunch is that the first would be easier, and parsed by all builders that open JUnit xml files the same way they are parsed now :)

You can also look at failed tests of this build:
https://circleci.com/gh/el-net/bs_karma_eval/21
correspond to the code here:
https://github.com/el-net/bs_karma_eval

Instead of having all results in one junit.xml file, there needs to be an option to produce a separate file per testcase.

This would allow tools like jenkins to separate the test logging output on a per testcase basis. Combined with #2, you would get log output that actually maps to each testcase on each browser.

How "testcase" maps to particular structures would be based on how each test framework reports "testcases".

When generating a report as the following:

<?xml version="1.0"?>
<testsuite name="Chrome 47.0.2526 (Linux 0.0.0)" package="" timestamp="2016-02-17T16:40:49" id="0" hostname="" tests="34" errors="0" failures="0" time="0.6">
  <properties>
    <property name="browser.fullName" value="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36"/>
  </properties>
  <testcase name="test" time="0.049" classname="Chrome_47_0_2526_(Linux_0_0_0).Filter: stripExtension"/>
  <system-out>
    <![CDATA[
]]>
  </system-out>
  <system-err/>
</testsuite>

and after parsing it with Sonar, an error occurred:

ERROR: Caused by: Unknown XML node, expected "unitTest" but got "testsuite"

See: 2e5e015

reproduced here
https://github.com/el-net/bs_karma_eval

when run locally as:

karma start karma/lc.js

reproduction is inconsistent :-(

full error:

Missing error handler on `socket`.
TypeError: Cannot read property '30193015' of null
    at specSuccess.specSkipped.specFailure (/Users/osher/ws/POC/bs_karma_eval/node_modules/karma-junit-reporter/index.js:93:22)
    at onSpecComplete (/Users/osher/ws/POC/bs_karma_eval/node_modules/karma/lib/reporters/base.js:78:12)
    at null.<anonymous> (/Users/osher/ws/POC/bs_karma_eval/node_modules/karma/lib/events.js:13:22)
    at emit (events.js:129:20)
    at onResult (/Users/osher/ws/POC/bs_karma_eval/node_modules/karma/lib/browser.js:216:13)
    at Socket.<anonymous> (/Users/osher/ws/POC/bs_karma_eval/node_modules/karma/lib/events.js:13:22)
    at Socket.emit (events.js:129:20)
    at Socket.onevent (/Users/osher/ws/POC/bs_karma_eval/node_modules/karma/node_modules/socket.io/lib/socket.js:330:8)
    at Socket.onpacket (/Users/osher/ws/POC/bs_karma_eval/node_modules/karma/node_modules/socket.io/lib/socket.js:290:12)
    at Client.ondecoded (/Users/osher/ws/POC/bs_karma_eval/node_modules/karma/node_modules/socket.io/lib/client.js:193:14)
    at Decoder.Emitter.emit (/Users/osher/ws/POC/bs_karma_eval/node_modules/karma/node_modules/socket.io/node_modules/socket.io-parser/node_modules/component-emitter/index.js:134:20)
    at Decoder.add (/Users/osher/ws/POC/bs_karma_eval/node_modules/karma/node_modules/socket.io/node_modules/socket.io-parser/index.js:247:12)
    at Client.ondata (/Users/osher/ws/POC/bs_karma_eval/node_modules/karma/node_modules/socket.io/lib/client.js:175:18)
    at Socket.emit (events.js:107:17)
    at Socket.onPacket (/Users/osher/ws/POC/bs_karma_eval/node_modules/karma/node_modules/socket.io/node_modules/engine.io/lib/socket.js:99:14)
    at WebSocket.emit (events.js:129:20)

Error happens if browser failed on start. Simplest case is when test case contained a Syntax Error. Here is the example, run grunt to se the issue.

It is rather Karma problem, because onBrowserComplete is triggered twice and suites is null'ed by the second call. Here is some debugging output:

--> onRunStart
INFO [karma]: Karma v0.12.12 server started at http://localhost:9876/
INFO [launcher]: Starting browser Chrome
INFO [Chrome 34.0.1847 (Mac OS X 10.9.2)]: Connected on socket zY0n_4LMncgG3V6ETzU9 with id 87250253
Chrome 34.0.1847 (Mac OS X 10.9.2) ERROR
  Uncaught SyntaxError: Unexpected end of input
  at /Users/siavorskyi/projects/CD/karma-junit-reporter-failure/test/syntax-error.spec.js:4

Chrome 34.0.1847 (Mac OS X 10.9.2): Executed 0 of 0 SUCCESS (0 secs / 0 secs)
--> onBrowserStart 87250253
--> initliazeXmlForBrowser
Chrome 34.0.1847 (Mac OS X 10.9.2): Executed 0 of 0 ERROR (0.002 secs / 0 secs)
--> onBrowserComplete 87250253 // suites is Object
--> onRunComplete
Chrome 34.0.1847 (Mac OS X 10.9.2): Executed 0 of 0 ERROR (0.004 secs / 0 secs)
--> onBrowserComplete 87250253 // suites is null
--> onRunComplete

v0.3.6 started to fail with in our environment:

Firefox 35.0.0 (Ubuntu 0.0.0): Executed 648 of 648 SUCCESS (3.967 secs / 3.86 secs)
 �[33mWARN [reporter.junit]: �[39mCannot write JUnit xml
     ENOENT, open '/data/slave/npsci/workspace/xxxx/front-end/test-reports/Firefox_35.0.0_(Ubuntu_0.0.0)/test-results.xml'

After upgrading to karma-jasmine 0.2.0, the classname attribute for each testcase is now incorrect.
The current code in the junit reporter does the following to generate a classname attr out of other metadata:

classname: (pkgName ? pkgName + ' ' : '') + browser.name + '.' + result.suite.join(' ').replace(/\./g, '_')

but with the upgrade to 0.2.0 it looks like result.suite is now an array of objects instead of strings, so by joining them you get

[object Object] [object Object], etc.

it seems like the matching classname content can be derived by taking the most recent result suite and grabbing its fullName property

result.suite.slice(-1)[0].fullName

Not sure about implementing that as an across-the-board fix, since it would break backward compatibility. Is it worth adding some check around the contents of result.suite to version check and execute accordingly?

I just upgraded karma-junit-reporter from v0.2.2 to v0.3.3. Test runs fail with error: "You must set an output directory for JUnitReporter via the outputDir config property". It seems that this setting had a default value before, but is now required. Can we please revert to the default value? It was working just fine. Thanks.

Instead of having all results in one junit.xml file, there needs to be an option to produce a separate file per browser.

This would allow tools like jenkins to separate the test logging output on a per browser basis.

Mostly a workflow issue when running under gulp. When running "progress" only, any syntax errors are spat out in console and watch gulp task continues to run. However when "junit" is added to the reporters the same situation will fail on att of undefined and watch will be exited.

Would checking for suite before attempting to use it be an acceptable fix? Go ahead and PR?

Same lack of validation as #16 and karma-runner/karma-coverage#70, the existing fix #17 seems to have been lost at some stage.

This doesn't make a lot of sense as a generic attribute, especially with the way its prefixed. A reasonable alternative would be to make the browser/etc be the default testsuite name instead. Since it already seems to exist there it seems reasonable to simply remove it from the other element.

Reading the Karma Runner v0.13 doc to integrate with Jenkins CI, there indicates to use outputFile

But in the last version (0.3.4 index.js) I don't found this support in the code, until 0.3.3 in the index.js was loaded in line 10 and used inside writeXmlForBrowser function

for now to generate report to Jenkins CI I'm using v0.2.2 and Jenkins CI config from Karma Runner v0.12

...
    // test results reporter to use
    // available reporters: https://npmjs.org/browse/keyword/karma-reporter
    reporters: ['junit', 'coverage'],

    // https://karma-runner.github.io/0.12/plus/jenkins.html
    // https://github.com/karma-runner/karma-junit-reporter
    junitReporter: {
      outputFile: 'tests_out/junit/test-results.xml'
    },

    // https://github.com/karma-runner/karma-coverage
    // https://github.com/yahoo/istanbul

    // optionally, configure the reporter
    coverageReporter: {
      type : 'lcovonly', // produces an lcov.info file
      dir : 'tests_out/coverage/',
      subdir: '.'
    },
...

Hi,

We are creating Sonar reports based on the data generated by karma-junit-reporter. We recently upgraded to Sonar 4.5.1 and It is not able to generate report based on the TEST-results.xml file generated. It is expecting the classnames to be relative paths of the corresponding spec files. It will be of great help if you can implement this feature or provide guidance on how to achieve it.

I started using the junit-reporter plugin in my angular js project and I notice that when I run the tests it starts creating nested Browser directories for each test result.

IE:

First run of tests = "target/javascript/unit/PhantomJS/unit-test-results.xml"

Second run = "target/javascript/unit/PhantomJS/PhantomJS/unit-test-results.xml"

Third run = "target/javascript/unit/PhantomJS/PhantomJS/PhantomJS/unit-test-results.xml"

If I run my tests against multiple browsers, the nesting happens with each browser (IE - replace "PhantomJS" with "Chrome").

My configuration for the junitReporter looks like this:

Any help would be appreciated

I get the following output for a jasmine test that is pending.

  <testcase name="Pending specs can be declared by calling 'pending' in the spec body" time="0" classname="PhantomJS_2_1_1_(Windows_8_0_0_0).Pending specs">
    <skipped/>
    <failure type="">Expected true to be false.
C:/Temp/KarmaTest2/test/spec/controllers/about.js:206:26
</failure>

I would expect the test to only be reported as a skipped test and not run.

I have karma/qunit tests that involve Quicktime and other media plugins, so they crash alot. When this happens the browser disconnects and say 12 out of 92 tests run. I automate this with Jenkins and use karma-junit-reporter to report JUnit results, but outside of scraping the Console output afterwards it appears as if all tests passed and everything is fine. I've filed this issue with Karma, but thought it might be the responsibility of karma-junit-reporter instead.

If I run a test in e.g. IE 11 via sauce labs and inadvertently include a backtick in my source code, the karma console will report something like

E 11.0.0 (Windows 7 0.0.0) ERROR
  Invalid character
  at /var/folders/m1/x8tv0dkd40d636mmqw7lq5_c0000gn/T/packages/test-helper-file/src/file.shim.js:93:0 <- /var/folders/m1/x8tv0dkd40d636mmqw7lq5_c0000gn/T/911df24c8515a218c68c8d72f41a3643.browserify:59086

but no report file will be produced. As a result, any CI tools that rely on structured output (read: jenkins) will consider the build a success.

Following the example on the readme, when I edit this line: outputFile: 'reports/results.xml', it won't save the report in the reports folder, nor rename the file to "results.xml"

The output from this reporter works with Jenkins, but there are probably ways to make it better.

Research how jenkins processes JUnit.xml and create an example of how this can be improved.

On my machine (Windows 10, node 4.2.3) the Karma Junit reporter produced an invalid path for the Junit Xml file, and always failed for me.

A commit that fixed the issue for me and describes the error message in more detail can be found here:
https://github.com/vermeeca/karma-junit-reporter/tree/absolute_outputFile

Opening this issue for feedback before submitting a pull request.

0.3.8 pass correctly.
problem:
14:37:48 [ERROR] Missing error handler on socket.
14:37:48 [ERROR] TypeError: Cannot read property 'charAt' of undefined
14:37:48 [ERROR] at posix (/opt/home/jenkins/workspace/APP/node_modules/karma-junit-reporter/node_modules/path-is-absolute/index.js:4:13)
14:37:48 [ERROR] at writeXmlForBrowser (/opt/home/jenkins/workspace/APP/node_modules/karma-junit-reporter/index.js:61:9)
14:37:48 [ERROR] at onBrowserComplete (/opt/home/jenkins/workspace/APP/node_modules/karma-junit-reporter/index.js:126:5)
14:37:48 [ERROR] at null. (/opt/home/jenkins/workspace/APP/node_modules/karma/lib/events.js:13:22)
14:37:48 [ERROR] at emitTwo (events.js:92:20)
14:37:48 [ERROR] at emit (events.js:172:7)
14:37:48 [ERROR] at onComplete (/opt/home/jenkins/workspace/APP/node_modules/karma/lib/browser.js:142:13)
14:37:48 [ERROR] at Socket. (/opt/home/jenkins/workspace/APP/node_modules/karma/lib/events.js:13:22)
14:37:48 [ERROR] at emitTwo (events.js:92:20)
14:37:48 [ERROR] at Socket.emit (events.js:172:7)
14:37:48 [ERROR] at Socket.onevent (/opt/home/jenkins/workspace/APP/node_modules/karma/node_modules/socket.io/lib/socket.js:335:8)
14:37:48 [ERROR] at Socket.onpacket (/opt/home/jenkins/workspace/APP/node_modules/karma/node_modules/socket.io/lib/socket.js:295:12)
14:37:48 [ERROR] at Client.ondecoded (/opt/home/jenkins/workspace/APP/node_modules/karma/node_modules/socket.io/lib/client.js:193:14)
14:37:48 [ERROR] at Decoder.Emitter.emit (/opt/home/jenkins/workspace/APP/node_modules/karma/node_modules/socket.io/node_modules/socket.io-parser/node_modules

configuration like:
junitReporter: {
outputDir: '../target/surefire-reports', // results will be saved as $outputDir/$browserName.xml
outputFile: undefined, // if included, results will be saved as $outputDir/$browserName/$outputFile
suite: '', // suite will become the package name attribute in xml testsuite element
useBrowserName: true, // add browser name to report and classes names
nameFormatter: undefined, // function (browser, result) to customize the name attribute in xml testcase element
classNameFormatter: undefined // function (browser, result) to customize the classname attribute in xml testcase element
},

writeXmlForBrowser should validate/handle outputFile before calling pathIsAbsolute.

testsuite element can have a skipped attribute that provides the number of skipped tests. the reporter should fill it with the skipped tests count.

Would be nice to have a more control on browser folder.
Browser folder could be masked as %browser% in outputDir like outputDir: "myfolder/%browser%/report"
and skip the browser folder generation when outputFile was specified (keep it only for file name).
Moreover is good to have %browserBasic% which drop browser version like Chrome_46.0.2490_(Windows_8_0.0.0) -> chrome
Or at least accepting a method in order to override it's behavior.

wdyt?
p.s. Can to do that.

When using:

reporters: ['dots', 'junit']

The reporter fails with: .Fatal error: Cannot call method 'ele' of undefined

It doesn't seems like the onRunComplete method is invoked and sets suites = xml = null;

Using: Mac OS, Karma 0.10.2, karma-junit-reporter 0.1.0

JUnit files are not generated as "reporters: ['progress', 'junit']" is missing from the karma config.

I'm trying to get karma to publish using the junit reporter on my windows machine,,, doesnt seem to be working. I can't find the result xml file anywhere and I also don't see any output with log-level debug.

Example config:

    // test results reporter to use
    // possible values: 'dots', 'progress', 'junit', 'growl', 'coverage'
    reporters: ['junit'],

    junitReporter: {
        outputFile: 'core-results.xml'
    },

End of console output:

DEBUG [web-server]: serving: c:/Users/micah/Dropbox/printmee/widget-site/src/Wid
get.Azure/Widget.Web/assets/js/core/test/helpers/pio.testing.fakes.models.app.js

DEBUG [web-server]: serving: c:/Users/micah/Dropbox/printmee/widget-site/src/Wid
get.Azure/Widget.Web/assets/js/core/test/helpers/pio.testing.fakes.models.order.
js
DEBUG [web-server]: serving: c:/Users/micah/Dropbox/printmee/widget-site/src/Wid
get.Azure/Widget.Web/assets/js/core/test/pio.models.customizationstate.spec.js
DEBUG [web-server]: serving: c:/Users/micah/Dropbox/printmee/widget-site/src/Wid
get.Azure/Widget.Web/assets/js/core/test/pio.models.sections.checkoutsection.spe
c.js
DEBUG [web-server]: serving: c:/Users/micah/Dropbox/printmee/widget-site/src/Wid
get.Azure/Widget.Web/assets/js/core/test/pio.models.state.spec.js
DEBUG [web-server]: serving: c:/Users/micah/Dropbox/printmee/widget-site/src/Wid
get.Azure/Widget.Web/assets/js/core/test/pio.services.orderserializer.spec.js
DEBUG [launcher]: Disconnecting all browsers
DEBUG [launcher]: Killing Chrome
DEBUG [launcher]: Process Chrome exitted with code 0
DEBUG [launcher]: Cleaning temp dir C:\Users\micah\AppData\Local\Temp\karma-3379
8751

If i have the output file defined it caused ENOENT errors because it can't create the folder for the file name.

In this case I have the following:

        junitReporter: {
            outputDir: 'test/client/results/junit',
            outputFile: 'test-results.xml',
            suite: ''
        },

I receive the following error

21 07 2015 10:33:44.303:WARN [reporter.junit]: Cannot write JUnit xml
    ENOENT, open '/Users/XXXX/Code/XXX/XXX/test/client/results/junit/PhantomJS_1.9.8_(Mac_OS_X_0.0.0)/test-results.xml'

If I make the folder to match the name above, it works. However, these change quite regularly, so this should be handled by the reporter. I think it just needs to make the directory first before attempting file creation.

Looking at issues such as #62 and #69, it certainly seems like different users of this plugin have different needs concerning how the testcase attributes 'name' and 'classname' are generated. In our case, a recent change introduced in e4f7ebd is pretty much the exact opposite of what we need.

It should be easy to allow users to add functions to their configs to handle this – in other words, for the 'classname' attribute we'd allow overriding the 'getClassName' function, and add something similar for 'name'.

Would you accept a PR for this if I implement it?

After upgrading to 0.3.5 or 0.3.6, when running karma from gulp with singleRun = false:

  new karma.Server({
    configFile: __dirname + '/webpack/karma.conf.js',
    singleRun: false,
  }, done).start()

And the following reporter configuration:

    junitReporter: {
      outputDir: 'build/tests',
    },

The first execution succeeds, but upon detecting file changes, the subsequent test runs fail:

29 09 2015 17:06:02.040:WARN [reporter.junit]: Cannot write JUnit xml
    ENOENT: no such file or directory, open '/home/xxxx/dev/xxxxx/build/tests/PhantomJS_1.9.8_(Linux_0.0.0)/home/xxxx/dev/xxxx/build/tests/TESTS-PhantomJS_1.9.8_(Linux_0.0.0).xml'

(edit: I was trying a few things to fix it and pasted the wrong error path originally)

It appears to be trying to append the full original output path to the new one.

Downgrading to version 0.3.4 fixes the problem.

The latest version of JavaScript Plugin (2.10) from SonarQube removes the ability to import unit test results using JUnit format, they recommend to use another plugin (Generic Coverage) that uses a different XSD Schema.

Anyone uses this feature? Do you have any solution for that?

It's related to #69

Please :)?

I'm tyring to rename my unit tests to get an JUnit output that is readable from Sonar. But I'm following the naming conventions of John Papas Angular Styleguide that includes dots in the filename. For examle:
avengers.controller.js
If this file is in dir/subdir/avengers.controller.js my classname shoult be:

describe('dir.subdir.avengers.controller.js', function(){...

according to the README here.
But this would try to find the file in
dir/subdir/avengers/controller.js

How should I name my test classname to find the file?

When you run multiple levels of tests (unit, midway / integration, e2e), each level of test is executed and overwrites the reports from the previous test runs.

It would be nice if the subsequent test runs would append rather than overwrite.

Workaround is to add a different test report name in the junitreporter config within each karma configuration block (once of unit, one for integration, one for e2e)

and yes, I know that karma e2e is on it's way out because it runs the tests in a frame, whereas protractor runs using selenium webdriver, hence protractor is being promoted as the e2e solution. But, if you are still running ngScenario tests, you might want to configure test file reporters separately.

Sorry but it's one of these "non reproductible", "not everytime" bug.

After some digging, it appears that in my karma run, this plugin stop with no logs exactly at the point of writing the XML file https://github.com/karma-runner/karma-junit-reporter/blob/master/index.js#L88. I put logs and try/catch approximatively everywhere in the plugin. Nothing seems wrong, the normal way of exiting works (onExit done callback stored to be called after writing the XML) but never finish as I never enter the write callback.

It only fails about 1 time on 2 and only on the integration server with Debian. It never failed on my laptop.

As a workaround, I tried a patch switching the write file for a synchronous one and it do the job.
https://github.com/Swiip/karma-junit-reporter/tree/write-sync

Any idea is welcome.

I tried upgrading to karma 0.12.1 in an angular project (not public).

When running karma test:unit I get two different errors. I sometimes get "Fatal error: Cannot call method 'push' of undefined" (probably line 23) and I sometimes get "Fatal error: Cannot call method 'ele' of undefined" (probably because it couldn't find the suite based on suites[browser.id] somewhere). There's a TODO on line 38 that talks about eventually forgetting about the 0.10.x series of karma. I'm not sure if we're there yet. Do we need a branch for 0.12.x (might be required to make 0.11.x work as well)? Any way to make the junit reporter work with both karma 0.10.x and newer stuff, too?

Similar issues:

• karma-runner/karma-teamcity-reporter#15
• litixsoft/karma-mocha-reporter#1
• dtabuenc/karma-html-reporter#3

Thanks a bunch.

Would it be feasible to also remove parenthesis from browser name to make easier to work with? The function writeXmlForBrowser currently replaces spaces with underscores and the default parenthesis in the name make it more difficult to work with on command line since they need to be escaped when copying, etc.

Example:
surefire-reports/TESTS-Chrome_51.0.2704_(Windows_10_0.0.0).xml

In addition would it possible to make an additional parameter to override the default prefix 'TESTS'. The default conventions for java surefire plugins is to use a prefix of 'TEST' (singular) and we would like to get the naming more consistent so just a bit easier to match on patterns when using the JUNIT files for other workflows.

Example
/surefire-reports/junitreports/TEST-com.mycompany.test.server.service.echo.EchoServiceTest.xml

steps to reproduce:

• configure karmaconf.js with browserNoActivityTimeout : 1
• activate reporters: ['junit']
• run tests

bug: karma crashes with this error

ERROR [karma]: [TypeError: Cannot call method 'att' of undefined]

There seems to be an issue with the report directory structure when specifying an output directory and outputFile.

Using the karma.conf:

junitReporter: {
        outputDir: 'reports/karma',
        outputFile: 'report.xml',
        suite: ''
    },

I would expect:
/reports/karma/$browserName/report.xml
/reports/karma/$browserName/report.xml
/reports/karma/$browserName/report.xml
etc etc for the various browsers, however my output appears like so:

Without specifying the outputFile the output is as expected

junitReporter: {
        outputDir: 'reports/karma',
        outputFile: undefined,
        suite: ''
    },

Recent commit d5dc808 causes karma-junit-reporter to no longer function if an outputFile is not defined in the config, even though outputFile is documented as optional and was previously optional. The error is TypeError: Cannot read property 'charAt' of undefined, and it's caused by calling pathIsAbsolute(outputFile) in writeXmlForBrowser without first checking to make sure outputFile was set.

We have a fake unit test and it's indeed running but no report generated. Is it supposed to be?

define(function (require) {
    'use strict';

    suite('Stub Test', function () {
        setup(function () {
        });

        teardown(function () {
        });

        test('stub Test', function () {
            assert.isTrue(true);
        });
    });
});

Hi,

Is there any chance you could add the time attribute onto the testcase element that is written out? This would allow the recording of time taken per test. Details of this attribute can be found here. It is this part of the spec that I mean:

<xs:attribute name="time" type="xs:decimal" use="required">
    <xs:annotation>
        <xs:documentation xml:lang="en">Time taken (in seconds) to execute the test</xs:documentation>
    </xs:annotation>
</xs:attribute>

If I include prototype.js as one of the files in the karma list, the reporter fails, with a stack trace like this:

ERROR [karma]: [TypeError: Object ["Tests for functions in rowHandler.jsfrag"] has no method 'join']
TypeError: Object ["Tests for functions in rowHandler.jsfrag"] has no method 'join'
at JUnitReporter.specSuccess.specSkipped.specFailure (D:\installs\node_modules\karma-junit-reporter\index.js:91:85)
at BaseReporter.onSpecComplete (D:\installs\node_modules\karma\lib\reporters\base.js:83:12)
at bindMethod (D:\installs\node_modules\karma\lib\events.js:15:22)
at EventEmitter.emit (events.js:126:20)
at Browser.onResult (D:\installs\node_modules\karma\lib\browser.js:213:13)
at Socket.bindMethod (D:\installs\node_modules\karma\lib\events.js:15:22)
at Socket.EventEmitter.emit as $emit
at SocketNamespace.handlePacket (D:\installs\node_modules\karma\node_modules\socket.io\lib\namespace.js:335:22)
at Manager.onClientMessage (D:\installs\node_modules\karma\node_modules\socket.io\lib\manager.js:518:38)
at WebSocket.Transport.onMessage (D:\installs\node_modules\karma\node_modules\socket.io\lib\transport.js:369:20)

Occasionally, I see the error Fatal error: Cannot read property 'ele' of null but it appears the xml file has been written correctly. I can't reproduce on demand and after eyeballing index.js nothing jumps out as a likely culprit. I'd be happy to supply more information if you can point me in the right direction.

In the generated XML, the timestamp looks like this:
<testsuite ... timestamp="2016-09-07T10:19:10"...>

and in this example, the timestamp is wrong by two hours (I executed the test at 12:19). The reason seems to be the code in index.js
var timestamp = (new Date()).toISOString().substr(0, 19)

which seems to ignore the timezome. This article describes that the time zone is ignored by default in JavaScript: http://stackoverflow.com/questions/10830357/javascript-toisostring-ignores-timezone-offset

It is not really a terrible bug, but I like to have a timestamp which is correct, because this timestamp is then propagated to the HTML report and when looking at the report, I am interested whether it is an old report or a recent one.

This might work very well on my machine (where I know that the difference is always two hours), but as soon as there is some continuous integration environment and I don't know where in the world this environment is running, I am no longer sure whether the generated report is recent or not.

Possible fixes are described in the stackoverflow article.
Is this something that you can fix?

Thanks & Cheers
Michael

Hello!

I can't get one reporter.xml for each browser that I run my tests (and I really need this).
Can someone please help me with the configuration file in order to achieve that?

Thx

Hey ho,

I'm using the ng-scenario plugin with karma and this junit reporter plugin.
Also a xUnit Jenkins Plugin to collect test at the CI stage.

This is a example output of a testCase:

<testcase name="Elements of StartScreen (Shell view)" time="0.881" classname="Chrome 30.0.1599 (Mac OS X 10.9.0).(XXX-2271) Home Screen and Tab Navigation">
      <failure type="">element ....</failure>
      <failure type="">http://localhost:38080/base/test/e2e/shell/home.nav.e2e.js?1382963509000:19:16: Selector .main-menu-logo did not match any elements.
     </failure>
</testcase>

If you may noticed, there are two failures.
But Jenkins Error log always only shows the first failure.

So I took a look at the jUnit schema spec to check what is going wrong.
https://github.com/karma-runner/karma-junit-reporter/blob/master/junit-schema.xsd

My Result: Only one element is valid, correct?

So the failing Code is in index.js:84

if (!result.success) {
      result.log.forEach(function(err) {
        spec.ele('failure', {type: ''}, formatError(err));
      });
    }

But also ng-scenario does emit's 2 errors.
So maybe there should be also a fix?
Just print one? Ignore all 1+n?

Rob :)

The reporting configuration of karma has become more complicated over time. But, the changes have not helped me at all. Actually, the changes have only caused me headaches. The junit reporter used to write to a file named 'test-results.xml'. That's pretty arbitrary, but at least my test runner could find it. Now the default name is based on the OS name and version and browser name and version -- most of which is hard to code into my test runner. Now it's possible to specify parameters to a reporter, but even though I set outputDir and outputFile, I don't get what I ask for. The file is still in a folder with a complicated directory name. Karma is trying to be too helpful. It should do what I ask and no more. Extra functionality to support this or that system, make it hard to use outside of those systems. The naming today seems about as arbitrary as it did when the name was test-results.xml. Maybe some people find it useful that the name includes browser and os info, but some don't (like me) and maybe some want other info in there. I should be able to select the output file name ... exactly ... without further modification. And, if you really want to make the file name flexible, then implement a tag system that allows the user to put name parts together as they want. For example: [browser-name]_[browser-version] [os-name][os-version]. Various other tags could be supported too.

Also, separating the reports by browser is the opposite of helpful to me. It's easier for me to deal with the combined results.