Coder Social home page Coder Social logo

mocha-teamcity-reporter's Introduction

NPM version TeamCity Build Status Monthly Downloads

mocha-teamcity-reporter

mocha-teamcity-reporter Teamcity reporter which makes it possible to display test results in real-time, makes test information available on the Tests tab of the Build Results page.

Version 4.x changes

  • Breaking: Only supported on node.js 6 and above
  • Breaking: Only Mocha version 6 and above is now supported
    • Please remain on mocha-teamcity-reporter@3 if this is an issue
  • New reporter option ignoreHookWithName to skip reporting for hooks with title containing some word (@DJ-Glock)
  • Added postfix _hook for flowId for hooks to ensure flowIds will never intersect.
  • Implement 'hook end' event (@DJ-GLock)
  • General maintenance and tidy up (@DJ-Glock)

Mocha@6 notes

  • recordHookFailures option may not work as intended as mocha6 is now doing this itself

Requirements

  • NodeJs 6+
  • Mocha 6+
  • Web Browser supporting ES5

To Install

In your project run a npm install command:

npm install mocha-teamcity-reporter --save-dev

Basically, have your project's package.json be like:

{
  "devDependencies": {
    "mocha-teamcity-reporter": ">=2.0.0"
  }
}

Usage

https://github.com/visionmedia/mocha/wiki/Third-party-reporters describes using third party reporters in mocha.

Then call mocha with:

mocha --reporter mocha-teamcity-reporter test

Running In Browser

  • Use lib/teamcityBrowser
  • Has option parsing stripped out for the moment
  • Example use can be found in test\browser
  • Custom log function can be set with window.customLogFunction

Customisation

TeamCity flowId

Can set flowId like: mocha test --reporter mocha-teamcity-reporter --reporter-options flowId=gobbledygook

Top-level suite name

Can set a top-level suite name, which will wrap all other suites.
This is useful for reading test output when running multiple suites in a single build

  • Environment variable: MOCHA_TEAMCITY_TOP_LEVEL_SUITE=
  • Reporter option: topLevelSuite=

log test failures with std error

To enable this please Please note this will probaly be made default in the next major version

  • Environment variable: USE_STD_ERROR=true
  • Reporter option: useStdError=true

Record hook failures

Record failures for hooks such as before/after etc Please note this will probably be made default in the next major version

  • Environment variable: RECORD_HOOK_FAILURES=true
  • Reporter option: recordHookFailures=true

Display Ignored tests as ignored

Display skip tests as ignored

  • Environment variable: DISPLAY_IGNORED_AS_IGNORED=true
  • Reporter option: displayIgnoredAsIgnored=true

Ignore hooks with title contains some text

This option should be used in pair with recordHookFailures. It allows you to skip reporting of hooks containing some word. Including root hooks.

  • Environment variable: IGNORE_HOOK_WITH_NAME=HookNoReporting
  • Reporter option: ignoreHookWithName=HookNoReporting

Example: mocha test --reporter mocha-teamcity-reporter --reporter-options recordHookFailures --reporter-options ignoreHookWithName=HookNoReporting

For root hooks defined the following way:

exports.mochaHooks = () => {
 return {
  beforeEach: [
   function beforeEachRootHookNoReporting() {
    assert.strictEqual(1, 1);
   }
  ],
  afterEach: [
   function afterEachRoot() {
    assert.strictEqual(1, 1);
   },
  ]
 };
};

beforeEach hook beforeEachRootHookNoReporting() will not be reported as testStarted. But hook afterEachRoot() will be reported:

Show diff between expected and actual values

This will allow a hyperlink to appear to compare actual vs expected Please note this requires the error thrown in mocha to have the properties actual and expected. For example an assertionError has this

  • Environment variable: ACTUAL_VS_EXPECTED=true
  • Reporter option: actualVsExpected=true

This will be shown in teamcity like this:

AssertionError [ERR_ASSERTION]: 2 == 1
     at Context.<anonymous> (test/test_data/simple.js:11:11)
 ======= Failed test run #10 ==========
 Show diff between expected and actual values

Setting options

  • Set with reporter-options:

mocha test --reporter mocha-teamcity-reporter --reporter-options topLevelSuite=top-level-suite-name mocha test --reporter mocha-teamcity-reporter --reporter-options useStdError=true mocha test --reporter mocha-teamcity-reporter --reporter-options useStdError=true

  • Set with environment variable

MOCHA_TEAMCITY_TOP_LEVEL_SUITE='top-level-suite-name' mocha test --reporter mocha-teamcity-reporter

Multiple reporters

This is not supported out of the box by this plugin but have a look at the following:

View on live Teamcity

Contributions

  • Always Welcome
  • Would prefer if customisation is added it is controlled via mocha options or environment variables
  • Only requirement is for code to pass linting and functional tests

Run example test in project

mocha test/test_data/simple.js --reporter mocha-teamcity-reporter or npm run test-teamcity-example

Reference Information

https://confluence.jetbrains.com/display/TCD10/Build+Script+Interaction+with+TeamCity

mocha-teamcity-reporter's People

Contributors

bdefore avatar chge avatar danjenkins avatar dj-glock avatar eagleeye avatar fernyb avatar jamie-sherriff avatar leninlin avatar mjesun avatar rajwilkhu avatar travisjeffery avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

mocha-teamcity-reporter's Issues

We would like to take over maintenance if you want?

If you would like us to, we at debitoor would like to take over maintenance of this module.
We are quite a few people at debitoor, and we have changing employees. So my suggestion would be that you transfer the repo to our GitHub org debitoor and add me on npm my npm name is ebdrup. Then I can add the other team members on npm.

Output to File

Is there no way to output results to file? I didn't see any under the options listed

Implement 'hook end' event

The problem

Does not emit testFinished message when a hook ends. (before, beforeEach, etc)
This means the grouping in TC's log is broken, because peers of the hook are erroneously shown as children of the hook.

Environment

  • What mocha-teamcity-reporter version are you using: 3.0.0
  • Version of NPM/yarn etc and Node.js: npm 6.13.4, node 14.0.0
  • Version of teamcity (If Applicable) 2019.2.1
  • Using mocha 7.1.1

Details

A "Test start" message is emitted when a hook is run, but no corresponding "Test finished"
https://github.com/travisjeffery/mocha-teamcity-reporter/blob/master/lib/teamcity.js#L131-L135

Error when using with mocha-phantomjs-core

As of version 1.1.0, mocha-teamcity-reporter cannot be loaded by mocha-phantomjs-core. It fails with the following output:

"C:\Users\johng\dev\MyProject\node_modules\mocha-teamcity-reporter\lib\teamcit
y.js" reporter not found
Failed to use load and use the custom reporter C:\Users\johng\dev\GeoNexus\no
de_modules\mocha-teamcity-reporter\lib\teamcity.js

That error is misleading because the reporter is actually being loaded, but Mocha.reporters.Custom is being set to null, which fails a check here.

It would appear that exports/module.exports is not being set correctly in this environment.

TeamCity reports less tests than there actually are with multiple 'describe' levels

We've come across an issue with TeamCity only reporting a limited number of Mocha tests when several 'describe' levels are present. We use this in conjunction with mocha-each:

const withThese = require('mocha-each');

describe('@tag1 @tag2 When I do A', function() {
    describe('@tag3 with params B', function() {
        describe('@tag4 and expect C', function() {
            let correctParameters = [
                [1],
                [2],
                [3]
            ];
            withThese(correctParameters )
                .it('does D with %s', (param1) => {
        });
    });
});

When this is run locally using the spec reported the amount of tests is correct but within TeamCity the summary is wrong. The log lists all the tests but the ultimate result of the test run is incomplete.

useStdError=true can bring false positive test results

The problem

I am not quite sure that this is fixable on the reporter side, but I feel I need to create the ticket just to bring an attention to the others.
Teamcity has a known issue of output stream synchronizations
Which means the order of messages across stdOut and stdErr is not guaranteed.
It also appears that teamcity cares about the message order in regards to test reports (at least it cares in 2020.1)

With useStdError=true the reporter will throw 'testFailure' into stdErr, but 'testFinished' into stdOut. if you are unlucky, 'testFinished' will arrive earlier and the test will be marked as passed

Environment

  • What mocha-teamcity-reporter version are you using: 4.0.0
  • Version of NPM/yarn etc and Node.js: node 14.13.1, npm 6.14.10
  • Version of teamcity (If Applicable): 2020.1 (on premises)

Details

An example test or code sample

To see the effect, you don't need to install mocha and teamcity reporter plugin.
Just create a teamcity build configuration with one linux shell step:

echo "##teamcity[testSuiteStarted name='suite0' flowId='40792']"
echo "##teamcity[testStarted  name='test1' captureStandardOutput='true' flowId='40792']"
>&2 echo "##teamcity[testFailed   name='test1' message='horrible error' captureStandardOutput='true' flowId='40792']"
echo "##teamcity[testFinished name='test1' duration='1424' flowId='40792']"
echo "##teamcity[testStarted  name='test2' captureStandardOutput='true' flowId='40792']"
>&2 echo "##teamcity[testFailed   name='test2' message='horrible error' captureStandardOutput='true' flowId='40792']"
echo "##teamcity[testFinished name='test2' duration='1481' flowId='40792']"
echo "##teamcity[testStarted  name='test3' captureStandardOutput='true' flowId='40792']"
>&2 echo "##teamcity[testFailed   name='test3' message='horrible error' captureStandardOutput='true' flowId='40792']"
echo "##teamcity[testFinished name='test3' duration='1583' flowId='40792']"
echo "##teamcity[testStarted  name='test4' captureStandardOutput='true' flowId='40792']"
>&2 echo "##teamcity[testFailed   name='test4' message='horrible error' captureStandardOutput='true' flowId='40792']"
echo "##teamcity[testFinished name='test4' duration='1623' flowId='40792']"
echo "##teamcity[testStarted  name='test5' captureStandardOutput='true' flowId='40792']"
>&2 echo "##teamcity[testFailed   name='test5' message='horrible error' captureStandardOutput='true' flowId='40792']"
echo "##teamcity[testFinished name='test5' duration='1525' flowId='40792']"
echo "##teamcity[testStarted  name='test6' captureStandardOutput='true' flowId='40792']"
>&2 echo "##teamcity[testFailed   name='test6' message='horrible error' captureStandardOutput='true' flowId='40792']"
echo "##teamcity[testFinished name='test6' duration='1466' flowId='40792']"
echo "##teamcity[testSuiteFinished name='suite0' duration='12740' flowId='40792']"
echo "Executing global teardown"

Run the build several times and observe random results

Failed tests appears as successful in TeamCity

The problem

I use mocha-teamcity-reporter with the following options:
--reporter-options recordHookFailures=true --reporter-options ignoreHookWithName=RootHook

In the raw log I can see that test fails and messages are fine:

�[36me2e_1  |�[0m ##teamcity[testStarted name='FtsUI - 1.2 Security selected #full #intfull' captureStandardOutput='true' flowId='19']
�[36me2e_1  |�[0m ##teamcity[testStarted name='"before each" hook: Clean search in "FtsUI - 1 Search Input "' captureStandardOutput='true' flowId='19']
�[36me2e_1  |�[0m ##teamcity[testFinished name='"before each" hook: Clean search in "FtsUI - 1 Search Input "' flowId='19']
�[36me2e_1  |�[0m ##teamcity[testFailed name='FtsUI - 1.2 Security selected #full #intfull' message='Test and Master SecuritySelected.png files should be equal: expected |'Screenshot /e2e/test-screenshots/FTS/SecuritySelected.png did not match with master screenshot /e2e/master-screenshots/FTS/SecuritySelected. DiffBounds: {"left":0,"top":0,"right":180,"bottom":31} Diff-image will be saved to /e2e/diff-screenshots/FTS/SecuritySelected.png.|' to equal true' details='AssertionError: Test and Master SecuritySelected.png files should be equal: expected |'Screenshot /e2e/test-screenshots/FTS/SecuritySelected.png did not match with master screenshot /e2e/master-screenshots/FTS/SecuritySelected. DiffBounds: {"left":0,"top":0,"right":180,"bottom":31} Diff-image will be saved to /e2e/diff-screenshots/FTS/SecuritySelected.png.|' to equal true|n    at Function.compareScreenshots (src/Tests/UIGeneric.ts:21:80)|n    at processTicksAndRejections (internal/process/task_queues.js:93:5)|n    at Context.<anonymous> (src/TestsPoMa/FTS/FTS.prl.ui.ts:45:7)' captureStandardOutput='true' flowId='19']
�[36me2e_1  |�[0m ##teamcity[testFinished name='FtsUI - 1.2 Security selected #full #intfull' duration='6166' flowId='19']

But in TeamCity:
image

I suspect that TeamCity parses the following line and thinks that test finished:
�[36me2e_1 |�[0m ##teamcity[testFinished name='"before each" hook: Clean search in "FtsUI - 1 Search Input "' flowId='19']

This was changed in scope of #54 before that testFinished was not logged for hooks:
https://github.com/travisjeffery/mocha-teamcity-reporter/pull/58/files

I believe that the root cause of this issue is a but on TeamCity side - it looks like they find the next testFinished line and think that test is finished. But this line is for hook!

I will raise now an issue for TeamCity to check if it was fixed or not (we are using 2020.2.1 TC version and are not going to update it soon). So If they confirm that issue is on their side - I will create a PR to revert hook end implementation.

Environment

  • What mocha-teamcity-reporter version are you using: 4.0.0
  • Version of teamcity (If Applicable): 2020.2.1 (build 85633)

Constants instead of event names

Please treat this issue as a gentle reminder. I will probably handle it later, if you don't mind.

The problem

Quote from mocha document https://mochajs.org/api/tutorial-custom-reporter.html:
Please use these constants instead of the event names in your own reporter! This will ensure compatibility with future versions of Mocha.

image

At the moment reporter use test names. So it's better to change it from using names to constants. To avoid issues cause by
breaking changes on mocha side.

Require import Incompatibility with mocha 6.0.0

When mocha 6 is used test run fails with error:

node_modules/mocha-teamcity-reporter/lib/teamcity.js:88
        Base.call(this, runner);
      ^
TypeError: Cannot read property 'call' of undefined

require('mocha').reporters.Base seems to be missing in mocha 6.

Stdout/stderr from tests aren't properly wrapped I think

The problem

My experience with Jest & Jest-teamcity has been that console.X messages are written wrapped with the relevant TeamCity adoration to assign the output to the running test it was for. However, this reporter seems to treat console.X messages as passthrough. Another niggle is that when using Miniflare 3, you can get output to stdio directly from workerd. So it would be nice if stdout and stderr were redirected into the reporter on launch and adorned with the appropriate TeamCity wrapper text to assign it to the specific test generating that output..

Environment

  • What mocha-teamcity-reporter version are you using
  • 4.2
  • Version of NPM/yarn etc and Node.js
  • npm 9.2, node 18.13
  • Version of teamcity (If Applicable)
  • 2021.2.3 build 99711

Details

An example test or code sample

example.test.js:

test('abc', async () => {
   console.error('waiting')
   await new Promise((resolve) => setTimeout(resolve, 100))
   process.stdout.write('finished wait\n')
})

I would expect waiting and finished wait to be printed adorned with TeamCity service messages assigning that output to test abc.

As I understand it, TeamCity only allows a single testStdOut / testStdErr per test so it sounds like you have to buffer stdout / stderr to memory/disk and then output it BEFORE writing out testFinished / testFailed. That being said, I have my own custom wrapper that I wrote using teamcity-service-message for non-mocha things and generating multiple testStdout / testStderr lines repeatedly seems to maybe work (although maybe I'm just not noticing the failure mode).

Pending tests are displayed as Passed in teamcity UI

The problem

Briefly describe the issue you are experiencing (or the feature you want to add)
Tests shown as pending in Build logs are incorrectly displayed as Passed/Successful in Overview and Tests tab in teamcity UI.

Environment

  • What mocha-teamcity-reporter version are you using
  • 4.1.0
  • Version of NPM/yarn etc and Node.js
  • npm version 6.14.11, Node.js version 12.21.0
  • Version of teamcity (If Applicable)
    2020.2.4 (build 86063)
    Cypress version - 8.3.0
  • Grep library version
  • 2.5.3

Details

  • If necessary, describe the problem you have been experiencing in more detail.
  • Such as a stacktrace or error message

We use --env grepTags=@SMOKE along with following parameters with docker run command
cypress run --reporter mocha-teamcity-reporter
Teamcity(TC) buildlog shows tests under Pending and Passing status correctly as shown below
image
However, the TC overview tab shows all Pending tests as passed as seen below
image
The TC Tests tab shows all tests as successful. When I filter by 'ignored' status, it does not return any tests as shown below.
image
Note: TC does not have test filter statuses like Pending, which are mocha specific as mentioned here. The only default filters TC has is 'successful', ''failed' and 'ignored'.

Differences b/w mocha teamcity reporter and cypress using mocha teamcity reporter can be found here.

Expected behavior:
Pending tests from cypress teamcity reporter should display as Ignored tests in Teamcity UI. It should not display under 'Successful'/Passed status on TC UI.

An example test or code sample

  • Please remember that with sample code it's easier to reproduce the bug and it's much faster to fix it.
  • Not needed for feature requests or questions

Issue with running two mocha runs in serial and parallel modes.

The problem

I use mocha 8.2.1 for running tests and mocha-teamcity-reporter as a reporter for teamcity. I have test files that can be run in parallel. And also I have some test files that should be run one by one - in serial mode. As I understood mocha documentation, mocha can run test files only in parallel or in serial mode. So I tried to do the following: run two mocha runs with & command in one yarn script, like this:

mocha --reporter mocha-teamcity-reporter --timeout 90000 -r ts-node/register/transpile-only src/Tests/.ts --parallel --jobs 4 & mocha --reporter mocha-teamcity-reporter --timeout 90000 -r ts-node/register/transpile-only src/Tests/seq.ts

But for some reason using this approach reporter does not work properly. It does not add ##teamcity messages. So TeamCity does not detect all tests that were executed. And this is a big problem :)
I have global hooks that are executed before and after launching each test to log "Started test:" and "Finished test:" messages. So it's was easy to find that some tests are not logged properly:
image
image

Environment

  • mocha-teamcity-reporter 3.0.0
  • mocha 8.2.1
  • yarn 2.3.3
  • Version of teamcity does not matter

Details

I'll be happy to provide any additional information, but not sure what. I believe the issue here is in combined mocha commands. Because when it was only one launch (parallel) everything was fine. My full command for running tests is this:

"test:smoke_dev": "mocha --reporter mocha-teamcity-reporter --timeout 90000 -r ts-node/register/transpile-only --require src/Generic/mochaHooks.ts src/Tests/**/*prl.e2e.ts src/TestsPoMa/**/*prl.e2e.ts --parallel --jobs 4 --grep '#devsmoke' & mocha --reporter mocha-teamcity-reporter --timeout 90000 -r ts-node/register/transpile-only --require src/Generic/mochaHooks.ts src/Tests/**/*seq.e2e.ts src/TestsPoMa/**/*seq.e2e.ts --grep '#devsmoke'",

Enable Travis

Hi Travis, If you get a chance, i had Travis setup on my fork for running tests. If you can enable it on your end that would be great!

Running inside mocha-webpack fails

The specific line https://github.com/travisjeffery/mocha-teamcity-reporter/blob/master/lib/teamcity.js#L18 decides whether to pull the reporters base from mocha import or phantomjs. When running tests inside mocha-webpack (e.g. using Vue.js) the reporter fails with cannot resolve module require('./base')

The fix should be in the test typeof window === 'undefined' to check if the environment being run in is mocha-webpack and if so simply use the first part of the if statement thus assuming mocha.

Currently I have simply copied the code into our project and replaced:

if (typeof window === 'undefined') {
  // running in Node
  Base = require('mocha').reporters.Base;
  log = console.log;
  logError = console.error;
} else {
  // running in mocha-phantomjs
  Base = require('./base');
  log = function (msg) {
    process.stdout.write(msg + '\n');
  };
  logError = function (msg) {
    process.stderr.write(msg + '\n');
  };
}

with

// running in Node
Base = require('mocha').reporters.Base;
log = console.log;
logError = console.error;

This will work as a fix for us but an actual fix to support mocha-webpack would be greatly appreciated and open up this tools use to vue.js projects on 2.0+

Duration is reported as NaN on testSuiteFinished

Added this reporter to some of my unit tests and I'm getting output like this:

##teamcity[testSuiteFinished name='messageRow(context, options)' duration='NaN']
##teamcity[testSuiteFinished name='Helpers' duration='NaN']
##teamcity[testSuiteFinished name='templateUtils' duration='NaN']

Run reporter in browser

Hello!

Is there any legal way to run mocha-teamcity-reporter in browser?
Simple adding

<script src="node_modules/mocha-teamcity-reporter/lib/teamcity.js"></script>

obviously leads to errors.

We test video players' iframes. In this way we run our tests in browser.
And we've set transferring log messages from browser console to node script, so we only need to get teamcity log messages in browser console.

"mocha-teamcity-reporter" reporter blew up with error:

20:19][Step 4/4] "mocha-teamcity-reporter" reporter blew up with error:
[08:20:19][Step 4/4] ReferenceError: window is not defined
[08:20:19][Step 4/4] at Object. (D:\tc\ba1\work\39f0d6d839da2ca9\src\node_modules\mocha-teamcity-reporter\lib\teamcity.js:25:5)
[08:20:19][Step 4/4] at Module._compile (module.js:541:32)

Version: 1.1.0

Previously. 1.0.2 works fine

Running tests in parallel

Trying to use this reporter with Cypress and parallelization mode : https://docs.cypress.io/guides/guides/parallelization.html#Overview
This means that my tests are distributed on a multiple number of machines which run tests in parallel.

Everything works fine with the exception that a failing test is wrongly displayed under another describe.

Example:
describe('Describe A', () => {
it('should test 1', () => { });
it('should test 2', () => { });
});
describe('Describe B', () => {
it('should test 3', () => { });
it('should test 4', () => { });
});

On Teamcity I may get reported that the failing test 'should test 3' or 'should test 4' belongs to 'Describe A' and 'should test 1' or 'should test 2' belong to 'Describe B'.

Is there any way to use this reporter and run the tests in parallel?

Cannot read property 'duration' of undefined

I've just noticed the following error being thrown when running our client side tests in TeamCity:

[12:50:27][Step 3/4] Cannot read property 'duration' of undefined
[12:50:27]
[Step 3/4] TypeError: Cannot read property 'duration' of undefined
    at Runner.<anonymous> (node_modules\mocha-teamcity-reporter\lib\teamcity.js:127:52)
[12:50:27][Step 3/4] C:\BuildAgent\work\2d7908fc7e0f07c9\WhiteLabelFrontEndTests\node_modules\mocha-teamcity-reporter\lib\teamcity.js:127
[12:50:27][Step 3/4] 		log(formatString(SUITE_END, 'mocha.suite', stats.duration, flowId));

Nothing's changed with the setup or package versions.

Any idea why the stats object would be undefined and what might be causing this?
We're using version 2.4.0.

Many thanks

Failed tests logged to stderr?

We use this module a lot. Thanks for making it!
Would it be possible to log failed tests to stderr? If yes, then I will happily make a pull request, otherwise I'll do our own fork. But I would rather do it here.

Don't report about root hook start - when recordHookFailures enabled

The problem

After adding flag recordHookFailures in scope of #35, the reporter started to add testStarted tag for any hook that was run. So TeamCity is now able to detect hooks as tests, regardless of their state. It works fine for hooks defined in scope of test suite. But it causes another problem - it generates tag for global hooks, including hooks beforeEach and afterEach that are being executed for every test. So when recordHookFailures is enabled we can see much more tests in the TeamCity that we want to see.

Example

We have a simple test suite with 2 tests, beforeEach, before and after hooks:

describe('Login - Login page tests.', () => {
  before('Before hook', async () => {
    console.log('Before hook');
  });

  beforeEach('Before each local hook', async () => {
    console.log('Before each local hook');
  });

  it('Test 1', async () => {
    console.log('Test 1 OK');
  });

  it('Test 2', async () => {
    console.log('Test 2 OK');
  });

  after('After hook', async () => {
    console.log('After hook');
  });
});

Also we have a global hooks that run for every test. Example from real life: beforeEach logs start of the test (for better navigation in log of parallel tests execution); afterEach hook checks test state and creates browser pages screenshot if test failed, if not - it does nothing. Simple example:

export const mochaHooks = async () => {
  return {
    beforeEach() {
      console.log(`Before each global hook`);
    },
    afterEach() {
      console.log(`After each global hook`);
    },
  };
};

So if we run the following suite with no recordHookFailures parameter, we will get the following log:
mocha --reporter mocha-teamcity-reporter -r ts-node/register/transpile-only --require src/Generic/mochaHooks.ts src/Tests/**/Test.e2e.ts

##teamcity[testSuiteStarted name='Login - Login page tests.' flowId='22296']
Before hook
##teamcity[testStarted name='Test 1' captureStandardOutput='true' flowId='22296']
Before each global hook
Before each local hook
Test 1 OK
##teamcity[testFinished name='Test 1' duration='0' flowId='22296']
After each global hook
##teamcity[testStarted name='Test 2' captureStandardOutput='true' flowId='22296']
Before each global hook
Before each local hook
Test 2 OK
##teamcity[testFinished name='Test 2' duration='0' flowId='22296']
After each global hook
After hook
##teamcity[testSuiteFinished name='Login - Login page tests.' duration='3' flowId='22296']

In TC it will look like this:
image

But if we run the same test suite with recordHookFailures=true parameter, we will get another log:
mocha --reporter mocha-teamcity-reporter --reporter-options recordHookFailures=true -r ts-node/register/transpile-only --require src/Generic/mochaHooks.ts src/Tests/**/Test.e2e.ts

##teamcity[testSuiteStarted name='Login - Login page tests.' flowId='25512']
##teamcity[testStarted name='"before all" hook: Before hook for "Test 1"' captureStandardOutput='true' flowId='25512']
Before hook
##teamcity[testStarted name='Test 1' captureStandardOutput='true' flowId='25512']
##teamcity[testStarted name='"before each" hook: beforeEach for "Test 1"' captureStandardOutput='true' flowId='25512']
Before each global hook
##teamcity[testStarted name='"before each" hook: Before each local hook for "Test 1"' captureStandardOutput='true' flowId='25512']
Before each local hook
Test 1 OK
##teamcity[testFinished name='Test 1' duration='1' flowId='25512']
##teamcity[testStarted name='"after each" hook: afterEach for "Test 1"' captureStandardOutput='true' flowId='25512']
After each global hook
##teamcity[testStarted name='Test 2' captureStandardOutput='true' flowId='25512']
##teamcity[testStarted name='"before each" hook: beforeEach for "Test 2"' captureStandardOutput='true' flowId='25512']
Before each global hook
##teamcity[testStarted name='"before each" hook: Before each local hook for "Test 2"' captureStandardOutput='true' flowId='25512']
Before each local hook
Test 2 OK
##teamcity[testFinished name='Test 2' duration='1' flowId='25512']
##teamcity[testStarted name='"after each" hook: afterEach for "Test 2"' captureStandardOutput='true' flowId='25512']
After each global hook
##teamcity[testStarted name='"after all" hook: After hook for "Test 2"' captureStandardOutput='true' flowId='25512']
After hook
##teamcity[testSuiteFinished name='Login - Login page tests.' duration='8' flowId='25512']

And we will see 8 tests in TeamCity: 2 real tests, 1 before and 1 after hooks, 2 afterEach hooks (global) and 2 beforeEach hooks (I assume global and local hooks were united by TeamCity), so in total there were 10 testStarted messages:
image

image

Proposed solution

It's great to see all local hooks as tests in TeamCity build.
But we would like to have an ability (another flag) to skip this reporting for global hooks that defined via separate file and parameter in mocha : --require filename. Let's say --ignoreGlobalHooks flag.

Is it possible?

Background

Let me give you a context of the problem. I'm trying to implement the ability to mute failed tests and hooks via TeamCity. TeamCity allows to mute tests to make build green. It works fine but only if we disable the following condition: one of build steps exited with an error (e.g non-zero exit code in command line runner):
image
If we do not disable it, build will be always treated as failed because mocha returns exit code > 0 if any test or hook was failed.
So we decided to switch it off. But we faced another issue - we will not be able to realize that any hook failed because it does not look as a test for TeamCity and exit code is ignored. The only way to see it is to check count of executed test. Not the best way actually. The main issue here is tests that are skipped by mocha because of hook failure. You can check mochajs/mocha#4392 issue for details
So we enabled recordHookFailures flag and faced this issue - a lot of tests appeared in each build.

Environment

mocha-teamcity-reporter: 3.0.0
mocha: 8.3.2

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.