bahmutov / cypress-and-jest

Cypress and Jest both with code coverage running unit tests

cypress-and-jest

Demo unit tests with code coverage from both Cypress and Jest

Use

# install and run tests
$ npm it
# runs Jest and Cypress tests headlessly
# generates combined code coverage report
$ open coverage/lcov-report/index.html

The application source code is just a single file src/calc.js. Let's cover this file with unit tests by using two test runners: Jest and Cypress.

Jest init

The Jest setting were initialized using npx jest --init command:

In the generated jest.config.js enable code coverage collection, and output into folder jest-coverage (to avoid clashing with Cypress coverage report).

// jest.config.js
module.exports = {
  // Indicates whether the coverage information should be collected while executing the test
  collectCoverage: true,
  // The directory where Jest should output its coverage files
  coverageDirectory: 'jest-coverage',
  // The test environment that will be used for testing
  testEnvironment: 'node'
}

The Jest unit tests are in the file tests/calc.test.js and the tests only run the add function. We can run the Jest tests and see the coverage summary.

The coverage reports in jest-coverage folder by default include JSON, LCOV and static HTML reports. The HTML report shows that the function sub was not reached by the Jest tests.

Cypress init

First install Cypress

$ npm install -D cypress

Initialize Cypress folder with npx @bahmutov/cly init command.

In cypress/integration/spec.js let's require sub function and test it

// cypress/integration/spec.js
const { sub } = require('../../src/calc')
it('subtracts 10 - 5', () => {
  expect(sub(10, 5)).to.equal(5)
})

The test passes

Cypress code coverage setup

Code coverage in Cypress is done via @cypress/code-coverage plugin. I suggest following the installation instructions in that repo.

First, install the plugin and babel-plugin-istanbul to instrument code.

npm i -D @cypress/code-coverage babel-plugin-istanbul

Add to your cypress/support/index.js file:

import '@cypress/code-coverage/support'

Register tasks in your cypress/plugins/index.js file:

module.exports = (on, config) => {
  require('@cypress/code-coverage/task')(on, config)
  on('file:preprocessor', require('@cypress/code-coverage/use-browserify-istanbul'))
  return config
}

Because we saved the Jest coverage report in jest-coverage, set Cypress to save its coverage to cypress-coverage. Since nyc is used to generate the report, add nyc object to package.json file.

{
  "nyc": {
    "report-dir": "cypress-coverage"
  }
}

Run Cypress with npx cypress open and a report should be saved. As you can see, we have missed the add function!

Merge coverage reports

We have two folders with coverage reports generated by Jest and Cypress.

$ ls -la *-coverage
cypress-coverage:
total 24
drwxr-xr-x   6 gleb  staff   204 Jul 22 23:04 .
drwxr-xr-x  20 gleb  staff   680 Jul 22 23:05 ..
-rw-r--r--   1 gleb  staff   744 Jul 22 23:05 clover.xml
-rw-r--r--   1 gleb  staff  1000 Jul 22 23:05 coverage-final.json
drwxr-xr-x  10 gleb  staff   340 Jul 22 23:04 lcov-report
-rw-r--r--   1 gleb  staff   201 Jul 22 23:05 lcov.info

jest-coverage:
total 24
drwxr-xr-x   6 gleb  staff   204 Jul 22 21:00 .
drwxr-xr-x  20 gleb  staff   680 Jul 22 23:05 ..
-rw-r--r--   1 gleb  staff   744 Jul 22 23:05 clover.xml
-rw-r--r--   1 gleb  staff  1000 Jul 22 23:05 coverage-final.json
drwxr-xr-x  10 gleb  staff   340 Jul 22 21:00 lcov-report
-rw-r--r--   1 gleb  staff   201 Jul 22 23:05 lcov.info

Let's combine the two reports and generate the final report. There is a script in package.json that does just that. It copies cypress-coverage/coverage-final.json and jest-coverage/coverage-final.json into a folder, runs nyc merge command, the creates the combined report using nyc report ... command.

$ npm run report:combined
...

coverage files in reports merged into coverage.json

> [email protected] report:combined /Users/gleb/git/cypress-and-jest
> nyc report --reporter lcov --report-dir coverage

The final HTML report shows that we have reached all source statements in calc.js file.

Coverage CI artifact

You can store the produced static HTML report on your continuous integration server. For example see .circleci/config.yml file.

More info

• Cypress code coverage guide
• @cypress/code-coverage plugin

cypress-and-jest's Issues

Check produced coverage using strict mode

Extract these scripts into reusable NPM module and check that coverage is 100% for calc.js and only for that file.

https://github.com/cypress-io/code-coverage/tree/master/scripts

We cannot instrument Cypress unit tests by using .babelrc file

Because when we add istanbul plugin there we get duplicate plugin error from Jest

Different folder level between Jest and Cypress

Hi,

When running unit coverage and e2e coverage, my reports are not at the same level:

coverage-unit > lcov-report > my-app-name > my code # Jest
coverage-e2e> lcov-report > my code # Cypress

Do you have any idea how to configure either Cypress or Jest to use the same l`evel (the easiest to configure probably being Jest)?

Also I hit Cannot read property 'loc' of undefined error when running nyc report --reporter lcov --report-dir coverage but that's probably a different issue.

Thanks a lot for all your config examples, I have managed to setup Cypress coverage in a TS + Next project, which was very tricky. I am at the last step of merging Jest and Cypress and very eager to see the result!

Edit: I also seem to have different line counts/branch counts between them.

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

Using it with TypeScript

I tried using samples from this repo in order to get coverage from TypeScript files. Unfortunately, it looks I'm missing the way to provide TypeScript config to the Browserify instance. With browserify it could be like that:

browserify({
  typescript: require.resolve('typescript')
})

I tried to do that:

module.exports = (on, config) => {
    require('@cypress/code-coverage/task')(on, config);

    const options = browserify.defaultOptions;

    options.typescript = require.resolve('typescript');

    on('file:preprocessor', browserify(options));

    return config;
};

But it fails with an error. How should I pass the TypeScript config to Browserify?

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update dependency @cypress/code-coverage to v3.12.42
• chore(deps): update dependency babel-plugin-istanbul to v7
• chore(deps): update dependency cypress to v13
• chore(deps): update dependency jest to v29
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

npm

package.json

• @cypress/code-coverage 3.9.12
• babel-plugin-istanbul 6.1.1
• check-code-coverage 1.10.5
• cypress 4.12.1
• jest 24.9.0

---

• Check this box to trigger a request for Renovate to run again on this repository

nyc merge behaves differently locally than on circle ci

I am wondering if you have may seen the same issue and taken the merge code out of your .circleci file. Anyway, here goes:
I am using nyc merge to combine my Cypress and Jest code coverage reports. When I execute the below steps locally, I get the combined report that I would expect (coverage is a bit more than either suite provides individually). I am executing the exact same steps in CircleCI and the combined report contains ONLY the Jest results. I have used SSH to verify that the Cypress coverage report is indeed being generated and copied around properly. I have verified that I am running the same version of nyc locally and on CircleCI (15.1.0).

            mkdir -p reports
            cp jest-coverage/coverage-final.json reports/from-jest.json
            cp cypress-coverage/coverage-final.json reports
            mkdir -p .nyc_output
            npx nyc merge reports
            cp coverage.json .nyc_output/out.json
            npx nyc report --reporter=lcov --reporter=text --report-dir=reports

Also posted to stackoverflow: https://stackoverflow.com/questions/66575311/nyc-merge-behaves-differently-locally-than-on-circle-ci

Error: Can't walk dependency graph: Cannot find module

Hey, I am experiencing an issue when I add this bit into my plugins file:

on('file:preprocessor', require('@cypress/code-coverage/use-browserify-istanbul'));

stack is: React/Typescript/Webpack/Jest/Cypress

Seems it can't follow imports from the root src directory. Node module imports are fine. I have aliased my imports. When remove that line above, all works as before. When I added, things break. I tried not aliasing files, same error.

import {CONSTANT1, CONSTANT2, CONSTANT3} from 'mocks/constants'; <-- breaks
import {UTIL_CONSTANT1} from 'utils/auth'; <--- breaks
import set from 'lodash/set'; <--- fine
import get from 'lodash/get'; <--- fine

import {CONSTANT1, CONSTANT2, CONSTANT3} from '../../src/mocks/constants'; <-- also breaks
import {UTIL_CONSTANT1} from '../../src/utils/auth'; <--- also breaks

full plugin file:

module.exports = (on, config) => {
    const options = {
      webpackOptions: require('../../tools/webpack.config')({ env: 'dev' }),
    };
    on('file:preprocessor', wp(options));
    // cypress coverage
    require('@cypress/code-coverage/task')(on, config);
    on('file:preprocessor', require('@cypress/code-coverage/use-browserify-istanbul'));
    return config;
};