bcoe / c8

output coverage reports using Node.js' built in coverage

c8 - native V8 code-coverage

Code-coverage using Node.js' built in functionality that's compatible with Istanbul's reporters.

Like nyc, c8 just magically works:

npm i c8 -g
c8 node foo.js

The above example will output coverage metrics for foo.js.

CLI Options / Configuration

c8 can be configured via command-line flags, a c8 section in package.json, or a JSON configuration file on disk.

A configuration file can be specified by passing its path on the command line with --config or -c. If no config option is provided, c8 searches for files named .c8rc, .c8rc.json, .nycrc, or .nycrc.json, starting from cwd and walking up the filesystem tree.

When using package.json configuration or a dedicated configuration file, omit the -- prefix from the long-form of the desired command-line option.

Here is a list of common options. Run c8 --help for the full list and documentation.

Option	Description	Type	Default
-c, --config	path to JSON configuration file	string	See above
-r, --reporter	coverage reporter(s) to use	Array<string>	['text']
-o, --reports-dir, --report-dir	directory where coverage reports will be output to	string	./coverage
--all	see section below for more info	boolean	false
--src	see section below for more info	Array<string>	[process.cwd()]
-n, --include	see section below for more info	Array<string>	[] (include all files)
-x, --exclude	see section below for more info	Array<string>	list
--exclude-after-remap	see section below for more info	boolean	false
-e, --extension	only files matching these extensions will show coverage	string | Array<string>	list
--skip-full	do not show files with 100% statement, branch, and function coverage	boolean	false
--check-coverage	check whether coverage is within thresholds provided	boolean	false
--per-file	check thresholds per file	boolean	false
--temp-directory	directory V8 coverage data is written to and read from	string	process.env.NODE_V8_COVERAGE
--clean	should temp files be deleted before script execution	boolean	true
--experimental-monocart	see section below for more info	boolean	false

Checking for "full" source coverage using --all

By default v8 will only give us coverage for files that were loaded by the engine. If there are source files in your project that are flexed in production but not in your tests, your coverage numbers will not reflect this. For example, if your project's main.js loads a.js and b.js but your unit tests only load a.js your total coverage could show as 100% for a.js when in fact both main.js and b.js are uncovered.

By supplying --all to c8, all files in directories specified with --src (defaults to cwd) that pass the --include and --exclude flag checks, will be loaded into the report. If any of those files remain uncovered they will be factored into the report with a default of 0% coverage.

SourceMap Support

c8 can handle source-maps, for remapping coverage from generated code to original source files (useful for TypeScript, JSX, etc).

Source map files versus inline source maps

Just-in-time instrumented codebases will often insert source maps inline with the .js code they generate at runtime (e.g, @babel/register can be configured to insert a source map footer).

Pre-instrumented codebases, e.g., running tsc to generate .js in a build folder, may generate either inline source maps, or a separate .map file stored on disk.

c8 can handle loading both types of source maps.

Exclude after remap

Depending on the size and configuration of your project, it may be preferable to apply exclusion logic either before or after source-maps are used to remap compiled to original source files.

--exclude-after-remap is used to control this behaviour.

c8 report

run c8 report to regenerate reports after c8 has already been run.

Checking coverage

c8 can fail tests if coverage falls below a threshold. After running your tests with c8, simply run:

c8 check-coverage --lines 95 --functions 95 --branches 95

c8 also accepts a --check-coverage shorthand, which can be used to both run tests and check that coverage falls within the threshold provided:

c8 --check-coverage --lines 100 npm test

The above check fails if coverage falls below 100%.

To check thresholds on a per-file basis run:

c8 check-coverage --lines 95 --per-file

If you want to check for 100% coverage across all dimensions, use --100:

c8 --100 npm test

Is equivalent to

c8 --check-coverage --lines 100 --functions 100 --branches 100 --statements 100  npm test

The --100 flag can be set for the check-coverage as well:

c8 check-coverage --100

Using Monocart coverage reports (experimental)

Monocart is an alternate library for outputting v8 code coverage data as Istanbul reports.

Monocart also provides reporters based directly on v8's byte-offset-based output. Such as, console-details and v8. This removes a complex transformation step and may be less bug prone for some environments.

Example usage:

c8 --experimental-monocart --reporter=v8 --reporter=console-details node foo.js

NOTE: Monocart requires additional monocart-coverage-reports to be installed:

npm i monocart-coverage-reports@2 --save-dev

Ignoring Uncovered Lines, Functions, and Blocks

Sometimes you might find yourself wanting to ignore uncovered portions of your codebase. For example, perhaps you run your tests on Linux, but there's some logic that only executes on Windows.

To ignore lines, blocks, and functions, use the special comment:

/* c8 ignore next */.

Ignoring the next line

const myVariable = 99
/* c8 ignore next */
if (process.platform === 'win32') console.info('hello world')

Ignoring the next N lines

const myVariable = 99
/* c8 ignore next 3 */
if (process.platform === 'win32') {
  console.info('hello world')
}

Ignoring all lines until told

/* c8 ignore start */
function dontMindMe() {
  // ...
}
/* c8 ignore stop */

Ignoring a block on the current line

const myVariable = 99
const os = process.platform === 'darwin' ? 'OSXy' /* c8 ignore next */ : 'Windowsy'

Supported Node.js Versions

c8 uses native V8 coverage, make sure you're running Node.js >= 12.

Contributing to c8

See the contributing guide here.

c8's Issues

Wrong ranges

Not necessarily relates to c8, but I was asked to create an issue here.

I have general struggle with coverage features in node.js.

I tried using c8 package and node.js' inspector module.
Neither of them gives correct lineNumber, columnNumber and ranges (startOffset, endOffset), unfortunately.

c8

when I run c8 node test.js where test.js is this file:

function foo() {
  console.log(123)
}

foo()

I get this in the generated report:

{"functionName":"foo","ranges":[{"startOffset":62,"endOffset":99,"count":1}]

Note the startOffset":62 when there is 44 characters total in test.js. I am not even speaking about "endOffset":99.

inspector

after not successing with c8 I tried to place in my test.js code from this example

and run node test.js
then I got this:

{"functionName":"session.post","scriptId":"66","url":"file:///home/nurbol/www/sandbox/test.js","lineNumber":5,"columnNumber":32},"hitCount":0,"children":[16]}

Note the "lineNumber":5 when there is no anything on line number 5!

Then @bcoe gave me an advice to take into accunt require('module').wrapper[0] that is inserted by node.js. But if done so, line number 5 contains session.connect and not session.post (see the json above)!

My setup:

Linux (Lubuntu).
Node 11.2.0 (also tested in 10.13.0 with the same results)
c8 3.2.1

commented out lines not counted as covered

very interesting tool

I am very interested in using this tool.
But I just cannot figure out how it works.
Some things I was able to figure out by googling the solution.
Other things, not so much.
There's just no documentation for all the possible cli options, or real world usage examples.
I am very intrigued, and yet, frustrated 😈

Feature: c8 --all

nyc --all is widely used, an option for c8 to accomplish the same will be needed before many projects are willing to migrate.

Half crazy though: should all: true be the default, c8 --no-all to disable it?

feat: ignore files outside of the project root by default?feat:

• Version: v12.1.0
• Platform: Windows 7 64-bit

I have some JavaScript configuration files that are dynamically generated during tests. It seems that those generated JS files are also counted as part of the source code. These config files are generated inside a temporary directory created by tmp, and they will be require()ed during the test.

Here is a simple repro, run npm test to see the error. It seems that this problem is Windows-only. At least, Travis doesn't show this error:

D:\Dev\node-test>npm test

> [email protected] test D:\Dev\node-test
> c8 --reporter lcov node index

file: D:\Dev\node-test\config.js error: Error: ENOENT: no such file or directory
, open 'D:\Dev\node-test\config.js'
    at Object.openSync (fs.js:454:3)
    at readFileSync (fs.js:354:35)
    at V8ToIstanbul.load (D:\Dev\node-test\node_modules\v8-to-istanbul\lib\v8-to
-istanbul.js:30:23)
    at Report.getCoverageMapFromAllCoverageFiles (D:\Dev\node-test\node_modules\
c8\lib\report.js:65:25)
    at Report.run (D:\Dev\node-test\node_modules\c8\lib\report.js:37:28)
    at exports.outputReport (D:\Dev\node-test\node_modules\c8\lib\commands\repor
t.js:24:16)
    at D:\Dev\node-test\node_modules\c8\bin\c8.js:31:13
    at ChildProcess.<anonymous> (D:\Dev\node-test\node_modules\foreground-child\
index.js:52:5)
    at ChildProcess.emit (events.js:196:13)
    at ChildProcess.cp.emit (D:\Dev\node-test\node_modules\cross-spawn\lib\enoen
t.js:40:29)

So:

1. Is there a way to exclude a path/pattern from being processed by c8?
2. Can we ignore files outside of the project root by default?

strange missing branch coverage at return statement

• v10.15.1:
• Linux home 4.18.0-16-generic #17-Ubuntu SMP Fri Feb 8 00:06:57 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux:

nyc reports 100% coverage when c8 reports only 75% coverage.

$ cat t.js 
t()
function t () {
  return 42
}
$ ./node_modules/.bin/c8 node t.js 
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |      100 |       75 |      100 |      100 |                   |
 t.js     |      100 |       75 |      100 |      100 |                 3 |
----------|----------|----------|----------|----------|-------------------|

$ node --version
v10.15.1
$ ./node_modules/.bin/c8 --version
3.4.0

LCOV report misaligned after upgrade to Node 10.16

• Version: 10.16.0
• Platform: Windows 10 x64 / Amazon Linux

After upgrading to NodeJS 10.16, the coverage report is off. Here is an example:

10.15.3

10.16.0

I'm assuming a relation to nodejs/node#26579

Integration tests sometimes failing

This is a weird issues, I'm not sure what's going wrong... if it's just me (I'm on Windows), or if it actually has something to do with integration tests

Sometimes the tests fail and sometimes they pass...

Here's a screen grab where I run the same test command:

The two tests that are failing are:

1. 'merges reports from subprocesses together'

   When it fails (see source for what's expected) the actual output is:

   ,first
   
   second
   
   --------------------|----------|----------|----------|----------|-------------------|
   File                |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
   --------------------|----------|----------|----------|----------|-------------------|
   All files           |    94.12 |    73.08 |        0 |    94.12 |                   |
   bin                 |    83.72 |    57.14 |      100 |    83.72 |                   |
     c8.js             |    83.72 |    57.14 |      100 |    83.72 |... 22,40,41,42,43 |
   lib                 |    96.41 |    65.38 |      100 |    96.41 |                   |
     parse-args.js     |    97.47 |    44.44 |      100 |    97.47 |             55,56 |
     report.js         |    95.45 |    76.47 |      100 |    95.45 |       51,52,53,54 |
   test/fixtures       |    95.16 |    89.47 |        0 |    95.16 |                   |
     async.js          |      100 |      100 |      100 |      100 |                   |
     multiple-spawn.js |      100 |      100 |      100 |      100 |                   |
     normal.js         |    85.71 |       75 |        0 |    85.71 |          14,15,16 |
     subprocess.js     |      100 |     87.5 |      100 |      100 |                 9 |
   --------------------|----------|----------|----------|----------|-------------------|
   ,
   

   which differs in the last line (subprocess.js)

2. 'omit-relative can be set to false'

   When it fails (see source for what's expected) the actual output is:

   ,first
   
   second
   
   ,fs.js:115
       throw err;
       ^
   
   Error: EISDIR: illegal operation on a directory, read
       at Object.readSync (fs.js:491:3)
       at tryReadSync (fs.js:330:20)
       at Object.readFileSync (fs.js:367:19)
       at new CovScript (C:\...\c8\node_modules\v8-to-istanbul\lib\script.js:14:23)
       at module.exports (C:\...\c8\node_modules\v8-to-istanbul\index.js:4:10)
       at Object.keys.forEach (C:\...\c8\lib\report.js:65:22)
       at Array.forEach (<anonymous>)
       at Report._getCoverageMapFromAllCoverageFiles (C:\...\c8\lib\report.js:62:32)
       at Report.run (C:\...\c8\lib\report.js:31:22)
       at module.exports (C:\...\c8\lib\report.js:86:10)
   

   It throws an "EISDIR: illegal operation on a directory" error instead of the expected "ENOENT: no such file or directory"

Edit: I'm testing this with #26 checked out, but this issue seems to exist without it too.

Reporting of empty lines as not covered

• Version: Node v10.15.3
• Platform: Darwin Kernel Version 18.5.0

After running my tests I get a coverage report that reports empty lines as being uncovered.

This is one of the files where it does this:

const config = require("config");
const logger = require("./logger");

const geocoderConfig = config.geocoder
const apiKey = geocoderConfig.apiKey;
const extra = {
  apiKey: apiKey,
  timeout: 30000
};

const geocoder = require("node-geocoder")("google", "https", extra);

exports.geocode = (address, zip, city, country) => {

  let params = {
    address: address + ' ' + zip + ' ' + city + ' ' + country
  };

  return new Promise((resolve, reject) => {
    geocoder.geocode(
      params, (error, result) => {

        /* c8 ignore next 3 */
        if (error) {
          logger.error("Error geocoding: " + JSON.stringify({error, result}, null, 3));
          return reject(error);
        }

        /* c8 ignore next 5 */
        if (!result || result.length === 0)
          return resolve({
            latitude: null,
            longitude: null
          });

        resolve({
          latitude: result[0].latitude,
          longitude: result[0].longitude
        });
      });
  });
};

The empty line right above c8 ignore next 3 is reported as uncovered

---------------------------|----------|----------|----------|----------|-------------------|
File                       |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
---------------------------|----------|----------|----------|----------|-------------------|
All files                  |    78.38 |    64.79 |    84.99 |    78.38 |                   |
  geocoder.js              |    97.62 |    71.43 |      100 |    97.62 |                22 |

I also have other files where it reports closing braces as uncovered, e.g. at the end of a file I have

      })
  })
}

and those 3 lines are reported as uncovered.

outputting coverage when process.exit() is called

I think @chrisdickonson came up with a good idea for how to approach this, if we capture an event on process.exit, we'll use spawnSync to output coverage in a subprocess.

const client = await CRI({port: port})
const {Profiler} = client

process.on('exit', () => {
  process.spawnSync('ws-write-coverage.js', ())
  client.close()
})

CC: @schuay, @bmeck

add support for report-dir variable

We use an alternate report directory at work, it would be nice if c8 supported this config option.

100% line coverage when 50% at most with ESM

I'm filing at the request of @bcoe. The bug may actually be in v8-to-istanbul.

To repro npm i -g c8 esm nyc

Then create a file foo.js:

const { log } = console

function a() {
  if (true) {
    log("hi")
  } else {
    log("false")
  }
}

function b() {
  if (true) {
    log("hi")
  } else {
    log("false")
  }
}

a()

and run c8 node foo.js which will report:

hi
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |       55 |       75 |        0 |       55 |                   |
 foo.js   |       55 |       75 |        0 |       55 |... 13,14,15,16,17 |
----------|----------|----------|----------|----------|-------------------|

now change the line const { log } = console to import { log } from "console"
and run c8 node -r esm foo.js which will report:

hi
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |      100 |      100 |      100 |      100 |                   |
 foo.js   |      100 |      100 |      100 |      100 |                   |
----------|----------|----------|----------|----------|-------------------|

If I run nyc node -r esm foo.js instead it will report:

hi
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |    42.86 |       25 |       50 |    42.86 |                   |
 foo.js   |    42.86 |       25 |       50 |    42.86 |        7,12,13,15 |
----------|----------|----------|----------|----------|-------------------|

and with report --reporter=html is

Multiple incompatible reports generated for the same file.

I am investigating a bug where the lcovonly reporter sometimes crashes. I cannot reproduce it reliably, but it happens about 60% of the time.

I was able to find part of the cause why it happens, but I need help to find the root cause and solve it.

The lcovonly report failed at this line because the branchMap and b properties of the file coverage had different keys. It is a broken invariant, meaning that the coverage map was malformed.

The bad file coverage object was for a file called test.esm.js. When looking into c8's tmp directory (coverage/tmp), I found two coverage file for test.esm.js: gist.
If you look at them, you see that they have different branchMap values. The first one has 2 properties, the second one has 4 properties.

When c8 builds its coverage map, it merges all the files in its temporary directory. One of the preconditions of the merge function is that if there are two reports for the same file, the file needs to have the same structure (ex. branchMap). https://github.com/istanbuljs/istanbuljs/blob/829e658dfa91e3a9533842be9ce940dbe7785c09/packages/istanbul-lib-coverage/lib/file.js#L246.

---

So here is the current state of my investigation: lcovonly breaks because it gets an invalid file coverage because multiple reports for the same file but with different structures are merged, thus breaking one of the preconditions of the merge function.

To solve this, c8 should ensure that the reports are unique for each file URL, and that they have the same structure.

v8-to-istanbul relies on file://

(Don’t think this is significant, but:)

• Version: 10.13.0
• Platform: Darwin

In v8-to-istanbul, the presence of file:// in the specifier is used to determine whether or not the CJS header should be used to offset the results:

https://github.com/istanbuljs/v8-to-istanbul/blob/v1.2.1/lib/script.js#L120-L125
https://github.com/istanbuljs/v8-to-istanbul/blob/v1.2.1/lib/script.js#L13-L16

In c8, the specifier is resolved to a non-URL filepath before being passed to v8-to-istanbul:

https://github.com/bcoe/c8/blob/v3.2.1/lib/report.js#L51-L52

ESM modules that were run natively as ESM get interpreted as CJS as a consequence — incorrect offsets get applied to all ranges. When this is changed so that the original specifier is used the ranges are correct.

This behavior was introduced in 3.2.1 when trying to solve for a different issue related to file: specifiers. It seems like file: is probably not a good signal to rely on for determining ESM vs CJS, but it also appears that there’s nothing better available to v8-to-istanbul presently.

bug: coverage reported incorrectly with source-maps on Node 12

This is a strange one, for the following repo, which uses TypeScript and source-maps:

https://github.com/googleapis/release-please

coverage is reported correctly on Node 11, but does not seem to be correct on Node 12; it's over reporting.

@shinnn this seems like a high priority issue, as it leads to over reporting of coverage, which can give false confidence to users.

shebang screws up line count in coverage

missing coverage of closing brace when `return` ends a `switch/case`

• Version: tested on 8.15.1, 10.15.3, 11.14.0, 12.0.0, 13.0.0-pre
• Platform: Darwin Fhqwhgads.local 17.7.0 Darwin Kernel Version 17.7.0: Wed Feb 27 00:43:23 PST 2019; root:xnu-4570.71.35~1/RELEASE_X86_64 x86_64

Code in foo.js:

function checkIt(arg) {
  switch (arg) {
    case true:
      return 'It was true!';
    case false:
      return 'It as not true';
  }
  return 'It was neither true nor false';
}

const val1 = checkIt(true);
const val2 = checkIt(false);
const val3 = checkIt('string');

console.log(val1);
console.log(val2);
console.log(val3);

Command:

c8 node foo.js

Output:

It was true!
It as not true
It was neither true nor false
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |    94.12 |    71.43 |      100 |    94.12 |                   |
 foo.js   |    94.12 |    71.43 |      100 |    94.12 |                 7 |

Expected output:

I expected to see 100% coverage...

fun: switch to async IO

c8 already requires a bleeding edge v8 engine, we should switch to async/await rather than using any synchronous IO.

see @TimothyGu's comments in #8.

I need process to be failed if lines coverage is low, is it possible via c8 api?

• Version: latest
• Platform: Mac

does not verify all conditions when || present

• Version: 12.2.0
• Platform: Darwin LIB-0F7FVH8-LT 18.2.0 Darwin Kernel Version 18.2.0: Thu Dec 20 20:46:53 PST 2018; root:xnu-4903.241.1~1/RELEASE_X86_64 x86_64

foo.js:

function test(a,b) {
  if (a || b) {
    return true;
  }
  return false;
}

test(true, false);
test(false, false);

Command + output:

$ npx c8 node foo.js
npx: installed 123 in 3.436s
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |      100 |      100 |      100 |      100 |                   |
 foo.js   |      100 |      100 |      100 |      100 |                   |
----------|----------|----------|----------|----------|-------------------|
$ 

Expected results:

I expected the branch coverage to be less than 100% because in a || b, the condition never occurs where b is true. nyc is showing the same behavior so maybe I'm confused? But it sure seems like the nyc documentation suggests this should not show 100% coverage. Anyway, not sure if the problem is my repro, my expectations, or if it's a bug in V8 and/or c8 and/or nyc/istanbul. Help?

maintenance: house-keeping proposal

Here is my suggestion on what we can do to keep the house in order in readiness for outside contribution, etc. If I get your approval @bcoe -- I'll go ahead and make the necessary pull request to add these.

Also, let me know any additions/modifications to make.

Proposal Checklist

• Add CONTRIBUTING.md (include git workflow, invite to tooling #c8 channel, etc)
• Add .github/ISSUE_TEMPLATE.md
• Add .github/PULL_REQUEST_TEMPLATE.md
• Add MAINTAINERS_GUIDE.md with basic guidelines (as per the project, not to over-kill)
• Create the documented tags (as agreed on MAINTAINERS_GUIDE.md)
• Create a dedicated channel for #c8 to free #general in node-tooling Slack

Detailed coverage

c8/bin/wrap.js

Line 17 in 38dff30

await Profiler.startPreciseCoverage({callCount: true, detailed: true})

Getting detailed coverage in node is currently still a bit tricky, since it requires that the source is reparsed/recompiled after detailed coverage is enabled.

In Chrome DevTools, that's easy. In Node, not so much. We're looking into ways to make this better :)

cc @hashseed @fhinkel

Support ignore rules

nyc and istanbul let you use ignore rules such as istanbul ignore next or istanbul ignore else.
It would be good to support those too. We could support the prefix c8 or more general coverage so you could write c8 ignore next or coverage ignore else.

Of course, V8 would return coverage even for the ignored parts, but they should not be included in the final report.

This may be a v8-to-istanbul issue.

Performance / optimizations with coverage enabled

Opening a new ticket here to avoid derailing nodejs/node#16531:

code transpiled for test coverage runs significantly slower than the code prior to transpilation; I'm betting collecting coverage directly from v8 will be fast (although the subprocess shenanigans might add some overhead).

Turning on code coverage disables optimization in many cases. It may be possible to allow optimization in some modes (e.g. precise binary coverage, or possibly all detailed coverage modes), but this probably requires investigation.

EISDIR error when converting to Istanbul format

I'm seeing this error:

{ Error: EISDIR: illegal operation on a directory, read at Object.fs.readSync (fs.js:675:18) at tryReadSync (fs.js:540:20) at Object.fs.readFileSync (fs.js:583:19) at new CovScript (/home/chris/Code/WK/UsageAnalyticsLib/node_modules/v8-to-istanbul/lib/script.js:14:23) at module.exports (/home/chris/Code/WK/UsageAnalyticsLib/node_modules/v8-to-istanbul/index.js:4:10) at allV8Coverage.forEach (/home/chris/Code/WK/UsageAnalyticsLib/node_modules/c8/bin/c8.js:95:20) at Array.forEach (<anonymous>) at writeIstanbulFormatCoverage (/home/chris/Code/WK/UsageAnalyticsLib/node_modules/c8/bin/c8.js:94:17) at executeWithCoverage (/home/chris/Code/WK/UsageAnalyticsLib/node_modules/c8/bin/c8.js:68:5) at <anonymous> errno: -21, code: 'EISDIR', syscall: 'read' }

I was able to fix it locally in my project by putting an isFile check before creating a CovScript so it doesn't attempt to create coverage files for directories. I then forked this repo and was going to put a fix in, but it looks like 2.0.0 on NPM and the code that is latest here aren't the same. So I'm creating an issue instead. Can someone fix this and release to NPM as a patch? I'd really like to use this tool as I can finally get code coverage with tests using JSDOM.

Question: Does this work with codecov.io

Does c8 work with codecov package?

It looks like they expect the same output as istanbul from the readme: https://www.npmjs.com/package/codecov

But I'm not sure the differences in the outputs or how to get c8 to change the output.

Thanks!

[Question] How is branch coverage calculated?

• Version: 3.2.1

Just curious about how branch coverage is calculated.
Below is the output I'm interested in,

Basically, this function just checks if raw is a string array.

I think I've exercised the code enough with my tests to have gotten 100% coverage.
But it seems like c8 disagrees.

Does it expect every if statement to have a matching else statement?

And if so, is there a way for me to relax the requirement?

Add watermarks CLI flags

The report command has a watermarks option, but it cannot be specified using the CLI because no yargs option for watermarks is defined.

There probably should be --statements, --lines, --functions and --branches CLI options.

Internal: Integration tests do not properly exclude the library files

The following integration test produces a snapshot:

c8/test/integration.js

Line 26 in 3679734

it('merges reports from subprocesses together', () => {

Here is the corresponding snapshot:

c8/test/integration.js.snap

Lines 8 to 22 in 3679734

	--------------------|----------|----------|----------|----------|-------------------|
	File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
	--------------------|----------|----------|----------|----------|-------------------|
	All files | 95 | 72.22 | 0 | 95 | |
	bin | 87.76 | 62.5 | 100 | 87.76 | |
	c8.js | 87.76 | 62.5 | 100 | 87.76 | 35,39,46,47,48,49 |
	lib | 96.65 | 62.07 | 100 | 96.65 | |
	parse-args.js | 97.47 | 44.44 | 100 | 97.47 | 55,56 |
	report.js | 96.15 | 70 | 100 | 96.15 | 56,57,95,96,97 |
	test/fixtures | 95.16 | 94.12 | 0 | 95.16 | |
	async.js | 100 | 100 | 100 | 100 | |
	multiple-spawn.js | 100 | 100 | 100 | 100 | |
	normal.js | 85.71 | 75 | 0 | 85.71 | 14,15,16 |
	subprocess.js | 100 | 100 | 100 | 100 | |
	--------------------|----------|----------|----------|----------|-------------------|

Expected:

• The snapshot only contains a report for the files in the fixtures directory.

Actual:

• The snapshot contains a report with the lib files.

This causes noise in PRs. Every time the lib is changed, the snapshot must be regenerated (even if you just add whitespace or a comment).

`c8 report` doesn't work on Windows

• Version:
  3.2.1
• Platform:
  Windows 10

c8 report always outputs an empty report block on windows, no matter what include or exclude options you do (or don't) provide. Looking at it, it looks like the coverage files being produced on windows have paths like file:///E:/Github/TypeScript/built/local/tsc.js, which c8 then shortens to /E:/Github/TypeScript/built/local/tsc.js internally, which I'm guessing don't glob match (anything) correctly inside test-exclude, since that's not how windows-style paths are typically rooted (though I'm not sure - that last bit's a guess, since I'm not too familiar with the package).

Improve c8 determinism

c8 uses uuid.v4 to generate unique file names:

c8/lib/wrap.js

Line 76 in f14508e

resolve(tmpDirctory, `./${uuid.v4()}.json`),

This introduces randomness. This is bad because it prevents reproducible execution in case an issue is found. For example #14 has a bug that depends on the order in which the result files are loaded.

c8 needs unique names, not random names. Given that c8 clears its temporary directory every time it runs, a counter should be enough for uniqueness. We could also improve the output slightly by using the url of the test file. Something like:

const digits = (allV8Coverage.length - 1).toString(10).length;
for (let i = 0; i <allV8Coverage.length; i++) {
  const v8 = allV8Coverage[i];
  const baseName = path.posix.baseName(v8.url); // Not the exact code
  const fileName = `${i.toString(10).padStart(digits, "0")}-${baseName}.json`;
  // ...
}

The code above is determinist if allV8Coverage is.

Don't overrider NODE_V8_COVERAGE

Is it possible to prepopulate --temp-directory with the content of $NODE_V8_COVERAGE if set? I am trying to integrate c8 in my workflow and I really like the ability to separate execution and reporting.

• NODE_V8_COVERAGE=/tmp/cov npm test
• NODE_V8_COVERAGE=/tmp/cov c8 report

Happy to contribute id accepted.
Cheers!

Dependencies update coordination

Following #31 and discussions with @bcoe, I plan to release a coverage based on different design decisions: using spawn-wrap, the inspector API and AST. To achieve this, I need to introduce updates to some dependencies. My high-level needs are stronger encapsulation and type annotations.

The goal is not to fork the dependencies but discuss my changes and try to merge them so they can benefit to everyone. I already know the end result I'd like to have, but did a large amount of refactoring to make the libraries more maintainable. Sending these end-results in block is overwhelming to [review] (and kinda rude) so I am planning to merge the changes gradually, sending small PRs, and discussing each of them. This is a long process across multiple repos. The goal of this issue is to keep track of these various PRs.

signal-exit

• tapjs/signal-exit#53
• tapjs/signal-exit#51

foreground-child

• tapjs/foreground-child#23
• tapjs/foreground-child#24
• tapjs/foreground-child#25
• tapjs/foreground-child#30
• tapjs/foreground-child#26
• tapjs/foreground-child#27
• tapjs/foreground-child#29
• tapjs/foreground-child#31

(more to come)

spawn-wrap

• istanbuljs/spawn-wrap#78

(more to come)

Thoughts on c8 & Cie

Hi,
I am opening this issue to share my thoughts on c8and Node coverage. I spent some time in August working on Node coverage, and I have some free time now so I am back at it.

My high level goal is to leverage V8's builtin coverage tools to get coverage for ES modules (currently behind the flag --experimental-modules).

c8 API

Using the builtin tools is not easy: it requires you to know how to interact with Node's inspector, how to enable profiling and then format the raw results to a human (or computer) readable report. This is were c8 comes in: I want it to be a library that provides an easy API to work with coverage.

c8 is currently a CLI tool only, but I would like to be able to use it as a library. My use case would be to integrate it with Gulp. I think that the CLI part should be a very thin wrapper on top of the API.

The core of the API would be a function called runWithCoverage (or similar name). It spawns an instrumented process and returns a promise for the V8 coverage. Additional functions would then let me generated reports from these coverages.

Since I want to use it as a library, I have strong expectations that the execution should be deterministic and encapsulated.
Determinism is simple: I want to have reproducible results. This means for example that random values should be avoided.
For encapsulation, it means that I can call the API multiple times and these calls do not interfere with each other. I should be able to perform multiple calls sequentially or concurrently. Encapsulation also means for me that it should avoid writing temporary files to the disk. Writing files to the disk may trigger side affects if the FS is watched, it exposes the files to be removed or modified.

Coverage properties

Here are some properties regarding coverage to compare various implementations:

Covered processes

The coverage can be either applied to a single direct child process or to a whole process subtree.
The earliest versions of c8 only supported coverage for direct node child processes. Subprocesses were not covered. This is simple, but often not enough.
The main reason why we need to support process sub-tree coverage is that some test runners (like Mocha) use child processes to run the tests.

The easiest solution is to support sub-tree coverage is to have some support in the covered process and use environment variables. Most programs (shell, Node) propagate the parent environment to the child processes so this is a clean way to pass the info. The environment variable is static though. It means that the covered process must look for it. In the latest Node version, Node looks for NODE_V8_COVERAGE to enable coverage and write the results at the end. This method breaks if a child process is created in a fresh environment, but this is pretty rare in practice.

The other solution is to watch the process subtree and intercept the creation of new processes. The library spawn-wrap can be used for this. It creates a temporary shim directory with a fake node executable. By changing the PATH or rewriting the spawn calls. This solution also depends on the environment to not be reset. It also uses some temporary files so it's not ideal. Still, this solution allows the most flexibility because you can run arbitrary code and do not need any support from the covered process.

Coverage results

Once a covered process finishes, it must return its results.
The two main ways to achieve it is via communication channels or by writing the files in an agreed-upon location.

c8@2 and c8@3 uses a temporary directory where the files are written. Each process in the subtree writes a file and at the end the root process reads all the files in the directory. It's an easy solution, but it breaks encapsulation: you cannot safely run multiple coverages concurrently if there's a risk that they use the same temporary directory.

c8@1 and my version use communication channels. Namely, the inspector API. The inspected processes are started in debug mode and the root process establishes an inspector session with them through a direct connection. Using the inspector API not only allows to get the results directly, it also allows to use the inspector to request more data (for example the module type or the source code).

Comparison

c8@1

• Can only inspect a direct Node child process.
• Uses the inspector API.
• No shim directory, no temporary directory, no dependency on the environment.

This version was a good start. It offers the cleanest implementation but the fact that it only support direct sub-process means that it is unusable with many popular test runners.

c8@2

• Can inspect process subtrees
• Uses spawn-wrap@1: for each process, starts an inspector on itself, executes main, and on exit synchronously writes the results.
• The results are written in a temporary directory, the files use uuid4 for uniqueness (caused real issues with determinism)
• Requires a shim directory and a temporary results directory.

This version enables a usable CLI tool. It's pretty great. The main issues are that it had some issues when generating the final reports (but it's a seconday issue) and that it left many files around.

c8@3

• Can inspect process subtrees.
• Uses the environment variable NODE_V8_COVERAGE and Node's builtin support for coverage.
• The results are written in a temporary directory.
• No shim directory.

The main change in this version is getting rid of spawn-wrap and its shim directories by using Node's builtin support. It still uses temporary files, but it should be possible to get implement some solution just to make sure that concurrent coverage does not interfere.

My current solution

I have a working c8 version on my next branch. It currently relies on unpublished changes in other packages but I'll try to make it available as soon as possible.

• Can inspect process subtrees
• Uses a heavily patched spawn-wrap version ("spawn-wrap@2"). Allows to establish inspector sessions with any process in a subtree.
• Uses the inspector API: no temporary files, has access to extra data.
• Uses a shim directory.

It still relies unfortunately on shim directories, but it has a direct access to the inspector API for any child process. This is extremely powerful.

The ideal solution

The ideal solution would have no shims, no temporary directories and grant full access to the inspector API.
This solution would use the inspector API and Node support to detect child processes.
This requires Node support to reverse the dependency between the inspected and inspector process. Currently, the inspected process is started first and then an inspector can connect to it. In our case, we have the opposite: a single root process and many child processes. The idea would be to start an "inspector server" and then pass its port through an environment variable. Any new process would then started in "inspected mode" and establish a connection with the root process.

This kind of the mechanism is not only useful for coverage, but it could be used by IDEs to support cross-process debugging/break points.

Comments

I hoped to quickly send a PR my c8 issues and move on to run coverage on my ESM files. I ended up going through a rabbit hole of fixes. Here are some unordered comments.

foreground-child

foreground-child is the lib used to display the output of the covered process. It proxys the I/O, IPC and signals of the root process to a child process.

My first issue with this lib is that it takes over control-flow. You can pass a handler to run some code once the child process is closed. You can retake control by not calling the end callback but the API design does not encourage this behavior. It prevents using it in a lib function called multiple times.

My second issue is that the function signature kinda mimicks child_process.spawn, but not completely:

function foregroundChild(program: string | ReadonlyArray<string>, cb?: CloseHandler): ChildProcess;
function foregroundChild(program: string, args: ReadonlyArray<string>, cb?: CloseHandler): ChildProcess;
function foregroundChild(program: string, arg1: string, cb?: CloseHandler): ChildProcess;
function foregroundChild(program: string, arg1: string, arg2: string, cb?: CloseHandler): ChildProcess;
function foregroundChild(program: string, arg1: string, arg2: string, arg3: string, cb?: CloseHandler): ChildProcess;
function foregroundChild(program: string, arg1: string, arg2: string, arg3: string, arg4: string, cb?: CloseHandler): ChildProcess;

foreground-child spawns the child process itself: you don't have control over the stream or environment variables. This is the reason why c8@3 has to mess with the global environment as a workaround.

foreground-child not only spawns the child process itself, it also uses a different spawn function based on the platform. On windows it uses cross-spawn, otherwise the builtin child_process.spawn. This is a workaround around the fact that argument parsing is basically broken for processes spawned on Windows and that you can't pass your own options.

It also has an inconsistency regarding the way it proxys IPC compared to signals. It clears all the message listeners on the parent process and does not "unproxify" its listeners. On the other hand, the signals are well handled IMO by only attaching and detaching its own listeners.

I wrote some patches to address these issues. The main change was to expose an additional function to proxy an existing process instead of spawning a new one. It means that the caller can create th child process however it wants and foreground-child.proxy just sets up the various proxys.

v8-to-istanbul

v8-to-istanbul is the lib used to convert V8 coverage to c8 coverage.

At its core, it should be a simple pure transformation from a ScriptCoverage (or FunctionCoverage[]) to an Istanbul file coverage.

v8-to-istanbul reads the files from disk itself. I don't think it should be it's responsibility. It plays badly with mocked FS (such as Gulp's Vinyl) and brings asynchronicity or blocking calls when they are not that needed.

The current version returns an istanbul report (many scripts) when converting V8 coverage for a single script. There is a mismatch.

For commonJS, there is a mismatch between the files on disk and the files evaluated by V8 due to the CJS wrapper (another reason IMO why it shouldn't read the files directly). Some users change the value of this wrapper, for example to inject "use strict" to every module. It would be nice to be able to control this behavior.

My main issue is that it uses a line-based approximation to get the number of covered functions, processes and branches. It should use syntax analysis to get proper results. This IMO the reason why I had issues with c8 (wrong results).

In my patches, I refactored v8-to-istanbul to focus on the conversion using Babel's parser for syntax analysis. Using my version I am able to get the exact same results as nyc for the fixtures in c8. It properly handles multiline statements, all kinds of functions and simple branches (I still need to add code for some of the branches like switch/case or logical operators).

spawn-wrap

Ok, this is a hack. But it can be pretty powerful.

First issue: it globally patches Node's internals. This immediately rules out any sort of concurrent calls involving child processes. You should be able to create independent wrapped instances of child_process.

The logic to rewrite the intercepted spawn calls is a bit brittle. It relies directly mutates the options and writes some internal values on it. It also uses string concatenation for path manipulation (so it has some bad results for windows). There also some issues around the handling of env variables on Windows.

The wrapper is statically configured. You can enforce some arguments and environment variables, but cannot change them based on the process being spawned (or generate a different value for each child process).

By looking at all the dependents on npm, it seems that the API does not really match the need it solves. The need is to spawn some custom module before the real main and pass some data to this custom modules. Using args and env vars is a bit contrived.

I did some major clean up on spawn-wrap. It still needs a shim directory, but appart from that I think it's pretty good. It solves the issues above. It also adds some helpers to handle the lifetime shim directory in a stack-based way.

c8

This message is already long so I won't go into the details. I just moved most of the logic to the lib part so the the cli script is just a small wrapper. I refactor the argument parsing/configuration detection so you can use it in a library. I also updated it to use my patched versions of the other libraries.
(Obviously, it gained a first class lib API)

Types

One of the main pain points when working with all these various libraries was documentation. The public API had a description in the README, but the internals had nothing. It means that finding out the real signature of foreground-child, or what are the expected fields in an Istanbul source location, or what are invariants involved in spawn-wrap's transformations was a pain.
I wrote type definitions/converted to Typescript most the libs I touched. Yeah, it adds some transpilation facilities and static checker, but the main benefit is that it pushes you to write a mostly unambiguous documentation for your functions. At the size of c8, involving all these dependencies, it pays off.

What next

I worked on an other projects in the last weeks of August/first weeks of September and couldn't submit all the PRs I had in mind. I wanted to finish them together to be sure that they all play nice with each other.
In the meantime, Node got basic builtin support for coverage and c8 was updated. It's great, but I still think that there is room for improvement.
In the long term, I truly believe that Node needs a general way to inspect subprocesses, for coverage and debuggers.
Before that, Node's builtin coverage and my lib both have value and tradeoffs. Even the c8@1 version supporting only direct child processes is valuable: it's most reliable self-contained solution if you only need to inspect a single process.
I think a good way would be to have all 3 versions available in the lib for the moment and I'd like to work on a merge request for this:

• Direct node subprocess with --inspect=0 and the inspector API
• NODE_V8_COVERAGE with temporary files
• "spawn-wrap@2" with the shim directory but no temporary files (inspector API)

Even before that, I'll work on closing my PRs for foreground-child and v8-to-istanbul.

process.exit breaks coverage

If you have process.exit in a file, it doesn't show any coverage.

Repro is simple:

index.js

// process.exit() // uncomment me

Run c8 node index.js

• Without process.exit()

  ----------|----------|----------|----------|----------|----------------|
  File      |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
  ----------|----------|----------|----------|----------|----------------|
  All files |      100 |      100 |      100 |      100 |                |
   index.js |      100 |      100 |      100 |      100 |                |
  ----------|----------|----------|----------|----------|----------------|
  

• With process.exit()

  ----------|----------|----------|----------|----------|----------------|
  File      |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
  ----------|----------|----------|----------|----------|----------------|
  All files |  Unknown |  Unknown |  Unknown |  Unknown |                |
  ----------|----------|----------|----------|----------|----------------|
  

Add command line option to turn off the outputReport

Node.js: 12.4.0
c8: 5.0.1
Platform: Linux eagle 5.1.7-300.fc30.x86_64 #1 SMP Wed Jun 5 12:32:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

My test suite runs child processes and the decision as to whether to run them through c8 is passed along as a command line argument. The child processes output a JSON object which is parsed from their stdout to assert correctness. The unconditional production of c8's outputReport() interferes with the JSON.parse() of the child process' stdout.

Commenting out these lines is a workaround. It'd be nice if they could be turned off with a command line parameter.

chore(release): proposal for next release

🤖 Here's what the next release of c8 would look like.

---

4.1.6 (2019-05-18)

---

• Should I create this release for you 🤖?

Incorrect coverage reported for ternaries in some situations

• Version: 11.9.0
• Platform: Darwin Fhqwhgads.local 17.7.0 Darwin Kernel Version 17.7.0: Thu Dec 20 21:47:19 PST 2018; root:xnu-4570.71.22~1/RELEASE_X86_64 x86_64

In file foo.js:

'use strict';

function testIt (a, b) {
 const thing = a && b;
 console.log(thing);
 return a && b;
}

if (testIt(true, false))
  console.log('foo');

I run this:

c8 node foo.js

I get this:

false
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |      100 |       60 |      100 |      100 |                   |
 foo.js   |      100 |       60 |      100 |      100 |              6,10 |
----------|----------|----------|----------|----------|-------------------|

I would have expected the coverage for line 4 (the assignment to thing) and line 6 (the return statement) to be the same. Instead, it is (correctly) reporting that 6 has some uncovered branches, but is omitting the same thing for line 4.

throw results in an uncovered line

• Version: v11.7.0
• Platform: Linux lt2 4.19.8-200.fc28.x86_64 #1 SMP Mon Dec 10 15:43:40 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

function testThrow(doThrow) {
	if (doThrow) {
		throw new Error('test throw');
	}
}

testThrow(true);

Run the above under c8, it reports that line 4 is not covered. Really since line 4 is only a } I think it should not even count, having no else statement means that line is non-significant. IMO significance of lines comment applies to lines 5 and 6 as well, the } of a function does not have any actual code. Line 6 is blank so that should not be counted.

nyc basically works the way I expect, though the function testThrow(doThrow) { line isn't part of the total or covered lines. I don't have a strong opinion about counting / not counting the function declaration as a line.

[Edit] no coverage when running on Alpine

I'm trying to upgrade from v2 to v3.2 and I'm not getting any coverage files generated anymore. With v2, when I run my mocha tests through c8, coverage files are generated in the set temp directory and then a report is generated in ./coverage. With v3.2, no coverage files are generated in the temp directory (which is what NODE_V8_COVERAGE is set to).

This is what Node's docs say about NODE_V8_COVERAGE:

NODE_V8_COVERAGE will automatically propagate to subprocesses, making it easier to instrument applications that call the child_process.spawn() family of functions. NODE_V8_COVERAGE can be set to an empty string, to prevent propagation.

At this time coverage is only collected in the main thread and will not be output for code executed by worker threads.

I would think mocha would work within those conditions, but something isn't working properly. I don't know anything about the internals of mocha, but is it possible they don't run on the "main thread"? I'll try looking in to it, but I just wanted to point this out. I've been getting unit test code coverage with c8 v2 (with a few issues, but still generally decent reports except the function reports), but with v3.2, nothing. I'll have to stick with v2 for now. If you know anything about what may be going on, please let me know. Thanks.

Function definition after the return statement

• Version: v11.10.0
• Platform: Windows 7 x64

1. Create foo.js:

   function test(foo = "foo") {
     return {bar};
     
     function bar() {
       console.log("test");
     }
   }
   
   test().bar();

2. Run c8 --reporter html node foo.js
3. Line 3, 4 are marked as uncovered:
   

It works fine if test has no default argument:

c8 v3.4.0

Code Coverage Report is Empty even will valid coverage/tmp/xxxxx.json generated...

• Version:
• Platform:

Run a c8 Code Coverage session with my Node API Project with this Command Line:
c8 -r html -x node_modules node server.js

I hit a few APIs and then cleanly terminate the Server.

coverage/tmp/coverage-8140-1545948692773-0.json gets created and its Coverage is as expected.

But the HTML report generated by Istanbul is essentially empty. Has the headings, but nothing is really there.

I can manually parse the file and create my own report out of it, but I don't really want to do that.

THANKS for c8... Once I get it working, it will truly change how I look at Testing...

Here is a snippet below from the Coverage JSON file:

{
"scriptId": "453",
"url": "file:///C:/Users/tsukiennik/terraPoint/terraPoint/rapierWorkflow/lib/rapierUtils.js",
"functions": [{
"functionName": "",
"ranges": [{
"startOffset": 0,
"endOffset": 41264,
"count": 1
}],
"isBlockCoverage": true
}, {
"functionName": "",
"ranges": [{
"startOffset": 1,
"endOffset": 41262,
"count": 1
}],
"isBlockCoverage": true
}, {
"functionName": "exports.setRunnerMode",
"ranges": [{
"startOffset": 1371,
"endOffset": 2297,
"count": 4
}, {

Question: how does --include work

• Version: v10.15.2

Hi, I have a question about --include parameter. I have tried to target a specific file (--include ["src/api/foo.ts"]) and to use a glob pattern (--include ["src/**"]) but the only thing I got was a completely empty report. Am I using this parameter correctly?

Btw I run tests with ts-node (I guess that could be important. Even though the basic c8 usage works perfectly :)

Thanks a lot!

Feature Request: Significance detection

c8 currently does not detect significant / insignificant lines. Some examples are lines that only contain:

• hashbang
• white-space
• comments
• block closing brackets
• closing parenthesis
• semi-colon / comma

The inclusion of these lines in results artificially inflates coverage percentages. Silly example for demo purposes:

#!/usr/bin/env node
/*
 * This file has a large comment.
 * This file has a large comment.
 * This file has a large comment.
 * This file has a large comment.
 * This file has a large comment.
 * This file has a large comment.
 * This file has a large comment.
 * This file has a large comment.
 */
if (false) {
  console.log('This is not hit');
  console.log('Neither is this');
}

c8 reports 80% of the lines covered (green), nyc reports 33% (red).

Incorrect coverage reporting with throw in a block

• Version: 11.9.0
• Platform: Darwin LIB-0F7FVH8-LT 18.2.0 Darwin Kernel Version 18.2.0: Mon Nov 12 20:24:46 PST 2018; root:xnu-4903.231.4~2/RELEASE_X86_64 x86_64

In file foo.js:

if (true) {
  throw new Error('foo');
}

I run this:

c8 node foo.js 

I get this output:

/Users/trott/io.js/foo.js:2
  throw new Error('foo');
  ^

Error: foo
    at Object.<anonymous> (/Users/trott/io.js/foo.js:2:9)
    at Module._compile (internal/modules/cjs/loader.js:734:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:745:10)
    at Module.load (internal/modules/cjs/loader.js:626:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:566:12)
    at Function.Module._load (internal/modules/cjs/loader.js:558:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:797:12)
    at executeUserCode (internal/bootstrap/node.js:526:15)
    at startMainThreadExecution (internal/bootstrap/node.js:439:3)
----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |       50 |    66.67 |      100 |       50 |                   |
 foo.js   |       50 |    66.67 |      100 |       50 |               3,4 |
----------|----------|----------|----------|----------|-------------------|

What I expected:

I would not expect there to be a line 4 mentioned in the coverage report because the script is three lines long.
I would not expect line 3 to be mentioned as uncovered because there is nothing executable on it.
I would expect Lines to be covered 100% (but not Branch).

Report directory can't be configured by CLI options

Version: v10.15.1
Platform: 4.18.0-14-generic #15-Ubuntu SMP Mon Jan 14 09:01:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Hello, @bcoe!

Thank you for so awesome tool.

I have tried c8 and it satisfies all my needs except one.
I have to generate two reports (one for unit tests coverage, other - for functional tests coverage). But the output directory is hardcoded for each run (see this line). So i am not able to collect artifacts on my CI for both: unit tests and functional tests. This is a very minor issue and if you don't mind, i will try to fix it and create a PR.

Best regards
shonie

Unable to get report for a node server

• Version:node v8.11.1
• Platform:Ubuntu 16.04LTS

I am trying to use c8 on a node server. I start the server like this $ c8 node app.js . After this I run my BDD tests on this server. Once I am done, I will stop the server by Ctrl+C. But when I run $ c8 report, I do not see any report.

What am I missing here? Please help.

Context: I am running the node server on a docker and BDD is initiated from another BDD docker. So I am looking for a way to generate code coverage report without modifying the server code base much.

Configuration: quality percentage

• Version: latest
• Platform: Mac

Is it possible to configure percentage of test coverage, and if it's lower than this value process fails

Does not report coverage for files with names "test-*" or "test.*"

• node v10.15.3:
• [email protected]:
• WSL on windows 10 (Ubuntu 16.04.5 LTS):

Does not report coverage for files with names "test-" or "test."

for example c8 node ./test-cover1.js and c8 node ./test.js does not work, but c8 node ./testcover1.js does work

Travis badge is green, however build log indicates a failure

The travis badge on the readme is green, which makes you think the build is solid, but the actual log indicates an early failure from an uncaught promise rejection (which does not currently trigger a nonzero exit code)!