Coder Social home page Coder Social logo

babel-plugin-transform-adana's Introduction

babel-plugin-transform-adana

Minimal, complete code-coverage tool for babel 6+.

build status coverage license version downloads

Has all the features (and more) of istanbul including line, function and branch coverage, but works as a babel plugin instead of relying on esparse and escodegen. Works great with west, mocha, jasmine and probably more.

Features:

  • First-class babel support,
  • Per-line/function/branch coverage,
  • Tagged instrumentation,
  • User-defined tags,
  • Smart branch detection.

Usage

Install babel-plugin-transform-adana:

npm install --save-dev babel-plugin-transform-adana

Setup your .babelrc to use it:

{
  "env": {
    "test": {
      "plugins": [[
        "transform-adana", {
          "ignore": "test/**/*"
        }
      ]]
    }
  }
}

IMPORTANT: This plugin works best when it runs as the first plugin in the babel transform list, since its purpose is to instrument your original code, not whatever other transformations happen to get made.

NOTE: This plugin is only responsible for instrumenting your code, not verifying the coverage information or reporting. You can install something like adana-cli to get something like instanbul check-coverage. See the adana-cli repository for more information.

mocha

Usage with mocha is straight-forward. The only thing you need to do after running your code is dump the coverage information to disk so it can be processed; mocha can do this via its -r flag.

Install the necessary packages:

npm install --save-dev \
  mocha \
  adana-cli \
  adana-dump \
  adana-format-lcov \
  babel-plugin-transform-adana

Start testing with mocha:

#!/bin/sh

# Run tests and dump the coverage information.
NODE_ENV="test" mocha \
  -r adana-dump \
  -r @babel/register \
  test/*.spec.js

# Upload coverage data to coveralls.
cat ./coverage/coverage.json \
  | ./node_modules/.bin/adana --format lcov \
  | ./node_modules/coveralls/bin/coveralls.js

jasmine

Usage with jasmine is less straight-forward than with mocha since there is no native babel support. The package jasmine-es6 can be used to use babel (and therefore adana) with jasmine.

Install the necessary packages:

npm install --save-dev \
  jasmine-es6 \
  adana-cli \
  adana-dump \
  adana-format-lcov \
  babel-plugin-transform-adana

Add the output tool as a helper to jasmine via jasmine.json in order to ensure your coverage data gets output:

{
  "spec_dir": "spec",
  "spec_files": [
    "**/*[sS]pec.js"
  ],
  "helpers": [
    "../node_modules/jasmine-es6/lib/install.js",
    "../node_modules/adana-dump/index.js",
    "helpers/**/*.js"
  ]
}

Start testing with jasmine:

#!/bin/sh
NODE_ENV="test" jasmine

# Upload coverage data to coveralls.
cat ./coverage/coverage.json \
  | ./node_modules/.bin/adana --format lcov \
  | ./node_modules/coveralls/bin/coveralls.js

west

TODO: Write me!

Tags

There is no ignore flag, but you can tag functions, branches or statements which can be used to determine relevant coverage information. This allows you to ask things like "Have I covered all the code that pertains to authentication in the file?" and "Has this run in IE covered all the IE-specific cases?". Existing ignore comments simply tag a function with the ignore tag.

  • Add a tag with +tag, remove a tag with -tag.
  • Tags above a function declaration apply to all code in that function.
  • Tags above a class declaration apply to all code in that class.
  • Tags before the first statement of a branch apply to the branch and its code.
  • Tags on a line apply to everything on that line.
/* adana: +ie +firefox -chrome */
function foo(i) {
  ++i; // +chrome
  console.log('foo', i); // adana: +test
  return i;
}


if (foo(1)) {
  /* adana: +chrome */
  console.log('bar');
}

FAQ

  • Why is let i;, function foo() {}, etc. not marked at all? โ€“ Some things are not executable code per se (i.e. declarations). They do nothing to effect program state and are therefore not instrumented.

Configuration

There are a couple of configuration options available to control how your program is instrumented. They can be set via the standard mechanism babel employs for configuring transforms.

{
  // Pattern to match to determine if the file should be covered. The pattern
  // must be matched for coverage to be enabled for the file. Takes precedence
  // over `ignore`.
  // See `only` of https://babeljs.io/docs/usage/options/
  only: 'src/**/*.js',
  // Pattern to match to determine if the file should NOT be covered. The
  // pattern must NOT be matched for coverage to be enabled for the file.
  // See `ignore` of https://babeljs.io/docs/usage/options/
  ignore: 'test/**/*',
  // Name of the global variable to store all the collected coverage information
  // in.
  global: '__coverage__'
}

API

Again, this plugin is simply a babel transformer that injects markers to determine if specific parts of the code have been run. Usage is as a normal babel plugin:

import {transform} from '@babel/core';

const result = transform('some code', {
  plugins: ['transform-adana']
});

// Access result.code, result.map and result.metadata.coverage

To collect information about code that has been instrumented, simply access the configured global variable, e.g. __coverage__.

import vm from 'vm';
const sandbox = vm.createContext({});
sandbox.global = sandbox;
vm.runInContext(result.code, sandbox);
console.log(sandbox.__coverage__);

The __coverage__ object has the following shape:

{
  // Hash of the file.
  hash: '2892834823482374234234235',
  // Path to the file being instrumented.
  path: 'some/file.js',
  // Detailed information about every location that's been instrumented.
  locations: [{
    id: 0,
    loc: { start: { line: 0, column 0 }, end: { line: 0, column: 0 } },
    name: 'foo',
    group: 'bar',
    tags: [ 'tagA', 'tagB' ],
    count: 5
  }, {
    ...
  }, ...]
}

More useful processing of this object can be done with adana-analyze.

babel-plugin-transform-adana's People

Contributors

izaakschroeder avatar olegskl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

babel-plugin-transform-adana's Issues

Shorthand methods aren't considered functions

I've been playing around with adana-analyze and noticed that I get less coverage than with my previous babel5+isparta setup. Apparently, shorthand methods aren't considered functions:

{
  foo(a, b) { // this is not treated as a function
    return a + b; // this is not covered
  }
}

Error thrown on quoted object key

Given the following .babelrc file:

{
  "presets": ["es2015"],
  "plugins": ["transform-adana"]
}

When there's an object declaration in a js file with a key surrounded by quotes:

export default {
  foo: 'bar', // this line is ok
  'baz': 'qux' // this line fails
};

Then Babel throws:

TypeError: foo.js: Property key of ObjectProperty expected node to be of a type ["Identifier","StringLiteral","NumericLiteral"] but instead got "SequenceExpression"
at validate (/Users/olegsklyanchuk/repos/test/adana-test/node_modules/babel-types/lib/definitions/index.js:101:13)
at Object.validate (/Users/olegsklyanchuk/repos/test/adana-test/node_modules/babel-types/lib/definitions/core.js:506:63)
at Object.validate (/Users/olegsklyanchuk/repos/test/adana-test/node_modules/babel-types/lib/index.js:269:9)
at NodePath._replaceWith (/Users/olegsklyanchuk/repos/test/adana-test/node_modules/babel-traverse/lib/path/replacement.js:201:7)
at NodePath.replaceWith (/Users/olegsklyanchuk/repos/test/adana-test/node_modules/babel-traverse/lib/path/replacement.js:179:8)
at instrument (/Users/olegsklyanchuk/repos/test/adana-test/node_modules/babel-plugin-transform-adana/dist/instrumenter.js:156:12)
at visitObjectProperty (/Users/olegsklyanchuk/repos/test/adana-test/node_modules/babel-plugin-transform-adana/dist/instrumenter.js:353:9)
at PluginPass. (/Users/olegsklyanchuk/repos/test/adana-test/node_modules/babel-plugin-transform-adana/dist/instrumenter.js:84:39)
at NodePath._call (/Users/olegsklyanchuk/repos/test/adana-test/node_modules/babel-traverse/lib/path/context.js:74:18)
at NodePath.call (/Users/olegsklyanchuk/repos/test/adana-test/node_modules/babel-traverse/lib/path/context.js:46:17)

Support tag comments.

Allow user-defined tags.

e.g.

// The following single line has all matching AST nodes tagged with `foo`
let i = 0; // adana: +foo

// Everything between `+foo` and `-foo` is tagged with `foo`.
/* adana: +foo */

/* adana: -foo */

Think about:

  • Block tags? Might be a nice convenience.
  • Branch tags? These are absolutely necessary.

Increase accuracy of line count algorithm.

Ultimately a (more sane) choice has to be made about how lines are counted. The line count is simply a computed result from things with the line tag. I believe this to be the correct approach, seeing as it is the simplest and most flexible in terms of the instrumenter; and the data as it stands now isn't the worst, but it could be more accurate.

Support HMR-style coverage.

HMR Spec Coverage

foo.spec.js -> foo.js: overwrite coverage info only for foo.js

On HMR:

  • clear coverage counters for reloaded modules
  • run reloaded code
  • replace coverage counters for reloaded modules

Other Notes:

// TODO: Allow "merging" of same-hash coverage where the counters are
// incremented. This could be for someone who has to run a program
// several times for the same files to cover everything.

Cannot read property 'start' of undefined

\D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-core\lib\transformation\file\index.js:520
      throw err;
      ^

TypeError: D:/Users/Adriaan/Workspace/Skedify/skedify-plugin/tests/test_helper.js: Cannot read property 'start' of undefined
    at PluginPass.visitConditional (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-plugin-transform-adana\dist\instrumenter.js:327:39)
    at D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-traverse\lib\visitors.js:271:19
    at NodePath._call (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-traverse\lib\path\context.js:72:18)
    at NodePath.call (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-traverse\lib\path\context.js:44:17)
    at NodePath.visit (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-traverse\lib\path\context.js:102:12)
    at TraversalContext.visitQueue (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-traverse\lib\context.js:151:16)
    at TraversalContext.visitMultiple (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-traverse\lib\context.js:106:17)
    at TraversalContext.visit (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-traverse\lib\context.js:193:19)
    at Function.traverse.node (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-traverse\lib\index.js:139:17)
    at NodePath.visit (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-traverse\lib\path\context.js:106:22)
    at TraversalContext.visitQueue (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-traverse\lib\context.js:151:16)
    at TraversalContext.visitSingle (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-traverse\lib\context.js:111:19)
    at TraversalContext.visit (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-traverse\lib\context.js:195:19)
    at Function.traverse.node (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-traverse\lib\index.js:139:17)
    at Object.traverse [as default] (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-traverse\lib\index.js:61:12)
    at File.transform (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-core\lib\transformation\file\index.js:475:31)
    at D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-core\lib\transformation\pipeline.js:46:19
    at File.wrap (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-core\lib\transformation\file\index.js:488:16)
    at Pipeline.transform (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-core\lib\transformation\pipeline.js:43:17)
    at Object.transformFileSync (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-core\lib\api\node.js:113:10)
    at compile (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-core\lib\api\register\node.js:111:20)
    at loader (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-core\lib\api\register\node.js:136:14)
    at Object.require.extensions.(anonymous function) [as .js] (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\babel-core\lib\api\register\node.js:146:7)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\mocha\bin\_mocha:304:3
    at Array.forEach (native)
    at Object.<anonymous> (D:\Users\Adriaan\Workspace\Skedify\skedify-plugin\node_modules\mocha\bin\_mocha:303:10)
    at Module._compile (module.js:434:26)
    at Object.Module._extensions..js (module.js:452:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:475:10)
    at startup (node.js:118:18)
    at node.js:952:3

npm ERR! Windows_NT 10.0.10240
npm ERR! argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\Adriaan\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js" "run" "test" "--" "--watch"
npm ERR! node v4.1.2
npm ERR! npm  v3.3.6
npm ERR! code ELIFECYCLE
npm ERR! [email protected] test: `mocha --compilers js:babel-core/register --require ./tests/test_helper.js "tests/**/*.js" "--watch"`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] test script 'mocha --compilers js:babel-core/register --require ./tests/test_helper.js "tests/**/*.js" "--watch"'.

Determine roadmap with regards to isparta/istanbul.

Pretty open to porting existing features from istanbul/isparta over. Obviously this project is nowhere near as mature, but maybe it's a good base to build new stuff on, especially with regards to instrumenting ES6 code. Pretty happy to have input here. Would be happy to have you guys on board! ๐Ÿ˜„

/cc @gotwarlost

Coverage and line counting issues.

I'm continuing working on the HTML reporter and starting to get visual representation of our code coverage. There are some issues, mostly due to the fact that the lines function in adana-analyze reports all lines between loc.start and loc.end as covered with coverage count of the parent node. This not only results in empty lines having coverage count, but also in the fact that empty lines have higher count than their neighboring lines with statements and the such (compare lines 18 and 19).

original

So, I changed the line counting loop to only include loc.start and use Math.max on counts. This solves two issues:

  1. The lines reported as covered are actually covered.
  2. Lines with function assignments actually show the count of function declaration, not the count of function executions. Same goes for a single line ternary expressions. If the falsy branch of a ternary is not covered, that doesn't mean that the ternary is not executed at all.

Below is the screenshot of the result. It's only a little bit better.

  1. I don't know what to do with functions. On one hand I think it's good that we're tracking the execution count for the "function" tag, but the visual representation is weird (see line 2).
  2. Method calls don't seem to be covered.
  3. Something wonky is going on with conditionals (see lines 11 and 13).

start-loc-only

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.