nadeesha / saul

Experimental annotation-based javascript unit tests :rocket:

What is it?

saul gives you a custom DSL that will help you write test framework agnostic unit tests for your javascript functions.

A simple example might look like:

// @t "should call saul when the threat is imminent"        shouldCallSaul('imminent') ~equals true
// @t "should not call saul when threat is not imminent"    shouldCallSaul('nodanger') ~equals false
function shouldCallSaul(threatLevel) {
    if (threatLevel === 'imminent') {
        return true;
    }

    return false;
}

What problems does it solve?

• Avoid writing unnecessary boilerplate code for trivial tests
• Quickly test functionality with // @t annotations in your code
• Have your tests co-located to the functionality it tests
• Self-document your functionality with a custom DSL

Installation

1. Install saul as a dev dependency:

yarn add --dev saul

2. Create a .saulrc in the root.

example:

{
    "fileGlob": "src/**/*.js",                      // files that contain the saul comments
    "customEnginesDir": "./src/custom-saul-engines" // optional: dir where you will put custom engine .js files
}

3. Invoke saul from your test.

Mocha/Jasmine

If you have some mocha tests already, your npm test would look like: mocha src/**/.js. Simple add saul's bin (node_modules/.bin/saul) right at the end:

mocha lib/*.test.js" node_modules/.bin/saul

Jest

Since jest requires a regex pattern for test files, you will have to create a file with a single file with a require, that will be matched by your jest regexPattern.

Example:

require('saul'); // will run all saul tests here

Usage with Babel

Any transformation that you apply to your tests will be inherited by saul when you run your tests. If you're running babel, this will include anything that you define in your local .babelrc.

For an instance, if you want to feed babel-transformed files to mocha, you will invoke mocha with mocha --compilers js:babel-register. You can simply add saul to the end of the command. (mocha --compilers js:babel-register node_modules/.bin/saul) - and things will Just Work™.

DSL Specification and Examples

expect

Assert the result using chai's expect. Comes with test spy support from sinon.

Example:

// @t "appends foo" appendFoo('bar') ~expect expect(result).to.contain('foo');
// @t "has no fizz" appendFoo('bar') ~expect expect(result).to.not.contain('fizz');
export function appendFoo (someString) {
    return someString + 'foo';
}

With spy support:

Calling spy(name: string), will create a sinon spy. You can assert on any of it's methods/properties like this:

// @t "calls only once"   testEvalSpy(spy('mySpy')) ~expect spy('mySpy').calledOnce
// @t "calls with obj"    testEvalSpy(spy('mySpy2'), 'foo') ~expect spy('mySpy2').calledWith('foo')
export function testEvalSpy (fn, str) {
  fn('foo', str);
}

matches-snapshot

Checks whether a previously saved snapshot image of the function's serialized output, matches the current output. (Saves a snapshot file on the first run - that should be checked in to the repo).

// @t "should render Date" Date({dateString: '1970-03-11'}) ~matches-snapshot
export function Date(props) {
    return <div className={'mydate'}>{props.dateString}</div>
}

// @t "returns all months" getAllMonths() ~matches-snapshot
export function getAllMonths() {
    return CONSTANTS.ALL_MONTHS.join('_');
}

equals

Checks whether the expected value is equal to the actual value. If the function returns a promise, resolves it before asserting

// @t "can sum" sum(1, 2) ~equals 3
export function sum(numOne, numTwo) {
    return numOne + numTwo;
}

// @t "testEqualsAsync" testEqualsAsync() ~equals 'foo'
export function testEqualsAsync() {
  return new Promise((resolve, reject) => {
    resolve('foo');
  });
}

contains

Checks whether the output contains the expected value.

Example:

// @t "can concat" concatanate('string1', 'something els') ~contains 'string1'
export function concatanate (a, b) {
    return a + b;
}

is-not

Checks whether the expected value is not equal to the actual value. (Opposite of equals)

// @t "can sum" sum(1, 2) ~is-not 4
export function sum(numOne, numTwo) {
    return numOne + numTwo;
}

throws

Checks whether the invokation would throw.

// @t "throws on null engine" executeTest({engine: null}) ~throws Error
export executeTest(options) {
    options.engine('foobar');
}

And more! See: extending saul.

Extending saul

Then engines are the "comparator" in the tests.

// @t "throws on null engine" executeTest({engine: null}) ~throws Error
                                      |                      |      └ expected value
                                      |                      |
                                      |                      └ comparator
                                      |
                                      └ actutal value

They are handled by the file of that name in src/engines/. (Example: src/engines/throws.js)

The "engines", are responsible for generating the tests. So, as long as you build a custom engine - it can pretty much test anything.

The default engines can do a few cool things out of the box. (check the src/engines/ directory). You can always write your own engines and put them in your customEnginesDir defined in .saulrc.

Examples

Just look through this repo for // @t annotated tests. saul is tested with saul! 🚀

Contributions

Please! Here are som TODOs that need being done.

• More engines! (If you would like to contribute an engine, please take a look at the engine files at src/engines)
• Documentation on writing engines.
• Extending the parsers for fixtures
• Better error handling for engines
• Tests for existing engines

saul's Issues

test async functions

Hi, I've been using saul to test my code, it's really fantastic.
However when testing with asyn functions, I wrote an test engine like this:

const expect = require('chai').expect;

module.exports = (
  testDescription,
  func,
  argsArray,
  expected,
  test
) => {
  test(testDescription, () => {
    func.apply(null, argsArray)
      .then(actual => {
        expect(actual).to.eql(eval(`var trouble = ${expected}; trouble`));
      });
  });
};

It suppose to act like deep-equals with function returns promise, but it's not working and it did not throw any error.
Could you help me with this? Thanks :)

Usage with Jest

Hey, this looks really good, trying to use it with an exisiting project which uses Jest but am unsure how to make that work, or if it does atm.

In a file in ./src:

...
// @t "should add numbers" add(1, 2) equals 3
export const add = (a, b) => a + b
...

.saulrc:

{
    "fileGlob": "src/**/*.js"
}

scripts in package.json:

...
"test": "NODE_ENV=test jest --config jest.config.json --verbose node_modules/.bin/saul",
...

outcome:

❯ npm t

> [email protected] test /Users/kav/Sites/playground/sauling
> NODE_ENV=test jest --config jest.config.json --verbose node_modules/.bin/saul

No tests found
In /Users/kav/Sites/playground/sauling
  10 files checked.
  testMatch:  - 10 matches
  testPathIgnorePatterns: /node_modules/ - 10 matches
  testRegex: \.test\.js$ - 1 match
Pattern: "node_modules/.bin/saul" - 0 matches

so it seems the jest cli uses that last bit of the command to look for matching tests. Same thing happens if i take node_modules/.bin/saul out of the npm script and run npm t node_modules/.bin/saul. npm t | node_modules/.bin/saul hail mary is all errors:

/Users/kav/Sites/playground/sauling/node_modules/saul/lib/testcase.js:6
var suite = exports.suite = describe;
                            ^

ReferenceError: describe is not defined

Is anything in the Jest config that can be set up for it to work?

Thanks heaps!

Describe is not defined

Hi,

When I want to start and watch the application for contributing, I receive an error with the following message:

var suite = exports.suite = describe; // eslint-disable-line no-undef

ReferenceError: describe is not defined

Would you please help me in solving this problem in development mode?

[RESOLVED] Current version not working

I've done as the readme says, but it does not test any file.

Results are like these:

[email protected] test /home/xinbg/Documents/pincloud.supe
nyc mocha src/**/*.test.js node_modules/.bin/saul

Warning: Could not find any test files matching pattern: src/**/*.test.js

0 passing (1ms)

----------|----------|----------|----------|----------|----------------|

File	% Stmts	% Branch	% Funcs	% Lines	Uncovered Lines
All files	Unknown	Unknown	Unknown	Unknown
----------	----------	----------	----------	----------	----------------

---

Also wondering how to exclude node_modules folder?