Coder Social home page Coder Social logo

lastmjs / guesswork Goto Github PK

View Code? Open in Web Editor NEW
3.0 2.0 0.0 314 KB

Framework for property-based testing in JavaScript, TypeScript, JSX, and TSX.

License: MIT License

TypeScript 75.63% JavaScript 23.72% HTML 0.65%
test-runner test-framework javascript headless-tests jsverify quickcheck karma browser-testing typescript zwitterion

guesswork's Introduction

CircleCI npm version dependency Status devDependency Status

Guesswork

Framework for property-based testing in JavaScript, TypeScript, JSX, and TSX. Uses Karma for automating browser runs and output. Uses JSVerify for property-based testing utilities in JavaScript, similar to what QuickCheck does in Haskell. Uses Zwitterion to seamlessly transpile source code on the fly. Can be run from a terminal or the web GUI. The web GUI allows fine-grained control over which tests to run and how many random inputs to generate. The terminal allows for automatic runs of your test suite.

Live Example

Check out the live example!

Installation

npm install guesswork

Use

Headless Runs

Headless runs will automatically execute all of your tests from the --entry file with 100 iterations each.

Run headless from the terminal:

node_modules/.bin/guesswork chromium firefox safari electron --entry test/index.js

Run headless from an npm script:

// package.json
{
  "scripts": {
    "test": "guesswork chromium firefox safari electron --entry test/index.js"
  }
}

Web GUI Runs

Web GUI runs will open up a port on localhost. You can go to that port in any browser and have fine-grained manual control over which tests to run and how many iterations of random inputs should be generated. To get the port to open, just leave out any browsers from the command line arguments.

Run the web GUI from the terminal:

node_modules/.bin/guesswork --entry test/index.js

Run the web GUI from an npm script:

// package.json
{
  "scripts": {
    "test-gui": "guesswork --entry test/index.js"
  }
}

Entry File

The entry point to your tests should be a JavaScript file. When you instruct the tests to execute, the file will be loaded into the browser as an ES Module. You can import all of your test suites into the entry file using ES Modules. You should also import the test-runner custom element from Guesswork. Once you have loaded all of your dependencies, write to the DOM and create your full test suite by inserting each of your test suite custom elements as children of your test-runner element:

// test/index.js

import './test-suite-1.js';
import './test-suite-2.ts';
import './test-suite-3.jsx';
import './test-suite-4.tsx';
import '../node_modules/guesswork/test-runner.ts';

window.document.body.innerHTML = `
    <test-runner>
        <test-suite-1></test-suite-1>
        <test-suite-2></test-suite-2>
        <test-suite-3></test-suite-3>
        <test-suite-4></test-suite-4>
    </test-runner>
`;

Check out the live example!

Test Suites

Each test suite is created as an HTML custom element. You must define a prepareTests method on the class of your custom element. That function has one parameter, which is the Guesswork test preparation function. This function is used to create individual test cases. Here is an example test suite:

// test/test-suite-1.js

import jsverify from 'jsverify-es-module';

class TestSuite1 extends HTMLElement {
  prepareTests(test) {
    test('Addition is commutative', [jsverify.integer, jsverify.integer], (arbInt1, arbInt2) => {
      return arbInt1 + arbInt2 === arbInt2 + arbInt1;
    });

    test('Addition is associative', [jsverify.integer, jsverify.integer, jsverify.integer], (arbInt1, arbInt2, arbInt3) => {
      return (arbInt1 + arbInt2) + arbInt3 === arbInt1 + (arbInt2 + arbInt3);
    });
  }
}

window.customElements.define('test-suite-1', TestSuite1);

Check out the live example!

Browsers

Specify which browsers you desire for your headless runs as command line arguments. The following browser command line arguments are available:

  • chromium
  • firefox
  • safari
  • edge
  • electron

You must install each of these browsers separately on the machine your tests will be running on. Only truly headless browsers will be run headless (Chromium and Firefox for now). Each browser launch is managed by its associated Karma browser launcher, which is installed along with Guesswork. If you have any questions about hooking up your browser, see the documentation in the appropriate browser launcher repo:

guesswork's People

Contributors

lastmjs avatar renovate-bot avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar

guesswork's Issues

Fix all timing issues

The setTimeout's and perhaps one or more other timing issues are bad. Fix them. Get rid of all non-deterministic timing code as much as possible. It has already broken the --auto-run option when the ts-warning option is on because the warnings take so long to run before allowing other components to load.

Use redux

State management has become a mess inside of test-runner. Manage all of the state with Redux, clean up side effects and mutations, clean up css and style mutations, use lit-html correctly

Generate test boilerplate

Given a web component as input, we should be able to generate a test suite element with all if the basic API test descriptions this would help tremendously with teaching newcomers how to test

Consider combining jsverify

If this is really supposed to be a property-based web component test runner, then I think I should abstract the jsverify function calls up into this library. Consider the implications of that.

Update documentation

  • Explain that Zwitterion is used, so in production you must do a static build
  • Explain all command line options
  • Add GIF of GUI and terminal
  • Give a quick introduction to JSVerify and how to create arbitraries
  • Explain the test suite structure, the file structure, the web component structure, etc
  • Show specific examples of test cases/test suites
  • Explain Zwitterion

Get rid of iframe

Once Karma supports ES Modules with the client.iframe: false setting along with client.runInParent. It will be nice to not have the extra overhead and complication of the iframe, as I don't see what benefit it provides. Also, it causes problems for electron

More thorough tests

I have a basic test suite, but it is more of a very high level integration test. It might be good to get into the nitty gritty of the public api for the test-runner, and potentially some better tests for guesswork.js and the command line and such.

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this πŸ’ͺ.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.


The push permission to the Git repository is required.

semantic-release cannot push the version tag to the branch master on remote Git repository.

Please refer to the authentication configuration documentation to configure the Git credentials on your CI environment.


Good luck with your project ✨

Your semantic-release bot πŸ“¦πŸš€

Fix this error

(node:12158) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 exit listeners added. Use emitter.setMaxListeners() to increase limit

Fix timing and template issues

Right now I'm using a couple of setTimeout functions to deal with some timing issues having to do with template stamping and when methods are available on other components. Get rid of all of the setTimeout functions so that everything is actually deterministic and not hacky. Upgrading to Polymer 2 and the v1 custom element spec I'm hoping will help with this.

Turn shadow dom on

Use a shadow root in all of the components once we switch to Polymer 2 and the v1 shadow dom spec

The automated release is failing 🚨

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this πŸ’ͺ.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.


Invalid npm token.

The npm token configured in the NPM_TOKEN environment variable must be a valid token allowing to publish to the registry https://registry.npmjs.org/.

If you are using Two-Factor Authentication, make configure the auth-only level is supported. semantic-release cannot publish with the default auth-and-writes level.

Please make sure to set the NPM_TOKEN environment variable in your CI with the exact value of the npm token.


Good luck with your project ✨

Your semantic-release bot πŸ“¦πŸš€

Figure out a better way to deal with asynchronous tests

Write now we have to wrap the code inside of the testWrap function in a promise so that we can wait for it to complete before running other tests...it would be nice to figure out how to abstract that so it isn't a burden to the user.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.