Got Scripts? Test’em!
Unit testing in Javascript can be tedious and painful, but Testem makes it so easy that you will actually want to write tests.
Features
- Test-framework agnostic. Support for
- Run tests in all major browsers as well as Node and PhantomJS
- Two distinct use-cases:
- Test-Driven-Development(TDD) — designed to streamline the TDD workflow
- Continuous Integration(CI) — designed to work well with popular CI servers like Jenkins or Teamcity
- Cross-platform support
- OS X
- Windows
- Linux
- Preprocessor support
- CoffeeScript
- Browserify
- JSHint/JSLint
- everything else
Screencasts
- Watch this introductory screencast (11:39) to see it in action! This one demonstrates the TDD workflow.
- Launchers (12:10) — more detail about launchers: how to specify what to auto-launch and how to configure one yourself to run tests in Node.
- Continous Integration (CI) Mode (4:24) — details about how CI mode works.
- Making JavaScript Testing Fun With Testem (22:53) — a thorough screencast by NetTuts+'s Jeffery Way covering the basics, Jasmine, Mocha/Chai, CoffeeScript and more!
Installation
You need Node version 0.6.2 or later installed on your system. Node is extremely easy to install and has a small footprint, and is really awesome otherwise too, so just do it.
Once you have Node installed:
npm install testem -g
This will install the testem
executable globally on your system.
Usage
As stated before, Testem supports two use cases: test-driven-development and continuous integration. Let's go over each one.
Development Mode
The simplest way to use Testem, in the TDD spirit, is to start in an empty directory and run the command
testem
You will see a terminal-based interface which looks like this
Now open your browser and go to the specified URL. You should now see
We see 0/0 for tests because at this point we haven't written any code. As we write them, Testem will pick up any .js
files
that were added, include them, and if there are tests, run them automatically. So let's first write hello_spec.js
in the spirit of "test first"(written in Jasmine)
describe('hello', function(){
it('should say hello', function(){
expect(hello()).toBe('hello world');
});
});
Save that file and now you should see
Testem should automatically pick up the new files you've added and also any changes that you make to them and rerun the tests. The test fails as we'd expect. Now we implement the spec like so in hello.js
function hello(){
return "hello world";
}
So you should now see
Using the Text User Interface
In development mode, Testem has a text-based graphical user interface which uses keyboard-based controls. Here is a list of the control keys
- ENTER : Run the tests
- q : Quit
- ← LEFT ARROW : Move to the next browser tab on the left
- → RIGHT ARROW : Move to the next browser tab on the right
- TAB : switch the target text panel between the top and bottom halves of the split panel (if a split is present)
- ↑ UP ARROW : scroll up in the target text panel
- ↓ DOWN ARROW : scroll down in the target text panel
- SPACE : page down in the target text panel
- b : page up in the target text panel
- d : half a page down target text panel
- u : half a page up target text panel
Command line options
To see all command line options
testem --help
Continuous Integration Mode
To use Testem for continuous integration
testem ci
In CI mode, Testem runs your tests on all the browsers that are available on the system one after another. To find out what browsers are currently available - those that Testem knows about and can make use of
testem launchers
Will print them out. The output might look like
$ testem launchers
Browsers available on this system:
IE7
IE8
IE9
Chrome
Firefox
Safari
Opera
PhantomJS
Did you notice that this system has IE versions 7-9? Yes, actually it has only IE9 installed, but Testem uses IE's compatibility mode feature to emulate IE 7 and 8.
When you run testem ci
to run tests, it outputs the results in the TAP format, which looks like
ok 1 Chrome 16.0 - hello should say hello.
1..1
# tests 1
# pass 1
# ok
TAP is a human-readable and language-agnostic test result format. TAP plugins exist for popular CI servers
- Jenkins TAP plugin - I've added detailed instructions for setup with Jenkins.
- TeamCity TAP plugin
Command line options
To see all command line options for CI
testem ci --help
Configuration File
For the simplest JavaScript projects, the TDD workflow described above will work fine. There are times when you want
to structure your source files into separate directories, or want to have finer control over what files to include.
This calls for the testem.json
configuration file (you can also alternatively use the YAML format with a testem.yml
file). It looks like
{
"framework": "jasmine",
"src_files": [
"hello.js",
"hello_spec.js"
]
}
The src_files
can also be unix glob patterns.
{
"src_files": [
"js/**/*.js",
"spec/**/*.js"
]
}
You can also ignore certain files using src_files_ignore
, and you can alternatively use a space separated string for the patterns
{
"src_files": "js/**/*.js spec/**/*.js",
"src_files_ignore": "js/toxic/*.js"
}
Custom Test Pages
You can also use a custom page for testing. To do this, first you need to specify test_page
to point to your test page in the config file (framework
and src_files
are irrelevant in this case)
{
"test_page": "tests.html"
}
Next, the test page you use needs to have the adapter code installed on them, as specified in the next section.
Include Snippet
Include this snippet directly after your jasmine.js
, qunit.js
or mocha.js
include to enable Testem with your test page
<script src="/testem.js"></script>
Launchers
Testem has the ability to automatically launch browsers or processes for you. To see the list of launchers Testem knows about, you can use the command
testem launchers
This will display something like the following
Have 5 launchers available; auto-launch info displayed on the right.
Launcher Type CI Dev
------------ ------------ -- ---
Chrome browser ✔
Firefox browser ✔
Safari browser ✔
Opera browser ✔
Mocha process(TAP) ✔
This displays the current list of launchers that are available. Launchers can launch either a browser or a custom process — as shown in the "Type" column. Custom launchers can be defined to launch custom processes. The "CI" column indicates the launchers which will be automatically launch in CI-mode. Similarly, the "Dev" column those that will automatically launch in dev-mode.
Running Tests in Node and Custom Process Launchers
To run tests in Node you need to create a custom launcher which launches a process which will run your tests. This is nice because it means you can use any test framework - or lack thereof. For example, to make a launcher that runs mocha tests, you would write the following in the config file testem.json
"launchers": {
"Mocha": {
"command": "mocha tests/*_tests.js"
}
}
When you run testem
, it will auto-launch the mocha process based on the specified command every time the tests are run. It will display the stdout and well as the stderr of the process inside of the "Mocha" tab in the UI. It will base the pass/fail status on the exit code of the process. In fact, because Testem can launch any arbitrary process for you, you could very well be using it to run programs in other languages.
Processes with TAP Output
If your process outputs test results in TAP format, you can tell that to testem via the protocol
property. For example
"launchers": {
"Mocha": {
"command": "mocha tests/*_tests.js -R tap"
"protocol": "tap"
}
}
When this is done, Testem will read in the process's stdout and parse it as TAP, and then display the test results in Testem's normal format. It will also hide the process's stdout output from the console log panel, although it will still display the stderr.
PhantomJS
PhantomJS is a Webkit-based headless browser. It's fast and it's awesome! Testem will pick it up if you have PhantomJS installed in your system and the phantomjs
executable is in your path. Run
testem launchers
And verify that it's in the list.
Preprocessors (CoffeeScript, LESS, Sass, Browserify, etc)
If you need to run a preprocessor (or indeed any shell command before the start of the tests) use the before_tests
option, such as
"before_tests": "coffee -c *.coffee"
And Testem will run it before each test run. For file watching, you may still use the src_files
option
"src_files": [
"*.coffee"
]
Since you want to be serving the .js
files that are generated and not the .coffee
files, you want to specify the serve_files
option to tell it that
"serve_files": [
"*.js"
]
Testem will throw up a big ol' error dialog if the preprocessor command exits with an error code, so code checkers like jshint can used here as well.
If you need to run a command after your tests have completed (such as removing compiled .js
files), use the after_tests
option.
"after_tests": "rm *.js"
If you would prefer simply to clean up when Testem exits, you can use the on_exit
option.
Custom Routes
Sometimes you may want to re-map a URL to a different directory on the file system. Maybe you have the following file structure:
+ src
+ hello.js
+ tests.js
+ css
+ styles.css
+ public
tests.html
Let's say you want to serve tests.html
at the top level url /tests.html
, all the Javascripts under /js
and all the css under /css
you can use the "routes" option to do that
"routes": {
"/tests.html": "public/tests.html",
"/js": "src",
"/css": "css"
}
DIY: Use Any Test Framework
If you want to use Testem with a test framework that's not supported out of the box, you can write your own custom test framework adapter. See customAdapter.js for an example of how to write a custom adapter.
Then, to use it, in your config file simply set
"framework": "custom"
And then make sure you include the adapter code in your test suite and you are ready to go. Here for the full example.
Growl or Growl-ish Notifications
If you'd prefer not to be looking at the terminal while developing, you can us growl notification (or simply desktop notifications on some platforms) using the -g
option.
But, to use this option, you may first need to install some additional software, see the node-growl page for more details.
Example Projects
I've created examples for various setups
- Simple QUnit project
- Simple Jasmine project
- Custom Jasmine project
- Custom Jasmine project using Require.js
- Simple Mocha Project
- Mocha + Chai
- Hybrid Project - Mocha tests running in both the browser and Node.
- Buster.js Project
- Coffeescript Project
- Browserify Project
- JSHint Example
- Custom Test Framework
- Tape Example
- BrowserStack Integration bleeding edge
- SauceLabs Integration bleeding edge
Known Issues
-
On Windows, Mocha fails to run under Testem due to an issue in Node core. Until that gets resolved, I've made a workaround for mocha. To install this fork of Mocha, do
npm install https://github.com/airportyh/mocha/tarball/windowsfix -g
-
If you are using prototype.js version 1.6.3 or below, you will encounter issues.
Contributing
If you want to contribute to the project, I am going to do my best to stay out of your way.
Roadmap
- BrowserStack integration - following Bunyip's example
- Figure out a happy path for testing on mobile browsers (maybe BrowserStack).
Contributors
Community
Credits
Testem depends on these great software
- Jasmine
- QUnit
- Mocha
- Node
- Socket.IO
- PhantomJS
- Node-Tap
- Node-Charm
- Node Commander
- JS-Yaml
- Winston
- Express
- jQuery
- Backbone
License
(The MIT License)
Copyright (c) 2012 Toby Ho <[email protected]>
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.