gallardo / citeproc-test-runner

Citeproc Test Runner (CTR for short) is a productivity tool for maintaining citation styles in the popular Citation Style Language (CSL) used by Zotero, Mendeley, Jurism and other projects. The script assumes that node, java and mocha are installed (the last globally with npm i -g mocha). With those in place, installation is a one-liner:

    npm install -g citeproc-test-runner

After installation, starting the program with cslrun will spit out instructions pointing back to this document. The remaining steps for setup are explained below. The instructions assume that you have a GitHub account (if you don't yet have an account, you should create a free account now).

Setting up

CTR can build tests for individual styles, using items from a shared public library. This allows style maintainers to quickly confirm that changes subsequently made to the style do not have unwanted side effects.

Tests are identified to a style using its slug name, which is the last portion of the style's CSL id string. For example, the slug name of the style http://www.zotero.org/styles/cell is cell.

Tests are written into and run from a local style tests directory. To facilitate the sharing of tests among maintainers, this directory should be a clone of your personal fork of the jm-style-tests repository on GitHub:

    <visit https://github.com/Juris-M/jm-style-tests and fork the repository>
    git clone https://github.com/my-github-name/jm-style-tests.git

After cloning your fork of the tests repository, set the path to it in the CTR configuration file, located in the user home directory ~/.cslrun.yaml:

groupID: 2319948
path:
  styletests: /path/to/jm-style-tests
  local: false
  std: false
  src: false
  locale: false
  modules: false
  cslschema: false
  cslmschema: false

By default, CTR builds tests from Jurism Style Test Items, a public group library (to use a different source library, set its id as the groupID value in ~/.cslrun.yaml). To make items available for a given style, join the group and create a top-level collection with the slug name of the style, and add some items to the collection. It is helpful to write a short description of the item to be tests (i.e. within 30 characters or so) into the Abstract field.

To build tests, use cslrun with the -w option to set the style to be tested, and the -U option to update test fixtures from the group library:

    cslrun -w /path/to/cell.csl -U

This will write test fixtures to a subdirectory of the style tests directory, named for the slug name of the style (if the subdirectory does not exist, it will be created). Finish the tests by using cslrun with the a option to run all tests, and the -k option to confirm output:

    cslrun -w /path/to/cell.csl -a -k

With the -k option, cslrun will show the style output as a failed test, prompting for confirmation that the output is correct. If Y is selected for each failing fixture, all tests will pass. Use ctrl-c to exit cslrun and return to the command line.

To share the tests. check them in and push them to your jm-style-tests repository on GitHub:

    cd /path/to/jm-style-tests
    git commit -m "Add tests for cell" -a
    git push

Then jump to GitHub and file a pull request to the main jm-style-tests project to have your tests added to the suite.

That's pretty much it. You can rerun the tests against the style anytime. If the -k option is omitted, cslrun will simply watch the CSL file, revalidating it and rerunning all tests whenever it is saved, to convert any plain text editor into a poorman's CSL IDE:

    cslrun -w /path/to/cell.csl -a

A few other options are available on CTR. See cslrun -h for further details.

Test layout

CTR works from tests written in the idiosyncratic CSL test format, documented in the CSL test-suite repository. The description there is for the self-contained tests used in the processor test suite, and the tests generated by CTR (see below) differ in a few respects. Specifically: CTR style tests omit the CSL section (because they use CSL code directly from the style); they include a KEYS array to memo the source of the input data; and they use a MODE of "all" to exercise the various forms that a citation may have in all contexts.

---

Enjoy!

FB