pwyf / bdd-tester

A very very basic BDD test runner

BDD Tester

Test data files using behaviour-driven development tests, in python.

Wait, what?

It’s a tool to run Gherkin tests (features) against (IATI) data – similar to Cucumber (in Ruby) or Behave (in Python). Here’s a blog that nicely explains the idea.

Install

pip install bdd_tester

Run

bdd_tester --feature [Feature file] --steps [Step file] [Data file]

For more options, run:

bdd_tester --help

Using it programmatically

from bdd_tester import BDDTester


tester = BDDTester(step_definitions_filepath)
feature = tester.load_feature(feature_filepath)
result = feature.tests[0](*args, **kwargs)

bdd-tester's Issues

Rename to something gherkin-related…

Options here.

Pumpkin? Squash? Calabash? Zucchini?

Naming things is hard.

Tests!

@edugomez points out that this testing tool is sorely lacking some tests. Valid point! He also recommends using pytest.

Split sample_features according to IATI standard version

This split should affect only rules that are different between versions. Common rules across versions should be reused by all versions.

Split feature sets out into separate repos

It’s convenient to keep the features and steps in here, but it’s not necessary, and isn’t really correct. At some point, these should be moved each into their own repository.

DQJSONFormatter occasionally misses closing `]`

I can’t tell what’s causing this at all, but I’m just going to:

1. Leave this issue as a placeholder
2. Go back to using DQLogFormatter (but I’ll leave DQJSONFormatter as default)

@edugomez have you experienced this problem? I wondered if it was because I’m using celery and running multiple workers, but I tried switching to one worker and the problem persists. At some point I’ll dig into this a bit more!

Set up outputs properly

Okay so there are three different outputs:

1. Summary of results i.e. pass/fail/not relevant activity counts per scenario
2. Log of reasons for each activity failing each scenario
3. General informational log

We may want to output (1) to stdout or to a file. We probably just want to output the others to files.

We currently solve (1) and (2) using formatters, and (3) using a logger.

Don’t create empty log files

Currently a log file gets created by DQLogFormatter at the start of a scenario, even if it is never populated.

Feature folder structure reflected in output

It would be really handy if the folder structure of the features was somehow reflected in the output dict… E.g. dicts of dicts for nested folders, that sort of thing.

Handle logging properly

We need a param to specify where the logging should go (e.g. a filename) and then environment.py should output it there.

Refs #5.

Consider moving all custom context stuff to context.config.userdata

Examples suggest it’s fine to store stuff on the context object… but it just might be a bit tidier to leave it all in one place.

context.config.userdata is a candidate.

Add current data test

currently just asserts True

Provide more details of params in the README

the API bit is very sparse at present.

Swap out behave for a custom solution

Testing large files is like wading through treacle, and renders this unusable for our purposes.

I tried swapping out behave for foxpath, and saw significant performance gains.

Behave probably does a bunch of stuff we don’t need, and doesn’t appear to be optimised for processing scenario outlines with thousands of rows. It’s not difficult to make a simple gherkin parser; foxpath already processes a step_definitions.py file that’s roughly the same format as here (though the context to each step is passed in in kwargs). So this is very doable.

Add an argparser

Current hack is v bad.