Feature File Incorrectly Reports as "Passed" about cucumber-reporting

[kingsleyh]

Hi

In the configuration you can set pending and skipped steps fail the build - if you want that to happen

--K

On 1 Aug 2013, at 21:09, msymons [email protected] wrote:

With the "Feature Statistics" table lists a nice long set of features, the easiest way to get a feel for status is to let your eye sweep across the vista of green "passed" and alight on red "failed". If all is green you are done almost instantly. If there's a red status then you zero on it just as quickly.

So it's important that nothing is green and marked as passed when it should really be red and "failed".

This is a feature that I believe should be "failed"

There's a background section in which

Given X
And Z

was essentially simply tweaked to

Given X
And Y
And Z

Thus, for each of the 59 scenarios, until such time as a developer updates the support code for the feature, step X still passes but Y is Pending and Z is skipped.

So, no step actually failed. But that should not mean the scenario itself passed. I suggest that the scenario (and thus the feature) should be flagged as failed

None of the above is arguing that the user wont already spot that something is wrong simply by seeing that there's a lot of pending and skipped steps. But that means that they have to stop and think.

—
Reply to this email directly or view it on GitHub.

[msymons]

Your answer was exactly what I needed. Knowing now that it was possible, I got the Jenkins administrator to actually show me the cucumber configuration for the job and there it was, in the "advanced" section:

So, I've now got my first reports generated with the newly changed config and am well happy. Sure, the percentage of failed scenarious is now high - but that's what I want because it's better representing what's really going on.

Double thanks then... thanks for the speedy reply that helped me fix my problem and thanks for the quality of the resulting report.

However, rather than close this issue, can I suggest that there might be something that could be tweaked... the documentation? I have been unable to find mention in the Jenkins plugin readme.md of the options shown in the above screenshot.