kevinrood / teamcity_formatter Goto Github PK

cannot load such file -- team_city_formatter/formatter
Error creating formatter: TeamCityFormatter::Formatter (LoadError)

cucumber ... --expand -f TeamCityFormatter::Formatter ...

Seems Like the problem is in class name and file name

Camel case replaced with _ when searching file.

Then, TeamCityFormatter::Formatter is going to be replaced on team_city_formatter, meanwhile your path is teamcity_formatter/formatter

Hello,

We are using cucumber to run end to end tests which include flaky 3rd party services. By using v3.0.0.pre.1 of cucumber I can use the --retry X flag to rerun tests X times. Functionally this works fine, failed tests are retried. However when it comes to reporting this to TeamCity I encounter a problem. Each test attempt is reported, so if one attempt fails and the retry succeeds the the build itself fails.

This is confused a bit further because tests run on different devices also show under the same test. So here is what running 1 test, on 2 devices, failing the test on the first attempt and passing on the retry attempt (on both devices).

So for us, this scenario has passed at least once on each device, so we would consider it as green, but we are getting red.

Any thoughts? I realise that this is an edge case, and yes you shouldn't have flaky tests but that's where we are at the moment.

Thanks,

Jon

Any chance this will be updated to be compatible with v3 of cucumber?

Hi,

I'm using a Scenario Outline with multiple examples tables, for different tags and when the formatter tries to process this I get the following crash:

undefined method `column_values' for nil:NilClass (NoMethodError)
teamcity_formatter-0.8.2/lib/team_city_formatter/formatter.rb:62:in `before_table_row'
cucumber-2.3.3/lib/cucumber/formatter/ignore_missing_messages.rb:10:in `method_missing'
cucumber-2.3.3/lib/cucumber/formatter/legacy_api/adapter.rb:825:in `before'
...

Here's an example of the kind of thing I'm doing with the Scenario Outlines:

Scenario Outline: Testing a certain feature
    Given I test the value: <test_value> and <second_value>
    Then I should have a positive outcome

  @smoke
    Examples:
      | test_value | second_value | 
      | A          | B            | 

  @regression
    Examples:
      | test_value  | second_value | 
      | C           | D            | 
      | E           | F            | 

@kevinrood Formatter does not display scenario outlines as separate tests in TC Build, something should be changed. Originally, these tests are show up in tab named 'tests' after they appear in the build log.

When a spec calls skip_this_scenario, it's skipped. However, the TC reporter reports it as OK. Should probably be reported as Ignored.

Hello,

First, this issue already exist in the teamcity_formatter for older version of cucumber.

The problem is here:
Teamcity offer a tool statistic with graph. If you want to create a graph with the number of failed cases, the graph will only display the builds with error. Because if there is no error, the value 'Number of Failed Tests' is not send to teamcity. ( Same with Passed Tests, Ignored Tests… ) You can see the data reported by a run in the tab Parameters -> Reported statistic values.

Run with all the type of result:

Run without error, the data Number of Failed cases is not given:

Is that possible to update that gem to send '0' for the Number of (Failed/Passed/Ignored) Tests when there is no (Failed/Passed/Ignored) Tests ?

Thank you.