Properties for test cases about mocha-junit-reporter

[bitcoder]

@dennisgrck , fyi, the links for the documentation you provided are not working.

[clayreimann]

@dennisgrck Sorry for the delay, I'd be happy to review a PR if you were to implement this feature. As long as nothing breaks most any contribution is welcome here

[dennisgrck]

@bitcoder Thanks a lot for letting me know, I've fixed the links now!

[dennisgrck]

@michaelleeallen @clayreimann I would be happy to implement this and prepare a PR if you think this would be interesting to review. I would make sure this is a completely tool-agnostic implementation and follows the conventions of other tools that already support this (such as pytest and various other frameworks).

IMO a tool-agnostic way to include test case properties in the resulting XML file will allow many developers to build better integrations with Mocha & various reporting, CI & testing tools that support test case properties already.

[bitcoder]

How would that fit exactly a mocha test and a Cypress test for example?

[dennisgrck]

The proposed API would work exactly like the existing APIs such as this.test.attachments and this.test.consoleOutputs, so it would fit well with Mocha and Cypress like the existing approach. Hope this clarifies it!

[bitcoder]

Well, not totally :)
How would you specify those right from the specification block (i.e., it/describe)? Not from within it. Because some metadata must be specified before and not during the test

[dennisgrck]

@bitcoder This is not the intention of this issue request and not in the scope of what I suggested. My suggestion (and possible PR) is to provide a simple API to specify properties, just like the existing attachment & output APIs. I don't think it's a good idea to extend the function signature for this and this is also not really necessary for my suggestion; any property that can be specified in the function signature can also be specified within the test instead, which works just like the existing APIs. Hope this clarifies it!

[dennisgrck]

To add some additional information to this request, other popular testing tools such as Pytest and Playwright also support test case properties the same way as the above suggestion, and they also support multiple properties with the same name. Here is an example for Playwright (which uses Playwright's annotations API):

test('user profile', async ({ page }) => {
  test.info().annotations.push({ type: 'severity', description: 'critical' });
  test.info().annotations.push({ type: 'language', description: 'english' });

  // Playwright also supports multiple properties with the same name
  test.info().annotations.push({ type: 'step', description: 'The first step' });
  test.info().annotations.push({ type: 'step', description: 'The second step' });
  test.info().annotations.push({ type: 'step', description: 'The third step' });

  // [..]
});

And here is the same example for Python & Pytest:

def test_function(record_property):
    record_property("severity", "critical")
    record_property("language", "english")
  
    # Pytest also supports multiple properties with the same name
    record_property("step", "The first step")
    record_property("step", "The second step")
    record_property("step", "The third step")

    assert True

Instead of using a tool-specific implementation like #153 from one vendor, I would like to suggest using a similar open API for this project. The mocha-junit-reporter package is used with many frameworks and tools, and it is difficult to change APIs later, so I think it would be important to implement this in a more generic way so users can use this with any tool in the future.

[dennisgrck]

Sounds good, I will work on this and will try to get the PR ready in the next days.